User:Prawsthorne/Assessment

	Mozilla Open Assessment (proposal)
	Owner: Peter Rawsthorne, and a bunch of other Mozilla team members (hopefully)	Updated: 2012-03-30
	Assessment could be built into all aspects of webmaking. This includes both the teachers and the learners. We may start with the more traditional master-apprentice type of assessment, then move into peer and collaborative based models so we can scale assessment. Mozilla Open Assessment would support and encourage deepening quality within Open Badges. In quality we mean every badge has a great amount of worth within the learner community and continues to have worth.

This proposal is a work in progress, for which I would like to make a request for feedback... if you have insights, references, questions or critical comments please add them to this pages discussion. Sincerely, Peter Rawsthorne 16:45, 25 March 2012 (PDT)

Background

In the last few weeks I have learned that the Mozilla Learning team has been hard at work defining a set of core web literacy skills and developing learning pathways for people to develop and hone these skills. Their goals are to develop and empower more webmakers by providing them with the opportunity to learn making by making. Mozilla open assessment is leveraging all of the great momentum, tools and content built through Webmaking, Hackasaurus and other similar initiatives.

NOTES:

• The Open Badges project is also part of the Learning Group. For more on that project, see http://openbadges.org.
• This OER Assessment blog post is further background on some of Peter's thoughts regarding Open Assessment.
• Review the results for a search within this wiki; https://wiki.mozilla.org/Special:Search?search=assessment
• Much work has been done with peer assessment approaches within current online education and traditional institutions, these should be referenced.
• Some of the badges / featured articles models within Wikipedia, Wikiversity and WikiEducator should also be considered.

About this Project

To support teachers and encourage learners the Mozilla Learning Group intends on utilizing badges to promote the recognition of Webmaker skills. These badges will be awarded to both teachers and learners as they progress through the WebMaker curriculum and Hackasaurus events. This project focuses on the how of creating assessment approaches and instruments to identify badge awardees.

NOTE: the initial focus of this project will be on teachers awarding badges to their students. As the project progresses focus will shift to building capacity toward peer-based assessment and collaborative open approaches of assessment. This will encourage community building and provide the scalability that teacher awarded badges could not.

BLUE SKY: the destination of this project would be to have created an assessment infrastructure of technology and approaches that would support badges across all subject domains, in all languages, around the globe. Mozilla foundation would have created assessment centres of excellence in all their primary locations to encourage further use of badges and to deepen the research around assessment in a localized context. Localization is important, as assessment is different around the world. Like many things; it is not the destination, but the journey.

Think Different

Just as many of the Mozilla initiatives (xray-goggles, popcorn, lovebomb, etc...) have focused on the fun of webmaking, so does helping people assess where they are with learning webmaking. Assessment should disappear into the activities and provide guidance as they go. It needs to be as fun as the making! There are very few (if any) learning sites doing this already, we need to be bold and creative and begin to run experiments in the real labs of webmaking events. We need to believe this kind of participatory assessment can be done. It returns to building community...

The Roadmap

1. In the immediate term we should build peer-assessment to utilize and promote the use of badges (see wikieducator: http://wikieducator.org/User:Prawstho or Mozilla badges for example). This will get us going quickly and then build upon this with greater automation and other assessment approaches.
2. In the near term we should build rubrics to allow people to perform self-assessments and then submit portfolio for a peer-review. Peer reviewers should also be given badges for executing N peer reviews. This does not require a lot of automation, can be implemented quite quickly.
3. In the close term we should build formative and summative assessment instruments that can be baked into websites, apps and tools. People should be able to edit / improve on instruments in a community kind of way.
4. Beginning immediately and through the next year we should start building apps and experimenting with mass collaboration for assessment.
5. In the medium term we should build a repository of guidelines, templates, assessment instruments and approaches.
6. In the long term we should build a fun community around assessment and broadening peoples skills and knowledge regarding assessment and accreditation.
7. As an ongoing initiative we should encourage research and centers of excellence around open assessment and its relationship with open accreditation.

1. Immediate term (Now, March 27, 2012)

Mozilla is predominantly awarding badges by the teachers identifying participants who have mastered the required skills. This needs to continue and will likely always continue. Assessment does not come as one approach, multiple pathways to assessment provides the strongest assessment toward mastery of skills. Capacity building needs to continue with this model where Mozilla builds instruments, templates, rubrics and community to support the teachers of Webmakers and Hackasaurus. Teachers as peers would be this initial community.

Tasks

This first set of tasks is in developing a solid understanding within the hackasaurus / webmaker facilitator community in how assessment can be baked into events... leading toward peer assessments amongst the participants. We need to start with the facilitators having an awesome understanding of how this all (rubrics, templates, encouraging peer assessment) works.

1. Identify courses / learning events who have issued badges
2. gather instructor testimonials regarding badge awarding for these courses (focus on the fun)
3. document approaches and best practices
4. create curriculum to assist in building understanding of badge awarding and instructors-as-peers
5. form online community of instructors, create instructor-as-peers assessment approaches
6. build community around instructor awarded badges
7. work on scale-out strategies with community (how do we keep instructors trained in awarding badges)

2. Near term (Q2, 2012)

The next phase would be to build upon the success achieved with the immediate term and its focus on building capacity within the Webmaker / Hackasaurus teacher and student community. This phase would add to the scale out of the previous phase. In the near term the Mozilla Learning group would build rubrics and other templates to allow people to perform self-assessments and then submit a portfolio for peer-review. Peer reviewers should also be given badges for executing N peer reviews. This would not require a lot of automation, and can be implemented quite quickly. Strong stewardship of this community would be important.

Tasks

Following quickly from the immediate term we start to build capacity within the workshops / events where the facilitators are good at having the peers assess each others work in a caring environment. Having created a community of practice amongst the facilitators in how to bake peer-assessment into the events would be important. This would have been the focus of the previous immediate term tasks.

1. Facilitate workshop / community building event reviewing previous phase and focusing on adding peer-review to the students experience.
2. Find and create examples of peer-review activities within the Webmaker / Hackasaurus communities. Because, most times peer-assessment happens. (within most events and very quickly participants know who gets it within the events cohort, this needs to be leveraged. One of the best ways for learning to happen is to have the students work together in helping those who struggle the most. This elevates the whole group so the top is even higher; everybody learns more.)
3. Create rubrics and templates to encourage peer assessment. have participants engage in this creation. Well facilitated, this can be a lot of fun for the group.
4. Attend Webmaker / Hackasaurus events; utilize rubrics and templates, and begin experiments toward peer-assessment. All this has to be driven into the events.
5. Build student and teacher community around peer assessments and supporting rubrics and templates. Encourage re-use.
6. Start a featured portfolio theme within MozLearning. Highlight those portfolios that are exemplary... tie them back to rubrics explaining why they are exemplary. Build community... great assessment is about love... sounds kinda flakey, but when it comes to kids... it is true.

3. Close term (Q3, 2012)

Build formative and summative assessment instruments that can be baked into websites, apps and tools. People should be able to edit / improve on instruments in a community kind of way. Fun should be the key... identify things (tags, styles, etc...) they have yet to play with. The thrust of this phase will be threefold;

1. to determine the challenges of creating assessment tools and approaches that can be baked into Webmaker and hackasaurus websites, apps and tools.
2. have assessment instruments to help people identify gaps in their understanding
3. all of this should be fun and integrate seamlessly with the joy and exploration of making.

Tasks

The theme of these tasks is trying to do something that has never been done online, in the open; baking assessment into the tools. Small iterative successes is how we will build something great! With all the knowledge gained within the first two terms, it will be easier to know where to start with this. WikiEducator already has templates to bake assessment right into the OER. I believe this could also be done within the fun of existing Mozilla tools.

1. identify candidate tools (xray-goggles, lovebomb, popcorn, etc...) that could be easily extended with formative and summative assessment instruments.
2. identify a hierarchy of competencies required to be proficient with these tools.
3. create assessment instruments for the identified competencies.
4. as an example; extend xray-goggles to assess depth of learning with the tasks they have already completed. Suggest other fun things to do within xray-goggles, these could include tasks from both CSS and HTML5.

4. Over the next year

Beginning immediately and through the next year start building approaches to embedding assessment, templates, apps and experimenting with mass collaboration for assessment. The focus is on Mozilla's helping build integrity and solid assessment techniques around the use of badges. This should include; gathering feedback from events on ideas that could work, and building apps as experiments that can be used by learners, study groups, communities of practice and institutions issuing badges.

5. Medium term (tbd)

In the medium term we should build a repository of guidelines, templates, assessment instruments and approaches. Work in progress...

6. Long term (tbd)

In the long term we should build a fun community around assessment and broadening peoples skills and knowledge regarding assessment and accreditation. Work in progress...

7. Ongoing

As an ongoing initiative we should encourage research and centers of excellence around open assessment and its relationship with open accreditation. Work in progress...