#TeachTheWeb Assessments as Stories OR Stories as Assessments

2 min read

I know I may sound like a broken record (is missing packets the mp3 equivalent?) but we need to stress stories of the different Mozilla Web Clubs.

Can we let folks tell their story as a form of assessment? We are looking for friction-free assessment so why not bake it into the workflow of the club leaders.

We are already stressing that Mozilla Web Club mentors or leaders (seeing different titles in different documentation) set up blogs or websites. When you examine the three questions we asked we are getting at mentors impression of the learners rather than the much more difficult to measure knowledge growth in the  learners.

Why not encourage this to be done as a blog post or a discourse thread? The open ended questions would have to analyzed and coded to be useful it would be no different with posts. Plus you get the added benefit of stories.

Stories, as much as people, will empower Mozilla learning.

Pre-Post

This would require more work on part of the mentor but you ask them to do a post before and after the learnign activity. They could be given the three questions we use now as optional post.

or

 

The first response could be a free response and the second could be a considered response.

Debrief

You could just have club leaders post to a blog or discourse after an activity. I lean more towards this. You give the three prompts and ask them to reflect.

To me teaching like a Mozillian would involve using assessments as an opportunity for reflective growth rather than a measure of knowledge growth.

Friction Free Assessment when Tests are the Sandpaper of #TeachTheWeb Learning

4 min read

Michelle Thorne posted a great round up of curriculum testing

Friction Free Assessment

She asked about friction-free assessment. They don't exist. We would be chasing a unicorn named oxymoron if we spent too much time looking for friction free assessments,

As Dan Hickey likes to remind us the introduction of assessments fundamentally alters the motivation of learning. Yet we need to assess learning.

So I guess we shouldn't look for friction free assessments but well oiled assessments, and I always thought badging was the major mechanism to ensure evidence of learning was collected. Digital badges create wells of evidence that can be mined for recognition, motivation, and credibility. You just have to drill down behind the badge (Did I take the metaphor too far?).

I have been fascinated to watch the diverse perspectives towards assessment in Mozilla Learning. You have analytics and design teams who will run a statistical test on a hex color A/B test to increase unique visitors (not sure why) and we also support the largest open badging platform.

When we were drafting the questions for testing the curriculum some wanted to get at the learners. Some wanted to get at the mentors. Some wanted to get at the curriculum. Not sure we did any of these.

The short  answer (but not easy) answer, "Choose assessments that align to your learning goals and philosophies."

The three open ended questions we asked more measured the mentors expectations and bias of what was learned. The analysis of these questions would also be quite time consuming. For example how does the question, "What are the top three strategies you think will be used by your learners to see if information on a website is credible?" capture growth? Would you count up the frequency of different strategies and run statistical tests to see if the top three strategies changed?

If you want it the assessment to be fast you have to use likert scales, have enough items, and then treat your ordinal data like numerical data (a step many in measurement disagree with).

If you want the assessment to be fast and reliable you would have to spend anywhere from 10K to millions to develop measures that act like traditional tests. This could be done for reading and writing, but participation would be hard. Furthermore many, in the connected learning camp, would argue that these assessments measure very little (Ian and I  do have credibility assessments and UCONN has made their online reading and research assessment available) and the variance and noise in scores is where real learning happens.

What are we to do?

Align assessments strategies to our philosophy and it may become evident that the only metric that matters is the number of makes submitted. Then encourage club mentors to have club members submit evidence for badges.

You then to ensure, credibility of badges, could audit a random set of submissions and the evidence included with the credentialing. We have to ensure that  the badges get external recognition. I don't think a sampling of badge applications would involve that much more work than the coding and analysis of three open ended questions.

The problem I see with badges and the curriculum is I, as an issuer, do not feel that the activities lead to a preponderance of evidence that would leave me comfortable with issuing a badge. It may take multiple activities. 

We may need more light weight or level up badges that mentors can give (but only the final web literacy badges go to backpack?).

Instead of friction-free assessments on a global scale we will need to provide mentors, especially the majority who do not come from education, on the principles of formative assessment. They need to know how to take the learning goal, teach the curriculum that elicits evidence of growth towards that goal, be able to analyze that evidence while they facilitate learning, and then adjust instruction. If you can figure out easy ways to teach these better practices please tell me so I can steal them.

It is up to us as a community to ensure the badges have value and weight. It is up to us to ensure they are baked into the ecosystem and allow for individual learning pathways.

 

 

Annotate the web

1 min read

I learn so much being part of co-learning in spaces like . The act of teaching is the act of learning.

 

This week we are focusing how you can annotate the web for critical evaluation. Dr Julie Coiro pointed out you can not separate argumentation and credibility. 
As a reminder though if you have not added your blog or rss url please use this form.Say hello on Twitter using the hashtag


This week we worked together as a community to create a codebook for annotation. You goal is to complete these tasks and write about it

Happy Pi Day to all,

Greg

12 March 2015 Web Literacy Map Version 1.5 Community Call

5 min read

Updated with correct times of Monday's Hack Session

This is a very incoherent rough draft written on a day when there was no time to write. Therefore it is full of grammar mistakes, broken code and inappropriate snark. 

On this week community call we were making the final push to finish the skills under the competencies of version 1.5 of the Web Literacy Map. We started off with Ian sharing a Cory Doctorow talk, but then dove write into the work.

Doug was on personal time off but checked in throughout the day and helped to set up the etherpad (learning to take PTO like an American I guess). 

We first reviewed the activity on Git as we shifted to asynchronous work to ensure Version 1.5 can ship by the end of Q1. The team is getting more comfortable with filing and responding to issues on Git and we encourage everyone to visit the issues to track our progrss,

We first reviewed the sharing competency as we noted earleir that this was our most problematic competency. Too much overlap existed with other comptencies in version 1.1 of the Map.

The following skills were proposed in Git

  • Contributing content and finding content for the benefit of others (Greg)
  • Creating, curating and circulating web resources to elicit peer feedback (Greg)
  • Understanding the needs of audience in order to make relevant contributions to a community(Greg)
  • Creating and using a system to distribute web resources to others (Greg)
  • Identifying when it is safe to contribute content in a variety of situations on web (Tim)

Ian and Mikko suggested that creating a system to share should be ordered as the first skill. The final list of skills included.

  • Creating and using a system to distribute web resources to others.
  • Contributing  and finding content for the benefit of others.
  • Creating, curating, and circulating web resources  to elicit peer feedback.
  • Understanding the needs of audiences in order to make relevant contributions to a community.
  • Identifying when it is safe to contribute content in a variety of situations on the web. 

Next we turned to Search

  • Developing questions related to your goal and knowing when these questions are answered. (proposed)
  • Using and revising keywords to make your web searches more efficient.(approved)
  • Evaluating search results to determine if the information is relevant to your needs. (edited needs approval)
  • Finding real-time or time-sensitive information using a range of search techniques. (approved)
  • Discovering information and resources by asking people within social networks (proposed)

These were discussed extenslvely. Gadit and Tim were looking for clarification on the questioning skill. We spent a good amount of time deciding if formulating questions was part of a searching skill. The group decided to keep the skill. The skills were edited to only include third person langauge and they were approved.

Next we turned to Composing for the Web

We began with the following proposed skills - 

  • Inserting hyperlinks into a web page (defined)
  • Embedding multimedia content into a web page (defined)
  • Creating web resources in ways appropriate to the medium/genre (defined)
  • Identifying and using HTML tags (defined)
  • Creating and controlling a space to publish on the Web (Doug)+1

We debated the last skill. Specifally if you create a space and why you have to control the space. Laura brought up the issue of how we started on hosted spaces and moved to open spaces. Greg argued for trying to push people out of silos and on to the open web. Mikko and Gatid supported a similar position. In the end we voted on:

  • Setting up and maintaining a space to publish on the web+1+1+1+1+1
  • establishing, acquiring+1+1, and setting up
  • maintaining     controlling     +1

The skills under the competency were then set.

Next we turned to remix, but once we realized that we could not come to agreement we decided to abandon the issue in kicked it to asynchronus debate. Ian argued for renaming the comptency syntheis. Greg argued to move the synthesis skills booted from search to remix. He felt you are making something new wit someone else idea. Gadit felt strong that, "Remixing is under building -- it should have a very specific 'publishing' aspect," and therefore should not include "in your head" skills. Laura questions wether we were discussing a web literacy skill or a life skill.,

There was also debate wether attribution fit under remix or under open practice. We did agree that the skill on remixing open content should be changed to, "Identifying and using content that can be ethically remixed." This seperated the skill from the open practices competency.

We also turned to the skill, "Creating something new on the web using existing resources" and felt it was too close to a new remix we added: "Shifting context and meaning by creating derivative content." There is a current debate on Git wether these are similar skills and only one is needed or if both skills are needed under the remix competency.

We tried then to turn to credibility and we nailed down a skill before time was up.

Wrapping Up

We have almost finished Version 1.5 of the Map. Here is our current state of progress

Finished competencies:

Unresolved competencies:
credibility
security
remixing
coding/scripting
collaborating
privacy
open practices

Final Steps

It was decided to host a hack session on Monday 5pm UTC (10am PT / 1pm ET / 5pm CET / 10.30pm IST). The group also decided to abandon the spreadsheet for work (though Ian will update it) and just finish off our work on Github.

Please visit the issues if you would like to contribute.

Check out @googleguacomole rightfully calling me out. https://hypothes.is/a/7ZBfSxUdQP6LlkQ2MNHjQQ

how well do work with n00bs? Did it work because we were used to networked spaces: https://hypothes.is/a/-gt4knpaRpWGucrZwLCXSQ

Thanks to @GoogleGuacomole I discovered a great new blog K-Log http://justywk.blogspot.com/ added to feedly.com/jgmac1106

@withknown Okay the add-on pages for Firefox doesn't scare much as they used to. Felt like used to be a bunch of shady youtube downloaders.

To all those who joined us on the https://docs.google.com/document/d/1WgN4l-0QESkWOkwAoAg_Tl800P0NESNiqlUjxd_hvsc/edit to thank you. h/t @EduQuinn for hosting us tonight

@GoogleGuacamole or @EduQuinn who is going to get the Prize?