Accountability and Portfolios

I am posting this message that did not get sent to the AAHE eportfolio listserv before it was shut down. I was responding to this message sent by Trent Batson from Rhode Island:

... My sense of the eportfolio phenomenon in the US is that the assessment/accountability end of the spectrum is where most of the money is right now. My own hope from the push toward assessment-management is that these systems will get eportfolios in place on many campuses and then other uses will be discovered

Trent, I agree with your statement that e-portfolios are being adopted because of the assessment/accountability needs of institutions. The challenge with that scenario is that we are turning off a lot of students (and perhaps also faculty) in the process because of high stakes accountability. Perhaps formal education is validating Lee Shulman's assertion that one of the five dangers of portfolios is "perversion"! As he says in Nona Lyons' book (1998) With Portfolio in Hand,

If portfolios are going to be used, whether at the state level in Vermont or California, or at the national level by the National Board, as a form of high stakes assessment, why will portfolios be more resistant to perversion than all other forms of assessment have been? And if one of the requirements in these cases is that you develop a sufficiently objective scoring system so you can fairly compare people with one another, will your scoring system end up objectifying what's in the portfolio to the point where the portfolio will be nothing but a very, very cumbersome multiple choice test? (p. 35)

At the IRA conference earlier this month, I got a round of applause for the statement, "High stakes accountability is killing portfolios for learning." In the drive to use portfolios as assessment OF learning, we are in danger of losing the power of portfolios to support reflection and assessment FOR learning. I'm starting to collect stories about student rebellion against this approach, like the college student in a midwest university who ran for student body president on a platform to get rid of the campus-wide assessment portfolio. Then, there are high school students in the Pacific Northwest who built a bonfire and burned their mandatory graduation paper-based portfolios (eSchool news quoted me on this story as the opening of their article about the TaskStream research project.... Of course they didn't quote me on the other story about the high school student who offered a $50 reward to recover her lost writing portfolio.) I tell both of those stories in more detail in the TaskStream White Paper.

I also think purpose is inextricably linked with process (per Activity Theory) and the tools tend to be developed to support the primary purpose. In my incomplete survey of different online tools to construct e-portfolios, it was obvious to me that the tools tend to favor one approach over the other. Those tools that purport to be more "assessment management systems" tend to provide an institution-focused structure that makes it much easier to "score" but more difficult for the learner to tell their own story of their learning. There were some systems that I tried where I could not create the portfolio that I wanted... I was forced to use a pre-set template. For me, the bottom line is "ownership" - and I was pleased at the ePortfolio conference in Vancouver, B.C. last month where the general consensus of the participants and presenters was that learners owned their own portfolios. Based on that statement, the tools should support that ownership in every way possible. I am finding that those systems based on an online database to capture assessment data provide far less creativity in appearance and organization than other tools. So I am making a plea to educational institutions for balance in the purposes for implementing e-portfolios, and to the software developers for more creativity and flexibility in the presentation tools.

Saskatchewan Learning

I just received notification about a new resource posted online at the Saskatchewan Learning Website:

A JOURNEY OF SELF-DISCOVERY: Facilitator's Guide to Reflection and Portfolio Development [PDF]
This guide has been developed to support facilitators as they lead learners through a process of thinking about what they know and can do (reflection). Through involvement in these activities, learners identify the knowledge, skills, and attitudes they have developed, and create evidence of their learning. These general activities are intended to be adapted by facilitators to meet the needs of any group.

This is a great resource for those facilitators who help learners with self-assessment in preparation for PLAR portfolios (Prior Learning Assessment and Recognition).

Reality Check!

Every few years, I need to do a hands-on technology project with school-age students, as a reality check on my theories about electronic portfolios or digital storytelling in education. This week, I worked with two dozen eighth grade English students (my daughter's students at a private school in Anchorage). The experience was eye-opening, exhausting and exhilarating! We produced 22 digital family stories, between one and three minutes long. We spent one day in a portable classroom, where only one computer had Internet access. I had a small office to do the audio recording, which did have high speed wireless Internet. As a private school, there are not the technology resources available (the classroom computers were still using Windows 98). There was no computer lab available to us during the period when the class met, but we reminded ourselves that is was a writing activity, not a computer lesson!

While we struggled with the technology (or lack thereof) as well as the wide variation in students' technology skills, we explored a variety of strategies to be able to accomplish this task with the resources at hand. Several students brought in their own computers, usually at the wrong time. After all of the student stories were written, recorded, and pictures collected, we were in the school until midnight last night, putting them all together using iMovie5, which worked for us flawlessly! We will duplicate a CD of the movies for all of the students next week. It's been an eye-opening experience for me: how to do 22 digital stories with 8th graders using two Mac G4 Powerbooks, two scanners, two digital cameras and a few other internet-connected computers for finding pictures.

We are both planning digital stories about the process. I was reminded that the project we did with these students in 6 hours of class time (plus a lot of pull-out time for individual work) is what we normally do with adults in 16-24 hours. These are not CDS-quality stories, and we ran out of time to select music to go along with any of them, but most of the students were very pleased when they privately reviewed their stories with me this morning. But I also realize that it would have been impossible for my daughter to do this project alone, with the constraints she has, both in block scheduling (we didn't see the students every day) and with the technology constraints. And she only had 13 students in each class! I have a greater appreciation for my fellow Apple Distinguished Educators who support these types of activities in schools every day! I also know why many, if not most, teachers would not take on such an ambitious project without a good support system, which is lacking in many financially strapped educational systems today. Nor is there time in the curriculum because of accountability demands....but that is another story!

10 Years Ago

Last Friday, I filled out the paperwork to officially retire from the University of Alaska. I am presently cleaning out a few boxes that were still here in storage, going through old files since 1992! One folder held an especially poignant document: the purchase order for the first web server in the School of Education, dated May of 1995! Ten years ago this month. It was from that web server that I published my first pages on electronic portfolios, using the transition.alaska.edu URL. That Mac web server had a 1 GB hard drive, and ran steadily for about 4 years before it was retired. I now have a flash drive in my purse with that much storage! My how technology changed over the last ten years! I just read an online article about the new standards for HD DVDs being debated by the tech companies: up to 30 GB per side!

Podcasting with Audacity

I'm up in Anchorage, working with a variety of people on Digital Storytelling. Last week, I met with my daughter's 8th Grade Language Arts classes, and gave them a brief introduction to Digital Family Stories. Tomorrow, we start constructing their own digital stories. Yesterday, I worked with a colleague and a young mother, to help them both begin the process of recording their stories. One had been working on her story for several months; the other for just a day. By the end of the day, they both had recorded their stories, adding a music track, and were ready to add images to the timeline of their video editors.

I was a little out of my comfort zone, mostly because I was helping them use software that I don't normally use (Audacity and MovieMaker2) on a platform I normally don't use (Windows XP). So, we plunged in together, learning as we went along, making the usual mistakes, and referring to the online reference manual only when we were desperate. By the end of the day, we produced what I suppose is called a "Podcast" file...a digital audio clip suitable for use in a variety of formats, including a digital story, a website, or an iPod or MP3 player.

I had seen Audacity before, but normally use Sound Studio on my Mac to record audio. We normally record each paragraph of a story as a separate file, and then place them in order on one audio track of the movie editor's timeline. On the second audio track, we place the music, and adjust the volume so that it doesn't overpower the voice. But since we were working in MovieMaker2, the free software that comes with Windows XP, we only had one sound track available. So we had to construct the sound track outside of the movie editor. That's why I plunged into learning Audacity.

Audacity is free, open source software, available for Mac OS X, Windows or even Linux. Once I experimented a few times, I determined how to work around the limitations of the software to meet our needs. For example, after recording a track, when you record a second track, it places it in the same file, at the beginning, so that both tracks play simultaneously. Maybe there is a setting I don't know how to change, but we figured out that if you open a new file, record the second clip, select all of it and copy it, you can paste it at the end of the timeline of the first clip (a process we repeated until the story was complete). Within about an hour, both women had their 3 minute stories recorded, paragraph by paragraph. Adding the music was another challenge, but I was able to import a second track in Audacity, and lower the volume under the voice-over track, and produce a final audio file that included both narration and music.

Why am I struggling with free software for this task? Why not purchase software that will do the job more effectively (and also, why not just use iMovie on a Mac???)? For the simple reason that I am working with novices who already have Windows XP computers, and they just want to get started learning the digital storytelling process. Rather than making an investment in new software, which has a higher learning curve (and level of frustration), we chose to use what they had, or could download for free. I warned them about the limitations of the tools, and that they might outgrow the software very soon, but I wanted them to have a successful first experience. Of course, I may grumble later about the difficulty of publishing these movies to more accessible formats, like DVD, but that will be all part of the learning process.

I have been having a debate with other digital storytellers about the pros and cons of using the more high end tools (Adobe Premiere, Final Cut Pro, Pinnacle Studio, etc.). Those are all what I would call "pro-sumer" tools: lots of capability, but with complexity comes confusion and frustration at the beginning of the learning process. And since I began my career studying how people teach themselves how to use personal computers, I know that a positive first experience is important, and the ability to intuitively explore a tool contributes to the process of self-directed learning. One of my mantras (from my dissertation) is, "When learning new tools, use familiar tasks, and when learning new tasks, use familiar tools." When we make it too hard, we turn off the beginners. My goal is to get them excited about what they have achieved, and to understand the process, so that they can transfer that enthusiasm and awareness to more advanced tools when they are ready.

I am convinced that philosophy works with adult learners. This week, I will have an opportunity to test it out with 8th graders. Stay tuned (and wish me luck!)!

The Work of Stories

I'm at MIT's Work of Stories conference in Boston, in a session on Stories in the Classroom. The presenter read a passage from a book and asked us to do a fast write. Here is mine:

That last passage sounds like the type of writing we do in blogs. It is the personal, reflective, first person narrative that is so powerful. I'm also convinced that the power of digital storytelling is in the storyteller's own voice...both literally and rhetorically. That is why I do what I do... what drives my work: to help people find their voice... in their words spoken from the heart.

This was an interesting conference. On the first day, I thought I was on another planet, where almost every presenter in the breakout sessions I attended read their papers to the audience. Is this what an academic conference is really like? Luckily, on the last two days, the presenters either told stories themselves, or had more engaging slides on the screen. There were the usual problems with the technology in a few of the rooms, like the sound didn't work. But I gained some new ideas. It is always delightful to hear Joe Lambert speak, since he provides both humor and quick insights. I also made some new acquaintances, people interested in similar topics. There were a group of us that tended to show up in the same breakout sessions.

The Saturday night session provided examples of MIT faculty storytelling projects. I was especially impressed by one graduate student's electronic brush, that was a combination micro video camera and a brush for electronically painting on a screen. We saw videos of kindergarten children using the brush to copy colors, objects or short video sequences, and then paint what they captured on the computer screen. I hope we will see this tool available as a commercial item soon.

The panelists at the closing session provided an overview of the three days, invoking a bit of controversy, but providing a good way to end the weekend. As one participant observed, "Some people said it was too academic, others said it wasn't academic enough!" That means the program provided both theoretical and practical insights. I'm glad I traveled all the way across the country to attend what was basically a free conference.

Whirlwind Workshops

On Friday and Sunday, I conducted a couple of hands-on workshops which were very interesting. On Friday, I led two half-day (2.5 hours) workshops at Kean University in New Jersey on the Tools for Digital Storytelling: Windows in the morning, Macintosh in the afternoon. I based the workshops on my webpage with a "Getting Started Tour of the Tools."

Digital Storytelling Tools for Windows XP
In this hands-on session, learn about Windows software used for digital storytelling. We will create a short digital story using a photo editor to manage images and two different programs to edit video. All software used will be available for free download from the WWW.
Digital Storytelling Tools for Macintosh OS X
In this hands-on session, learn about using the Macintosh iLife tools for digital storytelling. We will create a short digital story using iPhoto to edit images, and iMovie to edit video.

I was really pleased with the Windows workshop. The software was loaded on the laptops, but not on the presentation machine, so I quickly installed the software before I ran it, to show how easily it could be up and running. We walked through a sampler of Audio Editors and Image Editors, but focused most of our time on MovieMaker2, PhotoStory and Photo to Movie. We used my "Short Movie" exercise (7 images of D.C. and a 23 second recording of President Reagan). A few of the participants hung around almost an extra hour! My impression was that I was really learning along with the participants, not too far ahead of them with these tools, so we had a lot of fun. The afternoon workshop was the first I have conducted using the newest version of iLife (iPhoto5 and iMovieHD). Apple moved some of the commands between versions 4 and 5! I fumbled a little, but we got through all of the programs, including creating movies with those same 7 images using iPhoto, iMovie and Photo to Movie.

On Sunday, I did another hands-on workshop at the International Reading Association Technology Institute in San Antonio, immediately after my after-lunch keynote address. The lab at the San Antonio Convention Center had about 30 Windows computers and 30 Macs. I looked at the computers when I arrived in the morning and couldn't find MovieMaker on the Windows computers. So I loaded the sample files on the Mac desktops. However, one of the more tech-savvy participants found it, so she quickly helped those sitting at the Windows computers to launch the software and also, thanks to two flash drives, load the files. The podium had presentation stations for both platforms connected to a switch box, so I could do a short demo on iMovie for the Mac users, then switch over to MovieMaker2 for the Windows users. We were able to construct a rough edit (add the sound track and place the 7 images on the timeline) on both platforms at the same time, all in 40 minutes!

That is the shortest hands-on workshop I have ever conducted! But people left having some idea about how digital stories are built, using one of these tools. I also realized how similar and different the tools are. MovieMaker2 has the capabilities of iMovie five years ago, but the three step approach seems to scaffold the approach a little more (1. Capture Video, 2. Edit Movie, 3. Finish Movie). With iMovie, there are more options and capabilities (especially the still motion "Ken Burns" effect), but the initial experience for novices can be a little more confusing (how to get started? which tab to click?). To say the least, I was exhausted at the end of that short time, but felt good about what we were able to do, with thanks to Diane Tracey (who asked me to do the workshop and helped the Mac users) and that other techie, whoever she was, who helped with Windows. My public thanks!