One of my goals for this year is to think a lot more often and carefully about evaluating the informal teaching projects I work on. Until now, as a fairly novice academic staff developer, evaluation has been a somewhat haphazard process: feedback forms for workshops, conversations in the corridor with participants, chats with more senior colleagues about plans and screeds of departmental (government) paperwork. As a first step towards my goal, I decided to think more explicitly about evaluation in relation to a project I run in our unit called the Brown Bag series.
This series of meetings was suggested during a merger when two separate units needed to find as many ways as possible to communicate about the work different individuals were engaged with. The sessions are very “easy going” – a staff member nominates a topic for the session and presents/ chairs/ guides the session. Attendance in optional. There is no preset list of topics, and staff are typically alerted by email a couple of days before the session. The sessions are (given the variety of topics, and the voluntary attendance) fairly well attended. A loose interpretation of attendance suggests that all staff members have attended at least one session in the previous academic year, with some attending upwards of 75% of the sessions and most sessions attracting between 10 and 15 staff members.
Embarrassingly, I hadn’t really though much about how to evaluate the success of this type of learning event. I can plan an evaluation for a formal class, but the “looseness” of the voluntary learning event seemed to stymie me, perhaps because I was trying to pair it with the more rigorous forms of evaluation I am more familiar with from the formal classroom context. I enlisted the help of a colleague who specialises in evaluation to help me think it through and together we designed four activities to elicit feedback on particular areas we were interested in.
- Brainstorm the aims of the brown bag sessions in pairs feed into a google doc.
- Personal highlights. Post it activity.
- Advice for brown bag presenters. Five do’s and five don’ts.
- Brown Bag “dreams”
But, as the idiom reminds us, “The best laid schemes o’ mice an’ men/ Gang aft a-gley, [often go awry].” In true Brown Bag fashion, my planned progression through the activities was quickly appropriated and reordered by the group. And I landed up with some very useful but terribly unstructured information! While the information about the Brown Bag series is very useful to me, of more use generally was what I learned about doing evaluations for informal learning experiences.
- Collaboratively, develop a purpose for the evaluation.
- Have a plan. Don’t be wedded to it. Allow the needs (and mood) of the group to steer the session.
- Have an additional note taker / scribe. I just about got away with it because our unit is used to working in google docs, so various folk kindly took additional notes.
- Gather the feedback somewhere communal, share it, and allow for additional comment.
- Feed folks 🙂 anyone who give time to evaluating something that won’t directly help them deserves a cookie!