Effectiveness is measured by a combination of means:
Attendance - Attendance was counted in two ways: 1) Participants attending a live session; and 2) Individuals clicking on and watching the archive of the webinar session. In Winter & Spring Quarters, we logged over 400 attendees, and are tracking over 200 participants thus far for Sumer Quarter.
Satisfaction - The teams built a satisfaction survey using Survey Monkey to rate specific components of content and presenter effectiveness, as well as overall effectiveness of the workshop in which the individual participated. 92.1% of attendees rated the workshop as "Excellent or Very Good", with only a 1.2% rating of "Fair or Poor". Participants were also encouraged to provide comments:
"Keep on doing what you do because it was great and very helpful for students!"
"everything was what i needed to know on how to write a great paper."
"I really wish that this workshop was available to me in my first quarter not when I am almost done. This information is so helpful and useful I was amazed, almost made typing APA style papers EASY! I think this seminar should be for EVERY student in their first quarter just as much as in their last."
Peer Review - in Summer Quarter, our Library & Learning Center Assessment Committee developed a Peer Review rubric specifically for webinar evaluation. Online instructors, deans, and team members participated in observing several workshops throughout the quarter, and then sent the feedback to the speaker and to the Director of Library & Learning Resources for documentation purposes. While the rubric ratings & feedback are designed to help the presenter improve his/her technique for future webinars, the reviewer was also asked to rate whether or not the webinar should be offered in future quarters, and comment on proposed changes to format & content.
Formative Assessment/Work Product: Because student attendance is not mandated at webinars, nor tied to a specific course or assignment, formative assessment is difficult to implement and track. However, several presenters had success using the "polling" tools in the webinar technology to gauge learner understanding of objectives. Additionally, when we transitioned to the small-group work sessions on writing assignments, we were able to clearly denote which students were able to make changes to their content based on instruction during the webinar, and which had additional questions and required follow-up. While this is our greatest challenge area, it is our goal to specifically improve and continue to integrate more opportunities for formative assessment into each webinar, as this is the most effective way to guage the quality of the webinar experience for the student.
Faculty Satisfaction: Our measures of faculty satisfaction (based on their perceptions of effectiveness related to students' classroom success) are anectdotal, at best. We consult with faculty during the planning stage to garner suggestions for changes to webinars or additions; we also seek out faculty to serve as speakers. Faculty feedback also drove the changes to our marketing for Fall Quarter, 2011: specifcally, the move to organizing and marketing webinars by track.