The SUNY Learning Network (SLN) online teaching self-assessment survey for experienced online faculty

Award Winner: 
2010 Sloan-C Effective Practice Award
Author Information
Author(s): 
Alexandra M. Pickett
Institution(s) or Organization(s) Where EP Occurred: 
SUNY Learning Network
Effective Practice Abstract/Summary
Abstract/Summary of Effective Practice: 

The online teaching self-assessment survey for experienced online faculty turns theory into practice by assisting the experienced online instructors to self-assess on specific indicators of teaching presence from the COI model and the development of online class community in the design of their own online courses and how they teach them. Faculty are asked to self-assess on 20 specific indicators, the survey generates a report giving the instructor a numerical score for each indicator that corresponds to a key of score ranges. (See Appendix B (http://www.slideshare.net/alexandrapickett/teachingsurveyreport) for an example of this survey-generated report and the specific indicators isolated by the survey). The instructor can then see, based on his/her own self-evaluation, what specific areas in the online course need (1) redesign, (2) need some improvement, or (3) effectively demonstrate class community and teaching presence and need no improvement. A companion piece to the survey is a series of handouts that provide examples of the indicators, and suggestions that faculty can use to make improvements in those areas where their self-assessment indicated they needed some improvement: ( http://www.slideshare.net/alexandrapickett/sln-teaching-online-survey-course-review-materials ).

 

Description of the Effective Practice
Description of the Effective Practice: 

The SUNY Learning Network is committed to assisting SUNY campuses and faculty become highly effective online educators who understand online pedagogy/andragogy and best practices, can leverage the appropriate LMS tools and functionality available to meet specific learning objectives, and who can create and manage effectively designed online teaching and learning environments in which to teach and learn.

Every year SLN trains approximately 300 new fully online faculty.  Most of the faculty development efforts of the SLN program and the participating campus-based instructional designers target new online faculty development and course design. However, since 1994, we estimate that we have trained 3,000- 4,000 fully online faculty in the SUNY system to develop and teach effective online courses – that represents about 10% of all SUNY faculty. With this increasingly staggering number of “experienced” online SLN faculty (i.e., those that have participated once in the SLN new faculty development program and developed and delivered their first fully online course via SLN) we needed a way to continue to continue to inform and influence the quality of existing online courses and the ongoing professional development of the online faculty. The SLN faculty development approach defines online course design as an iterative process including a review and revision loop for the continuous improvement and evolution of the online course and the skills of the online instructor.  See Appendix A http://slnfacultyonline.ning.com/page/effective-practices. We developed the SLN experienced online instructor teaching self-assessment survey (http://SLNsuny.edu/teachingsurvey) as a mechanism to programmatically address that aspect of our model (Step 7/ Stage 4) and to specifically target the online course review and revision needs of this growing cadre of experienced online instructors in the SUNY Learning  Network. The SLN online teaching self-assessment survey for experienced online instructors was prototyped and piloted for the fall 2006 SLN online faculty development cycle.

Our research shows that high levels of "Teaching Presence" (Anderson, 2001) - effective instructional design and organization, facilitation of productive discourse and direct instruction -  positively and significantly influence the satisfaction and reported learning of online students. Our research also shows that thereis a positive and significant correlation between student satisfaction and faculty satisfaction.

There is also evidence to suggest that a strong sense of community in the classroom helps reduce student feelings of isolation and “burnout” associated with higher attrition levels in both classroom-based and distance learning. A positive sense of community also promotes the likelihood of student support and information flow, commitment to group goals, cooperation among members and satisfaction with group processes and efforts [e.g. Rovai (2002)].  We believe that thereis a relationship between teaching presence and the development of community in online learning environments - that courses characterized by effective teaching presence are more likely to develop a stronger sense of community on the part of students.

This self-assessment survey continues to be used today as the key element in our SLN returning faculty resources and events. It is used in our returning faculty instructional design institutes, as a stand alone self-assessment for faculty, and as a tool for campus-based online instructional designers to use with their online faculty. It is designed to branch questions to address the needs of both fully online and blended learning online instructors and their courses.

Supporting Information for this Effective Practice
Evidence of Effectiveness: 

How can a faculty development process help faculty to engage in behaviors that are likely to result in productive learning environments, high levels of learning and student satisfaction? To achieve this goal we developed the SLN online teaching self-assessment survey for experienced online faculty. Courses characterized by effective teaching presence are more likely to develop a stronger sense of community on the part of the students resulting in high levels of faculty and student satisfaction and reported learning. Attention to the principles espoused by Bransford and colleagues [3], Chickering and Gamson [4], as well as Garrison and colleagues [6] and Anderson and colleagues [5] (and articulated in our conceptual framework) have proven to be an approach to ensure high quality in the development of online teaching and learning environments. We continue to facilitate understanding of this emerging conceptual framework (See Figure 1 below http://tiny.cc/5zm24) with the SLN community and to seek to improve the experiences of students and faculty in the SUNY Learning Network.

-Fall 2006 faculty satisfaction results:

90.4% satisfied with SLN program support and services.

88.5% would recommend teaching online to a colleague.

95.4% would like to teach online again.

81.6 % were very satisfied with teaching their online course.


-Fall 2006 student satisfaction results:

83.6% % satisfied with SLN program support and services.

85.9% would take another online course in the future.

80.6% were satisfied with their online course.

78.4% reported that they learned a great deal in their online course.

How does this practice relate to pillars?: 

A unique aspect of SLN’s approach to online faculty development and effective course design is its theoretical foundation in social constructivism, and in the Community of Inquiry (COI) model developed by Garrison, Anderson, and Archer http://communitiesofinquiry.com/model , and our own conceptual framework for high quality higher education online learning environments that merges the COI model with  principles espoused by Bransford and colleagues [3], Chickering and Gamson [4], and Anderson and colleagues [5]. Our ongoing contributions to and use of research and scholarly work to inform all aspects of our SLN faculty development and course design practices is a fundamental aspect of our program. It  is articulated consistently throughout the program in the professional development of the 50+ online instructional designers participating in the program, with both new and returning online faculty, and in the effective online teaching and learning practices recommended and online course templates used by all faculty to quick start their online course design process in the program. The online teaching self-assessment survey for experienced online faculty turns theory into practice by assisting the experienced online instructor’s to self-assess on specific indicators of teaching presence from the COI model and the development of online class community in the design of his/her own online course and how they teach it. Faculty are asked to self-assess on 20 specific indicators, the survey generates a report giving the instructor a numerical score for each indicator that corresponds to a key of range of scores. (See Appendix B http://www.slideshare.net/alexandrapickett/teachingsurveyreport for an example of this survey-generated report and the specific indicators isolated by the survey). The instructor can then see, based on his/her own self-evaluation, what specific areas in the online course need (1) redesign, (2) need some improvement, or (3) effectively demonstrate class community and teaching presence and need no improvement. A companion piece to the survey are the hand outs that provide examples of the indicators, and suggestions that faculty can use to make improvements in those areas where their self-assessment indicated they need some improvement See Appendix B http://www.slideshare.net/alexandrapickett/teachingsurveyreport . 

Equipment necessary to implement Effective Practice: 

This online faculty self-assessment is a simple survey - all you need is a web browser and an email address.

Estimate the probable costs associated with this practice: 

This online faculty self-assessment is a simple survey, the innovation lies in the report that it generates to the faculty that aids the experienced online instructor to identify areas in his/her course that they them selves feel might need improvement (See Appendix B for an example report http://www.slideshare.net/alexandrapickett/teachingsurveyreport).  The results can then be used independently by the instructor to complete the review and revision cycle of the course design process to update the online course in preparation for the next delivery, or it can be used as a component of a faculty development event, or one on one with an instructional designer to pinpoint areas in a course that could be improved, thereby giving the instructor, the trainer, or the instructional designer specific areas on which to focus recommendations, suggestions, examples, tips for improvement.

Addressing the issue of course quality with this self-assessment utility just makes good business sense. Anything one can do to maintain and improve the quality of online course offerings and the effectiveness of online instructors will ultimately benefit the student, make it a more engaging and satisfying experience, which intern may attract new students, lead to retention, and speed time to completion. According to recent Eduventures research, one of the current trends in differentiating online education preferences among online student consumers is course quality.  “…more nuanced aspects of online offerings (e.g., platform, pedagogy, support services, partnerships, student body, scheduling, outcomes, etc.) will become much more developed and visible and necessitate a much more complex explanation of online value and for online growth.” (http://www.slideshare.net/alexandrapickett/richard-garretts-online-higher-education-market-update-2008-national-new-york-data - slide 17)

The survey was easy to develop and costs little to maintain. By using an online survey in this manner we also maximize the ways it can be used. We use it as a directed activity in online and f2f faculty development events/trainings and because it is online, it can also be used just-in-time and as a self-service tool by our online faculty and instructional designers. We get a lot of bang out of this little tool. Consistencies and economies of scale are achieved by using and maintaining one survey for use by all in the program.  

References, supporting documents: 

SUPPORTING LINKS

Link to the survey - http://sln.suny.edu/teachingsurvey

Link to the survey's companion handouts - http://www.slideshare.net/alexandrapickett/sln-teaching-online-survey-course-review-materials

Link to Appendix A - http://slnfacultyonline.ning.com/page/effective-practices

Link to Appendix B - http://www.slideshare.net/alexandrapickett/teachingsurveyreport

Link to Figure 1 - http://tiny.cc/5zm24

 

REFERENCES

1. Shea, P., Fredericksen, E., Pickett, A., and Pelz, W. A Preliminary Investigation of Teaching Presence in the SUNY Learning Network, Elements of Quality Online Education: Practice and Direction, Volume 4 in the Sloan-C series. Needham, MA: Sloan-C, 2003.

2. Shea, P., Pickett, A., and Pelz, W. A Follow- up investigation of “Teaching Presence” in the SUNY Learning Network. Journal of Asynchronous Learning Networks, 7(2) (July1003).

3. Bransford, J., Brown, A., Cocking, R., Donovan, M., and Pellegrino, J. W. How People Learn, National Academy Press, 2000

4. Chickering, A. W., and Gamson, A. F. Seven Principles for Good Practice in Undergraduate Education. Racine, WI: The Johnson Foundation, Inc/Wingspread, 1987

5. Anderson, T., Rourke, L., Garrison, D. R., and Archer W. Assessing Teaching Presence in a Computer Conferencing Context. Journal of Asynchronous Learning Networks, 5(2) (September2001).

6. Garrison, D. R., Anderson, T, and Archer, W. Critical Inquiry in a Text Based Environment: Computer Conferencing in Higher Education. The Internet and Higher Education, 2(2-3): 1-19, 2000.

7. Rourke, L., Anderson, T., Garrison, D. R., and Archer, W. Assessing Social Presence in

Asynchronous Text-based Computer Conferencing. Journal of Distance Education, 2001. http://cade.athabascau.ca/vol14.2/rourke_et_al.html

Useful Links: 
The SUNY Learning Network (SLN) online teaching self-assessment survey for experienced online faculty
Other Comments: 

the idea of having faculty reflect, review, evaluate, self-assess, and then revise and evolve their online course design and teaching practices is just good practice in terms of operationalizing a continuous improvement loop in any online faculty development and course design process – this gives you the ability to continue to inform and influence the quality of the course and the professional development of the instructor with advances in the field, new approaches, best practices, etc. Anyone could create such a programmatic practice by simply building program elements to assist faculty to self-reflect on what worked and what needs improvement in their online teaching and course design and taking the opportunity to update faculty with current research findings and approaches in the design and enhancement of effective online teaching and learning environments.

Our survey self-assessment is currently a resource open to for use by anyone. We provide public access to this resource http://sln.suny.edu/teachingsurvey in the spirit of open sharing, community and collaboration, and to promote effective practices in online faculty development and course design. There are a number of institutions, groups, and communities external to SUNY currently using the tool as part of their online faculty development efforts: e.g., FOR-PD - University of Central Florida; National Park Community College, Arkansas; STARLINK professional development network; and University of North Florida.

Contact(s) for this Effective Practice
Effective Practice Contact: 
alexandra m. pickett
Email this contact: 
alexandra.pickett@suny.edu
Tags:
© 2012, The Sloan Consortium