The CUNY Young Adult Program - Utilizing Social Networking to Foster Interdisciplinary and Cross-Cohort Student Communication during Workforce Training

Volume, Issue - Date:

Volume 14, Issue 3 - November 2010

Author(s):

Alissa Levine, CUNY Kingsborough Community College

Author(s):

Christoph Winkler, The City University of New York

Author(s):

Saul Petersen, Connecticut Campus Compact

Full article - Free: Click on the file to download.:

jaln_v14n3_levine_8.pdf

Keywords:

access, workforce development, community college, innovation, social networking, pedagogy, retention

Abstract:

The Center for Economic and Workforce Development (CEWD) at Kingsborough Community College is currently working on a workforce development project that contains innovative teaching tools that proved successful in overcoming issues of academic isolation

Tags:

Login or register to post comments

Incorporating the e-HIM® Virtual Lab into the Health Information Administration Professional Practice Experience

Volume, Issue - Date:

Volume 14, Issue 3 - November 2010

Author(s):

Amanda C. Barefield, Medical College of Georgia

Author(s):

Jim Condon, Medical College of Georgia

Author(s):

Charlotte McCuen, Macon State College

Author(s):

Nanette B. Sayles, Macon State College

Full article - Free: Click on the file to download.:

jaln_v14n3_barefield_7.pdf

Keywords:

access, online laboratory, professional practice, health information management

Abstract:

This article highlights the experiences of two baccalaureate Health Information Administration (HIA) programs in the adoption of the American Health Information Management Association’s (AHIMA) e-HIM Virtual Laboratory (Virtual Lab) into the Professional Practice Experience (PPE). Information is provided describing the implementation of the Virtual Lab, issues that were encountered, and subsequent feedback from students and faculty regarding this new technology.

Tags:

Login or register to post comments

The Sloan-C Pillars: Towards a Balanced Approach to Measuring Organizational Learning

Volume, Issue - Date:

Volume 14, Issue 2 - November 2010

Author(s):

Kee Meng Yeo, Amway

Author(s):

A. Frank Mayadas, The Alfred P. Sloan Foundation and e Sloan Consortium

Full article - Free: Click on the file to download.:

yeo.pdf

Keywords:

scale, learning assessment, online learning, e-learning, corporate training, Kirkpatrick method, Sloan-C Pillars

Abstract:

The Sloan Pillars have set the standard for university-wide online learning program assessment for more than a dozen years. In this paper, the authors propose the extension of the Pillars to corporate e-learning, offering an alternative to traditional enterprise learning assessments. Claiming that conventional methods stress individual courses or programs, rather than encompassing a company’s learning ecology, a new corporate version of Sloan Pillars is proposed. The authors claim that their holistic approach—assessing employee access, learning and cost effectiveness, and learner and management satisfaction—provides companies with the tools they need to assess the effectiveness of company learning efforts overall.

Tags:

Login or register to post comments

Professional development for and evaluation of online course instructors: Research-based best practices

Author Information

Author(s):

Barbara Storandt, ALTA Solutions Group, LLC

Institution(s) or Organization(s) Where EP Occurred:

PBS TeacherLine

Effective Practice Abstract/Summary

Abstract/Summary of Effective Practice:

This yearlong formative-summative research study evaluated PBS TeacherLine’s model for selecting, supporting and evaluating its online instructors. The study began during the fall of 2009, collected data during each of four course terms, and concluded in July 2010.

Description of the Effective Practice

Description of the Effective Practice:

Selecting the right instructors

Only high quality online instructors can provide high quality instruction. PBS TeacherLine requires all online course instructors to hold master’s degrees (16.3% hold CAS/EdS or PhD degrees) and have prior experience as online learners before facilitating a course. At least one third of instructors had enrolled in 30 or more online courses as learners before the Fall 2009 course term, and half had been teaching online for more than five years at that time.

Differentiated professional development

At the core of instructor training and support is the importance of positioning online instructors as learners themselves. PBS TeacherLine achieves this through three modes:

Two online courses developed by PBS TEacherLine convey best practices for online instruction early in his/her tenure. Nearly sixty percent of PBS TeacherLine’s instructors (59.4%) have completed both courses, and instructors rate the quality of these experiences very highly.

3D course reviews serve the dual purpose of providing professional development to online instructors through formative evaluation and providing a means by which to evaluate their performance through summative evaluation. 3D course reviews position three perspectives on an instructor’s performance during a course – instructor/self, the course provider and learners (i.e. k-12 teachers) – side by side for an individualized account of instructional strengths and areas of needed improvement. The 3D course reviews are designed to increase dialogue about high-quality online instruction and generate a culture of inquiry among instructors.

Embedded learning opportunities through faculty meetings, peer mentoring, a moderated professional learning community, and a resource library support instructors’ ongoing information and learning needs in real time.

Supporting Information for this Effective Practice

Evidence of Effectiveness:

Over the course of the study, each of PBS TeacherLine’s approximately 110 online course instructors participated in at least one, if not multiple, research activities. Online instructor trainers, mentors and the community manager also participated in the research.

Instructors rate the quality of PBS TeacherLine’s professional development and support very highly; superior to most of the preparation they have received from other online course providers. Further, instructors have very few unmet specific learning needs related to online instruction.

An instructor’s performance during a course term was assessed in three ways: by instructors through reflection logs, by PBS TeacherLine who used an evaluative rubric, and by learners (i.e. k-12 teachers) who completed post-course surveys. All feedback on an instructor’s performance from each source was aggregated into one document and returned to instructors through 3D course reviews. Key findings from the 3D course review process include:

As a whole, PBS TeacherLine’s instructors excel at: course climate and community building, instruction, interaction and discussion facilitation, course organization, and implementation of assessments.

Reflection logs prompted instructors to articulate areas for improvement in their own performance, which converged across course terms: landscape posts, presence on the discussion board, and providing constructive, valuable and specific feedback to learners.

Participation in PBS TeacherLine’s 3D course review process promotes critical self-reflection, with instructors gaining the capacity to critically assess their performance over time.

Learner evaluations, examined in aggregate across instructors and terms, also reveal high quality online instruction. Their aggregate ratings of instructor performance align with PBS TeacherLine’s ratings, and both are higher than instructors’ self-assessments. Learners’ satisfaction with course experiences and their descriptions of instructor excellence reinforce these findings.

Most k-12 teacher learners will immediately apply what they learned from the courses offered by PBS TeacherLine as a result of their instructor’s high quality performance.

How does this practice relate to pillars?:

PBS TeacherLine's 3D course review process provides comprehensive data to instructors about their performance, and also makes transparent the processes of self-reflection and performance evaluation.

Estimate the probable costs associated with this practice:

The most substantial cost associated with this practice is staff time required to complete annual 3D course reviews for each instructor, and to compile the data so that each course instructor receives individualized feedback. This cost will vary depending on the number of course instructors that need to be assessed.

Useful Links:

http://www.pbs.org/teacherline

Contact(s) for this Effective Practice

Effective Practice Contact:

Barbara Storandt

Email this contact:

barb@altasolutionsgroup.com

Effective Practice Contact 2:

Lia Dossin

Email contact 2:

lcdossin@pbs.org

Tags:

Faculty Satisfaction

Login or register to post comments

Integrating the Social Presence Model to Maximize Blended and Online Learning Experiences

Author Information

Author(s):

Aimee L. Whiteside

Institution(s) or Organization(s) Where EP Occurred:

University of Minnesota-Twin Cities

Institution(s) or Organization(s) Where EP Occurred:

University of Wisconsin-Stout

Effective Practice Abstract/Summary

Abstract/Summary of Effective Practice:

The Social Presence Model offers a framework to helps students, instructors, and other professionals around the world to maximize their online exchanges through increased awareness of linguistic nuances, social interaction, learning communities, instructor involvement, and prior knowledge and experiences. The Model creates an awareness of the importance of creating critical connections and cultivating relationships in learning communities, which can result in increased student motivation and elevated learning outcomes.

Description of the Effective Practice

Description of the Effective Practice:

This Model is based on programmatic research in social presence (Whiteside, 2007), which extends the Social Presence coding schemes of Rourke, Anderson, Garrison, and Archer (2001) and Swan and Shih (2005) . The previous coding schemes referenced three distinct social presence categories: Affective, Cohesive, and Interactive categories. The Social Presence Model extends the previous research by integrating five key elements that can help instructors and students maximize the learning experience: Affective Association, Community Cohesion, Interaction Intensity, Knowledge and Experience, and Instructor Investment. (See Figure 1 below).

Figure 1. The Social Presence Model

Affective Association addresses the emotional connections in the course. Affection Association examines emotion, humor, sarcasm, paralanguage, and self-disclosure. Community Cohesion represents the extent to which participants see the group as a community. This concept is examined through greetings and salutations used, referring to each other as a community, and the extent to which students and instructors reference each other by name. Interaction Intensity refers to the level of interaction among participants, including direct quotes, paraphrasing, complimenting, and asking questions. Knowledge and Experience involves sharing of additional resources and experiences. Finally, Instructor Investment refers to the extent to which the instructor is an invested, active partner in the learning community.

Supporting Information for this Effective Practice

Evidence of Effectiveness:

In a study involving two iterations of a 13-month graduate-level certificate program at a large Midwestern University (Whiteside, 2007), the instructors emphasized the importance of establishing relationships in their courses to spur social presence. They indicated that “learning stems from relationships” and if students “don’t have a relationship with somebody” or a connection to them, then students are not invested in each and they do not have “the incentive to interact.” Each of the instructors and students interviewed found that the initial community building activities were essential to establishing the foundation for social presence, building relationships, and extending overall learning. When that community building and social presence falters, a student suggested, so does the “overall learning.”

This case study research suggests that the Social Presence Model and its five key elements help facilitators increase the level of trust and respect in an online community, which can motivate student participants to take a more active role in their own and their peers’ construction of knowledge. The Model is important because many instructors are completely still new to effective online learning practices, and they may need scaffolding and assistance to help them create an ideal learning environment for their students. The Social Presence Model provides an essential framework to compliment the pedagogical and technical support that instructors new to campus and new to online learning need. Additionally, many students are also new to online learning and introducing them to a framework can help create an awareness of their actions and correspondences in a way that they might not have considered.

How does this practice relate to pillars?:

This practice improves three pillars: learning effectiveness, faculty satisfaction, and student satisfaction.

In the above mentioned study, instructors noted that “learning stems from relationships” and if students “don’t have a relationship with somebody” or a connection to them, then students are not invested in each and they do not have “the incentive to interact.” The Model relates to faculty satisfaction and learning effectiveness because it is a heuristic that can be integrated to help instructors design effective online learning practices—to help them create a more ideal learning environment for them and for their students. The Social Presence Model provides an essential framework for instructors who are new to campus, are new to online learning, or are who are seeking additional pedagogical assistance for blended/online learning environments.

In relation to student satisfaction and learning effectiveness, each of the students interviewed in the above mentioned study found that the initial community building activities were essential to establishing the foundation for social presence, building relationships, and extending overall learning. This data suggests that many students are either new to blended/online learning or they are still seeking advice for the blended/online learning environment. This framework can help students create an awareness of their actions and correspondences in a way that they may not have previously considered. When that community building and social presence falters, one student suggested, so does the “overall learning.”

Equipment necessary to implement Effective Practice:

No equipment is necessary to implement this Effective Practice.

Estimate the probable costs associated with this practice:

There are no probable costs associated with this Effective Practice.

References, supporting documents:

Anderson, T. (2008). Social software to support distance education learners. In T. Anderson (Ed.). Theory and practice of online learning (pp. 221-244). 2nd ed. Edmonton: Athabasca University.

Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice-Hall.

Bandura, A. (1973). Aggression: A social learning analysis. Englewood Cliffs, NJ: Prentice-Hall.

Bandura, A. (1977). Social learning theory. New York: General Learning Press.

Bai, H. (2003). Student motivation and social presence in online learning: Implications for future research. In C. Crawford et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2003 (pp. 2714-2720). Albuquerque, NM: Association for the Advancement of Computing in Education.

Chen, N. S., Kinshuk, Wei, C. W., & Wang, M. J. (2009). A framework for social presence in synchronous cyber classrooms. In Proceedings of the 9th IEEE International Conference on Advanced Learning Technologies (pp. 40-44). Riga, Latvia: IEEE Computer Society Press.

Collison, G., Elbaum, B., Haavind, S. & Tinker, R. (2000). Facilitating online learning: Effective strategies for moderators. Madison, WI: Atwood Publishing.

Conrad, R. & Donaldson, A. (2004). Engaging the online learner: Activities and resources for creative Instruction. San Francisco, CA: Jossey-Bass.

Dewey, J. (1916) Democracy and education: An introduction to the philosophy of education (1966 ed.), New York: Free Press.

Dewey, J. (1910). How we think. Boston: D. C. Heath.

Dunlap, J. C., & Lowenthal, P.R. (2009). Tweeting the night away: Using Twitter to enhance social presence. Journal of Information Systems Education, 20(2), 129-136.

DuVall, J. B., Powell, M. R., Hodge, E., & Ellis, M. (2007). Text messaging to improve social presence in online learning. Educause Quarterly, 3, 24-28. Retrieved from http://net.educause.edu/ir/library/pdf/EQM0733.pdf

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95-105.

Gunawardena, C. N., & Zittle, F. J. (1997). Social presence as a predictor of satisfaction within a computer-mediated conferencing environment. The American Journal of Distance Education, 11(3), 8-26.

Hingorani, K. K. (2008). Social presence, personality types, and IT-supported teaching methods. Issues in Information Systems, 9(2), 56-62.

Hodge, E. M., Tabrizi, M. H. N., Farwell, M. A., & Wuensch, K. L. (2007). Virtual reality classrooms: Strategies for creating a social presence. International Journal of Social Sciences, 2, 105-109.

Hofstede, Geert. (2001). Culture's consequences: Comparing values, behaviors, institutions, and organizations across nations. Thousand Oaks CA: Sage.

Joyce, K. M., & Brown, A. (2009). Enhancing social presence in online learning: Mediation strategies Applied to Social Networking Tools. Online Journal of Distance Learning Administration, 12(4). Retrieved from http://www.westga.edu/~distance/ojdla/winter124/joyce124.html

Jusoff, K., & Khodabandelou, R. (2009). Preliminary study on the role of social presence in blended learning. International Education Studies, 2(4), 79-83.

Kehrwald, B. (2007) The ties that bind: social presence, relations, and productive collaboration in online learning environments. In Proceedings of ASCILITE Conference, Singapore. Retrieved from http://www.ascilite.org.au/conferences/singapore07/procs/kehrwald.pdf

Kekwaletswe, R. M. (2007). Social presence awareness for knowledge transformation in a mobile learning environment. International Journal of Education and Development using Information and Communication Technology, 3(4), 102-109.

Lu, J., Huang, W., Ma, H., & Luce, T. (2007). Interaction and social presence in technology-mediated learning: A partial least squares model. In Proceedings IEEE 3rd International Conference on Wireless Communications, Networking and Mobile Computing. Shanghai, China.

Marcus, S. (2006). Measure by measure: How WBT can help create a social online presence. Campus-Wide Information Systems, 23(2), 56-67.

May, G. L., & Short, D. (2003). Gardening in cyberspace: A metaphor to enhance online teaching and learning. Journal of Management Education, 27, 673-693.

Mehrabian, A. (1969). Some referents and measures of nonverbal behavior. Behavior Research Methods and Instruction, 1(6), 205-207.

Mykota, D., & Duncan, R. (2007). Learner characteristics as predictors of online social presence. Canadian Journal of Education, 30 (1), 157-170.

Na Ubon, A. & Kimble, C. (2003). Supporting the creation of social presence in online learning communities using asynchronous text-based CMC. In Proceedings of the 3rd International Conference on Technology in Teaching and Learning in Higher Education (pp.295-300). Heidelberg, Germany.

Picciano, A. G. (2002). Beyond student perceptions: issues of interaction, presence, and performance in an online course. Journal of Asynchronous Learning Networks, 6(1), 21-40.

Polhemus, L., Shih, L. F., & Swan, K. (2001). Virtual interactivity: The representation of social presence in an online discussion. Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA.

Rettie, R. (2003). Connectedness, awareness and social presence. 6th International Presence Workshop. Aalborg.

Richardson, J., & Swan, K. Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks 6(1), 76–90.

Rourke, L., Anderson, T., Garrison, D. R. & Archer, W. (2001) Assessing social presence in asynchronous textbased computer conferencing. Journal of Distance Education, 14(2), 51-70.

Shen, K. N. and Khalifa, M. (2007). Exploring multi-dimensional conceptualization of social presence in the context of online communities. In Jacko, J. (Ed), Proceedings of the Human-Computer Interaction, 999-1008.

Short, J., William, E., & Christie, B. (1976). The social psychology of telecommunications. Toronto, ON: Wiley.

Stacey, E. (2002). Social presence online: Networking learners at a distance, education and information technologies. Education and Information Technologies, 7(4), 287-294.

Stein, D., & Wanstreet, C. (2003). Role of social presence, choice of online or face-to-face group format, and satisfaction with perceived knowledge gained in a distance learning environment. Paper Presented at the 2003 Midwest Research to Practice Conference in Adult Continuing and Community Education.

Swan, K. (2002). Building communities in online courses: The importance of interaction. Education, Communication and Information, 2(1), 23-49.

Tu, C. (2001). How Chinese perceive social presence: An examination inline learning environment. Educational Media Internal, 38(1), 45-60.

Tu, C. (2002). The measurement of social presence in an online learning environment. International Journal on E-Learning 1(2), 34-45.

Vesely, P., Bloom, L., & Sherlock, J. (2007). Key elements of building online community: Comparing faculty and student perceptions. Journal of Online Learning and Teaching, 3, 234-246.

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, England: Cambridge UP.

Whiteside, A. L. (2007). Exploring social presence in communities of practice within a hybrid learning environment: A longitudinal examination of two case studies within the School Technology Leadership graduate-level certificate program. Unpublished doctoral dissertation, University of Minnesota.

Whiteside, A. L., Hughes, J. E., & McLeod, S. (2005.) Interconnecting cognition, contact, and comprehension: The influence of social presence in a hybrid-model certificate program.
In Proceedings of the New Media Research Conference. University of Minnesota.

Wise, A., Chang, J., Duffy, T., & Del Valle, R. (2004). The effects of teacher social presence on student satisfaction, engagement, and learning. Journal of Educational Computing Research, 31(3),247-271.

Woods, R., & Ebersole, S. (2003). Becoming a “communal architect” in the online classroom—Integrating cognitive and affective learning for maximum effect in web-based learning. Online Journal of Distance Learning Administration, 6(1). Retrieved from www.westga.edu/~distance/ojdla/spring61/woods61.htm.

Yildiz, S. (2009). Social presence in the Web-based classroom: Implications for intercultural communication. Journal of Studies in International Education, 13, 46-67.

Contact(s) for this Effective Practice

Effective Practice Contact:

Aimee Whiteside

Email this contact:

whitesidea@uwstout.edu

Tags:

Login or register to post comments

A Tool for Faculty Peer Review of Online Teaching

Author Information

Author(s):

Ann H. Taylor

Institution(s) or Organization(s) Where EP Occurred:

Penn State University

Effective Practice Abstract/Summary

Abstract/Summary of Effective Practice:

To address the need for online course peer review in the College of Earth and Mineral Sciences at Penn State, the Dutton e-Education Institute has designed, implemented, and assessed a peer review process for online teaching. The Peer Review Guide for Online Teaching at Penn State is composed of two parts: 1) an Instructor Input Form to be completed for the reviewer by the reviewee in advance of the peer review and 2) the actual Peer Review Guide for Online Teaching at Penn State, which is to be completed by the reviewer during the peer review. Following the peer review, the reviewer summarizes her observations in a document that is to be included in the reviewee's dossier—identical to the procedure followed in resident instruction. Reviewers are encouraged to share the completed Guide with the reviewee, as well.

Description of the Effective Practice

Description of the Effective Practice:

The peer review of teaching–like the peer review of research–is a widely accepted mechanism for promoting and assuring quality academic work, and is required the purpose of promotion and tenure at Penn State. The peer review process in resident instruction typically involves a faculty reviewer observing a peer’s classroom. The reviewer then summarizes her observations in a document that is to be included in the reviewee’s dossier.

To address the need for online course peer review in the College of Earth and Mineral Sciences at Penn State, the Dutton Institute has designed, implemented, and assessed a peer review process for online teaching. The Peer Review Guide for Online Teaching at Penn State that we have developed is based on the “Seven Principles for Good Practice in Undergraduate Education,” a summary of 50 years of higher education research that addressed good teaching and learning practices. While instruments such as end-of-course surveys provide a measure of student satisfaction with a course, the Seven Principles provide a useful framework to evaluate the effectiveness of online teaching. Each adapted principle is described in detail in the Guide, including examples of evidence of how a principle may be met in an online course. Resources for additional information are also included.

The Peer Review Guide for Online Teaching at Penn State is composed of two parts: 1) an Instructor Input Form to be completed for the reviewer by the reviewee in advance of the peer review and 2) the actual Peer Review Guide for Online Teaching at Penn State, which is to be completed by the reviewer during the peer review. Following the peer review, the reviewer summarizes her observations in a document that is to be included in the reviewee's dossier—identical to the procedure followed in resident instruction. Reviewers are encouraged to share the completed Guide with the reviewee, as well.

Use of this peer review process has already been fully embraced by one academic department within the College of Earth and Mineral Sciences and is now being expanded to other units. While initial pilot tests utilized online teaching faculty as the peer reviewers, we will begin utilizing faculty who are only familiar with traditional face-to-face instruction as peer reviewers, as well. It is hoped that this "cross-fertilization" will break down many of the myths that still surround online teaching at our institution, as face-to-face faculty will be able to examine online teaching and learning effectiveness first hand.

Supporting Information for this Effective Practice

Evidence of Effectiveness:

Fictionalized examples of dossier letters (based on actual memos):

How does this practice relate to pillars?:

The peer review process we have developed helps to ensure adequate support for faculty in online course delivery. To begin with, faculty are given a copy of the Guide in advance of teaching their online course so they can review the criteria upon which their teaching will be evaluated. Through that use, the Guide serves as a valuable faculty development tool, especially when there is an opportunity to review the document with a learning designer or other faculty development expert. The peer review process itself provides reviewees with meaningful and focused feedback on their online teaching that includes information about resources they can use to improve their teaching practices. When necessary, departments can use the information obtained through this process to develop a customized faculty development plan for the reviewee that will help that individual improve future performance and teaching satisfaction.

Equipment necessary to implement Effective Practice:

Faculty reviewers simply need access to the reviewee's online course. The process encourages reviewer and reviewee to communicate throughout the process, especially when evidence of effective teaching practices outlined in the Guide are not clearly available. For example, the faculty reviewer may need to request from the reviewee copies of e-mail interactions between instructor and students when seeking evidence of faculty-student interactions.

Estimate the probable costs associated with this practice:

The only cost involved with this practice is time. The peer review process is designed to take approximately 2 hours for a reviewer to complete. Some faculty involved in our pilot test, however, reported voluntarily spending longer amounts of time reviewing their peer's online course out of professional curiosity!

References, supporting documents:

Chickering, A. & Gamson, Z. (1987). Seven principles for good
practice in undergraduate education. AAHE Bulletin (39 )7.

Useful Links:

Peer Review for Online Teaching at Penn State

Other Comments:

We presented our work at the 16th Annual Sloan-C Conference in November 2010.

Contact(s) for this Effective Practice

Effective Practice Contact:

Ann H. Taylor

Email this contact:

atb3@psu.edu

Tags:

Faculty Satisfaction

Login or register to post comments

The SUNY Learning Network (SLN) online teaching self-assessment survey for experienced online faculty

Award Winner:

2010 Sloan-C Effective Practice Award

Author Information

Author(s):

Alexandra M. Pickett

Institution(s) or Organization(s) Where EP Occurred:

SUNY Learning Network

Effective Practice Abstract/Summary

Abstract/Summary of Effective Practice:

The online teaching self-assessment survey for experienced online faculty turns theory into practice by assisting the experienced online instructors to self-assess on specific indicators of teaching presence from the COI model and the development of online class community in the design of their own online courses and how they teach them. Faculty are asked to self-assess on 20 specific indicators, the survey generates a report giving the instructor a numerical score for each indicator that corresponds to a key of score ranges. (See Appendix B (http://www.slideshare.net/alexandrapickett/teachingsurveyreport) for an example of this survey-generated report and the specific indicators isolated by the survey). The instructor can then see, based on his/her own self-evaluation, what specific areas in the online course need (1) redesign, (2) need some improvement, or (3) effectively demonstrate class community and teaching presence and need no improvement. A companion piece to the survey is a series of handouts that provide examples of the indicators, and suggestions that faculty can use to make improvements in those areas where their self-assessment indicated they needed some improvement: ( http://www.slideshare.net/alexandrapickett/sln-teaching-online-survey-course-review-materials ).

Description of the Effective Practice

Description of the Effective Practice:

The SUNY Learning Network is committed to assisting SUNY campuses and faculty become highly effective online educators who understand online pedagogy/andragogy and best practices, can leverage the appropriate LMS tools and functionality available to meet specific learning objectives, and who can create and manage effectively designed online teaching and learning environments in which to teach and learn.

Every year SLN trains approximately 300 new fully online faculty. Most of the faculty development efforts of the SLN program and the participating campus-based instructional designers target new online faculty development and course design. However, since 1994, we estimate that we have trained 3,000- 4,000 fully online faculty in the SUNY system to develop and teach effective online courses – that represents about 10% of all SUNY faculty. With this increasingly staggering number of “experienced” online SLN faculty (i.e., those that have participated once in the SLN new faculty development program and developed and delivered their first fully online course via SLN) we needed a way to continue to continue to inform and influence the quality of existing online courses and the ongoing professional development of the online faculty. The SLN faculty development approach defines online course design as an iterative process including a review and revision loop for the continuous improvement and evolution of the online course and the skills of the online instructor. See Appendix A http://slnfacultyonline.ning.com/page/effective-practices. We developed the SLN experienced online instructor teaching self-assessment survey (http://SLNsuny.edu/teachingsurvey) as a mechanism to programmatically address that aspect of our model (Step 7/ Stage 4) and to specifically target the online course review and revision needs of this growing cadre of experienced online instructors in the SUNY Learning Network. The SLN online teaching self-assessment survey for experienced online instructors was prototyped and piloted for the fall 2006 SLN online faculty development cycle.

Our research shows that high levels of "Teaching Presence" (Anderson, 2001) - effective instructional design and organization, facilitation of productive discourse and direct instruction - positively and significantly influence the satisfaction and reported learning of online students. Our research also shows that thereis a positive and significant correlation between student satisfaction and faculty satisfaction.

There is also evidence to suggest that a strong sense of community in the classroom helps reduce student feelings of isolation and “burnout” associated with higher attrition levels in both classroom-based and distance learning. A positive sense of community also promotes the likelihood of student support and information flow, commitment to group goals, cooperation among members and satisfaction with group processes and efforts [e.g. Rovai (2002)]. We believe that thereis a relationship between teaching presence and the development of community in online learning environments - that courses characterized by effective teaching presence are more likely to develop a stronger sense of community on the part of students.

This self-assessment survey continues to be used today as the key element in our SLN returning faculty resources and events. It is used in our returning faculty instructional design institutes, as a stand alone self-assessment for faculty, and as a tool for campus-based online instructional designers to use with their online faculty. It is designed to branch questions to address the needs of both fully online and blended learning online instructors and their courses.

Supporting Information for this Effective Practice

Evidence of Effectiveness:

How can a faculty development process help faculty to engage in behaviors that are likely to result in productive learning environments, high levels of learning and student satisfaction? To achieve this goal we developed the SLN online teaching self-assessment survey for experienced online faculty. Courses characterized by effective teaching presence are more likely to develop a stronger sense of community on the part of the students resulting in high levels of faculty and student satisfaction and reported learning. Attention to the principles espoused by Bransford and colleagues [3], Chickering and Gamson [4], as well as Garrison and colleagues [6] and Anderson and colleagues [5] (and articulated in our conceptual framework) have proven to be an approach to ensure high quality in the development of online teaching and learning environments. We continue to facilitate understanding of this emerging conceptual framework (See Figure 1 below http://tiny.cc/5zm24) with the SLN community and to seek to improve the experiences of students and faculty in the SUNY Learning Network.

-Fall 2006 faculty satisfaction results:

90.4% satisfied with SLN program support and services.

88.5% would recommend teaching online to a colleague.

95.4% would like to teach online again.

81.6 % were very satisfied with teaching their online course.

-Fall 2006 student satisfaction results:

83.6% % satisfied with SLN program support and services.

85.9% would take another online course in the future.

80.6% were satisfied with their online course.

78.4% reported that they learned a great deal in their online course.

How does this practice relate to pillars?:

A unique aspect of SLN’s approach to online faculty development and effective course design is its theoretical foundation in social constructivism, and in the Community of Inquiry (COI) model developed by Garrison, Anderson, and Archer http://communitiesofinquiry.com/model , and our own conceptual framework for high quality higher education online learning environments that merges the COI model with principles espoused by Bransford and colleagues [3], Chickering and Gamson [4], and Anderson and colleagues [5]. Our ongoing contributions to and use of research and scholarly work to inform all aspects of our SLN faculty development and course design practices is a fundamental aspect of our program. It is articulated consistently throughout the program in the professional development of the 50+ online instructional designers participating in the program, with both new and returning online faculty, and in the effective online teaching and learning practices recommended and online course templates used by all faculty to quick start their online course design process in the program. The online teaching self-assessment survey for experienced online faculty turns theory into practice by assisting the experienced online instructor’s to self-assess on specific indicators of teaching presence from the COI model and the development of online class community in the design of his/her own online course and how they teach it. Faculty are asked to self-assess on 20 specific indicators, the survey generates a report giving the instructor a numerical score for each indicator that corresponds to a key of range of scores. (See Appendix B http://www.slideshare.net/alexandrapickett/teachingsurveyreport for an example of this survey-generated report and the specific indicators isolated by the survey). The instructor can then see, based on his/her own self-evaluation, what specific areas in the online course need (1) redesign, (2) need some improvement, or (3) effectively demonstrate class community and teaching presence and need no improvement. A companion piece to the survey are the hand outs that provide examples of the indicators, and suggestions that faculty can use to make improvements in those areas where their self-assessment indicated they need some improvement See Appendix B http://www.slideshare.net/alexandrapickett/teachingsurveyreport .

Equipment necessary to implement Effective Practice:

This online faculty self-assessment is a simple survey - all you need is a web browser and an email address.

Estimate the probable costs associated with this practice:

This online faculty self-assessment is a simple survey, the innovation lies in the report that it generates to the faculty that aids the experienced online instructor to identify areas in his/her course that they them selves feel might need improvement (See Appendix B for an example report http://www.slideshare.net/alexandrapickett/teachingsurveyreport). The results can then be used independently by the instructor to complete the review and revision cycle of the course design process to update the online course in preparation for the next delivery, or it can be used as a component of a faculty development event, or one on one with an instructional designer to pinpoint areas in a course that could be improved, thereby giving the instructor, the trainer, or the instructional designer specific areas on which to focus recommendations, suggestions, examples, tips for improvement.

Addressing the issue of course quality with this self-assessment utility just makes good business sense. Anything one can do to maintain and improve the quality of online course offerings and the effectiveness of online instructors will ultimately benefit the student, make it a more engaging and satisfying experience, which intern may attract new students, lead to retention, and speed time to completion. According to recent Eduventures research, one of the current trends in differentiating online education preferences among online student consumers is course quality. “…more nuanced aspects of online offerings (e.g., platform, pedagogy, support services, partnerships, student body, scheduling, outcomes, etc.) will become much more developed and visible and necessitate a much more complex explanation of online value and for online growth.” (http://www.slideshare.net/alexandrapickett/richard-garretts-online-higher-education-market-update-2008-national-new-york-data - slide 17)

The survey was easy to develop and costs little to maintain. By using an online survey in this manner we also maximize the ways it can be used. We use it as a directed activity in online and f2f faculty development events/trainings and because it is online, it can also be used just-in-time and as a self-service tool by our online faculty and instructional designers. We get a lot of bang out of this little tool. Consistencies and economies of scale are achieved by using and maintaining one survey for use by all in the program.

References, supporting documents:

SUPPORTING LINKS

Link to the survey - http://sln.suny.edu/teachingsurvey

Link to the survey's companion handouts - http://www.slideshare.net/alexandrapickett/sln-teaching-online-survey-course-review-materials

Link to Appendix A - http://slnfacultyonline.ning.com/page/effective-practices

Link to Appendix B - http://www.slideshare.net/alexandrapickett/teachingsurveyreport

Link to Figure 1 - http://tiny.cc/5zm24

REFERENCES

1. Shea, P., Fredericksen, E., Pickett, A., and Pelz, W. A Preliminary Investigation of Teaching Presence in the SUNY Learning Network, Elements of Quality Online Education: Practice and Direction, Volume 4 in the Sloan-C series. Needham, MA: Sloan-C, 2003.

2. Shea, P., Pickett, A., and Pelz, W. A Follow- up investigation of “Teaching Presence” in the SUNY Learning Network. Journal of Asynchronous Learning Networks, 7(2) (July1003).

3. Bransford, J., Brown, A., Cocking, R., Donovan, M., and Pellegrino, J. W. How People Learn, National Academy Press, 2000

4. Chickering, A. W., and Gamson, A. F. Seven Principles for Good Practice in Undergraduate Education. Racine, WI: The Johnson Foundation, Inc/Wingspread, 1987

5. Anderson, T., Rourke, L., Garrison, D. R., and Archer W. Assessing Teaching Presence in a Computer Conferencing Context. Journal of Asynchronous Learning Networks, 5(2) (September2001).

6. Garrison, D. R., Anderson, T, and Archer, W. Critical Inquiry in a Text Based Environment: Computer Conferencing in Higher Education. The Internet and Higher Education, 2(2-3): 1-19, 2000.

7. Rourke, L., Anderson, T., Garrison, D. R., and Archer, W. Assessing Social Presence in

Asynchronous Text-based Computer Conferencing. Journal of Distance Education, 2001. http://cade.athabascau.ca/vol14.2/rourke_et_al.html

Useful Links:

The SUNY Learning Network (SLN) online teaching self-assessment survey for experienced online faculty

Other Comments:

the idea of having faculty reflect, review, evaluate, self-assess, and then revise and evolve their online course design and teaching practices is just good practice in terms of operationalizing a continuous improvement loop in any online faculty development and course design process – this gives you the ability to continue to inform and influence the quality of the course and the professional development of the instructor with advances in the field, new approaches, best practices, etc. Anyone could create such a programmatic practice by simply building program elements to assist faculty to self-reflect on what worked and what needs improvement in their online teaching and course design and taking the opportunity to update faculty with current research findings and approaches in the design and enhancement of effective online teaching and learning environments.

Our survey self-assessment is currently a resource open to for use by anyone. We provide public access to this resource http://sln.suny.edu/teachingsurvey in the spirit of open sharing, community and collaboration, and to promote effective practices in online faculty development and course design. There are a number of institutions, groups, and communities external to SUNY currently using the tool as part of their online faculty development efforts: e.g., FOR-PD - University of Central Florida; National Park Community College, Arkansas; STARLINK professional development network; and University of North Florida.

Contact(s) for this Effective Practice

Effective Practice Contact:

alexandra m. pickett

Email this contact:

alexandra.pickett@suny.edu

Tags:

Faculty Satisfaction

Login or register to post comments

A Quality Scorecard for the Administration of Online Education Programs

Award Winner:

2010 Sloan-C Effective Practice Award

Author Information

Author(s):

Kaye Shelton, Ph.D.

Institution(s) or Organization(s) Where EP Occurred:

A research study that involved multiple institutions.

Effective Practice Abstract/Summary

Abstract/Summary of Effective Practice:

As the demands for public accountability increase for the higher education industry, institutions are seeking methods for continuous improvement in which they may demonstrate the level of quality within educational programs. Because of the rapid growth of online programs, institutions are further called upon to demonstrate that quality education is being delivered to students at a distance. This study created such a method to provide institutions offering online education an instrument for assessing quality within their programs: a quality scorecard for the administration of online education programs.

A six round Delphi study was undertaken with 43 seasoned administrators of online education programs from a variety of institutions in higher education. This panel of experts agreed upon 70 quality indicators that administrators of online education programs should examine within their programs to assess quality. A method for scoring was also developed. The original set of 24 quality indicators from the Institute for Higher Education Policy study, Quality on the Line: Benchmarks for Success in Internet-Based Distance Education (2000) were used as a starting point and were determined to be still valid in 2010, but with modifications (wording, some were divided, and some were combined). Forty-five additional quality indicators were added which resulted in a quality scorecard which provides industry agreed upon standards for online education programs to use for quality assessment. Each quality indicator has a potential range of 0-3 points with a perfect score on the scorecard resulting in 210 points.

Description of the Effective Practice

Description of the Effective Practice:

The purpose for this study was the development of a scorecard to measure and quantify elements of quality within online education programs in higher education. The scorecard is an easy-to-use tool for online administrators to use for program evaluation. By evaluating each of the respective quality indicators within the established categories, an online administrator can determine strengths and weaknesses of their program. The identification of the weaknesses can be used to support program improvement and strategic planning initiatives. The scorecard could also be used to demonstrate to accreditors elements of quality within the program as well as an overall quality level.

A scorecard of 70 quality indicators was developed by a panel of experts with 86% of the panel have 9 or more years of experience Each indicator is worth up to three points. The administrator will determine at what level their program meets the intent of the quality indicator:

0 points = Not Observed. The administrator does not observe any indications of the quality standard in place.
1 point = Insufficiently Observed. The administrator has found a slight existence of the quality standard in place. Much improvement is still needed in this area.
2 points = Moderate Use. The administrator has found there to be moderate use of the quality standard. Some improvement is still needed in this area.
3 points = Meets Criteria Completely. The administrator has found that the quality standard is being fully implemented and there is no need for improvement in this area.

A perfect score = 210 points.

90-99% = 189-209 - Exemplary (little improvement is needed)

80-89% = 168-188 - Acceptable (some improvement is recommended)

70-79% = 147-167 - Marginal (significant improvement is needed in multiple areas)

60-69% = 126-146 - Inadequate (many areas of improvement are needed throughout the program)

59% and below = 125 pts and below - Unacceptable.

Supporting Information for this Effective Practice

Evidence of Effectiveness:

While there are rubrics being used to assess quality in online course materials, such as Quality Matters, until now, there was not an industry agreed upon instrument that was being used to evaluate online education programs. Many institutions prolifically advertise they offer quality online education but have not had a way to quantify or benchmark their programs. How do students know they are enrolling in a quality program? The scorecard developed as a result of this research study provides an instrument that could identify strengths and weaknesses of an online education program and be used as a benchmarking tool for evaluation against other like programs in the industry. The results of the scorecard may also provide valuable information for strategic planning and budgeting.

The quality scorecard was developed by a panel of online education administrators with all having more than 7 years of experience. The following table shows the distribution of the types of institutions represented.

Institutional Classification for Expert Panel Members

Institutional Classification	Type	Size	Total
Public (4 year)	Non-profit	Large	24
Public Community College (2 year)	Non-profit	Large	2
Private (4 year)	Non-profit	Large	4
Private (4 year)	For-profit	Large	1
Private Faith-Based (4 year)	Non-profit	Large	1
Public (4 year)	Non-profit	Medium	2
Private (4 year)	Non-profit	Medium	3
Private Faith-based (4 year)	Non-profit	Medium	3
Public (4 year)	Non-profit	Small	1
Private (4 year)	Non-profit	Small	2

How does this practice relate to pillars?:

Each of the Sloan-C pillars of quality for online education are focused on establishing minimum guidelines for quality effectiveness. While this effective practice actually supports all of the pillars of quality, it may be most closely aligned with the Scale pillar - The provider continuously improves services while reducing costs because the scorecard enables the institution or program director to determine if they meet minimum quality standards in all areas of the program.

Equipment necessary to implement Effective Practice:

Nothing more than a pen or pencil is needed and the time necessary for all processes and procedures to be carefully examined and reviewed before filling out the quality scorecard.

Estimate the probable costs associated with this practice:

$0

References, supporting documents:

Institute for Higher Education Policy. (2000). Quality on the line: Benchmarks for success in Internet-based distance education. Washington, DC: Author.

Pond, W. K. (2002). Distributed education in the 21st Century: Implications for quality assurance. Online Journal of Distance Learning Administrators, V(II). Retrieved from http://www.westga.edu/~distance/ojdla/summer52/pond52.pdf

Useful Links:

View the scorecard

Other Comments:

The identification of quality online education programs satisfies a great need in our field and has been requested by many online education administrators as a tool for program improvement. The assessment of quality online education has never been more important as fierce competition from for-profit programs as well as many non-profits programs continues to increase and students all over the world are clicking to find a respectable degree program. Quality in education really does matter as the ultimate impact is to our students.

The interactive scorecard on the Sloan-C website is open to institutional members, see http://sloanconsortium.org/node/add/scorecard-online-program.

Contact(s) for this Effective Practice

Effective Practice Contact:

Kaye Shelton

Email this contact:

kaye@dbu.edu

Tags:

Login or register to post comments

Rubrics: Constantly Seeking Quality

Submitted by janetmoore on September 7, 2010 - 9:43am

What's your favorite rubric?

Tags:

Announcing 2010 Effective Practice Awards

Submitted by janetmoore on July 6, 2010 - 6:13pm

Sloan-C Honors Effective Practices in Online and Blended Education

FOR IMMEDIATE RELEASE

Tags:

HOME	INSTITUTE	MEMBERSHIP	CONFERENCES	RESEARCH & PUBLICATIONS	COMMUNITY	WHY SLOAN-C	ABOUT SLOAN-C
Welcome	Workshops	Membership	Emerging Technologies	JALN	Mailing List	The Quality Scorecard	About Us
My Account	Mastery Series	Institutional Members Listing	Blended Learning	National Survey Reports	Social Networks	Catalog of Online Programs	Contact Us
	Certificate	Corporate Sponsors	International on Online Learning	Sloan-C Series Books	Awards	Jobline	Personnel
	Webinars	Member Resources		Effective Practices	Fellows	Advisory Services	Privacy Policy
	College Pass	Insider Newsletter		Sloan-C View	Commons		Flyers
	Research Symposia	Subscribe to listserv		Workshop Publications	News & Press Room		FAQs
				Free downloads	Advertise		501c3 Status

INDIVIDUAL	INSTITUTIONAL	CORPORATE
Check here if your institution is a member Membership Comparison Chart

Faculty Satisfaction