A Quality Scorecard for the Administration of Online Education Programs

Community

Join your colleagues and make new friends in the Sloan-C Community. Take part in discussion groups or start your own. Blogs, Groups, Discussions, Polls. More >>>

Join our Social Networks

Stay in touch, subscribe to our mailing list

Distance Education Report

A Simple Way to Score Administrative Quality
By Jennifer Patterson Lorenzetti

A Quality Scorecard for the Administration of Online Education Programs

An interactive version of this score card is available to Institutional Members. If you are not an Institutional Member, click here to learn more about membership and to register.

Buy the book now:

Print copy ($49.95)

PDF format ($29.95)

Recorded Webinar (Free)

This scorecard is for measuring and quantifying elements of quality within online education programs in higher education. It is an easy-to-use tool for online administrators for program evaluation. By evaluating each of the respective quality indicators within the established categories, an online administrator can determine strengths and weaknesses of their program. The identification of the weaknesses can be used to support program improvement and strategic planning initiatives. The scorecard could also be used to demonstrate to accrediting bodies, elements of quality within the program as well as an overall level of quality.

The scorecard contains 70 quality indicators--each indicator is worth up to three points. The administrator will determine at what level their program meets the intent of the quality indicator after examining all procedures and processes.

0 points = Not Observed. The administrator does not observe any indications of the quality standard in place.
1 point = Insufficiently Observed. The administrator has found a slight existence of the quality standard in place. Much improvement is still needed in this area.
2 points = Moderate Use. The administrator has found there to be moderate use of the quality standard. Some improvement is still needed in this area.
3 points = Meets Criterion Completely. The administrator has found that the quality standard is being fully implemented and there is no need for improvement in this area.

The quality scorecard is versatile enough to be used to demonstrate the overall quality of online education programs, no matter what size or type of institution. The following steps for use and implementation are suggested that will yield a measurable result:

The online education administrator examines the online program for evidence of each of the 70 quality indicators. Based upon the level of evidence observed, the administrator chooses one of the following values: 0 points - not observed, 1 point - insufficient, 2 points - moderate use, 3 points - completely meets criterion.
For each indicator, the online education administrator should provide examples of the observed evidence. For example, the first indicator listed in the Institutional Support category is: The institution has put in place a governance structure to enable effective and comprehensive decision making related to distance learning. To substantiate the score for this indicator, evidence should be documents such as digital copies of organizational charts, reporting structures, and advisory committee minutes demonstrating how a decision is processed.
The online education administrator totals the score for each indicator and then determines the level of quality observed:
A perfect score = 210 points.
90-99% = 189-209 - Exemplary (little improvement is needed)
80-89% = 168-188 - Acceptable (some improvement is recommended)
70-79% = 147-167 - Marginal (significant improvement is needed in multiple areas)
60-69% = 126-146 - Inadequate (many areas of improvement are needed throughout the program)
59% and below = 125 points and below - Unacceptable.

This scorecard was developed through a Delphi study involving 43 experts in online education administration (83% had more than nine years of experience).

Because the quality scorecard was developed for the administration of online education programs, the word “distance” has been replaced with the work “online.

An interactive version of this score card is available to Institutional Members. Not an Institutional Member? Learn more about membership.

The quality scorecard contains adaptations of the 24 quality standards identified by the Institute for Higher Education Policy report, Quality on the Line: Benchmarks for Success in Internet-based Distance Education (2000).

The order of quality indicators for each category does not, in any way, imply rank of importance.

QUALITY SCORECARD FOR THE ADMINISTRATION OF ONLINE EDUCATION PROGRAMS
			Not Observed	Insufficient	Moderate Use	Meets Criterion Completely	Score	Points Possible Per Category
	INSTITUTIONAL SUPPORT							12

	1.	The institution has a governance structure to enable effective and comprehensive decision making related to online education.	0	1	2	3
	2.	Policies and guidelines are in place to authenticate that students enrolled in online courses and receiving college credit are indeed those completing the course work.	0	1	2	3
	3.	Policy for intellectual property of course materials exists. (If policy exists, 1 pt.; If policy specifically addresses online course materials, 2 pts.; If policy specifically addresses online course materials and is publically visible online, 3 pts. )	0	1	2	3
	4.	The institution has defined the strategic value of online learning to its enterprise and to its stakeholders.	0	1	2	3

	TECHNOLOGY SUPPORT							18

	1.	A documented technology plan that includes electronic security measures (e.g., password protection, encryption, secure online or proctored exams, etc.) is in place and operational to ensure quality standards, adherence to FERPA, and the integrity and validity of information. ***	0	1	2	3
	2.	The technology delivery systems are highly reliable and operable with measurable standards being utilized such as system downtime tracking or task benchmarking. ***	0	1	2	3
	3.	A centralized system provides support for building and maintaining the online education infrastructure. ***	0	1	2	3
	4.	The course delivery technology is considered a mission critical enterprise system and supported as such.	0	1	2	3
	5.	The institution maintains system backup for data availability.	0	1	2	3
	6.	Faculty, staff, and students are supported in the development and use of new technologies and skills.	0	1	2	3

	COURSE DEVELOPMENT AND INSTRUCTIONAL DESIGN							36

	1.	Guidelines regarding minimum standards are used for course development, design, and delivery of online instruction (such as course syllabus elements, course materials, assessment strategies, faculty feedback). ***	0	1	2	3
	2.	Technology is used as a tool to achieve learning outcomes in delivering course content. ***	0	1	2	3
	3.	Instructional materials, course syllabus, and learning outcomes are reviewed periodically to ensure they meet program standards. ***	0	1	2	3
	4.	Courses are designed so that students develop the necessary knowledge and skills to meet learning objectives at the course and program level. These may include engagement via analysis, synthesis and evaluation. ***	0	1	2	3
	5.	Learning objectives describe outcomes that are measurable.	0	1	2	3
	6.	Selected assessments measure the course learning objectives and are appropriate for an online learning environment.	0	1	2	3
	7.	Student-centered instruction is considered during the course-development process.	0	1	2	3
	8.	There is consistency in course development for student retention and quality.	0	1	2	3
	9.	Course design promotes both faculty and student engagement.	0	1	2	3
	10.	Current and emerging technologies are evaluated and recommended for online teaching and learning.	0	1	2	3
	11.	Instructional design is provided for creation of effective pedagogy for both synchronous and asynchronous class sessions.	0	1	2	3
	12.	Curriculum development is a core responsibility for faculty (i.e., faculty should be involved in either the development or the decision making for the online curriculum choices).	0	1	2	3

	COURSE STRUCTURE							24

	1.	The online course site or other web site includes a syllabus outlining course objectives, learning outcomes, evaluation methods, textbook information, and other related course information, making course requirements transparent at time of registration. ***	0	1	2	3
	2.	The institution ensures that all online education students, regardless of where they are located, have access to library/learning resources adequate to support the courses they are taking (SACS statement). ***	0	1	2	3
	3.	Expectations for student assignment completion, grade policy, and faculty response are clearly provided in the course syllabus. ***	0	1	2	3
	4.	Links or explanations of technical support are available in the course (i.e., each course provides suggested solutions to potential technical issues and/or links for technical assistance).	0	1	2	3
	5.	Instructional materials are easily accessible and easy to use for the student.	0	1	2	3
	6.	The course adequately addresses the needs of students with disabilities via alternative instructional strategies and/or referral to special institutional resources.	0	1	2	3
	7.	Opportunities/tools are provided to encourage student-student collaboration (i.e, web conferencing, instant messaging, etc) if appropriate.	0	1	2	3
	8.	Documents attached to modules are in a format that is easily accessed with multiple operating systems and productivity software (PDF, for example).	0	1	2	3

	TEACHING AND LEARNING							15

	1.	Student-to-Student interaction and Faculty-to-Student interaction are essential characteristics and are facilitated through a variety of ways. ***	0	1	2	3
	2.	Feedback on student assignments and questions is constructive and provided in a timely manner. ***	0	1	2	3
	3.	Students learn appropriate methods for effective research, including assessment of the validity of resources and the ability to master resources in an online environment. ***	0	1	2	3
	4.	Students are provided access to library professionals and resources that help them to deal with the overwhelming amount of online resources.	0	1	2	3
	5.	Instructors use specific strategies to create a presence in the course.	0	1	2	3

	SOCIAL AND STUDENT ENGAGEMENT							3

	1.	Students should be provided a way to interact with other students in an online community.	0	1	2	3

	FACULTY SUPPORT							18

	1.	Technical assistance specifically for online course development and online teaching is provided for faculty. ***	0	1	2	3
	2.	Instructors are prepared to teach online education courses and the institution ensures faculty receive training, assistance, and support at all times during the development and delivery of courses. ***	0	1	2	3
	3.	Faculty receive training and materials related to Fair Use, plagiarism, and other relevant legal and ethical concepts. ***	0	1	2	3
	4.	Faculty are provided on-going professional development related to online teaching and learning.	0	1	2	3
	5.	Clear standards are established for faculty engagement and expectations around online teaching.	0	1	2	3
	6.	Faculty workshops are provided to make them aware of emerging technologies and the selection and use of these tools.	0	1	2	3

	STUDENT SUPPORT							51

1.	Before starting an online program, students are advised about the program to determine if they possess the self-motivation and commitment to learn online. ***	0	1	2	3
2.	Before starting an online program, students are advised about the program to determine if they have access to the minimal technology required by the course design. ***	0	1	2	3
3.	Students receive (or have access to) information about programs, including admission requirements, tuition and fees, books and supplies, technical and proctoring requirements, and student support services prior to admission and course registration. ***	0	1	2	3
4.	Students are provided with access to training and information they will need to secure required materials through electronic databases, interlibrary loans, government archives, new services and other sources. ***	0	1	2	3
5.	Throughout the duration of the course/program, students have access to appropriate technical assistance and technical support staff. ***	0	1	2	3
6.	Student support personnel are available to address student questions, problems, bug reporting, and complaints. ***	0	1	2	3
7.	Students have access to effective academic, personal, and career counseling.	0	1	2	3
8.	Minimum technology requirements for skills and equipment are established and made available to students.	0	1	2	3
9.	Student support services are provided for outside the classroom such as academic advising, financial assistance, peer support, etc.	0	1	2	3
10.	Policy and process is in place to support ADA requirements.	0	1	2	3
11.	Students are provided easy access to required course materials in print and/or digital format, such as ISBN numbers for textbooks, book suppliers, and delivery modes.	0	1	2	3
12.	Program demonstrates a student-centered focus rather than trying to fit service to the online education student in on-campus student services.	0	1	2	3
13.	Efforts are made to engage students with the program and institution.	0	1	2	3
14.	Students are instructed in the appropriate ways of communicating with faculty and students.	0	1	2	3
15.	The institution provides guidance to both students and faculty in the use of all forms of technologies used for course delivery.	0	1	2	3
16.	Tutoring is available as a learning resource.	0	1	2	3
17.	Students are instructed in the appropriate ways of enlisting help from the program.	0	1	2	3

EVALUATION AND ASSESSMENT							33

1.	The program is assessed through an evaluation process that applies specific established standards. ***	0	1	2	3
2.	A variety of data (academic and administrative information) are used to regularly and frequently evaluate program effectiveness and to guide changes toward continual improvement. ***	0	1	2	3
3.	Intended learning outcomes at the course and program level are reviewed regularly to ensure clarity, utility, and appropriateness. ***	0	1	2	3
4.	A process is in place for the assessment of support services for faculty and students.	0	1	2	3
5.	Course and program retention is assessed.	0	1	2	3
6.	Recruitment and retention are examined and reviewed.	0	1	2	3
7.	Program demonstrates compliance and review of accessibility standards (Section 508, etc.)	0	1	2	3
8.	Course evaluations are examined in relation to faculty performance evaluations.	0	1	2	3
9.	Faculty performance is regularly assessed.	0	1	2	3
10.	Alignment of learning outcomes from course to course exists.	0	1	2	3
11.	Course evaluations collect student feedback on quality of content and effectiveness of instruction.	0	1	2	3
Perfect Score = 210
*** Adapted from the Institute for Higher Education Policy's report Quality on the Line: Benchmarks for Success in Internet-based Distance Education (2000).

HOME	INSTITUTE	MEMBERSHIP	CONFERENCES	RESEARCH & PUBLICATIONS	COMMUNITY	WHY SLOAN-C	ABOUT SLOAN-C
Welcome	Workshops	Membership	Emerging Technologies	JALN	Mailing List	The Quality Scorecard	About Us
My Account	Mastery Series	Institutional Members Listing	Blended Learning	National Survey Reports	Social Networks	Catalog of Online Programs	Contact Us
	Certificate	Corporate Sponsors	International on Online Learning	Sloan-C Series Books	Awards	Jobline	Personnel
	Webinars	Member Resources		Effective Practices	Fellows	Advisory Services	Privacy Policy
	College Pass	Insider Newsletter		Sloan-C View	Commons		Flyers
	Research Symposia	Subscribe to listserv		Workshop Publications	News & Press Room		FAQs
				Free downloads	Advertise		501c3 Status

INDIVIDUAL	INSTITUTIONAL	CORPORATE
Check here if your institution is a member Membership Comparison Chart