This proposal reports on aResearch Study that longitudinally examined the impact various Web modalities (fully online, blended, and lecture capture) have had on the end-of-course Student Perception of Instruction (SPoI). Few traditions in higher education evoke more controversy, ambivalence, criticism and, at the same time, support than SPoI. Results from these end-of-course survey instruments serve two main functions: they provide instructors with formative input for improving their teaching, and they serve as the basis for summative profiles of professors' effectiveness through the eyes of their students. In the academy, instructor evaluations also can play out in the high-stakes environments of tenure, promotion, and merit salary increases, making this information particularly important to the professional lives of faculty members. At the research level, the volume of the literature for student ratings impresses even the most casual observer with well over 2,000 studies referenced in the Educational Resources Information Center (ERIC) alone (Centra, 2003) and an untold number of additional studies published in educational, psychological, psychometric, and discipline-related journals.
During recent decades, the emerging Internet has impacted students' evaluations of their instructors. This phenomenon is interacting with a generation of millennial students. For them, the Web, with its sharing, communicating, blogging, text messaging,Social Networking, group writing though wikis, and interactive social opportunitiesâis a seamless and continuous communication medium. These developments present a learning model far different from one directional, teacher-to-student techniques that served as the prototype for most SPoI research of the past decades. Today's students experience education though online and blended courses as well as mobile devices, podcasts, chat rooms, and other instructional media.
These educational technologies have implications for students and their instructors. One example of emerging issues is the website http://www.ratemyprofessors.com where students formed a worldwide community to share their perceptions about their instructors' teaching abilities. Further, they share their impressions onSocial Networking tools, such as Facebook (http://www.facebook.com) or Twitter (http://www.twitter.com) , or post videos of their instructors in the act of teaching on YouTube (http://www.youtube.com). On most campuses students rate their professors online rather than using the paper-and-pencil scantron sheets of old. Students respond, not only to their face-to-face courses, but evaluate any number of technology-mediated classes in which they might be involved.
These emerging trends make it even more important to explore elements that underpin effective teaching in the eyes of students. In order to do this, the author and researchers at UCF explored the use of data-mining techniques to develop rule-based models that best predict what students consider excellent and poor teaching in the academy.
A Faculty Concern Regarding Web Courses
In 2010 the Faculty Senate at the University of Central Florida passed a resolution criticizing the SPoI instrument that had been in place for the past three decades. That resolution came from a majority of senators whose faculty constituencies expressed concern to them that the traditional items on the instruments were invalid or inappropriate for Web-based courses because of the modality mismatch. Faculty members argued that their student ratings were being adversely affected, thereby penalizing them for promotion, tenure and teaching awards. As a result, research was conducted on the existing form, as well as new "modality specific" forms in an effort to determine whether these new items could better capture the nuances of Web teaching.
The Current Study
The current study examines over 1.5 million student perception of instruction (SPoI) forms over the course of the past 8 years, specifically looking at the impact of varied course modalities, increases in class size, and changes in both learning management system and SPoI forms on students' ratings of their instructors. Data was examined using Classification and Regression Tree analysis incorporating course college, department, modality, and class size along with the SPoI elements and findings were consistent with the prior work of Moskal and Dziuban and others (Wang, Dziuban, Cook, & Moskal, 2009; Dziuban, & Moskal, 2011; Dziuban, Moskal, Kramer & Thompson, 2013). Elements that students perceive are indicative of "excellent" overall teaching were identified, and found to be independent of class mode or size. Results indicate that students have a clear view of what they perceive is âgood teaching' and translate this to whatever format the class takes. The Q&A discussion will center on these results and what faculty can do to help improve their SPoI ratings in their Web and F2F courses.
Centra, J. A. (2003). Will teachers receive higher student evaluations by giving higher grades and less course work? Research in Higher Education, 44, 496-518.
Dziuban, C., Moskal, P., Kramer, L., & Thompson, J. (2013). Student satisfaction with online learning in the presence of ambivalence: Looking for the will-o'-the-wisp. Internet and Higher Education, 17, 1-8.
Dziuban, C. & Moskal P. (2011). A course is a course is a course: Factor invariance in student evaluation of online, blended, and face-to-face learning environments. Internet and Higher Education, 14, 236-241.
Wang, M.C., Dziuban, C.D., Cook, I.J., Moskal, P.D. (2009). Dr. Fox Rocks: Using data-mining techniques to examine student ratings of instruction. M.C. Shelley, L.D. Yore, B. Hand, Eds. Quality Research in Literacy and Science Education: International Perspectives and Gold Standards, Dordrecht, The Netherlands: Springer, 383-398.