Objective The purpose of this paper is to present a model for evaluating secondary school online learning courses. The model, based on a thorough review of the literature, focuses on four key components: organization and resources, engagement and participation, teacher interaction, and peer interaction. Description Student participation in K-12 online courses has risen markedly in the past five years (NACOL, 2008; Watson et al., 2009), however, relatively little research has been done on how to assess the quality of these online learning experiences (see Tallent-Runnels et al., 2006 for a review). Based on a review of reports evaluating K-12 and higher education (e.g., Bourne & Moore, 2004; Hosie et al., 2005; & Moore, 2005; Picciano & Seaman, 2007), four general constructs emerged and include: organization and resources, engagement and participation, teacher interaction, and peer interaction. The organization and resources construct included ease of navigation, accessibility of resources, organization of learning management system, quality of resources available, and clarity of assignments. The engagement and participation construct referred to engagement and interest in the online course, degree of participation, ability to keep up with material being covered, and contributions to online discussions. The teacher interaction construct looked at how quickly requests for help were met, regularity of formative feedback, clarity of summative evaluations, and contact time with the instructor. Finally, the peer interaction construct incorporated value of online discussions, getting help from other students, and the opportunity to get to know peers. Results Validity and reliability of Model Organization & resources (6 items), engagement & participation (4 items), teacher interaction (4 items), and peer interaction (3 items) were assessed using Likert scale items based on a five-point scale (Strongly Disagree to Strongly Agree). A principal components analysis was done to explore whether the items in these constructs grouped together. The resulting structure resulted in four factors that fit the proposed design model well, thereby establishing construct validity. Internal reliability estimates for each of the five constructs ranged from 0.72 to 0.87. Context of Learning The sample consisted of 142 students (male=36, female = 106) enrolled in grade 10,11, and 12 online courses in business, computer science, English, geography, health, history, law, philosophy and physics. Ninety-seven percent of the students reported being comfortable or very comfortable with technology, however, only 15% of the students had previously participated in online learning. From open ended responses, it was determined that that the main reason students chose to take an online course was "perceived necessity" - the course was not offered at their school, they had full time employment, they were committed to a rigorous sports program, or it provided more flexibility. Evaluating K-12 Online Learning Courses With respect to the sample population in this study, organization and resources were rated the highest - 60% the students felt the course website was well organized with good quality learning materials that were easy to find. Seven out of ten students reported that technology was not a problem for them while they participated in the online course. Teacher interaction was rated as the next highest construct by students with over 60% of the students stating that their teacher provided prompt "just-in-time" help and clear feedback on summative evaluations. On the other hand, less than 50% of the students felt they received regular formative guidance on their progress during the course. Engagement and participation in online courses was moderate to low in this study. While just over 60% of students stated they participated regularly , only 50% reported they were engaged and interested in the course being taught and just over 40% of students were involved in online discussions. Peer interaction was rated relatively low by students. Thirty percent of students reported being able to get help from their peers, 25% felt online discussion helped them learn, and only 10% of students were able to get to know other students in their online course. Predicting Success Students were asked to compare four aspects of online learning to their previous face-to-face experiences: participation, learning, assessment, and sense of community. High scores indicated they felt that online learning was better than face-to-face delivery methods. These scores were used as a measure of perceived success in online learning for this study. In other words, it was assumed a measure of success was achieved when students felt the online course delivery method of learning was equal to or better than face-to-face learning experiences. All four constructs in the model used in this study were significantly correlated with high ratings of the online delivery method compared to face-to-face approach. High quality organization and resources (r = 0.43, p <.001), higher engagement & participation (r = 0.50, p <.001), effective teacher interaction (r = 0.37, p <.001), and better peer interaction (r = 0.36, p <.001) correlated significantly with student perceptions of successful, online learning experiences. Conclusion Four constructs were used in this study to evaluate online learning for secondary school students: organization and resources, engagement and participation, teacher interaction, and peer interaction. The constructs demonstrated good construct reliability and internal reliability estimates. All four constructs were significantly correlated with perceived success in online learning as compared to a face-to-face delivery format. It is concluded that the proposed model in this study is a good starting point for evaluating online learning in a secondary school environment.

This presentation was selected Best-in-Track for the K-12 Online Learning track.