The purpose of this presentation is to highlight an effective model of the design and use of virtual school data systems to inform practice. The outcomes and implications reported are based on a two-year state virtual school evaluation process. Just-in-time and longitudinal uses of data for instructional application and school improvement are among the national goals articulated in the U.S. Department of Education Blueprint for Reform (2010). This plan calls for collection, analysis, and action based on achievement data, demographic data, and information about teaching and learning conditions in "data systems that determine "how schools and districts are progressing in preparing students to graduate from high school college- and career-ready" (p.12). By virtue of their robust and detailed data systems, virtual schools offer models to traditional schools for data-driven decision making.
The state-run virtual school described in this study offers online learning to children in grades K through 12 residing in the state. As a public school district operated under the authority of the state education agency, it has full access to disaggregated student record data at the school and state level. The state mandates specific demographic and performance data, the school collects additional achievement and other data, and the learning management system captures course behavior data. When the resulting data are systematically collected and linked, questions can be asked and answered that illuminate specific strengths and needs in the school. Deliberate attention in the early stages of a virtual school's development to the design of data systems and the staff expertise in data interpretation and communication are likely to benefit the school leaders and teachers through efficient and timely access to performance outcomes at course, department, and school levels (Smith, Clark & Blomeyer, 2005). Ongoing monitoring of these data in supplemental online programs via dashboards can support increasing student performance for all students, closing achievement gaps, and turning around low-performing schools where students take most of their courses. In addition, as innovations are implemented at course, program, and school levels, data systems can illuminate promising approaches early in their cycles of design so corrections may be made and effective practices may be scaled up.
One evaluation question concerned success factors in K-12 online learning. Data were extracted from the school's data systems during its first year for the 15 highest-enrollment high school courses. The data included over 14,000 course enrollments and associated information related to the course, teacher, student, learning management system logs, and grades. The influence of time students spent in the learning management system (LMS), number of times logged into the LMS, teacher comment, participation in free or reduced lunch programs, student status in the virtual school (full time or part time student), race/ethnicity, and grade level in the physical school student attends on student academic achievement were investigated using hierarchical linear modeling.
The design of the school's data systems enabled analysis of data across all course instances related to a wide range of factors with potential impact on student performance in the courses. Several factors were found to have significant influence on student achievement, and these factors varied depending on the level and content area of the course. This insight into the interplay of multiple factors has supported decision-making within the school. Time students spent online in the course was the most significant factor, influencing student academic outcome significantly in 11 of these 15 courses, regardless of student demographic group. The directions of the coefficients show students who spent more time in the course performed better academically, aligning with the belief that students participating in online academic activities at a higher level achieve better performance in online learning (Wang & Newlin, 2000) and the lower level of involvement in these activities at the beginning of the semester could be an early warning sign of failure later during the learning process. This finding provides support for online course design to make them more user-friendly with attractive interfaces that motivate students to spend more time in the system and engage them more effectively in academic learning activities. Participation in free or reduced lunch programs had a significant effect in 5 courses with students not participating in the lunch programs performing better than students participating in these programs. This finding highlights the importance of instructional strategies to support students from low-income families. Student status in the virtual school also had a significant effect in 5 online core courses, 3 of which were advanced math courses, with full time online students performing better than part time online students. This highlights the need for integration of components in course design particularly at advanced levels. Such components include time management skills or learning strategies for part time students to balance their life between online and traditional learning environments.
Given the dearth of research on success factors in K-12 online learning environment for high enrollment courses, this study informs researchers, educators, course designers, online program administrators, policy makers, and classroom-based educators. The investigation provides a deeper understanding of success in K-12 virtual learning environment by providing a foundation for the decision making process in virtual schools with respect to the improvement of course design and school policy.
Liu, F. & Cavanaugh, C. (in press). Online Core Course Success Factors in Virtual School: Factors influencing student academic achievement. International Journal of E-Learning.
Office of Planning, Evaluation and Policy Development. (2010). Blueprint for reform: The reauthorization of the elementary and secondary education act. Washington, DC: U.S. Department of Education.
Smith, R., Clark, T. & Blomeyer, R. (2005). A Synthesis of New Research on K-12 Online Learning. Naperville, IL: North Central Regional Educational Laboratory.
Wang, A. Y., & Newlin, M. H. (2000). Characteristics of students who enroll and succeed in psychology Web-based classes. Journal of Educational Psychology, 92, 137-143.
