Characterizing online, asynchronous discussions is an essential aspect for developing best practices for modern higher education. A 2007 survey of United States higher education indicated that 75% of the institutions offering hybrid or online courses made extensive use of discussions as an educational medium (Parsad & Lewis, 2008). An additional 17% reported moderate usage indicating that almost all, 92%, of online courses incorporate discussions. With this widespread usage, identification of best practices in this medium is essential.
Code-based content analysis is a common method for describing learning and comparing educational practices in this medium; however, this is not a mature methodology. A review of publications between 2002 and early 2009 identified over 50 different coding schemes applied for this purpose, and recent methods reviews have also identified concerns regarding units of analysis, coding scheme reliability, and specificity of coding definitions (DeWever, Schellens, Valcke, & VanKeer, 2006; Gerbic & Stacey, 2005; Spatriu, Hartley, & Benedixen, 2004; Woo & Reeves, 2007; Weltzer-Ward, 2009). These inconsistencies make it difficult to compare results between studies and expand upon previous research. A framework and coding scheme to facilitate consistent and thorough characterization of online, asynchronous discussions is an essential tool for research to identify best practices in this medium.
This poster will present reliability and variable correlation data for an overall coding scheme as part of a continuing effort to develop a composite coding scheme for synthesizing previous research regarding online, asynchronous discussion. The first step in this development, identifying and characterizing the many different coding schemes applied in the last seven years, was presented at the 15th Annual Sloan Consortium International Conference on Online Learning in late 2009 (Weltzer-Ward, 2009). The second step, eliminating overlaps between these frameworks to produce an overall coding scheme situated in the commonly accepted Community of Inquiry framework, was presented at the American Education Research Education Annual Meeting in early 2010 (Weltzer-Ward, 2010). This overall coding scheme will be applied to discussions reflecting multiple question types and a cross-section of educational disciplines during the summer of 2010 to produce an extensive data set for analysis of this overall coding scheme. This analysis will include assessment of reliability for each of the over 200 variables in the overall coding scheme, review of correlations and potential overlap between the variables, and validation through comparison to results from previous research. Future work will then focus on reducing the overall coding scheme into an efficient but still thorough composite coding scheme. This poster will present the reliability and variable correlation data for the overall coding scheme.
Although not the final product of this initiative, the reliability and variable correlation data will provide a useful resource to researchers assessing online, asynchronous discussion using code-based content analysis. As the overall coding scheme incorporates the schemes applied in many studies, this information will facilitate comparison of results between these studies. The correlation data will allow researchers to identify areas of similarity between frameworks, and the reliability data will assist in identifying the most effective coding definitions. In addition, this data will aid researchers who wish to apply the overall coding scheme. The details of the overall coding scheme as well as a special-purpose computer program designed to facilitate coding will be made available to interested researchers.
References:
DeWever, B, Schellens. T., Valcke, M., & VanKeer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers and Education, 46, 6-28.
Gerbic, P., & Stacey, E. (2005). A purposive approach to content analysis: Designing analytical frameworks. Internet and Higher Education, 8(1), 45-59.
Parsad, B., and Lewis, L. (2008). Distance Education at Degree-Granting Postsecondary Institutions: 2006–07 (NCES 2009–044). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.: Washington, DC.
Spatariu, A., Hartley, K., & Benedixen, L. D. (2004). Defining and measuring quality in online discussions. Journal of Interactive Online Learning, 2(4), 1-15.
Weltzer-Ward, L. M. (2009, October). Analysis of Coding Schemes for a Composite Coding Framework for Online Asynchronous Discussion Analysis. Poster presented at the 15th Annual Sloan Consortium International Conference on Online Learning, Orlando, FL.
Weltzer-Ward, L. M. (2010, May). Development and Preliminary Application of a Composite Coding Scheme for Online Asynchronous Discussion Analysis. Paper presented at the American Education Research Association, Denver, CO.
Woo, Y., & Reeves, T. C. (2007). Meaningful interaction in web-based learning: A social constructivist interpretation. Internet and Higher Education, 10(1), 15-25.
