Measuring Digital Learning in the 21st Century - Data Warehouses, Dialects, and Dashboards

Presenter(s)
Jennifer Stephens (American Public University System, US)
Phil Ice (American Public University System, US)
Frank McCluskey (American Public University System, US)
Dave Becher (American Public University System, US)
Session Information
November 4, 2010 - 3:10pm
Track: 
Learning Effectiveness
Areas of Special Interest: 
Online Learning and Community Colleges
Major Emphasis of Presentation: 
Practical Application
Institutional Level: 
Multiple Levels
Session Type: 
Group Presentation
Location: 
Curacao 1
Session Duration: 
80
Concurrent Session: 
6
Abstract
The American Public University System will discuss strategies to promote a culture of assessment, continuous improvement, and data-based decision-making. We will discuss tools and strategies used by faculty and administrators to integrate assessment into daily decision-making processes to enhance the student learning experience.
Extended Abstract
Historically there has been friction between individual faculty and the university about how learning is measured. Answering the question "What is good teaching?" is as much art as it is science. A teacher who is considered "great" to a small number of brilliant students may be considered a failure to the rest of the class. Is that bad teaching? There may be a teacher who is extremely popular with the students, whose teaching is ineffective or is not even current with the subject matter. Is that good teaching? Recently, there have been more attempts to validate entire programs and degrees at the university level. This has led to the use of standardized tests and learning measures that are independent of the faculty teaching experience. These assessment methods are often not popular, as faculty members often feel that they restrict their teaching and are not pedagogically sound. Faculty members often resist the use of resulting data in the classroom, seeing it as intrusive and not representative of student learning. This resistance has stalled the integration of data into higher education planning and program review, and has led to a negative perception of data as a tool for tracking consumer behavior, not student learning. In recent years, added pressure has been put on colleges and universities to justify their high cost and constant raises in tuition. For this, and other reasons, accreditation standards and validation of student learning, have become more prominent and more political in university administration. Colleges often see accreditation and data collection as something negative - something that is foisted on them from the outside and not part of their core functionality. To attempt to overcome some of these obstacles, we have utilized strategies to promote a culture of assessment, continuous improvement, and data-based decision-making at the American Public University System (APUS). APUS is an online for-profit university with more than 70,000 students and 1300 faculty. We recognize that faculty involvement and voice are essential to a quality educational experience for our students. As a fully online institution, we have the ability to collect and review an extensive amount of student data, which we recognize as integral to the continuous improvement of a quality online program. Our challenge is to create a balance between the need for accountability in teaching and learning, and faculty comfort with data collection. To build a culture of assessment, we embrace data collection, review and reporting at the classroom level. At APUS, we have done this by embedding our data collection, review and reporting at into the core of our program review process. Participants in this session will learn how an online institution engages faculty and administrators in an evolutionary process that has no specific end other than continuous improvement. You'll learn how we developed a committee to lead quality assurance processes, promoted accountability among faculty and administrators through the documentation of student learning outcomes evidence, informed internal and external stakeholders on the achievement of student learning outcomes, established a reciprocal relationship among the institution and appropriate stakeholders, and incorporated data into daily decision-making processes at the institution. This session will also briefly demonstrate the tools we use (dashboards, data warehouse, and fact books) to tap into a variety of data sources to provide useful information to decision-makers in a timely manner. As a result of these strategies, faculty and administrators are increasingly integrating assessment information into daily decision-making processes to improve on academic courses and programs.
Lead Presenter
Jennifer Stephens is the Dean of Assessment and Associate Vice-President at the American Public University System She holds a Ph.D. in Educational Psychology with an emphasis on learning, development, and instruction from Texas A&M University in College Station, Texas. She most recently worked with the Accelerated Schools Project at Stanford University, developing an outcome reporting system for the school reform organization. She has consulted with schools, school districts, state departments of education, and school reform organizations. She has presented at national and international conferences in the areas of school reform, distance learning, faculty development, and assessment.