NASULGC-Sloan National Commission on Online Learning Benchmarking Study: Preliminary Findings

Abstract: 

The goal of the benchmarking activity is to begin the process of identifying some of the key factors that lead to “successful” online programs at public colleges and universities. To date, much of the research regarding online learning has focused on the questions of “what are campuses doing” and “why are they doing it.” Not as much attention has been paid to the question of “how do campuses with successful online programs organize themselves.”
To begin to answer that question, the Commission has identified forty-five joint NASULGC/AASCU members, representing more than one million total enrollments and over 100,000 online enrollments. Through a combination of a well-defined short survey and in-depth interviews, the Commission is building a profile of the attitudes and successful practices of the participating institutions in order to identify “key factors” that could be shared with/replicated by other campuses.

Full Article (PDF): 

 

NASULGC-Sloan National Commission on Online Learning

The goal of the benchmarking activity is to begin the process of identifying some of the key factors that lead to “successful” online programs at public colleges and universities. To date, much of the research regarding online learning has focused on the questions of “what are campuses doing” and “why are they doing it.” Not as much attention has been paid to the question of “how do campuses with successful online programs organize themselves.”

To begin to answer that question, the Commission has identified forty-five joint NASULGC/AASCU members, including institutions identified by the Sloan-C survey as both “fully engaged” and “engaged” (high potential for growth). These institutions represent more than one million total enrollments and over 100,000 online enrollments. Through a combination of a well-defined short survey and in-depth interviews, the Commission is building a profile of the attitudes and successful practices of the participating institutions in order to identify “key factors” that could be shared with/replicated by other campuses.

The study will provide an in-depth examination of the attitudes and beliefs of senior academic leaders at these institutions, probing them for their goals and objectives. The in-depth interviews also document the range of institutional approaches used by those who have been the most successful in introducing online course and programs to their institutions. Special attention is being paid to the barriers that had to be overcome and the strategies that were successfully implemented. These responses can be compared to those from institutions that are not as far along the path of online learning implementation, to see what lessons can be applied to their situations.

A second aspect of the study is the first detailed cross-institutional examination of faculty attitudes and beliefs towards online learning. Faculty represent the second major constituency (after senior academic leaders) that is critical to building a high-quality online learning program. Both national studies and single-institutional examinations have demonstrated that faculty attitudes are critical to the growth of any online learning program. There is, however, a large vacuum of reliable information on the exact nature of faculty attitudes and their influences.

Why select “engaged” and “fully engaged” institutions?

The dual objectives of this study are to better understand the approaches taken by those who have the most successful online programs and to determine what barriers other institutions face in joining them. By mapping the actions and attitudes of the “fully engaged” and “engaged” institutions, we hope to prepare a resource of possible approaches that can be adopted by the other institutions, which represent the largest potential for growth.

The 2007 Sloan survey of online learning, Online Nation: Five Years of Growth of Online Learning, presents a detailed analysis of the characteristics of institutions by their stage in an online learning framework. Among the three types of institutions with online offerings there is a clear pattern of differences in attitudes and activities as you move from those with the least amount of involvement (non-strategic online) to those with the most involvement (fully engaged). Several differences stand out:

  • The proportion of an institution’s student who are taking at least one online course doubles when you move from “non-strategic online” (7.3%) to “engaged” (14.6%), and then doubles again (33.5%) when you move to “fully engaged” institutions.
  • The proportion of these institutions with full online programs (as opposed to just scattered online courses) shows this same steady increase, growing from 32.5% for “non strategic online”, to 45.6% for “engaged,” and to 68.8% for “fully engaged”.
  • The proportion of institutions expecting growth in their online enrollments, and the amount of that growth, also follows this same pattern.

Nationally, there are twice as many “fully engaged” institutions as there are “engaged” institution, yet they enroll over five times as many online students.

Why concentrate on faculty?

More than six years of data from the national Sloan survey of online learning have shown that faculty acceptance of online education has consistently been seen as a critical barrier to its wide-spread adoption. Institutions at all stages of online adoption have listed faculty attitudes as a critical component of any online learning strategy for their institution. Six years of data show only a small improvement in the proportion of institutions who say that their faculty fully accept the value and legitimacy of online education. A majority of institutions remain either neutral or negative on this issue.
We know, however, that there are huge differences in this belief between those who have no plans for online (where only 3.7% say their faculty accept it) and those institutions that are already fully engaged with online (where the percentage jumps to 62.1). No other variable shows this same pattern.

What is it about these fully engaged institutions that they believe that their faculty have such a different opinion of online from those institutions that are not engaged at all? Does it take a change in attitudes on the part of faculty in order for an institution to move forward with online courses and programs, or if an institution introduces online offerings in spite of a lack of faculty acceptance, do faculty change their attitudes when they experience online education?

Are the perceptions of faculty attitudes on the part of academic leader even correct? Many chief academic officers continue to report that their faculty do not embrace online learning, yet these same academic leaders are planning more and more online offerings. This study will allow us to see how accurate the academic leaders are in assessing the attitudes of their faculty members. It will also allow examination of a number of other critical faculty issues as well, many for the first time in a cross-institutional analysis:

  • What reasons do faculty who do not accept online give for their opinion.
  • Does experience with online (either directly through teaching or indirectly through seeing others) have a positive impact on faculty views?
  • What are faculty perceptions about the efficacy of any actions that institutions have taken to address their concerns? Do they believe the actions have been effective or not?
  • How similar and/or varied are faculty attitudes at a single institution? Are there any patterns by type of faculty (full-time vs. part-time), department, experience teaching online, or level (associate, assistant, or full)?

Process

The first step in the interviewing process was to interview the designated point of contact at each institution to gain a better understanding of the interests/motivations of the institutions – what they are trying to accomplish through their online initiative, what they hoped to contribute and/or learn from participating in the study; what areas/issues were of primary interest to their institution. Information gathered through these initial interviews lead to identification of six broad, thematic areas:

  • Faculty Incentives
  • Student Life Cycle/Support
  • Senior Administration
  • Academic Quality and Effectiveness
  • Administrative and Financial Models
  • Technology

Based on these preliminary interviews, we then developed a questionnaire that we sent back to each institution with the intent to have institutions self-identify those areas in which they felt they had something to contribute – or learn.
It was determined that several themes actually had more than one element; we eventually ended up with ten questions or “areas of focus.” The project team then created cohorts of approximately 6 institutions to each of the ten questions, based on initial interviews and responses to the campus questionnaire.

A team of NASULGC staff has been conducting interviews ranging from 20 minutes to an hour with more than 200 completed. We anticipate conducting a total of between 230 and 250 interviews. All subjects are guaranteed confidentiality to encourage candid responses; all interviews are recorded, transcribed and will be subject to text analysis to ensure we are capturing as much data as possible. Many conversations “bleed” across subjects, e.g., discussions of reporting relationships include comments about leadership issues; conversations about financial incentives often veer into observations about overall faculty training and support.

First Impressions

One of the most common themes we have heard across several different “areas of focus” is the essential role of senior administration in creating the proper environment for online programs to be strategic and successful. Time and again interviewees have stated the importance of the institutional leadership articulating the role of online learning as part of the institution’s way of doing business – a part of the fabric of the institution – not as a directive, but as an articulation of a vision or strategy. In addition, those senior administrators – Presidents and Chancellors and/or Provosts – must provide adequate resources to support the vision, and make it as easy as possible for faculty to engage in the design and delivery of online courses and programs.

Another factor that has emerged is the value of bringing a range of people together – administrators and faculty (especially early adopters) –to map out a strategic approach to online. These comments were made both affirmatively – “this was central to the success of our efforts” and wistfully – “we probably would have been further along if we had done that.”

Another element that has been referenced numerous times - in a variety of question areas – is the importance of having a single office or individual responsible for coordinating and “overseeing” the institution’s online activities. Again, not in issuing edicts or otherwise dictating to faculty what they should put online and how, but as a consistent place where faculty and administrators can go for guidance and assistance on matters of policy and procedure. The design support or student support may be provided in other places on campus, but at least everyone knows that there is one place to go to get answers to questions about online answered at the “conceptual level.”

Faculty Survey

As part of the overall benchmarking project, all faculty at participating institutions are being asked to participate in a survey about experiences and attitudes towards online learning. Over 11,000 faculty have opened the survey to date, with almost 10,000 responding in sufficient detail to be included in the analysis. The survey was directed to all faculty at the participating institution, not just those involved in online education. Topics includes demographic information about the respondent, teaching experience (both face-to-face and online), and their option on the potential role, if any, for online learning.
Faculty were asked about the potential benefits of online education, and students concerns were at the top of their list.

image

It has been a common perception that online teaching takes more effort on the part of faculty than face-to-face instruction. Faculty which experience teaching online or developing on online courses were asked to rate the level of effort involved compared to an equivalent face-to-face course. There was wide agreement among the responding faculty that online instruction does require more effort. This was especially true for among those who had created an online course (where over 80 percent said it took more effort to development an online course than a face-to-face one) – but the result was true for teaching online, where over 60 percent thought it takes more effort than to teach a face-to-face course.

Level of effort for teaching:

What was your experience with the level of effort required to teach an online course compared to a face-to-face course? Total Sample
Is a lot less effort 2.5%
Is somewhat less effort 10.2%
Is about the same level of effort 23.2%
Is somewhat more effort 33.1%
Is a lot more effort 31.1%

Level of effort for course development (from scratch):

What was your experience with the level of effort required to develop an online course from scratch compared to a face-to-face course? Total Sample
Is a lot less effort .7%
Is somewhat less effort 1.0%
Is about the same level of effort 13.1%
Is somewhat more effort 30.8%
Is a lot more effort 54.4%

Free-text responses

In addition to the directed questions about online learning, all faculty members were asked a number of open ended questions in which they could provide as much information as they desired. One such question asked of those with any experience teaching online was “What do you like most about online instruction?”
A total of 2,536 free-text responses were received from faculty members with online teaching experience. The most cited like was “flexibility” (779 mentions), typically stated in terms of the flexibility it provides students. The second most cited reason was “discussions”, (183 mentions), followed by “access” (147 mentions). The 2,500+ responses were overwhelmingly student centered responses (the word “student” was used over 1,500 times) – what faculty like most about online instruction is how it is better for students.

A second open-ended questions addressed the issue of “What do you like least about online instruction?” A total of 2,538 free-text responses were received to this question from faculty members with online teaching experience.The most cited reason was “time” (632 mentions) – typically stated as the greater amount of time required for online instruction. The second most cited dislike was the lack of “interaction” (310 mentions) – often stated as being less than for a face-to-face course, and included statements about lack of facial expressions and body language. Problems with “support” were next (142 mentions), followed by the increased “effort” (106 mentions).