UCF's Online Faculty Readiness Assessment

Award Winner: 
2011 Sloan-C Effective Practice Award
Author Information
Author(s): 
Thomas Cavanagh
Institution(s) or Organization(s) Where EP Occurred: 
University of Central Florida
Effective Practice Abstract/Summary
Abstract/Summary of Effective Practice: 
The University of Central Florida has developed a web-based form and corresponding rubric that assess a faculty member’s prior online teaching experience. This instrument is competency-based and determines if the candidate’s experience, practices, and teaching philosophy meet university standards and conventions and exempt him/her from faculty development training that is required for all online instructors.
Description of the Effective Practice
Description of the Effective Practice: 
The University of Central Florida (UCF) is one of the fastest-growing universities in the country, currently ranked as the second-largest public institution in the US with over 58,600 students. This kind of growth has placed enormous pressures on the academic delivery infrastructure, and online learning has proven to be a key strategy to meet student demand. Over 30% of the university's student credit hours are generated by online and blended courses and more than half of all UCF students take one or more online courses every year.   As new faculty are hired to meet that demand, an increasing number of them join UCF with prior experience teaching online elsewhere. This creates a challenge for UCF’s Center for Distributed Learning (CDL), which is responsible for the quality of online courses. The centerpiece of faculty development for UCF’s online course design and instruction is IDL6543, an eight week, award-winning graduate-level workshop that models how to teach online using a combination of seminars, labs, consultations, and Web-based instruction. IDL6543 is taught in a blended/hybrid format that requires classroom participation. However, IDL6543 is designed for faculty with no online teaching experience and the course has no accommodation for faculty who may be hired on a part-time/adjunct basis to teach only online (and who are not geographically able to participate).   For faculty who truly possess existing competence in the design and delivery of online courses, IDL6543 is too comprehensive and perhaps even unnecessary. College deans and department chairs have questioned the need to enroll newly-hired faculty with prior online teaching experience in the required IDL6543 program. However, because there are numerous models for teaching online, including template-based courses for adjunct assignments; original development for full-time faculty; courses that consist only of posted PDF documents; courses taught in wikis, blogs, or Second Life; Quality Matters rubrics; Blackboard Greenhouse rubrics; Sloan-C pillars; WCET guidelines; USDLA guidelines; and many others, there is no clear definition of what “prior online teaching experience” means. UCF prides itself on developing the highest quality online courses (as evidenced by numerous awards) and must ensure that new faculty who are exempted from IDL6543 meet the university’s expectations for quality design and delivery of online courses.   As a result, UCF has developed an online form and corresponding rubric intended for selected faculty that determine if their experience, competence, and teaching philosophy meet CDL standards and conventions. It is expected that a final assessment will result in one of several possible results: IDL6543 equivalency, a requirement to complete specific remedial elements from IDL6543 or other programs (including technical instruction on the course management system), a requirement to complete a different professional development program, or a requirement to complete IDL6543 in its entirety. The submitted information and course design/delivery artifacts are evaluated by instructional designers according to an established rubric.
Supporting Information for this Effective Practice
Evidence of Effectiveness: 
We conducted an initial round of instructional design reviews of the instrument and that resulted in several iterations of the instrument. Once we had a version of the instrument we liked, we tested it with some example data from a CDL administrator. While this was in process, the instructional design team also developed the rubric that would be used to evaluate submissions. This rubric turned out to have the most dramatic changes between versions. We initially began with a comprehensive Excel file that quantified each element and rolled them all up into a numerical summary. However, when implemented, the Excel-based rubric turned out to be too complex and cumbersome. The ID team then prepared a simplified rubric with more qualitative categories, relying more on their own expertise than a numerical score. So far, this qualitative review has proven to be an effective assessment method.   Although only a dozen or so faculty have been through the rubric and approximately half of those have been exempted from IDL6543, thus far those exempted faculty have not required any additional support from either their instructional designers or our help desk. As we continue to use the instrument, we intend to track all exempted faculty to determine if their support requirements and performance are comparable to faculty who have completed IDL6543. Although the sample size is small, thus far, this has been the case.
How does this practice relate to pillars?: 
This Online Faculty Readiness Assessment (OFRA)...   ...increases access by allowing more faculty to be credentialed to teach online without taking up valuable seats in our IDL6543 program. Each faculty member is typically responsible for several online courses. This results in dozens to potentially hundreds of seats in online courses (depending upon the discipline and demand) being available two semesters sooner than would otherwise be possible under the previous process.   ...enables scale by allowing more faculty to be credentialed than there are available seats annually in IDL6543.   ...improves faculty satisfaction by permitting faculty to "test out" of required training. Compelling faculty to complete training on subjects for which they are already experts is demotivating and frustrating. This instrument allows CDL to not only exempt faculty from IDL6543 in its entirety but to also exempt them from portions while only requiring targeted subjects for remediation, thus maximizing faculty time.
Equipment necessary to implement Effective Practice: 
There is no extraordinary equipment costs associated with this practice. We have placed the online form on an existing server and house all of the submitted data within our existing proprietary business intelligence database called the Executive Information System (EIS).
Estimate the probable costs associated with this practice: 
The only costs were labor costs associated with programmer, instructional designer, and administrator time devoted to developing the instrument and evaluating submissions. In practice, however, these costs are fixed and the impact is more akin to opportunity costs related to other tasks not completed or delayed during normal working hours. However, this instrument was considered a high-value project since it enabled us to credential more faculty, reserve seats in IDL6543 for those who truly need them, and improve faculty satisfaction by allowing those who were qualified to "test out" of training requirements.
References, supporting documents: 
We referred to many different sources to develop the instrument and rubric, including the Blackboard Greenhouse (Exemplary Course) Awards, Quality Matters rubric, Sloan Consortium guidelines, and Chico State internal documents.
Useful Links: 
UCF's Online Faculty Readiness Assessment
Contact(s) for this Effective Practice
Effective Practice Contact: 
Thomas Cavanagh
Email this contact: 
cavanagh@ucf.edu
Tags:
© 2012, The Sloan Consortium