An Online Supplemental Instruction Tool Array

Award Winner:

2010 Sloan-C Effective Practice Award

Author Information

Author(s):

Lawrence Perez

Author(s):

Patrick Quigley

Author(s):

Candice Harrington

Institution(s) or Organization(s) Where EP Occurred:

Saddleback College

Effective Practice Abstract/Summary

Abstract/Summary of Effective Practice:

This effective practice is a methodology with results, where student feedback is used to continuously evolve a unique online learning environment. In this particular case developmental math students were asked to define issues and concerns for the purpose of understanding their specific needs. Using an action research approach, a strategy organically developed over several years where student input served as the mechanism of design. The result is a free unrestricted online supplemental instruction tool array called Algebra2go^TM developed by students enrolled in a traditional classroom setting.

With 17% of the nation’s college students being 35 years or older (TCOHE, 2009), traditional approaches to basic skills mathematics instruction may need to expand by creating learning experiences more relevant to the adult learner (Knowlton & Simms, 2008). As research has shown that an integral part of the educational process involves teachers as innovator, researcher, and change agent (Casazza & Silverman, 2000), one approach could be that teachers solicit student feedback as part of a self-evaluation process. Since external obligations such as those to employers and family in some cases hinders persistence (Tinto, 1993), extra effort should be made to assure that returning adult learners are part of the evaluation process. This information is essential for instructors attempting to alter their pedagogical models in order to meet the needs of these students. Given the recent origin of the online learning environment, this type of research in action can lead to entirely new ideas in instructional design. It should be noted that initially there was no intention of using technology to support student learning in this case. Only through defined student need did the tool array find its way to the online environment.

Description of the Effective Practice

Description of the Effective Practice:

In 1999 a researcher began his tenure at Saddleback College located in Southern California. While teaching developmental mathematics, he quickly became aware of the diversity and academic background among his students. At the end of that year the researcher felt the need for some self assessment as this could provide an effective source of feedback to improve practice (Travis, 1995). He decided to capture his lectures by acquiring a set of lecture notes from one of his top performing beginning algebra students. This document provided him a view of himself through the eyes of his student making him more aware of his own personal teaching style. Noting his student’s mistakes and personal annotations, led to the realization that students could teach him how to teach.

The following year the researcher used these student captured notes to guide him through his lectures refining a teaching style so that it complemented the learning styles of successful developmental math students. Informing students that he was engaged in this process of self assessment demonstrated that he was open to mutual interaction between teacher and student rather than a one way projection of information. This paved the way for sincere focus group interviews that were video recorded in May 2001.

A focus group of ten beginning algebra students agreed to allow the researcher to video tape them informally discussing issues focusing on their past experiences taking pre and beginning algebra courses. What was surprising was the degree to which the affective domain colored their perception of mathematics. Students referred to their past experiences using terms such as fear, anxiety, and even terror. Fortunately, these interviews also documented many of the specific causes of these emotional associations. Most students could trace the source of their fear and anxiety to specific events. After evaluating the captured video content, it was obvious to the researcher that teaching classroom skills would not increase the likelihood of success unless student affect was addressed in concert.

Soliciting student input was not limited to students within the classroom environment. As a matter of fact, the researcher’s most informative conversations with his students occurred prior to the beginning of the semesters. Meetings of this nature allowed him the opportunity to present himself more as a mentor rather than that mean math instructor that they see on the first day of class. This then sets up a semester of sincere exchanges of information that foster new ideas for presenting lecture material and secondly allows a clearer assessment of student need. Defined student issues such as fear, ability, family obligations, and time constraints placed on working students, are among many factors that led the researcher to develop an online delivery system that could potentially meet his students’ needs.

In this pedagogical mode, the approach was to capture the advantages of the online learning environment while being mindful to the student perception of the cold and mechanical nature of seeing mathematics on a computer screen. Therefore, providing learning resources in this environment required both contextualization and humanization in an effort to re-create the social emotional experience similar to what exists in the traditional classroom setting. Mutual interaction between teacher and student is maintained within the video lectures as the researcher portrays himself as a student character named “Charlie” whose somewhat humorous personality commingles with the emotions of students in this virtual world. Portraits of faculty, classified staff, real students, and tutors add to the mix to reveal a humanistic side of mathematics. Soothing color tones are also used to promote a calm learning online environment.

To accommodate different learning modalities, it was necessary to develop an array of online learning tools designed to function individually or collectively. This gives students full control of their learning experience and the opportunity to develop individualized learning schemes. Hand written lecture notes provide a natural less threatening portrayal of mathematics. Worksheets that mirror video presentations promote active learning. Video presentations that include both professor and student create a unique learning experience designed to maintain focus, alleviate anxiety, and draw students into the presentation. Homework sets with solutions provide extra practice. Online quizzes give students the opportunity to test their skills. Finally, online assessment quizzes that yield individualized reports allow students to check mastery skills of major topic areas.

Supporting Information for this Effective Practice

Evidence of Effectiveness:

In Spring of 2009, researchers attempted to assess the impact of the tool array on student affect through the use of 5-point Likert scale surveys. Three sections of pre-algebra students, enrolled in classes where they were encouraged to use the tool array, were asked to complete a survey at the beginning of the semester, and were asked to answer the same questions shortly before the end of the semester. The surveys had five questions in common.

1. On a scale of 1 to 5, how would you rate your level of anxiety about taking this pre-algebra math course? (1 = No Anxiety to 5 = Extreme Anxiety)

2. On a scale of 1 to 5, how would you rate your level of motivation to succeed in this pre-algebra course? (1 = No Motivation to 5 = Extreme Motivation)

3. On a scale from 1 to 5, rate the likelihood that you will enroll in an online math course here at the college. (1 = Not Likely to 5 = Very Likely)

4. On a scale from 1 to 5, rate the likelihood that you will transfer to a four year school after leaving the college. (1 = Not Likely to 5 = Very Likely)

5. On a scale from 1 to 5, rate the likelihood that you will pursue a career that involves a large amount of mathematics. (1 = No Way!!!! to 5 = Very Likely)

Seventy-nine students completed both surveys. The responses for the pre and post surveys were paired and tested for significant change using a two-tailed paired t-test with α = .05. The students expressed a significant reduction in anxiety over the course of the semester (0.399 point average decrease in score; p-value = 0.0089), as well as a significant improvement in motivational level (0.184 point average decrease in score; p-value = 0.0421). These were the results that the tool array was intended to produce. The greatest change was in the willingness to enroll in an online math class (2.867 point average increase in score; p-value < 0.0001). This suggested to the researchers that online tool arrays of this nature might serve as a means to help students transition from traditional classroom settings to classes in the online environment. The last two questions were intended to assess any long-term effects of using the tool array. Unsurprisingly, these did not show significant changes. The researchers did not expect these materials to radically change the career paths of large numbers of students. However the shifts, though small, were in the desired direction. Students expressed slightly more willingness to transfer to a four year school (0.019 point average increase in score; p-value: 0.8603) or pursue a math-intensive career (0.013 point average increase in score; p-value: 0.9142).

Ideally, these results would be compared to surveys submitted to a similar group of pre-algebra students who did not have access to the tool array. The researchers were unable to find such classes onsite since the website is available to everyone and has been well-publicized at their college. Future studies in collaboration with other schools is a possibility.

In addition to the Likert scale questions, the post-survey also included open-ended questions regarding the students’ usage of and reaction to the tool array. One question asked, “If you watch the videos, how do you use them? Do you watch it straight through, pause it, fast forward it, do you watch it with another student, do you watch them at home, etc.” The majority of students viewed the videos at home by themselves using this resource as a supplemental study tool. Several students watched the videos straight through racing the instructor and student character to the solution. In a few cases, students indicated that they preferred working alone rather than with another student. Darlene wrote, “I watch them alone because I don’t like the distraction of study groups or partners.”

Another open-ended question asked students, “If you use the website, how has it affected your attitude towards learning math on a computer?” Many students who used the website found the resources easier to use than expected relieving some of the anxiety they were experiencing. Eric wrote, “I was nervous coming into math for the first time in 12 years. With the addition of the site and its tools, it has helped me tremendously.” Sharron responded with, “The website is so helpful, especially to the student who works fulltime like me. It boosts up my interest in learning math.” These responses suggest to the research team, that due to the unrestricted nature of the tool array, the website might serve to help build self-confidence in mathematics with adult learners who are returning to school after several years.

Students were also asked to describe their feelings regarding the researcher’s portrayal of himself as a student character within the video presentations. Mathew responded, “It makes me feel more relaxed seeing you as the student for some reason.” David response was, “I can relate to Charlie. He reminds me of myself sitting in the classroom.” Sharron simply stated, “It made me laugh.” Karen’s response was, “What? I’m going to watch them now!”

Generally, student responses appeared to be positive with self-directed learners that carried with them some level of fear or math anxiety. Several top performing self-directed learners found the videos to be a bit annoying. Jackie responded with, “Charlie is a little distracting in an annoying way like some of the kids in the class.” Louise’s response was, “Charlie was ok at first, but it got a little old.” The fact that students with low levels of math anxiety did not find this video dynamic useful while those who were fearful of math did find it engaging suggested this video format was successfully addressing student affect.

How does this practice relate to pillars?:

Student Satisfaction – Student feedback is at the core of this project. As this collected data is purely student centered obtained from students learning in both a face-to-face and online environment, student satisfaction itself drives the evolution of the online supplemental instruction tool array. The result is an unrestricted online bridge that appeals to many non-directed developmental math students leading them to become self-directed learners who can learn in both online and traditional math classroom environments. Understanding how this process occurs would likely be beneficial to collaborative efforts of instructional design where both student and developers work together to formulate new models for traditional, hybrid, and pure online learning environments.

Faculty Satisfaction – The methodology used in this body of work outlines an approach to the refinement of one’s personal teaching style. This approach can easily be initiated by engaging students in a mutual exchange of information. Should the process be replicated by faculty in other disciplines or in developmental math classrooms at other institutions, the outcome may differ from the results described in this project. The sharing of these results among faculty would be useful in identifying how students are satisfied in different disciplines and additionally different learning environments. This in turn could inspire new ideas that increase student success.

Learning Effectiveness – In addition to the solicitation of student feedback, electronic portfolios are collected mainly consisting of the students hand written exam work. None of the exams are multiple-choice and calculator usage is limited. These documents are used as a means of both evaluating the effectiveness of the projects online learning resources as well as designing new content. Analyzing students’ work to define common arithmetic errors allows the researcher to create new content while being mindful of these common mistakes. Secondly, these errors are conveyed to the students as well as other faculty members during faculty development events to increase awareness.

Equipment necessary to implement Effective Practice:

The result of this practice is free and unrestricted and can be used by anyone having a computer with internet access. With cases where internet access is an issue, the entire pre-algebra component can be delivered on a 4GB flash memory drive and the website in its entirety can be copied onto a DVD disc readable on recently manufactured computers.

Estimate the probable costs associated with this practice:

Depending on the type of learning resources developed, cost will vary. Video editing and production can be very expensive if you need to hire a crew. Tablet PC’s equipped with Camtasia software a very popular these days for creating digital content with a total cost of about $1500. Purchasing a $300 video camera is sufficient to capture individual or focus group interviews. A hosting service may be required to deliver content to users via the internet although video content can be delivered freely using the “YouTube” or “TeacherTube” websites. Numerous websites such as “issuu” offers a free service to publish your documents. To organize all the materials in a structured format, a dedicated knowledgeable person will be required to design a main home page preferably using your college as the hosting service.

References, supporting documents:

Casazza, M., & Silverman, S. (2000). Learning and development: Making connections to enhance teaching. San Francisco: Jossey-Bass

Knowlton, D., & Simms, J. (2008). Ideas in practice: Instructional design and delivery for adult learners. Journal of Developmental Education, 32(1), 20-23, 26-30.

Travis, J (1995). Models for Improving College Teaching: A faculty Resource. ASHE-ERIC Higher Education Report No. 6. Washington D.C.: The George Washington University, Graduate School of Education and Human Development.

The Chronicle of Higher Education (2009). Almanac Issue 2009-10, 56(1).

Tinto, V (1993). Leaving College: Rethinking the Causes and Cures of Student Attrition (2^nd ed.). Chicago: University of Chicago Press

Useful Links:

Algebra2go

YouTube/MuchoMath

Interview with Larry Perez and Patrick Quigley

Other Comments:

Brannick, T., Coghlan, D. (2005). Doing Action Research in your own Organization (2^nd ed.). Los Angeles, CA: SAGE Publications

Contact(s) for this Effective Practice

Effective Practice Contact:

Larry Perez

Email this contact:

Lperez@saddleback.edu

Tags:

Student Satisfaction

HOME	INSTITUTE	MEMBERSHIP	CONFERENCES	RESEARCH & PUBLICATIONS	COMMUNITY	WHY SLOAN-C	ABOUT SLOAN-C
Welcome	Workshops	Membership	Emerging Technologies	JALN	Mailing List	The Quality Scorecard	About Us
My Account	Mastery Series	Institutional Members Listing	Blended Learning	National Survey Reports	Social Networks	Catalog of Online Programs	Contact Us
	Certificate	Corporate Sponsors	International on Online Learning	Sloan-C Series Books	Awards	Jobline	Personnel
	Webinars	Member Resources		Effective Practices	Fellows	Advisory Services	Privacy Policy
	College Pass	Insider Newsletter		Sloan-C View	Commons		Flyers
	Research Symposia	Subscribe to listserv		Workshop Publications	News & Press Room		FAQs
				Free downloads	Advertise		501c3 Status

INDIVIDUAL	INSTITUTIONAL	CORPORATE
Check here if your institution is a member Membership Comparison Chart