Roblyer, M. D., Davis, L., Mills, S. C., Marshall, J., & Pape, L. (2008). Toward practical procedures for predicting and promoting success in virtual school students. American Journal of Distance Education, 22(2), 90–109. doi:10.1080/08923640802039040
Until relatively recently, distance education research and discussion on student success largely ignored the influence of the learning environment. As the internet increasingly came into the purview of distance education, the focus on individual learner characteristics came into question as the dominant source of insight into student success. Roblyer, Davis, Mills, Marshall, and Pape ask the question: “Can any measured student cognitive and background characteristics be combined with learning environment characteristics to predict the success or failure of high school students in online courses” (2008, p. 96)? This piece presents the second iteration of a model developed to predict the success of students in a virtual high school, conducted by Roblyer and Marshall in 2002.
The impetus for the work seems to be, first, that the drop-out rate in online education settings is significantly higher than for conventional learning environments (typical brick and mortar high schools), and, second, that there were no effective (in terms of percentage accuracy identifying a student’s likelihood to remain in school and achieve academically) models to help assess a student’s potential success in a virtual school. This study presents a revision of the Educational Success Prediction Instrument (ESPRI), referred to as ESPRI-V2. It has been edited to a 60-item Likert Scale measuring “technology use/self-efficacy (self-assessment of one’s ability with technology), achievement beliefs (confidence in one’s ability to learn, an aspect of locus of control), instructional risk-taking (willingness to try new things and risk failure in instructional situations, related to locus of control), and organization strategies (ways to organize for more efficient learning)” (p. 102). These four factors originally derived from an extensive literature review of previous educational psychology and distance education studies and work on learner success in asynchronous and/or geographically distributed education situations, and they are representative of what emerged as most influential from a direct logistic regression analysis. This output was combined with “two student background variables (age and self-reported GPA), and two environmental variables (home computer availability and school period for working on the virtual course)” to yield a highly reliable model (p. 99). (Both of these descriptive variables where shown in the review of the literature to be incontestably significant for predicting online learner success.)
The study was conducted using a very large and seemingly variegated sample, comprising over 2,000 students of the Virtual High School Global Consortium (VHS). Students were from different parts of the country and attended schools of different sizes, socioeconomic status, and settings (urban, suburban, and rural schools were represented). The main factor missing from the sample is a diversity of access to internet at home – all students in the sample had the internet (and related devices) at home. It would be interesting to apply ESPRI-V2 to learners whose access to the internet is more difficult and/or less reliable; these individuals would likely display less comfort overall with the platform and navigating the online world generally, which, one could reasonably assume, would impact findings. Additionally, 80% of study participants had a period during the school day designated for their online course work; it is logical to assume that less supervision and a schedule structure requiring more student self-management would impact learner success. This is the case in many online learning settings, particularly for remediation and credit recovery, where, often, students need credits to graduate and are not duly motivated to pace their work for quick completion (rather, if permitted, students may wait until the month of graduation to worry about their missing Geometry credit, for example).
ESPRI and the approach of Roblyer et al is rooted in the sociocultural tradition which values understanding “people’s everyday activities rather than focusing exclusively on formal educational contexts and academic subjects. The emphasis is on the ways psychological processes emerge through practical activities that are mediated by culture and are part of longer histories” (Ito, Gutierrez et al, 2013, pp. 42-43). They assessed the existing literature and discourse and observed an important under-appreciation of factors associated with the student’s learning experience and environment, including technology access and life circumstances. Their “results indicate that environmental variables can play as important a role in a students’ success as the characteristics and background students bring to the course” (p. 105). As a predictive tool, ESPRI-V2 is valid and robust, having predicted success (for its sample of VHS students) with 93% accuracy.
The important pursuit that follows from these findings is how to construct an academic program that is supportive of those predicted to succeed and as well as for those who are sure to struggle (if they choose to pursue the online option after the reflection opportunity ESPRI affords program advisors to facilitate during the pre-enrollment process). One approach for an online high school is sketched briefly in the following. Upon enrollment, a student will not only take a placement assessment, to help determine specific academic strengths and weaknesses, but will take a questionnaire including the measures comprising the Educational Success Prediction Instrument. The questionnaire will be augmented with questions eliciting a range of personal details, such as hobbies, siblings, passions, aspirations for the future, concerns, and expectations in the program. Armed with a wealth of data about each individual, including past academic performance (which schools receive on any incoming student in the form or report cards or transcripts), technology skills and access (self-reported), fears and hopes, as well as the precise output from the ESPRI-V2 element, a personal learning plan can be crafted with the student. The plan and associated correspondence with the student’s instructors will be driven not merely by academic requirements and school budgetary concerns (online charter schools in Arizona receive funding based upon each student’s average daily minimum number of instructional minutes, relative to the average number of required minutes annually). This working document will be unique to the individual, tailored to help the student connect his/her interests throughout their high school career, as well as to his/her potential strengths and weaknesses as an online student, as indicated by this onboarding questionnaire.
Prior to the student beginning any coursework articulated in the plan, the student will engage in a program and online learning orientation. Structured like a course itself, the orientation is an opportunity for students to get familiar with and connected to the program, peers, staff, and the technological tools the student will be expected to use. This has been recommended by several scholars thinking about student engagement and online learner success, e.g. Beyrer (2010) who developed and studied the utility of an online orientation course for online students at a small college, and Jagannathan and Blair (2013) who echo that orientating should be integral to the all-important efforts of a school endeavoring to engage students from “day 1” for retention and achievement. Important aspects of the orientation process would include: requiring students to communicate in multiple modalities; completing tasks by set due dates; collaborating with peers on small projects designed to build relationships and help students familiarize themselves on how asynchronous and synchronous activities and may work and the challenges therein; practicing using web-based learning resources in a safe and effective way; and engaging in lessons on online learning “tips” and digital citizenship.
The intent of ESPRI-V2, according to Robleyer et al, explicitly, is not for schools to use it to deter or exclude students from an online educational setting. One of the benefits of online education is that it has the potential to enhance access to high quality education widely, given institutional capacity to support technological device and skill development needs. However, there is value in using a tool such as ESPRI to help counsel families on the options their student is better suited for. Online learning is not for everyone. For those that choose to proceed, ESPRI furnishes schools with valuable data on each student’s potential for success. Schools may implement highly personalized engagement and support plans for each student, toward retention and achievement for its student body.
Beyrer, G. M. D. (2010). Online student success: Making a difference. MERLOT Journal of Online Learning and Teaching, 6(1). Retrieved from http://jolt.merlot.org/vol6no1/beyrer_0310.htm
Ito, Mizuko; Gutiérrez, Kris; Livingstone, Sonia; Penuel, Bill; Rhodes, Jean; Salen, Katie; Schor, Juliet; Sefton-Green, Julian; Watkins, C. S. (2013). Connected learning: an agenda for research and design (p. 99). Irvine, CA, USA: Digital Media and Learning Research Hub. Retrieved from http://dmlhub.net/
Jagannathan, U., & Blair, R. (2013). Engage the disengaged: Strategies for addressing the expectations of today’s online millennials. Distance Learning, 10(4), 1–7. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eft&AN=93996527&site=ehost-live
Roblyer, M. D., Davis, L., Mills, S. C., Marshall, J., & Pape, L. (2008). Toward practical procedures for predicting and promoting success in virtual school students. American Journal of Distance Education, 22(2), 90–109. doi:10.1080/08923640802039040
Roblyer, M. d., & Marshall, J. C. (2002). Predicting success of virtual high school students: Preliminary results from an Educational Success Prediction Instrument. Journal of Research on Technology in Education (International Society for Technology in Education), 35(2), 241. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=8948095&site=ehost-live