FINAL REPORT AND RECOMMENDATIONS

Task Group B3
How Well Do Students Learn

With the On-Line Curriculum Delivery Method?

John Drueke

Nobby Emmanuel

Cathleen Kennedy

Mary Polite

Ron Schaefer

Bryce Sullivan, convener
March 20, 2001

 
Our task group was not able to arrive at a definitive answer to the question "How well do students learn with the on-line curriculum delivery method?" We conducted a literature review of the research examining student learning and found little peer-reviewed research that has examined this issue. The research available on student learning in on-line courses is often not published in peer-reviewed journals, but there is some research available from journals such as the Journal of Asynchronous Learning Networks. There are also other documents available with information relevant to our question such as the one developed by the Institute for Higher Education Policy entitled Quality On the Line: Benchmarks for Success in Internet-based Distance Education. Overall, the research papers we could locate generally support the effectiveness of student learning via on-line courses.

The Journal of Asynchronous Learning Networks (JALN) presented a special issue on the effectiveness of asynchronous learning networks (ALN) in September of last year. ALNs include the type of on-line courses and web-enhancement the TLTR has been considering and are designed for "anytime - anywhere learning" (see the JALN website for a definition of ALN). Included in the JALN special issue were papers on such topics as "Student Satisfaction and Perceived Learning with On-line Courses." The most recent issue of the journal has an article that bears directly on our task group's question entitled "Measuring Learning Effectiveness: A New Look at No-Significant-Difference Findings." These articles are somewhat representative of the literature available on the effectiveness of on-line learning.

The article on Student Satisfaction and Perceived Learning with On-line Courses (Fredericksen et al., 2000) presents some useful data from over 1,400 students who took on-line courses. Some of the study highlights are as follows:

1.         "Interaction with the teacher is the most significant contributor to perceived student learning in these on-line courses."

2.         "Students who reported that they participated in their on-line classes at higher levels than in the regular classroom reported the highest levels of perceived learning."

3.         "Interaction with classmates is a significant contributor to perceived learning in on-line courses."

4.         "Student motivation for taking courses appears to play an important role in perceived learning."

In the article Measuring the Importance of Collaborative Learning for the Effectiveness of Asynchronous Learning Networks (ALN) Hiltz et al. (2000) presented evidence suggesting that ALNs can improve educational access and progress toward degree. Additionally, student learning was improved as measured by both student self-report and objective measures.

 

The following abstract from the article Measuring Learning Effectiveness: A New Look at No-Significant-Difference Findings (Joy & Garcia, 2000) identifies some of the key issues in the research on student learning via an on-line curriculum.

 

Researchers, instructional designers and consumers of ALNs [asynchronous learning networks] must be cautious when interpreting results of media comparison studies. Much of the literature purports to have found no significant difference in learning effectiveness between technology-based and conventional delivery media. This research, though, is largely flawed. . . ALN practitioners should not assume that students would learn better from technology delivery systems. Rather, ALN practitioners should adhere to time-tested instructional design strategies, regardless of the medium they choose. Learning effectiveness is a function of effective pedagogical practices. Accordingly, the question for ALN practitioners ought to be: "What combination of instructional strategies and delivery media will best produce the desired learning outcome for the intended audience?"

 

The report prepared by the Institute for Higher Education Policy, Quality On the Line: Benchmarks for Success in Internet-based Distance Education, provides specific benchmarks they deem as essential for quality on-line education. The benchmarks cover the following areas: institutional support, course development, teaching/learning, course structure, student support, faculty support, and evaluation and assessment. A number of these benchmarks are concerned with student learning. For example, one of the course development benchmarks states, in part, that "learning outcomes--not the availability of existing technology--[should] determine the technology used to deliver course content." The teaching/learning benchmarks and the evaluation and assessment benchmarks are also relevant to the effectiveness of student learning in an on-line environment.

 

This brief overview of selected research reports suggests that on-line learning can be an effective curriculum-delivery method. However, there were some concerns raised in our task group discussion about the broader range of learning that takes place in the collegiate experience that an on-line student will not experience. Such learning might take place through participation in a student service organization or through informal social exchanges. In this regard, there may be two profiles of students who have different needs for experiences outside the classroom. There are the traditional students who would miss out on the full collegiate experience if they only learned through online courses. There are also nontraditional students who would not be as likely to need these experiences for academic and personal growth and maturity.

 

Our task group therefore makes the following recommendations:

  1. that the University consider developing an ongoing evaluation process to gauge the effectiveness of web-enhanced teaching at SIUE. Evaluation and assessment benchmarks should be used as new on-line courses are developed. These benchmarks might reflect those suggested in the Institute for Higher Education Policy (2000) report. A thorough program of evaluation and assessment will show how well students learn with the on-line curriculum delivery method at SIUE.
  2. that the University consider creating competitive release time or graduate assistant grants for faculty to develop on-line courses. Such grants might assist faculty who have little experience with on-line courses or with emerging classroom technologies. In large measure the grant support could be aimed at establishing on-line courses where none or very few previously existed.
  3. that the University consider developing some kind of Distinguished on-line Scholar program. The program could emphasize cutting edge knowledge as well as distribution of that knowledge within the University. The Distinguished Scholar could focus on not only experimenting with new means of using technology in the classroom but also providing demonstrations and seminars to faculty throughout a semester or an academic year. Again, a competitive award on a yearly basis might be most productive.
  4. that the University carefully scrutinize any undergraduate degree programs which would be offered entirely in an on-line format. This would reflect the value SIUE places on the entire range of intellectual and social skills that comprise today's collegiate experience.

References

Fredericksen, E., Pickett, A., Shea, P. Pelz, W. & Swan, K. (2000). Student satisfaction and perceived learning with on-line courses: Principles and examples from the SUNY learning network. Journal of Asynchronous Learning Networks, 4(2). Retrieved March 14, 2001, from the World Wide Web: http://www.aln.org/alnweb/journal/Vol4_issue2/le/Fredericksen/LE-fredericksen.htm

 

Hiltz, S. R., Coppola, N., Rotter, N., Turoff, M., Benbunan-Fich, R. Measuring the importance of collaborative learning for the effectiveness of ALN: A multi-measure, multi-method approach. Journal of Asynchronous Learning Networks, 4(2). Retrieved March 14, 2001, from the World Wide Web: http://www.aln.org/alnweb/journal/Vol4_issue2/le/hiltz/le-hiltz.htm 


Institute for Higher Education Policy. (2000, April). Quality on the line: Benchmarks for success in internet-based distance education. Washington, DC: Author. Retrieved March 14, 2001, from the World Wide Web: http://www.ihep.com/quality.pdf

 

Journal of Asynchronous Learning Networks. (2000). Special Issue on Learning Effectiveness. Nashville, TN: ALN Center. Author. Retrieved March 14, 2001, from the World Wide Web: http://www.aln.org/alnweb/journal/jaln-vol4issue2.htm

 

Joy, E. H., & Garcia, F. E. (2000). Measuring learning effectiveness: A new look at no-significant-difference findings. Journal of Asynchronous Learning Networks, 4(1). Retrieved March 14, 2001, from the World Wide Web: http://www.aln.org/alnweb/journal/Vol4_issue1/joygarcia.htm