Research Methods are Complicated

This week, a study was released proposing clear evidence that face-to-face (F2F) instruction is superior to Internet instruction. Here's the excerpt from Educause:

"This paper, written by David N. Figlio, Mark Rush, Lu Yin, presents the first experimental evidence on the effects of live versus internet media of instruction. Students in a large introductory microeconomics course at a major research university were randomly assigned to live lectures versus watching these same lectures in an internet setting, where all other factors (e.g., instruction, supplemental materials) were the same. Counter to the conclusions drawn by a recent U.S. Department of Education meta-analysis of non-experimental analyses of internet instruction in higher education, we find modest evidence that live-only instruction dominates internet instruction. These results are particularly strong for Hispanic students, male students, and lower-achieving students. The paper also provides suggestions for future experimentation in other settings."

I haven't read the paper yet as I am awaiting the free-ish (since your email "must be connected to a subscribing college, university, or other subscribing institution," which basically means it isn't free) paper, but I see one glaring problem right off the bat, which is this:

"...live lectures versus watching these same lectures in an internet setting, where all other factors (e.g., instruction, supplemental materials) were the same."

My issue is that these things should not be the same online as in a F2F class. I understand that this is what makes the study an "experimental design," but it reinforces a common misconception that online courses are just regular courses uploaded to the Web.

This is like saying instructors should teach a course the same way for a 7 person seminar and a 307 person lecture. They shouldn't. Each has its own set of social dynamics and pedagogical techniques, as do online courses.

This isn't to dispute the results of this study, and I will re-visit this when I get my "free" copy, but we must be mindful of these nuances when talking about computer mediated instruction and Web culture.