The Null Hypothesis and Online Education

Bradford S. Bell and Jessica E. Federman write,

The meta-analyses reviewed above show that when instructional design characteristics are held constant across delivery conditions, e-learning and classroom instruction generally produce similar learning outcomes.

Pointer from Timothy Taylor–read his whole post.

The null hypothesis is that there is no difference in outcomes, and apparently the “meta analysis” does not reject the null hypothesis. Taylor interprets this as evidence that online learning has “caught up” to classroom learning in terms of quality. My more cynical interpretation is that there was never any catching up required, because what students learn does not appear to depend in any way on how they are taught. I am sure that there is a limit to this: presumably, if you do a randomized experiment in which one group of students gets $50,000 worth of instruction and another group gets zero, you will see some difference in outcomes. However, I would bet that, relative to what we do today, the way to improve cost-effectiveness in education is to slash costs. That is, my view of the null hypothesis is that most of what we spend on education has no marginal impact.

Note, however, that the authors of the study seem convinced that the null hypothesis is false. They believe that empirical evidence shows that pedagogical techniques do affect outcomes.

3 thoughts on “The Null Hypothesis and Online Education

  1. Since you mentioned slashing costs as the best way to improve cost-effectiveness, I’m curious what you thought of the cost discussion in Timothy Taylor’s post (that costs may be equal for classroom and e-learning) and in Daniel Lemire’s post linked to from Taylor’s post (that classroom is more cost effective only when you can nearly match classroom capacity and enrolled students).

    I also think an important part of the discussion needs to be networking opportunities afforded to classroom-based students and not to e-learning students. From my time in higher education, it seems campuses serve as loci of networking, and if educational outcomes are the same for classroom and e-learning, then this might tip the scales toward classroom based instruction from a future employment perspective.

    • There are more opportunities for networking in a physical classroom — but is a classroom the most cost-effective way to provide networking opportunities? Do classrooms provide the optimal amount of networking, or more than is really necessary?

      I recently attended the 15-year reunion of my elite, technical undergraduate institution. I was able to get “enough” networking done in those two days to satisfy me. Obviously I was building on relationships forged over four years, but did I really need *four* years to build that foundation, or would three have been enough?

  2. Yep and the big cost is delayed output. We should be looking for ways to minimize the years of school needed for a student to cover a certain range of material. This mostly involves asking the students to learn more.

Comments are closed.