Online learning is hardly a novel concept anymore. It’s hard to find a recent or current college student who hasn’t taken at least one course online. Whether or not they like the experience is another question—some seem to thrive on the flexibility and freedom of working at their own pace, others miss the face-to-face interaction with a live instructor.

The real question is not whether online learning is here to stay—it almost certainly is—but whether it is making a fundamental difference. Is online learning really transforming higher education as we know it? There is certainly plenty of hype on either side—hardly a day goes by without someone either extolling online learning as a panacea for all that ails higher education, or bewailing the imminent collapse of the sky on our heads.

A new study released today by Ithaka S+R takes a hard-nosed look at these questions. First, the report makes the crucial point that not all online courses are the same. Merely posting a video of a lecture doesn’t cut it anymore. Let’s stop using the term “online” for those copy-and-paste traditional courses and look at the real thing: courses that take advantage of intelligent systems, drawing on vast quantities of networked data, to allow computers to guide students’ instruction. Might those online courses change the world as we know it?

Our new study looked at a prototype of one of those courses (an introductory statistics course developed by Carnegie Mellon University) and conducted a rigorous test of how much students learned in an online course compared to students taking statistics taught the traditional lecture-textbook way. The students used the online course materials in a hybrid mode, with one hour of face-to-face time each week, though most of the instructional content was delivered by the online system. By “rigorous” we mean a real randomized test with treatment and control groups (just like you learned about in Stat 101!), and a respectable sample size of more than 600 student volunteers on six public university campuses.

The result? A big fat zero. Both groups learned the same amount; there were no significant differences between learning outcomes—measured in several different ways—for the two groups. And this zero really means something. It doesn’t mean “we don’t know.” It means we have convincing evidence that the students in the hybrid course truly learned just as much as the students in the traditional course.

There could be big ramifications to this finding. For one thing, it’s easy to see how teaching large introductory courses in a hybrid mode could save institutions a lot of money. And these courses will only improve over time. If the learning outcomes are the same (or even better), especially in the kinds of public universities where it could make a real difference for hundreds of thousands of students, then that is good news indeed.