More classrooms trying Khan Academy, finding it better than group projects

Last week I criticized what I considered the hasty proclamation of a Khan Academy (KA) classroom pilot as a “colossal success”.  Nevertheless, because I am a big fan of Khan Academy and believe in its potential, I am happy to see this.

This semester, at least 36 schools nationwide are trying out Mr. Khan’s experiment: splitting up the work of teaching between man and machine, and combining teacher-led lessons with computer-based lectures and exercises.

The results so far make me feel cautiously optimistic.

It is too early to know whether the Khan Academy software makes a real difference in learning. A limited study with students in Oakland, Calif., this year found that children who had fallen behind in math caught up equally well if they used the software or were tutored in small groups. The research firm SRI International is working on an evaluation of the software in the classroom.

But look closely at what was happening in this class before KA.  Students would work in groups for days at a time trying to solve just one problem, an exercise that didn’t seem to help them master the fundamentals.

In the past, math class at the Summit schools was always hands-on: the class worked on a problem, usually in small groups, sometimes for days at a time. But getting an entire class of ninth graders to master the fundamentals of math was never easy. Without those, the higher-level conceptual exercises were impossible.

So what exactly were they really getting out of this teacher-supervised, time-consuming group work? This is a question many parents and mathematicians have been asking.

KA offers students a new, engaging way to learn the basics.  It also tracks data that provides teachers with precise individualized information on each student’s progress.  Good, because this allows teachers to do what they can do best.

Ms. Tavenner says she believes that computers cannot replace teachers. But the computer, she recognizes, can do some things a teacher cannot. It can offer personal feedback to a whole room of students as they work. And it can give the teacher additional class time to do more creative and customized teaching.

The thing is, I’m a little wary of giving teachers more time to oversee days of creative group projects with questionable learning goals.  I’d rather see teachers focus on taking advantage of Khan data to proactively address individual learning gaps, letting them be the expert humans interacting with and teaching students in ways no machine can.

Problems with study that claims groups are better than lectures for learning

Last month the NY Times reported on a study that showed interactive group instruction is superior to traditional lectures.

In one of the initiative’s most visible studies, Dr. Wieman’s team reports that students in an introductory college physics course did especially well on an exam after attending experimental, collaborative classes during the 12th week of the course. By contrast, students taking the same course from another instructor — who did not use the experimental approach and continued with lectures as usual — scored much lower on the same exam.

Not so fast.

Yet experts who reviewed the new report cautioned that it was not convincing enough to change teaching. The study has a variety of limitations, they said, some because of the difficulty of doing research in the dude-I-slept-through-class world of the freshman year of college, and others because of the study’s design. “The whole issue of how to draw on basic science and apply it in classrooms is a whole lot more complicated than they’re letting on,” said Daniel Willingham, a psychology professor at the University of Virginia.

Dr. Willingham said that, among other concerns, the study was not controlled enough to tell which of the changes in teaching might have accounted for the difference in students’ scores.

Here are Dr. Willingham’s candid comments from his Facebook page.

There were a lot of problems with this study. The two methods compared were each tested in just ONE classroom–so no way of knowing whether the observed effects were just due to the teacher. The “group learning” condition was taught by a new teacher–the “lecture” was the same professor as had been teaching all semester. The critical test was opt-in, and lots of students decided not to take it–and the proportions were unequal across conditions. The study was a mess. I can’t imagine why Science published it….

Studies like this are fine for what they are–they are really more pilot studies, or they could be useful as qualitative research. (I think qualitative research *is* really quite useful.) But it didn’t have the strengths of qualitative research and was pitched as a quantitative study.

Other reports about this study:

• Science
• The Economist
• The Chronicle of Higher Ed