Is the jury still out on teaching with software?
A US Department of Education study, released earlier this spring, found that using reading or mathematics software in class had no significant effect on students’ test scores. The report, titled ‘Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort’, is available here.
The News Tribune, a Washington state newspaper, reported the findings last month thus:
The long-awaited report amounts to a rebuke of educational technology, a business whose growth has been spurred by schools desperate for ways to meet the testing mandates of President Bush’s No Child Left Behind law.
Over nine thousand students in 132 schools took part in the study that looked at 15 different software products, all used for either reading or maths teaching. Becta, the government’s educational communications and technology agency, has been looking into the effectiveness teaching software for some time, but has not (at least to my knowledge) come out with such an official overall verdict on the worth of teaching with software, as America’s Department of Education has done.
The US, interestingly, is actually behind the UK in terms of educational software. Since the government in the UK started subsidising educational technology British primary and secondary schools have been busy buying and using new software and hardware. Because of this, this US report has prompted a mild flurry of introspection and self-justification by American software providers, but in the UK the worth of software is more of a given.
The American Software and Information Industry Association has argued that the study was flawed, and that it doesn’t take into account how technology is implemented and used. Whilst we’re not able to say how good the software tested was, it can be argued that the main point of the report might be completely off - looking into the worth of teaching software per se is a little like researching the usefulness of blackboard and chalk.
The problem with this research is that the answer to the question “how do we raise test scores?” will never be “use software” or “buy new technology”. The answer is, and always has been, “teach better”. A good teacher using blackboard and chalk will always have better students than a bad teacher using an interactive whiteboard, top-of-the-line projector, interactive voting handsets and teaching software.
Thankfully, the second set of data to come from this multi-year American study will look at the effect of experience with using educational software and the individual effectiveness of certain products. But this will neglect the fact that teachers who exchange classroom skills for technical competence will still fail their students. What the technology should really be doing is giving them more time and mental space to maximise those classroom techniques. Instead, the report found that teachers lectured less, and that “students using reading or mathematics software products were more likely to be working on their own.” In other words, classroom interaction and collaboration went down - teachers presumably assumed students could be left to their own devices whilst the software taught them everything they needed to know.
Shoving a software package in front of students and expecting them suddenly to learn more than they did before is a mistake. We need to be sure the software is actually any good (and not simply an expensive textbook that you need a computer to read), we need to be sure that the teacher uses the software for what it’s good at, and nothing more, and we need to give everyone time to get used to it. The best software improves a teacher’s ability to teach and a student’s ability to learn, but the teacher still has to teach and the student still has to learn.
Some teachers and school bursars assume that acquiring snazzy technology will necessarily make everything right. When they find that it isn’t that simple, they inevitably get disillusioned. Having new technology without knowing how best to use it is a little like a cyclist buying a motorbike and wondering why the expensive new thing is much harder to push and has no pedals.
As for Maths-Whizz, it won’t do the laundry, feed the cat, switch on the computer or even make a student pay attention in the first place. Maths-Whizz relies on that attention and facility with computers being present in the first place before it can do what it does best - help motivate students and teach them maths.
Does this mean we reject the findings of the US government study? Not necessarily - a lot of effort went into it and a lot of students took part. The report instead (albeit inadvertently) reminds us of a universal truth - that technology is a means to an end, not an end in itself.
May 16th, 2007 at 11:45 am
Good post Duncan.
In fact, as I am sure you are aware, there have been several reports in the UK casting doubt on the effectiveness of ICT as implemented so far: the ImpaCT2 report (www.becta.org.uk/impact2), published in 2002/03, showed virtually no measurable gains from ICT in the classroom; a recent report on whiteboards (http://www.dfes.gov.uk/research/programmeofresearch/projectinformation.cfm?projectid=14213&resultspage=1) published in January 2007 found, in the same way, “no impact on pupil performance in the first year in which departments were fully equipped”; while OFSTED’s two reports “ICT in schools” in 2002 and 2004 were both critical of the way in which ICT was generally being implemented.
The trouble is that Becta has an interest in downplaying any negative evaluations and is therefore not as honest as its US counterparts. Its summary of the ImpaCT2 report was relentlessly optimisitic (see “Cook the books” at http://books.guardian.co.uk/departments/childrenandteens/story/0,,1457789,00.html); while on whiteboards, Becta publishes a “summary of the evidence” which is again 100% positive, while it completely ignores the most recent research. They persevere with poorly thought-out, prescriptive solutions, which are killing the kind of innovation which is required to make this work.
I agree with you that technology has the potential to make a very significant difference to learning, but that the key is in the interaction between technology and teachers.
Your whizz site advocates individual tutoring (I see the Chancellor is with you on that one). Alpha Learning provides a learning platform (see http://www.alphalearning.co.uk) which allows teachers to assign, track and manage student activities in the computer. Both approaches revolve around the key importance of teacher involvement. I think that there is a dual benefit to be gained from increased 1-on-1 contact while at the same time some of the more routine aspects of practice and drilling are automated.
All the best,
Crispin.
May 16th, 2007 at 5:25 pm
Crispin,
Thanks for the detailed response. I confess I’d forgotten about ImpaCT, but I’ve certainly noticed the cheerleading tone to some of Becta’s publications. Becta risks trumpeting technology for its own sake, rather than technology intelligently implemented and properly used.
Teachers confident enough to make the hardware and software work for them (rather than the other way about) are still, sadly, in the minority. We’ve been lucky in never having had to train our users but we know full well that much software is bought, installed, and never used.
Maybe Becta should direct its funding towards training all state-education teachers in technological best practice, if it hasn’t started doing so already. When the majority of teachers are confident about the benefits (and the limits) of software, we’ll see genuine improvements. Equally, educational publishers need to meet their users half-way, rather than throwing software at teachers and expecting them to be grateful that they have something to use with their interactive whiteboards!