Domestic, Elementary / Primary / Junior, Ethics, High school / Secondary 2, Private education, Public education, Publishers, Required, Students, Technology - Written by on Monday, October 10, 2011 18:31 - 0 Comments

Heard: Carnegie Learning & Other Edu Tech Firms Overpromising, Underdelivering?


by jennandjon via Flickr under Creative Commons license

Carnegie Mellon University in Pittsburgh, Pa., where professors created Carnegie Learning


The New York Times dials in with another thoughtful piece critical of marketing by Ed Tech players this weekend with a story by Trip Gabriel and Matt Richtel headlined, “Inflating the Software Report Card.” It suggests that the $2.2 billion a year education software industry isn’t exactly what it’s cracked up to be. In fact, they point to critics and data showing some of these products or companies just don’t get the results they say they do. They highlight Carnegie Learning, which the University of Phoenix purchased for $75 million in August. They write that the company, started by scientists at Carnegie Mellon University, promises to revolutionize math curriculum and classrooms. But they say a U.S. Department of Education review last year suggests this and other software market themselves heavily but offer “unproven results.”

The federal review of Carnegie Learning’s flagship software, Cognitive Tutor, said the program had “no discernible effects” on the standardized test scores of high school students. A separate 2009 federal look at 10 major software products for teaching algebra as well as elementary and middle school math and reading found that nine of them, including Cognitive Tutor, “did not have statistically significant effects on test scores.”

Amid a classroom-based software boom estimated at $2.2 billion a year, debate continues to rage over the effectiveness of technology on learning and how best to measure it. But it is hard to tell that from technology companies’ promotional materials.

Many companies ignore well-regarded independent studies that test their products’ effectiveness. Carnegie’s Web site, for example, makes no mention of the 2010 review, by the Education Department’s What Works Clearinghouse, which analyzed 24 studies of Cognitive Tutor’s effectiveness but found that only four of those met high research standards. Some firms misrepresent research by cherry-picking results and promote surveys or limited case studies that lack the scientific rigor required by the clearinghouse and other authorities.

Carnegie tells the Times that 600,000 students in 44 states are using its products. Teachers take Carnegie training and use its workbooks. They say the What Works study by the DOE has its flaws because it focuses on the wrong questions. The company was founded in 1998 by cognitive and computer scientists and math teachers at Carnegie Mellon University in Pittsburgh, some of whom have won high academic awards. Many consider the company to be a paragon of high standards. But the Times points out that Carnegie’s curriculum can cost a school roughly 3 times as much as a normal school textbook costs over six years.

At least one expert agrees with Carnegie Learning that the DOE clearinghouse reports are not perfect:

Karen Cator, a former Apple executive who directs the Office of Educational Technology at the Department of Education, said the clearinghouse reports on software should be “taken with a grain of salt” because they rely on standardized test scores. Those tests, Ms. Cator said, cannot gauge some skills that technology teaches, like collaboration, multimedia and research.

But The Times reporters point to other examples of eager marketing and unmet promises by education technology companies:

  • Carnegie, one of the most respected of the educational software firms, is hardly alone in overpromising or misleading. The Web site of Houghton Mifflin Harcourt says that “based on scientific research, Destination Reading is a powerful early literacy and adolescent literacy program,” but it fails to mention that it was one of the products the Department of Education found in 2009 not to have statistically significant effects on test scores.
  • Similarly, Pearson’s Web site cites several studies of its own to support its claim that Waterford Early Learning improves literacy, without acknowledging the same 2009 study’s conclusion that it had little impact.
  • And Intel, in a Web document urging schools to buy computers for every student, acknowledges that “there are no longitudinal, randomized trials linking eLearning to positive learning outcomes.” Yet it nonetheless argues that research shows that technology can lead to more engaged and economically successful students, happier teachers and more involved parents.

Via The New York Times

Link to the U.S. DOE Site on What Works in Education Tech

Leave a Reply


Campus Buzz

We welcome Tips & Pitches

What you need to know weekly:
The WiredAcademic newsletter.

* = required field

Latest WA Original Features

  • Twitter feed loading

Paul Glader, Managing Editor

Eleni Glader, Policy Editor

Elbert Chu, Innovation Editor

Ravi Kumar, Reporter & Social Media Editor

Derek Reed, Reporter

Brock Buesing, Contributor

error : cannot receive stock quote information

Cost of Education, Domestic, Education Quality, Ethics, For-Profit, Regulatory, Required, Students, University & College - Mar 11, 2012 21:17 - 0 Comments

Heard: Senators On Warpath Against For-Profit College Military Push

More In For-Profit

Blended Learning, Domestic, Elementary / Primary / Junior, Flipped Classrooms, High school / Secondary 2, International, Open Source Education, Required, School teachers, Startups, Students, Technology - Mar 12, 2012 19:04 - 0 Comments

Big Weekend For Sal Khan; Appears On 60 Minutes & Launches Free iPad App

More In Technology

Domestic, Education Quality, For-Profit, Friend, Fraud, or Fishy, Required, University & College - Feb 10, 2012 16:36 - 0 Comments

Heard: NYC Warns Against For-Profit Adult Education Scams

More In Friend, Fraud, or Fishy