For some time I've been researching in order to write a lengthy academic piece regarding how discredited the LA Times' VAM (and VAM in general) is amongst anyone knowledgeable about pedagogy and testing methodologies. In the meantime, the watershed study by the School of Education, University of Colorado at Boulder came out and saved me the trouble. The definitive university study doesn't just refute the specious Times' data and conclusions utterly, it drives a proverbial stake through the heart of the VAM pseudo-science, that like its cousin phrenology, should have passed off the scene long ago.
Initially some familiar with Russ Stanton's favorite school privatization hit-men, wanted to excuse them from their wanton recklessness and mendaciousness that ruined many lives and reputations. For me, there was no such deviation from principles. Here are my thoughts that were published when the news that the LA Times' grand VAM deception was first exposed as the sham it is:
I have a choice between believing distinguished university professors and researchers, confirming what nearly all the credible experts were saying about VAM, versus yellow journalists and amateur statisticians Jason Felch and Jason Song?
It's bad enough that the Gates Foundation paid for all the VAM pseudo-science hype, but I recently learned that Gates also supports the Discovery Institute to the tune of millions. They believe humans rode dinosaurs, which is about as scientifically correct as VAM.
A close friend recently giggled when I said during a discussion on my piece On Anschutz, Villaraigosa, LAUSD Privatization Candidates, and Riding Dinosaurs which further exposed the Gates Foundation's funding of the Discovery Institute:
I have no problems with religion, I'm a person with strong spiritual beliefs myself. However, when you start talking like the Flinstones is a documentary, you've lost me.
So rather than spend an inordinate amount of time writing about VAM, which should also go the way of the dinosaur, let me provide some reading material and some masterful quotes.
First, teacher and author Larry Ferlazzo has collected some excellent resources regarding VAM The Best Posts About The LA Times Article On "Value-Added" Teacher Ratings.
Most important are these documents from the National Education Policy Center (NEPC):
- Due Diligence and the Evaluation of Teachers
- Research Study Shows L. A. Times Teacher Ratings Are Neither Reliable Nor Valid
- Gates Report Touting ‘Value-Added’ Reached Wrong Conclusion
- FACT SHEET CONCERNING L.A. TIMES’ POSTS OF FEBRUARY 14, 2011
The "Conclusion" section of the last document cited has probably the best assessment of both the Times' original work, and their futile defense of it.
The Times has not been simply reporting on teacher evaluations or ratings. It has been creating them and publicizing them. This unusual position confers upon the Times a profound obligation to ensure that any ratings it publishes are both valid and reliable. It is incumbent on the paper‟s reporters and editors to cautiously report on the effort‟s weaknesses.
Moreover, this ethical obligation is amplified when the Times is presented with a critique of the social science work that the paper had commissioned and used. Yet inexplicably the story about the critique was assigned to the same reporter who wrote and has repeatedly defended the original story, and this assignment was apparently made by the same editor who worked on the original story. The result, not surprisingly, was an attempt to mislead readers and whitewash the critique.
As the two NEPC fact sheets document, this is not a matter of experts merely disagreeing. This is a matter of mistakes in judgment and in fact. We call upon the Times to stop trying to defend the indefensible, pull down its invalid teacher ratings, and set about the difficult business of getting its story right.
Stanton's community hating and teacher loathing publication still tries to defend the indefensible. Following one of their tepid apologies of their neo-phrenology, the distinguished Dr. Mike Rose of the UCLA Graduate School of Education and Information Studies posted a cogent comment that speaks to the heart of this issue:
I grew up with The Times, know and admire people on the paper, and have been fortunate to contribute to its opinion page. But I have to ask: What is happening at The Times? With each article in Value-Added methods it gets deeper and deeper into a mess of its own making and displays further hubris or ignorance.
Mr. Lauter claims that the big public policy question people have been arguing since The Times published its first value-added article is whether "teachers have a significant impact on what their students learn or whether student achievement is all about demographics." This statement is a little like claiming that a recently published article on, let's say, a lawsuit against a tobacco company got the public arguing about the link between tobacco and lung disease.
The complex web of issues involving parental income, student achievement, and the role of the teacher has been a central topic in education research and practice for a very long time and in many countries. I was writing about it over two decades ago, and a half-century worth of work preceded mine. To say that The Times study (or the one from Colorado) finds that the quality of the teacher matters, is perhaps news to Mr. Lauter, but not to others who know about schools.
In its eagerness to justify itself, The Times staff resorts to parsing out the details of The Times and Colorado studies -- something, by the way, that journalists always accuse researchers of doing -- but miss the big point. The paper acknowledges but finally sets aside the well-known fact that Valued-Added scores are not stable, and that the results one gets will significantly depend on the models and assumptions one uses. Given the way these measure work, it is unlikely that the instability will be refined away. These problems, therefore, will have a marked effect on the accuracy of the evaluation of teachers -- including the vast range in the middle -- and on their employment and career advancement. Furthermore, just as important, the use of this problematic method will affect the nature of the profession and recruitment into it.
I cringed at the cheap insinuation that the Colorado study is influenced by the source of some of its funding. Shall we consider the vested interest of Mr. Lauter, Mr. Felch, etc. in this project? Or the fact that Thomas Kane, who Mr. Lauter approvingly quotes, is a high-level official at the Gates Foundation, overseeing a project which has invested heavily in Value-Added methods? The point is that there are all kinds of personal, professional, and institutional investments in this debate, so if you're going to lay them out, lay them all out. And if you suspect a biasing influence, do the reporter’s job of demonstrating it.
But the big, big question for me is how is it that this newspaper moved so strongly toward advocating a particular technology in school reform? The Times is not just editorializing that we need reform, but within its news department is taking a side on a technique. The paper is no longer reporting the news, but creating it and spinning it.
Here's a thought experiment. The PSA test for prostate cancer has many supporters and detractors in the medical community. The Times has run excellent articles reporting on this debate. Imagine, though, if the Times decided to take a side in it, and hired a medical researcher to run a study for The Times in support of its stance. Preposterous, right?
But that is analogous to what The Times has done.
Mike Rose
Social Research Methodology Division
UCLA Graduate School of Education and Information Studies
Posted by: Mike Rose | February 14, 2011 at 08:56 PM
Thank you Professor Rose for phrasing so eloquently what we all wanted to say.
No comments:
Post a Comment