Saul Geiser and Maria Veronica Santelices [University of California, Berkeley, "Validity of High-School Grades in Predicting Student Success Beyond the Freshman Year: High-School Record vs. Standardized Tests as Indicators of Four-Year College Outcomes," Research & Occasional Paper Series: CSHE.6.07, 2007] looked at the high school GPAs and SAT scores of 79,785 students admitted to the University of California system and compared those data to two basic measures of student success at university: university cumulative GPAs and four-year graduation rates. Geiser and Santelices found that high school GPA more reliably predicted student success at university than any of the five SAT scores used in UC admissions (Geiser p.9, Table 4; HSGPA also more reliably predicted student success than parents' education, family income and the "Academic Performance Index" rank of the students' high schools). The study also notes that standardized tests show more bias against minority students than high school GPA.
These results support the position that the Madville Times has advanced previously: if you really want to know how well your school district is performing, it's silly to rely on measures like the standardized tests required by No Child Left Behind and prepared by for-profit out-of-state companies. The people with the best perspective on how well your school district is performing are the people in your school district, the teachers and administrators who actually work with your kids. In the name of "accountability," No Child Left Behind imposes a lot of extra expense on school districts (money that ends up leaving the community to pay testing companies and outside consultants to process the results). Yet standardized tests don't provide the best measure of what a majority of parents are interested in: "Is Junior learning enough to succeed in college, or are we going to waste a bunch of tuition dollars?"
Now, before you rush over to Stephanie Herseth Sandlin's next August recess event and hand her this study as proof positive that she should vote to kill No Child Left Behind, keep in mind that this study doesn't look at NCLB tests. The Geiser study looks at scores from the SAT,which is an aptitude test, purporting to measure a student's potential for success in college. No Child Left Behind requires achievement tests that measure what a student as already learned. The SATs are used nationwide; the testing companies craft different tests for different states, so it will take a lot of researchers and research dollars to do comparable studies on the gaggle of tests proliferated by NCLB.
Nonetheless, the Geiser study does put the lie to the conventional wisdom that local GPAs are less reliable than standardized tests. The SAT, as the grand-dame of all standardized tests, fails to do exactly what it was created to do: to provide a measure of student success more objective and reliable than can those darned teachers, who surely must be biased in evaluating their students and can't be trusted to provide good scientific data. Hmm. Turns out the teachers are providing better data than the well-funded College Board after all. Maybe before we reauthorize No Child Left Behind and impose further expense on our school districts, we should ask for some data that shows the testing companies are providing us with instruments that give any better measure of our students' achievement than the tests and assignments we're already paying our teachers to create. If we don't have that data, then maybe we should scrap No Child Left Behind and get back to trusting our local teachers.
But be careful -- start trusting your teachers, and then you might start getting even more radical ideas, like paying your teachers what they are worth....
p.s.: For more on why we should let No Child Left Behind die this September, see Edward Humes's review of Linda Perlstein's new book Tested ["Review: Linda Perlstein's 'Tested' Examines School," Los Angeles Times, 2007.08.12, posted on Newsday.com]:
The endless regimen of testing, drilling, report filing, student bribing and student berating that Perlstein describes could only have been conceived by politicians and ideologues who rarely set foot in actual public schools (and would never subject their own children to the Frankenstein classrooms their policies have created)....
...Perlstein depicts a school obsessed not so much with educating as with measuring education, and with doling out a kind of pallid simulation of knowledge. Stories, for example, are always analyzed for their structure, almost never for their actual content....
...Creativity and spontaneity only get in the way of data collection. And so the author treats us to the awful moment when bright kindergartners identifying long vowel sounds are told to stop - because rigid lesson plans say they are supposed to know only short vowel sounds....
...The school lavishes attention on troubled and unruly children, while the most gifted and co-operative are ignored, one of No Child Left Behind's most destructive unintended consequences.
Reviewer Humes notes that Perlstein doesn't connect the dots as damningly as she ought, but the observations do that job for us: No Child Left Behind kills local control of education, and often education itself. It is predicated on the notion that we can't trust our local teachers to do their job and report fairly on the results. It dooms certain schools to failure and "takeover, closure, or privatization" [Humes]. Let's eliminate NCLB now before it does any more harm.
I always enjoy it when you attack the particular aspects I actually like of things I in general do not like at all. Injecting accountability into the educational system is crucial to fixing problems. I have never heard of anyone saying that SAT or ACT scores are better predictors of college success. I heard the exact opposite long before I left the hallowed halls of MHS. But as you note it has nothing at all to do with the testing in NCLB. As I understand things, NCLB testing is actually quite weak. If a teacher can't bring students up to competency levels for those test and still have plenty of time for their own special emphasis in their course objectives and teaching style, including the important soft skills of creativity, than they really are failing. I am however, completely out of that loop. Since you are living in it, could you let us know what your personal experience is with the demands of these tests. Are they easy or tough for a average B- or C student? And what objective success does a school need to meet to not be a "failing" school? Do you know of any schools that are failing?
ReplyDeleteI like the concept, but the devil is in the details and I just don't know them.
I will agree, the NCLB tests are not killers. Of course, if we accept that the tests aren't killers, why are we giving them in the first place? Why don't we look at the overall grades the kids get from their own teachers, or other metrics that already exist, instead of imposing another layer of costly bureaucracy? Why create another level of accountability to the federal government, a far-distant entity with no real knowledge of your kids or your school district, when we already have what ought to be the fully sufficient accountability of the report card, parent-teacher conferences, school board meetings, and all the other information a responsible parent can get daily in the school and community?
ReplyDelete"Failing" the NCLB tests is also problematic. It's not like normal failing, where we say, "Get 7 out of 10 questions right to pass." NCLB requires that every year a school have more kids produce "proficient" scores. States set different benchmarks for "proficient," but let's say on one test in one state it means getting 7 out of 10 math questions right. On even the simplest tests, the scores should follow a normal distribution. Even the best teaching can only shift the bell curve so far to the right. There will always be variation in scores, and there will inevitably be some years when a school's overall score drops. Thus, inevitably, over time, every school will "fail" NCLB, because of natural variance in ability of student populations and other factors. That's not accountability; that's blind bureaucracy.
A little bit off topic here, but I would like to know what you think about the "tracking" that MHS does starting this year. I know my kid when he started high school was told by the school counselor that he was probably tech school material (based on his freshman grades and the major adjustment of starting high school). He would probably have been advised to take the minimum or middle track. But, he went on to graduate in the top of his class, was a member of the Natl Honor Society, and graduated from vet school only six years later!
ReplyDeleteI think tracking as a freshman only harms kids. Not all kids mature at the same rate, some have trouble adjusting to the changes of high school, and most don't have any idea what they want to do with their lives at that age.
And, on topic now, I think the idea behind NCLB was good, but the reality is that it doesn't really work as set up. My daughter-in-law teaches English as a second language, and her students, many of which can hardly speak English, had to take the same test as the English proficient kids, and their expected poorer scores reflected back on the school negatively. I'm not a teacher, but even I can see the foolishness of any scores obtained this way.
Good morning, Nonnie!
ReplyDeleteTracking -- indeed, off-topic, but an interesting question, and not one I (or anyone else) can answer with some pithy partisan swipe. It's a real issue that requires sensible discussion.
To relate it first to this post's topic, I'd suggest that if a school is going to track kids, it has to be very careful in what information it uses to determine that tracking. Is one standardized test able to determine down which career path we ought to send a fifteen-year-old? (High schools give such a test: the ACT-prep PLAN test includes UNIACT, an interest inventory that purports to tell kids what careers they would enjoy and succeed in. ACT says it has 20+ "years of research supporting the reliability and validity of UNIACT." Should we trust it any more than the above study says we should trust the SAT over the grades teachers give? Even a reliable test won't give valid results if the student doesn't feel like making an effort on the test the day it's given. If tracking is even possible, it should only be done by teachers, counselors, and parents who have a real connection with the student, who really listen and try to understand the student... and even then, there's not guarantee they'll make the right guess about a career that a kid is still ten years away from getting into.
That's a long way of saying tracking is a crapshoot. Who knows what a kid will decide to be after high school, or what opportunities the job market will give (or take away) in 5, 10, or 40 years? I had a student who insisted he didn't need to learn anything but small-engine repair, since that was all he was going to do for his entire life. But how does he know some other interest won't hit him at age 25 or 45? How does he know small engines won't become obsolete, that in 2030 lawn mowers won't run on rubber bands and fusion power? How does he know he won't have to find some other work to make a decent living?
The only thing certain is that each kid will grow up to be a citizen, a member of Western culture, and probably a parent. Thus, we have to provide a unified curriculum that prepares every student for those roles. We can have college-prep and career electives -- not everyone should have to take AP Calculus or Advanced Woodworking. But every student should share a certain core curriculum. Every kid should read Shakespeare. Every kid should do algebra. Every kid should study the Constitution, US history, and world history. Those are the elements of shared culture that unify us as a people. If high schools push specialization on kids, they do damage to our social cohesion. Kids have to see themselves as part of a whole community, not as cogs in a machine (there is a difference).
How's that for off-topic? :-)
Good answer, Cory. Thanks. I worry that if a kid is "tracked" as a freshman as a low achiever, he might accept that as his role in life and never strive to be more. I think all classes should be offered to all kids and no kid locked into thinking he is only as qualified as he appears to be on the start of his high school career. He/she is still a child at age 14 and nowhere near making decisions that will affect his life even two or three years down the road.
ReplyDelete