Q: Maria has hired a company to administer tests to a million students. The company performed fine and even made her an extra $15 million. What should Maria do?
A. Stick with the original company.
B. Hire a different company that has a spotty record that includes botched tests and data breaches.
C. The answer is absolutely, definitely NOT B.
The Utah State Office of Education was faced with basically that very question back in 2017. It chose B, signing a contract with a company called Questar Assessments, Inc. Maybe the state can blame it on computer glitches, because Questar has certainly had its share of those.
But, as my colleague Courtney Tanner reported, the state knew that going in. During the bidding process, the company had reported a string of problems administering standardized tests in other states.
To briefly recap: In New York and Tennessee, Questar suffered data breaches and had student data stolen. In Texas, Questar’s parent company paid $5 million for a “disruption of testing and reporting schedules.” In California, it paid $3 million for missed deadlines. In Missouri, test results had to be thrown out because they were unreliable. And in Tennessee, 9,000 tests were found to have been scored incorrectly.
To be fair, some of those problems were the fault of the parent company, ETS, not necessarily Questar, and the state office said it had been assured by Questar the issues had been resolved.
They hadn’t.
More than 18,000 Utah students were unable to complete tests on schedule. One district had to halt testing entirely, and now the state is unsure whether it can even rely on the scores it gets back.
Last week, rather than risk further disruptions, the state school board voted 12-1 to terminate the 10-year, $44 million contract after less than two years and said it may pursue damages from Questar for failing to live up to the deal.
Next week, the Legislature’s Public Education Appropriations Interim Committee will call in state education officials to explain what happened and what they plan to do about it.
“I just want to give the state board an opportunity to explain what happened,” said Sen. Lyle Hillyard, R-Logan, committee co-chairman. “I don’t know how much money they’ve lost or not lost, or how much can be salvaged. I just want to hear from them in front of what I think will be a friendly group, let them explain what’s wrong, how they’re going to correct it and it keep it from happening again.”
All of this after the state jettisoned another company, American Institutes for Research, which had provided the platform for the standardized SAGE tests and then facilitated licensing that test with other states that brought in $15 million in revenue.
It’s like the state office took my approach to home improvement projects: If it ain’t broke, futz with it until it is.
The main reason — the only reason given, really — for the switch was that too many parents were letting their kids opt out of the SAGE test, which isn’t a software problem so much as it is a policy problem.
The opt-out was created in part to accommodate conservatives opposed to the Common Core education standards. But the test was entirely optional and soon other parents — including me — were letting their over-tested kids skip SAGE.
The problem with the logic is that changing one testing platform for another doesn’t change the fundamental issues that led parents to opt out of the test. In reality, that’s nothing more than an attempt to rebrand the test and it went about as well as the cataclysmic “New Coke” experiment in the 1980s.
At any rate, when the bids came in for the new testing platform, the state board went with a cheaper option — and even then it didn’t get what it paid for.
In fairness to state school officials, it’s hard to fault them for looking at other options — it’s key to keeping up with new technologies and avoiding stagnation. And it appears not only were the normal procurement processes followed, but the contract gave the state a way out because of Questar’s failure.
Those are positives.
After the Questar fiasco hit The Tribune, John Brandt, a retired technology officer at the Utah State Office of Education and veteran of many bid processes, offered some recommendations. While it’s obviously important to be careful with taxpayer dollars, Brandt says the lowest cost is often over-emphasized in the scoring process.
Those with technical expertise in the field who understand the deceptive jargon in bids should be listened to, and so should the end users, like teachers, who have the most hands-on experience with the product.
It’s not a bad place to start. Beyond that, the state office should review its bidding procedures to try to prevent the next Questar-style mess.
Because as long as policymakers tie school funding and teacher performance to all of these high-stakes standardized tests, getting those measurements right is one of the most important things our schools can do — you know, next to the actual teaching.
Follow @RobertGehrke