ISTEP rescore plans reduced after lawmakers consider high costs, expert advice

Republican lawmakers have scaled back their ambitious plans to rescore the 2015 ISTEP test.

Although legislation introduced earlier this month originally suggested a full rescore of the controversial exam — meaning hundreds of thousands of tests would be re-opened and millions of student answers would be re-examined — the bill’s author now says that proposal would be too expensive, coming in at roughly $8 million to $10 million.

The author, Rep. Bob Behning, R-Indianapolis, announced during a House Education Committee meeting today that his bill — House Bill 1395 — now calls for just a partial rescore of a smaller sample of exams.
The smaller effort could boost public trust in the exam without breaking the budget, he said.

“(A rescore would) at least try to restore confidence in the assessment as we move forward,” Behning said.

The bill comes in response to heavy criticism of last year’s exam, which was plagued with scoring, test design and technical problems that accompanied new, more challenging standards and a brand-new test.

The tougher standards led to a major drop in scores. The statewide rate of students passing both the math and reading sections of the exam dropped by 22 percentage points to 53.5 percent. All but four of state’s 1,500 public schools saw their scores go down.

Behning’s bill would require the Indiana Department of Education to hire an outside company to rescore short-answer questions on the 2015 ISTEP. If the scores change, the bill would allow the state to use the new results as the baseline for calculating whether a school’s test scores improve or decline in future years. Improvement is a major factor in the state’s new A-F school grading system.

Even as his bill makes its way through the legislature, Behning says he’s still working on some of the details, seeking guidance from test experts and state officials that could lead to amendments.

Ed Roeber, a test expert from Michigan who has consulted on ISTEP for the Indiana State Board of Education, has told the legislature that a sample of 5,000 tests could be enough to verify the exam’s validity as long as the sample includes tests from urban, rural and suburban schools in all parts of the state.

“While this plan involves more steps than simply rescoring all responses to every prompt, it has the potential to answer the questions about the accuracy of hand-scoring without attendant expense of scoring all responses,” Roeber wrote in a letter to the Indiana General Assembly. “Thus, I believe you will be able to achieve your objective of checking on the accuracy of the scoring at lower cost.”

Indiana Deputy Superintendent Danielle Shockey told the committee today that a rescore isn’t necessary because the Department of Education has already conducted numerous reviews of exam, many of which were led by the very test experts Behning is consulting, including Roeber. So far, Shockey said, the state has not found any evidence of test flaws that would have affected student scores.

“The department would like to wait for data to support the need for a costly, very time-consuming rescore,” Shockey said.

It’s also not clear who would pay for the rescore. Behning, whose bill does not specify which state agency would be on the hook for the expense, raised the possibility that the company that made the test, CTB, could be asked to contribute.

“Obviously someone is going to have to pay for it,” Behning said. “We might have to … provide some additional flexibility so money could be moved from the general fund.”

Daniel Altman, a spokesman for Indiana state Superintendent Glenda Ritz, said the Department of Education has talked with the state Attorney General’s office about filing suit against CTB to recover damages for delays in scoring.

If Behning’s bill becomes law, the rescore would not be the first review of the troubled 2015 test. A panel of testing experts already conducted one review after concerns were raised in October over differences in difficulty between the paper version of the exam and the online version.

In December, the Indianapolis Star reported another scoring glitch that could have led to thousands of mis-scored tests. The state then convened a second panel to examine the data and found that the glitch did not affect student scores.

The state also allows parents to ask for a rescore of their child’s test each year. In 2015, more than 61,000 tests were rescored, Shockey said, about three times more than are usually requested. In about 11 percent of rescored exams, scores were changed by a point or more but only a fraction of rescores — 1.76 percent — led to a student moving to a different level, such as from “fail” to “pass.”

Rescores can only lead to a student’s score going up, not down, said Michele Walker, the education department’s test director.

Behning’s bill, which is up for a committee vote later this week, would also create a panel to review Indiana’s current A-F accountability system. The system might need to be adjusted to comply with the new federal Every Student Succeeds Act, which will replace the No Child Left Behind Act in the 2017-18 school year. The new law will still require Indiana to give most students a pass-fail test every year but will allow for some more flexibility.

The 20-person testing and accountability panel would include policymakers, educators and lawmakers appointed by Republican legislative leaders, Gov. Mike Pence and Ritz. Behning said he was open to adding Ritz, a Democrat, as a co-leader of the group.

“I think it’s appropriate that we spend time as policymakers talking to the brightest and best educators … and looking at what our next options are in terms of performance, standards and accountability metrics,” Behning said.

Shockey said the committee could help Indiana move on from the testing problems it’s seen over the past year. It would align with Ritz’s plan to review the state’s testing program.

Ritz has said that the state should consider a testing program that doesn’t rely so heavily on a single, end-of-year exam. She suggested a series of tests that would focus more on how students improve throughout the year.

“Indiana could be a leader in creating a more streamlined, student-centered assessment system,” Shockey said.