Are Children Learning

Test format questions hold up state board vote on ISTEP passing scores

PHOTO: Scott Elliott
The state board decided not to vote on projected ISTEP passing scores this month.

Last-minute questions about differences in the 2015 online and paper-pencil ISTEP tests postponed an Indiana State Board of Education vote to set passing scores on the exams today.

The delay put off a discussion about big drops expected in the number of students who passed ISTEP last year if the board adopts the recommended passing scores. The passing rate declines are connected to tougher new academic standards, which made for more difficult ISTEP tests.

Board members, who often have been at odds with state Superintendent Glenda Ritz and the Indiana Department of Education staff, quickly complained that the latest delay was another example of the Ritz’s team failing to provide timely information to the board.

This delay is the second for ISTEP scores after the testing company that creates the exams, California-based CTB, reported problems with short-answer questions back in August.

“In my view this was avoidable,” board member Gordon Hendry said. “We could’ve voted on cut scores today if the information was delivered like it should have been.”

But Danielle Shockey, deputy state superintendent, said nothing was withheld from the board

A draft study doesn’t actually need to be done first, she said. The process used to come up with the recommended passing scores could still be approved, and technical adjustments could come later.

“It is the same valid work we’ve done previously for other tests,” Shockey said. “There is absolutely no reason to pause or delay the test cut-score setting today.”

ISTEP scores factor into many critical statewide decisions, such as school A-to-F grades and teacher evaluations, as well as teacher pay decisions. At last month’s board meeting, the Indiana Department of Education said school grades wouldn’t be released until January 2016.

The comparison study is very technical. It details statistical tests performed on test scores and questions from both the online and paper-pencil tests. The findings seem to suggest that some questions on certain tests are harder on the online test than the paper-pencil test and vice versa, particularly in the math section.

“The concerns center on a report by the Department of Education from October 2, but was not provided to the test’s Technical Advisory Committee, a team of state and national experts hired to monitor the testing process, and the State Board of Education until Tuesday night,” said a statement from the state board. “That report raises serious questions about potential differences in difficulty between the online and paper/pencil tests.”

Comparison studies are not unusual, in Indiana or in general. They are done to see whether different test formats affect the validity of the test scores.

But the state board wants its own test experts to review the study, and it says the department didn’t fulfill its end of the bargain. Board member Sarah O’Brien said she requested this study back in July and was told by the education department that it would be finished before work on passing scores started.

“In order for us to justify such a huge decision for the field, we need to be able to say we have this unbiased information saying we are on-track,” said O’Brien, a teacher from Avon. “I don’t feel we have that.”

Karla Egan, a test expert on the technical advisory committee that helps oversee the score-setting process, said the comparison study doesn’t necessarily show something is wrong. But the test experts should have it before they approve recommended cut scores.

“We don’t know that there’s a problem,” Egan said. “We just need more time.”

State board test director Cynthia Roach said comparison studies should, and typically have been, completed before cut-off scores are created to begin with. When asked why the process continued without the study, state board spokesman Marc Lotter said they were assured the tests were comparable by a CTB representative, but no actual data was given.

Michelle Walker, the department’s test director, said based on the draft study by CTB, she’s confident the paper-pencil and online tests are comparable and will prove valid. The state board’s review of the process doesn’t have to hold things up.

“The draft came to us on (Oct. 2) so we could discuss it with CTB last week, which we did do,” Walker said. “If they want feedback … I think that’s fine. I think that’s a third-party, secondary piece in the process.”

The board was expected to vote today on ISTEP passing scores, which projections said could drop by about 16 percentage points in English and 24 percentage points in math compared to 2014. The drops are worse than in prior years, as ISTEP scores have rarely swung up or down by more than a few percentage points since 2010.

Part of that difference is because the 2015 ISTEP test was written to measure new, more challenging academic standards. For that reason, lower scores were expected, and some educators and advocates say other states that have switched standards saw similar changes.

Ritz said it’s possible the board could vote on the recommended passing scores in two weeks when it holds a special meeting to discuss new proposed high school diplomas on Oct. 28.

more digging

Kingsbury High added to list of Memphis schools under investigation for grade changing

PHOTO: Shelby County Schools
Kingsbury High School was added to a list of schools being investigated by an outside firm for improper grade changes. Here, Principal Terry Ross was featured in a Shelby County Schools video about a new school budget tool.

Another Memphis high school has been added to the list of schools being investigated to determine if they made improper changes to student grades.

Adding Kingsbury High School to seven others in Shelby County Schools will further delay the report initially expected to be released in mid-June.

But from what school board Chairwoman Shante Avant has heard so far, “there haven’t been any huge irregularities.”

“Nothing has surfaced that gives me pause at this point,” Avant told Chalkbeat on Thursday.

The accounting firm Dixon Hughes Goodman is conducting the investigation.

This comes about three weeks after a former Kingsbury teacher, Alesia Harris, told school board members that Principal Terry Ross instructed someone to change 17 student exam grades to 100 percent — against her wishes.

Shelby County Schools said the allegations were “inaccurate” and that the grade changes were a mistake that was self-reported by an employee.

“The school administration immediately reported, and the central office team took the necessary actions and promptly corrected the errors,” the district said in a statement.

Chalkbeat requested a copy of the district’s own initial investigation the day after Harris spoke at the board’s June meeting, but district officials said they likely would not have a response for Chalkbeat until July 27.

Harris said that no one from Dixon Hughes Goodman has contacted her regarding the investigation as of Thursday.

The firm’s investigation initially included seven schools. Kingsbury was not among them. Those seven schools are:

  • Kirby High
  • Raleigh-Egypt High
  • Bolton High
  • Westwood High
  • White Station High
  • Trezevant High
  • Memphis Virtual School

The firm’s first report found as many as 2,900 failing grades changed during four years at nine Memphis-area schools. At the request of the board, two schools were eliminated: one a charter managed by a nonprofit, and a school outside the district. The firm said at the time that further investigation was warranted to determine if the grade changes were legitimate.

The $145,000 investigation includes interviews with teachers and administrators, comparing teachers’ paper grade books to electronic versions, accompanying grade change forms, and inspecting policies and procedures for how school employees track and submit grades.

Since the controversy started last year, the district has restricted the number of employees authorized to make changes to a student’s report card or transcript, and also requires a monthly report from principals detailing any grade changes.

Silver Lining Playbook

Memphis’ youngest students show reading gains on 2018 state tests — and that’s a big deal

PHOTO: Caroline Bauman
A student works on reading comprehension skills at Lucie E Campbell Elementary School in Memphis and Shelby County Schools.

Those working to improve early literacy rates in Shelby County Schools got a small morale boost Thursday as newly released scores show the district’s elementary school students improved their reading on 2018 state tests.

The percentage of Memphis elementary-age students considered proficient in reading rose by 3 points to almost one-fourth of the district’s children in grades 3 through 5. That’s still well below the state average, and Superintendent Dorsey Hopson said “we obviously have a long way to go.”

PHOTO: Caroline Bauman
Superintendent Dorsey Hopson has overseen Tennessee’s largest public school district since 2013.

Strengthening early literacy has been a priority for the Memphis district, which views better reading skills as crucial to predicting high school graduation and career success. To that end, Shelby County Schools has expanded access to pre-K programs, adjusted reading curriculum, and made investments in literacy training for teachers.

Hopson said the payoff on this year’s TNReady scores was a jump of almost 5 percentage points in third-grade reading proficiency.

“It was about five years ago when we really, really, really started pushing pre-K, and those pre-K kids are now in the third grade. I think that’s something that’s really positive,” Hopson said of the gains, adding that third-grade reading levels are an important indicator of future school performance.

TNReady scores for Shelby County Schools, which has a high concentration of low-performing schools and students living in poverty, were a mixed bag, as they were statewide.

Math scores went up in elementary, middle, and high schools in Tennessee’s largest district. But science scores went down across the board, and the percentage of high school students who scored proficient in reading dropped by 4 percentage points.

The three charts below illustrate, by subject, the percentages of students who performed on track or better in elementary, middle, and high schools within Shelby County Schools. The blue bars reflect the district’s most recent scores, the black bars show last year’s scores, and the yellow bars depict this year’s statewide averages.

Hopson said he was unsure how much the scores of older students — all of whom tested online — were affected by technical problems that hampered Tennessee’s return this year to computerized testing.

“From what people tell me, kids either didn’t try as hard in some instances or didn’t take it seriously,” Hopson told reporters. “We’ll never know what the real impact is, but we have to accept the data that came from these tests.”

But students in two of the district’s school improvement initiatives — the Innovation Zone and the Empowerment Zone — showed progress. “We’re going to double down on these strategies,” Hopson said of the extra investments and classroom supports.

In the state-run Achievement School District, or ASD, which oversees 30 low-performing schools in Memphis, grades 3 through 8 saw an uptick in scores in both reading and math. But high schoolers scored more than 3 percentage points lower in reading and also took a step back in science.

The ASD takes over schools in the state’s bottom 5 percent and assigns them to charter operators to improve. But in the five years that the ASD has been in Memphis, its scores have been mostly stagnant.

Tennessee Education Commissioner Candice McQueen said she and new ASD Superintendent Sharon Griffin are reviewing the new data to determine next steps.

“We are seeing some encouraging momentum shifts,” McQueen said.

Chalkbeat illustrator Sam Park contributed to this story.