A new initiative from the U.S. Chamber of Commerce Foundation, The Future of Data, Assessments, and Accountability in K-12 Education initiative, is exploring the effectiveness of data and assessments in America’s K-12 public schools. The first phase of the project examines the impact of data-driven accountability education policy over the last 20 years, from No Child Left Behind (NCLB) to the Every Student Succeeds Act (ESSA).
The foundation hosted a webinar on March 29 to support the release of the report, Looking Back to Look Forward: Quantitative and Qualitative Reviews of the Past 20 Years of K-12 Education Assessment and Accountability Policy, a quantitative research review authored by Dan Goldhaber and Michael DeArmond of CALDER at the American Institutes for Research and a qualitative analysis by Chris Stewart at brightbeam.
“This report is a crucial tool for the field, especially now as we grapple with the historic learning loss that has occurred over the pandemic,” said Caitlin Codella Low, vice president of Policy and Programs at the foundation. “We want to make sure we understand the criticisms: that families don’t find assessment information data timely or useful, teachers complain that tests take too much time, and from civil rights advocates who are opening our eyes to the need for culturally relevant content for students. We want to see those concerns addressed while holding public schools accountable — that’s why we call the initiative ‘the future’ of education data.”
Findings
While the national narrative about the success of NCLB is mixed, researchers found it to be the most effective education policy reform of the last 20 years. Unsurprisingly, the way in which it was implemented made a difference in the initiative’s success in improving student outcomes.
“What was surprising?” asked report co-author Goldhaber. “Just how little infrastructure existed to evaluate large initiatives prior to NCLB. I also think it’s surprising how baked in conclusions seem to be about whether a program worked or didn’t without research backing.
“Top line findings on NCLB accountability is positive and particularly significant in math and for younger kids and those that were on the accountability margins,” he continued. “To us, that suggests that incentives made a difference, and that in particular, schools were responding in ways that were consistent with the theory of action around accountability.”
Goldhaber said the data is all based on assessment outcomes, but noted that while there has been backlash surrounding a reliance on testing for measuring schools, “there are significant relationships between how well students do on tests during their K-12 experience and how well they do later in life.”
“In terms of some of the other federal initiatives — Common Core, school improvement grants, teacher evaluation — it does not look like there are any broad-scale, national, positive effects, but there is quite a lot of variation across states,” he said.
Both the quantitative and qualitative analyses found that NCLB revolutionized the collecting of national data in public education and was especially valuable in its requirement to disaggregate data by student group. However, the qualitative data revealed the deep dissatisfaction with the initiative on the ground.
Stewart shared that stakeholders felt NCLB narrowed the curriculum and focused on tests in a way that “started rating schools and rating the performance of students in ways that labeled them, creating a culture or shame and blame that caused distress for students, families and teachers without a lot of answers of how they can get out of that position.”
Panelist Ivan Duran, superintendent of Highline Public Schools in Washington State, spoke about the detriment of testing-focused accountability. “We were used to having assessments, but the shift to accountability and outcomes created a lot of stress and started to label a lot of schools. They were schools that had a large number of second-language learners and students of color. I’ve appreciated the focus on subgroups, and it got us thinking more about what we need to do to be prepared, but it wasn’t the end-all be-all. I think we lost some good practices that we are still trying to recover now, especially after the pandemic.”
While the national narrative around NCLB at its close was mostly negative, “I think, where this gets complex, is that you need to start looking into stories of how people’s lived experience was affected by NCLB, based upon their own state laws and their own local areas,” Stewart said.
Local circumstances and implementation matter, and that is why local control is more important than ever, said panelist Maya Martin Cadogan, executive director of Parents Amplifying Voices in Education.
“As we think about what the future looks like, we need to make sure that those decisions about what accountability and assessments look like are localized,” she said. “Involve all of your stakeholders. We’re thinking about accountability as something that is a part of a community being able to hold itself accountable for what it wants to see for its children.”
NCLB was top-down, from the federal government to the states. “How do we do that bottom-up?” asked Cadogan, “Where states are giving local communities the power to determine what accountability looks like, but making sure that we have accountability, because parents want to know how their children are doing.”
Reflecting on the qualitative portion of the report, Stewart discouraged the use of binary thinking. “Some things clearly did help and push us forward in ways where the story is being told too little, and some things did not work in other places where, for instance, the implementation might have been the problem or the lack of other types of supports affected it. Having a yes-no binary about whether these things worked or not leaves the public confused and nihilistic about trying anything.”
