Understanding the New NYS Accountability Measures

Part 2 of 3  |  How will your school be evaluated?

DR. ROBERTA LENGER KANG
Center Director, CPET

A few months ago, I shared my initial take on New York State’s new accountability measures, including a breakdown of the two biggest changes to accountability: new school designations, and new criteria used to calculate what “success” looks like. Shortly after, schools received their new ESSA designations, which surfaced even more questions about the updated standards for NYS.

Most schools I’ve talked with have been relieved to get a notice that they’re now a school “in good standing.” And for many schools that have been fighting the Focus or Priority label for the last few years, it must feel great to finally get a fresh start after years of striving. But these new rankings also prompt a few questions: How were the new designations determined? How is the new evaluation system structured?

These are important questions because while the state has essentially given schools a free restart when it comes to accountability, this year’s data will be used to determine next year’s rating within the new paradigm, and that doesn’t leave very much time for strategic planning.

---

A Paradigm Shift: how the new designations have been determined

NCLB measures effectiveness...	ESSA measures effectiveness...
Based on hitting specific targets: measuring Adequate Yearly Progress (AYP) toward fixed goals Based on growth over time: offering Safe Harbor for schools who show 10% growth each year	Based on comparison with other schools: measuring performance by ranked school data across six indicators; averages specific data points into generalized levels

Over the last six months, New York State used all of last year’s school data to restructure its evaluation system, expanding the number of measures -- from three, to six -- that it will use to hold schools accountable to short and long-term goals. At the same time, the state has also fundamentally shifted what it means for a school to be “successful”.

Under No Child Left Behind (NCLB), schools were evaluated on their ability to meet Adequate Yearly Progress (commonly known as AYP), based on the school’s proximity to meeting a set of fixed benchmarks for performance on English and Math State tests, and graduation rate. However, these fixed and rigid goals quickly became unattainable for most schools, so the State implemented the concept of Safe Harbor. Safe Harbor allowed schools who did not meet the fixed standard to avoid accountability consequences if they were able to increase scores by 10% each school year. This system placed the emphasis of effectiveness on how a school grows year after year, compared with its own past performance.

Conversely, ESSA accountability does not measure schools against a fixed and rigid set of national performance targets, nor does it measure schools against their own individual performance over time. Under this new system, schools will now be evaluated on their ability to meet short- and long-term goals for student performance in the four core content areas, graduation, attendance, and college, career, and civic readiness.

ESSA brings a paradigm shift to New York State school accountability that will determine effectiveness using a series of bell curve metrics that will ultimately compare data and rank order schools across the state based almost entirely on student performance on state tests. That is to say, student population, demographics, past growth, enrollment of Special Education or English Language Learners, economic status or other criteria will not be calculated or considered mitigating factors in how a school is evaluated by the State.

The state’s new system is a complex web of data points and metrics that will eventually rank-order schools across the state. Using data from the 2017-18 school year, the state reset their expectations for school performance and applied the new metrics for evaluation. Then, they rank ordered every school in the state based on these new criteria. The lowest 5% of schools in this statewide ranking were identified for “Comprehensive School Improvement” (CSI). The other 95% of schools are identified as being in Good Standing -- for now.

Understanding this new trend helps us to understand how hundreds of schools across the state went from being a Focus or Priority school to a school in Good Standing, virtually overnight.

Bottom line: Under NCLB, schools had to be better than they were the year before in order to mark effective progress. Under ESSA, schools will now be competing with other schools across the state, within a series of complex metrics.

---

New Indicators and Metrics: how the new system is structured

When we begin to explore the details of this new evaluation system, things get very complicated, very quickly. ESSA doubles the number of accountability measures compared to NCLB. Even though both systems rely heavily on student performance on state tests, ESSA will use the same data points across indicators.

NCLB data points	ESSA data points
ELA State test Math State test Graduation rate	Composite performance Student growth or graduation rate Progress of ELLs Academic progress in ELA & Math Chronic absenteeism College, career and civic readiness (HS only)

With a focus on ranking school performance, this evaluation system identifies a wide range of data sets and metrics, but then aggregates all of the data into four performance levels. As a result, the State will equate performance in one area with the performance in another, even if they’re comparing the same data set by using two different metrics, or if they’re comparing two completely different types of data with completely different metrics.

For some indicators, the levels are an identification of rank-order percentile brackets (ex: Level 1 = lowest 10%). For other indicators, the levels are based on meeting or falling short of State Goals (ex: Level 1 = Did not meet goal) or it might be the result of student performance on the exam (ex: Level 1 = 0-64 on state test), or it might be the result of the leveling Performance Indices (ex: Level 1 = PI 23 - 118). So the State has identified 5-6 criteria to evaluate schools, and each criteria has its own unique way of measuring student and school performance. After each indicator has been evaluated, the findings will be translated into 1 of 4 performance levels, and those outcomes will end up being the determinations of future Accountability Status.

Below, we’ve outlined ESSA’s six accountability indicators, which schools they apply to, what data informs them, how that data is being evaluated, and what the associated ratings are.

Composite performance (elementary & middle)

LEVEL: Elementary and middle school

DATA SET: State test performance in ELA, Math & Science

METRIC: Average Performance Index of the Weighted Average Achievement Index and the Core Subject Performance Index

FINAL RATING: Level 1-4

composite performance (high school)

LEVEL: High school

DATA SET: Weighted State test performance in ELA (x3), Math (x3), Science (x2), and Social Studies(x1)

METRIC: Average Performance Index across all four weighted content areas

FINAL RATING: Level 1-4

STUDENT GROWTH

LEVEL: Elementary & middle school

DATA SET: State test performance in ELA and Math

METRIC: Compares student scores to the scores of similar students in prior years

FINAL RATING: Level 1-4

graduation rate

LEVEL: High school

DATA SET: 4, 5, and 6 year cohort graduation rate

METRIC: Compares graduation rate against long-term and short-term targets

FINAL RATING: Level 1-4

Progress of english language learners

LEVEL: Elementary, middle, and high school

DATA SET: Student performance on the NYSESLAT

METRIC: Increase scores between .75 and 1.25 quartile depending on ELL Level

FINAL RATING: Level 1-4

academic progress in ela & math

LEVEL: Elementary, middle, and high school

DATA SET: ELA and Math State test performance

METRIC: Measures progress on State tests against long and short-term targets

FINAL RATING: Level 1-4

chronic absenteeism

LEVEL: Elementary, middle, and high school

DATA SET: Attendance

METRIC: Measures students who miss 10% or more days of instruction

FINAL RATING: Level 1-4

college, career, and civic readiness

LEVEL: High school

DATA SET: Diplomas, credentials, advanced courses, CTE Certification, etc

METRIC: Measures the percentage of students in a cohort with the designations

FINAL RATING: Level 1-4

---

What does this mean for schools?

Compared with NCLB, the new ESSA system is complex! With double the number of indicators and data points analyzed with different types of measurements, school and district leaders will need to think critically about what they can do to ensure their schools are hitting the right targets. How can you become a school in good standing? How can you stay a school in good standing? These are the questions we’ll explore next time, as we outline the tipping points between being a school in good standing, a school identified as Target Support for Improvement (TSI) or as Comprehensive Support for Improvement (CSI).

In the meantime, consider joining us at Teachers College as we delve deeper into the new state accountability standards, and support leaders in making strategic plans for their schools. Details on our upcoming sessions can be found here! You can also reach out to me with questions at kang@tc.edu.