Part 3 of 3  |  How will NYS accountability components be measured, and how will these measurements determine accountability?

DR. ROBERTA LENGER KANG
Center Director, CPET

​As we near the end of a school year, our focus naturally begins to shift towards understanding our school’s data and making plans for next year, particularly as we sit down to develop the state mandated Comprehensive Education Plan (CEP). This is where ESSA policies begin to hit close to home. 

We’ve already explored the major differences between NCLB and ESSA, and the ways NYS is breaking down accountability into different components. We know that ESSA primarily measures effectiveness by comparing schools with one another, and that the number of accountability measures has doubled under ESSA. Now, the most important thing to understand is how these new components will be measured, and how those measurements determine accountability.  

Statewide rankings and long-term goals

​NYSED has developed two rating methods: statewide rankings, and annual and long-term goals.

Statewide rankings measure the following components:

• Composite Performance Index (CPI)
• Combined CPI & Graduation Rate
• Math & ELA Progress

​
Ranking performance across the state is exactly what it sounds like. In three of the seven components, the state will rank order the performance of every school and then assign a Performance Level on a scale of 1-4. Performance across the state will be ranked without making any considerations for enrollment demographics. Large or small, vocational or specialized, urban, suburban or rural — all schools across the state will be compared and rank ordered by performance. 

​The following components are measured by annual and long-term goals:

• Graduation Rate
• English Language Proficiency
• Chronic Absenteeism
• College and Career Readiness

In addition to ranking school performance, the NYSED has developed a system for statewide growth based on short- and long-term goals. 

Here’s how it works:
For each of the four components list above, NYSED has identified a statewide end goal to represent the ultimate expectations for schools in New York.

Then, using data from 2016 and 2017, the state identified baseline data points for every school. The difference between the Baseline and the End Goal is called the gap.

In order to close the gap between current school performance and the End Goal, the state has identified long-term goals, which require schools to close 20% of the gap within 5 years.

The state divides the Long-Term Goals evenly across the five years to determine annual goals called Measures of Interim Progress or MIPs.

Measures of Interim Progress

MIPs are preset, annual goals that are established in five year sets as they build toward the Long Term Goal. There are two types of MIPs: an individual school MIP aligned to the school’s baseline data, and a statewide MIP, aligned to the state’s baseline data. MIPs require schools to increase 4% every year, starting from their baseline. MIPs do not fluctuate based on year-to-year performance. 

At the end of each year, every school will be measured by their ability to meet their school’s individual MIP, as well as the State MIP. If a school is performing below the statewide average, their school MIP will be their lower goal. If a school is performing above the statewide average, the school MIP will be their higher goal. 

Schools will be assigned a Performance Level 1-4 based on their ability to meet or exceed their MIPs.

• Level 1 = If a school does not meet their school MIP and they do not meet the state MIP
• Level 2 = If a school meets the lower of the two MIPs
• Level 3 = If a school meets both MIPs, or meets their long-term goal 
• Level 4 = If a school exceeds their long-term goal 

Accountability status

In 2018, 95% of New York schools were designated to be in good standing for the 2018-19 school year. Performance data at the end of the 2018-19 school year will be used to determine the accountability status under ESSA’s system for the first time. 

Every ESSA component, whether measured by statewide ranking or MIPs, will receive a rating of Level 1 through 4, and then each of those levels will be used to determine the accountability status of the school. The two accountability pathways are Comprehensive School Improvement (CSI) and Targeted School Improvement (TSI). While the pathway to accountability is the same for CSI and TSI, CSI is reserved for schools who are struggling to meet the needs of the “all students” group, while TSI can be triggered by subgroups (either by ethnicity, students with disabilities, English Language Learners, or economically disadvantaged). 

Following the old saying, “three strikes and you’re out,” these are scenarios that trigger CSI or TSI:

Strategic planning

So what’s next? With a solid understanding of the methods for evaluation, you can begin looking closely at your school’s data, your goals, and where you might be in danger of triggering TSI or CSI. With advanced planning and support, you can use this data to inform your CEP goals, target areas for improvement for next year, and highlight areas for increased professional development.

If you’d like help breaking down your ESSA data, contact us for a free consultation, or consider joining our Exploring ESSA: Unpacking State Accountability online course. 

Part 2 of 3  |  How will your school be evaluated?

DR. ROBERTA LENGER KANG
Center Director, CPET

A few months ago, I shared my initial take on New York State’s new accountability measures, including a breakdown of the two biggest changes to accountability: new school designations, and new criteria used to calculate what “success” looks like. Shortly after, schools received their new ESSA designations, which surfaced even more questions about the updated standards for NYS.

Most schools I’ve talked with have been relieved to get a notice that they’re now a school “in good standing.” And for many schools that have been fighting the Focus or Priority label for the last few years, it must feel great to finally get a fresh start after years of striving. But these new rankings also prompt a few questions: How were the new designations determined? How is the new evaluation system structured?

These are important questions because while the state has essentially given schools a free restart when it comes to accountability, this year’s data will be used to determine next year’s rating within the new paradigm, and that doesn’t leave very much time for strategic planning.

A Paradigm Shift: how the new designations have been determined

Over the last six months, New York State used all of last year’s school data to restructure its evaluation system, expanding the number of measures -- from three, to six -- that it will use to hold schools accountable to short and long-term goals. At the same time, the state has also fundamentally shifted what it means for a school to be “successful”.

Under No Child Left Behind (NCLB), schools were evaluated on their ability to meet Adequate Yearly Progress (commonly known as AYP), based on the school’s proximity to meeting a set of fixed benchmarks for performance on English and Math State tests, and graduation rate. However, these fixed and rigid goals quickly became unattainable for most schools, so the State implemented the concept of Safe Harbor. Safe Harbor allowed schools who did not meet the fixed standard to avoid accountability consequences if they were able to increase scores by 10% each school year. This system placed the emphasis of effectiveness on how a school grows year after year, compared with its own past performance.

Conversely, ESSA accountability does not measure schools against a fixed and rigid set of national performance targets, nor does it measure schools against their own individual performance over time. Under this new system, schools will now be evaluated on their ability to meet short- and long-term goals for student performance in the four core content areas, graduation, attendance, and college, career, and civic readiness.

ESSA brings a paradigm shift to New York State school accountability that will determine effectiveness using a series of bell curve metrics that will ultimately compare data and rank order schools across the state based almost entirely on student performance on state tests. That is to say, student population, demographics, past growth, enrollment of Special Education or English Language Learners, economic status or other criteria will not be calculated or considered mitigating factors in how a school is evaluated by the State.

The state’s new system is a complex web of data points and metrics that will eventually rank-order schools across the state. Using data from the 2017-18 school year, the state reset their expectations for school performance and applied the new metrics for evaluation. Then, they rank ordered every school in the state based on these new criteria. The lowest 5% of schools in this statewide ranking were identified for “Comprehensive School Improvement” (CSI). The other 95% of schools are identified as being in Good Standing -- for now.

Understanding this new trend helps us to understand how hundreds of schools across the state went from being a Focus or Priority school to a school in Good Standing, virtually overnight.

Bottom line: Under NCLB, schools had to be better than they were the year before in order to mark effective progress. Under ESSA, schools will now be competing with other schools across the state, within a series of complex metrics.

New Indicators and Metrics: how the new system is structured

When we begin to explore the details of this new evaluation system, things get very complicated, very quickly. ESSA doubles the number of accountability measures compared to NCLB. Even though both systems rely heavily on student performance on state tests, ESSA will use the same data points across indicators.

With a focus on ranking school performance, this evaluation system identifies a wide range of data sets and metrics, but then aggregates all of the data into four performance levels. As a result, the State will equate performance in one area with the performance in another, even if they’re comparing the same data set by using two different metrics, or if they’re comparing two completely different types of data with completely different metrics.

For some indicators, the levels are an identification of rank-order percentile brackets (ex: Level 1 = lowest 10%). For other indicators, the levels are based on meeting or falling short of State Goals (ex: Level 1 = Did not meet goal) or it might be the result of student performance on the exam (ex: Level 1 = 0-64 on state test), or it might be the result of the leveling Performance Indices (ex: Level 1 = PI 23 - 118). So the State has identified 5-6 criteria to evaluate schools, and each criteria has its own unique way of measuring student and school performance. After each indicator has been evaluated, the findings will be translated into 1 of 4 performance levels, and those outcomes will end up being the determinations of future Accountability Status.

Below, we’ve outlined ESSA’s six accountability indicators, which schools they apply to, what data informs them, how that data is being evaluated, and what the associated ratings are.

What does this mean for schools?

Compared with NCLB, the new ESSA system is complex! With double the number of indicators and data points analyzed with different types of measurements, school and district leaders will need to think critically about what they can do to ensure their schools are hitting the right targets. How can you become a school in good standing? How can you stay a school in good standing? These are the questions we’ll explore next time, as we outline the tipping points between being a school in good standing, a school identified as Target Support for Improvement (TSI) or as Comprehensive Support for Improvement (CSI).

In the meantime, consider joining us at Teachers College as we delve deeper into the new state accountability standards, and support leaders in making strategic plans for their schools. Details on our upcoming sessions can be found here! You can also reach out to me with questions at kang@tc.edu.

Part 1 of 3 | A summary of the biggest adjustments coming to New York schools, including new accountability designations and new metrics to measure school performance. ​

DR. ROBERTA LENGER KANG
Center Director, CPET
​

A few weeks ago, the New York State Education Department released a memo outlining the changes to school accountability measures as a result of the legislation from the 2015 Every Student Succeeds Act, which has replaced the widely known No Child Left Behind Act from 2001.

There are a few people, like myself, who geek out about education policy, and we’ve been waiting for this moment since President Obama signed the Every Student Succeeds Act (ESSA) into law in 2015. No Child Left Behind (NCLB) imposed strict policies and guidelines for states to hold schools accountable for meeting student performance expectations, or face very real consequences. These consequences ushered in what I’ve described as the “Accountability Era” where school leaders, and teachers are held personally responsible for student performance on state tests. Where NCLB established federal expectations for testing and achievement benchmarks, ESSA removes the federal government from the equation and puts states back into the driver’s seat when determining what policies should be enacted in the state. Educators have been diligently watching the news to see how states are determining their priorities and expectations for their schools.

Like many other states, New York has been following an implementation timeline that has held steady on NCLB expectations while working to restructure a state system that policymakers can agree to, with input from the public. Though the NYS ESSA plan was approved earlier this year, there haven’t been many details as to how the new plan would be implemented, or how it might directly affect schools.

That is, until last month, when the NYSED released a set of changes related to ESSA. The 21-page memo is largely a technical manual, outlining the new system for school accountability in the state, new metric systems, and new designations for identifying schools who are below, meeting, or exceeding state expectations. I would never discourage anyone from reading the primary source, but most teachers and school leaders will likely find themselves a bit too busy to pore over all the formulas and definitions outlined there.

If you’re interested in the understanding the changes, but don’t have the time to review the entire memo, we’ve put together a summary of some of the biggest adjustments coming to New York schools, including new accountability designations and new metrics to measure school performance. ​

Change #1: New accountability categories
As of October 2018, NCLB’s categories like Priority School or Focus School have been discontinued, and have been replaced by the categories Comprehensive Support and Improvement (CSI) and Targeted Support and Improvement (TSI). The state explained that schools will receive a new designation before the end of December 2018.

Comprehensive Support and Improvement is specifically for schools that fall in the bottom 5% of performance for the “all students subgroup, or have a graduation rate of 67% or lower.

One notable change is the identification of CSI schools based on how they compare with other schools across the state, rather than marking their progress against their school’s individual goals. Under NCLB, schools were given an accountability ranking based on their ability to meet their own unique target goals. The shift to focusing on a schools within the lowest 5% shifts the focus from an internal benchmarking system to an external benchmarking system where schools only need to be better than those in the lowest 5%. Likewise, schools that have seen major increases in performance, but struggle with the neediest populations, are likely to have a difficult time showing their growth in the shadow of traditionally high performing schools.

Change #2: Indicators for evaluation
During the NCLB era, schools were evaluated on annual test scores in English Language Arts and Mathematics, as well as graduation rates. Under ESSA, New York has broadened the number of measures that will be used for evaluating school performance to include five indicators for elementary and middle schools, and six indicators for high schools.

• Academic achievement in core content area state tests (ELA, Science, Social Studies and Math)  
  • This indicator of school success is called “Composite Performance”, as schools will be evaluated based on student performance across all four core subjects and given a single “composite” score.
• Student growth & academic progress in ELA and Math
  • In elementary and middle school, this indicator of school success is based on comparing current student performance with similar students’ performance in the previous year. This type of comparison is intended to determine if students are achieving at predictable rates based on how similar students performed in the past. The state interprets increases or decreases in student performance based on historical trends as related to teacher and school performance. Additionally, elementary, middle and high school student growth in ELA and Math will be evaluated against new annual “Measures of Interim Progress.” This indicator is called Academic Progress.
• Progress of ELLs towards English proficiency
  • This indicator of school success is based on the progress made by students learning English as a new language, as measured by the NYSESLAT. With this, there is an expectation that students should be able to make progress one level per year, at a minimum.
• Chronic absenteeism
  • Attendance is a key factor for student learning, therefore the state is evaluating schools’ ability to motivate students to attend school with a rate of 90% or higher. NYC schools have been monitoring this data point over the past few years, but this is new to the state.
• College, career & civic readiness (high school only)
  • This indicator measures the percentage of students who leave high school prepared for college or careers. The state will be considering the number of students with Advanced Regents diplomas, CTE certifications, as well as AP or other college level course credits.
• Graduation rate (high school only)
  • One of the longest measures of effective schools, graduation rate, continues to be a major metric for the state. Schools will be measured on their progress towards annual goals, with a baseline expectation of 67% or higher. Schools with graduation rate lower than 67% will automatically be considered a Comprehensive Support and Improvement (CSI) school.

Each of these new areas of measurement taps into a host of new questions about how the state is defining student achievement and teacher performance. While we don’t yet have a full picture of how the evaluations will be structured at the state level, this basic blueprint helps us to identify the new and continuing expectations for school performance.

Coming soon: we’ll delve into additional components of this legislation, including the exploration of the state’s approach to setting annual and long-term goals, as well as new formulas for determining a school’s performance index. Stay tuned!

For more support with strategic planning in response to ESSA, connect with us at cpet@tc.edu / 212.678.3161, or join Dr. Kang for an upcoming session of Exploring ESSA. 
​