Why DC’s Low Graduation Rates?

[Some changes to the conclusions in this post; see edit at the end and entry 21 on Mistakes page]

US News: DC Schools Brace For Catastrophic Drop In Graduation Rates. “Catastrophic” isn’t hyperbole; the numbers are expected to drop from 73% (close to the national average of 83%) all the way down to 42%.

There’s no debate about why this is happening – it’s because the previous graduation rate was basically fraudulent, inflated by pressure to show that recent “reforms” were working. Last year there was a big investigation, all the investigators agreed it was fraudulent, DC agreed to do a little less fraud this year, and this is the result. It’s pretty damning, given how everybody was praising the reforms and holding them up as a national model and saying this proved that Tough But Fair Education Policy could make a difference:

As far as scandals in the education policy world go, D.C. schools so profoundly miscalculating graduation rates at a time when the high-profile school district had been so self-laudatory about its achievements may be difficult to top […] Indeed, when Michelle Rhee took the reins of the flailing school system a decade ago, it galvanized the education reform movement, which had just begun blossoming around the country, and ushered in a host of controversial changes that included the shuttering of multiple schools, firing of hundreds of teachers and the institution of new teacher evaluation and compensation models.

The changes not only dramatically altered the local political landscape in Washington but also shined a national spotlight on D.C. schools that prompted other urban school districts and education policy researchers to consider the nation’s capital a bellwether for the entire education reform movement.

Well, darn.

But the interesting bit isn’t just that DC schools are doing worse than we thought. It’s that DC schools are doing amazingly, uniquely, abysmally bad, below what should even be possible. We make fun of states like Mississippi and Alabama, but both have graduation rates around 80%. The lowest graduation rate in any of the fifty states is in Oregon, which still has 69%. And we are being told DC is 42%!

When we discussed this in the last links thread, people had a couple of explanations:

1. Washington DC has a terrible school system, with uniquely incompetent administrators.

2. Washington DC is poorer, blacker, and more segregated than any other state, and that leads to unique challenges other school systems don’t face. Even though everyone is doing their best, they face insurmountable structural difficulties.

3. Maybe the fraud was so bad that DC over-corrected, and now has stricter standards than anywhere else.

Which of these is most important?

Let’s start by looking at test scores. Here’s a sample of DC’s NAEP scores compared to some other states (full list is here).

In both reading and math, in all grades, DC does abysmally. But they don’t do as abysmally as a 42% graduation rate might predict. They are sometimes last, sometimes second- or third- to last, and in any case they’re rarely that different from other low-performers like Alabama and Mississippi. They might be best described as a member in good standing of the lowest-performing tier of US states.

But there are massive racial inequalities in education, and DC is by far the blackest “state” in the nation. How does it do when we adjust for this?

The full table is here. DC has by far the highest white test scores in the country – probably because a lot of its white students are the kids of well-off bureaucrats and think-tank types. And its black test scores range from lowest-tier-member to mediocre.

Nor can this just be a Simpson’s Paradox, where both white and black students do fine but the difference is driven by a greater number of black students. The US average black graduation rate is 68%. No state has a black graduation rate lower than 56%. DC – again – is supposed to have a 42% total graduation rate.

This seems to refute hypotheses 1 and 2 – that DC just has a terrible school system, or just has an unusually disadvantaged population. Its white students do very well. Its black students do poorly but not too much worse than they would in other states. There is no gap – among either race or among both combined – that corresponds to the gap between Mississippi’s graduation rate (total 80%, black-only 77%), and DC’s graduation rate of 42%.

This leaves us with hypothesis 3 – that DC got burned so badly in the fraud investigations that its standards are now much higher than anywhere else’s. Maybe a graph will help:

The horizontal axis is each state’s test scores – specifically the average of its 8th grade NAEP reading and math. The vertical axis is its graduation rate. The previous, supposedly fraudulent DC rate clusters together more or less with everybody else. The new, supposedly non-fraudulent rate is an extreme outlier, showing a graduation rate about 20 percentage points lower than test scores would predict.

So DC’s old graduation rate was normal relative to their test scores, and their new graduation rate is an outlier. But their old graduation rate is widely considered to have been maintained by fraud and really low standards, and their new graduation rate is widely considered correct. Does that mean that everywhere else with the same levels of poverty and segregation as DC also uses fraud and really low standards to keep their graduation rates up?

Maybe. Detroit is often used as a symbol of inner-city educational dysfunction, but even the district with the worst Detroit schools has a 61.5% graduation rate. How do they do it? Given that fewer than 5% of their students pass exams, I assume they do it through fraud and really low standards. Los Angeles? Fraud and really low standards. Chicago? Fraud and really low standards. Baltimore? Given stories like the one where one of the city’s highest-graduation-rate schools has zero percent of students score at “meets expectations” or even “approaches expectations” on statewide exams, it looks like fraud and really low standards.

I understand this is a really strong claim. But others seem to agree, and it’s the only way I can make sense of DC’s abysmally low projected graduation rates, in the context of their merely-awful exam scores.

Or maybe the key word is “projected”. From the US News article:

The report estimated that just 42 percent of seniors are on track to graduate at the end of the current school year, down from 73 percent who graduated in the 2016-17 school year. It noted that 19 percent of students were “moderately off-track” and could still earn enough credits to graduate.

Plausibly, kindly school officials will find loopholes that allow all of those 19% of moderately-off-track children to graduate, DC’s actual graduation rate will be 62% (right on the trend line for the graph above), and this whole episode will be remembered as “that one time a scary report underestimating the graduation rate came out”. Plausibly, this is the main way that Detroit and Los Angeles and all those other cities keep their rates up, and the more dramatic scare stories are just that.

I hope this happens. Think how unfair it will be for DC students if it doesn’t. Somebody who would graduate comfortably from any other high school in the country will be held back because they happen to have been in DC the year it decided to enforce standards nobody else enforced. If the true value of education is signaling, then the most important thing a school district can do is make sure it’s speaking the same signaling-language as everyone else. Probably somebody should fix the system in general, but that needs to happen on a national level if it’s not going to leave thousands of unfairly-failed children as collateral damage.

[EDIT: More discussion at Highlights From The Comments On DC Graduation Rates. Main update is that I underestimated the importance of absences, which are what’s causing a lot of the non-graduations, which there might be more of in DC, and which DC might be stricter about than other areas.]