What’s the easiest way to a secondary Ofsted Outstanding?

Secondary school leaders would like to think that they are judged on the difference they make and not on the pure outcomes of their school irrespective of context.  The whole language of the most recent framework is about progress made, taking into account pupils and schools various starting points.  While there has to be regard to national average attainments, there is a general sense that Ofsted inspectors try to take into account starting points when judging a school.

But are they succeeding?  For example, do those schools which have low prior-attainment intakes have that taken into account properly?  Do those with very able intakes get the appropriate challenge from Ofsted?  In a previous post, focusing on Grammar Schools and Secondary Moderns, the evidence seemed to be saying no.  It was unusual that Secondary Moderns had a very wide spread of prior attainment, so I wondered whether Ofsted grades might be linked not just to the selectivity of intake, but to the prior attainment of intake.

Simply, yes!  There is a high degree of correlation between a secondary school’s average KS2 points score (for the Y11 cohort in 2013) and the Ofsted grade most recently awarded.  Here’s what it looks like …

Ofsted Grade by Prior Attainment as of 30 April 2014

A few notes of explanation:

• These are the 2,684 secondary schools in England with both a KS2 prior attainment score and a current Ofsted grade.
• 85% of these schools have a KS2 average point score (APS) of higher than 26, but lower than 30 i.e. the four columns labelled 26, 27, 28, 29 in the chart
• The number of schools in each APS “bucket” is shown at the top of the bar so there are 748 schools with KS2 APS of 27 or higher, but less than 28.

You can see that there is a far higher likelihood of a school with higher attaining pupils getting an Ofsted Outstanding than there is of a school with low attaining pupils from primary school.  For example, look at the 462 schools in the KS2 APS=26-27 band. Just 9% of those schools are outstanding.  By contrast, 48% of the 133 schools in the KS2 APS=29-30 band are outstanding.  Those schools are five times more likely to be Outstanding.  The figures for Good or Outstanding for those two are almost as stark: 91% versus 54%.  The schools with the higher attaining intake are nearly twice as likely to be Good or Outstanding.

What about RI & Inadequate schools?  46% of the APS=26-27 band are in Ofsted grades 3 or 4, as opposed to 9% of those in APS=29-30 band (none of which are Inadequate).  The schools with the lower intake are five times as likely to be RI or Inadequate.

So what is the explanation?  Well, here are a few pre-digested ones to pick from:

1. It’s just a coincidence.  Correlation doesn’t prove causation.  Don’t get so worked up!  (I don’t find this convincing, do you?)
2. Schools with able intakes attract the best teachers and leaders so it is no surprise that they are more likely to be graded better.  (Are you bristling at this?)
3. Schools with able intakes are more likely to have pupils with the middle class values that prize education, making it easier to demonstrate the sorts of things inspectors want: good behaviour, strong progress, a thirst for knowledge.
4. Prior attainment is not the only variable that is different between schools with different prior attainment intakes.  Something else varies across schools of different prior attainment bands which influences school quality – this is known as a confounding variable.  (What could that be?  Social class? Something else?)
5. The Ofsted framework is biased against secondary schools with weaker intakes and expects more of them. (I’m not suggesting it is biased by design, just by implementation).
6. Inspectors cannot help being seduced by the Halo Effect and don’t expect enough from schools with more able intakes because they have such lovely, articulate and well-socialised pupils.

Perhaps you can think of some more.  My favourites are 5 and 6, though I think that it is hard to be too upset by number 5.  If schools are to address disadvantage, that’s just how it has to be.  However, that therefore means the inspection process is biased and inherently unfair on some schools, their teachers and leaders.  Ofsted do seem to expect more of those who have less, in a kind of anti-Matthew effect.  The longer term consequences of this for the schools affected aren’t good.

Wait! I know some will cry. Ofsted put a new framework in at the start of 2013, so it is not fair to use historic data on inspection, particularly when you are mixing it with 2013 KS2 APS scores.  Yes, you’re right.  Here’s the same chart just with Ofsted grades awarded since 1 September 2013.

Ofsted Grade by Prior Attainment since 1 September 2013

Not much difference.  Overall a bit harsher, and the prior attainment effect is slightly weaker, but still strong.  Band 26-27 schools are half as likely to be Outstanding and three times as likely to be RI or Inadequate as the band 29-30 schools.

Oh, what how about the slight amendments since 1 January 2014?  You should really look at those.  OK, here they are.

Ofsted Grade by Prior Attainment since 1 January 2014

We’re down to only 242 schools now, and my two favourite bands comprise 57 and 20 schools respectively. With these recent inspections, band 26-27 schools are one-seventh as likely to be Outstanding and twice as likely to be RI or Inadequate as the band 29-30 schools.  That’s a big difference.

All these bands are complicated, so let’s split schools into two halves in terms of their last year’s Y11 Key Stage 2 point scores (chart below).  If you were lucky enough to have “clever” Year 6 pupils join you in 2008 who gained their GCSEs last year, your school would be 3.5 times more likely to have an Outstanding grade (34% vs. 10%) compared to schools with KS2 APS below the median.  It would also be less than half as likely to be an RI or Inadequate school (19% vs. 43%).

Ofsted Grade by Prior Attainment as of 30 April 2014

So, it is easy.  If you want your secondary school to get an Outstanding Ofsted grade and you want to avoid RI or Inadequate, make sure your pupils’ previous attainment on intake is as high as possible.

Depressingly, this is our accountability system.  This is why it is getting difficult to recruit good people to struggling schools.  The odds are against them in more ways than the obvious ones.

As an organisation Ofsted has encouraged us to self-evaluate for many years, but how far is it evaluating its own biases?  Have Ofsted looked at themselves like this?  If not why not, given that they employ an Inspection Data & Insight Team of 100 staff?   I’ll let you know if I get an answer.

Ofsted Grade Predictor

Meanwhile, cheer yourself up by finding out the probability of your school getting each grade by using this handy Ofsted Grade Predictor! I took the unchangeable data (KS2 APS, religious nature and gender basis) of schools and modelled the Ofsted grades using those as predictors. Of course, inspections are too complicated to predict from just those items (I purposely didn’t use outcome data which any inspector would), but really, the point is none of these should be expected to influence inspection grades? Play around to see what you have to do to get the grade you want … or fear.

Technical notes:

The 2013 KS2 APS came from the DfE performance tables website  – data download area

The most recent Ofsted grades came from Ofsted’s Monthly Management Information area

All the schools with no inspection data (mainly recently converted academies) were removed from the dataset.  This left 2,684 secondary schools with both a KS2 APS score available and an Ofsted grade.  697 of these had been inspected since 1st September 2013, and 242 of those since 1st January 2014.

I’m happy to send you the R code I used to create the charts if you drop me a line.