Mind your gaps

The performance of vulnerable groups in schools has been a concern for some time, but particularly since the introduction of the pupil premium.  Gaps between attainment & performance of the disadvantaged and others even form part of the inspection framework, so in my school we’ve focused a lot on closing these gaps.  Now is the time of year when we are looking forward to what we want our Year 11 and Year 10 students to achieve so I’ve been looking at the national picture of attainment & performance gaps.  How does your school measure up?

It’s tricky to get a good feel for this.  The government want to keep it simple, but as is often the case, have made it too simple by just comparing everything to a national average.  Here are the averages from the most recent performance tables for 2013 for the key indicators:

National averages for gaps in key indicators – all secondary schools, 2013 performance tables
Indicator All pupils Disadvantaged Others The Gap
5 A*-C inc Eng + Maths 60.6% 41.0% 67.9% -26.9%
Expected Progress – English 70.4% 56.6% 75.4% -18.8%
Expected Progress – Maths 70.8% 54.1% 76.8% -22.7%

Does the gap vary depending on the level of pupil premium in the school?  Well, it does seem to, as you can see from this screen shot of the Swarm.  To enlarge, just click the chart.

The gap between the disadvantaged & others vs proportion of disadvantaged (AC5EM, 2013)

The gap between the disadvantaged & others vs proportion of disadvantaged (AC5EM, 2013)

Going up along the y-axis is the gap between pupil premium and others for 5A*-C including English and Maths in 2013.  The smaller the gap, the higher up the y-axis.

Going along the x-axis is the proportion of pupil premium students in Y11.  Each dot is a school.  The horizontal dotted red line is the average gap and the vertical dotted red line is the average proportion of pupil premium students.

Looking at this I get three messages.

Message 1: there is a massive variability in the gaps at individual schools between attainment of 5A*-C with English & Maths.  It is possible some of this is to do with the variability of the pupil premium cohort itself in each school, about which there is no further information in the tables.  For example, if your pupil premium students were quite able at Y6, you would appear to have a very small gap without having done much and vice versa.  However, the variability is so huge that it seems at least plausible some of it is down to schools’ good or poor work.  The picture is similar for expected progress in English and in Maths.

Message 2: the variability is higher for schools with low proportion of pupil premium students in Year 11 and gets smaller as the proportion of pupil premium students rises.  This means you need to compare yourself with “similar” schools for pupil premium proportions if you are to compare yourself reliably. The picture is again similar for expected progress in English and in Maths.

Message 3: schools with a larger proportion of pupil premium students in Year 11 seem to do better than those with a smaller proportion, although there is a lot of variability.  This is another reason to compare your school with schools who have a similar proportion of pupil premium students in Year 11.  The picture is similar for expected progress in Maths, but less pronounced for expected progress in English.

How good is my gap?

Rough and Ready method

It might be enough for you to get a general idea of whether you’re doing well or not.  If this is what you want:

  1. Go to the Swarm which currently shows 2013 data
  2. Towards the top left choose for y-data “FSM gap 5A*-C +EM”
  3. Also choose as x-data “%FSM/CLA in KS4”
  4. Where it says “Search for a school name” type in the first few letters of your school e.g. “Bash St” for Bash Street School
  5. Your school will appear as a fat coloured disc. If you hover it will show some school details and the gap figure for your school appears at the left in a white rectangle.
  6. Then judge whether your school is above the general trend or below it.  Above is good, below is bad.  The horizontal dotted red line gives the average gap for all schools of -26.9%.
  7. You can repeat the exercise for expected progress in English and Maths by choosing different y-data: “FSM gap Eng Prog” or “FSM gap Ma Prog”

Making it quantitative

You may prefer to know how you place compared to all secondary schools – top 20%, bottom 10% and so on.  In that case, consult the table below.

Centiles for gaps in key indicators – all secondary schools, 2013 performance tables
centile (2013 Performance data) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
% 5 A*-C + EM gap -70 -38 -33 -29 -26 -23 -20 -17 -13 -7 22
% expected progress English gap -61 -31 -26 -22 -19 -16 -13 -11 -7 -2 50
% expected progress Maths gap -71 -34 -28 -25 -22 -19 -16 -13 -10 -5 24

Use it like this.  Let’s say your gap between the disadvantaged and others for 5A*-C including English and Maths was -30%.  Remember this is the difference between the figure for the pupil premium group and the non-pupil premium group, not just the difference between pupil premium students and the overall figure.  Then look up from the left until you get to the smallest figure ABOVE yours.  In this case it would be -29 in the “30%” column.  That means 30% of all schools have -29 or worse, so your school is in the bottom 30% of all schools (but not in the bottom 20%).  If your school figure was -15, your school would be higher than the 70% figure but less than the 80% figure so it would be in the top 30% of all schools.

As a final example, you would have to have a difference between pupil premium students and others of better than -2% to be in the top 10% of all schools for the gap in Expected Progress in English.  If your gap in Expected Progress in English was worse than -31% you would be in the bottom 10% of all schools for the gap.

By the way, notice where the national average gets you.  If you hit the national average of -26.9% for the 5 A*-C +EM figure you score in the bottom 40%.  National average figures for expected progress in English and Maths give you bottom 50% and bottom 40%.  This is because of skewed distributions.  The very large gaps of a few schools at the bottom drag the mean gap below the middle 50% school.  If you remember your O-level statistics, the mean happens to be below the median.

Quantitative AND right

The problem is that “all” schools aren’t “similar” schools and given how the gap changes with pupil premium proportion you really ought to use just schools of a similar pupil premium proportion.

I looked at the gaps for 2,940 secondary schools from the 2013 performance tables who had both a published figure for pupil premium proportion in KS4 and a published figure for the relevant gap.  Then I divided the schools into ten bands (deciles) each with 294 schools in them according to the proportion of pupil premium students in Year 11.  For each of these deciles, I analysed the range of gaps to find out what the top 10%, the next 10% and so on to the bottom 10% performance would look like.  Here are the tables first.  Let’s begin with 5A*-C including English & Maths, and explain it before moving to the others.

Deciles for the gaps in % 5A*-C including English & Maths (2013 performance table figures)
Y11 PP deciles 1st decile 2nd decile 3rd decile 4th decile 5th decile 6th decile 7th decile 8th decile 9th decile 10th decile
Y11 pupil premium% 3-9.9% 10-12.9% 13-15.9% 16-19.9% 20-23.9% 24-27.9% 28-34.9% 35-41.9% 42-52.9% 53-92%
Top 10%  +1  -10  -13  -14  -11  -13  -11  -9  -7  0
Top 20%  0  -15  -17  -17  -16  -17  -14  -15  -12  -6
Top 30%  -8  -20  -21  -21  -20  -20  -18  -18  -14  -9
Top 40%  -14  -24  -25  -25  -22  -22  -21  -21  -16  -12
Top 50%  -20  -28  -28  -28  -25  -25  -23  -23  -19  -14
Top 60%  -25  -31  -31  -31  -27  -28  -26  -25  -21  -17
Top 70%  -31  -34  -34  -33  -30  -30  -28  -27  -24  -20
Top 80%  -36  -37  -37  -36  -34  -33  -31  -31  -27  -23
Top 90%  -44  -42  -42  -41  -37  -38  -35  -35  -30  -28
Top 100%  -70  -69  -59  -60  -52  -50  -52  -44  -39  -40

How to use the tables

Start with your own pupil premium proportion in Year 11.  Let’s say you had 32 students out of 170 – that would be 18.8%, so using the row marked “Y11 pupil premium %” you find your school was in the 4th decile for pupil premium proportion because 18.8% is between 16% and 19.9%.  That is the column you need to use.  Then use your gap between pupil premium students and “others”.  Let’s say that was -18%.  This is worse  than a gap of -17%, but better than -21%, so you are in the top 30% of similar schools.

Last year, your pupil premium proportion was about the same, so you still use the 4th decile column, but then, the gap was much worse then at -30%.  This is worse than -28% but better than -31% so you were then in the top 60% of similar schools.  An alternative way of looking at it is that you are in the bottom 50% of similar schools because you are worse than the bottom of the top 50% band.

There is a little less variation in the expected progress in English figures, so you may be happier comparing with all schools, however, here are the deciles.

Deciles for the gaps in % making expected progress in English (2013 performance table figures)
Y11 PP deciles 1st decile 2nd decile 3rd decile 4th decile 5th decile 6th decile 7th decile 8th decile 9th decile 10th decile
Y11 pupil premium% 3-9.9% 10-12.9% 13-15.9% 16-19.9% 20-23.9% 24-27.9% 28-34.9% 35-41.9% 42-52.9% 53-92%
Top 10% +3 -1 -5 -5 -4 -5 -4 -5 -2 -2
Top 20% +1 -7 -10 -11 -9 -10 -8 -8 -5 -2.6
Top 30% -5 -11 -13 -14 -13 -13 -12 -12 -9 -5
Top 40% -8 -15 -16 -17 -16 -15 -14 -14 -11 -8
Top 50% -13.5 -19 -18 -19 -18 -18 -16 -16 -13 -10.5
Top 60% -18 -23 -23 -22 -21 -21 -18 -19 -16 -12
Top 70% -22 -27 -26 -25 -24 -24 -21 -22 -17 -15
Top 80% -26 -30 -29 -28 -27 -27 -25 -25 -20 -19
Top 90% -34 -36 -36 -32 -32 -32 -29 -29 -26 -22
Top 100% -52 -61 -61 -51 -49 -52 -42 -40 -54 -43

The expected progress figure for Maths shows more variability.  Depending on which decile you are in for pupil premium proportion a gap of -18% could put you in top 50%, top 40%, bottom 50% or bottom 30%.  Comparing a -18% gap with all schools gave top 50%

Deciles for the gaps in % making expected progress in Maths (2013 performance table figures)
Y11 PP Deciles 1st decile 2nd decile 3rd decile 4th decile 5th decile 6th decile 7th decile 8th decile 9th decile 10th decile
Y11 pupil premium% 3-9.9% 10-12.9% 13-15.9% 16-19.9% 20-23.9% 24-27.9% 28-34.9% 35-41.9% 42-52.9% 53-92%
Top 10% +2 -5 -7.6 -8.6 -7 -8 -7 -9 -6 0
Top 20% +0.4 -11 -13 -13 -13 -13 -11 -15 -10 -5
Top 30% -5 -15 -15 -16 -16 -16 -14 -18 -13 -8
Top 40% -10 -18 -18 -19 -18 -19 -17 -21 -15 -10
Top 50% -13 -22 -22 -22 -21 -22 -20 -23 -17 -13
Top 60% -22 -25 -25 -25 -23 -24 -22 -25 -19 -15
Top 70% -27 -28 -28 -28 -26 -26.6 -25 -27 -20 -18
Top 80% -37 -31 -32 -31 -30 -29 -28 -31 -23 -21
Top 90% -52 -38 -38 -37 -34 -34 -32.4 -35 -27 -25
Top 100% -70 -71 -59 -52 -44 -49 -50 -44 -37 -55
Advertisements

One thought on “Mind your gaps

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s