School League Tables: Not enough data to judge a school

But Lindsay Paterson, Emeritus Professor of Education Policy, School of Social and Political Science at the University of Edinburgh, said inconsistent exam delivery creates another problem: “You cannot place any faith on one year’s worth of data,” Prof Paterson said.

It sounds dire, but this doesn’t mean that the data from this year’s Herald School League Tables is useless.

School League Tables: How has your child's school placed? 

Measuring student attainment, alongside key demographic data, provides a snapshot of how schools performed last year.

But, as many critics of the league table format have argued, the numbers need to be put in the right context. 

Comparing schools with vastly different demographics – assigning labels of “good” or “bad” – aren’t possible off the back of one year’s results.

Prof Paterson explained why we need to be careful what conclusions we draw from the statistics, and how looking at change over time gives us better insight into what’s really happening in schools.

And, perhaps, it provides some hope for the future.

According to data from the Scottish Government, the average secondary school size is between 600-700 pupils. That leaves roughly 100 pupils per year, and far less at rural schools.

Prof Paterson said that a sample size of 100 or below means that random chance and outliers can wreak havoc on the data.

“That’s an enormous amount of random variation from year to year.

“Nobody would ever dream about doing an opinion poll on the basis of 100 people.

“Now if you look at change over a longer period of time and you see a consistent downward or upward trajectory, then you’re on firmer ground.”

But taking change over time into account will prove difficult this year. Attainment data is complicated by the different systems employed since 2020.

Exams in the first years of the pandemic were cancelled outright; in 2020/2021, they were replaced with a system more reliant on teacher judgment; the 2021/2022 school year saw the return of national SQA exams, but with a tweaked appeals system which factored in coursework over the year – as well as a cloud of doubt cast by the fact that the Scottish Government announced it would eventually replace the SQA altogether.

READ MORE: Why Scotland's exam system is due a massive overhaul

What we need, Prof Paterson said, is a long view. Taking a decade or more into account allows for more accurate conclusions, which can start to tell us the full story behind each school's attainment statistics.  

One of the key metrics for placing each school in its context is the measure of deprivation among its population.

The Scottish Index of Multiple Deprivation (SIMD) measures the socio-economic makeup at every Scottish school.

The Herald League Tables include data on what percentage of at each school pupils live in SIMD1 postcodes - the 20 per cent most deprived in Scotland.

Prof Paterson has researched trends in Scottish schools for decades, and he said this number shouldn’t be overlooked.

“The gross differences between schools are largely due to the social circumstances of their catchment areas,” Prof Paterson said.

Students with more resources and more educated parents at home tend to score higher on exams.

But families in lower socioeconomic brackets aren't at fault, he said.

“The child in a poor home, who doesn’t have a room of her own, doesn’t have a laptop of her own, doesn’t have fast broadband or access to books – that child, through no fault of her parents or her own, does not have the same opportunities as the child who can afford all these things.”

But data over the last decade provides a glimmer of hope.

“The key reason for optimism is that that inequality has lessened over time,” he said. “At any given moment in time, there are big inequalities, but the inequalities have reduced and that’s important.”

According to recent statistics from the Scottish Government, the attainment gap is steadily closing.

A report showed that a record 93.5 per cent of Scottish pupils moved on to positive destinations – meaning further education, training, work, volunteering or other suitable next steps – within nine months of leaving secondary school.

Within that dataset, the government reported that the gap between students from the most and least deprived areas reached a record low of seven per cent.

As recently as 2009, that number was above 18 per cent.

Education Secretary Jenny Gilruth called the trend “greatly encouraging”.

“These figures are testament to the hard work of learners across the country.

School League Tables: How has your child's school placed? 

"We know the pandemic had a significant impact on our education system, but Scotland’s teachers and support staff are clearly preparing young learners for the next stage in life very well, whether that be work or further study.

"This Government has prioritised tackling the gap in achievement between the most and least deprived areas, so I am heartened to see this gap continuing to close.

"As we start to embark on a programme of education reform in the months ahead, it is clear that we will be building on a very strong platform."

Full data on where school leaver destinations after the 2021/2022 school year can be found online.