Exam results: Disruption to education has not gone away

The numbers don’t lie. Pass rates are down and the attainment gap is up once again – crucially, the gap has grown so much that it is now bigger than it was in the years before the pandemic.

Read more: Exam results: How much worse would they have been?

By comparing the latest results to 2019, the government has been able to claim that although pass rates have fallen, the attainment gap is still lower than pre-pandemic levels. What many people do not realise, however, is that the divide between pupils from the richest and poorest areas had been growing every year – so while 2023 figures are indeed lower than those from 2019, they are actually higher than the three preceding years.

Education researcher Barry Black spent results day crunching the numbers. He compared his findings to data from the past seven years, going all the way back to 2016 – the first set of results after the ‘judge me on my record’ speech.

“The claim that the poverty-related attainment gap is narrower now than ‘pre-pandemic’ cannot be backed up by the evidence,” says Black. “In fact, the attainment gap increased each year from 2016-2019 for Highers and National 5. The Higher gap was actually at its highest levels since the full introduction of the new Higher qualifications.”

He also rejects the notion that the government was on track to close the gap until the emergence of Covid.

“What we see in the data is a stagnant – and significant - attainment gap between the richest and poorest pupils. It is also false to say that there was progress towards closing it before the pandemic, and that the pandemic set that back. The evidence clearly shows that is not the case, and the claim should stop being made.”

All of which paints a fairly grim picture, at least so far as the SNP’s credibility is concerned. Put simply, we can see a critical attainment gap growing instead of shrinking, so much so that it now exceeds pre-pandemic years – and that’s even after applying what the SQA has referred to as a “sensitive approach” to grading.

According to the exams body, this meant adjusting some grade boundaries (and therefore manually boosting results) where performance wasn’t at the levels expected, but specific data on the impact of their method is not available. During the media briefing on exam results day, SQA officials were asked if they could quantify the impact of their “sensitive approach” to grading – in essence, to explain how bad results could have been without their interventions – but they confirmed that this was not possible.

Read more: Exam results: Are we closing the attainment gap?

That lack of data, and the problems it could be concealing, is a particular problem for Black, who said that it is “concerning that the SQA have not been able to set out the exact impact” of its approach.

“We simply then cannot know its effectiveness in mitigating some of the COVID disruption for pupils who sat exams this year,” he argued. “Nor are we able to assess what may happen when there are no such mitigations next year.”

“The government hasn't done nearly enough to assess and to understand the impact that COVID has had.”

What happens next year is now the big question.

Indications so far are that education authorities want to see an effective return to normal.

Course adjustments – such as reductions in the coursework demands for some subjects – are to be removed; given that these changes were made to lighten the load on teachers and students, it seems uncontroversial to assume that their abolition will have the opposite effect.

Read more: Exam results explained: How we got here and what's next?

The Scottish Secondary Teachers Association immediately labelled that decision a “bad news story for all secondary school teachers and the young people they teach” and argued that it was “more about SQA taking back control and cementing a place for itself in the developing education landscape.” It has also threatened the nuclear option of boycotting the exams themselves.

On top of that, we do not yet know what sort of appeals system, if any, will be available, nor do we know whether grading processes will be adjusted to account for any ongoing Covid-related impacts.

We do, however, know that if Covid is effectively, but prematurely, declared over in an educational sense then results are likely to get worse. If that happens, further increases in the attainment gap would be inevitable. That will look bad, statistically, but it will also have an enormous impact on the lives of thousands and thousands of mostly working class young people.

“It is clear that the disruption to education has not - and cannot - go away for the young people at school during the period, so it stands to reason that there will be a negative impact when the mitigations end.”

There is another, particularly uncomfortable, question raised by the data: in making adjustments to exam grading – both in order to protect pupils and to ensure that overall outcomes were remotely palatable – have we perhaps, entirely unintentionally, and even obscured some of the full impact of the pandemic on Scotland’s young people?

Black says that changes to course and marking approaches “were of course necessary once the decision was made to return to the exam system as was before the pandemic”, but points once more to lack of hard data on the impact of those adjustments as both “frustrating” and “a concern.”

What’s more, the failure to quantify to full impact of the Covid exam mitigations makes it near impossible to be confident about plans for the coming year. For Black, however, the concerns of teachers are paramount.

“When considering whether adjustments should continue next year, the evidence shows that teachers – the experts in all of this – think they should. Only 15% of SSTA members surveyed saw the reintroduction of pre-Covid assessment as a positive at Higher level. EIS members have also raised concerns on the impact. We should listen to them.”