Scotland’s national literacy and numeracy statistics are based on something called Achievement of Curriculum for Excellence Levels (ACEL) data, which probably sounds a bit more vague and unhelpful than it actually the case – although not by much.

ACEL data is gathered for pupils at four points during their time at school, each aligned with a particular Curriculum for Excellence (CfE) level: primary 1 pupils are generally expected to have achieved the Early Level; most primary 4 pupils will successfully achieve Level 1; the majority will achieve Level 2 by the end of primary 7; and we broadly expect to see S3 pupils complete Level 3, with some also going on to Level 4.

Information is submitted for pupils tracking their progress in literacy and numeracy, although the former is calculated by combining the data for reading, writing, and listening and talking.

Most read:

Teachers report record achievement levels for literacy and numeracy

Analysis: Is attainment really at a 'record high' in Scottish schools?

Jenny Gilruth has targeted maths in curriculum overhaul

Small town and ASN pupils trail in literacy and numeracy

However,  unlike earlier approaches – such as the 5-14 system that preceded CfE – there are no specific, national tests to determine whether or not a particular level has been achieved.

Whereas in the past a pupil might sit a Level E reading test, or attempt the Level F maths assessment, achievement is now, at least in theory, supposed to reflect a teachers’ judgement of each young person’s overall work. Rather than outsourcing their decisions to some true or false questions about Mog the Forgetful Cat, teachers are instead expected to assess a whole range of different materials in order to arrive at their judgement – and that judgement is then turned into national data.

Such an approach comes with several potential problems, all of which have plagued Scotland since the ACEL system was introduced by the SNP.

The first issue is that, for such a system to work well, everyone has to be clear about the standards. If you’re going to base official statistics, national policymaking and huge amounts of public spending  on the number of primary 7 pupils able to work at or beyond Level 2 requirements, for example, then you’d better be absolutely sure about what Level 2 actually means. In Scotland, this is not - and has never been - the case.

Ever since the gradual introduction of CfE from the beginning of the last decade, teachers have warned that the vagueness of curricular documents and lack of exemplification of standards are a serious problem. I know this because I was there, trying to get my head around the new documentation.

The lack of specificity is less of an issue in the classroom, or when working with an individual student, but as soon as we try to understand what is happening across a school, a council, a region, or indeed the country itself, that lack of agreement on standards becomes a major problem, and it is one that teachers regularly raise even today.

Read more: PISA - what does the data really tell us about Scottish schools?

On top of that, the assessment approach in CfE has never been properly resourced. It may well be better for students for be assessed in the broader, more holistic terms available under the new curriculum, but doing so properly asks more of teachers than was ever the case under the previous system.

They needed time, space, and well-run support networks to have any chance of making it all work – they got none of it. Scotland’s teachers have just about the lowest levels non-teaching time in the developed world and the removal of a number of support structures over the years has made the job harder than ever.

Another concern is that a subjective system such as ACEL puts significant negative pressure on teachers and schools, although there are many who would argue that this was always part of the government's plan.

Many have forgotten, but when Sturgeon announced her doomed crusade to close the attainment gap, she made it clear that she wanted to achieve this, in no small part, by publishing school-by-school data and test scores. The idea behind that sort of approach is always much the same: if you threaten schools publicly enough, the data will get it better.

If you’ve also got a measurement system that is based on submissions from teachers themselves, and where the performance data is linked directly to schools, then the pressure to produce the ‘right’ results can become immense. Ultimately it's not even conscious choices to alter results that are the big threat here - it's the subconscious pressure to demonstrate improvement, no matter what, that alters behaviour.

That said, over the years I’ve also lost count of the number of teachers who’ve told me about coming under pressure to improve ACEL data by moving a few pupils up a level, regardless of whether the evidence supports such a shift.

The consequence of all this is simple: even if standards don’t change, and children don’t see any benefit, the data will allow politicians to claim that they are succeeding.

And that is where we are now.

To be clear, none of this means that there hasn't been any improvement, or teachers’ judgements of their pupils are automatically unreliable – it’s just that we’re asking teachers to use their knowledge in a way that doesn’t really work.

If you want to know all about how an individual pupil has been progressing, or what the next steps in their learning should be, then you should ask their teacher; if you want to know how well our national education system is functioning, and how effectively we are developing the literacy and numeracy skills that underpin so much of young people’s learning, then you need a better, more targeted approach.

And the worst part is that we used to have one.

Up until 2016, we used a system called the Scottish Survey of Literacy and Numeracy (SSLN). This was a sample-based system not dissimilar to PISA but built entirely around the Scottish curriculum. Every year, a selection of pupils from across the country sat the assessments, which alternated between literacy and numeracy each time, in order to produce an objective snapshot of the situation across the country.

There was no way to prepare for the SSLN and no way to know if any of your students would be involved. Even if these things had been possible, there was no pressure to game the system in any way because results were not tied to pupils, teachers or schools. As a result, the data produced by the SSLN was extremely reliable – perhaps too reliable for some.

Having accepted that the SSLN results in 2014 and 2015 showed declines in literacy and numeracy standards in Scotland, the SNP then decided to scrap this objective measure and replace it with the subjective ACEL data. Whether that was a glaring error or a cynical manipulation, the result is the same: we have less reliable information about the state of Scottish education, and young people are therefore being let down.

The Scottish Government is well aware of all of this. Indeed, even the OECD identified the problem in their 2021 review of Scottish education, arguing that we should “re-develop a sample-based evaluation system to collect robust and reliable data necessary to support curriculum reviews and decision making.”

That’s a very diplomatic way of saying that scrapping the SSLN and replacing it with ACEL was the wrong decision, and one that the Scottish Government, if it is serious about improving education in this country, should undo.

Until that happens, it's hard to really know the truth about literacy and numeracy in Scotland. Perhaps that suits some people just fine.