Statistics on pupils’ skills ‘cannot be relied upon’

Statistics are labelled “experimental” when they are still subject to testing and will only lose that description when they become useful and credible.

The Scottish Government has already scrapped the only alternative to the figures, the Scottish Survey of Literacy and Numeracy (SSLN).

Keir Bloomer, chairman of a commission on school reform set up by the Reform Scotland think-tank, said ministers had created a vacuum of school information.

He said: “The fact that these statistics are experimental is basically an admission that thus far these have been a failure.

“Last year’s figures revealed huge inconsistencies in teacher judgments in different areas and a general degree of over-optimism.

“We have now got nothing that replaces the data from the SSLN and I cannot see these teacher judgments improving very much so we are left without a proper measure of performance for the foreseeable future.”

Political opponents also criticised the Government for removing the SSLN, with Liz Smith, education spokeswoman for the Scottish Conservative Party, calling for better measures of performance.

She said: “Every educational expert in the land is telling the Scottish Government Scotland needs to improve both the quantity and quality of the data set which can measure progress in our schools.

“The SNP has already removed Scotland from some key international indicators and with the decision to end the SSLN it is even more vital that we have good quality, accurate data.”

Iain Gray, education spokesman for the Scottish Labour Party, said: “Once again we have only experimental information about attainment in schools.

“We used to have the respected and statistically valid SSLN, but of course that produced results which were embarrassing for the SNP Government, so they abolished it.”

The issue has arisen because the figures published this week are based on the judgments of individual teachers on what level pupils have reached under the Curriculum for Excellence (CfE) rather than a formal test or survey and are therefore subjective.

Last year, when the figures were published for the first time, there were considerable differences between performance levels in different council areas.

Andrea Bradley, assistant secretary of the Educational Institute of Scotland teaching union, said the CfE level data was not suitable for school by school comparisons.

She said: “The most useful assessment data for teachers is that which is gathered carefully by them, from a range of sources, about individual children and young people, and which is indicative of the strengths, areas for development and next steps for each individual learners. CfE levels data is not intended to provide any sort of benchmark to measure performance at school level.”

A Scottish Government spokesman said more data than ever before was being published on children’s progress under CfE.

He said: “The statistics will show attainment levels based on teacher professional judgment in literacy and numeracy of every child in P1, P4, P7 and S3.

“Unlike the SSLN, the new arrangements allow people not just to see the national trends, but also how their local schools are performing. Furthermore, it will give teachers and parents data on the attainment of their own children, something the SSLN survey did not do.”

The Government expects the introduction of national assessments to help standardise teacher judgements across the country from 2018 onwards improving the quality of the data available.

In May the results of the last SSLN revealed that fewer than half of Scotland’s 13 and 14-year-olds were performing “well” in writing compared to 55 per cent in 2014 and 64 per cent in 2012.

Analysis: Complex story of why schools lack reliable measure of standards 

By Andrew Denholm

THE reason why Scotland currently has no credible national measure for standards of literacy and numeracy in primary schools is complex.

It begins with the results of the Scottish Survey of Literacy and Numeracy (SSLN) which, since 2015, has shown a decline in standards.

This was no cataclysmic drop and was perhaps expected at a time of budget cuts and the confusing roll-out of a new curriculum.

Nonetheless it was concerning and so the Scottish Government asked councils to provide more detailed information on literacy and numeracy under the Curriculum for Excellence (CfE).

What they got back set alarm bells ringing because, although all councils were using some form of standardised assessment, these were not comparable across authority areas and so were incapable of giving a national picture.

When Nicola Sturgeon became First Minister she made improving education her mission and, with concern over the quality of information to the fore, she announced a new programme of standardised national tests for all pupils in P1, P4, P7 and secondary third year.

It was clear Ms Sturgeon wanted better national data from these tests and that she intended to publish the information to provide a transparent record of progress, even if it meant the compilation of school league tables. With such detailed information available on a school-by-school – and even pupil-by-pupil – basis the government decided it no longer needed the SSLN and scrapped it.

This, however, was where bold political ambition met the realities of delivering a workable policy in a climate where standardised tests are toxic for powerful teaching unions.

Unions argue that when such tests are introduced teachers start coaching pupils to pass them and they cease to be a measure of progress. Unions also don’t like how they can be used to judge the effectiveness of individual teachers.

With the Educational Institute of Scotland threatening a boycott of the tests, a series of behind-the-scenes compromises were agreed which led to a significant watering down of the policy.

The most important of these emerged when the government announced it was no longer going to publish the results of the tests, but instead would release school-by-school data on the CfE levels pupils have reached.

Crucially, these levels are based on the judgement of teachers which, while suitable for school-level discussions with parents about progress, are by their nature subjective and therefore not suitable for national benchmarking.

That is why the current statistics have been labelled “experimental” with the previous year’s release highlighting significant inconsistencies between councils.

The government hopes as standardised assessments are rolled out teachers will have a more consistent approach to the assessments they make on progress. But until that happens Scotland has only one national data set on standards which, in its current form, cannot be used to judge progress.