Back in 2007 when the SNP first came to power, they made one of the better decisions since Scottish devolution. They set out a series of outcomes, targets and ambitions across all service areas by creating the National Performance Framework (NPF).

For the first time, a government could now be openly challenged on its performance across all its services, many central to the wellbeing of the Scottish people and its businesses. In stating short and long-term objectives it also set a framework for the whole public sector to work within.

The NPF also provides the government of the day with the opportunity to claim kudos about its performance in delivering targets and outcomes.

Scottish local government welcomed this commitment to measuring the outcomes of the collective performance of ministers, government departments and the country’s 110 quangos. After all, government departments and a host of statutory inspectors had been peering into each council’s performance for decades, then sagely announcing from on high their assessment of the performance of individual councils.

With John Swinney MSP (the then Finance Secretary) enthusiastically raising the banner for the NPF, things looked promising.

What emerged was initially encouraging with national outcomes (now 11) being established, each stating a clear ambition with their own set of national indicators measuring performance.

Recently updated, there are 81 performance indicators, each one based on a single or a cocktail of measures. However, 19 of these indicators have no results. They are simply described as “performance to be confirmed”. Of the remainder, 17 are improving, 13 declining with the remaining 32 flat lining.

Setting to one side this unconvincing performance, there are a number of issues concerning the way these improving, declining and flat lining conclusions are reached.

This can best be illustrated by looking at two of the most important responsibilities of the Scottish Government: education and health. Both have nine indicators.

Read more: Holyrood needs to stop trying to be our biggest local authority

Education fares badly with four of its most important indicators – educational attainment, confidence of children and young people, resilience of children and young people and engagement in extra- curricular activity – having no results. Of the remainder, three state that performance is improving.

The first looks at the 2007-2020 period; the second at 2011-2020 and despite the performance in 2020 being the same as 2011 it recorded as performance improving. In the third, the 2020 performance is poorer than 2017 but it also gets a performance improving measure.

Looking at the Healthy and Active National Outcome’s indicators, although all nine have an end date of 2020 or 2021, their start dates vary from 1997, 2003, 2004, 2006, 2009 and 2012. A sceptic might say that this pick and mix approach with varying start dates might help generate many improving assessments.

However, only two health indicators are judged to be improving and closer scrutiny suggests that for one of them, active travel, this conclusion is questionable.

A more detailed analysis of all 81 indicators could well raise similar issues which brings us to the core of our concerns.

The excellent idea of a National Performance Framework is being weakened for the following reasons: assessments for almost a quarter of performance indicators is absent; there is no standard approach to start and end dates when measuring trends in performance; and it is unclear who ultimately decides which indicators are judged to be improving, flat lining or worsening.

Why is this important? We are effectively talking about the government’s report card and as the parliament’s finance and public admin committee advocated in late 2022, the NPF should sit at the core of government decision-making and should drive public sector delivery.

This seems contrary to our current preoccupation with the likes of the Ferguson Marine ferries contract, single issues that steer us away from examining our devolved government’s overall performance.

To do this we need confidence in the accuracy of the NPF. With so many performance indicators not yet determined and bearing in mind the earlier comments on accuracy, the process could benefit from independent external scrutiny and especially validation.

Look for example at the work of the Office for National Statistics and the Office for Budget Responsibility, both reporting to the UK parliament. Both organisations, independent of government, are sources of highly relevant information that the public trust.

The Scottish Parliament should in our opinion, appoint an independent external organisation to validate the NPF. The public could then assess our devolved government’s performance with a high degree of confidence and avoid concerns that the Scottish Government is marking its own report card.

George Thorley is writing on behalf of the Mercat Group of former Scottish Council CEOs.