‘PARTYGATE’ damages the Prime Minister. It also risks tarnishing the civil service’s reputation as a Rolls-Royce machine, which purrs along smoothly no matter how bumpy the road or wayward the driver. Smoke is billowing worryingly from the bonnet. Is some fresh oil needed or a reconditioned engine?

Managing this Prime Minister can’t be easy. And an organisation’s culture is set from the top. Subordinates take their cue from their leader’s example. That’s why Boris Johnson’s attempts to shift responsibility, for what went on under his own roof, won’t wash.

I worked for 6 years in No 10 for two different premiers. It’s hard to imagine the boundaries between work and play becoming so blurred on their watch. Or that they would have been so insensitive to the heightened obligations expected during a national crisis.

READ MORE: Nicola Sturgeon and BBC's Raworth in interview clash. And the winner was?

However true this is, it’s also evident senior civil servants in and around No 10 haven’t covered themselves in glory. The job of civil servants is – in part – to save their ministers from themselves, protecting the integrity of our governing institutions. In this instance they’ve failed. Instead, the Prime Minister finds himself neck deep in a tub of very hot water, with the hand of his unfortunate private secretary, Martin Reynolds, on the tap.

Drawing general conclusions about the civil service’s current state, from the collapse of discipline inside the rarified atmosphere of No 10, would be unfair to thousands of hard-working and professional public servants.

The affair does though, inescapably, put the service under the spotlight. An unflattering glimpse of what goes on behind the arras is bound to raise questions about its overall performance. Specifically, is the civil service match fit for the rigours of a world changing dramatically at unparalleled speed?

Our modern civil service can trace back its roots to the 1854 Northcote-Trevelyan Report, co-authored by a future Chancellor of the Exchequer and a senior Treasury mandarin. The report was commissioned out of concern that public administration was suffering “both in internal efficiency and in public estimation”. Plus ca change it seems.

The report led to the founding of the country’s first permanent, impartial civil service. Peter Hennessy, the leading modern historian of British government, describes this as a service ensuring “core values of integrity, propriety, objectivity and appointment on merit, able to transfer its loyalty and expertise from one elected government to the next”.

As a government adviser and minister, I encountered many civil servants. All were fine people – some of the best I’ve ever worked with – and dedicated to public service. Yet one can admire individually civil servants and what they stand for, and still wonder whether the system within which they toil is as good as it needs to be in the 21st century.

READ MORE: Are 'woke' worries spoiling telly greats?

The ‘system’ struggled badly at the outbreak of Covid. Understandable you might think. Three years before, government had war-gamed a response to a pandemic – Operation Cygnus. There was only one problem – officials north and south of the border planned for a flu pandemic, not a coronavirus. And in early 2020 it showed.

And this summer, government was caught out again by the sudden collapse of the Afghan government and fall of Kabul. As a result, the subsequent evacuation was at times chaotic, raising already present concerns that the calibre of the Foreign Office is not what it used to be.

And before patting ourselves smugly on the back, we Scots should reflect on the Scottish Government’s own embarrassing failures. The handling of complaints against Alex Salmond, and the procurement of CalMac ferries, are hardly shining examples of effective civil service management.

Too often for comfort, the country’s bacon has been saved by that great British quality – an ability to improvise. The hugely successful vaccine procurement is a classic of the genre. Unconventional, short-circuiting cumbersome processes, some calculated risk taking and relying on external subject-matter experts, rather than gifted in-house generalists.

In a 2020 lecture, Michael Gove made the case for civil service reform. Two of his central themes were re-focusing government away from process towards results, and recruiting, encouraging and rewarding “creators of original policy”.

He’s right on both counts. Governing isn’t an end in itself. It’s about improving peoples’ lives. Gove offered a depressing statistic. Only 8% of the UK Government’s 108 major programmes were assessed to judge if they were delivered effectively, and achieved their desired effect – little incentive there for future performance improvement.

Governing today is also very different from 20 years ago. We’re living through a transformative information and technology revolution. Big data and artificial intelligence open up huge opportunities, as well as posing significant risks. Does government face enough challenge from officials within its own ranks, who aren’t orthodox thinkers, and can help it capitalise on these new opportunities and mitigate the risks?

Here in Scotland the SNP Government – surprise, surprise – wants a separate Scottish civil service. But Scotland needs more joint working, sharing of capabilities and cross-fertilisation of ideas and talent, not less. Appointing John-Paul Marks as the Scottish Government’s new Permanent Secretary is a good start. He has real experience of a big, genuinely innovative – if underfunded – welfare reform programme.

The concern is that the civil service – in Scotland and across the UK – has become too insular. Frets of senior management recruiting and promoting in their own image. Prioritising career paths, with frequent switching of roles and departments, at the expense of institutional memory and the continuity needed to build on policy successes. Under-performers sheltered with a sideways move. Poor Mr Reynolds eased out of Downing Street into another ambassadorial posting, though now more likely in downtown Ulan Bator.

Our single UK civil service is a great national asset. We’ll need it on top form for the demanding decade ahead. To reform and renew the country, government must first reform itself. The time for a 21st Century Northcote-Trevelyan report has come.

Andrew Dunlop was an adviser to former Conservative prime minister David Cameron during the 2014 independence referendum