THE new Labour government has underscored the transformative potential of artificial intelligence (AI) in its manifesto, particularly highlighting its use in enhancing diagnostic services within the NHS. While the promise of AI in revolutionising the public sector is appealing, it is crucial to proceed with caution. The Post Office-Horizon scandal serves as a cautionary tale of what can go wrong when technological solutions are not accompanied by rigorous oversight and accountability measures.
The introduction of AI in the public sector should not be undertaken without strict and enforceable public sector procurement rules.
But rules alone are not enough. For the rules to be effective, the procurement regime should encompass a special-purpose body to authorise adoption and police compliance. That way, instead of waiting for problems to arise - which an already over-burdened court system cannot possibly cope with - problems would get nipped in the bud.
In the event someone is wronged, compensation should be easy to obtain, preferably through an efficient and well-staffed complaints-handling department sitting inside the new body.
Arguably, had a regime like this been in place, the Post Office-Horizon scandal would never have seen the light of day.
Above all, and whatever its precise form, the new regime should be designed with four key principles in mind.
Transparency: One of the foremost concerns with AI systems is their complexity. AI algorithms often function as "black boxes," making it challenging to understand how they arrive at specific conclusions: a concept known as explainability. Where the NHS is concerned, the importance of explainability may be secondary to diagnostic accuracy. But as a general rule, AI systems used in public services should be explainable. Even in healthcare, a minimum base level of explainability can ensure that professionals and patients alike have confidence in the diagnoses provided.
Accountability: The Post Office-Horizon scandal highlighted the consequences of a lack of accountability. To prevent a repeat of this sorry saga, it is crucial to establish clear lines of responsibility for AI systems. There must be mechanisms in place for affected individuals to seek recompense, and these mechanisms should be easily accessible, affordable, and efficient.
Data privacy and security: The deployment of AI in healthcare involves handling vast amounts of sensitive personal data. Ensuring the privacy and security of this data is paramount. Legislators must use existing legislation to enforce strict data protection. Patients should be given brief, reader-friendly policies setting out how their data will be used.
Continuous evaluation: The field of AI is rapidly evolving, and laws governing its use must be adaptable. Legislators should establish frameworks for the continuous evaluation of AI systems and be prepared to update rules and regulations as new challenges arise.
The Labour Government’s plan to use AI in NHS diagnostics is commendable. However, it must have comprehensive legislation to address governance. Prioritising transparency, accountability, data privacy, and continuous evaluation will ensure effective and just AI deployment in the public sector.
Dr. John Zerilli is Chancellor’s Fellow in AI, Data, and the Rule of Law, University of Edinburgh
Agenda is a column for outside contributors. Contact: agenda@theherald.co.uk
Why are you making commenting on The Herald only available to subscribers?
It should have been a safe space for informed debate, somewhere for readers to discuss issues around the biggest stories of the day, but all too often the below the line comments on most websites have become bogged down by off-topic discussions and abuse.
heraldscotland.com is tackling this problem by allowing only subscribers to comment.
We are doing this to improve the experience for our loyal readers and we believe it will reduce the ability of trolls and troublemakers, who occasionally find their way onto our site, to abuse our journalists and readers. We also hope it will help the comments section fulfil its promise as a part of Scotland's conversation with itself.
We are lucky at The Herald. We are read by an informed, educated readership who can add their knowledge and insights to our stories.
That is invaluable.
We are making the subscriber-only change to support our valued readers, who tell us they don't want the site cluttered up with irrelevant comments, untruths and abuse.
In the past, the journalist’s job was to collect and distribute information to the audience. Technology means that readers can shape a discussion. We look forward to hearing from you on heraldscotland.com
Comments & Moderation
Readers’ comments: You are personally liable for the content of any comments you upload to this website, so please act responsibly. We do not pre-moderate or monitor readers’ comments appearing on our websites, but we do post-moderate in response to complaints we receive or otherwise when a potential problem comes to our attention. You can make a complaint by using the ‘report this post’ link . We may then apply our discretion under the user terms to amend or delete comments.
Post moderation is undertaken full-time 9am-6pm on weekdays, and on a part-time basis outwith those hours.
Read the rules here