SO, what’s your favourite fictional portrayal of journalism?

Novels including Tom Wolfe’s Bonfire of the Vanities and Evelyn’s Waugh’s Scoop still offer insight into the moral, political and economic imperatives of news, as do films such as All The Presidents Men, Network and The Post, not to mention TV shows like State of Play and The Wire.

For me, however, the 2015 film Spotlight is the best of the bunch.

Based on the true story of how reporters at the Boston Globe uncovered not just the scale of child abuse by Catholic priests in the early 2000s but the institutional conspiracy covering it up, what makes this Oscar-winning film stand out for many journalists is the authentic tone and feel, right down to the cramped office with strip lighting and bad carpets.

Even the most talented writers tend to get the newspaper industry wrong, either because they don’t understand how newsrooms work or because they – perhaps understandably – choose to sacrifice reality for the sake of drama. But there is no showboating in Spotlight. Instead, it focuses on the hard, often thankless graft involved in newspaper journalism, the complex and unglamorous nature of much of the work and, ultimately, the overriding importance of the values that drive the film and the journalists portrayed in it: seeking truth and holding the powerful to account.

The Papers, the new documentary series hitting our screens this week, may not win any Oscars, but the same fundamental values are to be found within it. I must, of course, declare an interest. The two-part BBC film by Sarah Howitt follows the work of this very newspaper and its sister titles The National and the Evening Times, and yes, I pop up a couple of times.

This is no vanity project, however, rather an honest examination of the realities of modern newspaper journalism and anyone who has even a passing interest in the complex mechanics and ethics of news, most especially Scottish news, will find it a thoroughly engaging, maybe even surprising watch.

It offers, after all, a unique opportunity to see what goes on behind the curtain. Never before has there been such full and unrestricted access to a newsroom and its staff for this long – six tumultuous months – and believe me, viewers see it all: the blood, sweat and tears required to put out quality products amid cuts to staff and budgets; the race to be first with news in an age where the number of rival outlets grows by the day and political leaders have the power to bypass the traditional media altogether; the difficulties inherent in analysing and explaining events in a wildly confusing and fractured political and social landscape that is changing by the hour.

At the same time, it also brings to our attention the changing expectations of a politically and culturally divided audience that is losing trust in institutions and experts, turning away from traditional outlets and fracturing into silos and echo chambers that represent increasingly narrow and aggressive viewpoints within Remain or Leave, Yes or No, particular strands of identity politics. Accurate, thoughtful news reporting and comment (the two are increasingly conflated by audiences) has never been more important and yet relatively few people – practically no young folk at all – are willing to pay for it.

The structural, financial and intellectual ramifications of all this across an industry that is struggling to adapt and evolve in the digital age is laid bare in this documentary.

I don’t doubt that many of those who watch, and indeed many reading this now, will be experiencing, or will have already experienced, similarly painful restructures in their own industries and workplaces as digital advances continue to revolutionise – some may even say vandalise – old models and certainties. With this in mind, it is surely right and proper that those turning the spotlight on others should be willing to have it shone on themselves.

That doesn’t make it easy to watch, however. I got a sneak preview of The Papers last week and it’s fair to say that as someone who has spent a large part of my 20-year career in The Herald newsroom, it was a strange, at times rather traumatic, experience. But the emotion I felt most strongly was pride, at the passion, professionalism and commitment shown by my colleagues every day, how hard they work, often under very difficult circumstances, to record and scrutinise the world around us for our audiences.

If nothing else, this very honest documentary will help people understand more about the realities, limitations and possibilities of news in the digital age. Most all, however, I hope it will help us all agree that good journalism, in whatever form it comes, still matters.