For Alistair Moffat, history is rooted in the personal. You need only to glance at his burgeoning bibliography to confirm this. Though he has written many books, on such diverse subjects as DNA, Charles Rennie Mackintosh and Tuscany, it is clear that he is most fascinated by Scotland and, in particular, the Borders, where he grew up, still lives and runs the popular Melrose-based book festival.

Few areas of Borders lore lie unexplored by Moffat. He has written about Hadrian’s Wall and the reivers, as well as the stories of Hawick and Kelso, where he was born and which he described in Homing, an affecting family memoir. Now, however, having reached pensionable age, he has produced what is undoubtedly his most ambitious work. Scotland: A History From Earliest Times encompasses 500 million years, from when the tectonic plates were shifting to form the land mass we recognize today to the referendum and its aftermath.

“Our history,” Moffat writes at the outset, “is written in our rocks just as surely as it is in monastic chronicles, census returns or the stones and bones of archaeological digs.” Five hundred pages later, having covered everything and everyone from Calgacus (nicknamed “the Swordsman”), Mary, Queen of Scots (“who was said to have played a round of golf after the murder of her first husband, Henry Darnley”) to Andy Murray, he reflects on identity and iconography, on tartan, whisky, bagpipes, the Bens and Glens, and how they have been applied not to just to the Highlands but to the entire country. “Walter Scott has a great deal to answer for. We gamely seek alternatives but they are difficult to find because, like most nations, our identity is complex and not given to instant impression. So we put a kilt on it.”

We meet in Melrose shortly after a 30-mile stretch of the old Waverley line has been opened by the Queen and days before the anniversary of the vote a year previously which might have sundered the 1707 union and separated Scotland from the rest of the United Kingdom. It is illustrative of the fact that history is dynamic and unpredictable, and that hindsight is the historian’s most precious – if fluctuating – currency. In comparison, the recent past is a blur of uncertainty, assertion and incomprehension.

In May, on the morning after the General Election, Moffat recalls, he found himself in Gloucestershire and, with the book’s deadline pressing, he was required to dash off 1,000 words on the result and its implications. When he pressed ‘send’, he thought: “I wonder what the hell that’s like?” Reflecting on the outcome, which sent 56 SNP MPs to Westminster, he traces the collapse of the Labour vote to the early hours of September 19, 2014, when David Cameron made “his incendiary statement about English Votes for English Laws. It was no rush of blood to the head but a coldly made calculation.” Momentarily, he looks lost for words. “In our lifetime, we’ve never seen anything like that and it’s still difficult to comprehend. The swingometer broke.”

Having said which, the distant past is often similarly mist-shrouded. Increasingly, historians have been inclined to look beyond kings and queens, preferring to study the lives of the masses who, for much of our history, were embedded in the land. Urbanisation and industrialisation, says Moffat, are relatively recent phenomena. “My grannie was a bondager and worked on singling and weeding in the fields. I don’t think that as individuals and in families we easily forget the association with the land. So I would hope that this book is more to do with the countryside, to do with agriculture, to do with the land, because my argument is that it is a very present and clear memory for us.”

Moffat was born in 1950, his upbringing typical of the postwar era. As a young boy, roving as freely as the Ettrick Shepherd, he was made constantly aware of the potency and proximity of history and in his book he returns often to the Borders, which for centuries has been disputed territory and the locus of conflict and violence. In Moffat’s case this was manifested on the rugby pitch where tribalism remains rife. He played in the front row for Kelso and as a raw teenager he was once selected to play against the Under-16s Welsh Schools team who were built like wardrobes. En route to the game his father handed him an envelope in which was a page with words in capital letters: “REMEMBER WHERE YOU COME FROM”.

It is advice he has never forgotten. Likewise, he is appreciative of how living standards and class distinctions have changed over the centuries. His aforementioned grannie, who was born on a farm, lived in cramped, overcrowded conditions not too dissimilar to those experienced by slum dwellers in the cities. For a spell she was a maid at Floors Castle on the banks of the Tweed. One of her many tasks, she told her grandson, was to wash the soiled linen napkins the toffs used in preference to toilet paper.

Such asides are as much part of our history as the Sturm und Drang of politics. The problem for Moffat, however, was not what to put in but what to ignore. He was eager, he says, “not to have a procession of high heid-yins. In general, my rule of thumb was to leave out what’s already been done. If it’s not been well done, and it’s not clear, get into it.”

One episode he could not ignore was the visit to Scotland in 1822 of the recently crowned George IV, an unlikely pivotal moment in the formation of the nation. Masterminded by Walter Scott, then at the height of his fame, it was, argues Moffat – no mean master of ceremonies himself – “one of the greatest publicity stunts in British history”. Scott persuaded the king to wear a Royal Stewart kilt which hitherto had been proscribed. Whether the great novelist also recommended his majesty don flesh-coloured tights which hid his hideous varicose veins is less certain. Should anyone feel inclined to discover more on this subject, Moffat urges caution. “If you’re interested in hosiery and so on, never Google ‘men’s stockings’. You will see things you don’t want to see.”

The influence of George’s visit was immense and long-lasting, not least in the Tweed valley where 19th-century entrepreneurs quickly realised that there was capital to be made out of the fashion for tartan and plaid. The local mills struggled to meet demand as the gentry rushed to ape the monarch. But not everyone was impressed. Scott’s own son-in-law, J G Lockhart, deemed the whole affair a “hallucination” and complained that the Scots were being portrayed as “a nation of Highlanders”, which, says Moffat, was true.

“But such was Scott’s sureness of touch and judgement of the public mood that the hallucination hardened into reality. Now, two centuries after George IV appeared resembling a tartan dumpling, few formal occasions are kilt-free. Lowland bridegrooms and their male guests routinely put on the dress of men who were believed by their ancestors to be sub-human savages. By contrast, few Highlanders or Islesmen are ever seen wearing kilts.”

Moffat’s sympathy for Gaeldom is as deeply felt as is his affinity with the Borders. In a previous existence, as an executive at Scottish Television, he felt he should learn Gaelic if he was going to commission programmes in the language. But despite sterling efforts to keep it alive, it appears to be in terminal decline. “Gaelic speakers are dying faster than they’re being born,” says Moffat. “We’re down below 50,000 now; technically that’s when a language is dead. Everybody’s bilingual, but very few people now live in Gaelic, whereas a lot of people live in Welsh.”

He believes the problem can be traced to the moment at which government decided legislatively to act. In the case of Welsh, the revival was timeous; for Gaelic, it may have come too late. Whether it will survive remains to be seen. Moffat is hopeful if not bullishly optimistic. Taking the long view, he says that had things been different we might not have been speaking any of Scotland’s three current languages. Pictish is one possibility, Danish another. Who knows: had the Romans stuck around a bit longer we may all be speaking Italian. Such, it seems, are the tricks history feels it necessary to play on its pawns.

Scotland: A History From Earliest Times by Alistair Moffat is published by Birlinn, priced £25