Following the Rutherglen and Hamilton West by-election in late 2023, I was asked to speak to SNP MPs and staffers at Westminster about the Scottish polls and the party’s electoral prospects. I was repeatedly told that they lost that by-election because of turnout: voters who had voted SNP in 2019 had stayed at home, not switched to Labour. If they could find a workable independence message, they would get “their” vote out.

As the July general election proved, they were greatly mistaken. A 20.4-point swing from the SNP to Labour in Rutherglen and Hamilton West, the fourth largest in Scottish postwar by-election history, became a 15.8-point swing across Scotland in July, and a further 35 SNP-held seats flipped to Labour. If there were lessons to be learned from defeat in Rutherglen and Hamilton West, the SNP failed to understand them.

Read more by Mark McGeoghegan

Electoral defeat, particularly when it is unexpected or on a massive scale, can be traumatic for a political party. On top of the emotional scarring and loss of sense of purpose that can come with rejection by the electorate, many of the party’s elected members and their staff lose their jobs, and the spine of the party – the activists – who have spent months pounding pavements and knocking doors can feel that that expense of energy was a pointless waste.

As a result, in the immediate aftermath of defeat often comes recriminations and relitigation of old disagreements, whether about policy, strategy, or personality.

Perhaps unsurprisingly, most of those who weigh in in the days immediately following defeat will find its roots in their pet concerns or long-standing squabbles. In the wake of Vice President Kamala Harris’ defeat by President-elect Trump in last Tuesday’s US Presidential election, the various factions of the Democratic Party have done just this – their defeat was because of misogyny and racism; it was because they focused too much on identity politics and not enough on class, or because President Biden failed to drop out early enough.

The same can be said of both the Conservative Party and the SNP after July’s general election. The Conservative leadership contest was a walking litigation of the reasons for their defeat, with the notion that they failed to fight the culture wars successfully winning out despite all the evidence that the public neither care about those issues nor agree with the Tories on them.

In the SNP and wider independence movement, factional fighting intensified. One ex-MP publicly decried Nicola Sturgeon as the reason for her defeat, despite her voters manifesting the fourth-largest swing against the SNP. If the SNP-to-Labour swing in her constituency had matched the national average, she would have held her seat. Such excessive swing was not down to a national leader who had resigned more than two years prior.

These immediate spasms of recrimination are both understandable and unhelpful for political parties. They can embed myths about defeat and effectively inoculate the party against the lessons that it needs to learn to win again, both of which can dump the party in a cycle of repeated defeats until sufficiently strong leadership comes along to break them out.

Kamala Harris' defeat has sparked soul searching by many centre-left parties across the worldKamala Harris' defeat has sparked soul searching by many centre-left parties globallyI would go so far as to say that any political party – whether the US Democrats, the Conservatives, or the SNP – looking to return to power should completely ignore everything said in the immediate aftermath of defeat. Take your time. Electoral cycles are long, and even in the cases of the Democratic Party and the SNP, where US mid-terms and Holyrood elections lurk around the corner, there are at least 22 months between elections to learn lessons and correct course.

Just as importantly as taking your time, any party looking to learn from defeat must do so systematically. While the views of activists and politicians in the party are important, they are each only a single data point. The key is developing a process to capture these views in a structured manner and combine them with other sources of data to achieve a rounded view of the defeat and its causes.

In the UK and the US, political parties are lucky enough to have plentiful sources of data on voters produced in the wake of elections. In the UK, we have the British Election Study, and in Scotland, we have the Scottish Election Study. In the US, voter files containing a wealth of data on the electorate are published in the months following elections – and a multi-billion dollar research industry produces a plethora of analyses of this data in short order.

This data is, of course, worthless if a party is unwilling to believe what it tells them. So here’s a third piece of advice: believe the voters, even if you don’t like what they’re telling you. If the data is telling you that they do not care about your priority issues and voted for your opponent because the price of a carton of eggs has doubled, don’t go casting around for alternatives – believe it.

Read more

And believe, also, that voters who have turned their backs on you can be won back over. The word ‘realignment’ is used far too liberally to describe voter shifts in elections, the Red Wall being a case in point. Once dyed in the wool Labour voters did not become Tory partisans in 2019 thanks to Brexit, as evidenced by their swing back to Labour in July. If anything, voters have become dealigned, meaning that they are more winnable than in the past. Even if they voted against you this time around, you can win them back if you listen to them.

Lastly, focus on what you can control. Every incumbent party to contest a democratic election this year has lost ground in the face of rampant inflation and its consequences, a global trend that those parties could not have changed. Election post-mortems should focus on what you, as a party, could have done differently to outperform expectations given the circumstances. Part of that is to recognise not just what you got wrong, but also what you got right.

The often traumatising and vicious aftermaths of election defeats rarely shed much light on the causes of defeat. Effectively learning from defeat takes a much longer, systematic, and humbling process, one that most political parties prove woefully inept at.

Mark McGeoghegan is a Glasgow University researcher of nationalism and contentious politics and an Associate Member of the Centre on Constitutional Change. He can be found on BlueSky @markmcgeoghegan.bsky.social