The Scottish Conservative leader Ruth Davidson was in a good mood as she addressed a fringe meeting at her party's conference at the weekend. Scottish Tories, she believes, are on course to return their largest number of MSPs in May's Holyrood elections. "That is the challenge I have laid down to my candidates," she said. But, no matter how optimistic Ms Davidson feels, it is going to be a hard challenge to meet.

One justifiable ground for optimism for Ms Davidson is the fact, as she pointed out in her speech, that many of her supporters voted tactically in May under the first past the post. The hope for the Tories is that those tactical voters will return to the party under the list system for Holyrood.

Some of the other political stars may also be aligned in Ms Davidson's favour. The state of the Labour party, for example, may help her in the ambition to become the effective opposition to the SNP, but it remains to be seen what effect Jeremy Corbyn will have in Scotland – it is likely to be negative in big swathes of England, but it may be that his old-school, left-of-centre policies help Labour to some modest recovery in Scotland.

However, even if Labour remains in the doldrums in Scotland, the prospect of old Labour voters switching directly across the Tories in any great numbers remains slim because of the persistent toxicity of the Conservatives and what they are seen to stand for. Ms Davidson is an effective performer, and has attracted largely positive press; she has also engaged effectively with the issues. But that is not the same thing as a recovery for the party, which still seems as far off as ever.

The main problem for Ms Davidson – and it is an old one for the Tories in Scotland – is the effect the leading figures in the UK party have on its image north of the border. During the Scottish leadership campaign, Ms Davidson's rival Murdo Fraser suggested the party should re-establish itself as independent of the UK party and the jury is still out on whether that would have been a better idea than sticking with the status quo that has not worked in any meaningful way in Scotland for at least 30 years.

In its current situation, the Scottish Conservatives face two factors that will create a drag on any chances they have of a recovery in Scotland.

The first is George Osborne, whose speech to the conference this week boasting of new infrastructure projects is the latest step in a strategy aimed at winning him the leadership after David Cameron. If there was any doubt about that, the flattering profiles of the Chancellor in the tabloid press at the weekend should remove them. But the idea that a Tory party whose UK leader was George Osborne would do significantly better in Scotland is far fetched.

The second drag factor on Ms Davidson's chances is the economy. Yes, it is recovering, and has been for some time, but voters in Scotland are still feeling the effects of the wage depreciation that has dragged on for eight years. The Prime Minister has also resisted pressure to back down on the planned cuts in tax credits despite the Institute for Fiscal Studies warning that millions of people will lose up to £1,300 a year from the changes.

In the fact of all that, what are Ms Davidson's chances of a significant recovery? It is clear that Mr Osborne would like to offer an income tax cut at some point soon, and Ms Davidson would like to promote a tax-cutting agenda when the next tax powers come to Holyood. But, effective as she is, the odds are still against Ms Davidson and her party – the road to recovery remains as steep and slippery as ever.