Boris Johnson has faced criticism from his ethics adviser over his handling of the partygate scandal, as the prospect of a leadership challenge moved closer.

After another bruising day for the Prime Minister as more Tory MPs called for him to resign, Lord Geidt suggested that Mr Johnson’s fixed penalty notice (FPN) may have breached the Ministerial Code.

He said a “legitimate question” had arisen as to whether the FPN, issued for a June 2020 birthday party thrown in Mr Johnson’s honour in the Cabinet Room, might have constituted a breach of the “overarching duty within the Ministerial Code of complying with the law”.

Lord Geidt, the independent adviser on the Ministerial Code, also questioned the Prime Minister’s willingness to “take responsibility for his own conduct” in relation to the ministerial rules and delivered a withering assessment of exchanges with Downing Street officials.

Downing Street partygateCabinet Office photo showing Prime Minister Boris Johnson at a gathering in 10 Downing Street for the departure of a special adviser during lockdown (Sue Gray Report/Cabinet Office/PA)

This included Lord Geidt advising them that Mr Johnson should be “ready to offer public comment” on his obligations under the code, noting this had “not been heeded”.

Mr Johnson, in a letter released on Tuesday evening, responded by claiming the FPN “did not breach” the Ministerial Code as there was “no intent to break the law”.

He also insisted he had taken “full responsibility for everything that took place on my watch” in light of lockdown-busting gatherings in Downing Street and pointed to his House of Commons apology.

Amid suggestions Lord Geidt was considering his position over the handling of the issue, the Cabinet Office pushed back and insisted he is not quitting.

But Lord Geidt appeared to hint about the prospect when, in his annual report, he noted he had attempted to avoid offering advice to Mr Johnson about his obligations under his own Ministerial Code.

He added: “If a Prime Minister’s judgment is that there is nothing to investigate or no case to answer, he would be bound to reject any such advice, thus forcing the resignation of the independent adviser.

“Such a circular process could only risk placing the Ministerial Code in a place of ridicule.”

The exchanges were made public after more Tory MPs publicly called for Mr Johnson to quit in the wake of Sue Gray’s report on lockdown parties Downing Street.

PM’s flat refurbishmentLord Geidt is the independent adviser on the Ministerial Code (Dominic Lipinski/PA)

There is a growing belief at Westminster that it is only a matter of time before the 54 letters from Conservative MPs needed to trigger a confidence vote are reached.

Former Tory leader William Hague said the Prime Minister was in “real trouble” while another ex-leader, Sir Iain Duncan Smith, appealed to colleagues to halt their plotting until celebrations for the Queen’s Platinum Jubilee are over.

Former cabinet minister Dame Andrea Leadsom became the latest senior figure to publicly criticise Mr Johnson, saying Ms Gray’s report had exposed “unacceptable failings of leadership that cannot be tolerated and are the responsibility of the Prime Minister”.

In a letter to her constituents shared on social media, she stopped short of calling for him to go but said she and other Tory MPs “must now decide individually on what is the right course of action that will restore confidence in our Government”.

Meanwhile, Carlisle MP John Stevenson became the latest Tory backbencher to announce publicly that he has submitted a letter to the chairman of the 1922 Committee Sir Graham Brady calling for a vote of no confidence.

Baroness Davidson of Lundin Links, a former leader of the Scottish Tories, predicted there could be more letters submitted in the coming days.

She said MPs will have constituents raising their feelings during recess, telling Piers Morgan Uncensored on TalkTV: “I think that the trickle that you’ve seen in the last couple of days is part of that and we may see a few more.”

But Culture Secretary Nadine Dorries struck a bullish tone about Mr Johnson’s prospects, telling Sky News: “180 MPs are what would be needed to secure a vote of no confidence in the Prime Minister, that is never going to happen.”

Since the end of last week a steady stream of MPs – having had a chance to study Ms Gray’s findings in detail and consult with their constituents – have come forward calling on Mr Johnson to quit.

Under party rules, the chairman of the backbench 1922 Committee is required to call a vote of confidence in Mr Johnson’s leadership if 54 Tory MPs – 15% of the parliamentary party – submit a letter calling for one.

So far, more than 25 MPs have publicly called on the Prime Minister to stand down – although not all of them have said whether they have written to Sir Graham.

However, it is also widely believed in Westminster that a number of others have put in letters without declaring their intentions amid speculation the tally is approaching the total needed to trigger a vote.