The Tory MP under investigation for allegedly watching pornography in the Commons chamber has been backed by the safeguarding minister over his refusal to stand down immediately.

Rachel Maclean suggested on Saturday the “appropriate measures will be put in place” to protect Neil Parish’s constituents, after he vowed to continue with his duties while inquires are ongoing.

Senior Conservative Caroline Nokes said it will be “difficult” for Mr Parish to spend the required time in Parliament as a select committee chair.

She also accused the Tory whips of “dither and delay” in not suspending the MP from the parliamentary party until Friday, despite colleagues airing the claims in a meeting three days earlier.

Mr Parish has pledged to continue his “duties” as the MP for Tiverton and Honiton, in Devon, and as chairman of the Commons Environment, Food and Rural Affairs Committee.

He said he may have opened the pornography accidentally, as he referred himself to Parliamentary Commissioner for Standards Kathryn Stone for investigation.

But senior Tory Karen Bradley urged him to stay away from Parliament and Harriet Harman, the longest-standing female MP, called for his immediate resignation.

Ms Maclean, the minister responsible for safeguarding women, said “clearly what he’s done is unacceptable” and that he should resign if found to have watched the material deliberately.

Asked on Times Radio if it was appropriate for him to carry on while under investigation, she said: “I genuinely think that’s a matter for him.”

But she added: “I want to be clear, he is not continuing his business as normal, he’s under investigation, and I’m confident that the appropriate measures will be put in place to safeguard any of his constituents, particularly women and girls.”

The Conservative Party has been asked what action will be taken.

Ms Nokes, who chairs the Women and Equalities Committee, agreed “it’s right those investigations are carried out before any further action is taken”.

However, she suggested it will be a challenge for Mr Parish, who has been an MP since 2010, to continue leading his committee examining environmental matters.

“It’s a senior and responsible position and as a select committee chair myself I know how seriously I take that job,” she told BBC Radio 4’s Today programme.

“That’s a matter for Neil to discuss with the party whips and to take a position on, but I think whilst he’s under investigation it’s going to make it very difficult for him to be in the Commons as much as he needs to be.”

Ms Nokes also raised concerns about how quickly Chief Whip Chris Heaton-Harris acted despite two MPs raising their concerns on Tuesday evening.

“I felt by leaving it until Friday before we knew that action had been taken by the whips that it felt like unnecessary dither and delay,” she said.

Mr Parish, a 65-year-old farmer, said he will resign if found guilty by Ms Stone, who is yet to say whether she will open an investigation.

Separately, the Independent Complaints and Grievance Scheme (ICGS), which looks into claims of bullying and sexual harassment, is understood to have begun examining events after at least one witness made a referral.

Mr Parish said in a statement on his website he “will be co-operating fully with any investigation, and whilst it is ongoing I will continue to perform my duties”.

Later asked by reporters at his Somerset farmhouse if he opened something in error in the Commons, he said: “I did, but let the inquiry look at that.”

However, there have been suggestions that a Tory minister also witnessed him watching porn on a second occasion, in a committee meeting.

Ms Bradley, who chairs the Procedure Committee, said she “would urge him not to come into Parliament” while under investigation and was clear he must quit if found guilty.

“I’m struggling to find an excuse, I’m struggling to find a reason that might mean it was acceptable, but let’s let the investigation take its course,” she told Channel 4 News.

Labour’s Ms Harman called on Mr Parish to stand down amid a “new low for the House of Commons”.

She told BBC Radio 4’s PM programme: “If this is what he has done, he should stand down from Parliament right away. It is not right for him to go through the investigation processes if that is what he has done.

“Clearly he is not fit to be in Parliament. He should accept that and not drag the processes out.”

Liberal Democrat deputy leader Daisy Cooper called on Prime Minister Boris Johnson to order Mr Parish to “resign immediately”.