IT was his famous visit to the Easterhouse housing scheme in Glasgow that provided the inspiration for many of his reformist ideas.

But, 11 years on, Iain Duncan Smith, who was praised for pledging the Tories would listen and learn from the experiences of its residents, received a very different reception in Scotland.

The Work and Pensions Secretary was branded a "ratbag" and ran a gauntlet of abuse in his home city of Edinburgh over the controversial benefit cuts and other reforms in the Coalition Government.

Mr Duncan Smith, who was Tory leader during the 2002 Easterhouse visit, was defending his controversial welfare reforms, including the "bedroom tax" at a conference at the George Hotel.

Poll tax campaign veteran Willie Black of the anti-cuts group Edinburgh North Fights Back subjected him to a lengthy harangue that ended: "We want a different Scotland, one that cares for people who are in distress and poverty. You are going to make millions of people homeless, people that desperately need help."

Claiming the bedroom tax would be the new poll tax in Scotland, he added: "You're making the rich richer and the poor poorer. You are creating your new poll tax, that's what you're going to do. We're going to see the end of you – back to England where you belong, you ratbag."

Mr Black stormed out of the venue where the outsourcing group Capita's conferences event, Welfare Reform Scotland Conference, was being held. But there was worse to come for keynote speaker Mr Duncan Smith, who insisted he was not cutting welfare benefits but "managing growth at a lower level".

As he ploughed on with his speech, two partially-sighted protesters, one with a guide dog, began lengthy and insistent heckling about the bedroom tax and other cuts. The minister admitted: "I'm not saying that ending the spare room subsidy will not present difficult cases."

Arguing that Scottish councils had been given an extra £10 million to manage the transition, he insisted: "These are changes you simply can't walk away from. These are changes that are required to get the housing system back into balance."

He also took a swipe at the prospect of an independent Scotland maintaining benefit, saying demographic change would kick in at the same time oil revenues began to fall.

But Cameron McKay and Sasha Callaghan continued heckling "speak to us, don't talk about us" and "you don't know us" before they were finally removed from the chandeliered ballroom where the event was held.

Mr McKay, who has a guide dog after losing his sight in a series of strokes, said: "Someone once said evil happens when good men stand back and do nothing. I think I am a good man, and this Government is evil."

Once a well-paid project director with a consultancy firm, he said the bedroom tax would wreck his chances of independent living. "Instead of having a carer living in to help me, I am going to spend more time in hospital. I just want to live with a bit of independence. It's demeaning."

His partner in protest, Sasha Callaghan, said she was there not for herself but for her son, who has learning difficulties. "Duncan Smith is guilty of hate crime in talking of those on benefits as scroungers," she said.

Mr McKay said he also felt let down by those from the charity sector who attended the con-ference at a cost of £350-£500 per delegate. "There were one or two who said 'hear, hear' at what we said, but I'd like them to have sent a message with a round of applause."

Ms Gallagher said: "Capita can see the break-up of the social security system will lead to greater profits for them in the long run."

Mr Black said the current mood was reminiscent of the poll tax era, with people joining the bedroom tax protests even if they were not personally affected. "We are going to build this into a campaign on a massive scale. It's got that kind of feel," he said.

Mr Duncan Smith later had a private meeting at Holyrood with members of the Welfare Reform Committee, having refused to take part in a full public session with them. Leaving the building he said he was answerable to Westminster.

Welfare Reform Committee convener Michael McMahon said he could not express strongly enough how disappointing the meeting with Mr Duncan Smith had been. "I think he has absolutely no understanding of the impact and scale of this, and he doesn't seem to understand we do things differently in Scotland," he said.

The Scottish Government dismissed a Labour demand for emergency legislation on the issue, saying SNP councils had given a no-eviction guarantee which Labour controlled authorities had failed to give. Labour's health spokeswoman Jackie Baillie said a refusal to take action centrally would lead to a postcode lottery.