Prime Minister David Cameron has rejected calls for Take That star Gary Barlow to hand back his OBE over claims the pop star invested in a tax avoidance scheme.

Mr Cameron said he did not think that removing the honour from Barlow was "necessary".

The Prime Minister told ITV's Good Morning Britain: "I mean Gary Barlow has done a huge amount for the country, he has raised money for charity, he has done very well for Children in Need, so I'm not sure... the OBE is in respect of that work and what he has done.

"But clearly what this scheme was was wrong and it is right that they are going to have to pay back the money."

Mr Cameron repeated his condemnation of "aggressive" tax avoidance schemes.

"I am against these aggressive tax avoidance schemes but I am not just against them - this Government has taken a huge amount of steps to legislate and toughen the laws and go after aggressive tax avoidance schemes for the very simple reason that if people go after these schemes and aggressively avoid tax they are making it the case that everyone else has to pay higher taxes as a result," he said.

Barlow and two other members of Take That refused to comment on reports over the weekend that they face having to pay tens of millions of pounds in tax after a court ruled a partnership in which they invested was a tax avoidance scheme.

The singer along with Howard Donald, Mark Owen and their manager Jonathan Wild invested £66 million into two partnerships styled as music industry investment schemes, according to reports.

Judge Colin Bishopp ruled that 51 partnerships set up by Icebreaker Management were to secure tax relief for members and HM Revenue and Customs is now expected to demand repayment.

It was alleged in 2012 that Barlow, Donald, Owen and Wild invested at least £26 million in a scheme run by Icebreaker Management.

At the time Take That's lawyers insisted the band mates believed the investments were legitimate enterprises and that all four named paid "significant tax".

Mr Cameron added on Good Morning Britain: "I think we should be very clear, tax evasion is illegal, and for that, you can be prosecuted, you can go to prison for tax evasion.

"Tax avoidance is, in these cases, these very aggressive tax avoidance schemes, they are wrong, and we should really persuade people not to do them.

"That is why we have these court cases where the court looks at whether a scheme is really about avoiding tax rather than anything else. The court was very clear in this case."

Labour's Margaret Hodge, chair of the Public Accounts Committee who has shone a spotlight on tax avoidance, said Barlow - who has previously been seen on the campaign trail with Mr Cameron - "might want to show a bit of contrition by giving back his OBE".

Liberal Democrat Chief Secretary to the Treasury Danny Alexander said: "People who don't pay the taxes that they should undermine the economy, damage our public services and place an extra, unfair burden on hard-working families and companies who play by the rules."

Conservative MP Charlie Elphicke said: "People who have seriously abused the tax system should be stripped of their honours."

Barlow, 43, who has spent more than 20 years in the public eye, returned to Buckingham Palace in November 2012 to be awarded his OBE by the Queen.

He was given the honour for services to the entertainment industry and to charity.

Barlow masterminded the Queen's Diamond Jubilee concert - staged at the Palace during a special bank holiday weekend in June 2012 - which featured a host of stars including Stevie Wonder and Sir Paul McCartney who played to a televised audience of millions.

The anthem Sing, co-written with Andrew Lloyd Webber, and featured at the Jubilee concert, topped the singles charts during the summer and his OBE was announced days later in the Queen's Birthday Honours.

As the creative force behind Take That between 1990 and 1996 and since they reformed in 2005, and with his solo success, Barlow has topped the singles charts 14 times and received five Ivor Novello awards for his song-writing.

His charity work includes gathering a team of celebrities to climb Mount Kilimanjaro in 2009 for Comic Relief and organising charity concerts for Children in Need in 2009 and 2011.

In November 2012 Sir Elton John presented Barlow with the Music Industry Trusts Award.

Previous winners of the award, which raised funds for Nordoff Robbins and The BRIT Performing Arts & Technology School, include Sir Elton and Kylie Minogue.

Mr Cameron added on Good Morning Britain: "I think we should be very clear, tax evasion is illegal, and for that, you can be prosecuted, you can go to prison for tax evasion.

"Tax avoidance is, in these cases, these very aggressive tax avoidance schemes, they are wrong, and we should really persuade people not to do them.

"That is why we have these court cases where the court looks at whether a scheme is really about avoiding tax rather than anything else. The court was very clear in this case."

Labour's Margaret Hodge, chair of the Public Accounts Committee who has shone a spotlight on tax avoidance, said Barlow - who has previously been seen on the campaign trail with Mr Cameron - "might want to show a bit of contrition by giving back his OBE".

Liberal Democrat Chief Secretary to the Treasury Danny Alexander said: "People who don't pay the taxes that they should undermine the economy, damage our public services and place an extra, unfair burden on hard-working families and companies who play by the rules."

Conservative MP Charlie Elphicke told the Times newspaper: "People who have seriously abused the tax system should be stripped of their honours."

The Prime Minister's spokesman told a regular briefing at Westminster that the Government was tackling "aggressive" tax avoidance.

"We have taken this power through GAAR (general anti-abuse rule)... it is designed to deal with this issue around aggressive tax avoidance schemes that are often linked with tax advisers and tax experts that are marketed," the spokesman said.

"That is what the Government has been doing as part of a wider agenda."