The last five months have been something of a whirlwind. At the end of March, the Cabinet Secretary for Justice, Michael Matheson, asked me to chair an Independent Advisory Group that would draft a code of practice dealing with police stop and search, and also consider whether the tactic of “consensual” stop and search should end.

I was aware of the issues, mainly because of the media attention over the last few years, and I had followed the academic research in the area, mainly by Dr Kath Murray of Edinburgh University.

The other members of the Advisory Group were all experts in their respective fields, including human rights with particular emphasis on children and young people. I was glad to have this group working with me or else I could not have completed the report by the end of August – the punishingly short timescale that was dictated by Mr Matheson’s perfectly reasonable wish to address the issues in time to incorporate amendments to the Criminal Justice Bill which is back before the Parliament this month.

The process was fascinating. I have practised in the criminal courts for almost 30 years, and so have an idea of some aspects of policing, but I needed to find out much more. We got assistance with this from a considerable number of people and organisations, especially those involved with children and young people and also some within the policing community, both serving and former officers. In the report I acknowledged the integrity of the police officers I met. They had a passion for the job, which was obvious. All recognised that the tactic of “consensual” stop and search was, or had become, a problem.

Almost everyone we heard from welcomed the idea of a code of practice but the position on “consensual” stop and search was more divided. Significantly, some of the most persuasive and vocal critics of the tactic came from within the policing community. Some had never used it and were aware of no gaps. Others saw a place for it but thought that targets had driven officers to use it when their own discretion and instincts should have led to different decisions. Some thought that it had been useful at a particular time as part of the package of measures used to tackle serious violent crime, but even they thought that it was now doing more harm than good for policing.

Poor, untargeted use of the tactic, informed by a sense of pressure to use it in specific areas with specific groups (mainly children and young people), had reduced public confidence in the fairness of policing overall.

“Policing by consent” has been a key part of policing in Scotland throughout its history. Having the confidence of only some of the community isn’t enough when so much crime happens in the areas which had been on the receiving end of a substantial amount of “consensual” stop and search. In those areas there was a perception by many of the police exercising what was, in effect, a power of random stop and search. Ending “consensual” stop and search may help to restore some confidence in those areas.

The Scottish Human Rights Centre were certain that the tactic was unlawful – it had no framework in law and operated on the basis of a fiction of “consent” in many cases where either the age or capacity of the individual, or the general power imbalance, meant that the answer “no” was never a real possibility when a request was made to be allowed to carry out a search.

The success rate of “consensual” stop and search was very poor. If alcohol was stripped out of the searches/seizures, the success rate is about 3%. It seemed to us, and to many officers, that it was a poor use of officer time.

The police will not be left naked by the implementation of our recommendations. They continue to have considerable statutory powers. The recent presumption by Police Scotland in favour of using those powers has seen an increase in the success rate of statutory searches. Thus the evidence supports the perhaps obvious – if the police search individuals because they have reasonable grounds to suspect possession of a prohibited item, they are far more likely to find something than where they rely on consent and an absence of any grounds that can be articulated. Recent evidence to this effect here in Scotland is matched by evidence from England where there has been no power of “consensual” search for many years. They do not need it and have not asked to have it restored.

As for the suggestion that officers will be hobbled by the ending of “consensual” stop and search, I am afraid that the evidence suggests the opposite. This is the right time to make the necessary arrangements, already underway to some extent, to end an ineffective practice of questionable lawfulness that has driven a further wedge between the police and some of the key communities they serve.

John Scott QC is a human rights lawyer and former chair of the Scottish Human Rights Centre. He chaired the independent advisory group into stop and search procedures by Police Scotland. The report's recommendation of a new statutory code of practice is to be implemented by the Scottish Government