In 1979 Michael Rutter published the seminal book Fifteen Thousand Hours.

That's the amount of time a child spends in school, and his assertion seems obvious, but in 1979 it was radical. Different schools influence differentially how well children perform: some schools do better than others.

The Thatcher Government, elected in 1979, disliked comprehensive schools. Committed to market forces, it used Rutter's assertions to force competitive concepts on a comprehensive system it could not easily dismantle. Its answer – never Rutter's intention – was to compare schools, using league tables.

Since then research on improving schools' effectiveness has been bedevilled by that ideological baggage. Some, especially Scottish teachers, see any attempt to analyse school outputs as Thatcherism: labelled and dismissed, end of matter.

Most, however, accept that schools can improve and that analysing what works and what does not can be helpful. But examining outputs alone is flawed. Judging by exam results, for example, needs an awareness of the social and cultural background of pupils.

Christopher Chapman and other authors have now updated such research with the publication of a collection of essays, School Effectiveness and Improvement: Research, Policy and Practice (Routledge). They have asked a big question: how do we promote equity as well as effectiveness so that one school or student's gain is no longer another's loss?

In England, raw results, with no contextual allowances, remain the principal means of judging school performance. They easily support league tables and simplistic headlines. The development of reliable value-added statistics, taking different student intake into account, becomes a priority. Research is also required on which approaches best support disadvantaged children.

Chapman wants school improvement research to reconnect with classroom teachers. Researchers in Manchester have linked with coalitions of schools and teachers to compare, contrast and analyse lessons. The academics merely support the practitioners; their conclusions and insights come from and belong to the teachers.

Manchester University's Professor Mel West points out that both England and Scotland lie in the bottom five (of 52) OECD countries for the influence of socio-economic circumstances on pupil attainment.

Our poorest children are, across the developed world, among those most poorly served by their schools. He criticises the Blair Government's prescriptive and micro-managing "closing the gap" approaches.

His alternative is greater collaboration among schools. Prof West poses radical options. He states the drive for improved exam results led to "orchestrated changes in school populations": the exclusion of poorly attaining pupils, hidden enrolment policies, course choices geared to attainable rather than valuable courses.

He claims interventions targeted at pre-school children are among the most effective in improving outcomes. He states there is little evidence that academies improve attainment.

John McBeath of Cambridge University, formerly of Strathclyde University, stresses that Scottish education is different: 97% in state schools, all of which are comprehensive, no Ofsted, no compulsory National Curriculum. The local authorities are an obstacle to school-level decision-making and flexibility.

Prof McBeath points to several factors, outwith school effectiveness, which have major impacts on attainment: peer pressure, private tutoring and family influence.

Another contributor, Jim O'Brien, former dean at Edinburgh's Moray House, sees teacher quality as the significant variable. The professional development of teachers, then, becomes the key to school improvement.

David Hopkins reviews American, English and other programmes to raise attainment. Like Prof West he doubts the effectiveness or sustainability of top-down national initiatives. Schools and teachers must lead the next phase of reform, moving to a new era of teacher professionalism. Glasgow University's Tony Townsend is also sceptical about top-down national projects, and points out that despite countless school improvement programmes in the US, reading test scores there have remained virtually unchanged over 40 years.

Too much educational research is as dry as dust and inaccessible to even the experienced practitioner. Although some of the statistical and methodological material in this volume of essays is specialist, the bulk of the work is not.

It starts from the assertion, which any teacher, head or deputy head, should accept, that we can all do better than we presently do. It suggests that neither naming nor shaming through league tables or major national initiatives will make sustainable improvements.

Above all it offers professional teachers the opportunity to deliver their own improvements by working collectively. This fits neatly with the mind-set of Scotland's young teachers who are open to dialogue and self-evaluation.

The success of this approach requires two things. Firstly, the small number of teacher dinosaurs who believe that once qualified, their professional learning is complete, need to change or go.

Secondly, government (national and local) should set broad strategic directions for education but end the micro-management habit.

Chris Chapman and his colleagues have done British education a valuable service with this publication.

Alex Wood is a former headteacher and educational consultant. School Effectiveness and Improvement: Research, Policy and Practice is published by Routledge, priced £28.99.