THE PAST MASTER In his most candid interview yet, art supremo Sir Timothy Clifford reveals his dreams for the future and tells how dyslexia shaped his controversial reign over the National Galleries of Scotland

Clifford has allowed us into his personal world. He says he wants to explain things, to talk things over in the one place he feels most relaxed, most himself.

In his time Clifford has been a controversial and a transformative presence at the galleries on The Mound in Edinburgh. He will leave in January next year, when he turns 60, to be succeeded by John Leighton, the quietly spoken director of the Van Gogh Museum in Amsterdam. Right now, though, Clifford is talking about boxing.

He boxed for one of his schools, Sherborne in Dorset, which he attended after an unhappy time at schools in Kent, where he grew up the son of Derek Plint Clifford, a poet, arts expert and writer, and Anne, who excelled at pottery.

Perhaps it was at Sherborne where Clifford learned to stand his ground. Still, it seems odd now to think of the elegant art expert, fists up, popping a bounder on the nose. His territory is usually sharp words, sharp suits and an even sharper mind.

"I had to do it to defend myself, " he says of his pugilistic past. "If you are an aesthete, you have to be tough, don't you think? If as a youth you are interested in wild orchids, as I was, people think you are mad. In fact, I boxed for Sherborne for three years running. I won the school competition a couple of times. I think going to an English public school toughens you up. It prepares you very well for prison.

Maybe they are not like that now, but in those days it was very, very tough. So it was very useful to be able to box."

The spit and blood of the ring are a world away from the carefully maintained treasure trove of a home where he sits now. But this is a different, difficult time for Clifford: in just over two months he will leave the post he has made his own, and, he feels, it is time for a reckoning, of sorts.

As he sits on a pale sofa, clad in a pale jumper and blue striped shirt, pale chinos and scuffed loafers, it is a different "Sir Tim" to the one that the public is used to. This is not the slightly exaggerated art connoisseur seen when the National Galleries unveils its latest purchase - running from painting to painting in a breathless treatise on the links between each and every work of art. Nor is it the effervescent, enraptured academic seen during his television appearances, explaining why one Renaissance masterpiece is inspired by another in his trademark theatrical style.

That Clifford is something of a front, he says.

A "simplification", a mask that has stood him in good stead these last two decades. Today this is the Clifford no-one in the public sphere sees, and it is quite disarming: softly spoken, albeit with that ice-cutting public school accent, modest, restrained, obviously moved, sometimes to tears, by art, but generous with his knowledge and never patronising.

He doesn't look 60 and is fit and healthy. His hair, though silver, gleams, and his eyes are clear under those black, beetling brows. "I've been working like a Trojan getting the place tidy and ready all morning, " he says, "but it is so lovely to have you here."

Earlier, as he scuttled around his kitchen, trying to make a simple cup of coffee, he seemed rather confused and flustered. He mistakes coffee beans for crystals of brown sugar. He can't find the dratted coffee filters.

And where is the china? You suspect he is not a domestic god.

His wife, "Lady" Jane, is in Italy with four female friends on a bridge-dominated break.

That sounds like fun, I venture. "More fun if it was me and four women, " he says, laughing and pouring coffee, minus filter, into a small china cup. The coffee swirls with black lumps.

"Oh no, " he says, devastated. He filters the grainy coffee through a tea-strainer. Clifford is a fan of perfection, and this won't do. "I'm terribly worried about the coffee beans, " he says and disappears again.

As he searches for something, the kitchen's sumptuous art slowly reveals itself. Over there is a lovely portrait by David Hockney. In another corner is a striking self-portrait by Peter Howson from 1982 - "Peter at his best, I think". A small John Bellany piece leans against the tiled kitchen wall, and, in the hall outside hangs "just a little Renoir". Even in the kitchen and the scullery, art permeates and dominates Clifford's life. "I think people get a better idea of me when I'm at home, " he says.

"Art is everything in my life and it doesn't stop when I stop working. It is everything."

He is quiet, more free with his words than he has ever been in public. In an unguarded moment Clifford once said he had to manufacture himself as a figurehead. So is all that effervescence, that drama, merely for show?

"You do have to do that a bit, " he admits. "You have to simplify yourself, to package yourself so people can understand what you are about.

Obviously a lot of what I am telling you now I have not told anyone before. Because it is simply too complicated. I suppose the most important thing you can do is get across to the public that the visual arts are absolutely fundamental to your life, and hope you can share your own enthusiasm with them."

He admits that some in the arts world - especially academics - have a dim view of his multi-faceted enthusiasms and expertise. "A lot of people see me as superficial, rather than knowledgeable, " he says. "They don't know the private me. They don't realise I'm working on these things all the time. I do not naturally fit into the university world in Scotland, nor the university world anywhere. I am very interested in butterflies and moths, and I might get a reputation for being a dilettante and fluttering from flower to flower.

"But the point about it is this: unless you build a holistic view of things, you miss so much of the understanding of the individual works of art. I think working in a narrow area is impoverishing."

The driveway to Tyninghame House is appropriately inspiring. In the bracing chill of an early autumn morning, the stands of parkland trees and fields of neatly stacked bales of straw shimmer as if in an Impressionist painting. The long drive curves to a coralcoloured minor palace: Clifford's home for nearly 20 years. To the front there is a swathe of closely cropped green turf, adorned with an obelisk and a view to the sea. To the rear the house looks over oak-studded parkland and the gentle roll of East Lothian.

Clifford clearly loves Tyninghame House - he lives in one part of the redeveloped building - and relishes the lounge, a magnificent tennis-court-sized living museum of paintings, furniture, chandeliers, vintage wallpaper, family snaps and books. It looks like a cross between a royal drawing room, a museum and a family home. You expect a warder to be sitting in the corner, minding the paintings. "Yes, we have plenty of kit, " Clifford says quietly.

A lot of this art is inherited from his father, who collected all his life. He died two years ago, aged 88, and a touching bust of Derek Plint Clifford in the prime of life adorns the dining room. It is clear his son takes after him, in both looks and enthusiasms.

It is a suitably bracing, clear day for such a candid, clear talk. The coastal air is scouring and cold. "I do love this room, " Clifford says as we settle in the drawing room, "but I'm not sure how many people I want to see it. I don't think it would go down very well with first ministers, " he adds with a chuckle. He has no intention of leaving Scotland and his home.

"We'll always be here, " he says. "We cannot really move. We are plugged in here, like lobsters stuck in a pot."

He will, however, spend more time working in his Italian home in the hilltop town of Casteldilago in Umbria - he has already transported thousands of books there. He also hopes it will become the base for a new centre of learning: an academic foundation where his big project - the joint and intermingled history of fine art and applied art - can be studied.

Clifford will certainly not be idle when he finally departs the Scottish scene - there is a book to write and a series of lectures to be made in New York. There is scholarship, research and, no doubt, more dabbling in the art market to come. Which makes it all the more remarkable that this man of learning admits he is dyslexic.

He is not alone: around four per cent of the population is severely dyslexic. The term comes from the Greek and means "difficulty with words". It denotes a range of learning difficulties, from erratic spelling to hesitant reading and misreading words entirely.

Clifford says reading "is not a problem" now, but the issue remains. "I have always had that:

problems with words and with numbers, " he says, reluctantly. "I think you'll find an enormous amount of people involved in the visual arts are sometimes acutely dyslexic."

Figures such as Picasso, Winston Churchill, Auguste Rodin and even Leonardo da Vinci are said to have suffered from the condition.

His tough, seemingly lonely time at school, when he was transfixed by birdspotting and the detail and arcana of the natural world, was made worse by the fact he had no awareness of - or label for - his learning disability. He once signed his name Droffilc Yhtomit.

"No-one had heard of it [dyslexia], " he says.

"It was only very recently that I realised. Also, I have a complete lack of ability with numbers.

I don't know what it is, but I have a complete part of my brain that just refuses to operate. If you asked me now what the number plate of my car was, I wouldn't have a clue. If you asked me to ring home, I wouldn't know what my home telephone number is, or my work telephone, any of those things. I have, after an enormous amount of work, memorised my cash card number."

He is not, to his dismay, "a natural linguist", although he speaks Italian. He is bad at recalling vocabulary and genders, but, adds:

"If you show me any picture, I could tell you when it is painted. I could give you all kinds of historical dates. But it's very odd - my mind just rejects all kinds of other information when it comes in. I think it's like a disability. You compensate. One part of me is very good - and one part of me is completely bloody useless."

The upside to this condition is his remarkable visual memory - the learned "eye" for art that curators in galleries from Europe to the USA revere and respect. "You have to memorise things, because you cannot write them down, " he says. "I have a very, very good visual memory, because otherwise I am so bad." This helped him find a Michelangelo drawing at the Smithsonian's Cooper-Hewitt Museum's archive in New York where it had previously been missed. It has helped him attribute, sometimes controversially, works to artists and find hidden jewels at auctions. I ask if he thinks a recently re-discovered portrait of Mary Magdalene was indeed a Leonardo da Vinci work, as its supporters, including an expert at UCLA, have claimed.

"Oh, that's just bollocks, " he says, smartly.

"It's not a Leonardo at all." He scoffs at the suggestion, as if it's the most ridiculous thing he has ever heard. There's one thing you can say for Sir Timothy Clifford, boxer, orchid fancier, caterpillar expert and dyslexic: he knows his art.

Clifford was once described as a "Regency aesthete who woke up one morning to find himself in the mid-20th century". He hoots with laughter when reminded of this, and thinks back to his first years at the National Galleries. He says the Scottish arts world and administrative system, and the Edinburgh mandarin class that dominates it, "can be narrowing" for the character. All those chattering aesthetes, the lords and ladies, the press, the interested bankers and money men: he felt the pressure of their gaze and their expectation.

He was, and is, resolutely the English aristocrat in manner. He often still appears unworldly - he says that when he first moved to Edinburgh at the age of 38, in 1984, he asked whether his horses could use Arthur's Seat as a paddock. Not surprisingly, he was refused.

"I remember years ago, when I first came to the job, I was told anything I said or did would be repeated over breakfast the next day in Edinburgh. I thought, 'I can't operate like this, ' so I carried on as if I was in London. The thing is, Scotland is a very small society and everyone does know everybody else, and I am sure a lot of things have been said about me - some of them have been correct and some of them have been completely incorrect."

Earlier, he had purred as he perused the programme of Choice, the upcoming exhibition based on the purchases made during his time in the post. The show, at the Royal Scottish Academy in Edinburgh, will be a suitable tribute to the purchasing power of the galleries under him. The Three Graces by Canova, the Venus Anadyomene by Titian, El Greco's Fabula and Botticelli's Virgin Adoring the Sleeping Christ Child will all be present.

"I never stop thinking of so much unfinished business still to do, " he says. "There is still a number of areas in the collections that I would like to have strengthened."

He has one major regret: the failure of his dream of a national gallery for Scottish art, which he wanted to open in Glasgow. For a time, he wanted it housed in the Post Office building in George Square. The idea fell apart in the mid-1990s, shortly after another disappointment, when he lost out on the directorship of the Victoria and Albert Museum in London. He once described the latter blow as a "public humiliation".

"I wanted to make a gallery in Glasgow, and rather like Bloody Mary had Calais written on her heart, I will have the Scottish Gallery written on mine, " he says. "I feel terribly sad it hasn't happened. Hopefully my successor or my successor's successor will grapple with that. It has got to happen, got to happen."

He has other scars. He has often found himself the target of criticism, not so much recently, but in the earlier years of his reign.

He received bad press over the otherwise laudable shared purchase of Canova's Three Graces (when he suggested that John Paul Getty II was helping fund the deal because of a grudge against his father who had sought the sculpture for his own museum in California) and for his re-hang of the National Gallery.

The opulence he brought to the gallery was even compared to a high-class brothel. Such criticisms obviously hurt.

"I had a very rough ride early on, " he says.

"I think they [the press] did not understand me. I also think that there was also a fear of failure. There is a fundamental thing about people in Scotland, that they are fearful that people are going to sneer at them and think they have done a bad job. And I think that's really why the Scottish Gallery didn't work - the confidence was not there to think, when the gallery opened, it would be, 'Wow.' They thought they were going to be laughed at. The truth is it would have been fabulous."

He adds: "I suffered greatly because of the Scottish Gallery. I also suffered from being English - but then, you cannot change that."

But, he says, he tried to not let any criticism hurt him. "You don't do this job if you are going to be hurt by things, " he says. "I just wish we had the Scottish Gallery, because right now we would not only be talking about the Playfair Project, but a GBP63 million building in Glasgow and people would be saying, 'Have you seen those wonderful rooms full of Raeburn, Ramsay, Wilkie, McTaggart?' I believe you need to dignify Scottish art and to dignify it properly, it needs its own place."

Clifford, it seems, is an ongoing art project himself, a self-made accumulation of knowledge gleaned about the most disparate and eclectic enthusiasms. As he walks around his home, one moment he is explaining the intricacies of a plasterwork in his dining room and the Norwich school of painting, the next he is describing the life cycle of the caterpillars that live on the lime tree in the grounds.

He is one departing artwork Scotland will miss, although it barely realises that yet. He is both a throwback to a more patrician age ("a fossil, " he once admitted) and an idiosyncratic one-off. It is unlikely another director of galleries will last for 20 years.

His main aim when he retires is to write a history of the links between fine art (painting and so on) and applied art (design, silverware, ceramics and so forth). Asked how long this will take, he replies: "Oh, the rest of my life."

To explain how big this job will be, he shows me his library. Reached by a plain stair from his drawing room, its size is remarkable. In a series of cramped, cluttered dark rooms lined with packed bookshelves, there are thousands of files, box files, cabinets, books, volumes, papers and packets of papers. There are file after file of Clifford's own cuttings, photocopies of designs, diagrams and drawings. He has the sales and logbooks of ceramic companies, or potteries, painters and poets.

The whole collection is bewildering.

On the wall hangs a heraldic family tree of the Cliffords - they originally came to Britain from Normandy with William the Conqueror.

There have been a lot of fighting Cliffords over the years. He points to the family tree and says, laughing: "They were all slain in battle against the Scots."

Now, with his departure from office, there will be more time for his own family. He is a grandfather now. A year ago, his daughter, Pandora, an actress with whom he shares an affectionate relationship, gave birth to Mungo.

"He is an extremely sweet little fellow, " Clifford says, admitting he cannot quite remember when his birthday is.

We walk outside and Clifford shows me his garden: the sweet smelling roses, the parkland beyond. At the bottom of the house's garden is a final curio: a Norman church, ruined, but kept in trim. I ask whether he is religious.

"I do keep thinking of converting to Roman Catholicism, " he admits. "I am continually on the brink. I do love all the pomp and the circumstance of it. I do get moved by religion, and religious art, there is emotion there. But then I think about the religion, and the truth behind it, and I dare to apply my mind to it - and that's where it falls down. And that is when I tend to draw back from the brink."

We walk back to the house and Clifford starts work again - on a lecture he must deliver the following day, on the preface to his new catalogue, on a draft of a press release and on even more plans for the future. The work, the thinking, the exploring never stops.

Choice opens on November 2 at the Royal Scottish Academy, Edinburgh.