Interview: Andy Goldsworthy – the world in his hands

It's already collapsed once. Last evening. He sits now looking at it, taking in the line where stone meets wood. "Yesterday was sunny and it was really difficult," he tells me in a voice still marked by his Yorkshire childhood. "It's hard to see the join and the line and I messed up. I was quite happy that it did fall down."

His hands. On the drive down to Penpont I'd been thinking about what they might be like and remembering my father's. My dad was a bricklayer. His hands were coarsened, all hacks and hardened skin. As abraded and abrasive as the material he worked with.

Goldsworthy's though. I guess I had expected something similar, but no. He reaches his hand out to me when I ask. There's a black nail but otherwise the are clean and smooth, as smooth as the whinstone at his feet, the whinstone he spent yesterday morning digging up from where the farmer had left it when ploughing the field maybe years before.

They do roughen up when he's working on stone, Goldsworthy says. But he usually wears gloves. He examines the hand I've just touched. "They're not too bad."

Is he two hands attached to a brain or a brain attached to two hands, I wonder? "The relationship between the hand and the brain is really important," he says. "Jacob Bronowski said the hand is the cutting edge of the brain. And it is. There's a huge difference between just looking and touching. Touch feeds you. It takes you beyond just looking. Looking is important but that contact, that physical friction between hand and material is what provokes and generates sensations, feelings, ideas that are really important to making the mind work."

How does his mind work? Andrew Marvell's line "a green thought in a green shade" might sum it up. With apologies to Richard Long, Goldsworthy is Britain's most famous environmental artist. He has reshaped the world around us by reshaping the materials of that world. The results range from the monumental – like the Striding Arches, a handsome semi-circle of stone at Cairnhead on the Southern Uplands, or his early signature work of taking walls for a walk through the landscape – to the evanescent.

He has now published a book dedicated to the latter category. Ephemeral Works is a photographic archive of a decade of art between 2004 and 2014 made from ice, ferns, twigs, rain, muck or moss. Here are ice walls in Dumfries, rain shadows in Times Square, New York.

It's his first book in eight years. To get him to commit to the project his publisher Abrams gave him total control. And so there is no title on the cover, no publisher's logo and very few words inside. If he'd had his way there wouldn't even be a barcode. There is, but it's on a removable belly band.

It's the only compromise he allowed. "I've taken everything superfluous out," he says. "All that writing on the flaps that says you're the best artist ever. All that bullshit that goes on books …"

You don't like being called the best artist ever then, Andy? "I don't. I don't think it's necessary to sell the book."

I suppose you can always know it in your heart, I say. He barks a laugh. "It's irrelevant. It's boasting. No, it's not right."

What he certainly is, these days, is a success. Goldsworthy moved to Scotland from Cumbria in 1986, after his former wife got a job in Carlisle and they moved across the Border. His four children grew up here.

"It was quite cheap. Expensive to me, but I could just afford to buy the studio and house here. I came here because of economic reasons and now when I can live wherever I want, I choose to live here because people have been very tolerant of me."

Does he feel Scottish now? "I think I'm an amalgamation. My children are fully-fledged Scots and I love being here. I feel a commitment. I don't know whether I feel Scottish, Yorkshire or anything."

During his first eight years as an artist he didn't earn enough to file a tax return but that has all changed. A couple of days after our meeting, he will head off to work in America. How does he make his money? "The main source of income is private commissions and some public ones. I'm very fortunate that there are a lot of people who commission work from me. Obviously they [the works] can't be moved. They're not done for investment. They're done out of a deep love of the place."

A love of place is clearly something Goldsworthy feels too. His work often attempts to make us look at the landscape anew. He gets a little annoyed when I suggest he's mainly a rural artist. "That's a slight misunderstanding," he says. "Whilst I do like the landscape a lot of the work happens in cities."

He reminds me that he's lain down in Times Square to create rain shadows on the ground. "Nature doesn't stop for me at the city. The difference is I don't have that same sense of lineage of materials there that I do here. There's a connection to the place whereas in the city it's more complicated. It's interesting."

What do people say to him when he's lying on the ground in the rain in Times Square? "Oh Christ. I'm a very private person so I can't say I enjoy it. I just zone out. The first one I did in Times Square I lay there for 15 minutes thinking, 'This is amazing, this is going to work', and I was just about to get up and I looked up at the camera and the wind blew it over. Then this person said, 'Are you Andy Goldsworthy?' And I'm in this state of shock having this work ruined and she's wanting her friends to take a photo of me with her."

The coffee finished he picks up a stone and returns to work on the tree. Who's going to see this? "The farmer comes here most days. The odd walker."

Does the work have to be seen? No, he says, "The ephemeral work is done in the spirit of trying to learn to look at what's there. It's a way of finding out about the world around me and exploring it."

But what is that world? Here at the start of the 21st century our relationship with the landscape is deeply dysfunctional. On one hand members of the Green party are no longer dismissed as cranks and nature writers such as Robert Macfarlane are in the bestsellers chart. Yet fracking is being pushed as the next energy source and at government levels we are in deep denial over climate change.

Goldsworthy's work is seen in that nexus. He has his detractors. He is guilty of sins of commission or omission, depending on the critic. Either it's been dismissed as a twee tidy-up of nature's unruliness – the "meddling humans" line of attack, you might say – or condemned for failing to acknowledge the mess we've made of the place – the "where's the meddling humans?" line of attack.

Goldsworthy, though, comes at the subject from a different angle. "When I first started, human involvement with the landscape was a negative thing. 'Whenever we touch it we mess it up.' And that's not the case. We can leave the place richer for our being here. Ultimately the big danger is we don't see ourselves as nature. For me that's a very dangerous position to take. And it's a very dangerous position to think that the city is not nature too."

He goes on a mild rant about how these days, people on planes are too busy staring at screens to look out of the window. "Isn't anybody interested in where we've just left or where you're going or the process in between? The wonder of seeing the world below us? It perplexes me enormously."

At another point he talks about an installation he did which involved building a huge wooden structure inside a glass house. "There was a lot of anger for me using these trees." Yet he didn't chop the trees down. That had already happened. "They were on their way to become MDF or paper. Now if I made a work out of paper or MDF no-one would have got angry.

"We have to accept that we are part of the landscape and we affect the landscape through what we eat, what we wear, what we build. So we are connected. The danger is that we think we are not. We are bound up with the land; we are part of it. Now that's not a message I'm trying to preach. It's just something I understand through my work."

That couldn't be clearer where we are today, he says. "I like to work on other people's land. It's very different making this here in the farmer's field than in my own field. You've got the farmer to deal with and that makes you aware of the social history of the landscape. The stone that I'm working with has been taken out of the field by farmers and dumped over the side. So it's already had the hand of a farmer on it and my hand is another hand on his. These are important connections to the material to the place.

"It's really difficult in this country. The reaction to making contemporary artwork seems to provoke such a response in the press and among locals. I welcome a debate about a work. I'm really happy to go and answer questions and talk about it, but sometimes controversy seems to be generated to sell newspapers."

As if, Andy.

The question I want to ask about the work, is how it reflects not just geography but biography. His biography. Can he see his own life writ large in the work? "Absolutely. There's a work in the burn here that I made for my mum after she died. These events are written into it, all of those times are written into the work. The kids have always come out and seen me when I've been working. Oh they are definitely in there."

Andy Goldsworthy was brought up on the outskirts of Leeds. His father was a mathematician, his mother a housewife, though she worked in hospitals from time to time.

He always knew he wanted to be an artist. He has often told of how he started working for a neighbouring farmer in his early teens. "Farming was a very sculptural thing to do," he says now. "Every time you plough a field, lay a hedge, build a wall. It's a visceral sensation." He talks of "the brutal experiences" of farm life. "This isn't all pastoral cuteness. There's dead animals around and decay and shit and stone and hard work and sweat."

All of this fed into the artist he would become. Eventually. "I had these experiences and I didn't really tap into them until I went to art school because I had this conventional idea that art was paintings."

But that began to change. "I remember feeling dissatisfied with being inside and wondering why I was there. You're taught that art is this vehicle for self-expression but at 17, 18, what have you got to express?

"I was at art college in Lancaster and lived in Morecambe. One day in first year I went out to the beach and dug things, made lines, and the tide came in and washed it away.

"I learned more about the tide, the sand, the texture, I learnt so much in that couple of hours. And I shifted to working outside. I didn't really go back in again."

Goldsworthy's partner Tina arrives with the photographer. We walk down to the burn to take some photographs. He shows me a work he did a few weeks before. Ferns on a fallen tree. He's thrilled it's still in a reasonable state. Broken up but not broken. He shows me where the tree used to stand on the bank. Change is inevitable and, he tells me, not necessarily always diminishing.

"That tree. It's become so rich in its effect on that place. It's difficult to describe it as just the death of a tree because it gives growth to that place. You can't really describe it as decay."

Goldsworthy is 58. Does it bother him that in five, 10, 15 years he might not be able to be out in fields hoicking big lumps of stone about? "I don't know. When it comes to it you're just going to have to suck it and see. You'd like to think you'd be able to respond to your body. The great thing about art is that there is a possibility that your art can get better as you get older. You perceptions change. I would hope there's a depth to what I do now."

Is his art a bid for immortality? "No. But as I've said when you work with stones that have been put there by a farmer who's been working the field, there's a sense of human presence. The things that I'm making have been written into the story of this place. There will always be a pile of stones around that tree unless I move them away and there will be people going, 'Why is there a pile of stones there?' I've left things like that all over the place. They're in the memory."

I leave him to it. His hands pick up a stone. Then another. And another …

Ephemeral Works, 2004 -2014, by Andy Goldsworthy, is published by Abrams, priced £50.