When Patti Smith first met the celebrated Beat poet Allen Ginsberg over a cheese sandwich in New York in the late 1960s, she was a mother in name only and not yet the godmother of rock legend.

It was November 1969 and she was 22. She had given up a child for adoption two years earlier, and the son and daughter she would raise with musician husband Fred 'Sonic' Smith wouldn't be born for another 15 years. Meanwhile, her "godchildren" - the generation of late 1970s musicians who would come to view her as the high priestess of punk - were still in school. Or should have been.

So yes, Smith was well-read and clued-up on Rimbaud, Genet, Kerouac and the rest. Yes, the scruffy beatnik girl from New Jersey had suffered a few hard knocks even before she was out of her teens. But, as the title of her 2010 memoir Just Kids implies, her adult self was still a work in progress.

"I was a young girl. I dressed sort of boyish. I had a long coat on and a cap. I guess I had a very androgynous way," she says of that first meeting with Ginsberg, who was then in his early 40s and a vocal opponent of the Vietnam War.

"I was in this automat really hungry and I didn't have quite enough change, and I heard this voice offering to help. I turned round and it was Allen Ginsberg. Of course, he didn't know me - I was just a girl that worked in the bookstore, who lived at the Chelsea Hotel - but I knew who Allen Ginsberg was. I couldn't even speak, I was so shocked."

Ginsberg, openly gay, mistook the angular Smith for a young man, which may have been the reason for his largesse. He only realised his mistake when she opened her mouth to speak. But he sat with her, and the pair talked.

"It was very funny," she recalls. "Even when we said goodbye I never expected to meet him again. But I did - I wound up becoming friends with William Burroughs and [fellow Beat poet] Gregory Corso. Then I met Allen again and we read poetry together, and I reminded him of the story and he actually remembered. We laughed so much about that."

Smith and photographer Robert Mapplethorpe, Smith and poet Jim Carroll, Smith and punk rock, Smith and Smith: the stories of these relationships have been told, celebrated and picked over in the decades since Horses, Patti Smith's incendiary 1975 debut album. But it's her long relationship with Ginsberg, who died in 1997, and the mutual friendship they shared with minimalist composer Philip Glass which is the focus of the show the 66-year-old brings to the Edinburgh International Festival this month.

That's why, over a phone line to Milan, she's back in Ginsberg's orbit, feeling his pull, talking about his love of the blues, her days watching beside him as he lay dying, the oration she read at his funeral - and the time he bought a hungry young wannabe a cheese and lettuce sandwich.

The show is called The Poet Speaks and it's a mouthwatering prospect. As Glass accompanies her on the piano, Smith reads a selection of Ginsberg's work, among them his great anti-war poem Wichita Vortex Sutra. She also performs one or two poems of her own as well as some songs - Ghost Dance, perhaps, from her celebrated 1978 album Easter, or People Have The Power, from 1988's Dream Of Life. Or maybe 2004 song My Blakean Vision, "because Allen loved William Blake".

Behind the pair as they perform, a visual collage is displayed, some of it drawn from Smith's own collection of Polaroid portraits. "Because his poems are so demanding I sometimes forget there's visual material behind us," she says. "Sometimes I'll turn round and be surprised to see Allen naked, or William Burroughs's face."

At other times she may lose her place in Ginsberg's complex verse and be forced to improvise or, in her words, "just rap on the poem" until she finds her way again. During one performance of Footnote To Howl, an adjunct to Ginsberg's seminal 1955 poem, her sheaf of notes fell on the floor and the papers scattered. Cue more rapping.

"Philip loves when these things happen," she laughs. "He can be very studied for certain things he does, but he loves nothing better than when we're on the edge of the tightrope and things seem a little shaky, and then we pull it through."

That's not to say there's anything ad-hoc about The Poet Speaks. Smith and Glass have been performing it sporadically for some years now and have the sort of onstage understanding that doesn't need much rehearsal. Moreover its genesis lies in Glass's own work with Ginsberg, most fully realised in his 1990 opera Hydrogen Jukebox. The title itself is taken from a line in Howl and consists of Ginsberg's readings of his own verse set to Glass's music.

"One of the reasons we developed this was that Philip was heartbroken at never being able to perform Allen's poems again," Smith explains. "They're difficult poems and he couldn't imagine anyone doing them. And when he asked me to do one at Allen's memorial I had trepidation. I wasn't certain I could pull it off. But I found a way to slip into his words and Philip was so happy. He liked our chemistry so much that he really invited me to step into the place that Allen had had with him. So it's quite an honour. What I'm doing with Philip is what Allen did with Philip."

Of course Smith is no slouch when it comes to performing verse. She may be best known as the high priestess of punk, but in the early 1970s it was on New York's performance poetry circuit that she sought to establish herself. She lived in the Chelsea Hotel with then-lover Robert Mapplethorpe, acted in the play Cowboy Mouth (a two-hander she co-wrote with another lover, Sam Shepherd: the production notes call for a woman who "looks like a crow"), and read her own work at places such as St Mark's Church in the East Village. There, in February 1971, she performed in front of an audience that included Ginsberg.

"I know it sounds irreverent but I had no fear as a young person. I'm not saying I was a great poet but I had a lot of guts and I had no fear of Ginsberg or Burroughs or any of them. But I was absolutely proud to perform for them," she says of those early appearances.

"They all liked me. I was very, very lucky. They all saw something in me even though I was quite raw, quite the diamond in the rough - and you have to put the emphasis on rough. But they all took me in hand and taught me things about poetry and we did many readings together. I read with William in Amsterdam and in a lot of other places, and Allen and I did a lot of anti-war work right up to the end of his life."

Ginsberg died on April 5 1997 as a result of liver cancer. During his last days his friends, among them Smith and Glass, kept vigil in his apartment. At his funeral Smith read one of his poems. Familiar as she was with his work, she realised it was the first time she had ever read his words out loud.

But even at the end, Smith says, Ginsberg was showing the span of his interests or, as she puts it, "the multitudes of personas that Walt Whitman speaks of". As an example, she tells me about the last record Ginsberg played. It wasn't the jazz with which he and the Beats are so closely associated, but a simple blues holler by Big Mama Thornton.

"Allen really loved the blues," she says. "He had stacks of blues records. And the day he died, when he knew he was dying, he went into his collection and found the record with See See Rider on it and that was the last record he chose to listen to. Despite the complexity of his work, at the end of his life he was reading poems for children and listening to See See Rider. So Allen embraced everything."

And now, through his words, his old friends can embrace him.

The Poet Speaks: Homage To Allen Ginsberg performed by Patti Smith and Philip Glass is at the Edinburgh Playhouse on August 13 at 8.30pm as part of the Edinburgh International Festival, www.eif.co.uk. Patti Smith and Tony Shanahan perform An Evening Of Words And Music at Oran Mor, Glasgow on August 12, doors 7pm, www.oran-mor.co.uk