In theory, Randy Newman is one of a generation of great American songwriters now reaching the twilight of their careers.

Like his peers – Bob Dylan, Paul Simon, Leonard Cohen and Tom Waits – at 68 Newman is of an age once regarded as, well, old. He knows it, too.

His latest record, Live in London, recorded with the BBC Concert Orchestra, contains a version of the acerbic I'm Dead (But I Don't Know It) prefaced by a complaint about "geriatric rock" and the fact that "nobody is retiring in rock'n'roll". Given that Newman is popular music's arch-ironist, it's worth considering whether he's being serious or merely tongue-in-cheek. A bit of both, as it turns out.

"There's a lot of evidence that people produce their best work 45 or 50 years before they reach 70," he says. "It's like chess, people doing stuff at 24 that they can't do at 44 or 54." Would he apply that rule to his own career? "I don't think it's true in my case. It's always been important to me for it not to be. I'm not getting any worse, and in some aspects I think I've got better. I think I sing better now, but on the other hand styles crystallise and you can stagnate. I don't know if I'm busting my ass harmonically the way I used to."

It's slightly disconcerting to hear one of the great songwriters state bluntly: "I never liked writing, I would have done a lot less if I wasn't pushed." Although Newman hasn't written a new song for a couple of years and claims that "retirement isn't anathema to me", he has certainly been keeping busy. Much of his energy has been spent curating his back catalogue. Since his last record of new material, the excellent Harps and Angels in 2008, Newman has released the recent live album and two volumes of the Songbook series, in which he revisits his classic songs using primarily voice and piano. Although he claims to be "very hard on myself in every area", the process, he says, has made him more at ease with the broad sweep of something he has always regarded as work rather than art.

"I like what I've done," he says. "I was gratified by the quality of it going back a long time. I thought it was really pretty good. I was real surprised, there's some kind of consistency to it, but I haven't really settled down enough to think, Oh look what I've done. Maybe when I quit, when I'm off oil painting somewhere."

Nowadays Newman is as well known for his film work as for his conventional albums and tours. He is the Grammy and Oscar-winning composer of a multitude of movie scores, spanning everything from Monsters Inc and Awakenings to, perhaps most famously, the three Toy Story movies, the first of which included the inescapably affecting You've Got a Friend in Me. Some of these compositions have found their way into his live set. Does he make a distinction between these film pieces and his other songs?

"Well, they all sound like me," he shrugs. "You've Got a Friend in Me could be about a used car salesman, given the way I write. It's all equal. I never thought pop music was a pedestrian calling, even though my family was writing orchestral music." Three of Newman's paternal uncles were noted Hollywood composers. "They didn't have an open mind about much but they did about music – a song was just as valuable as a score, and I always felt that way. The lyrics in the film stuff are generally more benign and more direct, but I'm grateful to be kept in the middle of the road, or as close as I'll ever get, through doing that. Otherwise I'm just not there. I can't do it, though not because I disdain it in any sense. If I could write the way ABBA did I'd probably do it."

Back in the 1970s Newman looked a little like Woody Allen crossed with Gene Wilder. Nowadays he has a more distinguished, professorial air, but on songs such as A Few Words in Defence of Our Country he proves he can still be as funny as both those men once were. From somewhere near the very beginning of his career, with Rednecks and Short People, Newman established himself as an "unreliable narrator", an ironist and caustic wit unafraid of adopting an unpopular or outrageous position in order to find a new way into a subject.

"I got tired real early of straight ahead I-love-you, you-love-me songs, maybe because I was shy or not as good at it as Carole King," he says. "I thought I could be otherwise: why can't you be a short story writer in song?"

He laughs. "I know the answer to that: because the medium is very successful when it's direct, when people think you're talking to them from your own heart, even though it's bulls***. Some of these people who write songs about the meadows and fields are criminals. But it doesn't matter. Who is going to hear irony when you're driving at 60mph in a car? It's an odd thing to have done, but it's what I chose to do and I'm not dissatisfied."

Newman is nevertheless capable of turning his hand to other emotions. The loose, lascivious grooves of Mama Told Me Not to Come and You Can Leave Your Hat On became hits for Three Dog Night, Joe Cocker and Tom Jones, while he can be as emotionally direct as any heart-on-sleeve singer-songwriter. The man who wrote I Think it's Going to Rain Today, now a bona fide American standard, I Miss You, Feels Like Home and Real Emotional Girl could hardly be said to be all surface and no feeling.

"Paul Simon told me fairly recently that he likes the straight love songs best," he says. "My fans do, too, but it's really not what I choose to do. It surprises me that they'll like a song like Feels Like Home. It's indistinctive, lyrically, but it moves them. I sort of understand that, because I can feel it too, but it's not something I'd like to do all the time. It doesn't exactly bore me, but it doesn't interest me like a story or comedy."

Newman's Glasgow concert later this week will find him in his preferred format: just him and his piano "playing songs from everywhere. I'm thinking of doing some things I haven't done in a while".

He admits that some of the sharper corners of his back catalogue – Christmas in Capetown, for instance, or Old Man – are simply "too rough" for a live audience.

"You make some very pedestrian considerations on stage, which is a more conservative medium than songwriting," he says. "I have a set list but I'll change it depending on the audience.

"Like, say, Munich is different from Hamburg. In Munich they're laughing but they may be pretending; in Hamburg they're not laughing but they like it." He ponders this with some satisfaction. "Always different, every crowd."

Randy Newman plays Glasgow Royal Concert Hall on Friday. Live in London is out now on Nonesuch Records.