Next Monday, the winner of one of the biggest art prizes in the country will be announced: the 29th BP Portrait Award at the National Portrait Gallery, which also mounts an exhibition of 55 of this year's 1,727 entries from established artists and newcomers alike, of all ages and from all over the world. The winner pockets £25,000 cash and an NPG commission. The sum may be equivalent to the Tate's Turner Prize, but its value to those concerned is arguably far larger.

Whereas the Turner selects artists at the top of their game, all four names on the BP Portrait shortlist — Simon Davis, Peiyuan Jiang, Robert O'Brien and Craig Wylie — are unknown to the wider public, and this year younger, too, ranging in age from 24 to 40. The show attracts record crowds, last year drawing 200,000 visitors, many of whom never enter a museum or art gallery the rest of the year yet each season flock to this most human of exhibitions: a parade of life in all its guises, where it's not rude to stare at oddness, beauty, frailty, old age.

“It's intimate and alive, fascinating but not daunting. That's one of the central elements of portraiture,” observes Sandy Nairne, 55, director of the NPG since 2002. During that time he has also chaired the Fourth Plinth committee, a role now relinquished in favour of the Institute of Contemporary Arts head, Ekow Eshun. Nairne recently had several spats with Boris Johnson over public art but advised the Mayor over the decision, announced last week, that the scheme will continue. “I'm relieved that the idea of commissioning new work for the site will go on and I hope its ambitions will be international as well as national.”

A fiery-eyed proselytiser as well as a muscular ex-Oxford rower who fills a room, Nairne comes from an artistic dynasty with a strong sense of public duty, handed down by his father, a top Whitehall mandarin who is also a skilled watercolourist. Sandy Nairne is the oldest of six children, five of whom are actively engaged with art. His younger brother Andrew runs Modern Art Oxford, where Sandy himself started out alongside Nicholas Serota in the 1970s. After working at the Tate, the ICA and the Arts Council, Nairne has found his ideal niche at the NPG.

“There's a fantastic sense of engagement with paintings of the human face,” he says, “ of entering a private world, just you and the sitter, seeing someone — it might be Shakespeare, or a monarch, or someone's unnamed girl-friend or ageing parent — exactly as the artist did. A portrait can be an incredibly direct expression of human relationships: of how we see ourselves and others see us.”

Britain was the first country to have a gallery devoted to portraiture, which opened in 1856 and moved to its current St Martin's Place site in 1896. There are only nine such galleries in the world, including Scotland, America, Australia and Canada. By its nature the form, like landscape and still-life, tends to be thought traditional, even in recent examples by artists not considered conservative, such as Lucian Freud, David Hockney or Gerhard Richter.

The reasons are obvious. Among professionals, portrait commissions often subsidise the rest of an artist's more precarious output. If a rich patron pays a high price to have his wife painted, he may demand his money back if her ears and eyes are not in the usual place. (Before anyone mentions Picasso, except as a young man none of his portraits — of wives, children, mistresses, friends — was commissioned in the normal sense.)

But in the age of digital photography, who needs conventional painted portraits? Most of us can now take a good snap on a half-decent mobile phone. For most people the most flattering remark to make about a portrait is “it really looks like him/her”. Three of this year's shortlisted BP artists work in minutely observed photorealist style — a recent and some think unfortunate, unpainterly trend.

“A photograph captures a transient moment, whereas a painting can be a summation of the whole person, as perceived by the artist. But of course we have to ask all the time, what is the purpose of the National Portrait Gallery?” Nairne agrees. “Brian Sewell asked that very question in the Evening Standard recently and decided there wasn't one, for reasons I find very peculiar.”

Sewell is not alone. An influential art blogger has called the NPG “a temple of middle-class moronism”, where celebrity comes first, art second. Sewell's article argued that the NPG's collection of 330,000 works is “hum-drum”, that “nine times out of 10, the painted portrait is a bore” and that the contemporary works are “of precious little substance”. His solution was to abandon painted portraiture and opt instead for photographic material “culled from television newsreels”, the emphasis firmly on archive rather than aesthetics.

“In fact we've got plenty of video and multimedia art,” Nairne protests, “including Sam Taylor Wood's study of David Beckham sleeping and lots in the Sporting Lives: Contemporary Portraits of Athletes and Olympians opening next month, with an amazing video of Duncan Goodhew swimming.” The NPG collection contains 220,000 photographic images. Several examples of computer-generated art already exist and next Monday a new one, a self-portrait of Julian Opie, winking slowly on a plasma screen, will be unveiled. Way back in 1973, having no access to his “sitter”, Warhol made his silk-screen prints of the Queen — the woman whose image appears most in the NPG collection — from photographs.

Abandoning painted portraiture may sound an extreme proposition but it already happens, successfully, in the mixture of media favoured by the National Portrait Gallery, Washington. The difference is that in America painted portraiture is all but dead. The vital iconic images of recent presidents —Nixon at the time of Watergate, George Bush and 9/11 — all come from television footage. An exception is John F Kennedy, depicted by the abtract expressionist Elaine de Kooning, which she was working on at the time he was assassinated.

“Here in Britain there's a living and expanding tradition, as the BP Award proves. In the gallery, we invite commissions from major artists who might not otherwise think of doing a portrait at all, such as Michael Craig-Martin” — whose new portrait of Zaha Hadid will be presented to the gallery in September — “but the reason we need the NPG is that it represents this country through the extraordinary achievements of many individuals and it gives us a path back through history. And it's the only national collection that causes new work to be made. We have six or seven major new commissions a year, chosen by the trustees and curators.” Recent BP Award sitters include several high-profile figures such as JK Rowling, Helen Mirren, Fiona Shaw, Harold McKellen and Kazuo Ishiguro.

Sometimes sitters have strong views. John Gielgud wanted to be painted by Francis Bacon, who refused. Paula Rego accepted a portrait commission only because she knew her subject, Germaine Greer. Novelist Antonia Byatt chose Patrick Heron because she wanted something semi-cubist and abstract because “what I wanted was the presence of the idea of me, not of a record of the whole of my face that I don't much like”.

The waxen clones of Madame Tussaud's should play no part in the NPG where at its best lifelike means alive with all the challenges that art can bring. “Of course, there's lots of terrible portaiture,” Nairne agrees. He and his fellow jurors immediately rejected all but 250 of this year's submissions.

“But when we whittled down our choices, we found some very powerful work. We're all fascinated by faces. That's what celebrity magazines are all about and that's why everyone loves the BP Award, even if some are snooty about it. The greatest portraiture transcends the particularity of place and time.”

It may be that only one or two can rise to this great claim but we are guaranteed to be intrigued and entertained by the endeavour in the BP Portrait show.

The BP Portrait Award show opens at the National Portrait Gallery on Thursday and runs until 14 September. Admission free. www.npg.org.uk.