What exactly has Ken got against Brian Haw?

Recognising me, one of Mr Haw's support team, a rather intense young American, asked whether Dr David Kelly had been murdered. When I said no, she immediately got on a megaphone to denounce me to the rush-hour traffic as an Establishment sellout.

Mr Haw, inasmuch as I could hear him over his colleague's heavily amplified shrieks of "bourgeois criminal", can be heavy going, too. It's pretty hard to get a word in edgeways, and there's not much love lost between him and the capitalist press.

Yet though Brian Haw may be obsessive, the view of Iraq he represents is shared by the majority of the British people. And though he may be paranoid, that doesn't mean the authorities are not out to get him.

As Gordon Brown and Jack Straw today launch a consultation on "democracy", including perhaps, we're told, relaxing the ban on demonstrations near Parliament, you'd think Mr Haw might be a model for them.

The demo ban has been one of New Labour's most symbolic, and most heavily criticised, attacks on liberty - leading, among other things, to the arrest of a woman for reading out the names of the Iraq dead at the Cenotaph. Mr Haw, in situ before the new law arrived, escaped it on a technicality and represents the slender thread of continuity in a British freedom, the right to peaceful protest, that the Government didn't quite manage to stamp out.

Yet even as Gordon and Jack tell us how committed they are to liberty, Mr Haw - and others who want to oppose Labour - are facing a new, subtler but arguably nastier attempt to shut them down. You may not be totally astonished to learn that it comes from that celebrated progressive the Mayor of London.

Mr Livingstone controls Parliament Square - indeed, under section 380 of the Greater London Authority Act 1999, he appears to have the same power to permit or refuse demonstrations that was granted to the police only in 2005. And for all his alleged opposition to the Iraq war, Ken has over the past two months been orchestrating an unpleasant campaign of smears and harassment against the Parliament Square peace camp and Mr Haw personally.

In August, large numbers of GLA wardens and private security guards employed by the Mayor forcibly tore down the tents of the camp and evicted the occupants, apart from Haw, before putting up seven-foot metal fences around Parliament Square. When something far less than this was done by the police, in 2005, there was a massive row but this time, because it was Ken, he seemed to get away with it.

In a press release about the raid, City Hall claimed, ludicrously, that the peace campers were causing a "growing impact on the health of [GLA] staff" and accused them of turning the square into an "open-air public toilet", something the campers totally denied and something I certainly saw no evidence of when I visited just before the evictions. Haw and co say they use the public toilets in Westminster Tube station, although according to Jenny Jones, a member of the London Assembly, the Mayor's security men often try to prevent them from using these facilities as part of the harassment.

Last week, the fences came down - but Livingstone's campaign stepped up. First in The Guardian, then in other newspapers, including the Standard, anonymous GLA officials briefed that Mr Haw had been "cautioned" for supposed "racial abuse", "bullying" and "intimidation" of the authority's security men.

I can't help thinking that it is rather difficult for one man to "bully" or "intimidate" 15 uniformed security guards. The "caution", rather than being the official police sanction that the term implies, turns out to be no more than a letter from some GLA bureaucrat warning Mr Haw to mind his Ps and Qs.

As for the alleged racism, I wouldn't rule out the possibility that heated words were exchanged across the fences. But Haw is no racist. As he put it himself last night, his whole purpose in being in the square is to protest at the British Government's attacks on people of colour in Iraq.

Smearing his opponents as racists and fascists is one of Livingstone's favourite, and most unpleasant, tactics, and has been deployed by the Mayor and his supporters at least three times in the past two months alone.

As well as Haw, Boris Johnson has been on the receiving end. And on LBC radio last week, Ken used it to describe the taxi drivers, black and white, who oppose his bizarre plan to spend £2.2 million encouraging non-English speakers to become London cabbies.

Racism is one of the most serious charges you can make against a person. To fling it around with no cause other than simple disagreement invites obvious parallels with the way Senator Joe McCarthy brandished the charge of communist.

It is a tactic to intimidate, a strategy to stifle debate and the clear hallmark of an authoritarian. And worse, using it against mere opponents devalues it in the eyes of reasonable people and lets the real racists off the hook.

Yet the fact is this. Even if Haw were a racist, which he is not, or a bore and a nuisance, which he might well be, that does not invalidate his right to protest. In a free society, nobody can decide which demonstrations are "acceptable" and which are not. The only exceptions are for protests which use, or incite, violence.

The Mayor's campaign against Haw is another example of his betrayal of progressive values. Let us hope today that Brown and Straw, at least, realise that if rights are to mean anything, they cannot simply be for people of whom we, or even Ken Livingstone, approve.