Tories embroiled in fresh race row after peer's controversial 'nigger in the woodpile' remark

Lord Dixon-Smith, the Tory spokesman for communities and local government, referred to a problem with a new government housing agency as the 'nigger in the woodpile'.

The phrase, which means something looks suspicious or wrong, described fugitive slaves who hid in piles of firewood as they fled persecution in the American Deep South in the mid-19th century.

David Cameron has come under pressure to dismiss Lord Dixon-Smith, 73, as it comes less than a month after Boris Johnson, the Tory Mayor of London, was forced to fire a senior adviser who said that Afro-Caribbeans who were unhappy with his election could 'go home.'

Lord Dixon-Smith has aplogised for his controversial comment but Tory leader David Cameron has come under pressure to dismiss the peer

Mr Cameron said last night that the remark – which is recorded in Hansard – was 'not appropriate', but he refused to dismiss him.

Instead Lord Dixon-Smith went twice to apologise to Lord Strathclyde, the Tory leader in the Lords.

He made the remark during a debate on the Housing and Regeneration Bill, which merges some functions of the Housing Corporation and English Partnerships, creating a new Homes and Communities Agency.

Lord Dixon-Smith was outlining his objections to the new agency in the debate, and was agreeing with the objections of Baroness Hamwee, the Liberal Democrat planning expert.

He said: 'The Homes and Communities Agency is not a body to which we object in principle.

'As the minister has explained, it is an amalgamation of the Housing Corporation and English Partnerships.

'Of course, the nigger in the woodpile, as the noble Baroness, Lady Hamwee, has already pointed out, is that it still incorporates what I call the hangover of the new towns legislation.

'If it were not for that, we would have little difficulty with the foundation of this agency.'

Another Tory peer, Lord Brooke of Sutton Mandeville, intervened after Lord Dixon-Smith had finished and suggested that he reconsider his choice of words.

He said: 'My Lords, before my noble friend sits down, he used a phrase about a woodpile.

'If your Lordships’ House were happy, I think it would perhaps be helpful if the wording of the phrase were revised.'

Lord Dixon-Smith replied: 'I apologise, my Lords. I left my brains behind. I apologise to the House.'

He then went twice to apologise to Lord Strathclyde, the Tory leader in the Lords.

He said that the remark had 'slipped out without my thinking' and said: 'It was common parlance when I was younger, put it that way.'

He emphasised that he now considered the matter closed, but this prompted a wave of condemnation across Parliament.

Keith Vaz, the Labour chairman of the Home Affairs Select Committee, said: 'I regard this remark as racist, because it’s deeply offensive.

'It shows a lack of understanding and sensitivity to the ethnic community and seems to come from a throwback age when people used that kind of phrase as if it was normal.

'We will judge Mr Cameron on how he responds.'

Lord Dixon-Smith, 73, a Eurosceptic farmer from Essex, has courted controversy before.

In March he said that Britain 'might have had rather fewer problems' if there had been less immigration.

In November a Tory councillor in Bedfordshire resigned after using the same words.