White men are becoming an endangered species in some of the most senior positions in business as firms try to recruit more women and people from minority ethnic backgrounds, Tesco’s chairman has said.

John Allan said the latter groups were in a better position now than they have been in the past when it came to non-executive roles, despite concerns that each remains under-represented in boardrooms. He later said his comments were meant to be humorous in nature.

“If you are female and from an ethnic background – and preferably both – then you are in an extremely propitious period,” Allan told the audience at a panel discussion.

“For a thousand years, men have got most of these jobs, the pendulum has swung very significantly the other way now and will do for the foreseeable future, I think. If you are a white male, tough. You are an endangered species and you are going to have to work twice as hard.”

He told the Retail Week Live conference that he stood by his comments when it came to recruitment for the less powerful non-executive directorships, according to that magazine’s own report.

“There is loads of female talent out there, you have got to be prepared to look for it and you need to tell the headhunters that you really want them to serve up the most talented people they can find but to ensure there is a good mix,” he said.

“Tesco have appointed an almost completely new board … in the last 18 months, we now have eight non-executive directors, six of whom are new, three of whom are women.”

There are no full-time female executives on Tesco’s board.

The proportion of female directors among FTSE 100 companies is 26%, while only 10% of executives at those same firms are women. According to reports last year, only 8% of those directors were not white. By contrast, people from minority ethnic backgrounds made up 14% of the UK’s workforce at large.

In 2015, the number of women on FTSE 100 boards exceeded the 25% target set out by Mervyn Davies in a government-backed review in 2011. But figures released last year indicated less than a quarter of the boardroom recruits in the six months to March 2016 were women, the lowest level since 2011.

That prompted the then minister for women and equalities, Nicky Morgan, to say that the “momentum has not kept pace”. She added that there are “still more [people] called John chairing FTSE 100 boards than there are women”.

Speaking to the Guardian on Friday night, Allan insisted that his comments were meant partly in jest. “The context was [that] I was talking to a bunch of aspiring non-executive directors, many of whom were women, and I wanted to give them some encouragement and, therefore, I used that rather colourful turn of speech,” he said. “And the audience, I think, was quite amused and quite enjoyed it.

“It was intended to be humorous, a bit hyperbolic. Clearly, white men are not literally an endangered species but I was actually wanting the make the reverse point, which is that it is a great time for women and people of ethnic minorities who want to get on in business.”