The British government faces a €2bn (£1.7bn) fine for negligence that allowed criminal gangs to flood European black markets with illegal Chinese goods, EU anti-fraud investigators have said.

The European anti-fraud office (known as Olaf from its French name, Office de Lutte Anti-Fraude) has recommended the UK pay €1.98bn into the EU budget to compensate for lost customs duties, as a result of a failure by British customs officials to crack down on criminal gangs using fake invoices and making false claims about the value of clothes and shoes imported from China.

Olaf said the fraud was ongoing and pointed out the cost to national exchequers was even greater. France, Germany, Spain and Italy are estimated to have lost a combined €3.2bn from 2013 to 2016 in VAT revenues, as a result of British failures in handling imports at its ports.

The €2bn fine and accusations of neglect could stir bad feeling between the UK and the continent as Britain prepares to embark on negotiations to leave the EU, where dispute has already broken out over an estimated €60bn Brexit bill.

Relations between Britain and some member states, such as France and Italy, were already strained over trade. The British government has been in the vanguard of countries defending EU rules that allow artificially cheap (but legal) Chinese steel into European markets, hurting European jobs and companies.

The EU’s 28 member states belong to a customs union, where a common tariff is levied on imports from outside the bloc. About 80% of customs duties are used to fund the EU budget.

Anti-fraud investigators said they had repeatedly warned British customs officials about the scale of the fraud. A spokesperson said: “Despite repeated efforts deployed by Olaf, and in contrast to the actions taken by several other member states to fight against these fraudsters, the fraud hub in the UK has continued to grow.”

In one example given by the anti-fraud team, women’s trousers imported from China were declared in the UK at an average price of €0.91 a kg, although market prices for cotton were €1.44kg, a disparity that failed to raise the alarm for the British.

A senior French anti-fraud investigator said the UK had mostly not responded to requests to trace goods. Bruno Collin, at the French National Directorate of Intelligence and Customs Investigation, told Politico that his British counterparts had not made an effort. “UK authorities are not interested at all in co-operating in this field, probably because the phenomenon does not directly affect them.”

Politicians opposed to Brexit said the findings raised more questions about the government’s plans to quit the EU’s customs union. “The UK border force is asleep at the wheel and it’s going to cost the taxpayer billions,” said Alistair Carmichael, a Liberal Democrat MP. “This doesn’t bode well for reckless plans to leave the customs union and set up border checks for all goods coming into UK.”

The decision about whether to fine the UK will be taken by the European commission.

Asked whether the UK would pay a fine, Theresa May’s official spokesman said: “We’ve not received a bill from the European Commission. This report is not a bill. It’s an estimate. We don’t recognise the figures. HMRC are looking at it now.” The spokesman said the issue was entirely separate from Brexit negotiations.

Her Majesty’s Revenue and Customs said it had an “excellent record in tackling fraud and rule-breaking of all kinds” and was considering Olaf’s findings and recommendations.