The government has been forced to pay nearly £100m in a settlement with two US companies for mishandling the way it awarded a £6.1bn nuclear decommissioning contract.

Ministers have ordered an inquiry headed by the former boss of National Grid to find out why the procurement process was so flawed. Labour said the payout showed “dramatic levels of incompetence”.

The government body tasked with decommissioning old reactors will also terminate the contract it awarded for cleaning up a dozen of the UK’s old nuclear sites nine years early.

In a written statement, Greg Clark, business secretary, said: “This was a defective procurement, with significant financial consequences, and I am determined that the reasons for it should be exposed and understood; that those responsible should properly be held to account; and that it should never happen again.”

The debacle dates back to a 2012-2014 tender process by the Nuclear Decommissioning Authority (NDA) for dismantling 12 sites. They include old reactors at Sizewell in Suffolk, Dungeness in Kent and Hinkley Point in Somerset, where the UK is building the first new nuclear power station in two decades.

A 14-year deal was awarded to international consortium Cavendish Fluor Partnership in 2014, but the bidders who lost out immediately launched a legal challenge.

On Monday, the Department for Business, Energy and Industrial Strategy said it had settled with US-headquartered engineering companies Energy Solutions and Bechtel, for £85m and £12.5m respectively. Had their litigation cases gone to trial, Clark said the “very substantial costs” had the potential to rise much further.

Energy Solutions had claimed a scoring mechanism had changed at a late stage in the tender process and that there was inconsistency in the way bids were evaluated, among other failings.

Former National Grid chief executive Steve Holliday has been appointed to lead an independent inquiry into what went wrong. The inquiry will look at how the mistakes were made and by who, how the litigation was handled, and the relationship between the NDA and the government departments.

Holliday will report to Clark, with an interim report due to published in October.

The government now has the daunting task of starting a new tendering process for the 12 sites, as the deal with Cavendish Fluor Partnership will end early, in September 2019 instead of 2028. Clark said he wanted to stress the deal was “no reflection” on the performance of the consortium, which will continue clean-up work over the next two years.

The contract is being terminated early because the NDA underestimated the scale of the decommissioning required to clean up the Magnox sites, which form part of the UK’s first generation of nuclear power stations.

The share price of Babcock International Group, which has a stake in the Cavendish Fluor Partnership, has fallen by more than 3% since the government announcement.

Rebecca Long-Bailey, shadow energy secretary, said: “By cancelling just two years into a 14-year contact the government has shown dramatic levels of incompetence in the procurement process of this deal.

“British tax payers who stand to lose nearly £100m should be asking themselves not just whether they are willing to put up such ineptitude but also whether the government actually has a well thought out and long term nuclear decommissioning strategy.”

Unions said they were concerned at any potential job losses as a result of the contract ending early.

Mike Clancy, general secretary of Prospect, said: “This is an extraordinary situation given the scale and importance of the Magnox contract to the UK nuclear industry.

“The public, and our members, will want reassurance that the termination process and uncertainty over the future of decommissioning will not lead to standards deteriorating or the loss of UK expertise.”

The Unite union said the financial mess involved in awarding the contract showed the clean-up project should be taken into public ownership.

Stephen Thomas, emeritus professor of energy policy at the University of Greenwich, branded the NDA’s handling of the contract “an immense screw-up.” He added that there would be a further hit to the taxpayer because of the cost of a new tender.

The NDA has an annual budget of £3.1bn, two thirds of which is spent on Sellafield in Cumbria, which stores most of Britain’s nuclear waste.