AI Builders: Verify vs Trust

I have to admit that I’ve felt my trust in AI builders slipping lately.

The steady accumulation of credible reporting about questionable practices can’t be overlooked.

The list involving OpenAI (none of which have yet been proven true, to be clear) includes questions about whether significant safety incidents were disclosed candidly, whether benchmark results had sufficient context, whether published performance claims were later changed, and allegations involving academic researchers.

OpenAI is not alone, though, when it comes to reporting on questionable practices. No need to recap all of them. They’ve been featured enough.

The larger problem this has now created is how much trust we should be placing in organisations whose enormous commercial interests are expected to diverge from those of the broader societal good.

The reason I’ve felt that trust slipping is the serious information asymmetry.

The frontier AI companies:

• Typically know about safety incidents before anyone else

• Decide what gets disclosed

• Conduct much of the testing

• Choose which benchmark results to promote

• Determine how their models are described

• Possess model information that customers, regulators and researchers may never independently see

• And, at the same time, have enormous commercial incentives to move faster than competitors.

The implicit AI governance model to date has effectively been: Trust the builders.

But that becomes much harder to justify when the same organisations hold so much of the technical knowledge, control so much of the disclosure, and face such powerful commercial incentives.

Lord Acton’s maxim that “power tends to corrupt” has been coming to mind quite often lately.

Not because power inevitably produces misconduct, but because concentrated power without sufficient independent scrutiny creates exactly the conditions in which trust starts to erode.

It feels like the past few weeks may have been an inflection point, tipping us towards a necessary new implicit AI governance model: Verify the builders.

Through:

• Independent testing

• Independent audit

• Clear disclosure requirements

• Transparent benchmark methodologies

• Traceable changes to published performance claims

And governance that does not depend solely on voluntary corporate promises surviving commercial pressure.