A Safety Promise Is Not a Safety Mechanism

AI companies publish safety frameworks, define risk thresholds and promise that development or deployment will be slowed or stopped if certain capabilities become too dangerous. I would much rather have those commitments than not have them. But we should be careful not to confuse a promise with a mechanism of control.

If the company developing the model also defines the threshold, evaluates whether the threshold has been crossed, carries the financial cost of stopping and ultimately decides what happens next, then society is still relying on voluntary restraint. That may work perfectly well while safety and commercial interests point in the same direction. The interesting moment comes when they don’t, when honoring the principle means delaying a product, losing a lead, writing off an investment or watching a competitor move ahead.

A safety mechanism exists when somebody has the power to enforce the safe outcome even when the company would prefer another one. That distinction matters far beyond AI. Responsible behavior is welcome, but governance cannot consist of hoping powerful institutions voluntarily constrain themselves forever. A promise to stop is not the same thing as somebody having the power to stop you.