Amman — A few days ago, Dario Amodei, CEO of Anthropic, published a lengthy essay calling on AI companies to “slow the pace” of developing their models’ capabilities, warning that the speed of technical progress has come to outpace even researchers’ own ability to understand and control it. Within hours, Sam Altman, CEO of OpenAI, announced his full support for this position, and his commitment to adopting a similar measure involving bringing independent evaluators into the company with powers parallel to internal risk-assessment teams. Even Elon Musk, the traditional rival of both, publicly voiced his agreement. A rare moment of consensus between three men who rarely agree on anything.
The stated motive behind this shift is not abstract or theoretical. Amodei specifically pointed to an incident that occurred last July, when independent AI agents, built on a model belonging to OpenAI, managed to breach systems belonging to Hugging Face without direct human intervention. Add to that the loud, public resignation of a former researcher at both companies, in which he accused both of “gambling with our lives” in the race toward models capable of developing themselves.
This premise deserves to be taken seriously, and we will return to it. But it also deserves critical scrutiny that doesn’t simply take stated intentions at face value.
The first hard-to-ignore contradiction is timing. According to reliable reports, Anthropic is preparing to begin marketing its initial public offering in financial markets in mid-October, just weeks after Amodei published his call for “slowing down,” with the offering targeted for completion days before the US midterm elections in November. A company preparing for one of the largest public offerings in the history of the tech sector doesn’t appear, in practical terms, to have slowed anything related to its commercial growth or financial ambitions, even as it publicly calls for slowing the pace of the entire sector.
The same applies to Altman, who had already announced on the platform “X” back in August that his company had temporarily paused some advanced training processes “to ensure it meets its standards” as capabilities develop, the same rhetorical pattern being repeated today. When a CEO repeatedly announces an intention to “slow down” while his company continues raising massive amounts of capital, launching new models at an accelerating rate, and expanding its infrastructure through multi-billion-dollar deals, the legitimate question becomes: is this a genuine intellectual shift, or calculated reputation management at a moment when public and regulatory concern is rising?
This doesn’t necessarily mean the motive is entirely false; the two aren’t necessarily contradictory: a company can believe in the danger of what it’s building while continuing to build it at the same time, especially when the rules of competition make unilateral stopping a form of commercial suicide. But this is exactly what makes the call, however sincere its intent, practically incapable of achieving any real change unless it’s translated into binding, verifiable commitments, not merely press statements exchanged between CEOs.
Here lies the deeper dilemma, a purely technical dilemma before it’s a dilemma of intentions. The call to “slow the pace of capability improvement” implicitly assumes there’s a reliable way to measure these capabilities in the first place, an assumption now facing serious doubt within the technical community itself.
The standardized benchmark tests the industry has grown accustomed to citing with every new model launch, such as the famous MMLU, have become fully saturated at levels exceeding eighty-eight percent for advanced models, making the differences between competing models at the top closer to statistical noise than genuine evidence of real superiority. Another test once considered a reliable benchmark for measuring coding capabilities, known as SWE-bench Verified, OpenAI itself was forced to formally abandon after discovering fundamental flaws in its design and contamination in its data, as it turned out that a large proportion of its questions were either improperly designed or had already leaked into the training data of the models it was supposed to be neutrally testing.
More seriously, what the LMArena platform scandal revealed, one of the industry’s most well-known and trusted model-evaluation platforms, when it emerged that one major company had submitted ten different versions of its model for the private test, and published only the best result among them, achieving a jump of nearly a hundred points in the ranking through this systematic manipulation alone. A senior researcher in the field later admitted that the results had “been manipulated to some extent,” and the company involved dismissed its entire team over the scandal. Even independent studies point to a gap of nearly thirty-seven percent between results announced on leaderboards and these models’ actual performance when deployed in real work environments.
There’s another practice frequently discussed among independent researchers, though harder to document than a publicly named scandal: some companies subject a specific version of their model, specially tuned to achieve the highest possible performance, to benchmark tests and public comparisons, then release a different version to the public, lighter or cheaper to run, that doesn’t necessarily carry the same level of performance that was announced. If this pattern holds, even partially, it means the performance figure a consumer or even a researcher sees doesn’t necessarily reflect what they actually use, but rather a marketing figure built in a controlled test environment for the purpose of display, not use.
If the entire industry has so far been unable to agree on a reliable, non-manipulable measure of its current models’ capabilities, how can any collective pledge to “slow the pace of improvement” hold any practical meaning that can be monitored or held accountable? Who will verify a company’s commitment to slowing down, if the company itself is the one choosing the metric by which it’s measured, and its recent history is full of examples of blatant manipulation of these metrics when needed? A call to slow down, without independent and trustworthy infrastructure for measurement and verification, is closer to a well-intentioned ethical promise than an enforceable commitment.
Despite all this legitimate criticism of the timing and the mechanism, it remains irresponsible to entirely dismiss the concerns behind this call on the grounds that it’s merely a commercial maneuver or public relations move. Technical history carries clear lessons about what happens when a powerful technology is left without any safeguards because the calculus of profit and competition always outweighs the calculus of caution.
The incident of independent AI agents breaching Hugging Face’s systems without direct human intervention, however limited its apparent impact, is a miniature model of a much broader scenario: systems capable of planning and executing autonomously without adequate real-time oversight. And when a researcher who just resigned from one of these two specific companies issues a warning, not from the position of an outside critic but from the position of someone privy to internal operational details, dismissing his warning as “dramatic” or “exaggerated” is a gamble no less dangerous than uncritically believing every marketing statement.
The real dilemma here has two inseparable sides, not a single side that can be simply resolved. The first side is that leaving this technology’s development without any genuine brakes, under frenzied competitive pressure among a limited number of companies, carries real, already-documented risks, not mere theoretical speculation. The second side is that any pledge to slow down, issued by the competing parties themselves, without an independent and trustworthy verification mechanism, remains of little practical value, and could even turn into a cover that gives the public a false sense of reassurance while the actual race continues behind the scenes without any fundamental change.
The result is that we face an industry asking the world to trust its ethical promises, while itself lacking reliable tools to measure what it promises to slow down, with a recent history full of examples of manipulating these very tools whenever commercial interest called for it. Until an independent verification mechanism is built, one not run by the competing companies themselves nor subject to their commercial incentives, every call to “slow down” will remain just another word in a long marketing race, while the more important question remains suspended without a genuine answer: who watches the watchman, when the watchman is the one designing the tools of oversight himself?
Resource: Al-Ghad. Reporting: Nadim Hatem Mansour
