I trace the score, not the press release. When the AI safety index landed on my desk, my first instinct wasn't to discuss the letter grades. It was to check the methodology. What I found was a vacuum of transparency dressed as a governance report.
In a world where blockchain projects are routinely torn apart for opaque tokenomics and unverified audits, the AI industry is now being evaluated by a metric that would fail the smell test of any seasoned crypto investigator. This article is not an attack on AI safety. It is a forensic dissection of a scoring system that, if applied to a DeFi protocol, would be laughed out of the auditor's room.
The Scorecard That Isn't
For context, the AI safety index in question assigns Anthropic a C+ and OpenAI a C. The entire industry, according to the report, is underperforming. The article also flags concerns about deepening ties between AI companies and military institutions. On the surface, this sounds like a credible warning. But as someone who has spent years auditing smart contracts and tracing wallet flows, I know that a grade without a methodology is just a headline.
Let me be clear: I am not defending OpenAI or Anthropic. I am questioning the tool used to judge them. The AI safety index appears to measure governance commitments, transparency, red-teaming, and external audits. These are all important. But the index does not disclose its scoring weights, its data sources, or whether it accounts for actual safety incidents like jailbreaks, data leaks, or misuse cases. In crypto, we call this a "black box." And we reject black boxes.
Core: Systematic Teardown of the Safety Index
Hype is the only asset in a vacuum mint. The AI safety index is minted from hype. It claims to rank companies on safety, but it provides no reproducible evidence. Let me break this down using the same rigor I apply to a DeFi protocol audit.
First, the index lacks a verifiable on-chain or off-chain trail. There is no public ledger of the data points used to derive the scores. In crypto, we demand that a DeFi project publish its smart contract code. The AI safety index publishes nothing. It is a centralized judgment delivered by an anonymous or semi-anonymous body. This is the equivalent of a yield farm promising 1000% APY without a single line of verified code. The burden of proof is on the issuer, not the audience.
Second, the index conflates “safety commitment” with “safety outcome.” A company can issue a thousand press releases about safety and still have a model that is easily jailbroken. We saw this in Terra-Luna: all the marketing in the world could not save a fundamentally flawed algorithm. The AI safety index appears to reward documentation over results. In my experience auditing the 0x protocol, I found that the team’s initial dismissals of my vulnerability report were accompanied by strong public commitments to security. The commitment was fake; the vulnerability was real. The index would have given them a high score on governance while the exploit was live.
Third, the index does not distinguish between technical safety and governance safety. A model might be perfectly aligned but still be used for harmful purposes by its operators. The military ties mentioned in the report are a separate ethical dimension, not a measure of model safety. By lumping them together, the index creates a false composite that obscures where the real risks lie.
When the yield is too high, the exit is rigged. The AI safety index’s yield is the promise of a clear, comparable metric. The exit is rigged because the metric is not comparable. Without a standardized audit framework, each company can cherry-pick its best practices. OpenAI might highlight its content moderation; Anthropic might highlight its constitution training. The index then assigns a grade that is essentially a popularity contest among the scorers, not a rigorous assessment of algorithmic robustness.
During the DeFi Summer of 2020, I warned that the leverage loops were unsustainable. I was ignored. Today, I am warning that the AI safety index is a similar illusion: it gives the appearance of oversight without the substance. The industry needs a cryptographic commitment to safety data, not a press release on letter grades.
Contrarian: What the Bulls Got Right
To be fair, the AI safety index does one thing correctly: it raises the salience of safety as a competitive dimension. For years, AI companies were evaluated almost exclusively on model capability—who has the largest parameter count, the best benchmark scores, the most viral demos. The index shifts the conversation toward governance, transparency, and ethical boundaries. This is a net positive.
Moreover, the C+ and C grades are not meaningless. They reflect a consensus among experts that both Anthropic and OpenAI have significant room for improvement. Even if the methodology is flawed, the direction of the signal is correct: the industry is not safe enough. The report’s mention of military ties is also a legitimate concern that deserves more scrutiny, not less. In a world where AI systems are being deployed in defense contexts, the line between safety and complicity becomes blurred.
The bulls might argue that the index, despite its flaws, serves as a catalyst for better practices. They are right. Much like how Crypto Briefing’s coverage of the Terra collapse pushed regulators to act, the AI safety index pressures companies to invest in real safety measures. But a catalyst is not a replacement for a rigorous audit. The index is a starting point, not a conclusion.
Takeaway: Accountability Through Transparency, Not Grades
The AI safety index is a governance mirage. It gives the public a letter grade without the underlying data. In crypto, we have learned the hard way that trust is not a score—it is a set of verifiable actions. If we want to hold AI companies accountable, we need to demand on-chain transparency: publish the red-team results, release the audit logs, and commit to a publicly verifiable safety framework.
I trace the wallet, not the whisper. The AI safety index is a whisper. The real story is what happens when we demand the wallet. Until AI companies release their safety data with the same cryptographic rigor that we expect from a DeFi protocol, every grade is a fiction. The question is not whether Anthropic is better than OpenAI. The question is: why are we accepting a grading system that would be rejected in a blockchain audit?
The answer is that we are still in the early stage of AI governance. The market is euphoric, and the hype machine is working overtime. But as a cold dissector, I see the cracks. The AI safety index is a symptom of a deeper problem: the absence of a standardized, transparent, and verifiable method for evaluating AI safety. Until that changes, every letter grade is just another piece of marketing collateral.
A profile picture is not a shield against fraud. Neither is a C+.