The market is wrong about Vals AI. Wrong to frame it as a breakthrough in AI evaluation. Wrong to accept $400 million valuation as a signal of industry maturity. The real story is a liquidity trap dressed in a16z's halo. A centralized oracle for AI trust, built on shaky foundations, propped up by narrative hunger. The funding event is real—$40 million Series A, a16z leading, valuation at $400 million. But the underlying technology, commercialization, and industry impact are riddled with unverified claims, ambiguous metrics, and structural conflicts of interest. This is not a pioneer. It's a productized SWE-bench with a marketing budget.
Vals AI claims to revolutionize model evaluation by pulling real-world tasks from GitHub pull requests, testing models on hidden tests, and serving clients like OpenAI, Anthropic, Google, Meta, and xAI. The pitch is seductive: move beyond static benchmarks, evaluate models on your own codebase, trust the results. But the devil is in the details—or rather, the lack thereof. No independent verification of the evaluation methodology. No disclosure of dataset construction, contamination prevention, or statistical sample size. The revenue growth claim—"8 times the full-year 2025 revenue"—is mathematically ambiguous. The company itself admits the data is self-reported.
Note: Sentiment turning bearish on L2s.
Let me be clear: I am not dismissing the need for credible third-party evaluation. The AI industry is drowning in benchmark pollution. GSM8K, HumanEval, even MMLU have been compromised by training data leakage. The open-source ecosystem is a mess of overfitting. Vals AI's approach—dynamic, private, codebase-specific evaluation—addresses a real pain point. But the solution is not a centralized agency with a16z ties. It's a decentralized, verifiable infrastructure that leverages blockchain for immutable audit trails.
Based on my experience auditing DeFi derivatives in 2020, I learned that trust in centralized oracles is a fragile thing. The same lesson applies here. Vals AI is a centralized oracle for AI quality. It has no on-chain verification, no transparency into its test generation, no mechanism for users to independently replicate results. The company claims to generate hidden tests from GitHub PRs, but unless those PRs are from private repositories, the training data overlap risk is non-trivial. The article from the monitoring channel "Dongcha Beating" lacks independent media attribution. The source is anonymous. The claims are unverified.
Core Technology: Engineering Innovation, Not Breakthrough
Vals Smith's core insight is that model evaluation should mirror real-world developer workflows. By extracting tasks from historical pull requests and using hidden tests to judge completion quality, the system mimics SWE-bench-style dynamic evaluation. This is a productization of an existing academic concept, not a new algorithm. The innovation is in the integration, not the architecture. The company extends this to finance, law, and medicine—but the dataset construction, task generation, and contamination prevention methods are undisclosed. The risk of "reverse hacking" by model vendors is high. If Vals's tests are private but not cryptographically sealed, they can be leaked or reverse-engineered.
My own experience with oracle feed latency in DeFi taught me that centralized data sources are the Achilles' heel. Chainlink's nodes are centralized in practice, despite rhetorical decentralization. Vals AI faces the same problem. Its evaluation is a black box. The model vendors it serves are also potential customers. The conflict of interest is glaring. The company's claim that OpenAI, Anthropic, etc., cite its results in their model cards is unverifiable. Even if true, the citation is a one-way transaction—no guarantee of continued usage.
Commercialization: B2B Evaluation-as-a-Service, But Metrics Are Foggy
The $400 million valuation implies a bet on a new category: AI evaluation infrastructure. The 9–10% dilution for a $40 million Series A is standard. But the revenue claim is where credibility fractures. "This year's revenue has reached 8 times the full-year 2025 revenue" is a phrase that defies logical parsing. The most charitable interpretation is 8x year-over-year growth from a small base. The least charitable is outright fabrication. No customer count, no average contract value, no retention rate. The article mentions "model card citations" as a proxy for adoption, but citations are not revenue.
Note: Sentiment turning bearish on L2s.
The business model—free GitHub integration for developers, enterprise subscriptions for teams—echoes the standard SaaS playbook. But the unit economics are unclear. Evaluation tasks require human validation, especially in regulated domains like law and finance. The labor cost is not disclosed. The company may be burning cash on manual review to maintain quality, a common trap for AI startups that scale too fast. a16z's involvement suggests a strategic play: Vals AI could be integrated into a16z's portfolio companies, providing captive demand. But "independent" evaluation from a venture-backed firm with ties to the same ecosystem is an oxymoron.
Industry Impact: A Signal of Maturation, But Not a Panacea
The narrative that Vals AI's funding marks a turning point for third-party evaluation is plausible if the claims hold. But the industry impact is limited by geopolitics. Chinese AI labs and open-source communities are unlikely to submit to a US venture-backed evaluator. The standardization of evaluation will be fragmented, not unified. The company's reach is constrained by the English-speaking, US-centric developer ecosystem.
On the positive side, Vals AI lowers the information cost for enterprise AI procurement. Instead of relying on leaderboards, companies can test models on their own codebases. This is a genuine improvement. But the solution is temporary. The next step is decentralized evaluation networks where results are verifiable on-chain, using zero-knowledge proofs to ensure test integrity without revealing the test set. Vals AI is a centralized proxy for that future. It may be acquired or disrupted once the technology matures.
Contrarian Angle: The Real Innovation Is Missing
Here is the counter-intuitive take: Vals AI's centralized model actually reinforces the very problem it claims to solve. The industry needs a trustless evaluation layer, not a trusted one. The company's technology could be ported to a blockchain-based platform where evaluation tasks are posted as smart contracts, results are verified by multiple validators, and rewards are distributed via tokens. The current centralized approach is a stepping stone, but it's also a trap. Investors betting on Vals AI are betting on the status quo, not the future.
My experience with the Terra/Luna collapse in 2022 taught me that algorithmic trust mechanisms without proper risk frameworks are ticking time bombs. Vals AI's evaluation is an algorithmic trust mechanism. It relies on the company's integrity, not on cryptoeconomic guarantees. The moment a conflict of interest arises—say, a model vendor pays for a favorable evaluation—the system breaks. The company's independence is only as good as its firewall. a16z's involvement adds a layer of complexity. The venture firm has stakes in multiple AI companies. The potential for soft pressure to produce favorable evaluations is real.
Takeaway: The Next Narrative Is Decentralized Evaluation
The market is sideways. Chop is for positioning. Vals AI's funding is a signal that the evaluation narrative is gaining traction, but the centralized form is a dead end. The next wave will be protocols that combine ZK proofs with AI task verification, creating a verifiable, decentralized evaluation layer. Projects like ANON or EvalDAO are already experimenting. The liquidity will flow to those that solve the trust problem without centralizing it.
Note: Sentiment turning bearish on L2s.
For now, Vals AI is a story of narrative capture. The $400 million valuation is a bet on the category, not the company. The technology is incremental, the revenue claims are unverified, and the business model is fraught with conflicts. The market is wrong to celebrate. The real opportunity is in the decentralized alternative. I am watching the space, but I am not buying the narrative.
Tags: AI, Vals AI, a16z, Funding, AI Evaluation, Blockchain, Narrative, Market Analysis, Decentralized Trust, ZK Proofs
Prompt: Generate prompt for an article illustration showing a centralized oracle with a broken chain, representing the fragility of third-party AI evaluation, with a background of a sideways market chart and a faint blockchain grid overlay.