Code doesn't.
Kimi just pushed PerceptionBench to GitHub. 3000 visual tasks. 10 atomic capabilities. Every major AI model scored below 60%. GPT-5.6-Sol? 58.2%. Claude-Fable-5? 59.1%. Gemini-3.1-Pro? 56.8%. The highest? Some internal test model called "K3" at 58.5%.
Volume precedes price. Always.
Right now, the volume is in tweets about "AI's vision ceiling." The price? That's the attention these models are about to lose — or the money Kimi is about to raise.
I've been watching this space since 2018. I audited ICOs where the code didn't match the whitepaper. Here, the benchmark doesn't match the model names. GPT-5.6-Sol? That's not a real OpenAI model. Claude-Fable-5? Anthropic never shipped that. Gemini-3.1-Pro? Google's numbering doesn't line up.
Context: Why Now?
PerceptionBench isn't just another benchmark. It's a surgical strike on the hallucination problem. Kimi claims that existing benchmarks (MMLU, VQA, etc.) test reasoning, not raw perception. So they built a task set that isolates visual parsing: counting dots, detecting colors, identifying rotated letters, spotting mirror images. Each task is a binary or multiple choice ground truth. No language bias. No common sense shortcuts.
Why now? Because every AI company is sprinting toward multimodal. Apple's LLM-integrated Vision Pro. Tesla's FSD V12. OpenAI's ChatGPT with vision. The market for "reliable visual AI" is projected at $12 billion by 2027 (if you believe the VC decks). Kimi is staking a claim: "We are the ones who can see correctly."
But there's a catch — the model names in the report don't exist. That's not a typo. It's a signal.
Core: The Data Trail
Let's parse the original paper (yes, I read the code). The benchmark has 3000 tasks, divided into 10 categories: object counting, symmetry detection, color discrimination, orientation recognition, face detection, text recognition, pattern matching, depth estimation, occlusion handling, and spatial reasoning. Each category has 300 tasks. All synthetic. No real-world images.
Top score? 59.1% (Claude-Fable-5). Random chance across multiple-choice tasks is roughly 50% (for binary) to 25% (for four-option). So models are barely above random. On object counting, even the best model (K3) scored 62%. On occlusion handling, all models crashed below 40%.
Here's the forensic angle: the model names are fake. I cross-referenced OpenAI's API list, Anthropic's documentation, Google's Model Cards. None of these names appear. GPT-5.6-Sol? The last OpenAI release was GPT-4o with a date suffix, not a number like 5.6. Claude-Fable-5? Anthropic uses code names like "Opus" and "Sonnet." Gemini-3.1-Pro? Google releases are Gemini 1.0, 1.5, and now 2.0. No 3.1.
Why would Kimi publish results with fake model names? Three possibilities: 1. Test network nodes — Like crypto testnets, these could be internal release builds or early checkpoints. Kimi used placeholder names but didn't update the paper. 2. Deliberate obfuscation — To avoid legal liability or to prevent direct comparison. "We didn't claim it was the production model." 3. Marketing fiction — The entire benchmark is a lead magnet for their token (if there is one) or for their next funding round.
Based on my 2020 DeFi yield crisis analysis, I've seen this pattern before. Projects release a flawed audit report (or benchmark) that shows everyone is vulnerable except themselves. Then they pitch their "solution" as the only safe haven. Here, Kimi's own model (K3) is second place, within the margin of error of first. Convenient.
Contrarian: Not a Dip. A Liquidity Trap.
Not a dip. A liquidity trap.
Everyone is reading this as: "AI still can't see properly — invest in perception startups!" That's the surface narrative. The contrarian angle is that PerceptionBench is actually a honeypot for venture capital.
Here's how: Kimi open-sources a benchmark. Media runs with "AI models all fail." VCs panic about hallucination risk. Kimi whispers: "We have a model that scores 58.5% on our own benchmark — better than anything else." VCs write checks. Kimi raises $200M. Then they release a new model that scores 72% on PerceptionBench (after secretly training on the test set). Narrative flips: "Kimi solves vision!" Token launch (if any) pumps. Whales dump.
Sound familiar? It's the same pattern as the 2021 NFT floor manipulation I exposed. A syndicate creates artificial volume to pump a collection. Here, artificial scarcity of "correct perception" pumps a company's valuation.
The model names are the red flag. In crypto, when a wallet appears that doesn't match known patterns, you flag it. Same here. If Kimi can't even get the names of the competitors correct, what else is wrong? The dataset could be overfitted to their own model. The tasks could be designed to favor their architecture. Without independent replication, this is just a press release with code.
Takeaway: Next Watch
Watch for three signals in the next 90 days: 1. Kimi's official technical report — Will they clarify the model identities? If they release a corrected paper with real model names and similar scores, the benchmark gains credibility. If they stay silent, the trap is set. 2. Third-party replication — Will OpenAI, Anthropic, or Google run their own models on PerceptionBench and publish results? If they achieve similar low scores, the benchmark is real. If they score 70%+, the benchmark was flawed. 3. Kimi's next model release — If they suddenly announce a model that scores 70%+ on PerceptionBench, assume data contamination.
Volume precedes price. Always. The volume of discussion around this benchmark is the noise. The price is the eventual investment into Kimi. Smart money waits for verification. Code doesn't.