# The AI Rankings Just Got Reshuffled *The AI Daily Brief — Thursday, 2026-10-29 · https://aidailybrief.ai/e/2026-10-29* **Benchmarks are a cat-and-mouse game with reality, and the new scoreboard finally matches the vibes.** Benchmarks are inherently limited but important. Without them we only have vibes. Artificial Analysis's Intelligence Index V5 is the latest move in a constant chase to match how people actually use AI. It shifts to mostly private datasets to blunt benchmark-maxing, drops long-context tests that compaction made routine, and goes all-in on agentic coding, agentic science, and granular white-collar work. The result reshuffles the leaderboard. A handful of US labs pull clear of everyone else, the open-weights gap jumps from 3 points to 12, and Google's Gemini 4 Carbon takes first place. Most people say the new rankings finally match their own experience. The open questions are weighting, effort settings, and whether we should stare at one headline number at all. --- ## By the numbers - **59** — Gemini 4 Carbon's score on AA Index V5, a 9-point jump and first place - **$7.76** — Carbon's cost per task: 23% cheaper than Opus 5.5, 20% pricier than Astra - **3 → 12** — Points between top open-weights and US proprietary models after the V5 rebalance - **48%** — Azure's year-over-year growth, to $33B in its first standalone breakout - **$10B** — OpenAI's contribution to Azure revenue, around a third of the business - **$30.7B** — Google Cloud quarterly revenue, double the year-ago figure - **12%** — Google Search growth, down from 17% and 19% the two quarters prior - **$300B** — Google's guided CapEx for next year, a 50% increase - **$13B** — Nvidia's Hugging Face acquisition, now under DOJ questioning - **3.3 months** — One analyst's estimate of the US–China model gap, and closing ## Headlines ### Gemini 4 Carbon lands just two weeks after Argon `[01:00]` It's one of the shortest gaps yet between versions of ultra-large models. With a new base model in hand, Google can iterate quickly through reinforcement learning instead of waiting on a lengthy pre-training run. *For: Eng* Link: https://aidailybrief.ai/e/2026-10-29#gemini-4-carbon-two-weeks ### I would expect the next 12 months to see continued acceleration because of RSI. `[02:00]` *— Sundar Pichai, Google CEO, on the earnings call* On Wednesday's earnings call, Pichai said Google is at the point where AI is helping increase its training and launch velocity. He framed Carbon's quick arrival as evidence of "incredible model momentum." *For: Exec* Link: https://aidailybrief.ai/e/2026-10-29#pichai-acceleration-rsi ### Carbon tops six of eight key benchmarks `[02:00]` In coding, Carbon gained 8 points on TerminalBench 4.0 and 6 on Frontier Suite. In cybersecurity, it added 12 on Exploit Gym. In computer use, it picked up 7 on OSWorld and 8 on Agent's Last Exam. It leads the new AA Index V5 at 59, ahead of Opus 5.5, and ties Sonnet 5.5 for second on the AA Coding Index. *For: Eng* Link: https://aidailybrief.ai/e/2026-10-29#carbon-tops-benchmarks ### From second tier to back in the race `[03:00]` When Argon shipped, some felt Google had missed its window, leaving it in a second tier alongside Meta. If real-world use matches the benchmarks, Carbon puts Google genuinely back at the frontier. It does so at a middle-of-the-road $7.76 per task. Link: https://aidailybrief.ai/e/2026-10-29#google-back-in-frontier-race ### Carbon goes to cyber defenders first `[03:00]` Google says it is following the administration's pre-release model testing guidelines and taking a "deliberate approach." A subset of cyber defenders and trusted partners get access first. Developers, businesses, and consumers wait while Google iterates on guardrails, so there are few first-hand testimonials yet. *For: Legal, Eng* Link: https://aidailybrief.ai/e/2026-10-29#carbon-deliberate-rollout ### Don't judge LLMs on 3D renders `[04:00]` Arena's render and game-design review of Carbon was slammed for looking less polished than Opus 5.5. NLW isn't convinced 3D design is the right yardstick for LLMs, however much social media fixates on it. Arena's Peter Gastev still said Carbon feels "a lot better than Argon." Link: https://aidailybrief.ai/e/2026-10-29#3d-design-wrong-yardstick ### Inside Google, launches now feel like periodic checkpoints `[05:00]` Gemini post-training researcher Xiao Zhang contrasted Carbon with the big-bang celebration around Gemini 3 Pro. The team has been dogfooding for weeks: overnight loops running dozens of experiments, root-causing bugs, and swarms inspecting RL data. DeepMind researchers also hinted that an even stronger unreleased Gemini is waiting. *For: Eng* Link: https://aidailybrief.ai/e/2026-10-29#launches-periodic-checkpoints ### Google still has the benefit of the doubt, but suspend judgment `[06:00]` Google has been out of the early-adopter AI conversation for about 10 months. Even so, builders are already reopening Antigravity and calling it "comeback season." NLW's stance is to hold judgment until people get their hands on the model, though it looks promising. Link: https://aidailybrief.ai/e/2026-10-29#google-benefit-of-doubt ### Azure grows 48%, and OpenAI is a third of it `[06:00]` Microsoft broke out Azure for the first time: $33B, up 48%, driving 19% overall growth to $92.3B. OpenAI contributed $10B of Azure revenue. The stock hit a new all-time high after hours, up as much as 4.8%. *For: Finance* Link: https://aidailybrief.ai/e/2026-10-29#azure-breakout-openai-third ### Google Cloud doubles while Search slows `[07:00]` Google Cloud doubled to $30.7B, nearly closing the gap with Microsoft. Search growth slowed to 12% from 17%, though, and YouTube to 9.2%. Pichai acknowledged that AI Mode's lower monetization is weighing on results. Some analysts see Google forced to cannibalize Search, and the stock fell as much as 10% after hours. *For: Finance, Exec* Link: https://aidailybrief.ai/e/2026-10-29#google-cloud-doubles-search-slows ### The build-out may have hit peak velocity, but not for the reason bears predicted `[08:00]` Google, Microsoft, Oracle, and Meta are all spending below plan this quarter, even as Google guides CapEx up 50% to $300B next year. Bears expected faltering demand to prick the bubble. Instead, hyperscalers appear to want to keep accelerating and are running into components, power, labor, and community support. *For: Finance, Exec* Link: https://aidailybrief.ai/e/2026-10-29#capex-peak-velocity-physical ### The DOJ's Nvidia probe is far bigger than anyone thought `[09:00]` Reuters reports DOJ questions about whether owning Hugging Face, a $13B deal, would let Nvidia control a key AI model distribution point. The multi-year probe predates the Groq acqui-hire. It also covers the traditional Run:ai acquisition and another data-center deal, plus venture investments, sales practices, and customer pressure. No charges have been filed. *For: Legal, Exec* Link: https://aidailybrief.ai/e/2026-10-29#doj-nvidia-top-to-bottom ## Main episode ### Benchmarks are inherently limited and still essential `[15:00]` Without benchmarks, there's nothing but vibes to tell models apart. Yet almost never should one score, or even the aggregate, be taken as gospel. Done well, they're a guide to where different models fit into your stack. Link: https://aidailybrief.ai/e/2026-10-29#benchmarks-limited-but-important ### From Chatbot Arena vibes to a cat-and-mouse game `[16:00]` When LLMs were glorified search and writing assistants, the Chatbot Arena ranking was the pinnacle test. Coding and research brought objective benchmarks, then harder versions as each one saturated. The index has repeatedly lagged reality. The early V4.2 update came because GPT-6 Astra's computer-use gains weren't showing up in its score. Link: https://aidailybrief.ai/e/2026-10-29#from-chatbot-arena-to-agentic ### V5 goes majority-private to fight benchmark-maxing `[18:00]` For the first time, most benchmarks in the Intelligence Index are built on private datasets. Benchmark-maxing means getting test questions and answers into training data, so a score reflects memory rather than capability. Private data makes it harder to train to the test. *For: Eng* Link: https://aidailybrief.ai/e/2026-10-29#v5-private-datasets ### Long context got solved enough to leave the index `[19:00]` V5 adds Cognition's Frontier Code 2.0, which scores on whether a maintainer would merge the code, plus Terminal Bench Science. It swaps GDPVal for granular white-collar tests and drops the long-context reasoning test. In January long context was critical, but compaction and memory gains mean it's now taken for granted. *For: Eng* Link: https://aidailybrief.ai/e/2026-10-29#long-context-dropped ### A 25% weighting on science is a forward-looking bet `[20:00]` The index now weights four categories equally: knowledge work, coding, science, and general. Coding benchmarks measure only agentic performance, and science tests real agentic research rather than stored knowledge. Science's quarter share seems to bet on the science and math race Anthropic and OpenAI have been trading headlines over. Link: https://aidailybrief.ai/e/2026-10-29#science-25-percent-bet ### The leaderboard reshuffle: agentic models rise, Wonder and Kimi fall `[20:00]` Before Carbon, Opus 5.5 led at 58 and GPT-6 Astra jumped from sixth to second at 57. OpenAI's Sol rose from ninth to third and GPT-6.1 Terra from tenth to fifth. Sonnet 5.5 slipped to fourth, Meta Wonder fell from third to sixth, and Kimi K3.1 Pro dropped from fourth to ninth. Link: https://aidailybrief.ai/e/2026-10-29#leaderboard-reshuffle ### The open-weights gap widens from 3 points to 12 `[21:00]` Kimi K3.1 Pro is the top open-weights model at 46. DeepSeek V4.1 Flash and GLM 5.3 sit at 32, below Claude Haiku 5.5's 36. Anthropic, OpenAI, Google, and Meta all have models at 50 or above, followed by a 10-point cliff. Adding more agentic tasks drew a clear line between the frontier and everyone else. *For: Exec* Link: https://aidailybrief.ai/e/2026-10-29#open-weights-gap-frontier-tier ### Frontier costs range from 96 cents to $13 per task `[22:00]` GPT-6.1 Terra is the cheapest top-10 model at $0.96 per task, and Fable 5.1 the priciest at $13.04. Among comparably sized models, Opus and Sonnet run around $10 and GPT-6 Astra $6.24. Gemini 4 Carbon, Grok 4.7, Wonder, and Muse land in the high-$7 range. *For: Finance, Eng* Link: https://aidailybrief.ai/e/2026-10-29#frontier-cost-spread ### Most people say the new rankings finally match the vibes `[23:00]` The dominant reaction was that V5 tracks subjective experience far better. OpenAI's Dean Ball said it's the first AA ranking to roughly approximate his own. Others noted it confirms a deeper Anthropic/OpenAI lead and that no "Fable-class" open model has appeared yet. Link: https://aidailybrief.ai/e/2026-10-29#v5-matches-vibes ### The case for open weights: strong at agents, weak at science `[24:00]` Brookings' Kyle Chan noted private benchmarks only rose from 45% to 60% of the index. Kimi leads all models on Terminal Bench 4.0 at 67% and beats Claude on private AutomationBench, but scores 34% on Terminal Bench Science versus Astra's 63%. His read: this isn't necessarily benchmark gaming, but a real gap in advanced scientific reasoning. Link: https://aidailybrief.ai/e/2026-10-29#kyle-chan-kimi-defense ### The next fight is over weighting `[24:00]` OpenAI and Anthropic are clearly emphasizing scientific discovery. For the average user, though, does science matter as much as agentic coding? That argues either for a different weighting or for skipping the headline number and focusing on the benchmarks for your own use cases. *For: Exec* Link: https://aidailybrief.ai/e/2026-10-29#weighting-debate ### Leaderboards run on max settings, but nobody uses max `[25:00]` AA does factor effort into its cost analysis. On a practical level, though, NLW thinks people would be better served by sorting the index by effort setting as well as by model. *For: Eng, Finance* Link: https://aidailybrief.ai/e/2026-10-29#leaderboard-by-effort ### A new Progress Index charts every model since November 2022 `[26:00]` AA's new long-term index tracks how each lab's leading model improves over time. People were quick to mine it. One analysis said OpenAI's models lean hardest on reasoning, and another estimated the US–China gap at 3.3 months and closing. Link: https://aidailybrief.ai/e/2026-10-29#progress-index-us-china *Today's sponsors: KPMG, CData, Granola, Blitzy — offers at https://aidailybrief.ai/sponsors* --- Transcript: https://aidailybrief.ai/e/2026-10-29/transcript.md Listen: https://pod.link/1680633614 · Ad-free: https://patreon.com/aidailybrief © 2026 The AI Daily Brief — Until next time, peace ✌