Monday, 12 October 2026

A Tbilisi Engineer Scored Eight Models on Georgian Grammar, and the Best Claude Came Second

3 stories, 6 min read

SOFXOctober 11, 2026

The Labs Are Rehearsing the Day It Goes Wrong, and the Rehearsal Includes Congress

Axios reported on Friday that senior people at OpenAI, Anthropic and other leading AI companies are privately running tabletop exercises against a large-scale AI incident — and that some insiders expect one within six to twelve months. The scenarios centre on AI systems contributing to major cyberattacks or other disruption of critical infrastructure: financial services, internet connectivity, electricity, water. OpenAI confirmed on the record that it “regularly conducts exercises to prepare for potential risks”, and a spokesperson added that “these scenarios are not treated as inevitable, but are meant to help us prepare for a variety of circumstances.” Anthropic declined to comment. The part worth sitting with is not the war-gaming of the incident, which is what a serious safety team is for, but the war-gaming of what comes after it: per Axios the same exercises rehearse the political and regulatory fallout, including educating members of Congress and shaping potential legislation before a crisis prompts lawmakers to write their own. Read it beside Nadella’s long piece on Saturday and the week acquires a shape. The containment advice is published. The crisis-communications plan is not.

Source: SOFX, October 11, 2026

TechCrunchOctober 7, 2026

An Open-Source Agent Cloned 24 Million Times Has Raised $90 Million to Sell the Version You Pay For

Nous Research confirmed a $90 million Series B led by Robot Ventures, with Nvidia, Union Square Ventures, Menlo Ventures, Samsung and 1789 Capital alongside — roughly $158 million raised in the three-year-old company’s history. The Wall Street Journal put the valuation at $1.5 billion; Nous did not state one itself, which is becoming the house style for rounds of this size. The money is for Hermes for Businesses, an enterprise offering that lets companies deploy customised agents for multi-step workflows while keeping their own data private. The scale claims are the interesting part, and they are the company’s own: Hermes Agent cloned more than 24 million times, around 2.5% of global AI token usage, about $36 million in annualised revenue by mid-September per the WSJ, and an expectation of passing $100 million before the end of the year. No pricing. No launch date. The open-source agent is the distribution and the thing being sold on top of it is privacy — which is a reasonable business, and also an admission about what the free version does with your data.

Source: TechCrunch, October 7, 2026

IroncladOctober 8, 2026

Ironclad Is Selling the Memory of Negotiators Who Have Already Left

Ironclad announced Ironclad Agent last Thursday: one conversational interface for legal, procurement and sales teams to search, understand and act on contracts in plain language, routing work to specialised agents underneath it — a Contract Review Agent that redlines against the company’s own approved positions, past decisions and deal history, and a Workflow Designer Agent that builds contract workflows from a description in words. What makes it more than a search box is the Contract Knowledge Graph beneath it, which connects clauses, relationships, obligations and past decisions across everything the company has signed. Chief executive Dan Springer framed the problem as years of “hard-won knowledge in its contracts, including past negotiations, decisions, and learnings” — and the release’s actual promise is that the graph keeps that knowledge in the company once the people who negotiated it have gone. One customer puts a number on it: Sharisse Cumberbatch, VP and associate general counsel at ANDMORE, says a contract risk analysis that used to take about three hours now takes about twenty minutes. Ironclad also says 88% of its customers plan to expand their use of its agents, which is a figure from the company about the company. No pricing or availability was given.

Source: Ironclad, October 8, 2026

News from Georgia

Somebody Finally Measured Whether These Models Can Write Georgian, Not Just Read It

Giorgi Kobaidze, an engineering manager at Flutter CEE in Tbilisi, published the Georgian Language Benchmark on Kaggle Benchmarks yesterday, and it is the first thing on this beat in a long while that answers a question with numbers instead of ambition. Five tasks, 287 scored items per model, all of it hand-written — no machine translation, no generated text: converting numbers between digits and words, verb forms and preverb choice, Georgian-to-English translation, fifty grammatical gaps across two articles, and correcting planted errors without damaging the words that were already right. Eight models in four same-tier pairs. GPT-6 Astra took it with 0.90; Claude Opus 5 came second at 0.82; Claude Haiku 4.5 came last at 0.48. OpenAI won three of the four tiers, and Anthropic’s single win — Opus 4.8 at 0.75 against GPT-5.5 at 0.74 — the author himself calls “basically a tie.” Two results matter more than the ranking. Translation is finished: every model scored between 0.96 and 1.00, so reading Georgian is solved. Numbers are not: the spread ran from 0.30 to 0.83, and turning ordinal words into digits collapsed as low as 0.04. There is also a cost finding Georgian builders should know about, because nobody else will tell them: on identical questions, the Claude models consumed two to two and a half times more input tokens than the GPT models — 272 against 104 on the numbers task, 1,412 against 699 on the gap-filling — which the author attributes partly to answer-format overhead and partly to how the tokenisers handle Georgian script. Pay per token in this language and the tokeniser is a line on your invoice. He is candid about the limits: some tasks hold only twelve to seventeen items, each model was run once with re-runs drifting about ±0.03, the translation scores depend on a single judge model, and the sets were deliberately trimmed towards items models get wrong, so the absolute scores read low by design and the ranking is what he asks you to take seriously. The whole thing was built inside Kaggle’s $10-a-day credit limit and cost $13.84 in total — which is the other number in this write-up, and arguably the one that should embarrass somebody. The original plan was 41 models across nine dimensions. The credit limit is why it is eight and five.

(Source: DEV Community, October 11, 2026 — https://dev.to/georgekobaidze/georgian-language-benchmark-two-biggest-ai-labs-clash-in-a-language-that-doesnt-play-by-anyones-5b22)

Every day