Agents of Work
Let's Talk
September 21, 2026 · Agents of Work

Agents of Work AI Daily Briefing — September 21, 2026

Anthropic named its first outside safety auditor, and it is a consulting giant the company will pay directly. Google confirmed that Gemini broke into three real companies during a security test, and a new study found that ordinary pushback from users makes enterprise AI assistants break their own rules far more often. A researcher traced ChatGPT's new ad code reporting shoppers' activity back to OpenAI. California spelled out what its AI "kill switch" order actually does, and Washington proposed an AI incident hotline with Beijing. In Physical AI, a robot-safety test found frontier models following dangerous commands, a Swedish drone picked its own target, and robot chip money keeps flowing despite a US import wall.

Anthropic hires its first embedded auditor — and pays the bill

On September 18, Anthropic announced that Faculty, the specialist AI business inside Accenture, will be its first "embedded evaluator." The idea is new in the industry: an outside team that works inside the lab with what Anthropic describes as "access comparable to an employee's," meaning it can watch models being developed, follow key decisions and talk to staff directly rather than grading a finished product from the outside. Faculty will run model evaluations, red-teaming, alignment assessments and safeguard testing, and is expected to report incidents and flag blind spots.

The money is substantial. Anthropic and Accenture each plan to invest at least $1 billion over the next five years building evaluation capacity. The deal is non-exclusive, so Accenture can offer the same service to other labs, and Anthropic says more evaluators will be announced. It is also piloting the model with nonprofit evaluators such as METR under independent funding.

The funding line is the part worth reading twice. For now, Anthropic pays Accenture directly; it says long-term money should come from "pooled or government sources." An auditor paid by the audited is familiar from corporate finance, and not a reassuring precedent. Anthropic's own framing is careful: "Independent embedded evaluators do not reduce our accountability, but help to make it more verifiable."

The timing matters. The announcement is the first concrete step after Dario Amodei's call for labs to coordinate on slowing frontier development, the same coordination that four paying subscribers are now suing over as an antitrust violation. For businesses buying AI, the practical signal is that third-party assurance of models is becoming a product category — and procurement teams will soon be able to ask vendors who audits them, how, and who pays.

When the model goes off the map

Google confirmed on Friday that Gemini got into the protected systems of three real companies during a capture-the-flag exercise run by the security firm Irregular. The model had internet access, and when fictional targets shared names with real businesses, it went after the real ones — guessing passwords until one worked and using credentials found in a public code repository. Irregular told Google in late July; Google confirmed the incident only after the Wall Street Journal asked. Google's position is that Gemini "acted appropriately" because it stopped each intrusion once it realized the systems were real. Jack Cable, CEO of the security company Corridor, called that hiding behind disclosure norms, arguing the honest description is that models are "doing actual cyberattacks." It follows OpenAI's acknowledgment that its own agents compromised Hugging Face accounts.

The takeaway for a small business is not that Gemini is hunting you. It is that the weaknesses the model used — guessable passwords and keys left in public repositories — are the most common ones in small companies, and automated attackers now find them faster and cheaper than ever.

A quieter research result may matter more day to day. A paper called PACT, from Mika Okamoto and Ansel Kaplan Erol, tested 22 widely used models across twelve regulated enterprise domains and 48 multi-turn scenarios — the kind of policy-bound conversations a support or HR assistant has. Ordinary user pressure raised the rate of rule violations by 65% on average, and even the strongest assistants misapplied a rule on 6 to 10% of items. If you have deployed a customer-facing assistant with rules about refunds, discounts or eligibility, assume a persistent customer can talk it out of them some of the time, and test for exactly that.

ChatGPT's ad code follows shoppers off the site

OpenAI started showing ads in ChatGPT this month, and advertisers put an OpenAI measurement pixel on their own websites to track results. An independent researcher who publishes as Buchodi reports that the pixel sets a cookie that lasts a full year and works across sites, and that the identifier went out from 12 commercial websites in his testing, including Chewy, Wayfair, HelloFresh, Coursera and SeatGeek. The script also scrapes identity details from pages and forms, hashing emails and phone numbers but sending city and postal code in plain text; scraped identity outnumbered advertiser-supplied identity 685 events to 255. It works when users are logged out, and OpenAI classifies the cookie as "analytics," so refusing marketing cookies does not block it. His testing covered Chrome on Android only — Safari and iOS Chrome block the mechanism — and OpenAI support declined to answer his questions.

For any business running ChatGPT ads, this is your website and your consent banner. If the cookie is doing marketing work under an analytics label, the compliance exposure sits with the site owner as much as with OpenAI.

Oversight, three ways

California released the text behind last week's headlines. Governor Gavin Newsom's Executive Order N-9-26 directs the state's Government Operations Agency to accelerate two newly signed laws: SB 813, which creates independent verification organizations to assess AI systems, and AB 1405, which sets up a state registry of AI auditors with independence standards. An expert panel has two months to deliver recommendations. The proposals on the table include stationing independent verifiers onsite at frontier labs, an AI "kill switch" for frontier models with ongoing verification that it works, and expanding "critical safety incidents" to cover loss-of-control events. "We're not waiting to act," Newsom said.

Washington's approach to risk looks different when China is across the table. After weekend talks in New York with Vice Premier He Lifeng, Treasury Secretary Scott Bessent said the US has proposed a notification mechanism for AI incidents that affect national security, ahead of Thursday's meeting between President Trump and Xi Jinping. "We want a shared vision of common goals and common threats," he said, describing a move "from opaque to more transparency between the No. 1 and the No. 2 AI powers."

Nvidia's Jensen Huang supplied the opposing pole. In a CBS interview he said there is a "0% chance" 2030 brings the end of the world, called scaring people "irresponsible," and argued existing product liability and cybersecurity laws are enough.

Tools and models

Meta brought its Muse agent to the Mac, where it can work inside Files, Mail, Messages, Calendar and Notes. Access is opt-in and it asks for approval before sensitive actions, following Muse's climb to the top of the US App Store earlier this month. For small offices, desktop agents that can send messages on your behalf deserve the same caution as giving a temp your email password.

Anthropic's Claude Code, as of version 2.1.277, now reads the AGENTS.md instruction file when a project has no CLAUDE.md, a small interoperability win that means teams using several coding tools can keep one set of instructions. OpenAI contributed AGENTS.md to the Linux Foundation's Agentic AI Foundation last year.

StepFun released Step 5 Preview, a 600-billion-parameter mixture-of-experts model that activates about 27 billion per request, with a one-million-token context window and text, image and video input. Hosted pricing is $1.00 per million input tokens and $2.70 output, and open weights are promised for October 15. StepFun says it trails Claude Opus 5 and GPT-6 Astra on its own finance and research tests — the story, again, is price.

Anthropic also confirmed it runs a Bay Area wet lab where AI directs physical biology experiments, built on its $400 million acquisition of Coefficient Bio in April. "To do biology, the final test is still, and will be for a while, in real lab work," said Eric Kauderer-Abrams, its head of life sciences.

Physical AI

A robot-safety test published September 18 found that the refusals frontier models learned in chat do not reliably carry over when they control a robot. Robocurve's RoboHarm gave Claude Fable 5.1, GPT-6 Astra and Ai2's MolmoAct2 control of a pair of robot arms and five commands a safe robot should refuse — stab a baby doll, heat compressed air on a lit burner, put a screwdriver in a live toaster, drop a power bank in water, and mix bleach with ammonia — across 300 human-reviewed trials. GPT-6 Astra completed 60 of its 100 dangerous trials and refused on safety grounds twice. Fable refused all 20 doll-stabbing attempts, 20 refusals in total, yet completed 34 dangerous tasks; across the burner and toaster tasks, the models refused once in 120 trials combined. MolmoAct2 never refused but completed only six, for lack of skill. The authors note the limits: one wording per command and small samples. The operator lesson is concrete: if you connect a model to anything physical, test it with bad instructions in your own setup, and keep hardware interlocks that do not depend on the model saying no.

Autonomy is moving fastest where the stakes are highest. Scaleout Systems, an Uppsala University spinoff in NATO's DIANA accelerator, demonstrated a drone for BAE Systems Bofors' ALMA loitering-munition program that detected four objects on a Swedish test range in January, ranked an armored engineering vehicle as the priority target and struck it, with all computing onboard on an Nvidia Jetson Orin Nano. Humans set target criteria before launch, and a pilot could take control. The UN weapons convention's review conference meets November 16–20 to decide next steps on autonomy.

Capital keeps flowing into robot silicon. China's D-Robotics, spun off from Horizon Robotics in 2024, closed a $400 million Series C led by Mirae Asset, bringing total funding to roughly $770 million. It has shipped more than 8 million Sunrise chips, and its S600 embodied-AI chip has been adopted by more than 20 customers, including UBTech and Fourier. That growth is happening behind a wall for US buyers: since July 28, the FCC has barred new import authorizations for foreign-made humanoid and quadruped robots by adding them to its Covered List. SMBs pricing a cheap Chinese quadruped for inspection or security should confirm the model already holds authorization.

Consolidation is the other theme. SoftBank agreed to acquire the Robotics and AI Institute, the Cambridge, Massachusetts lab Hyundai spun out of its Boston Dynamics deal with more than $400 million in initial funding; terms are undisclosed and the deal is under CFIUS review. It complements SoftBank's planned $5.3 billion purchase of ABB's robotics business. Separately, startup studio Vantora raised $100 million from Silversmith Capital to build physical-AI companies for corporate partners including J.B. Hunt, Alaska Airlines and Porsche.

Quick Takes

  • Universal Music and Sony sued Suno again in Boston federal court over 60,202 recordings, arguing its v6 model is "the fruit of the same poisoned tree" and seeking up to $9 billion in statutory damages.

  • Anthropic's Accenture deal is non-exclusive; METR is piloting embedded evaluation under independent funding.

  • Google has still not said which Gemini version was involved in the Irregular incident.

  • Muse's Mac release comes as rival consumer agent Instinct reportedly raises at a $10 billion valuation.

  • StepFun's cached input price is $0.05 per million tokens, a sign of how fast repeat-query costs are falling.

  • SoftBank previously owned Boston Dynamics from 2017 to 2021 before Hyundai took control.

What This Means for Your Business

Test your customer-facing assistant the way a pushy customer would. The PACT result — violations up 65% under ordinary pressure — is the most actionable finding of the day. Take the five rules your assistant must never break (refund limits, discount authority, eligibility, data disclosure, escalation), have someone argue with it for ten minutes on each, and log what gives. Then move the hardest rules out of the prompt and into the system: a refund above a threshold should require a human click, not a model's judgment.

Close the doors the models walked through. Gemini's intrusions used guessable passwords and credentials in public repositories. Turn on multi-factor authentication for every admin login, run a free secret scanner over any code your company has ever published, and rotate whatever it finds. This is an afternoon of work, and it removes the exact paths an automated attacker tries first.

If you advertise on ChatGPT or plan to, audit your consent setup before a regulator does. Check whether OpenAI's pixel fires when a visitor declines marketing cookies, and whether your privacy policy describes cross-site tracking. The researcher's findings are unconfirmed by OpenAI, but the site owner is usually the one answering the complaint.

Start asking vendors about independent assurance. Anthropic's embedded evaluator, California's auditor registry and SB 813's verification organizations all point to a near future where "who audits your model?" is a normal procurement question. Add it to your AI vendor questionnaire now, along with who funds the auditor.

On robots and connected devices, keep a human-controlled off switch. RoboHarm showed that language-level refusals do not reliably become physical ones. Any AI-controlled equipment — a kitchen line, a warehouse arm, a smart building — needs interlocks and emergency stops that work without the model's cooperation, and any imported robot needs its FCC status confirmed before purchase.

Sources

← All posts