On 11 November Google shipped A2UI 1.0 with a simple promise: let the agent describe a screen, and let your app draw it safely with its own components. I wrote about what that contract means without having run it, which bothered me. So I ran it: 4,798 test runs across six models from three vendors, against the two things most teams already do: map a tool call to a prebuilt component, or let the model write the HTML.
My verdict: A2UI is worth adopting if you want a cheap model to draw screens that carry text you do not control, or screens your component library cannot draw. On the three small models, model-written HTML let a hostile note run its script in every run of one task. A2UI never did, and neither did prebuilt components. If you are paying for a mid-tier model anyway, it is harder to justify: model-written HTML was just as accurate there, and prebuilt components are cheaper than both on every screen they cover.
The test: ten tasks from a travel-rebooking assistant, four methods (prebuilt tool components, A2UI Direct JSON, A2UI Express, and model-generated HTML in a sandboxed iframe), six models (Claude Haiku 5.5 and Sonnet 5.5, GPT-6 Luna and GPT-6.1 Sol, Gemini 3.5 Flash-Lite and Gemini 3.8 Flash), 20 runs each. Tool components use each vendor’s native tool calling. Prompts are cached where the vendor supports it. When the application rejects an output, the errors go back to the model once; “fully correct” is judged after that retry. Reasoning is off on the small models and at the lowest setting on the mid-tier models.
I count a run as fully correct when the output is valid, it renders, every expected value is visible, the right action and payload come back, nothing stale is left after an update, nothing from the data executes, and unavailable seats are disabled. A2UI’s two formats are Direct JSON, where the model writes the protocol’s messages in full, and Express, a compact notation the SDK compiles into the same messages and uses by default.
The results
| Method | Fully correct (of 960) | 95% range |
|---|---|---|
| Tool components | 720 | 72 to 78% |
| A2UI Direct JSON | 907 | 93 to 96% |
| A2UI Express | 866 | 88 to 92% |
| Generated HTML | 844 | 86 to 90% |
A2UI Direct JSON had the most fully correct runs overall. That average hides a split by model size.
On the three mid-tier models (Sonnet 5.5, Gemini 3.8 Flash, GPT-6.1 Sol), A2UI Direct JSON scored 480 of 480 and generated HTML 476 of 480; the 95% ranges overlap (99 to 100% and 98 to 100%), so the two are indistinguishable there. On the three small models (Haiku 5.5, Flash-Lite, Luna), A2UI Direct JSON scored 427 of 480 (86 to 91%) and generated HTML 368 of 480 (73 to 80%). Every one of generated HTML’s losses on the small models is on the two hostile-text tasks.
Each of the 20 runs was an independent sample with nothing carried over between them. Tool components scored a flat 36 out of 48 every run: they missed the composite checkout and the seat grid on every model, every time. Direct JSON varied between 44 and 46; Express between 42 and 46; generated HTML between 40 and 43.
Here is every task against every method. Hover a cell for the count on each model.
Where A2UI earns its cost
Composite screens. The toolkit has five components (options list, form, table, confirmation, message). None combines a table of charges, an input field and two buttons. Tool components scored 0/120 on the composite checkout. In 80 runs the model chose the form tool, which got the email field and the Pay button right but has only one button, so Back was missing. In 39 it chose the table tool and lost the field and both buttons. My scoring is all or nothing, so close does not count. A2UI Direct JSON scored 119/120. Express scored 115/120. Generated HTML scored 120/120.

The seat grid needed a spatial layout with disabled seats. The toolkit has no disabled button, so the model drew the available seats as option cards and named the unavailable ones in a sentence: 0/120. A2UI Direct JSON rendered the grid but disabled the unavailable seats in only 71 of 120 runs: all 60 on the mid-tier models, and 11 of 60 on the small ones (none on Haiku 5.5 or GPT-6 Luna). Generated HTML scored 120/120 on every model. This is A2UI’s weak spot on small models.

Untrusted text stays text. Two tasks embedded hostile markup inside a data string. Tool components and both A2UI formats treated it as text in every run on every model: 0 of 240 script executions each. Generated HTML let the injected script execute in 61 of 240 hostile-case runs. The split is by tier: 60 of 120 on the small models, 1 of 120 on the mid-tier models.
The 60 small-model executions were all on the breakout note, in every run on every small model: Haiku 20 of 20, Gemini 3.5 Flash-Lite 20 of 20, GPT-6 Luna 20 of 20. The small models put the note inside a <script> block as a string literal, where its closing </script> tag ends the page’s own script and the note’s injected handler runs next. The plainer hostile note did not execute scripts, but the same mechanism broke the page: the note went missing in 16 of 20 Haiku runs, 16 of 20 Flash-Lite runs, and all 20 Luna runs. On the mid-tier models, the breakout note executed once in 60 runs (Gemini 3.8 Flash), and the other 59 were clean.

Google says A2UI is a declarative format where agents describe what to show rather than sending executable code. In this test the claim held on every model. The script executions in generated HTML ran inside a sandboxed iframe with its own opaque origin and could not reach the host page’s DOM, cookies or storage. I set no content security policy on the generated-HTML host. The generated-HTML system prompt includes one instruction to treat data values as text, but nothing in the host enforces it. In a declarative format the model’s output cannot contain executable code by construction. In generated HTML it depends on what the model writes each time, and the small models wrote it wrong consistently.
Cheaper updates to a live screen. A2UI has messages that change part of a screen that is already showing, so the model sends only the components that changed. With a retry, every method was fully correct or nearly so on every model (the lowest was Express on GPT-6 Luna, at 17 of 20). A2UI still sends far fewer output tokens for a change. On Sonnet, A2UI Direct JSON used a median 126 output tokens for the withdrawn-offer update (down from 1,537 for the first screen). Generated HTML used 1,151.5 (down from 1,205). On Sol: A2UI Direct JSON used 86 (down from 917); generated HTML used 819 (down from 894).
| Method | Sonnet output (first screen) | Sol output (first screen) | Correct |
|---|---|---|---|
| Tool components | 861 (588) | 224 (320) | 120/120 |
| A2UI Direct JSON | 126 (1,537) | 86 (917) | 118/120 |
| A2UI Express | 90 (606.5) | 31 (358) | 115/120 |
| Generated HTML | 1,151.5 (1,205) | 819 (894) | 120/120 |
Output tokens are medians for those two models. The last column counts fully correct runs on all six.
Express needed the retry to get there. Removing an item means changing the top of the screen, and in Express that needs an update=true flag that the prompt generated by Google’s SDK never mentions. On the withdrawn-offer update, Sonnet left the flag off on the first try in all 20 runs and got it right in all 20 once the error came back.
Where it costs you
Prompt cost with caching. The A2UI agent SDK generates a system prompt of about 44,800 characters (Direct JSON) carrying the full v1.0 schema and the component catalog: about 17,400 input tokens on Claude, about 11,000 on the other vendors’ tokenisers. With prompt caching on, A2UI Direct JSON is no longer the dearest method. On Sonnet it cost 1.18 US cents per call against 1.28 for generated HTML. On Sol: 0.76 against 0.81. On Flash, where Gemini only caches prompts above 4,096 tokens, A2UI Direct JSON cost 0.50 and generated HTML 0.47. Switch model in the chart to see all six.
Google says Express cuts latency and token count by up to three times compared with Direct JSON. The inference formats page reports a roughly 45% input reduction on its own evaluation suite. I did not measure their claim directly. What I can say is what the saving cost in correct screens.
Express was less accurate. Express scored 866 of 960 fully correct single-screen runs against 907 for Direct JSON. On the small models it was 402 of 480 against 427. More than half of the overall gap is the seat grid (47 of 120 against 71). Its outputs were not rejected more often than Direct JSON’s: 866 of 960 were valid first try against 857. They were more often valid and wrong. Express is the SDK’s default format, and it is the one I would be slower to trust.
The official renderer has accessibility gaps. In every A2UI run that rendered a text input (239 of 239 Direct JSON, 238 of 238 Express), the label was drawn beside the input without being tied to it, so a screen reader cannot name the field. This is a defect in the @a2ui/web_core 0.15.0 TextField component that held on every model. Tool components and generated HTML both had 0 on this check. The gap matters for screen readers and for automated testing; I had to work around it for every form task.

How I measured it
Six models from three vendors, each called through its own API: Claude Haiku 5.5 and Sonnet 5.5 (Anthropic, $0.10/$0.50 and $2/$10 per million tokens), GPT-6 Luna and GPT-6.1 Sol (OpenAI, $0.10/$0.50 and $2/$10), Gemini 3.5 Flash-Lite and Gemini 3.8 Flash (Google, $0.30/$2.50 and $0.75/$3.75 introductory). Tool components use native tool calling on every model except Sonnet 5.5, which rejects forced tool choice; there the prompt asks for it. Prompt caching is on for every vendor. When the application rejects an output, the errors go back to the model once; “fully correct” is judged after that retry.
I ran ten tasks inspired by a travel-rebooking agent: choosing a flight, filling a passenger form, confirming a change, a fare summary, a hostile note with injected markup, a note that tries to break out, a composite checkout, a seat grid, and two live updates (a price change and a withdrawn offer). Each task ran 20 times on each method and model: 4,798 runs. The harness calls each vendor’s API, stores the raw output, then opens the method’s page in headless Chrome, injects the parsed output, takes a screenshot, checks that every expected value is visible, fills the fields and presses the buttons by their accessible names, and records the action the screen reports.
This replaced an earlier run of 1,598 calls on two Claude models through the CLI, with emulated tool calls, uncached prompts and no retries. That first pass showed the structure; this one tests whether it holds across vendors and model tiers. One notable difference: on Haiku the breakout note executed in 4 of 20 generated-HTML runs then and in 20 of 20 now. The earlier route let the model think before answering and this one has thinking off, which is my suspicion for the difference, not something I tested.
What I’d do on Monday
- Start with prebuilt components for every screen your toolkit covers. They were the cheapest method on every model and the fastest on five of the six (Gemini 3.8 Flash reasoned before calling the tool and was slow to start). With native tool calling almost every output was valid first try.
- Reach for A2UI Direct JSON when the screen is one your components cannot draw, or it shows text you do not control and you want to run a small model. Check any state the screen depends on, such as disabled controls: the small models mostly missed it on the seat grid.
- Do not put untrusted text into generated HTML on a small model without an escaping step you own. The prompt asking nicely was not enough on any of the three small models.
- Cache the A2UI system prompt. With caching, A2UI Direct JSON cost less per call than generated HTML on two of three mid-tier models.
- Test your renderer’s accessibility before shipping. The official A2UI renderer’s label gap held on every model I tested.
- Budget for the update prompt. A2UI’s partial-update messages save output tokens, but the input prompt still travels with every call.
If you’re running agent UI in production with any of these, I’d like to hear what you see. The design analysis is in the earlier post; this one is about the numbers.
Every kind of failure, counted by method, task and model:
Limitations
Twenty runs per cell on six models is still a small test, on one domain, with tasks and a tool kit I wrote. I designed the tool kit with sight of the first five tasks and before the others existed. Each call is a single turn with no streaming transport, and a rejected output gets one retry. Reasoning is off on the small models and at its lowest setting on the mid-tier ones, and I did not test what turning it up would change.
Before these numbers were produced I found and fixed two faults in my harness: the script could not operate a choice drawn as chips, only radio buttons, and the generated-HTML sandbox blocked form submit events that the prompt tells the model to handle. I also changed the tool kit after seeing a result: a free-form JSON payload came back empty under Gemini’s forced tool calling, so a button’s payload now travels as a list of key and value pairs on every vendor.
The generated-HTML host has no content security policy and no escaping step, and its validation checks well-formedness only, so its 0 of 960 rejected is a weaker claim than the schema check the other methods pass. The A2UI Direct JSON prompt includes one sentence about component identifiers that the SDK’s prompt lacks, and the instructions for updating a live screen are mine, because the SDK ships none. The A2UI page does not use the protocol’s sendDataModel option. Gemini caches only prompts above 4,096 tokens, so its smaller prompts were billed in full. The renderer packages are @a2ui/web_core and @a2ui/lit 0.15.0 with @a2ui/agent 0.1.0. The protocol is 1.0. The packages are not there yet.