Discussion on "I built desktop control so an agent could test my Chrome extension. Then I benchmarked it against four free tools."

Kuldeep Singh · 2026-10-15T21:05:32.371Z

Testing my Chrome extension by hand meant opening its side panel and reloading the extension after every change. Browser tools can't reach the side panel. Codex's computer use showed me an agent could

I'd like to see the pc_do call broken out on its own, since folding a form into one or two model calls looks like where most of your time gap comes from. Have you checked whether the model makes more mistakes when it plans a whole form at once, say on the seven-field form with the pre-ticked box?

Good question. I haven't tested pc_do on its own yet, so I can't say for sure how much of the speed comes from it. My guess is most of it, because the model spends far more time thinking between calls than the tool spends clicking.

And yes, doing a whole form in one go did break once, on that exact form. The next keys went in before the dropdown had opened, so one field never got set, but every step still said "ok". That was my bug, not the model's. It waits for the menu now, and the form passed every run in the latest benchmark.

Running the same tasks with pc_do switched off is a good idea. I'll try it and share what I find.

Ran it. Same tasks, same model, with pc_do switched off so every click is its own call.

Both versions passed every run, including that seven-field form, so doing a whole form at once didn't cause extra mistakes. Without it, runs took about 30% longer and made twice as many calls. Most of that was Calculator, where every key is its own button. On web pages the difference was small.

So pc_do is part of the speed but not all of it. The rest seems to come from how heavy each call is.

Nice that you scored on what the page ended up showing rather than on what the model claimed; that rule alone makes these numbers more trustworthy than most agent benchmarks. The Tessa Bright failure is interesting because the tool did exactly what it was told, so a cheap guard could be a pre-action echo for destructive buttons: return the target row's label alongside the element ref and let the model confirm before pressing. It would also be worth reporting the tool-list size as its own column, since 49k vs 3.2k tokens per call probably explains more of the time gap than anything about how the tools actually drive the UI.

Thanks, this is a good catch. Looking back at the failed run, the click answer only said it pressed "Approve", not whose row that was, so the model had nothing to check before it pressed. Putting the row label in that answer is cheap and I'm adding it.

You're right about the tool list too. It's the main reason for the token gap, so it should have its own column. I'd be careful about time, though. Most of a run is the model deciding, and Cua Driver made 441 calls against 216, so I think the call count explains more of the time gap than the list does. I haven't measured the split.

Fair point on call count; 441 vs 216 round trips is a bigger lever than prompt size once the model's thinking time dominates. If you want to separate them cheaply, log time-to-first-token per call: that mostly tracks input tokens, while the gaps between calls track planning, so the two costs show up separately without rerunning anything. Your pc_do-off run fits that too, since the extra 30% came mostly from Calculator's per-key calls.

Search Hashnode

I built desktop control so an agent could test my Chrome extension. Then I benchmarked it against four free tools.

Responses6