# gaisho benchmarks

Every agent shops better with gaisho. Same agents, same shopping queries — run native, then with gaisho plugged in.

Headline figures (best agent per metric):

- 69% fewer tokens per task: Dot, 110,400 → 34,150 tokens
- 50× more matching products: median per query 358.5 with gaisho vs. 5–7 native (72× vs. Dot and MUSE, 51× vs. Grok)
- 73% less time per answer: Grok, 879.3 s → 237.5 s

## Paired results

Each agent native and with gaisho, averaged across 30 simulated shopping queries.

| Agent | Condition | Answer quality | Effective coverage | Avg. tokens | Avg. RT | Risk deduction |
|-------|-----------|----------------|--------------------|-------------|---------|----------------|
| Dot   | Native    | 83             | 54                 | 110,400     | 60.9 s  | −2             |
| Dot   | + gaisho  | 98             | 87                 | 34,150      | 48.3 s  | 0              |
| Grok  | Native    | 93             | 74                 | 1,112,513   | 879.3 s | −5             |
| Grok  | + gaisho  | 99             | 95                 | 687,701     | 237.5 s | 0              |
| MUSE  | Native    | 87             | 73.33              | 368,768     | 415.4 s | −3             |
| MUSE  | + gaisho  | 99             | 91                 | 242,833     | 121.3 s | 0              |

Quality and effective coverage: higher is better. Average tokens and response time: lower is better. Risk deductions are signed points; zero means no deduction. RT is in seconds. The page charts tokens and response time with each agent's native run set to 100.

## Product coverage

Matching products per query, native vs. with gaisho. Medians: gaisho 358.5; Dot 5, Grok 7, MUSE 5.

| Query                                                                                                                                                | gaisho | Dot native | Grok native | MUSE native | Note                                                            |
|------------------------------------------------------------------------------------------------------------------------------------------------------|--------|------------|-------------|-------------|-----------------------------------------------------------------|
| D1: What are some good robot vacuums for a home with two cats? It's mostly hardwood floors with a couple of rugs.                                    | 680    | 5          | 6           | 6           |                                                                 |
| D2: I'm looking for a washable 8x10 area rug for my living room, something neutral. What are my options?                                             | 0      | 9          | 8           | 5           | gaisho returned 0 exact matches (15 partial matches) on Oct 22. |
| D3: I want to start making espresso at home. Which machines should a beginner look at?                                                               | 1,714  | 7          | 8           | 5           |                                                                 |
| D4: Which magnesium supplements are worth considering for better sleep?                                                                              | 2,929  | not run    | 14          | 6           | Not run on Dot.                                                 |
| F1: I need a replacement water filter for my Samsung fridge, part number DA29-00020B. Genuine Samsung or an NSF-certified compatible one, under $40. | 0      | 5          | 2           | 3           | Category not indexed by gaisho on Oct 22. Dot: at least 5.      |
| F3: Queen sheet set, 100% cotton percale, deep pockets that fit a 16-inch mattress, under $80.                                                       | 37     | 3          | 3           | 2           |                                                                 |

Native: Dot (Nov 4, 5 of the 6 queries), Grok (Nov 4) and MUSE (Oct 22). + gaisho: gaisho results for the same 6 queries (Oct 22).

## Method

Each chart compares the same agent in two conditions: native, and with gaisho plugged in. gaisho provides shopping search capabilities and is not a standalone competing agent.

**Token usage** measures total tokens consumed per task.

**Response time** measures the time from query submission to a complete response. Token usage and response time are averaged across 30 simulated shopping queries covering budget limits, specific brands or models, product specifications, use-case recommendations, product comparisons, and cross-platform price comparisons. All agents receive the same queries and are evaluated using the same criteria. Reductions are calculated relative to the native result.

**Product coverage** counts the matching products an agent puts in front of the user for one query. Native: the distinct products the agent actually examined that meet every constraint in the query (Dot and Grok on Nov 4, 2026; for MUSE, the products its browser agent verified on Oct 22, 2026). With gaisho: the exact matches gaisho returned for the same queries on Oct 22, 2026. gaisho’s results don’t depend on the host agent, so one figure serves all three. Medians are taken per query over 6 shared discovery queries; Dot ran 5 of them. The count is how many qualifying options a query surfaces, not recall against a full catalog.

**Reading the charts.** In Token efficiency and Response time, each agent’s native run is set to 100 and the gaisho bar shows the same agent with gaisho as a share of it; raw averages are labeled above the bars and the tag shows the change. In Product coverage, one dot is one matching product (median per query); gaisho is drawn once because its results are the same for every agent. The large figures at the top of the page and above each chart are the best result among the three agents; the charts show all three.