Hopscotch and Cheaper Inference, call for call.
The public watcher sends the same prompts to the same models through both gateways. This is every result behind our ads, the losses included, with the model we left out and the conditions each call ran under.
No markup or fees
Long answers: past five minutes, one gateway keeps streaming.
On Cheaper Inference, no answer finished after five minutes, and 14 of its 19 stream cuts landed between 300 and 307 s. Hopscotch keeps the stream open for up to nine, so an answer that needs six gets to its last line.
124 calls. The chart leaves out 8 Cheaper Inference and 2 Hopscotch calls that were refused before any output; the table below counts them.
answers finished whole on Hopscotch after five minutes
The longest took 534 s, and none finished that late on Cheaper Inference.
answers stopped or refused by the gateway
Cheaper Inference against Hopscotch, out of 62 identical tasks each. Hopscotch finished 37 whole and Cheaper Inference 30.
Kimi K3 answers finished whole on Hopscotch
Against 6 of 12 on Cheaper Inference, the clearest case of the five models.
DeepSeek V4 Flash
2 stopped by the gateway · 0 out of tokens
1 stopped by the gateway · 0 out of tokens
GLM-5.2
0 stopped by the gateway · 6 out of tokens
3 stopped by the gateway · 4 out of tokens
GLM-5.3 Flash
2 stopped by the gateway · 10 out of tokens
11 stopped by the gateway · 0 out of tokens
Kimi K3
2 stopped by the gateway · 0 out of tokens
5 stopped by the gateway · 1 out of tokens
MiniMax M2.7
0 stopped by the gateway · 3 out of tokens
7 stopped by the gateway · 0 out of tokens
Qwen3.6, left out
- The nine-minute limit [LIMIT]
- 4 Hopscotch answers were still streaming when it hit at 540 s. If your jobs run past nine minutes, split them.
- The wrong advice [OUR BUG]
- When Hopscotch stops a stream at nine minutes, its error message tells you to stream the response, even though you already are.
- Refused calls [REFUSED]
- Hopscotch refused 2 calls before any output, against 8 on Cheaper Inference.
- Billing [TIE]
- Neither gateway charged for the streams it stopped, so the difference is whether the answer finishes, not what you pay.
How we measured long answers.
The watcher runs in public and keeps every call, so you can check each dot on the chart against the raw data.
- Where
- A Cloudflare Worker in London streams each task to both gateways at the same moment.
- When
- Every 30 minutes, one model per run in turn. This snapshot is 23 and 24 Sep 2026, 62 tasks per gateway.
- Tasks
- Either a 24-function JavaScript module with docs and tests, or a full pricing page in HTML with light and dark themes.
- Settings
- stream: true, max_tokens 24k, and a 10-minute client timeout, so neither the client nor the token budget is what ends an answer inside the window being tested.
- Done
- Every answer has to end on an exact final line. If that line is missing, the answer was cut, whatever the status code says.
- Causes
- Cut mid-stream, refused before output, or a lowered token cap count against the gateway. A model that spends its whole budget counts against neither.
- Left out
- Qwen3.6, on both gateways. Hopscotch loses on it: 7 of 22 long answers finished whole, against 17 of 24 on Cheaper Inference, because Hopscotch's Qwen3.6 host was slow and refused calls for stretches.
Doesn't raising max_tokens fix it?+
No, because the five-minute limit is on time, not tokens, and every task already had a 24k-token budget.
Is this cherry-picked?+
Every call is on the chart, including the ones that make Hopscotch look bad, and the raw data is public on the watcher.
What are the grey dots?+
Those are answers where the model spent its whole 24k-token budget without reaching the last line. That is the model, not the gateway, so it doesn't count against either one.
Why leave Qwen3.6 out of both?+
Because leaving it out of one would be the cherry-picking. The numbers are in the table above.
What a response says about how it was served.
On 23 and 24 Sep 2026, every call Hopscotch served named both the model and the host in its headers. Cheaper Inference names the model in the response body, and over the same days none of its calls named the host.
- status
- HTTP 200
- x-hopscotch-served-by
- moonshot-kimi/kimi-k3
- x-hopscotch-provider
- moonshot
- x-hopscotch-retry-attempt-count
- 0
- x-hopscotch-served-via
- not sent
- provider_elapsed_ms
- 5969
- outside_provider_ms
- 1008
- status
- HTTP 200
- body.model
- kimi-k3
- host
- not sent
- retries
- not sent
- provider time
- not sent
Hopscotch calls carried the retry count and the provider's own time
Every successful call, on all five models.
Cheaper Inference calls carried either one
The model name comes back in the body, and nothing else about the route.
median time outside the provider on Hopscotch
Wall time minus the provider's own time, so the most the gateway could have added.
- deepinfra
- DeepSeek V4 Flash
- z-ai
- GLM-5.2, GLM-5.3 Flash
- moonshot
- Kimi K3
- minimax
- MiniMax M2.7
- No retries to show [0 IN WINDOW]
- No call in this window needed an extra attempt or a backup route, so Hopscotch's count said 0 every time. It is on the response either way.
- The same model either way [TIE]
- Both gateways served the model we asked for on effectively every call. Cheaper Inference just doesn't say which host served it.
- Its own overhead figure [NOT COMPARABLE]
- Cheaper Inference reports a routing overhead of its own instead of the provider's time, so its calls can't be split the same way.
How we measured responses.
The watcher keeps the full receipt for every call, so you can check each line on this page.
- Prompt
- Five short questions, each with one right answer, streamed to the same five models on both gateways every 5 minutes from London.
- Receipt
- The watcher keeps the headers and usage of every response, and checks each line against them.
- Window
- 23 and 24 Sep 2026, 1,954 successful Hopscotch calls and 1,892 on Cheaper Inference.
- Models
- DeepSeek V4 Flash, GLM-5.2, GLM-5.3 Flash, Kimi K3 and MiniMax M2.7, the same on both gateways. Qwen3.6 is left out here too.
Run your own calls.
Make an account, point your base URL at Hopscotch, and every response comes back with its receipt.
No markup or fees