Latency
By HubAI editorial team · Source checked: 24 October 2026What does it mean?
The delay between a request and a response or milestone. In AI products, time to first token and time to the complete usable answer measure different things.
Picture this
One assistant begins typing immediately but takes a long time to finish. Another starts later yet returns the complete short answer sooner. Which feels faster depends on the job.
Where you’ll come across it
Interactive assistants, voice applications and automated workflows. Measure realistic end-to-end requests, including retrieval and tools, rather than only an isolated model call.
What it doesn’t mean
A single fast demo is not a service guarantee. Compare typical and slow-case behaviour, concurrent use and the time required for human correction.
Measure the answer you can actually use
A short exercise · use fictional or non-sensitive material.
- Pick a short fictional task and start a timer when you submit it.
- Record both the first visible response and the completed usable answer. Include checking and any necessary retry separately.
- Repeat a few times under comparable conditions. Do not turn a small sample into a service guarantee.
Are you comparing the same milestone, rather than first token against finished answer?
Did that make it clearer?
Likes are a private on-page choice, not a public rating. For a correction or complaint, send the pre-filled form to our editorial inbox. Nothing is sent until you submit it.Check the source
Written and explained by HubAI. The reference below lets you go deeper; it is not an endorsement of a particular product.
Technical reference: developers.google.com ↗