Throughput
By HubAI editorial team · Source checked: 24 October 2026What does it mean?
The amount of work a system completes in a period, such as requests or tokens per second. It describes capacity and is not the same as the delay experienced by one user.
Picture this
A batch service processes many documents per hour, while a live chat service prioritises quick responses to individual users. They may optimise for different measurements.
Where you’ll come across it
Capacity planning, batch processing and comparing serving setups. State the unit, hardware and workload when quoting a number.
What it doesn’t mean
High throughput can coexist with long queues. Count successfully completed useful work, and include failures and retries when estimating real capacity.
Keep a tiny scorecard
A short exercise · use fictional or non-sensitive material.
- Choose five representative fictional tasks and define what counts as correct before running them.
- Record success, correction needed and time to a usable result for each attempt.
- Repeat after one change, keeping the task set fixed. Treat this as an exploratory check, not a reliable benchmark.
What failed, and would that failure matter to the person using the result?
Did that make it clearer?
Likes are a private on-page choice, not a public rating. For a correction or complaint, send the pre-filled form to our editorial inbox. Nothing is sent until you submit it.Check the source
Written and explained by HubAI. The reference below lets you go deeper; it is not an endorsement of a particular product.
Technical reference: developers.google.com ↗