Distillation
By HubAI editorial team · Source checked: 24 October 2026What does it mean?
Training a model using outputs or signals from another model, often to transfer useful behaviour into a smaller or more efficient system. The result is not a lossless copy.
Picture this
A larger model supplies examples for a smaller classifier. The team checks the smaller model on a held-out set rather than assuming it inherits all the larger model’s strengths.
Where you’ll come across it
Reducing serving cost or latency for a bounded job. It is most useful when you can define the behaviour you need and measure it.
What it doesn’t mean
Teacher mistakes can pass to the student. Training-data rights and provider terms also matter; access to outputs is not blanket permission for every training use.
Compare on a job, not a label
A short exercise · use fictional or non-sensitive material.
- Choose a small set of non-sensitive tasks, including an awkward case and a missing-information case.
- Keep the inputs and scoring checklist the same when you compare two model setups.
- Record useful completions, corrections and elapsed time. Keep the model name and settings with the results.
Is the improvement still useful after you include checking and correction time?
Did that make it clearer?
Likes are a private on-page choice, not a public rating. For a correction or complaint, send the pre-filled form to our editorial inbox. Nothing is sent until you submit it.Check the source
Written and explained by HubAI. The reference below lets you go deeper; it is not an endorsement of a particular product.
Technical reference: developers.google.com ↗