Say it like this
- run inference on new data
- inference latency
- inference performance
- inference pipeline
- real-time inference
Use it at work
AI work
- Inference costs are higher than training now.
- Let's move inference to GPUs.
Performance
- Inference takes 300 ms per request.
- Can we batch inference requests?
General
- From the logs, my inference is that the job never started.
- That's an inference, not a fact. Let's check.
Don't say this
Inference is the process of applying a trained model, not improving it. Training improves the model itself.
Don't mix it up
| Term | What it is | Who writes or uses it | When |
|---|---|---|---|
| inference | Using a trained model to predict | ML engineers | In production |
| training | Teaching a model from data | ML engineers | Before use |
Practice
Try it
4 quick exercises
1. After training the neural network for two weeks, we deployed it to production and began _______ on real customer data.
2. You are explaining the model lifecycle. Which sounds most natural?
3. Which sentence uses inference incorrectly?
4. Rewrite this using the word inference: 'Our system uses the machine learning model to make decisions about which products to show users.'
Say it out loud
Phrase 1 of 5
"run inference on new data"
We don't save your recording. The word check uses your browser's speech recognition.
More about inference
Real examples(3)
- in a standup
"We deployed the updated model yesterday, and inference times have dropped by 30% on the production servers."
- in a code review
"This inference function needs to handle edge cases more gracefully when the model receives unexpected input formats."
- in a team meeting
"Our inference costs are climbing because the model is too large for edge devices, so we're looking into quantization."
More mistakes(2)
Handle with care. Often confused with prediction or assumption. Inference specifically means the process of running a trained model on new data to produce output, not the process of training.
- ✕The inference phase happens before deployment.✓The training phase happens before deployment; inference happens after deployment in production.
Inference occurs when the model is live and processing real data, not before deployment.
- ✕We are doing inference on our training dataset.✓We're running inference on our test dataset or new unseen data.
Inference is meant for new, unseen data. Running inference on training data does not validate real-world performance.
Other forms(3)
- inference noun
"The inference latency must be under 50 milliseconds for our use case."
- run inference / perform inference verb phrase
"We run inference on millions of images daily using our GPU cluster."
- inference (used attributively) adjective
"The inference pipeline includes preprocessing, model execution, and post-processing steps."
Often used with(6)
- inference latency · The time it takes for a model to produce output; critical for real-time systems
- inference cost · The computational or financial expense of running predictions
- inference engine · The software or hardware component that executes inference operations
- batch inference · Processing multiple inputs together rather than one at a time
- real-time inference · Producing predictions with minimal delay for immediate use
- edge inference · Running models on devices at the edge of the network rather than in the cloud
Similar words(3)
- prediction · Similar meaning but less technical; inference emphasizes the computational process
- model serving · Broader term covering the infrastructure and deployment of inference
- execution · More general; inference is the specific execution of a trained model
Opposites(2)
- training
- learning
Questions(5)
- What does inference mean in software engineering?
- Inference is the process of using a trained machine learning model to generate predictions or outputs on new data. It is the opposite of training—training teaches the model, and inference applies what it has learned. In production systems, inference happens continuously when the model processes real-world inputs.
- How do you use inference in a sentence at work?
- You typically use inference as a noun: 'We need to reduce inference latency,' 'Run inference on the test set,' or 'The inference engine processes queries.' You can also use it in phrases like 'inference cost,' 'inference pipeline,' or 'batch inference.'
- Is inference formal or informal?
- Inference is formal and technical. It is used in technical discussions, meetings, documentation, and code reviews. Non-native speakers should use it confidently in professional settings without worrying about it sounding too casual.
- What is the difference between inference and prediction?
- Inference and prediction are related but not identical. Inference is the computational process of running a model on data. Prediction is the actual output or result. So you run inference to make a prediction. In casual conversation, they overlap, but in technical writing, inference is more precise.
- Why do teams talk about inference latency?
- Inference latency is the time it takes a model to produce output. For user-facing systems like search engines, recommendation engines, or mobile apps, low latency is critical. If inference is too slow, users experience delays, so teams focus on optimizing inference latency as a key performance indicator.
