DATA SCIENCE 9 min read · Updated 13 October 2026

Overfitting vs underfitting: the bias-variance trade-off explained

A model that memorises fails on new data; a model that is too simple fails everywhere. How to recognise overfitting and underfitting, what bias and variance mean, and the practical fixes.

Careers Ninza data faculty

Careers Ninza · Kolkata, India

Learn this live in Program Zero

Overfitting means a model has memorised its training data, including noise, so it performs well there but poorly on new data. Underfitting means the model is too simple to capture the real pattern and performs poorly everywhere. The bias-variance trade-off describes the balance between them, and the goal is a model that generalises to data it has never seen.

Every machine learning project runs into this problem, and it is one of the most common interview topics. The good news is that it is easy to detect if you split your data properly, and there are reliable fixes.

What do overfitting and underfitting look like?

UnderfittingGood fitOverfitting

Training performancePoorGoodExcellent

Validation performancePoorGood, close to trainingMuch worse than training

CauseModel too simple, too few featuresRight complexityModel too complex, too little data

AnalogyStudent who did not studyStudent who understoodStudent who memorised past papers

What are bias and variance?

—Bias is error from wrong assumptions: a straight line trying to fit a curve. High bias leads to underfitting.

—Variance is error from sensitivity to the particular training data: a model that changes wildly if you change a few examples. High variance leads to overfitting.

—The trade-off: making a model more flexible usually lowers bias but raises variance. The best model balances the two for your data size.

How do you detect overfitting?

Hold out data the model never trains on. Split your data into training, validation and test sets, or use cross-validation for smaller datasets. If training accuracy is much higher than validation accuracy, the model is overfitting. Plotting training and validation loss over time shows the moment validation loss starts rising while training loss keeps falling.

Data leakage creates fake good results: information from the test set sneaking into training, such as scaling data before splitting or using features that would not be known at prediction time. Leakage hides overfitting until the model fails in production.

How do you fix overfitting?

—Get more, and more varied, training data.

—Simplify the model: fewer features, shallower trees, smaller networks.

—Use regularisation such as L1 and L2 penalties that discourage extreme weights.

—For neural networks, use dropout, data augmentation and early stopping.

—Use ensembles such as random forests that average out variance.

—Use cross-validation to choose settings rather than tuning on the test set.

How do you fix underfitting?

—Use a more expressive model, such as gradient boosting instead of a linear model.

—Engineer better features that capture the real signal.

—Train longer or reduce over-strong regularisation.

—Check the data: sometimes the pattern simply is not there.

What does this look like in a real project?

Suppose you predict delivery times for a food platform in Bengaluru. A linear model on distance alone underfits: it ignores traffic and restaurant load. A very deep tree trained on a small dataset overfits: it memorises that one restaurant was slow on one rainy Tuesday. A gradient-boosted model with sensible features such as distance, time of day, weather and restaurant load, tuned with cross-validation, generalises well. Testing on later weeks, not random rows, confirms it works on the future.

Does this apply to deep learning and LLMs?

Yes, though large models behave in some surprising ways, sometimes generalising well despite having more parameters than training examples. Fine-tuning a language model on a small dataset can still overfit: it starts repeating training examples or loses general ability. Evaluation on held-out data remains the safeguard. Our guide to evaluating LLM applications covers that side.

How much data do you need to avoid overfitting?

There is no fixed number; it depends on the model’s complexity and how noisy the problem is. A useful rule is to compare learning curves: train on increasing amounts of data and watch validation performance. If it keeps improving, more data will help; if it has flattened, you need better features or a different model instead. Transfer learning, starting from a pretrained model, sharply reduces how much data you need for images and text.

What is the difference between validation and test sets?

The validation set guides your choices during development: which model, which settings, when to stop training. The test set is touched once, at the very end, to report how the final model performs on unseen data. If you tune against the test set, it stops being an honest estimate, and your reported results will be optimistic.

How do you explain this in an interview?

State the definitions, describe how you detected the problem with a validation set or curves, explain the fix you chose and why, and mention leakage. A concrete example from your own project is far more convincing than textbook definitions. Our guide to data science portfolio projects shows how to present that.

Program Zero covers model evaluation, cross-validation and the bias-variance trade-off in its classical machine learning phase and regularisation for neural networks in the deep learning phase.

Program Zero runs live online, so you can learn this from anywhere in India: metros such as Chennai, Hyderabad, Bengaluru and Pune, growing cities such as Coimbatore, Kochi, Indore and Nagpur, and smaller towns such as Ranchi, Patna, Bhubaneswar and Raipur. Every learner joins the same evening classes, at the same fee.

Want to learn this live, with mentors?

Program Zero is Careers Ninza’s 18-month live programme in AI, LLMs and full-stack development: classes Monday to Friday, 9:00–10:30 PM IST, all recorded, Saturday projects and Sunday doubt sessions. ₹17,999 for all 18 months; the batch starts 9 January 2027.

See what Program Zero teaches Explore Program Zero

PROGRAM ZERO · 18-MONTH LIVE PROGRAMME

Program Zero: AI, LLM & Full-Stack Developer Programme

Learn AI, large language models and full-stack development live with mentors: from your first line of code to building and deploying your own language model.

₹17,999 for all 18 monthsStarts 9 January 2027Mon–Fri, 9:00–10:30 PM ISTEvery class recorded

BOOK YOUR SEAT SEE FULL CURRICULUM →

Frequently asked questions

What is overfitting in machine learning?+

Overfitting happens when a model learns its training data too closely, including noise, so it performs very well on training data but poorly on new data. It is usually caused by a model that is too complex for the amount of data.

What is underfitting?+

Underfitting happens when a model is too simple to capture the real pattern, so it performs poorly on both training and new data. More expressive models, better features or less regularisation usually help.

What is the bias-variance trade-off?+

Bias is error from overly simple assumptions and variance is error from sensitivity to the particular training data. Making a model more flexible usually lowers bias but raises variance, so the goal is a balance that generalises well.

How do I know if my model is overfitting?+

Compare performance on training data with performance on a validation set the model never trained on. A large gap, or validation loss rising while training loss keeps falling, indicates overfitting.

How can overfitting be prevented?+

Use more and more varied data, simpler models, regularisation such as L1 or L2, dropout and data augmentation for neural networks, early stopping, ensembles, and cross-validation for choosing settings.

What is cross-validation?+

Cross-validation splits data into several parts, trains on some and validates on the rest, rotating through the parts. It gives a more reliable estimate of performance on new data, especially for smaller datasets.

What is data leakage?+

Data leakage is when information that would not be available at prediction time, such as test data or future values, influences training. It produces misleadingly good results that collapse in production.

Where can I learn model evaluation and machine learning in India?+

Program Zero by Careers Ninza covers evaluation, cross-validation, the bias-variance trade-off and regularisation across its machine learning and deep learning phases, within an 18-month live programme for Rs 17,999.

We teach this, live

Every article here comes from something we teach. Sit in on a free masterclass and judge the mentors yourself.

FREE MASTERCLASS CALL +91 90888 39993