Training Data and Test Data

One-sentence summary

A machine learning model learns patterns from training data, and we check whether it truly learned by testing it on test data it has never seen before.

Why does it matter?

In the previous lesson we saw that machine learning is a program finding patterns in examples. But how do we know a model actually works?

Think of it like an exam. All year you study by solving practice questions. But on the exam, your teacher asks new questions, not the same ones. The goal is not for you to memorise the answers, but to understand the topic and apply it to new questions.

Machine learning uses the same idea. We train a model on examples, then test it on examples it has never seen before. If we do not split our data into two parts, we can never tell whether the model has really learned or has simply memorised the answers. This is the basis of judging any AI honestly.

What are training data and test data?

We usually split the examples we have (the data) into two parts.

Training data (for learning)

Training data is the set of examples a model uses to learn patterns. The model sees these examples many times and tries to find the relationships between them.

For instance, if we show a model hundreds of dog and cat photos and tell it what each one is, the model starts noticing patterns like ears, noses and fur. Those photos are the training data.

Test data (for trying it out)

Test data is a set of examples the model has never seen and that we did not use while it was learning. We test the model on these new examples and check how accurately it predicts.

We set the test data aside and keep it hidden. The model never looks at it while learning. This makes the test a fair exam.

Short rule: Test data is data not used in training. If they mix, the exam is not honest.

Why must they be separate? The memorising problem

If we test a model only on examples it has already seen, we get a misleading result.

Memorising is not the same as learning

Suppose a friend prepared for a maths test by memorising the answer key. If you give the same test again, they get full marks. But when the questions change a little, they do not know what to do. They never learned the topic; they only memorised the answers.

In machine learning this is called overfitting. The model practically memorises the training examples; it does very well on them but fails on new examples. Keeping test data separate lets us catch this trap.

Two everyday examples

Example 1 — Studying for a test: You study by solving practice questions (training data). The real exam has new questions (test data). Memorising the answers to the practice questions does not help much on the real exam.

Example 2 — Recognising fruit: If you show a young child only red apples and say "this is an apple," they may be confused by a green apple. Learning without seeing different examples stays incomplete. Test data is exactly what lets us check those different examples.

What is accuracy?

A simple way to measure how well a model does on the test data is accuracy.

Accuracy is calculated like this:

Accuracy = Number of correct predictions / Total number of predictions

For example, if a model correctly predicts 8 out of 10 test examples, the accuracy is 8 / 10 = 0.8, which is 80 percent.

A simple experiment: Guess the fruit by its weight

Suppose we have the weights of some fruits and we make a prediction with a simple rule: if the weight is below 200 grams we say apple, otherwise we say orange. This is not a real model; it is just a simple rule we chose by hand. Let us try this rule on the test data and measure its accuracy.

# Test data: (weight_grams, real_fruit)
test_data = [
    (150, "apple"),
    (180, "apple"),
    (250, "orange"),
    (300, "orange"),
    (210, "orange"),
]

# Simple rule: lighter than 200 grams is apple, otherwise orange
def predict(weight):
    if weight < 200:
        return "apple"
    return "orange"

correct = 0
for weight, real in test_data:
    if predict(weight) == real:
        correct += 1

accuracy = correct / len(test_data)
print("Correct predictions:", correct, "/", len(test_data))
print("Accuracy:", accuracy)

In this short experiment we did not train a model; we only tried a simple rule on the test data and counted its accuracy. Yet the core idea is the same: test the model on data it has not seen and measure its accuracy. Accuracy alone is not always enough, but it is an easy and honest way to start.

Hands-on mini task

Design a small "exam" with a friend.

1. On a sheet of paper, write 10 short general-knowledge questions and their answers. These are the training data.
2. Let your friend study until they have memorised those 10 questions.
3. Now prepare 5 new questions on the same topic. These are the test data.
4. Let your friend answer the 5 new questions and calculate the accuracy: number correct divided by 5.

If the topic was truly learned, the score on the new questions will also be high. If your friend only memorised the first 10 answers, they will struggle with the new questions. This is exactly why we separate training and test data.

Common mistakes

Using test data in training too

If you both teach and test with the same examples, the result is misleading. The model scores high because it has seen those examples, but that does not show real ability. Test data must stay separate and hidden.

Deciding based on too little data

Testing with only 2 or 3 examples and saying "the model is great" is not fair. The more test examples you have, and the more varied they are, the more reliable the result.

Treating high accuracy as everything

100 percent accuracy is not always good news; sometimes it just shows the model memorised. Always check accuracy on new, unseen data.

Ignoring how similar the test and training examples are

If you train only on red apples and test on green apples, low accuracy is no surprise. The data should reflect the real world.

Safety note

• Do not upload personal information (names, addresses, phone numbers or photos of yourself or others) into online tools to train or test a model. Use non-personal examples like weights or numbers in your experiments.
• Use AI tools under the guidance of an adult and follow each platform's age rules.
• Even with high accuracy, a model can make mistakes and be wrong. Do not accept a model's output on its own for an important decision; check it together with an adult.
• Remember: AI is not magic; it is just a tool that finds patterns in data. We humans are responsible for how we use it and which data we give it.

Lesson summary

• Training data is the examples a model uses to learn; test data is unseen examples used to check it.
• If we do not split the data, we cannot tell whether the model learned or merely memorised.
• Memorising training examples but failing on new ones is called overfitting.
• Accuracy is the ratio of correct predictions to total predictions, and a model must always be tested on unseen data.
• Do not upload personal data to online tools, and verify important results with an adult.

Review questions

1. What is the main difference between training data and test data?
2. Why do we not use test data during training?
3. What does overfitting mean? Give an example.
4. If a model correctly predicts 15 out of 20 test examples, what is its accuracy?
5. A friend says, "My model is 100 percent correct on the training data." Does this prove the model is great? Why?

Answers

1. Training data is the examples a model uses to learn; test data is examples it did not see while learning, set aside for a fair exam.
2. Because if we both teach and test with the same examples the result is misleading; the model scores high simply because it has seen them, which does not show its real ability.
3. Overfitting is when a model memorises the training examples and does very well on them but fails on new examples. For example, memorising an answer key and then struggling with new questions.
4. Accuracy = 15 / 20 = 0.75, which is 75 percent.
5. No, it does not. Success on the training data may come from the model memorising. To see real ability, you must test it on unseen test data.

Source and verification note

For “Training Data and Test Data”, verification focuses on whether the relationship between What are training data and test data? and Test data (for trying it out) remains consistent across examples. Datasets in this module are small and educational; real personal data should not be used. An AI result should be evaluated not only for accuracy but also for data balance, error distribution and explainability.

Next lesson

Faulty and Biased Data: How missing, faulty or biased data affects a model's results, and what to watch for so we can notice it.