Test doubles (hands-on): mocks, stubs, fakes and spies, and how a mock lets a real bug through

Your checkout code calls a payment provider. You don't want your tests to charge a real card, wait two seconds for the network, or fail because the provider's test server is down. So you replace the provider with something that pretends to be it. That stand-in is a test double, named after the stunt double who stands in for the actor.

Test doubles make tests fast and reliable. They can also make tests lie. A double only knows what you told it, and you told it the same thing you believed when you wrote the code. If that belief is wrong, the code and the double are wrong together, and the test passes.

This post shows the five kinds of double with one example. Then it runs a real experiment: the same checkout code, tested with four kinds of double, with two bugs planted in it. Every command and line of output below was run while writing this post.

The five kinds of test double

The names come from Gerard Meszaros's book xUnit Test Patterns (2007). People often call all of them "mocks", but they do different jobs:

Double	What it does	FoodNow example
Dummy	Fills a parameter that the test never uses	A logger=None passed in only because the function needs one
Stub	Gives fixed answers to the calls the code makes	The payment provider always answers "declined", so you can test the declined path
Spy	A stub that also records how it was called, for checks afterwards	Records the amount charged, so the test can check it was 55,000 paise
Mock	Set up with the calls it expects; the test fails if they don't happen	"charge must be called exactly once"
Fake	A working, simpler version of the real thing	An in-memory payment provider that approves, declines and remembers charges

In Python, unittest.mock.MagicMock can act as a dummy, a stub, a spy or a mock depending on how you use it. A fake is usually a small class you write yourself.

Step 1: The code under test

You need Python 3.10 or newer and pytest:

mkdir foodnow-pay && cd foodnow-pay
python -m venv .venv && source .venv/bin/activate
pip install pytest==9.2.0

We used Python 3.11 and pytest 9.2.0 on Linux. unittest.mock is part of Python, so there's nothing else to install.

gateway.py is the client for the payment provider. It sends one HTTP request and returns a Payment with a status of "succeeded" or "declined". The amount is in paise, the smallest unit of the rupee, and every charge needs an idempotency key, so the provider can tell a retry from a new charge.

import json
import os
import urllib.request
from dataclasses import dataclass

PAYMENTS_URL = os.environ.get("PAYMENTS_URL", "https://api.payments.example")


@dataclass
class Payment:
    id: str
    status: str  # "succeeded" or "declined"


class PaymentGateway:
    """Client for the payment provider's HTTP API."""

    def charge(self, amount_paise: int, idempotency_key: str) -> Payment:
        request = urllib.request.Request(
            f"{PAYMENTS_URL}/v1/charges",
            data=json.dumps({"amount": amount_paise, "currency": "INR"}).encode(),
            headers={"Content-Type": "application/json", "Idempotency-Key": idempotency_key},
            method="POST",
        )
        with urllib.request.urlopen(request, timeout=5) as response:
            body = json.load(response)
        return Payment(id=body["id"], status=body["status"])

(.example is a domain reserved for examples, so this URL never reaches anyone.)

checkout.py charges the customer and marks the order paid or failed. The developer wrote it from memory of an older version of the client:

from gateway import PaymentGateway


def pay_for_order(order, gateway=None):
    gateway = gateway or PaymentGateway()
    payment = gateway.charge(amount=order["total"])
    if payment.ok:
        order["status"] = "paid"
    else:
        order["status"] = "payment_failed"
    return order

It has two bugs:

• Bug A, wrong arguments. The client takes amount_paise and idempotency_key. This passes amount=, in rupees, and no key.
• Bug B, a field that isn't there. Payment has a status, not an ok.

In production, the first bug makes every checkout crash before charging anyone. Fix only that one, and the second makes every checkout crash after the customer was charged, with the order still unpaid.

Step 2: MagicMock: both bugs, all green

The most common way to test this is a MagicMock in place of the gateway:

from unittest.mock import MagicMock

from checkout import pay_for_order


def test_successful_payment_marks_the_order_paid():
    gateway = MagicMock()

    order = pay_for_order({"id": 7, "total": 550}, gateway)

    assert order["status"] == "paid"
    gateway.charge.assert_called_once()


def test_declined_payment_marks_the_order_failed():
    gateway = MagicMock()
    gateway.charge.return_value.ok = False

    order = pay_for_order({"id": 8, "total": 550}, gateway)

    assert order["status"] == "payment_failed"

$ python -m pytest -q tests/test_with_magicmock.py
..                                                                       [100%]
2 passed in 0.02s

Both pass, with both bugs in the code. Two reasons:

1. A MagicMock accepts any call. charge(amount=550) is fine, and so is charge(banana=True). It has no idea what the real charge looks like.
2. Any field you ask a MagicMock for is another MagicMock, and a MagicMock counts as true. payment.ok doesn't exist on a real Payment, but on the mock it's there, and it's truthy, so the order is "paid".

Look at the second test again: gateway.charge.return_value.ok = False. The person who wrote the code believed a payment has an ok field, and they wrote that same belief into the test. The mock agrees with the code because the same person wrote both.

Step 3: An autospec mock catches the wrong arguments

create_autospec builds a mock from the real class. It only allows the methods the class has, called the way the class defines them:

from unittest.mock import create_autospec

from checkout import pay_for_order
from gateway import PaymentGateway


def test_successful_payment_marks_the_order_paid():
    gateway = create_autospec(PaymentGateway, instance=True)

    order = pay_for_order({"id": 7, "total": 550}, gateway)

    assert order["status"] == "paid"

tests/test_with_autospec.py F                                            [100%]

=================================== FAILURES ===================================
_________________ test_successful_payment_marks_the_order_paid _________________
tests/test_with_autospec.py:10: in test_successful_payment_marks_the_order_paid
    order = pay_for_order({"id": 7, "total": 550}, gateway)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
checkout.py:6: in pay_for_order
    payment = gateway.charge(amount=order["total"])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.../lib/python3.11/unittest/mock.py:1122: in __call__
    self._mock_check_sig(*args, **kwargs)
.../lib/python3.11/unittest/mock.py:131: in checksig
    sig.bind(*args, **kwargs)
.../lib/python3.11/inspect.py:3195: in bind
    return self._bind(args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
.../lib/python3.11/inspect.py:3110: in _bind
    raise TypeError(msg) from None
E   TypeError: missing a required argument: 'amount_paise'

(We shortened the path to Python's own files to .../lib/python3.11.)

Bug A is caught. The developer fixes the call:

    payment = gateway.charge(
        amount_paise=order["total"] * 100, idempotency_key=f"order-{order['id']}"
    )

Now the autospec test passes, and so do the MagicMock tests:

$ python -m pytest -q tests
...                                                                      [100%]
3 passed in 0.04s

Bug B is still there. An autospec mock checks how you call a method, but what the method returns is still a plain MagicMock, so payment.ok is still a truthy mock.

Step 4: A stub with a real answer catches the missing field

The fix for the test is to make the double answer with a real Payment, the same type of object the real client returns. Now it's a stub:

from unittest.mock import create_autospec

from checkout import pay_for_order
from gateway import Payment, PaymentGateway


def stub_gateway(status):
    """Checks the call matches the real signature, and answers with a real Payment."""
    gateway = create_autospec(PaymentGateway, instance=True)
    gateway.charge.return_value = Payment(id="ch_1", status=status)
    return gateway


def test_successful_payment_marks_the_order_paid():
    order = pay_for_order({"id": 7, "total": 550}, stub_gateway("succeeded"))
    assert order["status"] == "paid"


def test_declined_payment_marks_the_order_failed():
    order = pay_for_order({"id": 8, "total": 550}, stub_gateway("declined"))
    assert order["status"] == "payment_failed"

Both tests fail with AttributeError: 'Payment' object has no attribute 'ok': exactly what would happen in production.

Step 5: A fake, for the behaviour you can't stub

Some things a stub can't test, because they depend on what happened before. The provider uses the idempotency key so that a double click, or a retry after a timeout, doesn't charge the customer twice. To test that your code sends the same key for the same order, you need a double that remembers. That's a fake:

from gateway import Payment


class FakeGateway:
    """Works like the provider's sandbox, in memory: no network, same answers."""

    def __init__(self, decline=False):
        self.decline = decline
        self.charges = {}  # idempotency key -> (amount, Payment)

    def charge(self, amount_paise: int, idempotency_key: str) -> Payment:
        if not isinstance(amount_paise, int) or amount_paise <= 0:
            raise ValueError(f"amount must be a positive whole number of paise, got {amount_paise!r}")
        if idempotency_key in self.charges:  # a retry: same answer, no second charge
            return self.charges[idempotency_key][1]
        status = "declined" if self.decline else "succeeded"
        payment = Payment(id=f"ch_{len(self.charges) + 1}", status=status)
        self.charges[idempotency_key] = (amount_paise, payment)
        return payment

The tests read like a description of the business rules. The first also uses the fake as a spy: it checks the amount that was charged.

from checkout import pay_for_order
from fakes import FakeGateway


def test_successful_payment_marks_the_order_paid():
    gateway = FakeGateway()

    order = pay_for_order({"id": 7, "total": 550}, gateway)

    assert order["status"] == "paid"
    assert [amount for amount, _ in gateway.charges.values()] == [55000]


def test_declined_payment_marks_the_order_failed():
    gateway = FakeGateway(decline=True)

    order = pay_for_order({"id": 8, "total": 550}, gateway)

    assert order["status"] == "payment_failed"


def test_paying_twice_charges_the_customer_once():
    gateway = FakeGateway()

    pay_for_order({"id": 9, "total": 550}, gateway)
    pay_for_order({"id": 9, "total": 550}, gateway)  # double click, or a retry

    assert len(gateway.charges) == 1

With Bug A fixed and Bug B still in, all three fail at the same line:

tests/test_with_fake.py FFF                                              [ 66%]
...
checkout.py:9: in pay_for_order
    if payment.ok:
       ^^^^^^^^^^
E   AttributeError: 'Payment' object has no attribute 'ok'

Step 6: Fix the code, and watch the mocks fail

The fix for Bug B is one line:

    if payment.status == "succeeded":

The stub and fake tests now pass. And here is the part most people don't expect. We ran the old MagicMock and autospec tests against the correct code:

tests/test_with_autospec.py F                                            [ 16%]
tests/test_with_fake.py ...                                              [ 66%]
tests/test_with_magicmock.py F.                                          [100%]

=================================== FAILURES ===================================
_________________ test_successful_payment_marks_the_order_paid _________________
tests/test_with_autospec.py:12: in test_successful_payment_marks_the_order_paid
    assert order["status"] == "paid"
E   AssertionError: assert 'payment_failed' == 'paid'
E
E     - paid
E     + payment_failed
_________________ test_successful_payment_marks_the_order_paid _________________
tests/test_with_magicmock.py:11: in test_successful_payment_marks_the_order_paid
    assert order["status"] == "paid"
E   AssertionError: assert 'payment_failed' == 'paid'
E
E     - paid
E     + payment_failed
=========================== short test summary info ============================
FAILED tests/test_with_autospec.py::test_successful_payment_marks_the_order_paid
FAILED tests/test_with_magicmock.py::test_successful_payment_marks_the_order_paid
========================= 2 failed, 4 passed in 0.04s ==========================

The mock tests passed with the bugs and failed with the fix. A MagicMock's status is a MagicMock, not "succeeded", so the correct code marks the order as failed. These tests weren't checking the payment logic. They were checking that the code matched the mock.

Step 7: The patch that missed

There's one more common trap. This test doesn't pass a gateway in. It uses patch to replace the PaymentGateway class while the test runs:

from unittest.mock import patch

from checkout import pay_for_order
from gateway import Payment


@patch("gateway.PaymentGateway")
def test_paid_with_patched_gateway(gateway_class):
    gateway_class.return_value.charge.return_value = Payment(id="ch_1", status="succeeded")

    order = pay_for_order({"id": 7, "total": 550})  # no gateway passed: checkout makes its own

    assert order["status"] == "paid"

gateway.py:25: in charge
    with urllib.request.urlopen(request, timeout=5) as response:
...
E   urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
=========================== short test summary info ============================
FAILED tests/test_with_patch.py::test_paid_with_patched_gateway - urllib.erro...

(We cut the part of the traceback inside Python's network code.)

The test tried to make a real network request. checkout.py did from gateway import PaymentGateway when it was imported, so it has its own name for the class. Patching gateway.PaymentGateway changes the name in gateway, not the one checkout uses. Here the URL goes nowhere, so the test failed. With a real provider URL in your test settings, it would have sent a real charge to the provider.

Patch where the name is used, not where it's defined:

@patch("checkout.PaymentGateway")

$ python -m pytest tests/test_with_stub.py tests/test_with_fake.py tests/test_with_patch.py
============================= test session starts ==============================
platform linux -- Python 3.11.17, pytest-9.2.0, pluggy-1.7.0
rootdir: ~/foodnow-pay
collected 6 items

tests/test_with_stub.py ..                                               [ 33%]
tests/test_with_fake.py ...                                              [ 83%]
tests/test_with_patch.py .                                               [100%]

============================== 6 passed in 0.03s ===============================

Passing the gateway in as an argument, as the other tests do, avoids the trap completely. That's one reason people design code so its dependencies can be passed in.

Which double should you use?

• Answers only (stub): most of the time. Use create_autospec so calls are checked against the real signature, and return real objects, not mocks.
• Behaviour across calls (fake): retries, idempotency, "add to cart, then check out". Write it once and share it across tests.
• "Was it called?" (mock or spy): only when the call is the result, such as "an email was sent" or "the card was charged once". Prefer checking the outcome when you can.
• Nothing at all: your own code, simple data classes, and pure functions. Don't replace what you're testing.

How to keep doubles honest

A double is a copy of your beliefs about the real thing. Keep it close to the real thing:

1. Return real objects. A Payment, not a MagicMock, so missing fields fail like they would in production.
2. Use autospec. create_autospec, or patch(..., autospec=True), so a changed signature breaks the test, not production.
3. Test the fake against the real thing. Run a small set of the same tests against both the fake and the provider's sandbox, for example nightly. If the sandbox starts answering differently, the fake is out of date. This is the idea behind contract testing.
4. Keep some integration tests. A test suite where everything external is replaced proves the parts work as you imagine. A few tests against the real database or sandbox prove they work as they are. Part 4 found three bugs that no mock could have.

Common mistakes

• MagicMock everywhere. It accepts every call and every field, so it can't fail for the reasons production does. Step 2 passed with two bugs.
• Mocks that return mocks. gateway.charge.return_value.ok = False writes your assumption into the test. Return the real type.
• Checking calls instead of results. assert_called_once_with(...) for every step ties the test to how the code works, not what it achieves. Refactor the code and every test breaks, even when nothing is wrong.
• Patching in the wrong place. Patch where the name is looked up, as Step 7 shows.
• Faking your own code. If the test replaces the thing it's meant to test, it tests the double.

What's next

In Part 6 of Testing in Practice we test a mobile app: an Android emulator, Appium and pytest, on the FoodNow app.

ProvaLab's pytest plugin sends every result from tests like these to the ProvaLab test management system, with the history of each test across runs. So when the tests that were green with the bug go red with the fix, as in Step 6, you see that change in the test's history next to the code change that caused it. See it in the two-minute product tour.