ScreenshotNeo

BlogHow-to

How to Run a Method After Every pytest Assertion Failure

Use pytest_runtest_makereport to run code after failed test calls, capture diagnostics, handle setup errors, and troubleshoot hook behavior.

By the ScreenshotNeo team30 September 20269 min read

How to Run a Method After Every pytest Assertion Failure

To run a method after a pytest assertion failure, implement pytest_runtest_makereport in conftest.py (or in a pytest plugin). The hook receives a test report after each execution phase. Run your method when report.failed is true, and add report.when == "call" when you only want failures from the test body.

This callback runs after the assertion exception has unwound from the test call. It does not run between individual assertions and it cannot make pytest continue executing statements after a failed assertion. If one test contains several ordinary assertions, the first uncaught failure ends that call; pytest then creates the report and invokes the hook.

Minimal implementation

Create a conftest.py file in the test directory (or a parent directory visible to the tests):

import pytest


def run_after_failure(item, report):
    print(f"Failure in {item.nodeid}")
    print(report.longrepr)


@pytest.hookimpl(wrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    report = yield
    if report.when == "call" and report.failed:
        run_after_failure(item, report)

The wrapper lets pytest finish producing the report before your code inspects it. Current pytest documentation shows wrapper=True; older pytest and pluggy examples use hookwrapper=True. Check the versions installed in your project before choosing syntax. The pytest API reference documents the hook and its report phases.

Understand the three pytest phases

pytest normally creates separate reports for:

pytest creates reports for setup, call, and teardown; filter the phase before running diagnostics.
pytest creates reports for setup, call, and teardown; filter the phase before running diagnostics.
Phase When it runs Typical failure Use this filter
setup Before the test function Fixture creation, parametrization setup report.when == "setup"
call The test function body Assertion or other exception report.when == "call"
teardown After the test function Fixture cleanup report.when == "teardown"

Use report.failed without a phase check if your method must run for setup, call, and teardown failures. This can produce more than one diagnostic action for a single test item, so make the behavior explicit.

Run after failures in the test body only

For screenshots, logs, or notifications that describe the state asserted by the test, call-only behavior is usually the clearest:

from pathlib import Path
import re


def safe_name(nodeid):
    return re.sub(r"[^A-Za-z0-9_.-]+", "_", nodeid)


def save_failure_details(item, report):
    output = Path("test-artifacts")
    output.mkdir(exist_ok=True)
    path = output / f"{safe_name(item.nodeid)}.txt"
    path.write_text(
        f"nodeid: {item.nodeid}\n"
        f"when: {report.when}\n"
        f"outcome: {report.outcome}\n\n"
        f"{report.longrepr}\n",
        encoding="utf-8",
    )


@pytest.hookimpl(wrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    report = yield
    if report.when == "call" and report.failed:
        save_failure_details(item, report)

item.nodeid includes the file, test name, and (when relevant) parameter values. Sanitizing it avoids invalid path characters when parametrized tests contain punctuation.

Include setup and teardown failures

Fixtures can fail before the test body starts, and cleanup can fail after it ends. To run your method for every failed phase:

@pytest.hookimpl(wrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    report = yield
    if report.failed:
        run_after_failure(item, report)

If you need different handling per phase, branch deliberately:

@pytest.hookimpl(wrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    report = yield
    if not report.failed:
        return

    if report.when == "setup":
        handle_setup_failure(item, report)
    elif report.when == "call":
        handle_test_failure(item, report)
    elif report.when == "teardown":
        handle_teardown_failure(item, report)

Make the hook safe and useful

Do not hide the original failure

A diagnostic method should not replace the assertion traceback. Catch errors inside the method and report them separately:

import logging

log = logging.getLogger(__name__)


def run_after_failure(item, report):
    try:
        collect_logs(item)
    except Exception:
        log.exception("Failure hook could not collect diagnostics for %s", item.nodeid)

Whether to re-raise depends on your goal. For most diagnostics, logging the secondary error preserves the original test result and still tells you the collector failed. Avoid calling pytest.fail() from this hook: the test has already failed, and a new failure can obscure the useful traceback.

Capture output and exception details

report.longrepr contains pytest’s formatted failure representation. Depending on plugins and pytest versions, reports can also expose sections such as captured stdout and stderr. Treat these as optional attributes:

def report_text(report):
    chunks = [str(report.longrepr)]
    for name, content in getattr(report, "sections", []):
        chunks.append(f"[{name}]\n{content}")
    return "\n\n".join(chunks)

Prevent duplicate work

A hook can be called for setup, call, and teardown, and other plugins can add related behavior. Use the phase filter, a per-test marker, or an idempotent artifact name when the action must happen once:

def run_once(item, report):
    if getattr(item, "_failure_action_done", False):
        return
    item._failure_action_done = True
    collect_logs(item)

For parallel execution with pytest-xdist, write worker-specific files or use a unique name containing the worker id. Multiple workers may process failures at the same time.

Use a reusable pytest plugin

Keep the hook in conftest.py when it belongs to one repository. Package it as a plugin when several projects need the same behavior. A plugin module can expose the hook directly:

# failure_artifacts.py
import pytest


def pytest_addoption(parser):
    group = parser.getgroup("failure artifacts")
    group.addoption(
        "--failure-artifacts-dir",
        action="store",
        default="test-artifacts",
        help="Directory for failure diagnostics",
    )


@pytest.hookimpl(wrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    report = yield
    if report.when == "call" and report.failed:
        # Read configuration from item.config.getoption(...)
        save_report(item, report)

Register the plugin through normal pytest plugin packaging or load it for a run with -p failure_artifacts. A local conftest.py is discovered only for tests in that directory subtree and its parents. If a hook appears not to run, first check that the file is in the discovery path.

  • pytest_runtest_makereport: react to setup, call, or teardown outcomes. This is the general failure callback.
  • pytest_assertion_pass: called whenever an assertion passes. It requires enable_assertion_pass_hook = true and is not a failure hook.
  • pytest_assertrepr_compare: customize the explanation shown for comparison assertion failures. It formats failure output; it does not run a general post-failure method.

See the pytest hook specification for signatures and the documented lifecycle.

Assertions, exceptions, and multiple checks

The hook is triggered by a failed test report, not by every assertion expression. This distinction matters:

A screenshot service can clean common overlays before storing a failure artifact.
A screenshot service can clean common overlays before storing a failure artifact.
def test_values():
    assert 1 == 2       # test call stops here
    assert 3 == 4       # never reached

To evaluate several conditions and report all of them, use a plugin such as an assertion aggregation tool or collect failures yourself, then raise once. The report hook still runs once for the resulting failed call.

A non-assertion exception in the test body also produces report.failed for the call phase. If your method should react only to assertion failures, inspect the exception information available on the report, but keep that code defensive because report representations can vary by pytest version and plugin.

Or skip the browser setup

If your failure method saves a browser screenshot, you can call a screenshot API after the report is created instead of maintaining browser startup, navigation, cookie handling, and cleanup. ScreenshotNeo accepts one GET request and returns PNG, JPEG, WebP, or PDF. Its capture can load lazy images, select one element, apply custom CSS or JavaScript, wait for a selector, delay, or network idle, and use device, viewport, dark mode, geolocation, headers, cookies, or an authorization header.

In a failure hook, keep the request bounded and write the response only when it is a successful capture:

import os
from pathlib import Path
import requests


def capture_failure_page(item, report):
    url = os.environ["FAILED_PAGE_URL"]
    response = requests.get(
        "https://api.screenshotneo.com/v1/shot",
        params={
            "access_key": os.environ["SCREENSHOTNEO_API_KEY"],
            "url": url,
        },
        timeout=90,
    )
    response.raise_for_status()
    Path("test-artifacts/failure.webp").write_bytes(response.content)

See the ScreenshotNeo API documentation for the complete option list and response headers. The service removes cookie consent banners, newsletter popups, and chat widgets before capture. Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed, and the response identifies the result with X-Page-Verdict and X-Billed headers. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' }); const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

There are 1,000 free screenshots each month with no card. Paid plans start at $5 for 3,000 shots. Create a free ScreenshotNeo account.

Troubleshooting

The hook never runs

  • Confirm the file is named conftest.py and is above the test directory in the tree.
  • Run pytest --trace-config to inspect loaded plugins and configuration.
  • Check that the function name and parameters are exactly pytest_runtest_makereport(item, call).
  • If using a plugin, verify it is installed or loaded with -p.

It runs for setup or teardown unexpectedly

Add report.when == "call". If you need all phases, keep the broader condition and branch on report.when.

wrapper=True causes an error

Your pytest or pluggy version may use the older wrapper spelling. Consult the versioned documentation and try @pytest.hookimpl(hookwrapper=True, tryfirst=True).

The diagnostic method masks the assertion

Catch exceptions raised by the diagnostic method and log them. Do not replace the original report or raise a second pytest failure from the hook.

Artifacts are missing in CI

Write to a known workspace directory, create it before writing, and configure your CI job to upload that directory even when tests fail. Use unique names for parametrized tests and parallel workers.

The screenshot request times out

Set a finite timeout, use an appropriate wait strategy, and handle the response as a separate diagnostic failure. With ScreenshotNeo, failed loads and timeouts are not billed; inspect X-Page-Verdict and X-Billed when deciding whether an artifact is usable.

Performance, reliability, and cost

  • Keep failure work conditional: screenshots and uploads should run only after a failed report, never on every passing test.
  • Use bounded time: a hook that waits indefinitely can make the test suite appear hung. Set network and subprocess timeouts.
  • Prefer asynchronous delivery for slow systems: enqueue a diagnostic job or use a service’s async webhook instead of blocking the test process.
  • Control volume: retries, parametrization, and teardown failures can create many artifacts. Include the node id and phase in filenames.
  • Cache stable pages: when an identical capture is acceptable, a chosen cache TTL can reduce repeated work. Treat cache hits according to the API’s billing and verdict headers.
  • Protect secrets: do not print API keys, authorization headers, or private cookies in failure logs.

FAQ

Can I run code after every individual assert?

Not with pytest_runtest_makereport. It runs after a test phase report, and an uncaught assertion stops the call. Use explicit helper functions or an assertion aggregation approach if you need per-check behavior.

Should I put this in conftest.py or a plugin?

Use conftest.py for one test tree. Use a packaged plugin when several projects share the hook and its configuration.

Does the hook run for skipped tests?

A skipped or expected-failure outcome is not the same as report.failed. Inspect report.outcome if your policy must include skips or xfails.

Can the method modify the test result?

It can influence pytest behavior if it raises or changes hook data, but diagnostic code should preserve the original result and avoid introducing a second failure.

Where can I customize comparison failure text?

Implement pytest_assertrepr_compare. That hook is for formatting comparison explanations, while pytest_runtest_makereport is for post-phase actions.