Smoke, Sanity, and Regression Testing: Differences Explained

Short answer: Smoke testing checks whether a build’s essential functions work well enough to begin planned testing. The ISTQB glossary source available for this guide gives sanity testing the same definition, so the terms can overlap. Regression testing has a different purpose: after a software or environment change, it checks whether previously tested areas have developed unintended defects.

Use the test names to communicate the question and decision your team needs answered. Agree on what “smoke” and “sanity” mean locally, because those labels are not used consistently. A regression check should focus on behavior that could have been affected by a change.

What each test is meant to establish

Test	Question	Typical trigger	Scope and decision
Smoke	Do the main functions work well enough to start planned testing?	A new build or candidate arrives for testing.	A broad check of essential functions. If it fails, deeper testing may be blocked.
Sanity	Do the main functions work properly before planned testing begins?	Usage varies by team.	The cited ISTQB glossary source gives it the same definition as smoke. Some teams use “sanity” for a focused check after a limited change; treat that as a local convention, not a universal rule.
Regression	Did a change introduce a defect in previously tested behavior that was not meant to change?	A software modification, fix, or environment change.	Selected previously tested behavior that may be affected; the goal is to detect unintended side effects.

These definitions follow the available ISTQB glossary and Foundation Level material, reproduced by ASTQB. ASTQB: ISTQB Foundation Level syllabus and glossary material.

Smoke and sanity testing can mean the same thing

In the cited ISTQB glossary reproduction, “sanity test” and “smoke test” are synonyms with the same main-functionality definition. That is why a rigid universal distinction—such as “smoke is broad, sanity is always narrow”—can mislead. Teams may use the labels differently, so document the intended scope and decision in the test plan or build handoff.

A clear test name helps more than an assumed taxonomy. For example, “checkout release smoke: sign-in, cart, payment entry” says what is checked and what a pass permits. If your team calls a targeted post-change check a “sanity test,” state that convention explicitly.

Regression testing is about unintended effects of change

Regression testing asks whether a modification caused a failure elsewhere in behavior that had already been tested. The change may be a bug fix, a feature update, or an environment change. Regression scope should be selected according to risk and the areas the change could affect.

Regression testing is not simply another name for checking whether the original defect was fixed. The ISTQB Foundation Level material distinguishes:

Confirmation testing: check that the specific fix resolved the reported problem.
Regression testing: check whether the fix caused failures in other areas.

Both can be needed after a fix: first verify the original behavior, then check relevant surrounding behavior for side effects. ASTQB’s reproduction of ISTQB Foundation Level material covers this distinction.

A practical example: a checkout change

Suppose a team receives a new checkout build. A smoke suite checks that a user can sign in, add an item to the cart, and reach payment. Its purpose is to decide whether the build is ready for more detailed testing.

On build arrival: run the agreed smoke checks for essential paths. If sign-in or checkout entry fails, report the build as not ready for planned testing.
After changing tax calculation: run a check that confirms the new calculation produces the expected result. Call it a “sanity test” only if that is your team’s defined label.
Then assess regression risk: check previously working checkout paths that could be affected, such as discounts, saved addresses, totals, and payment handoff.

The example separates the questions without claiming every team uses the same names. A test can contribute to more than one decision when its scope and result are clear.

How to choose and combine the tests

State the decision first. Is the build ready for detailed testing, did a specific fix work, or did the change leave other behavior intact?
Choose the scope that answers it. For build readiness, cover essential functions. For regression, select previously tested behavior at risk from the change.
Choose labels your team understands. Since smoke and sanity overlap in the cited glossary and usage varies, define them in team documentation.
Record the result and next action. A failed readiness check may block planned testing; a failed confirmation check returns the fix for investigation; a regression failure identifies an unintended effect to triage.
Revisit scope when risk changes. A small code change can still affect shared components or integrations. Base selection on the change and system risk, not on the label alone.

Do not define these tests by whether they are manual or automated, or by how long they take. Those may be implementation choices, but they do not establish the purpose of the test in the cited material.

Testing includes more than running test cases

The ISTQB Foundation Level material describes testing as broader than dynamic execution: it includes static review and analysis, as well as planning and evaluation. That matters when a team decides how to establish quality and manage risk. A review can reveal issues before a build is executed, while dynamic checks exercise the running software.

Testing and debugging are separate activities: a test can expose a failure, while debugging investigates and locates its cause. Keep the failure report, diagnosis, fix, confirmation check, and regression selection connected, but do not treat them as one activity. ASTQB’s reproduction of ISTQB syllabus material.

Scope, reliability, and cost considerations

Scope: A smoke check should cover enough essential behavior to make the readiness decision meaningful. Regression scope should follow change impact and risk; it is not automatically every test in the suite.
Reliability: State expected outcomes and prerequisites clearly. A failed test caused by unavailable test data or a broken environment does not by itself establish a product defect; investigate the failure context before making a release decision.
Cost: The source material does not establish that one test type is universally faster, cheaper, or more effective. The practical cost depends on the selected scope, test setup, and how failures are investigated.
Execution mode: Manual and automated checks can both serve these purposes. Choose based on repeatability, risk, and team workflow rather than treating execution mode as part of the definition.
Test evidence: Record build or environment, scope, outcome, and follow-up. This helps teams compare results without relying on ambiguous labels.

Common mistakes and how to fix them

Mistake	Why it causes trouble	Better approach
Assuming sanity always means a narrow, deep check	The available ISTQB glossary source gives sanity and smoke the same definition; team usage varies.	Write down your local definition and describe the actual scope in the test name.
Calling a fix check “regression” without checking other areas	That may only establish whether the original defect was fixed.	Separate confirmation of the fix from checks for side effects elsewhere.
Running a broad regression suite without change-risk selection	The suite may consume effort without clearly targeting behavior the change could affect.	Select previously tested areas based on dependencies and risk, and explain the selection.
Starting planned testing despite a failed essential-function check	Further results may be blocked or unreliable when core paths do not work.	Define the readiness gate and escalation path before the build arrives.
Using duration or automation as the definition	Neither duration nor manual-versus-automated execution distinguishes the purposes in the cited sources.	Describe the question, trigger, scope, and decision instead.

Using screenshots to document visual checks

Smoke, sanity, and regression are testing purposes, not screenshot techniques. For visual checks, a screenshot can preserve what a page looked like in a particular run, but it does not by itself establish that behavior is correct. Compare the captured page with an expected result and keep viewport, state, and relevant test data consistent.

For browser-based screenshot capture, you can configure a browser yourself or use ScreenshotNeo, a website screenshot API and MCP server. The API can return PNG, JPEG, WebP, or PDF captures, and its options include viewport and device presets, full-page capture, element capture, custom CSS, JavaScript, waiting conditions, and request controls. See the ScreenshotNeo API documentation for parameter details.

Or skip the browser setup

Make one GET request with the page URL to receive a screenshot. Replace the placeholder API key and target URL with your own values. The following uses the documented API endpoint and Python’s requests library:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Equivalent cURL and Node.js calls:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);

ScreenshotNeo accepts cookie or consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients. The free plan includes 1,000 shots per month with no card; paid plans start at $5 for 3,000. Every feature is available on every plan. Read the API documentation and ScreenshotNeo overview.

FAQ

Is a sanity test the same as a smoke test?

The cited ISTQB glossary reproduction treats the terms as synonyms and gives them the same definition. Teams may use them differently, so define the terms locally.

Does regression testing confirm that a bug fix worked?

Confirmation testing checks the specific fix. Regression testing checks for unintended effects in other previously tested behavior.

Does regression testing have to cover the entire application?

No. The purpose is to check previously tested behavior that may have been affected. Select scope based on the change and its risks.

Are these tests necessarily automated?

No. Automation is an implementation choice; it does not define the purpose of smoke, sanity, or regression testing.

Sources and further learning

ASTQB: ISTQB Foundation Level syllabus and glossary material — definitions, regression and confirmation testing, and the broader scope of testing.
ISTQB — syllabi, glossary, certification scheme, and further learning resources.

The smoke/sanity equivalence in this guide is attributed to the accessible glossary reproduction described in the research source set; the official glossary interface did not provide readable text during that research pass.