How to Assess Software Quality Maturity

To assess software quality maturity, first decide whether you are evaluating the quality of a software product or the maturity of the processes used to build and test it. Define the decision and scope, select a model that fits that subject, gather evidence about work as performed, compare it with the model’s criteria, and turn the gaps into a small, measurable improvement plan.

Product quality and process maturity are related, but they are different assessment questions. A product can be evaluated against quality characteristics without establishing whether its development and testing processes are repeatable. Likewise, a process maturity rating does not by itself prove that a particular product meets its users’ quality needs.

1. Define what the assessment must decide

Write down the decision before selecting a framework. Common purposes include improving internal practices, deciding whether a process is suitable for a requirement, or evaluating a supplier’s processes against a contract. ISO/IEC TS 33010:2023 provides guidance on selecting assessment models, documented processes, and instruments for process assessments. ISO/IEC TS 33010:2023

Set a scope that is narrow enough to assess consistently. Record:

Products and services: which applications, components, or systems are included.
Teams and locations: which development, QA, operations, or supplier groups are in scope.
Lifecycle stages: for example, requirements, design, implementation, verification, release, and maintenance.
Practices: the particular quality or testing processes the assessment will examine.
Assessment boundary: what is explicitly outside the scope, and why.
Intended users of the result: who will make decisions from the findings and what decisions they need to make.

Do not combine teams with materially different work into a single rating unless the selected method supports that aggregation. A company-wide label can hide local strengths and weaknesses.

2. Choose the right assessment subject and model

Choose the reference model based on the question, rather than treating every quality framework as interchangeable.

Question	Suitable reference	What it helps assess
What quality characteristics does this product have, and how can they be specified or evaluated?	ISO/IEC 25010:2023	A product-quality model for ICT and software products.
How capable or mature are our organizational processes?	A process reference, assessment, or maturity model meeting the assessment need	Process expectations, assessment models, and maturity-model requirements.
How mature are our software testing processes?	TMMi is a testing-focused candidate	Testing process areas and their improvement, assessed against the model’s goals and practices.

IEC describes ISO/IEC 25010:2023 as a product-quality model applicable to ICT and software products. ISO/IEC 33004:2015, in contrast, sets requirements for process reference models, process assessment models, and maturity models. A product-quality checklist alone therefore cannot establish how mature an organization’s processes are. IEC: ISO/IEC 25010:2023 · ISO: ISO/IEC 33004:2015

There is no universally best model in the sources reviewed. Compare candidates against the purpose of the assessment, the processes they cover, their criteria and instruments, the evidence they expect, and how their results can be used. Confirm any formal-assessment or assessor requirements before committing to a method.

When TMMi may fit

The TMMi Foundation describes TMMi as a staged model for improving software testing processes, with progression from ad hoc and unmanaged practices through managed, defined, and measured stages to optimization. The Foundation also says it can complement CMMI, with more detailed support for software and system testing. These are the Foundation’s descriptions of its model, not an independent comparative evaluation. TMMi Model · TMMi aims and objectives

The Foundation says the TMMi model is freely available, while its TMMi Assessment Method is licensed for accredited assessors and lead assessors. Check the current method and qualification requirements if you need a formal assessment. TMMi Foundation · TMMi Assessment Method

3. Plan the assessment and identify evidence

Translate the chosen model into an assessment plan before interviewing teams or collecting records. Identify the criteria to examine, the people and artifacts that can provide evidence, how findings will be recorded, and who will review judgments. For a formal assessment, follow the selected method’s rules and use qualified assessors where required.

Look at work as it actually happens, not only at process documents. A practical evidence set may include:

Process descriptions, working agreements, and role definitions.
Requirements, risk records, review records, and traceability where relevant.
Test plans, test cases, automation code, execution records, and results.
Defect and incident records, including how issues are prioritized and resolved.
Release decisions, quality gates, and records of exceptions or waivers.
Measures used by teams and examples of decisions changed by those measures.
Interviews or observations from people doing the work in different roles and teams.

This is a practical evidence-gathering approach, not a universal checklist mandated by every model. Match evidence to the chosen criteria. A written procedure shows that a process is documented; records and observation help establish whether teams use it, adapt it, and learn from its results.

4. Compare evidence with the model

For each criterion, record the expectation, the evidence reviewed, the judgment, and any limitations. One simple internal recording scale is:

Finding	Meaning	Example record
Demonstrated	Evidence supports that the practice is in place for the assessed scope.	Practice, teams covered, records sampled, and date.
Partly demonstrated	Evidence shows inconsistent, incomplete, or locally limited use.	Where it works, where it does not, and what is missing.
Unsupported	The assessment found no sufficient evidence for the expectation.	Sources checked and remaining evidence gap.

This scale is a lightweight way to organize internal findings; it is not a substitute for the rating rules in a formal assessment method. Preserve evidence for every conclusion and distinguish “not done” from “not verified.” If evidence conflicts—for example, a procedure describes a review but teams report skipping it—record the discrepancy and investigate its scope rather than averaging it away.

For TMMi, the Foundation describes assessment as comparing process-area goals and practices with current practice. Its method aims to identify testing strengths, weaknesses, risks, and improvement opportunities. TMMi Assessment Method

5. Interpret measures without reducing maturity to a score

Use measures that answer the assessment’s business question and fit the model. Process performance and process-quality indicators can help show whether practices are producing useful results; product-quality measures help assess the resulting product. Maturity progression can include developing measurement and optimization, but no single metric set works for every organization.

Do not infer maturity from the count of automated tests, test cases, or reported defects alone. For each measure, state its scope, definition, time period, data limitations, and the decision it informs. A rising defect count, for example, could reflect changing product risk, better detection, or reporting changes; it needs context before it can support a conclusion.

The TMMi Foundation reports that 88% of TMMi users in its survey reported benefits in effectiveness or product quality, and 77% reported benefits in test efficiency. The source page does not establish the survey’s sample size, date, or independent verification. Treat these as Foundation-reported survey results, not proof that adoption causes those outcomes. TMMi Foundation: Model Aims and Objectives

6. Turn findings into an improvement plan

Choose a small number of gaps linked to business needs. For each improvement, document:

The gap: what the model expects and what the evidence showed.
The risk or need: why this gap matters to users, delivery, compliance, or operations.
The change: a specific practice or capability to improve.
An owner and timeframe: who will act and when the next review will happen.
A measure of progress: evidence that can show whether the change is working.
A reassessment point: when to gather new evidence and check the result.

Avoid turning every observation into a separate initiative. Prioritize by impact, risk, dependencies, and the organization’s capacity to make changes. ISO/IEC TS 33010 frames assessment as useful for improvement, and the TMMi Foundation says its assessment produces an organization-specific improvement plan. ISO/IEC TS 33010:2023 · TMMi Assessment Method

7. Common mistakes and how to avoid them

Mistake	Why it weakens the result	Correction
Confusing product quality with process maturity	A product checklist does not establish how consistently teams work.	State the assessment subject and choose a model designed for it.
Choosing a framework before defining the decision	The model may not cover the processes or outcomes the organization needs to assess.	Write the purpose and scope first, then compare models and methods.
Scoring documents instead of actual practice	Policies can exist without being followed or useful in daily work.	Triangulate documents with records, interviews, and observation.
Using one team’s evidence to represent every team	Different products and teams may use processes differently.	Sample across the defined scope and report where evidence came from.
Treating one metric as a maturity rating	A number without meaning, scope, and context can mislead.	Use multiple evidence sources tied to model criteria and business decisions.
Making a score the goal	A rating alone does not identify the change that will improve outcomes.	Translate findings into owned actions and reassess with evidence.

8. Troubleshooting assessment problems

Teams give conflicting accounts of the process

Cause: practices vary by team, or the documented process differs from daily work. Fix: record the variation, sample relevant teams, check recent work records, and clarify whether the scope permits local tailoring.

There is too little evidence to rate a practice

Cause: the assessment plan did not identify evidence sources, or the practice is not recorded. Fix: mark the finding as unverified or unsupported according to the method, seek additional evidence, and avoid treating absence of records as proof that the practice never occurs.

The chosen model does not fit the organization

Cause: its process areas, lifecycle assumptions, or assessment instruments do not address the original decision. Fix: revisit the purpose and scope, compare suitable models, and document any limits before continuing. ISO/IEC TS 33010 specifically addresses selection of assessment models and instruments.

Stakeholders disagree with a rating

Cause: criteria are ambiguous, evidence is incomplete, or expectations were not explained. Fix: review the model’s rating rules, present the evidence and scope, invite factual corrections, and retain a clear record of the decision. In formal assessments, follow the method’s governance.

The results produce too many improvement actions

Cause: findings have not been prioritized against business need and capacity. Fix: group related gaps, identify high-impact risks and dependencies, assign owners, and select a manageable first set of actions.

Performance, reliability, and cost considerations

An assessment’s reliability depends on a clear scope, fit-for-purpose criteria, evidence from more than one source where appropriate, consistent rating rules, and transparent records. Keep a sampling log so a later review can see which teams, projects, and records informed each judgment. If the assessment supports a supplier decision or formal recognition, follow the applicable contract or method requirements and verify assessor qualifications.

Assessment effort grows with the number of teams, products, lifecycle stages, and criteria included, as well as the rigor of evidence collection. Start with the decision-critical scope. Estimate interviewer and assessor time, evidence preparation, and follow-up capacity before scheduling. Do not assume that a higher maturity rating guarantees a particular defect reduction, delivery speed, or return on investment; the reviewed sources do not establish such guarantees.

Or skip the browser setup

If part of your assessment involves collecting website evidence or screenshots, ScreenshotNeo is a website screenshot API and MCP server from Yorker Media. One GET request returns an image or PDF, and the API can also capture an element, wait for page conditions, and use custom headers or cookies. See the ScreenshotNeo API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://example.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://example.com'
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
const image = Buffer.from(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', image));

Cookie banners, newsletter popups, and chat widgets are removed before the shot. Bot checks, blank pages, failed loads, timeouts, and cache hits cost nothing; response headers say which page verdict and billing status applied. An MCP server lets AI agents using Claude, Cursor, or another MCP client take screenshots. The free plan includes 1,000 shots per month with no card; paid plans start at $5 for 3,000 shots. Sign up for 1,000 free screenshots a month, with no card.

Frequently asked questions

How often should we reassess?

Reassess after the planned improvements have had time to affect the work, or when a material change in products, teams, requirements, or risk makes the earlier evidence stale. Set the review point in the improvement plan.

Can one assessment cover several teams?

Yes, if the scope, sampling, and chosen method support it. Keep findings traceable to teams and projects so organization-wide summaries do not hide important local differences.

Does a high maturity level guarantee better software?

No. Maturity describes practices against a model’s criteria. Product quality still needs to be evaluated for the product and its intended use.

Should a small team use a formal assessment?

That depends on the decision. A team seeking internal improvement may use a focused, evidence-based review; contractual or formal-recognition needs may require a specific method and qualified assessors.