DevOps Pipeline: Stages, Tools, and Best Practices

A DevOps pipeline is an automated, repeatable route that takes source code or a prebuilt artifact through validation and into a test or production environment. A useful pipeline connects each deployed change to its source, build inputs, artifact, deployment, and production signals. Its exact stages depend on the application, team ownership, compliance needs, and release risk.

Google Cloud describes a broad lifecycle of development inner loop, continuous integration, and continuous delivery. In practice, teams often split those phases into explicit steps: source review, validation, build, security checks, artifact publication, deployment, and production observation. CI validates changes; CD promotes tested artifacts and provides rollout, monitoring, and rollback controls. Google Cloud’s lifecycle overview is one vendor’s model, not a universal stage definition.

What is a DevOps pipeline?

A pipeline is the automated delivery path from a change or existing artifact to a running environment. It can include human decisions, such as pull request review or production approval, but routine work should be repeatable and recorded. The pipeline is also part of the production system: its definitions, runners, repositories, dependencies, artifacts, and credentials need protection.

The terms CI and CD describe related responsibilities. Continuous integration (CI) integrates and validates changes through builds, tests, and security checks. Continuous delivery (CD) keeps validated software ready to release and promotes it through environments. Some teams use continuous deployment to mean that qualifying changes are automatically released to production; teams should state which meaning they use.

What are the stages of a CI/CD pipeline?

Develop, commit, and review. Developers change application code or infrastructure definitions in version control. A commit or pull request starts automated checks. Peer review and protected branches can add a useful control point; the right review policy depends on the team’s risk and workflow.
Validate and build. The CI system retrieves source and dependencies, runs checks such as formatting, static analysis, and unit tests, then builds the application. For infrastructure as code, validate configuration, evaluate policy, and produce a reviewable plan before applying changes. Keep validation separate from deployment so an invalid plan cannot progress to resource changes.
Secure and package. Run security checks early enough to catch issues before release. Scan dependencies, source, and produced artifacts as appropriate to the system. Record the source revision and build inputs associated with the output so the artifact can be traced and verified.
Publish and promote an artifact. Publish a versioned package or image to an artifact repository. Promote the same tested artifact through environments where possible, rather than rebuilding different bytes for each environment. Google Cloud’s example builds a container image in CI, stores it in Artifact Registry, and uses a separate delivery pipeline to deploy it to GKE; the provider-specific services illustrate a general separation of responsibilities.
Deploy progressively and observe. Deploy to an environment with suitable risk controls, verify behavior, then promote or expand the rollout. Use production approval where governance and risk call for it. Monitor the release and retain a practical rollback path.
Operate and improve. Use metrics, logs, traces, alerts, incident learning, and user feedback to guide subsequent changes. Observability, test automation, version control, database change management, and deployment automation are ongoing capabilities, not merely boxes to check in a linear stage list.

Google Cloud’s secure-pipeline guide defines the concept this way: “A deployment pipeline is an automated process that takes code or prebuilt artifacts and deploys them to a test environment or a production environment.” See Design secure deployment pipelines for its security guidance.

Which tools are used in a DevOps pipeline?

Select tools by the job they perform and the boundaries they need to respect. A team does not need a separate product for every row, and adding tools does not by itself improve delivery.

Pipeline job	Typical tool category	Questions to ask
Source and review	Git repository and pull request workflow	Does it support review, branch protection, audit history, and ownership?
Build and orchestration	CI/CD system	Does a central controller push deployments, or do agents near resources pull and deploy? Can the team operate and recover it?
Validation and policy	Unit and integration test frameworks, static analysis, security scanners, policy-as-code	Which checks catch meaningful failures while keeping feedback useful?
Infrastructure	Infrastructure-as-code tools such as Terraform	Can proposed changes be planned, reviewed, and policy-checked before apply?
Artifact management	Package or container registry	Are artifacts versioned, retained, access-controlled, and traceable to builds?
Deployment and runtime	Deployment automation and target platform	What environment boundary, rollout strategy, and rollback mechanism fit this workload?
Operations	Monitoring, logging, tracing, and alerting	Can the team detect a failed release and understand its impact?

Google Cloud’s secure-pipeline guidance names Jenkins and GitLab as examples of central CI/CD systems. They are examples, not a ranking. In a push model, a central pipeline initiates deployments. In a pull model, a resource-local agent retrieves approved artifacts and deploys them. Compare access boundaries, target topology, management overhead, ownership, and recovery requirements; the available guidance does not establish one model as universally better.

For larger organizations, Google Cloud’s foundation blueprint separates foundation, infrastructure, and application pipelines, each with scoped responsibilities and identities. That structure can suit distinct platform and workload owners; smaller teams may not need the extra separation. See Google Cloud security foundations as a provider-specific example.

DevOps pipeline best practices

Make the path repeatable. Keep build and test steps in version control where practical, pin or otherwise control important dependencies, and make inputs and outputs identifiable.
Preserve traceability. Link a release to its source revision, build inputs, artifact, deployment, and relevant observations. Promote verified artifacts across environments.
Keep permissions narrow. Give each pipeline stage only the access it needs, scoped to the specific resources and environment. Separate identities or pipelines when doing so reduces the blast radius.
Protect the whole delivery chain. Review permissions for pipeline definitions, CI infrastructure, repositories, dependencies, artifacts, and credentials—not only cloud runtime roles. Protect secrets and avoid exposing long-lived credentials to untrusted changes.
Put integrity checks before deployment. Use static analysis, security scanning, and policy-as-code where they reduce meaningful risk. Keep changes reviewable and address failed checks rather than silently bypassing them.
Use rollout and rollback controls that match the service. Choose staged rollout, health checks, approval points, and rollback behavior based on impact and recovery needs. Verify that rollback is possible for application and data changes; database changes may require a forward fix or a separate recovery plan.
Make production observable. Define signals and alerts that can reveal a bad release, and ensure responders can connect them to a deployment. Feed incidents and operational findings back into the pipeline.
Plan for pipeline failure too. Map dependencies such as source control, build workers, artifact storage, identity systems, and deployment targets. Set recovery objectives based on business criticality and rehearse restoration.
Measure outcomes, not stage count. Use delivery and reliability signals to find bottlenecks and failure modes. Do not treat a particular number of tools, stages, or a generic benchmark as a universal target.

Google Cloud has documented pipeline attack techniques including cache poisoning in GitHub Actions, OIDC token extraction, and subversion of mutable action tags. These are examples from its security article, not an exhaustive list or proof that every pipeline has the same exposure. Apply defense in depth across the software delivery lifecycle. See Google Cloud’s software supply chain security discussion.

How to build a practical pipeline

Map the release path. Identify source, build dependencies, artifact storage, deployment targets, runtime signals, owners, and any required approvals.
Start with fast feedback. Run inexpensive validation and unit checks on each proposed change. Add slower integration or end-to-end checks at a suitable point so failures are actionable.
Build once and record what happened. Produce a versioned artifact and capture the source revision and relevant inputs. Keep the artifact immutable or otherwise protected from unnoticed replacement.
Verify before promotion. Apply security and policy controls, then deploy the same artifact to a test environment. Check both service health and relevant user-facing behavior.
Roll out with recovery in mind. Promote gradually when the platform and workload support it. Define what signals stop a rollout and how to roll back or mitigate.
Review the system after releases. Use failed builds, deployment incidents, and operational feedback to improve checks, ownership, access boundaries, and recovery procedures.

For infrastructure changes, separate plan and apply. Validate and policy-check the proposed change, make the plan reviewable, and ensure the identity that applies changes has only the necessary scope. Approval requirements are a governance choice, but a plan should not reach apply when its validation has failed.

Security, reliability, and cost considerations

Security

A pipeline can change production, so treat it as privileged infrastructure. Restrict who can change pipeline definitions, isolate untrusted contributions from sensitive credentials, control dependency and action sources, protect artifacts from unauthorized replacement, and scope identities by stage and environment. Google Cloud’s foundation blueprint illustrates separate least-privilege service accounts for stages and optional manual approval; adapt that pattern to the systems and responsibilities in use.

Reliability

Pipeline reliability includes the ability to deploy safely and the ability to restore the delivery system. Document dependencies and owners, set recovery time and recovery point objectives according to the pipeline’s business importance, and exercise recovery plans. Keep the rollback or mitigation procedure available during a release, and make sure monitoring can identify a failed rollout.

Performance

Reduce feedback time by running independent checks concurrently when resource limits allow, caching dependencies safely, and placing quick checks early. Keep cache keys tied to relevant inputs and treat shared caches as security-sensitive: Google Cloud’s security guidance specifically discusses cache poisoning as an attack technique. Avoid optimizing away checks that provide meaningful protection. Measure queue time and stage duration to identify the actual bottleneck.

Cost

Pipeline cost comes from compute, storage, network transfer, retained artifacts and logs, and engineering time spent operating tools. Use concurrency and retention policies that fit delivery needs, and avoid rebuilding artifacts unnecessarily. Self-managed systems add maintenance and recovery work; hosted services shift some operations to a provider but still require secure configuration and ownership. Compare total operating burden and recovery requirements rather than the sticker price alone. The source material does not establish a universal cost or speed ranking.

Common pipeline problems and fixes

Symptom	Likely cause	What to check or change
Build works locally but fails in CI	Different runtime, dependency resolution, environment variables, or undeclared local state	Pin expected runtimes and dependencies, declare required configuration, and reproduce the CI command in a clean environment.
Tests pass but deployment fails	Deployment identity lacks required scope, target configuration differs, or artifact is missing	Check the deployment stage’s identity and target settings; verify artifact name, version, permissions, and availability.
Infrastructure plan is unexpected	Configuration drift, changed provider inputs, or state mismatch	Review the plan and state, validate inputs and policy, and resolve drift deliberately before apply.
Pipeline has access to too much	Shared credentials or broad permissions used across stages	Map which stage needs each permission, scope identities to resources and environments, and split stages or pipelines where that reduces exposure.
Deployment succeeds but users see errors	Health checks do not cover the affected behavior, or rollout proceeds too quickly for detection	Add relevant service and user-facing signals, halt promotion on unhealthy behavior, and verify rollback or mitigation steps.
Pipeline outage blocks releases	Undocumented dependency or untested recovery plan	Map toolchain dependencies, define recovery objectives, and rehearse restoring CI, artifact access, and deployment capability.
Builds are slow or queues grow	Runner capacity, serialized checks, repeated dependency work, or oversized test scope	Measure queue and stage times, add safe parallelism or caching, and move fast, high-value checks earlier.

Or skip the browser setup

If a pipeline needs a clean screenshot of a deployed page for a release record, visual check, or downstream workflow, ScreenshotNeo provides a website screenshot API. One GET request returns an image or PDF. The same API can also support screenshot capture as part of a delivery workflow.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

See the ScreenshotNeo API documentation for options and setup. ScreenshotNeo accepts cookie and consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. Its MCP server gives AI agents tools for screenshots, page information, and PDF capture. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Learn about ScreenshotNeo or sign up for 1,000 free screenshots a month, with no card.

Frequently asked questions

Is every pipeline supposed to deploy automatically to production?

No. Teams can automate validation and delivery while keeping a production approval step when risk, compliance, or ownership requires it. Choose controls based on the service and organization.

Should CI and deployment be one pipeline?

They can be combined for a simple workload or separated to create clearer artifact, permission, ownership, and recovery boundaries. The useful design is the one the team can secure and operate.

Do all teams need a separate foundation pipeline?

No. Separate foundation, infrastructure, and application pipelines can help when ownership and permissions differ across those layers. A small team may reasonably keep fewer pipeline units.

Is a pipeline complete once deployment succeeds?

No. Production observation, incident response, rollback, and feedback are part of reliable delivery. A successful deployment command does not prove that the change behaves correctly for users.