How to Handle Large-Scale Website Conversions

A large-scale website conversion is a controlled change to a site’s URLs, hosting, platform, design, or several of these at once. The safest plan starts by identifying whether visible URLs will change.

• URL-changing move: a domain, protocol, path, or site merger. You need an old-to-new URL map, relevant permanent redirects, updated canonicals, and new sitemaps.
• Hosting or CDN move: the visible URLs stay the same. The main work is preparing the new infrastructure, changing DNS, and verifying that users and crawlers receive the same content correctly.

A redesign or CMS change can accompany either path. When possible, sequence major changes so you can identify which change caused a problem. Google’s site-move guidance recommends testing a representative section first when a large site can support a pilot, while warning that one section may not expose every issue in the full migration.

1. Define the conversion before changing anything

Write a short migration brief that answers these questions:

1. Will the protocol, hostname, path structure, or URL parameters change?
2. Is this also a hosting, CDN, CMS, design, or content change?
3. Which teams own SEO, engineering, analytics, operations, and approvals?
4. What is the launch window, rollback condition, and incident contact?
5. What evidence will prove that the old environment can be retired?

Classify the project as one of the following:

Conversion type	Visible URL change?	Primary controls
HTTP to HTTPS, domain, path, or merged sites	Yes	URL map, relevant 301/308 redirects, canonicals, sitemaps, Search Console
Hosting or CDN replacement	No	Infrastructure parity, DNS cutover, logs, capacity, cache and TLS checks
Combined migration	Usually	Separate changes where practical; otherwise maintain a detailed change log and isolated checks

Do not assume a navigation export is a complete inventory. Important pages can exist only in server logs, analytics reports, external links, Search Console link data, feeds, APIs, or old campaigns.

2. Build a complete URL and asset inventory

Export candidate URLs from at least four sources:

• CMS and database listings, including archived and paginated content.
• Server access logs, with a time window that covers seasonal traffic.
• Analytics landing pages and referral reports.
• Search Console links, indexed pages, and submitted sitemap data.

Include more than HTML documents. Record image, video, JavaScript, CSS, font, feed, and downloadable-file URLs when they are externally linked or required for rendering. Mark each record with its status, traffic, backlinks, template type, language, and intended destination.

A useful inventory schema looks like this:

old_url,new_url,status,template,owner,notes
https://old.example.com/guides/a,https://www.example.com/guides/a,301,guide,seo,unchanged
https://old.example.com/products/x,https://www.example.com/products/x,301,product,product,slug changed
https://old.example.com/events/2022,,410,event,content,expired and no replacement

Map each old URL to the closest useful new destination. A redirect to the home page is not a safe default for unrelated pages: Google says that irrelevant mass redirects can confuse users and may be treated as soft 404s. If a page has no replacement, decide deliberately whether it should return a useful 404 or 410.

3. Prepare the destination and test it before launch

Build the new environment while the current site remains available. Test representative templates and high-value URLs, including pages with unusual parameters, localization, authentication boundaries, large media, and deep paths.

Checks for a URL-changing move

• Every mapped destination returns the expected status and content.
• Redirects are server-side and permanent where the move is permanent.
• Redirect chains and loops are absent.
• New canonical annotations point to new URLs.
• Internal links, hreflang annotations, structured data URLs, feeds, and downloadable links use the new locations.
• The new sitemap contains the new canonical URLs, not the old URLs.
• Staging-only noindex directives and robots.txt blocks are removed at launch.
• Search Console properties are verified for the relevant old and new sites.

Checks for a hosting or CDN move

• HTML, status codes, headers, TLS certificates, compression, caching, and redirects match the intended behavior.
• Origin and edge logs identify real users and Googlebot correctly.
• Capacity covers normal demand plus increased crawling after the switch.
• DNS records, IPv4/IPv6 behavior, firewall rules, WAF policies, and regional routing are ready.
• Cache invalidation and purge procedures are documented.
• The old environment can remain available while traffic shifts.

Google’s hosting-move guidance describes a temporary crawl-rate drop followed by a rise over the next few days as normal when Googlebot can still fetch content successfully.

4. Pilot a representative section when feasible

For a large site, move a bounded section before the full cutover. Choose a section that is stable, has meaningful traffic, and contains several templates without being dominated by unpredictable events. Measure its crawl errors, indexed URLs, redirects, rankings, traffic, and server load.

A pilot reduces blast radius, but it is not proof that every template works. Keep separate test cases for international pages, faceted navigation, media, JavaScript-rendered content, login flows, and high-volume APIs even if they are absent from the pilot.

5. Execute the launch in a controlled sequence

1. Freeze the migration inputs. Export the final URL map, sitemap, configuration, and list of launch checks.
2. Deploy the destination. Confirm that it is accessible to authorized testers and that staging blocks are still active until the launch step.
3. Enable redirects. For URL moves, turn on mappings at the edge or origin and test old URLs from multiple representative groups.
4. Update discovery signals. Publish the new sitemap, canonicals, internal links, feeds, and structured data URLs.
5. Remove temporary blocks. Delete launch-inappropriate noindex directives and robots.txt disallows.
6. Use Search Console. Submit the updated sitemap and use the domain-move workflow where applicable.
7. Change DNS for hosting moves. Keep the previous infrastructure online while public resolvers and caches update.
8. Record the exact cutover time. This timestamp makes log, analytics, and crawl comparisons possible.

Large sites may receive heavier crawling because requests to old URLs can be redirected while other URLs are crawled. Plan capacity with the hosting provider before launch.

6. Monitor the first hours, days, and weeks

Use several independent signals. A single analytics dashboard cannot show whether a crawler is receiving errors or whether redirects are wrong.

Signal	What to inspect	Response
Search Console	Submitted and indexed URLs, crawl errors, indexing status, manual actions	Inspect affected URL samples and correct the underlying rule
Analytics	Landing-page traffic, referrals, conversions, country and device splits	Compare old and new properties using the same time windows
Server and edge logs	Googlebot status codes, redirect targets, latency, origin errors, capacity	Trace failing requests and increase capacity or fix routing
URL tests	Old-to-new mapping, canonical, robots, sitemap and rendered content	Run samples continuously and after configuration changes

The normal direction after a URL move is old-site traffic falling while new-site traffic rises. Expect fluctuations while systems process the change. Google says medium-sized sites can take a few weeks or more for most pages to move in its index, and larger sites can take longer; there is no fixed crawl frequency. That is an operational timeframe, not a ranking guarantee.

Permanent redirects preserve PageRank signals according to Google, but that does not guarantee unchanged rankings or traffic during a major migration.

7. Troubleshooting common migration failures

Old URLs redirect to the wrong page

Cause: a broad rewrite rule, stale mapping, or path normalization error. Fix: test exact URL samples and compare the final destination with the mapping table. Add specific rules before generic rules.

Many URLs redirect to the home page

Cause: missing destination mappings or a catch-all fallback. Fix: map each valuable URL to a relevant equivalent. Return a deliberate 404 or 410 where no replacement exists.

Redirect loops or long chains appear

Cause: old and new environments both rewrite protocol, host, or trailing slash rules. Fix: make one layer authoritative, point directly to the final URL, and remove duplicate transformations.

The new site is not being indexed

Cause: staging noindex, robots.txt disallow, authentication, blocked resources, or a broken canonical. Fix: inspect the raw response and rendered page, remove launch-only blocks, and verify the sitemap and canonical.

Traffic falls but there are few 404s

Cause: redirects may resolve technically while sending users to irrelevant content, or analytics attribution may have broken. Fix: sample landing pages, compare destination relevance, check referral and campaign parameters, and inspect logs for status and latency changes.

Googlebot receives capacity errors

Cause: insufficient origin capacity, an aggressive WAF, rate limits, or a cache miss storm. Fix: review edge and origin logs, allow legitimate crawlers according to your security policy, scale capacity, and verify recovery with URL checks.

Hosting cutover serves mixed versions

Cause: DNS propagation, stale caches, regional routing, or an old origin still answering. Fix: query public DNS from several locations, inspect response headers, keep both environments compatible during the transition, and delay shutdown until logs show the old system is no longer serving users or crawlers.

8. Automate URL and rendering QA

Automated checks make it practical to test thousands of URLs before and after launch. A minimal shell check can record status and the final redirect target:

while IFS=, read -r old_url expected_url; do
  result=$(curl -L -sS -o /dev/null -w '%{http_code} %{url_effective}' "$old_url")
  printf '%s,%s,%s\n' "$old_url" "$expected_url" "$result"
done < redirect-samples.csv

For visual regressions, capture the same representative URLs from the old and new environments at a controlled viewport. Compare full pages and critical elements, and repeat after JavaScript, cookie, personalization, and responsive breakpoints have loaded. A crawler such as Screaming Frog is one option for migration crawling and redirect audits; Search Console and scripts remain useful regardless of crawler choice.

9. Reliability, performance, and cost planning

• Reliability: keep rollback instructions, DNS records, redirect rules, sitemap versions, and configuration snapshots in the change record.
• Performance: compare time to first byte, cache hit rate, asset latency, image behavior, and JavaScript errors before and after cutover.
• Crawling: budget for additional requests during a URL move and coordinate capacity with your host.
• Observability: retain enough logs to correlate a URL, user agent, status, redirect chain, and origin response.
• Cost: include temporary duplicate infrastructure, log storage, crawl tooling, image or PDF capture, and engineering time. Do not retire the old stack solely because DNS has changed.

Search processing is URL by URL. Avoid declaring success or failure from one day of aggregate traffic. Establish thresholds for severe HTTP errors, redirect mismatch rates, origin saturation, and high-value landing-page loss, then escalate when those thresholds are crossed.

10. Or skip the browser setup

When migration QA requires repeatable screenshots of old and new URLs, ScreenshotNeo provides a single screenshot API request. Its clean-shot process accepts consent banners and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed, and responses identify the result with X-Page-Verdict and X-Billed headers. It also has an MCP server for AI agents with take_screenshot, get_page_info, and capture_pdf tools.

See the full option list and request behavior in the ScreenshotNeo documentation. The same endpoint supports full-page or element capture, device presets, custom viewports, retina scale, dark mode, custom CSS and JavaScript, waits, request blocking, headers, cookies, user agents, authorization, timezone, geolocation, resizing, caching, signed links, asynchronous jobs, bulk capture of up to 100 URLs, PDF output, HTML/CSS rendering, and a usage API.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Free usage includes 1,000 screenshots per month with no card. Paid plans start at $5 for 3,000 screenshots, and every feature is available on every plan. Create a free ScreenshotNeo account to capture migration samples without setting up and maintaining a browser worker.

11. Migration checklist

• Classified the project as URL-changing, hosting-only, or combined.
• Exported URLs from the CMS, logs, analytics, and Search Console.
• Included important non-HTML assets.
• Built and tested relevant old-to-new mappings.
• Tested canonicals, robots.txt, noindex, sitemaps, and internal links.
• Tested representative templates and edge cases.
• Planned capacity for normal demand and increased crawling.
• Prepared Search Console properties and launch notifications.
• Recorded a rollback plan and exact cutover time.
• Monitored logs, Search Console, analytics, DNS, and URL samples after launch.
• Retired old infrastructure only after evidence showed it was no longer serving users or crawlers.

FAQ

How long does a large migration take to settle in Google?

Google says medium-sized sites can take a few weeks or more for most pages to move in its index, and larger sites can take longer. There is no fixed crawl schedule.

Should I combine a redesign with a domain move?

Separate major changes when feasible so you can diagnose effects. If they must ship together, maintain a detailed change log and checks that distinguish URL, rendering, infrastructure, and content problems.

Do redirects permanently preserve rankings?

Google says permanent redirects do not cause a loss in PageRank signals. Rankings and traffic can still fluctuate during processing, so monitor individual URLs and technical errors.

When can I shut down the old hosting environment?

For a hosting-only move, wait until DNS checks, access logs, and URL tests show that users and Googlebot are consistently served by the new environment. For a URL move, keep redirects available for as long as users and crawlers may still request old URLs.