What Is Agent Search Optimization and How to Track It

Agent Search Optimization (ASO) is the work of making a website discoverable, understandable, and usable by AI agents acting for a person. To track it, measure four things separately: whether relevant crawlers can reach your pages, whether your site appears accurately in answers to a repeatable set of questions, whether those answers send visits, and whether visits lead to qualified actions.

ASO is an emerging practitioner term, not a standardized discipline with a universal score. It overlaps with search engine optimization (SEO), answer engine optimization (AEO), and generative engine optimization (GEO). The useful distinction is that an agent may not stop at retrieving and summarizing information: it may evaluate pages or take a user-directed action. That makes access, accuracy, and task usefulness worth measuring alongside traditional search visibility.

1. What Agent Search Optimization means

Traditional SEO focuses on eligibility, crawling, indexing, and ranking in search results. AEO and GEO commonly describe efforts to make content useful for direct answers and generative search experiences. ASO extends the conversation to AI agents that retrieve information, compare choices, and may interact with websites on a person’s behalf. The terms overlap, and different practitioners use them differently; define ASO when reporting it rather than assuming one industry-wide definition.

For practical purposes, ask whether an agent can:

Discover the relevant page through search or a user-requested fetch.
Evaluate the page’s content and identify accurate, current information.
Use the information to answer a question or support a user-directed task.

A crawler request alone proves only that a fetch occurred. A brand mention alone does not prove the answer was accurate, that a user visited, or that the interaction created business value.

2. Track four measurement layers

Layer	Record	What it tells you	What it does not tell you
Access and technical reach	Verified crawler or agent requests, requested URL, timestamp, status code, blocked requests, and verified source IP	Whether documented crawlers or fetchers reached pages	Whether a page appeared in an answer or was useful
Answer visibility and quality	Fixed prompt, engine, date, brand and competitor mentions, cited URL, context, factual accuracy	Whether the site appears in sampled answers and how it is represented	Total impressions across all users or engines
Referral traffic	Source where available, landing page, sessions, engaged sessions	Visits that can be attributed to a recognizable referral source	Answer exposures that did not result in a click
Business outcomes	Qualified signup, lead, purchase, booking, or another defined conversion; assisted conversion where available	Whether observed traffic is associated with useful outcomes	Causality from a simple before-and-after comparison

Keep these layers separate in dashboards and reports. They describe different steps in a path, not interchangeable versions of one ASO score.

3. Build a repeatable answer-visibility test

Choose representative questions. Use the actual information needs of your audience: informational questions, comparisons, and task-oriented questions where appropriate. Keep wording fixed for each measurement round.
Name the engines and context. Record the product or engine, date, language, target geography, and any available mode or model label. Results can vary with these conditions.
Define inclusion rules. Decide what counts as a brand appearance, a cited URL, a relevant competitor mention, and an accurate answer before collecting results.
Run the same set on a regular cadence. Save the complete answer or a sufficient excerpt along with the structured fields. Note major site, prompt, or platform changes in a reporting log.
Review quality, not just presence. Check whether the cited page supports the claim, whether facts are current, and whether the answer describes the brand in the right context.

A simple CSV can make the process reproducible:

date,engine,language,geography,prompt,brand_appears,cited_url,accuracy,competitors,context,notes
2026-10-04,Example engine,en,US,"How do I compare options for X?",yes,https://example.com/guide,accurate,"Example competitor","Cited in comparison answer","Illustrative row; replace with observed data"

Use the same prompt set, geography, language, engines, cadence, and scoring rules when comparing periods. Store answers or evidence securely, and record the sample size. This is a practical measurement design, not an official cross-platform standard.

A minimal Python tracker for manual observations

This standard-library script appends one observation to a CSV file. It does not query answer engines or verify claims; enter observations you collected manually. Save it as track_aso.py and run it with Python 3:

import csv
import sys
from datetime import date
from pathlib import Path

FIELDS = [
    "date", "engine", "language", "geography", "prompt",
    "brand_appears", "cited_url", "accuracy", "competitors",
    "context", "notes",
]

if len(sys.argv) != 11:
    raise SystemExit(
        'Usage: python track_aso.py ENGINE LANGUAGE GEOGRAPHY PROMPT '
        'BRAND_APPEARS CITED_URL ACCURACY COMPETITORS CONTEXT NOTES'
    )

row = dict(zip(FIELDS[1:], sys.argv[1:]))
row["date"] = date.today().isoformat()
path = Path("aso-observations.csv")
with path.open("a", newline="", encoding="utf-8") as file:
    writer = csv.DictWriter(file, fieldnames=FIELDS)
    if file.tell() == 0:
        writer.writeheader()
    writer.writerow(row)
print(f"Saved observation to {path}")

Example invocation (quote prompts that contain spaces):

python track_aso.py "Example engine" en US "How do I compare options for X?" yes https://example.com/guide accurate "Example competitor" "Cited in comparison answer" "Collected manually"

4. Check crawler access without confusing the bots

Review server or CDN access logs for requests to your public pages. User-agent strings can help identify candidates, but they can be spoofed. When making access decisions or reporting verified traffic, check the provider’s current published IP ranges and documentation. Do not copy a static IP list into a guide or firewall rule and assume it remains current.

OpenAI: OAI-SearchBot is used to surface websites in ChatGPT search; GPTBot is associated with potential model-training crawls; ChatGPT-User may fetch pages in response to a user request. These have different purposes. OpenAI documents independent robots.txt controls for OAI-SearchBot and GPTBot, and says ChatGPT-User does not determine search appearance. See OpenAI’s crawler documentation.
Perplexity: PerplexityBot is used for search results, while Perplexity-User fetches pages at a user’s request. See Perplexity’s crawler documentation and use its current IP guidance when verifying traffic.
Google: Google Search Console reports its AI Overview and AI Mode activity within overall Google Search reporting, in the Web search type. This is Google-specific data, not a measurement of external answer engines or all user-triggered agent sessions. See Google’s AI features guidance.

Before changing robots.txt or firewall rules, decide which documented crawler use you intend to allow or restrict. A training crawler, a search crawler, and a user-triggered fetcher are not equivalent. Preserve the relevant logs and distinguish successful page responses from denied requests, errors, and requests for unrelated assets.

5. Use Google Search Console for Google-specific signals

Google says appearances in AI Overviews and AI Mode are included in Search Console’s overall search traffic reporting. The data can help you evaluate Google Search visibility alongside ordinary search performance, but it does not supply a universal ASO score or cover other providers’ answer engines.

Use Search Console to review Google Search performance and landing pages, then join that view to your analytics and conversion reporting where your measurement setup permits. Check the current interface and documentation when publishing or building a report because feature labels and reporting details can change. Google’s guidance emphasizes ordinary search eligibility and people-first content; it does not require a special AI text file or special markup for its AI Search features. Its generative AI guide also says llms.txt is not a required Google optimization. See AI Features and Your Website and Google’s generative AI guidance.

6. Connect visibility to visits and outcomes

Use your web analytics to examine referral sources and landing pages where attribution is available. Then compare those sessions with defined conversion events, such as qualified leads or signups. A missing referral does not prove that an answer never mentioned your site: many exposures do not produce a click, and platforms may not expose complete referral data.

For business reporting, set a baseline and annotate the dates of meaningful changes. Avoid attributing a conversion increase to an ASO change from a simple before-and-after comparison; seasonality, campaigns, product changes, and platform behavior can also affect results. Report what was observed and the limits of the attribution.

7. Improve the pages agents need to use

Make important pages accessible to the search and fetch systems you intend to support; inspect robots.txt, access controls, and server responses.
Give each page a clear purpose, descriptive headings, and direct answers supported by the page’s evidence.
Keep factual details, dates, pricing, and instructions current. Make important information available in the page content rather than relying solely on a fragile visual interaction.
Use ordinary structured data only when it accurately describes visible page content and is supported by the relevant search platform’s guidance.
Check mobile and rendered-page behavior, including consent overlays, popups, and client-side content that may obscure or delay the main information.

Google’s guidance does not call for special markup or a new machine-readable AI file to appear in its AI Search features. A file, schema addition, vendor score, or dashboard should not be presented as a guaranteed ranking lever.

8. Capture rendered pages while investigating agent usability

When a page’s rendered state matters—for example, a consent banner covers the content or a client-side section appears only after loading—a screenshot can help document what a person or rendering system sees. ScreenshotNeo is a website screenshot API and MCP server from Yorker Media; it can capture a page as an image or PDF. Use page captures as diagnostic evidence alongside logs and prompt observations, not as a substitute for answer-visibility or conversion measurement.

For a manual browser-based check, open the target page in a browser, wait for the content and overlays to settle, capture the viewport or full page, and record the URL, time, viewport, and relevant interactions. Repeat under the same conditions when comparing changes. A rendered screenshot can reveal obstructing UI, but it cannot establish that an AI engine crawled, cited, or recommended the page.

Or skip the browser setup

Make a screenshot API request for a page you want to inspect. See the ScreenshotNeo documentation for request options.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webp

Python

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://example.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://example.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', Buffer.from(await res.arrayBuffer())));

ScreenshotNeo removes cookie banners, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers report the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Sign up for 1,000 free screenshots a month, with no card required.

9. Choose a tracking approach

A manual prompt-and-log workflow is enough to begin: it is transparent, adaptable, and keeps the test design in your hands, though it takes recurring effort and requires careful recordkeeping. A SaaS visibility dashboard may automate prompt monitoring, historical views, or citation details, but coverage and methodology vary. Compare tools on supported engines, prompt coverage, repeatability, cited-URL detail, historical retention, export access, and cost. Treat vendor scores as that vendor’s measurement unless the method is transparent and independently validated.

For page-rendering evidence, ScreenshotNeo is the screenshot service to try first: it removes common overlays before capture, bills only clean shots, and its paid plans start at $5 for 3,000. That solves a different problem from prompt visibility monitoring, so use it to inspect pages rather than infer cross-engine answer share.

10. Troubleshooting tracking problems

Symptom	Likely cause	What to do
You see no crawler requests	The provider has not fetched the site in the period, the relevant bot is blocked, or logs omit the request	Check logging and CDN coverage, robots.txt, access controls, and current provider documentation. Do not infer that a missing log proves a page cannot appear.
A user-agent appears but the IP is unfamiliar	User-agent strings can be imitated, or provider ranges changed	Verify against current published IP ranges and provider guidance before treating traffic as genuine.
A crawler request returns 403 or 429	Firewall, bot management, rate limiting, or access policy denied the request	Review the response and security logs; adjust access only if you intend to permit that documented crawler purpose.
The site appears in an answer but has no referral session	The answer may not link to the site, the user may not click, or attribution may be unavailable	Record visibility separately from referral traffic. Do not treat absent referrals as proof of absent exposure.
Prompt results change between runs	Answers vary by time, wording, location, engine state, or other context	Keep test conditions stable, record them, retain samples, and report variation rather than collapsing results into a false precise score.
A citation points to an outdated or weak page	The cited URL may be stale, unclear, inaccessible, or less relevant than another page	Check the URL and content, improve the relevant page, and monitor subsequent observations; no change guarantees a citation.
A screenshot is blank or shows an overlay	The page may still be loading, require interaction, or display a consent or popup layer	Wait for the relevant content, inspect page behavior and capture options, and compare with a browser view. ScreenshotNeo reports page verdict and billing headers for API responses.

11. Performance, reliability, and cost

Keep the first measurement set small enough to review accurately, then expand it to cover important audiences and tasks. A consistent sample with saved evidence is more useful than a large set whose prompts, conditions, and scoring rules change each run. Set a cadence appropriate to your reporting needs and note platform or site changes that could affect comparisons.

Access logs are useful but incomplete: retention, CDN configuration, caching, and provider behavior affect what you observe. Prompt samples are not a census of all answers. Referrals undercount views that do not lead to clicks, while conversions can have multiple causes. Report denominators, dates, coverage, and known gaps.

Manual tracking has little software cost but requires staff time. Dashboards can reduce manual work, but subscription cost and platform coverage should be checked against your actual use case. There is no broadly accepted cross-engine ASO benchmark or established effect size in the sources reviewed here. Avoid claiming a guaranteed return from a vendor score or a single technical change.

Frequently asked questions

Is Agent Search Optimization an official Google ranking factor?

ASO is an emerging practitioner term, not an official Google ranking factor or a standardized cross-platform discipline. Google’s published guidance describes its own search features and reporting.

How do I check if Google-Agent is visiting my site?

Inspect access logs and verify the request against current official crawler documentation and IP guidance. Do not rely on a user-agent string alone; Google Search Console provides Google Search performance data, which answers a different question from raw server access.

Does llms.txt make a site visible to AI agents?

Do not treat it as a guaranteed visibility mechanism. Google’s guidance says it is not required for its AI Search features; other systems may have their own documentation and behavior.

Can one score tell me how well my site performs across AI search?

No universal cross-engine ASO score is established in the reviewed sources. Track access, sampled answers, referrals, and outcomes as separate measures.

Should I block training crawlers to improve search visibility?

Training and search crawlers have distinct purposes and controls. Make the choice based on your access policy and the provider’s current documentation; blocking one does not itself establish better search visibility.