How to Scrape Google Search Results with an API

Short answer: Google’s supported API for programmatic search results is the Custom Search JSON API. Send an HTTPS GET request to https://www.googleapis.com/customsearch/v1 with an API key (key), Programmable Search Engine ID (cx) and URL-encoded query (q). The response is JSON containing result titles, links, snippets and query metadata.

There is an important availability limitation: Google says the Custom Search JSON API is closed to new customers. Existing customers have until January 1, 2027 to transition to an alternative. If you are starting a new project, evaluate the alternatives in this guide before designing around this API.

What this API does—and what it does not do

The Custom Search JSON API queries a configured Programmable Search Engine. You configure which sites or web scope the engine can search, then call the API to receive JSON. This is different from automating a browser against google.com result pages.

Question	Answer
Is this a live copy of every Google SERP?	No. Results come through your Programmable Search Engine configuration and its supported scope.
What does `cx` mean?	The identifier of your Programmable Search Engine.
What does `q` mean?	The search query, URL-encoded in the request.
What does the API return?	JSON with query metadata and, when matches exist, result items such as title, link and snippet.
Can a new customer sign up?	Google currently states that Custom Search JSON API enrollment is closed to new customers.

Prerequisites and setup

Confirm access status. If your account is not an existing Custom Search JSON API customer, plan for an alternative such as Vertex AI Search or a commercial SERP provider.
Create or select a Programmable Search Engine. In the control panel, define the sites or web scope it can search and copy its engine ID, called cx.
Create an API key. Keep it on your server or in a secret manager. Do not place an unrestricted key in browser JavaScript.
Record the request limits. Google’s overview documents 100 free queries per day for existing customers, then $5 per 1,000 additional requests, capped at 10,000 queries per day. Verify the current page before budgeting.

The request format

The REST endpoint is:

GET https://www.googleapis.com/customsearch/v1?key=API_KEY&cx=SEARCH_ENGINE_ID&q=encoded+query

The REST guide documents a 2,048-character request-length limit. Build requests with a URL encoder rather than concatenating unescaped user input.

Complete cURL example

curl --fail-with-body -G \
  "https://www.googleapis.com/customsearch/v1" \
  --data-urlencode "key=${GOOGLE_API_KEY}" \
  --data-urlencode "cx=${GOOGLE_SEARCH_ENGINE_ID}" \
  --data-urlencode "q=how to scrape google search results with an API" \
  --data-urlencode "num=10" \
  --data-urlencode "start=1"

Save the response and inspect it with a JSON tool:

curl -sS -G "https://www.googleapis.com/customsearch/v1" \
  --data-urlencode "key=${GOOGLE_API_KEY}" \
  --data-urlencode "cx=${GOOGLE_SEARCH_ENGINE_ID}" \
  --data-urlencode "q=example query" | jq .

Python: request and parse results

import os
import requests

API_URL = "https://www.googleapis.com/customsearch/v1"

params = {
    "key": os.environ["GOOGLE_API_KEY"],
    "cx": os.environ["GOOGLE_SEARCH_ENGINE_ID"],
    "q": "how to scrape google search results with an API",
    "num": 10,
    "start": 1,
}

response = requests.get(API_URL, params=params, timeout=30)
response.raise_for_status()
data = response.json()

# A successful search can have no items. Treat that as zero results.
for item in data.get("items", []):
    print(item.get("title", ""))
    print(item.get("link", ""))
    print(item.get("snippet", ""))
    print()

# Useful operational metadata
print("query:", data.get("queries", {}))
print("search information:", data.get("searchInformation", {}))

Install the dependency with python -m pip install requests. A production client should catch HTTP errors, record the response body for diagnosis and avoid logging the API key.

Node.js: request and parse results

const endpoint = 'https://www.googleapis.com/customsearch/v1';
const params = new URLSearchParams({
  key: process.env.GOOGLE_API_KEY,
  cx: process.env.GOOGLE_SEARCH_ENGINE_ID,
  q: 'how to scrape google search results with an API',
  num: '10',
  start: '1'
});

const response = await fetch(`${endpoint}?${params}`);
const body = await response.json();

if (!response.ok) {
  throw new Error(`Google API ${response.status}: ${JSON.stringify(body)}`);
}

for (const item of body.items ?? []) {
  console.log(item.title);
  console.log(item.link);
  console.log(item.snippet ?? '');
  console.log();
}

console.log('queries:', body.queries ?? {});
console.log('search information:', body.searchInformation ?? {});

This uses the built-in fetch available in modern Node.js releases. Set GOOGLE_API_KEY and GOOGLE_SEARCH_ENGINE_ID in the process environment.

Parameters you will commonly use

Parameter	Purpose	Implementation note
`key`	Google API key	Keep it server-side and restrict it where Google supports restrictions.
`cx`	Programmable Search Engine ID	Required; it determines the configured search scope.
`q`	Search text	URL-encode it; enforce your own input length limit below Google’s request limit.
`num`	Number of results requested	Use the smallest page size your application needs.
`start`	Pagination offset	Read the `queries` metadata to discover next-page links and bounds instead of assuming pages exist.

Google’s reference documents additional parameters for filtering and presentation. Add only parameters supported by your configured engine and verify behavior against the current reference.

Parsing responses defensively

Do not assume that items is present. A valid response can contain no matching results, in which case your code should return an empty list. Inspect:

items[].title, items[].link and items[].snippet for result data.
queries for the original request, pagination information and next-page metadata.
searchInformation for information such as total result estimates and search time.

Persist the raw response shape in logs or fixtures during development so a schema change or configuration issue is visible without exposing credentials.

Pagination pattern

function collectItems(page) {
  return (page.items ?? []).map(({ title, link, snippet }) => ({
    title: title ?? '',
    link: link ?? '',
    snippet: snippet ?? ''
  }));
}

function nextStart(page) {
  const next = page.queries?.nextPage?.[0];
  return next?.startIndex ?? null;
}

Use the returned pagination metadata and stop when there is no next page. This avoids making requests that cannot return additional items.

Display, attribution and terms

If your application displays Programmable Search results to users, follow Google’s attribution and branding guidance, including placement adjacent to the relevant search box or results. API use also requires acceptance of Google’s API terms, Programmable Search Engine terms and the additional Custom Search terms. Review those terms for your deployment and data-handling model.

API retrieval versus browser scraping

An API response is structured, easier to validate and less fragile than parsing changing HTML. Browser automation against Google result pages is a separate compliance and engineering question. The official material used here does not provide a jurisdiction-by-jurisdiction legal opinion on direct HTML scraping, so obtain current legal advice for your use case rather than assuming it is universally allowed or prohibited.

Choosing an alternative for a new project

Path	Best fit	Trade-offs to check
Custom Search JSON API	Existing enrolled customers using a configured Programmable Search Engine	Closed to new customers; configured scope; documented quota and attribution rules.
Vertex AI Search	New projects that can use Google’s named alternative	Check supported data sources, ranking behavior, pricing and regional requirements.
Commercial SERP API	Applications that need live Google SERP-style data or richer SERP features	Verify current pricing, quotas, feature coverage, retention, compliance and service limits vendor by vendor.

Compare access status, result fidelity, quota and cost, compliance and attribution, operational burden, caching, retries and schema stability. A vendor exhibit can establish that the SERP API category exists, but it is not evidence of current pricing or feature parity.

Or skip the browser setup

If your actual requirement is capturing rendered pages, ScreenshotNeo provides a website screenshot API and MCP server. It is separate from Google’s search-results API, but useful when your workflow needs visual evidence of a result page or any other URL.

One GET request returns PNG, JPEG, WebP or PDF. See the ScreenshotNeo API documentation for all options.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Cookie banners, newsletter popups and chat widgets are removed before the shot.
Bot checks, blank pages, failed loads and timeouts are not billed; response headers identify the page verdict and whether it was billed.
An MCP server lets Claude, Cursor and other MCP clients call take_screenshot, get_page_info and capture_pdf.
Free accounts include 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots.

Create a free ScreenshotNeo account to try it without a card.

Reliability and performance checklist

Cache identical queries for a short, documented period when freshness permits.
Set connect and read timeouts; retry only transient failures with exponential backoff and a cap.
Do not retry invalid credentials, invalid engine IDs or quota errors until configuration or quota changes.
Track request count, HTTP status, latency, empty-result rate and quota responses.
Bound concurrent requests so one user or batch cannot consume the daily limit.
Store the query, engine ID, pagination position and request timestamp for reproducibility, but redact API keys.
Return a stable internal schema so callers do not depend directly on every field in Google’s response.

Cost and quota planning

For existing customers, Google documents 100 free queries per day, followed by $5 per 1,000 additional requests, with a 10,000-query daily cap. These are legacy figures and availability details can change, so verify Google’s current overview before publishing a price or committing to a forecast. Estimate worst-case usage from users, retries and pagination—not only from the number of visible searches.

Troubleshooting

Symptom	Likely cause	Fix
HTTP 400 with an invalid request message	Missing `q`, malformed parameters or an overlong URL.	Use a URL encoder, confirm `key`, `cx` and `q`, and keep the request under Google’s documented 2,048-character limit.
HTTP 400 or 403 mentioning the engine	The `cx` value is wrong or the engine is not configured for the requested scope.	Copy the engine ID again and review the Programmable Search Engine configuration.
HTTP 403 for credentials or quota	The key is invalid, restricted incorrectly, or the project has exceeded quota.	Check the key and API enablement, inspect quota, then correct restrictions or wait for the quota window.
HTTP 429 or intermittent failures	Rate limiting or a burst of concurrent requests.	Reduce concurrency and retry transient failures with exponential backoff.
Successful response but no `items`	No results for the configured scope, or filtering excluded all matches.	Treat it as an empty result set and inspect `queries` and `searchInformation`.
Results differ from google.com	Programmable Search Engine scope and ranking are not identical to Google Web Search.	Review the engine’s included sites and use an appropriate alternative if live SERP replication is required.
Key appears in browser traffic	The credential was embedded in client-side code.	Move requests behind your server and rotate the exposed key.

FAQ

Can I get a Google Search API key today?

You can create Google API credentials for supported services, but Google currently states that the Custom Search JSON API is closed to new customers. Confirm eligibility before building around it.

Is `cx` the same as an API key?

No. key authenticates the Google Cloud project; cx identifies the Programmable Search Engine that supplies the search scope.

Why is my response missing snippets?

Fields can vary by result and configuration. Read optional fields defensively and always provide a fallback for missing snippets.

Should I scrape Google’s HTML instead?

That is a separate browser-automation and compliance decision. The Custom Search JSON API is the documented structured product; review current terms and legal requirements before using another method.

How should I handle a query with zero matches?

Return an empty list, preserve the query metadata, and avoid treating a missing items array as a transport failure.