ScreenshotNeo

BlogHow-to

How to Scrape Google Jobs in 2026

Google has no stable public Jobs scraping endpoint. Learn the compliant workflow for collecting authorized job data and exporting it reliably.

By the ScreenshotNeo team30 September 202610 min read

How to Scrape Google Jobs in 2026

Direct answer: Google does not document a stable public endpoint for scraping the Google Jobs panel. Google says automated queries and scraping Search results without express permission violate its spam policies and Terms of Service. A defensible 2026 workflow is to collect data from employer career pages, your own job pages, or an authorized feed/API, then normalize the listings and export them to CSV.

If you own the listings, publish one page per job with JobPosting structured data, make that markup visible and consistent with the page, and notify Google about changes through the Indexing API. If you have written permission to collect another source, document the permitted URLs, geography, refresh rate, rate limits, retention period, and attribution requirements before writing code.

1. What “scraping Google Jobs” means in 2026

The Google Jobs panel is a Search presentation layer. Its markup, localization, pagination and anti-automation behavior can change. Google’s spam policies describe automated queries and scraping Search results without express permission as machine-generated traffic. Google’s Terms of Service also restrict automated access that violates machine-readable instructions and prohibit scraping content that does not belong to you.

Google does not publish a supported “Google Jobs scraper” endpoint. Google API terms add separate restrictions: unless expressly permitted, you may not scrape content returned from Google APIs, build databases from it, or retain permanent copies beyond allowed cache periods.

Source When it is appropriate Main risk
Your own career pages You control the pages and data rights Schema and deployment mistakes
Employer pages with permission A contract or written permission defines collection Terms, rate limits and retention must be followed
Licensed jobs feed or API The provider documents provenance and rights Coverage, geography and resale limits vary
Google Search or Jobs result scraping Only where Google has expressly authorized the access Policy exposure, breakage and unstable markup

2. The supported path for job owners

Google’s JobPosting documentation says the markup belongs on the most specific page describing one job. Google recommends JSON-LD, validation with the Rich Results Test and URL Inspection, and consistency between structured data and visible content.

The Indexing API documentation limits supported pages to those containing JobPosting or BroadcastEvent structured data. It can notify Google when a URL is new, updated or removed. Google states: “For job posting URLs, we recommend using the Indexing API instead of sitemaps because the Indexing API prompts Googlebot to crawl your page sooner.”

Minimal JobPosting JSON-LD

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "JobPosting",
  "title": "Backend Engineer",
  "description": "Build and operate APIs for our product.",
  "datePosted": "2026-09-01",
  "validThrough": "2026-10-01T23:59:59Z",
  "employmentType": "FULL_TIME",
  "hiringOrganization": {
    "@type": "Organization",
    "name": "Example Employer",
    "sameAs": "https://example.com",
    "logo": "https://example.com/logo.png"
  },
  "jobLocation": {
    "@type": "Place",
    "address": {
      "@type": "PostalAddress",
      "addressLocality": "Austin",
      "addressRegion": "TX",
      "addressCountry": "US"
    }
  },
  "identifier": {
    "@type": "PropertyValue",
    "name": "Example Employer",
    "value": "ENG-1234"
  }
}
</script>

3. A compliant collection workflow

  1. Define the permitted source. Record whether the source is your own pages, an authorized feed/API or employer pages whose terms permit collection. Write down geography, language, refresh cadence and retention.
  2. Prefer structured data. Parse JSON-LD JobPosting objects when present. Preserve the source URL and exact retrieval timestamp.
  3. Normalize fields. Store a stable internal ID, identifier, title, hiring organization, date posted, valid-through date, employment type, location, salary when supplied, canonical URL and source employer.
  4. Keep evidence. Retain the raw JSON or an integrity hash, parser version and retrieval time so later corrections can be audited.
  5. Deduplicate. Merge reposts only when employer, title, location and identifier or canonical URL support the merge. Keep every source URL that contributed fields.
  6. Respect controls. Read each site’s robots.txt and terms. Robots.txt manages crawler access and traffic; it is not authentication. Google’s robots specification documents a 500 KiB file-size limit and generally up-to-24-hour caching.
  7. Monitor change. Track parser failures, schema changes, HTTP status changes, duplicate rates and stale validThrough values.
  8. Publish provenance. Include source employer, source URL, retrieval time and last-seen time in exports. Do not call a listing open solely because it appeared in an earlier result.
A compliant workflow preserves structured job data and provenance from an employer page to an export.
A compliant workflow preserves structured job data and provenance from an employer page to an export.

4. Python collector for authorized employer pages

The following script fetches pages you are authorized to collect, extracts JSON-LD, keeps only JobPosting objects and writes a CSV. It does not query Google Search or bypass access controls.

#!/usr/bin/env python3
import csv
import hashlib
import json
import sys
from datetime import datetime, timezone
from urllib.parse import urljoin

import requests
from bs4 import BeautifulSoup

HEADERS = {"User-Agent": "AuthorizedJobCollector/1.0 (+contact@example.com)"}
TIMEOUT = 30


def jsonld_objects(soup):
    for tag in soup.select('script[type="application/ld+json"]'):
        try:
            value = json.loads(tag.string or tag.get_text())
        except json.JSONDecodeError:
            continue
        values = value if isinstance(value, list) else [value]
        for item in values:
            if isinstance(item, dict) and item.get("@type") == "JobPosting":
                yield item


def text(value):
    if isinstance(value, dict):
        return value.get("name") or value.get("value") or ""
    return value or ""


def location(item):
    loc = item.get("jobLocation") or {}
    if isinstance(loc, list):
        loc = loc[0] if loc else {}
    address = loc.get("address", {}) if isinstance(loc, dict) else {}
    if isinstance(address, str):
        return address
    parts = [address.get(k, "") for k in ("streetAddress", "addressLocality", "addressRegion", "postalCode", "addressCountry")]
    return ", ".join(p for p in parts if p)


def collect(url):
    response = requests.get(url, headers=HEADERS, timeout=TIMEOUT)
    response.raise_for_status()
    soup = BeautifulSoup(response.text, "html.parser")
    retrieved = datetime.now(timezone.utc).isoformat()
    raw_hash = hashlib.sha256(response.content).hexdigest()
    rows = []
    for item in jsonld_objects(soup):
        org = item.get("hiringOrganization") or {}
        identifier = item.get("identifier") or {}
        rows.append({
            "source_url": response.url,
            "retrieved_at": retrieved,
            "raw_sha256": raw_hash,
            "title": item.get("title", ""),
            "employer": text(org),
            "job_id": text(identifier),
            "date_posted": item.get("datePosted", ""),
            "valid_through": item.get("validThrough", ""),
            "employment_type": text(item.get("employmentType")),
            "location": location(item),
            "salary": json.dumps(item.get("baseSalary", ""), ensure_ascii=False),
            "canonical_url": item.get("url") or response.url,
        })
    return rows


if __name__ == "__main__":
    urls = sys.argv[1:]
    if not urls:
        raise SystemExit("usage: python collect_jobs.py https://employer.example/jobs/role")
    rows = []
    for url in urls:
        try:
            rows.extend(collect(url))
        except requests.RequestException as exc:
            print(f"failed {url}: {exc}", file=sys.stderr)
    fields = ["source_url", "retrieved_at", "raw_sha256", "title", "employer", "job_id", "date_posted", "valid_through", "employment_type", "location", "salary", "canonical_url"]
    with open("jobs.csv", "w", newline="", encoding="utf-8") as output:
        writer = csv.DictWriter(output, fieldnames=fields)
        writer.writeheader()
        writer.writerows(rows)

Run it

python -m pip install requests beautifulsoup4
python collect_jobs.py \
  https://careers.example.com/jobs/backend-engineer \
  https://careers.example.com/jobs/data-engineer
cat jobs.csv

5. cURL and Node.js equivalents

For a one-off authorized page, cURL can preserve the raw response for inspection:

curl --fail --location --user-agent 'AuthorizedJobCollector/1.0 (+contact@example.com)' \
  'https://careers.example.com/jobs/backend-engineer' \
  --output job.html

Node.js can fetch and parse JSON-LD with Cheerio:

import { writeFile } from 'node:fs/promises';
import * as cheerio from 'cheerio';

const url = process.argv[2];
if (!url) throw new Error('usage: node collect.mjs https://employer.example/jobs/role');
const response = await fetch(url, {
  headers: { 'user-agent': 'AuthorizedJobCollector/1.0 (+contact@example.com)' }
});
if (!response.ok) throw new Error(`${response.status} ${response.statusText}`);
const html = await response.text();
const $ = cheerio.load(html);
const jobs = [];
$('script[type="application/ld+json"]').each((_, element) => {
  try {
    const value = JSON.parse($(element).text());
    for (const item of (Array.isArray(value) ? value : [value])) {
      if (item?.['@type'] === 'JobPosting') jobs.push(item);
    }
  } catch {}
});
await writeFile('jobs.json', JSON.stringify({ source_url: response.url, jobs }, null, 2));
console.log(`wrote ${jobs.length} job postings`);

6. Exporting, deduplication and updates

A useful export includes one row per current source record plus provenance. Recommended columns are:

Column Purpose
canonical_url Stable page identity
employer, job_id Employer and source identifier
retrieved_at, last_seen_at Freshness and audit trail
title, location, employment_type Normalized search fields
date_posted, valid_through Lifecycle dates
salary Preserve only when supplied by the source
raw_sha256, parser_version Change detection and reproducibility

Use employer + job_id as the strongest deduplication key. If no identifier exists, require agreement on canonical URL, employer, title and location before merging. Keep a source URL history instead of overwriting evidence. Mark a record closed only after the source indicates removal, an expired validThrough, or a documented absence policy.

7. If you own the listings: notify Google

After publishing or changing a page, validate the structured data with Google’s Rich Results Test and URL Inspection. Use the Indexing API only for supported JobPosting or BroadcastEvent pages. Send notifications for URL updates and removals, and keep your own publication state as the source of truth.

8. Browser automation: when it is justified

Some authorized career sites render jobs only after JavaScript runs. In that case, use a normal browser session against the permitted employer domain, wait for the job container, extract the rendered JSON-LD or visible fields, and stop on consent, login, CAPTCHA or access-denied pages. Do not rotate proxies, spoof fingerprints or bypass a CAPTCHA to defeat controls.

import asyncio
from playwright.async_api import async_playwright

async def main(url):
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()
        await page.goto(url, wait_until="networkidle", timeout=60_000)
        for script in await page.locator('script[type="application/ld+json"]').all():
            try:
                print(await script.text_content())
            except Exception:
                pass
        await browser.close()

asyncio.run(main("https://careers.example.com/jobs/backend-engineer"))

9. Or skip the browser setup

If your task is to capture an authorized employer page for review or archiving, ScreenshotNeo provides one GET request that returns PNG, JPEG, WebP or PDF. Its consent step accepts cookie banners and removes more than 60 known consent platforms, newsletter popups and chat widgets before capture; each step can be turned off. Bot checks, blank pages, timeouts, failed loads and cache hits are not billed, and the response identifies the result with X-Page-Verdict and X-Billed headers. It also offers an MCP server with take_screenshot, get_page_info and capture_pdf for Claude, Cursor and other MCP clients.

Consent and overlay handling can produce a clean capture for an authorized page.
Consent and overlay handling can produce a clean capture for an authorized page.

See the ScreenshotNeo API documentation for all options.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Free accounts include 1,000 screenshots each month with no card. Paid plans start at $5 for 3,000 screenshots. Create a free ScreenshotNeo account.

10. Troubleshooting

Symptom Likely cause Fix
No JobPosting objects found Page has no JSON-LD, uses a different type, or renders it after JavaScript Inspect raw HTML, use an authorized browser session, or ask the employer for a feed
HTTP 403 or 429 Permission, rate limit or access policy Stop, review terms and robots.txt, reduce cadence, and obtain written authorization
Duplicate rows Reposts, tracking URLs or multiple source pages Normalize canonical URLs and deduplicate with employer plus identifier evidence
Expired jobs remain Stale export or ignored validThrough Re-fetch on the permitted cadence and track last-seen state
Salary is missing The source did not supply salary fields Preserve an empty value; do not infer compensation
Consent dialog blocks capture Client-side banner or regional consent requirement Handle it as a normal visitor where permitted, or use ScreenshotNeo’s consent handling
Screenshot is blank Page timeout, bot check or failed load Inspect X-Page-Verdict, wait for a selector or network idle, and fix the source page

11. Performance, reliability and cost

  • Fetch only what you need. Use the source’s permitted refresh interval and conditional requests where supported.
  • Separate discovery from detail. Keep a queue of known URLs, then fetch individual pages for full structured data.
  • Make jobs idempotent. Hash raw payloads, store parser versions and upsert by a stable source key.
  • Bound concurrency. Parallel requests can trigger limits and make failures harder to replay.
  • Measure freshness. Track retrieval age, parse success, duplicate rate and stale postings.
  • Budget managed services by successful captures. ScreenshotNeo bills only clean shots; bot checks, blank pages, timeouts, failed loads and cache hits cost nothing.
  • Verify provider rights. For any managed jobs API, confirm authorization, data provenance, coverage, retention, geography and affiliate terms independently.
  • Have written permission or a documented ownership basis for every source.
  • Read terms, robots.txt and rate limits before collecting.
  • Do not present CAPTCHA evasion, proxy rotation or fingerprint spoofing as normal practice.
  • Keep source URL, retrieval time, raw hash and parser version.
  • Honor deletion, retention and attribution requirements.
  • Label exports with last-seen time and source employer.
  • Re-check policy and contract terms when scope, geography or volume changes.

13. FAQ

Is there an official Google Jobs scraping API?

Google does not document a stable public endpoint for scraping the Jobs panel. The supported Google APIs in this area are for indexing pages containing eligible structured data, not for downloading a Jobs database.

Can I scrape Google Jobs if the results are publicly visible?

Public visibility alone does not grant permission for automated access. Google’s policies address automated Search access and scraping without express permission.

Can robots.txt make a page private?

No. Robots.txt is a crawler instruction and traffic-management mechanism, not authentication or a security boundary.

What is the safest replacement for Google Jobs scraping?

Use employer career pages, a licensed feed, or a provider that can document its authorization and data rights. Preserve provenance so users can verify each listing.

Should I store the full source response?

Store raw JSON or an integrity hash when your terms and retention policy allow it. A hash plus parser version provides change evidence without requiring permanent copies of every response.

How do I capture a job page for an internal report?

Use a permitted browser session or a screenshot service such as ScreenshotNeo, then retain the source URL and capture time alongside the image or PDF.