Urlwatch Is Not Detecting Page Changes: Troubleshooting Guide

If urlwatch is not detecting a page change, check the whole pipeline in order: whether the command ran, whether the job fetched the intended page, whether JavaScript-rendered content was present, whether filters preserved the changed text, whether the comparison produced a visible diff, and whether a reporter was enabled. A missing alert alone does not tell you which stage failed.

Urlwatch compares each job’s current output after filtering with its previously retrieved output, then invokes enabled reporters when it finds a difference. Start with the earliest stage and move forward. [urlwatch documentation]

1. Confirm the scheduled command runs

Urlwatch only checks jobs when its command runs. A correct job cannot detect a change between invocations. Check the scheduler entry, the operating-system account that runs it, the working directory, environment variables, and command output or logs. A command that works in your interactive shell may fail under cron because it uses a different PATH, configuration directory, or account.

Run the same urlwatch command manually as the scheduled account, if possible.
Check the scheduler’s execution history and capture standard output and errors.
Confirm it points to the intended configuration and database locations.
Set an interval that fits the monitoring need and the site’s policies. Urlwatch documentation uses an interval no more frequent than every 30 minutes as an introductory cron example; it is guidance, not a universal requirement. [Scheduling documentation]

If the command never runs or exits with an error, fix that before investigating filters or reporters.

2. Confirm the job exists and targets the right page

List configured jobs and inspect their indices and URLs:

urlwatch --list

Jobs are stored in the YAML job list, commonly urls.yaml. Edit it with urlwatch’s editor or your preferred editor:

urlwatch --edit

Check for a stale URL, a redirect to a different page, an accidental duplicate job, or an index that refers to another job. A regular URL job retrieves what the server serves; it does not automatically execute client-side JavaScript. [Job configuration documentation]

3. Determine whether the page requires JavaScript

Some sites send a mostly empty HTML shell and fill the changing content in the browser. In that case, a URL job may successfully fetch the page while never seeing the text you want to monitor.

Compare the server response with what a browser displays. If the target text is absent from the server-served content but present after the page renders, use a browser job with navigate. Urlwatch browser jobs use Playwright; Playwright and its browser binaries must be installed. Browser jobs also consume substantially more resources than ordinary URL jobs. [Browser jobs documentation]

Use a URL job when the content is in the server response. Choose a browser job only when the rendered page is necessary, and account for its installation and resource requirements.

4. Inspect the content that actually reaches comparison

Filters transform fetched content before urlwatch compares it. A filter that selects the wrong element, strips a relevant attribute, or removes too much text can make a real page change invisible.

Use --test-filter with a job index or URL to preview the current filtered output:

# Replace 3 with the job index shown by --list
urlwatch --test-filter 3

# You can also identify the job by URL
urlwatch --test-filter https://example.com/page

Inspect the output and make sure it includes the precise text or structure expected to change. If the target is missing, revise that job’s filter pipeline, then run --test-filter again. [Filter documentation]

5. Account for snapshots created before a filter change

Urlwatch filtered the previous snapshot when it retrieved that version. Editing the filter does not rewrite the stored historical snapshot. Consequently, an unchanged report is not a reliable way to test a newly edited filter: the stored comparison input may have been produced by the old filter.

Use urlwatch --test-filter to apply the current filter to current page content. The official configuration documentation explicitly recommends this approach instead of using an unchanged report to test filters. [Configuration documentation]

6. Check diff and display settings, then verify reporting

A change can be detected in the comparison stage but disappear from the report. A diff_filter can reduce a detected change to an empty diff. The display.empty-diff setting controls whether urlwatch includes such a change. The display.unchanged setting controls unchanged jobs; new-job and error display are controlled separately.

Review the configuration’s display section and check the settings relevant to the result you expect.
Inspect any diff_filter for rules that remove the changed lines.
Confirm the intended reporter is enabled and configured, and check its errors or delivery logs.

Do not infer that fetching failed just because no notification arrived. First establish whether filtered output and a diff exist, then check display behavior and reporter delivery. [Configuration documentation]

7. Tune browser readiness for late content

When a browser job is needed, the page may not contain the target content as soon as navigation returns. Browser jobs support wait_until values load, domcontentloaded, networkidle, and commit, plus a wait_for locator. Choose a condition tied to the content you need to inspect. Urlwatch documentation discourages relying on networkidle and recommends readiness assertions instead. [Browser job options]

For example, configure a browser job to navigate to the page and wait for a locator that identifies the changing content. Use the actual selector and URL for your target:

kind: browser
navigate: https://example.com/page
wait_for: "main .release-notes"

Check the installed urlwatch version’s configuration documentation for the exact schema supported by that version. A wait condition that succeeds before the target content appears can produce consistent but incomplete snapshots.

Troubleshooting by symptom

Symptom	Likely stage	What to check	Fix
No run output or no recent check	Scheduling	Scheduler history, command path, account, environment, config path	Correct the scheduled command and capture its errors; verify it runs as the intended account.
Job is missing or checks an unexpected page	Configuration or retrieval	`urlwatch --list`, job index, URL, redirects	Correct the YAML job and confirm the intended target.
Browser shows changed text but filtered output does not	Rendering or filtering	Whether text is in the server response; output from `--test-filter`	Use a Playwright browser job if JavaScript is required, or revise the filter.
Filter was just changed but report says unchanged	Historical snapshot	Whether the old snapshot was created with the previous filter	Test current content with `--test-filter`; do not use an unchanged report to validate the new filter.
Diff exists but notification is absent	Diff, display, or reporting	`diff_filter`, `display.empty-diff`, reporter enablement and delivery errors	Adjust the relevant setting or filter and repair reporter configuration.
Browser job intermittently misses late content	Readiness	`wait_until` and whether the target locator is present before capture	Wait for the target locator or a suitable readiness condition; avoid treating network idle as a universal signal.
Browser job fails after setup	Dependencies	Playwright installation and browser binaries for the environment running urlwatch	Install the required Playwright components in that same runtime environment.

Performance, reliability, and monitoring cost

Polling frequency determines how soon a scheduled check can notice a change; it does not guarantee an alert if retrieval, filtering, display, or reporting is broken. Pick an interval based on how quickly the change matters and the site’s acceptable request rate. The urlwatch documentation’s 30-minute-or-longer cron example is a starting point, not a measured performance claim.

Prefer a URL job for server-returned content: it avoids the extra Playwright and browser installation burden. Use a browser job only for content that genuinely depends on rendering, and make the readiness condition specific so the browser does not wait unnecessarily or snapshot too early. Keep filters focused on stable, relevant content; broad extraction can admit volatile page material, while aggressive filtering can hide the very change being monitored.

Urlwatch’s resource cost depends on how often it runs, how many jobs it checks, and whether jobs launch browsers. The research sources do not provide a universal runtime, resource estimate, or monetary cost, so measure those in your own environment rather than assuming a benchmark.

Or skip the browser setup

If the reason you need browser rendering is to capture a page as an image or PDF, ScreenshotNeo is a website screenshot API and MCP server. Its API returns a screenshot or PDF from one GET request. See the API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://example.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://example.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);

Cookie banners, popups, and chat widgets are removed before the shot. Bot checks, blank pages, and failed loads are never billed; response headers identify the page verdict and billing status. An MCP server lets AI agents use screenshot tools. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Screenshot capture does not replace urlwatch’s text comparison and alert pipeline when that is what you need.

FAQ

Does urlwatch check continuously?

No. It checks jobs when the command runs, so the scheduler determines the check interval.

Can an unchanged result prove my new filter works?

No. The stored snapshot may reflect the filter that was active when it was captured. Preview current filtered content with --test-filter.

Should I always use a browser job?

No. Use a regular URL job when the server response contains the content. Browser jobs add Playwright and browser dependencies and use more resources.

Is `networkidle` the best browser wait condition?

Not by default. Urlwatch documentation discourages it; wait for a locator or another readiness condition that reflects the content you monitor.