How to Monitor Multiple URLs with Urlwatch from One YAML File

To monitor multiple URLs with Urlwatch, define one job per URL in urls.yaml and put a line containing only --- between jobs. Give each job a descriptive name, use url for ordinary HTTP pages, and run urlwatch periodically with cron or another scheduler. Filters can reduce noisy page changes before Urlwatch compares each result with its previous run.

Urlwatch stores the jobs in urls.yaml; global settings and notification reporters are configured separately in urlwatch.yaml. The official [Urlwatch introduction](https://urlwatch.readthedocs.io/en/latest/introduction.html) recommends using urlwatch --edit to edit and validate jobs and suggests running no more often than every 30 minutes.

1. Install and initialize Urlwatch

Install Urlwatch using the [official installation instructions](https://urlwatch.readthedocs.io/en/latest/install.html) for your environment. Then run it once to initialize or migrate its data:

urlwatch

Open the jobs file through Urlwatch’s editor:

urlwatch --edit

This opens urls.yaml. If the editor does not launch, set EDITOR or VISUAL in your shell, for example export EDITOR=nano, then run the command again. The editor checks the configuration before activating it.

2. Put multiple URL jobs in one YAML file

Here is a compact example with two websites. Replace the example URLs with pages you are allowed to monitor.

name: "Service status"
url: "https://status.example.com/"
---
name: "Product release notes"
url: "https://example.com/releases"

Each job is a YAML mapping. The separator is three hyphens by themselves on a line; it is not a list marker and should not be indented. Each job must define exactly one job type: url, navigate, or command. For standard pages, url makes an HTTP GET request by default. A readable name is strongly recommended because it makes reports easier to interpret. See the [Urlwatch jobs reference](https://urlwatch.readthedocs.io/en/stable/jobs.html).

Add useful filters

Web pages often change in ways that do not matter to you: navigation, timestamps, rotating promotions, or formatting. Filters transform or select the fetched content before comparison. This illustrative job focuses on the main content and converts it to text:

name: "Release notes"
url: "https://example.com/releases"
filter:
  - xpath: '//main'
  - html2text:

Filter order matters: the output of each filter is passed to the next one. XPath selection narrows the HTML to the chosen element, and html2text converts the result to text. The example assumes the page has a useful <main> element; inspect the target page and adjust the XPath. Urlwatch’s [filter documentation](https://urlwatch.readthedocs.io/en/latest/filters.html) describes available filters and their options.

3. Choose the right job type for each page

Job key	Use it when	Trade-off
url	The server response already contains the content to compare.	Usually the simplest and lightest option; JavaScript-rendered content may not appear.
navigate	The page needs a browser to execute JavaScript before the relevant content exists.	Uses a headless browser and is more resource-intensive. Install the optional browser dependencies described by Urlwatch.
command	A shell command produces the text or data you want to track.	You manage the command’s dependencies, execution time, and output.

Use url first when it provides the content you need. Switch only the affected job to navigate if an ordinary fetch cannot see the rendered content. Each job still gets its own separator and name. The job types are documented in the [Urlwatch jobs reference](https://urlwatch.readthedocs.io/en/stable/jobs.html).

4. Validate, list, and run the jobs

1. Save the jobs file from urlwatch --edit.
2. List configured jobs to check their names and positions: urlwatch --list.
3. Run a check manually: urlwatch.
4. Review the output and refine filters if irrelevant changes create noisy diffs.

The first run establishes the comparison baseline. Later runs report differences against the previous stored result. For predictable reports, make sure the same Urlwatch installation and data directory run each time.

5. Schedule recurring checks

Urlwatch does not schedule itself from the job YAML. The interval is controlled by how often an external scheduler invokes it. For cron, edit the crontab with crontab -e. This example runs every 30 minutes:

*/30 * * * * /usr/bin/urlwatch

Use the actual path to the Urlwatch executable in your environment; find it with command -v urlwatch. The official introduction recommends not running more often than every 30 minutes. If your environment uses systemd timers, a container scheduler, or a hosted job runner, configure that system to invoke the same command at your chosen interval.

6. Configure notifications separately

Job definitions belong in urls.yaml. Reporter and global settings belong in urlwatch.yaml, edited with:

urlwatch --edit-config

Urlwatch supports console output and configurable reporter integrations, including email and chat or push services. The setup varies by reporter, so follow the relevant [configuration documentation](https://urlwatch.readthedocs.io/en/stable/configuration.html). Keep credentials private; avoid committing tokens or passwords to a shared repository.

7. Handle duplicate URLs and preserve history

Monitor one URL with different filters

Urlwatch identifies URL jobs by their URL. If you need two jobs for the same page with different filters, give their URLs unique fragments:

name: "Find pricing text"
url: "https://example.com/pricing#pricing"
filter:
  - grep: "Plan"
---
name: "Find support text"
url: "https://example.com/pricing#support"
filter:
  - grep: "Support"

The fragments distinguish the job identities in Urlwatch; they are not a way to request two different server pages. This behavior and the identity rules are described in [Advanced Topics](https://urlwatch.readthedocs.io/en/latest/advanced.html).

Change a URL without losing its stored history

Changing a job’s URL normally creates a new identity with no prior history. If you are moving a job and want to retain its history, use --change-location with the old and new URL:

urlwatch --change-location 'https://old.example.com/page' 'https://new.example.com/page'

Check the command’s help and your installed Urlwatch version if the syntax differs. The [advanced guide](https://urlwatch.readthedocs.io/en/latest/advanced.html) documents this command.

Configuration options to consider

Start with the required job type and add only settings needed for the target. URL jobs support options for request behavior, such as headers, cookies, method, timeout, proxies, and cache handling. Browser and shell jobs have their own settings. Global defaults can be applied by job category in urlwatch.yaml, and reporters are also configured there. Consult the [jobs reference](https://urlwatch.readthedocs.io/en/stable/jobs.html) and [configuration reference](https://urlwatch.readthedocs.io/en/stable/configuration.html) for the supported options in your installed release.

• Filters: Select meaningful content and normalize irrelevant variation before comparison.
• HTTP behavior: Provide required headers or cookies for pages that legitimately need them. Do not disable TLS verification as a general fix.
• Timeouts and retries: Adjust request behavior for slow origins where supported; avoid setting excessive retries that multiply the load.
• Defaults: Use category-specific defaults for settings shared across many jobs, while keeping exceptions on individual jobs.
• Reporters: Configure delivery separately from the watched URLs, and test the destination with a manual run.

Reliability, performance, and cost

A run’s work grows with the number of jobs and the time each origin takes to respond. Ordinary HTTP jobs are generally lighter than browser-rendered navigate jobs. Keep the schedule aligned with how quickly the source can meaningfully change, and avoid monitoring more frequently than necessary. Urlwatch itself is software you run; practical costs come from the machine or runner, network use, and any external notification services you configure.

Reliability depends on repeated runs using the same persistent Urlwatch data, valid credentials where required, and successful delivery through the configured reporter. A one-off run that reports no changes is not proof that a target will always be reachable. For important monitoring, review execution failures as well as change reports, and keep backups of the configuration and state if you need history across machine replacement.

Troubleshooting

Symptom	Likely cause	Fix
YAML parse error	Bad indentation, a missing quote, or malformed separator.	Use spaces consistently; make --- a line by itself; reopen with urlwatch --edit to run its validation.
Editor does not open	EDITOR or VISUAL is unset or points to an unavailable command.	Set it to an installed editor, such as export EDITOR=nano, then retry.
Only one page is checked	The next job is not separated correctly, or the apparent separator is indented or has extra characters.	Put a plain --- between complete job mappings and verify with urlwatch --list.
Duplicate job or identity conflict	The same URL is defined more than once.	Use distinct URLs, or append unique fragments when monitoring one URL with different filters.
JavaScript content is missing	The server response does not include content rendered in the browser.	Use the navigate job type and install its optional dependencies, or monitor a suitable underlying endpoint if available.
Too many irrelevant changes	The whole page is being compared, including dynamic or unrelated regions.	Filter to the relevant section and normalize the output; check the filter order and XPath against the current page structure.
History appears to have reset	The job URL changed, so Urlwatch treats it as a new identity.	Use urlwatch --change-location OLD NEW when intentionally moving a job and retaining history.
No notification arrives	The run may be using console output, reporter configuration may be incomplete, or the destination may reject the message.	Inspect the manual run output and urlwatch.yaml; verify reporter credentials and destination settings.
Scheduled run behaves differently from a shell run	Cron often has a limited PATH, environment, or working directory.	Use the executable’s absolute path and ensure the scheduled process can access the same Urlwatch configuration and state.

Or skip the browser setup

Urlwatch is a good fit for tracking text changes. If the job is to capture a rendered page as an image or PDF, [ScreenshotNeo](https://screenshotneo.com) provides a website screenshot API and MCP server for developers. It can accept cookie and consent banners and remove more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and response headers report the page verdict and billing status. AI agents can use its MCP server tools, including take_screenshot, get_page_info, and capture_pdf.

One GET request returns an image or PDF. See the [ScreenshotNeo API documentation](https://screenshotneo.com/docs/) for the available options.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

ScreenshotNeo includes full-page and element capture, device and viewport choices, retina scale, PDF options, custom CSS and JavaScript, wait conditions, request blocking, headers and cookies, caching, signed public image links, asynchronous jobs, bulk capture, and more. Every feature is on every plan. The free plan includes 1,000 screenshots a month without a card; paid plans start at $5 for 3,000 screenshots. [Create a free ScreenshotNeo account](https://screenshotneo.com/account/sign-up/) to get 1,000 screenshots a month with no card.

FAQ

Can I keep all monitored URLs in one file?

Yes. Put each job in urls.yaml, with --- between jobs.

Does Urlwatch schedule itself?

No. A scheduler such as cron invokes Urlwatch at the interval you configure.

Do I need a filter for every page?

No. Add filters when they help isolate meaningful changes or reduce noisy diffs.

Where do I configure email or chat notifications?

Use urlwatch --edit-config to edit reporter settings in urlwatch.yaml.