How to export Urlwatch results and keep a history of page changes

To export a Urlwatch run, redirect its standard output to a file: urlwatch > report.txt. To inspect the cached history for one monitored job, use urlwatch --dump-history JOB. To keep the comparison history Urlwatch uses between runs, preserve its cache database, normally $XDG_CACHE_HOME/urlwatch/cache.db. These are three related but different things: a report from one run, one job’s historical cached data, and the database that stores state for diffing.

1. Save the report from a run

Urlwatch prints report information to standard output by default. Redirect it to create a text artifact for that invocation:

urlwatch > report.txt

To append each run’s output to the same file, use the shell’s append operator:

urlwatch >> report.log

Appending creates a sequence of run reports. It does not create a structured archive of every cached snapshot or replace Urlwatch’s cache database. Standard-output redirection is also distinct from Urlwatch’s configured reporters: the reporter system can send notifications through configured channels, and reporter-specific dependencies or configuration may be required.

Choose what to save

One run’s report: use > to replace a file or >> to append.
A job’s cached history: use --dump-history and optionally redirect its output.
Ongoing comparison state: retain the configured cache database.

2. Dump one job’s historical data

Use the job identifier accepted by your installed Urlwatch version:

urlwatch --dump-history JOB

To save the command’s output separately:

urlwatch --dump-history JOB > job-history.txt

This is a per-job history command. The CLI reference describes it as dumping historical cached data for a job; it does not define the output as a portable, whole-database export format. Check the output and job-selection syntax for your installed version before building an archival workflow around it.

3. Preserve the cache database

Urlwatch’s cache database stores job state history used for diffing. The documented default is $XDG_CACHE_HOME/urlwatch/cache.db. If you run Urlwatch with a custom cache location, the --cache FILE option identifies the file to preserve.

# Default cache location (when XDG_CACHE_HOME is set)
$XDG_CACHE_HOME/urlwatch/cache.db

# Run with a specific cache file
urlwatch --cache /path/to/urlwatch-cache.db

Keep the actual configured database with your backups. A redirected report is useful as a record of output, but it does not retain the underlying state history Urlwatch uses to compare runs. The reviewed CLI documentation does not specify a complete procedure for safely copying a live cache, restoring it across versions, or exporting all records to a portable interchange format. For database-level archiving, validate the procedure against your installed version and treat it as a SQLite workflow outside the documented history-dump command.

4. Avoid losing history when a monitored location changes

Urlwatch keys a job’s history using its url value. Editing the URL directly in urls.yaml creates a new job without the prior history. When changing a monitored location, use the documented migration command instead:

urlwatch --change-location https://old.example/page https://new.example/page

The command is also relevant to browser and shell jobs: it changes their navigate and command values, respectively. Use the exact old and new location values from your job configuration.

5. Check cache cleanup before relying on long history

The --gc-cache RETAIN_LIMIT option removes older cache entries while retaining the requested latest number. The documented default retain limit is one. If you need multiple historical entries, inspect the cleanup settings and command you use before running garbage collection.

# Example: retain the latest 10 cache entries
urlwatch --gc-cache 10

Confirm the option’s behavior in the documentation for your installed version before applying cleanup. Garbage collection can remove the older entries you intended to keep.

6. Select a report destination

Urlwatch’s reporter system supports standard output and configured notification reporters. The documented reporter list includes options such as email, shell, Slack, Telegram, Matrix, and Discord. Availability depends on the reporter configuration and, for some reporters, additional dependencies. Consult the current reporter documentation before choosing a channel.

The configuration guide also covers combining stdout with HTML email and choosing whether reports are combined or separated per job. A configured reporter determines where notifications go; shell redirection only saves standard output from the command invocation.

7. Troubleshoot export and history problems

Symptom	Likely cause	What to do
The report file is empty or lacks notifications	You redirected stdout, but the configured reporter sends output elsewhere, or that run had no report content.	Check the configured reporters and run output. Redirecting stdout does not capture messages sent directly through another reporter.
`--dump-history` does not find the job	The job identifier or selection syntax does not match what this Urlwatch version accepts.	Check the installed version’s command reference and use the supported job location or index.
A job has no history after its URL changed	Editing the URL directly created a new job identity.	For future moves, use `urlwatch --change-location OLD NEW`. The new job may not contain the old job’s history if it was already created separately.
Older entries are missing	Cache cleanup may have removed them; the documented default retain limit for `--gc-cache` is one.	Review cleanup commands and retain limits before running them. Previously removed entries cannot be recovered from the current cache alone.
Urlwatch appears to use a different cache	A custom cache path may have been supplied with `--cache FILE`.	Preserve and inspect the configured file rather than assuming the default path.
A notification reporter fails to send	The reporter may need configuration, credentials, or a reporter-specific dependency.	Check that reporter’s current documentation and dependency requirements; use stdout redirection when you need a local run report.

8. Reliability, performance, and storage considerations

Keep both artifacts when needed: save run reports for a readable record and retain the cache for Urlwatch’s comparison state.
Use append deliberately: >> accumulates output in one growing text file; rotate or archive it according to your own storage needs.
Back up the configured cache: identify whether you use the default location or --cache FILE. The documentation reviewed does not specify live-copy safety or cross-version restore guarantees.
Protect history from cleanup: review --gc-cache and the retain limit before executing it.
Preserve identity during migrations: use --change-location when the monitored location changes so existing history is retained.

These steps describe file and cache handling; the reviewed sources provide no benchmark or storage-growth estimate. The practical cost is the local storage used by any report files and the cache you retain.

9. ScreenshotNeo for visual snapshots

Urlwatch is designed to monitor pages and report detected changes. If your workflow also needs a rendered image or PDF snapshot of a URL, ScreenshotNeo is a website screenshot API and MCP server for developers. A screenshot is a visual artifact, so it complements report and cache history rather than replacing Urlwatch’s comparison database.

Or skip the browser setup

Make one GET request to capture a page. See the ScreenshotNeo API documentation for options.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Cookie banners are accepted and removed before capture; ScreenshotNeo also removes known consent platforms, newsletter popups, and chat widgets. Each step can be turned off.
Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed. Responses identify the page verdict and billing status in headers.
An MCP server lets AI agents use take_screenshot, get_page_info, and capture_pdf.
The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots.

10. FAQ

Does a report file contain all page-change history?

No. It contains the standard output captured from that invocation. Preserve the cache database for Urlwatch’s comparison state.

Can I export every cached job in one documented command?

The documented --dump-history command is for a selected job. The reviewed reference does not promise a whole-cache export format.

Will changing a page URL preserve its history?

Use --change-location OLD NEW. Editing the URL directly creates a new job without the old history.

Which Urlwatch versions do these commands cover?

The research references Urlwatch 2.29. Verify commands, defaults, reporters, and storage details against your installed version.