How to make Urlwatch ignore timestamps and rotating content

To make Urlwatch ignore timestamps and rotating content, add a filter to the job’s YAML configuration so Urlwatch compares only the content that matters. Use CSS or XPath to select stable content or exclude a changing element, re.sub to normalize a timestamp embedded in useful text, and sorting when a list’s order is irrelevant. Preview the filtered result with urlwatch --test-filter before relying on it.

Urlwatch filters transform retrieved content before comparison, and you can chain filters. The right approach depends on where the changing content appears and whether its changes should still trigger an alert. See the Urlwatch handbook and filter reference for the version installed in your environment; the current handbook and filter reference in the research dossier identify as version 2.29.

1. Find where the changing content appears

Inspect the page and decide which of these cases applies:

It is in a separate HTML element: select the stable region with CSS or XPath, or select a containing region and exclude the changing element.
It is embedded in useful text: use a specific regular-expression substitution to remove or standardize just the volatile part.
It is an item in a reordered list: sort the selected items, but only if their order has no meaning.

Prefer structural selection for HTML or XML when the changing part has a distinct element. A broad text substitution can accidentally hide real changes; sorting can hide meaningful order changes.

2. Edit the job’s YAML filter

Urlwatch jobs are configured in a YAML job list, commonly edited with urlwatch --edit. Add a filter chain to the relevant job. The examples below are templates: replace selectors and patterns with ones that match the page you actually monitor.

Select stable content and exclude dynamic elements

This example selects a stable content region, excludes a timestamp and rotating banner within it, converts the result to text, and strips surrounding whitespace:

url: https://example.org/status
filter:
  - css:
      selector: "main .stable-content"
      exclude: ".last-updated, .rotating-banner"
  - html2text
  - strip

You can also select only the stable elements directly. CSS and XPath filters are available for HTML, and the documentation also describes XML mode. Selectors depend on the page’s markup, so check that the chosen region still contains all content you want alerts about.

Remove a timestamp embedded in text

When a timestamp is part of otherwise useful text, apply re.sub to a pattern tailored to the site’s actual format. For example, this illustrative pattern removes a line beginning with a four-digit-year date:

url: https://example.org/status
filter:
  - html2text
  - re.sub:
      pattern: "Last updated: \\d{4}-\\d{2}-\\d{2}.*"
      repl: ""
  - strip

The expression is only an example; it has not been tested against a particular page. Match the timestamp’s real format and, where possible, anchor the expression to nearby stable text. If the format includes a time zone, seconds, or a different date order, adapt the expression. Avoid a pattern that can consume adjacent meaningful content.

Ignore order changes in a list

If the same items appear in different orders and order is irrelevant, sort the selected elements:

url: https://example.org/status
filter:
  - css:
      selector: "ul.items > li"
      sort: true
  - html2text

For plain text, the sort filter sorts lines by default and accepts a custom separator for other item boundaries. Use it only when item order is genuinely unimportant: sorting will suppress alerts when the order itself changes.

Choose selector options carefully

The selector filters also support excluding matched elements and limiting results with skip and maxitems. Use maxitems only when selecting one among duplicate matches is correct. If the page adds a meaningful match later, a limit may hide it from monitoring.

3. Preview the result before depending on it

Test the filter and inspect its plaintext output:

urlwatch --test-filter 0

Replace 0 with the job index you intend to test. The documented filter-test command can also be used with a URL. Confirm that the result contains the content that should trigger alerts and omits only the noise. For the exact syntax accepted by your installed version, check urlwatch --help and that version’s documentation; the research dossier’s details for this command refer to the official 2.20 filter reference.

A filter change affects newly retrieved content. Stored old content was processed using the earlier filter, so the first comparison after changing a filter may compare content treated differently on each side. Review that initial difference before treating it as a meaningful page change.

4. Troubleshoot noisy or missing alerts

Symptom	Likely cause	What to check
The timestamp still causes alerts	The selector does not include the timestamp element, or the substitution does not match its actual format.	Inspect the page markup and filtered plaintext. Adjust the selector or pattern to match the real timestamp and its surrounding text.
Meaningful text disappeared	The selector is too broad, the regular expression consumes too much text, or `maxitems` excludes later matches.	Reduce the selected region, narrow the expression, or remove the item limit. Check the preview for content that must remain monitored.
Reordering still causes alerts	The filter is not sorting the selected items, or the items are not separated as expected.	Apply sorting within the CSS or XPath filter for selected elements, or configure a suitable separator when sorting plain text.
A reorder no longer alerts, but it should	Sorting suppresses order differences.	Remove sorting if sequence is meaningful, such as a ranked list or ordered procedure.
The first run after an edit reports a large difference	The saved old snapshot may reflect the previous filter while new content uses the updated filter.	Inspect the filtered output and account for the different processing when reviewing the first comparison.
The filter preview does not match the page you expected	The job index or URL may not identify the job you intended, or the selector may no longer match the current markup.	Check the job configuration and preview the correct job or URL; verify the selector against the current page structure.

5. Keep the filter reliable and focused

Prefer stable structure: CSS or XPath selection and exclusion make the filtering boundary easier to review when a page has distinct elements.
Keep substitutions narrow: remove only the known volatile string so that nearby content changes remain visible.
Preserve meaningful order: do not sort ranked, chronological, or otherwise ordered items unless the sequence itself is irrelevant.
Recheck after page redesigns: selectors depend on markup and may stop matching when the site changes its structure.
Preview after every filter edit: validate the content that remains, not just the absence of the noisy part.

These filters determine what content is compared. They do not establish whether the site’s content is correct or whether the page will keep the same markup.

Or skip the browser setup

If the task is to capture a clean website screenshot rather than monitor text changes, ScreenshotNeo returns a screenshot or PDF from one GET request. The [ScreenshotNeo API documentation](https://screenshotneo.com/docs/) lists the available parameters.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.org/status -o shot.webp

ScreenshotNeo removes cookie and consent banners, newsletter popups, and chat widgets before the capture; each cleanup step can be turned off. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers say which page verdict and billing status applied. Its MCP server gives AI agents tools to take screenshots, get page information, and capture PDFs. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000.

FAQ

Can I ignore several changing elements in one job?

Yes. Exclude multiple elements in a selector, or chain filters to handle different kinds of volatility. Preview the combined output to ensure it retains all monitored content.

Should I use CSS, XPath, or a regular expression?

Use CSS or XPath when the changing content occupies a separable HTML or XML element. Use re.sub when volatility is embedded in text you still need to monitor.

Will sorting ignore changes to the list’s contents?

No. Sorting addresses order differences; additions, removals, or changed item text should remain visible if the selected items and filter chain preserve them.

Are the sample selectors and regular expression ready for my site?

No. They illustrate filter shapes only. Inspect the target page, adapt the selector or pattern, and verify the output with the filter test command before relying on the job.