How to Build a Website Monitoring Script in Bash
Build a Bash and curl website monitor with timeouts, status and content checks, logs, cron scheduling, and missed-job alerts.
Short answer: build a Bash script that sends a bounded curl request, checks the final HTTP status and an optional stable content marker, records the timestamp, duration and reason, then exits with status 0 for healthy and nonzero for failure. Run it from cron or a system timer. Add a heartbeat ping only after a successful run so you can detect when the scheduled job did not execute.
A single request from one machine cannot prove that a website is available everywhere. DNS, routing, TLS, firewalls and regional outages can affect the result. Treat this script as one monitoring vantage point and choose health criteria that match the application.
1. Decide what “healthy” means
Before writing code, define the acceptance policy for this site:
- Target: the URL or named endpoint to check.
- Status: for example, any final
2xxresponse, or one exact status such as200. - Redirects: whether redirects are expected and whether the final host is acceptable.
- Content: an optional stable marker such as a page heading, build identifier or health text.
- Timing: a connection timeout and a total transfer timeout.
- Retries: whether a transient network error should be retried before alerting.
curl does not normally treat an HTTP 4xx or 5xx response as a command failure. Use --fail when every 4xx/5xx should fail, or capture the response code with --write-out and apply your own policy. See the curl manual and curl HTTP scripting guide.
2. A complete Bash monitor
This script follows redirects, limits connection and total time, captures the response body, checks for a 2xx final response, optionally checks a content marker, logs evidence, and returns a useful exit code.
#!/usr/bin/env bash
set -o pipefail
# Usage: ./website-check.sh [URL]
URL="${1:-https://example.com/}"
LOG_FILE="${LOG_FILE:-./website-check.log}"
# Leave empty to disable the content check.
EXPECTED_MARKER="${EXPECTED_MARKER:-}"
CONNECT_TIMEOUT="${CONNECT_TIMEOUT:-5}"
MAX_TIME="${MAX_TIME:-15}"
MAX_REDIRECTS="${MAX_REDIRECTS:-5}"
fail() {
local reason="$1"
printf '%s FAIL target=%s reason=%s\\n' "$(date -u +%FT%TZ)" "$URL" "$reason" | tee -a "$LOG_FILE" >&2
exit 1
}
command -v curl >/dev/null 2>&1 || fail "curl_missing"
command -v date >/dev/null 2>&1 || fail "date_missing"
[[ -n "$URL" ]] || fail "url_missing"
body_file="$(mktemp)" || fail "mktemp_failed"
trap 'rm -f "$body_file"' EXIT
start_ns="$(date +%s%N)"
status="$(curl --silent --show-error --location \\
--max-redirs "$MAX_REDIRECTS" \\
--connect-timeout "$CONNECT_TIMEOUT" \\
--max-time "$MAX_TIME" \\
--output "$body_file" \\
--write-out '%{response_code}' \\
"$URL")"
curl_rc=$?
end_ns="$(date +%s%N)"
# Nanoseconds are available on GNU date. Fall back to zero if arithmetic data is unavailable.
if [[ "$start_ns" =~ ^[0-9]+$ && "$end_ns" =~ ^[0-9]+$ ]]; then
duration_ms=$(( (end_ns - start_ns) / 1000000 ))
else
duration_ms=0
fi
if (( curl_rc != 0 )); then
printf '%s FAIL target=%s curl_rc=%s duration_ms=%s\\n' \\
"$(date -u +%FT%TZ)" "$URL" "$curl_rc" "$duration_ms" | tee -a "$LOG_FILE" >&2
exit 1
fi
if [[ ! "$status" =~ ^2[0-9][0-9]$ ]]; then
printf '%s FAIL target=%s status=%s duration_ms=%s reason=http_status\\n' \\
"$(date -u +%FT%TZ)" "$URL" "$status" "$duration_ms" | tee -a "$LOG_FILE" >&2
exit 1
fi
if [[ -n "$EXPECTED_MARKER" ]] && ! grep -Fq -- "$EXPECTED_MARKER" "$body_file"; then
printf '%s FAIL target=%s status=%s duration_ms=%s reason=marker_missing\\n' \\
"$(date -u +%FT%TZ)" "$URL" "$status" "$duration_ms" | tee -a "$LOG_FILE" >&2
exit 1
fi
printf '%s OK target=%s status=%s duration_ms=%s\\n' \\
"$(date -u +%FT%TZ)" "$URL" "$status" "$duration_ms" | tee -a "$LOG_FILE"
exit 0Make it executable and run it:
chmod +x website-check.sh
./website-check.sh https://example.com/
EXPECTED_MARKER='Example Domain' ./website-check.sh https://example.com/The log contains a UTC timestamp, target, status or curl exit code, duration and reason. Avoid putting access tokens, session cookies or other secrets in the URL or log.
3. Status checks, redirects and content checks
Accepting status codes
The example accepts every final 2xx response. To require exactly 200, replace the regular-expression test with [[ "$status" == 200 ]]. If your endpoint intentionally returns 204, include it in the policy.
Following redirects
--location follows redirects and --max-redirs prevents an endless chain. If redirects should be considered unhealthy, remove --location and inspect the first response. If the final destination must remain on your domain, capture headers or use a separate redirect policy rather than assuming every redirect is safe.
Checking application content
A successful TCP connection or HTTP status does not prove that the expected application is working. Choose text that changes rarely: a dedicated health endpoint response, a stable heading or a deployment marker. Do not match rotating timestamps, advertisements or personalized text. For JSON, use a parser such as jq and validate a field explicitly.
4. Add intentional retries
Retries can reduce alerts caused by transient failures, but they also delay detection and still depend on the network path. Keep the number small and use a delay. This wrapper retries the complete check twice:
#!/usr/bin/env bash
set -o pipefail
attempts=3
for attempt in $(seq 1 "$attempts"); do
./website-check.sh "https://example.com/" && exit 0
[[ "$attempt" -lt "$attempts" ]] && sleep 2
done
exit 1Keep the overall retry window below the scheduler interval. If a check can take 15 seconds and you run it every minute, three attempts plus delays may overlap with the next run.
5. Schedule the check with cron
Edit the crontab for the account that owns the script:
crontab -eThis runs every five minutes and appends stdout and stderr to a local scheduler log:
*/5 * * * * /opt/monitor/website-check.sh https://example.com/ >> /opt/monitor/cron.log 2>&1Use absolute paths because cron has a minimal environment. Ensure the script, log directory and any tools such as jq are available to that user. Keep the interval longer than the worst-case request and retry duration. A systemd timer is another option when you need explicit service dependencies and journal logging.
6. Detect a missed cron run with a heartbeat
A local log cannot alert you when the host itself is offline. A heartbeat service can alert when an expected success ping is missing. Send the ping only after every check succeeds, and configure grace time above the normal run duration. Healthchecks.io documents this pattern for cron jobs and shell scripts.
#!/usr/bin/env bash
set -o pipefail
HEARTBEAT_URL="https://hc-ping.com/your-check-uuid"
if ./website-check.sh "https://example.com/"; then
curl --silent --show-error --max-time 10 --retry 2 "$HEARTBEAT_URL" >/dev/null
heartbeat_rc=$?
if (( heartbeat_rc != 0 )); then
printf '%s FAIL reason=heartbeat_failed rc=%s\\n' "$(date -u +%FT%TZ)" "$heartbeat_rc" >&2
exit 1
fi
exit 0
fi
# Do not ping success when the website check failed.
exit 1Healthchecks.io also documents appending /fail or an exit status to signal failure. When a pipeline must fail if an earlier command fails, use set -o pipefail; Bash otherwise reports the status of the last command in the pipeline. Sending monitoring signals over the public internet is inherently unreliable, so treat the heartbeat as a second signal rather than proof of global availability. See the shell guide and reliability tips.
7. Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Script says OK for a 500 response | curl does not fail on HTTP status by default, or status was not checked. | Capture %{response_code} and enforce your accepted range, or use --fail for a simple 4xx/5xx policy. |
| curl exit code 28 | Connection or total transfer timeout. | Inspect latency and server behavior; adjust --connect-timeout and --max-time deliberately. |
| curl exit code 6 or 7 | DNS resolution or connection failure. | Check DNS, routing, firewall rules and the host from the monitoring machine. |
| Too many redirects | Redirect loop or unexpected canonicalization. | Inspect with curl -I -L, lower or raise --max-redirs only after confirming the intended chain. |
| Marker check fails while browser looks fine | Content differs by user agent, locale, authentication or JavaScript rendering. | Use a stable health endpoint, add required headers or cookies, or use a browser-based capture for rendered content. |
| Works interactively but not in cron | Relative paths, missing PATH, permissions or environment variables. |
Use absolute paths, set required variables in the script, and redirect cron output to a file. |
| Heartbeat never alerts | Ping runs even after failure, or grace period is too long. | Send success only after checks pass and set grace above expected run time. |
| Duplicate alerts | Retries or overlapping cron jobs. | Bound retries, lengthen the interval, or add a lock so only one run is active. |
8. Reliability and security checklist
- Use HTTPS and validate the certificate; do not disable TLS verification to hide an error.
- Set both connection and total timeouts.
- Choose redirect behavior explicitly.
- Check a stable application marker when status alone is insufficient.
- Keep secrets out of command arguments, URLs and logs.
- Use a lock or equivalent if a slow run could overlap the next schedule.
- Rotate or limit log files so a long-running monitor cannot fill the disk.
- Run checks from more than one network location when regional availability matters.
- Record enough context to investigate: UTC time, target, status, duration and failure reason.
9. When Bash is enough—and when to use hosted monitoring
Bash plus cron is a good fit for a small project, one monitoring location, simple HTTP checks and local logs. Hosted monitoring becomes useful when you need external alert delivery, history retention, multiple locations, TLS reporting, dashboards or protection from the monitored host being unavailable. Compare services by vantage points, check frequency, HTTP and content-check support, alert destinations, retention, setup effort and cost.
10. Or skip the browser setup
If your monitoring workflow also needs a rendered screenshot, browser waiting, consent handling or a PDF, ScreenshotNeo provides a website screenshot API and MCP server. It accepts one GET request and returns PNG, JPEG, WebP or PDF. Cookie and consent banners, newsletter popups and chat widgets are removed before capture; bot checks, blank pages, failed loads, timeouts and cache hits are not billed, and response headers identify the page verdict and billing result. Its MCP server exposes take_screenshot, get_page_info and capture_pdf for Claude, Cursor and other MCP clients.
See the ScreenshotNeo API documentation for all options, including wait conditions, custom headers, cookies, JavaScript, CSS selectors, device presets, caching, async jobs and bulk capture.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpimport requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);There are 1,000 screenshots per month free with no card. Paid plans start at $5 for 3,000 screenshots. Create a free ScreenshotNeo account.
FAQ
Does a 200 response prove the site is healthy?
No. It proves that this request received a successful HTTP response. Add a stable content or JSON-field check and monitor from additional locations when needed.
Should I use curl --fail?
Use it when any 4xx or 5xx should fail. Capture and inspect the status yourself when your application accepts a wider or narrower range.
How long should the timeout be?
Set it from the endpoint’s expected behavior and your alerting needs. Keep connection timeout short and total timeout bounded; do not choose a value so large that failures overlap scheduled runs.
Can cron tell me that the server running cron is down?
No. A remote heartbeat can report a missing success ping, while a direct check from another location measures the website.
Why use a browser screenshot service for monitoring?
Use one when the signal depends on rendered JavaScript, visual layout, consent cleanup, element capture or a PDF. A plain curl check remains simpler for status and text endpoints.
