ScreenshotNeo

BlogHow-to

How to Save PhantomJS Webpages with Dynamic Data

Wait for JavaScript data in PhantomJS, then save a reliable PNG, JPEG, or PDF without capturing an incomplete page.

By the ScreenshotNeo team30 September 20267 min read

How to Save PhantomJS Webpages with Dynamic Data

Direct answer: open the page with PhantomJS, verify that page.open returns success, wait for a page-specific signal that the asynchronous data is present, and only then call page.render(). A load callback alone does not guarantee that timers or XHR requests have finished.

PhantomJS has JavaScript enabled by default. Configure settings such as resourceTimeout before the initial page.open; changing them afterward does not affect that navigation. The render filename extension selects the output format, including PNG, JPEG, PDF and other formats supported by the Qt build. See the official open, render, settings and automation references.

1. A complete PhantomJS script

Replace #results with an element that is populated only after the data you need has arrived. The script polls that element, stops after a bounded timeout, and renders only when the condition is met.

The capture sequence: navigate, wait for application data, then render the file.
The capture sequence: navigate, wait for application data, then render the file.
var page = require('webpage').create();
var system = require('system');

var url = system.args[1] || 'https://example.com/dashboard';
var output = system.args[2] || 'capture.png';
var readinessSelector = '#results';
var maxWaitMs = 30000;
var pollEveryMs = 250;
var startedAt;

page.settings.javascriptEnabled = true;
page.settings.resourceTimeout = 10000;
page.viewportSize = { width: 1440, height: 900 };

function fail(message) {
  console.error(message);
  phantom.exit(1);
}

function waitForData(done) {
  startedAt = Date.now();

  function poll() {
    var state = page.evaluate(function (selector) {
      var node = document.querySelector(selector);
      if (!node) return { present: false, hasData: false };
      var text = (node.textContent || '').replace(/\\s+/g, ' ').trim();
      var loading = node.getAttribute('aria-busy') === 'true' ||
                    /loading|pending/i.test(node.className || '');
      return { present: true, hasData: text.length > 0, loading: loading };
    }, readinessSelector);

    if (state.present && state.hasData && !state.loading) {
      done(true);
      return;
    }

    if (Date.now() - startedAt >= maxWaitMs) {
      done(false);
      return;
    }

    setTimeout(poll, pollEveryMs);
  }

  poll();
}

page.open(url, function (status) {
  if (status !== 'success') {
    fail('Unable to load ' + url + ' (status: ' + status + ')');
    return;
  }

  waitForData(function (ready) {
    if (!ready) {
      fail('Timed out waiting for ' + readinessSelector);
      return;
    }

    if (!page.render(output)) {
      fail('Could not render ' + output);
      return;
    }

    console.log('Saved ' + output);
    phantom.exit(0);
  });
});

Run it with:

phantomjs save-dynamic.js https://example.com/dashboard dashboard.png
phantomjs save-dynamic.js https://example.com/report report.pdf

2. Choose the right readiness condition

Dynamic pages differ, so the readiness test must describe the data you actually need.

Page behavior Condition to check Why it is useful
Results appear in a known container Selector exists and has non-empty text Simple and easy to inspect
Loading indicator is removed Spinner is absent or aria-busy="false" Works when the application marks completion
Application sets a state flag window.appReady === true More precise than text matching
Table rows are loaded document.querySelectorAll('table tbody tr').length > 0 Ensures actual records exist
Several requests must finish Check all required selectors or a page-specific completion marker Avoids rendering after only the first request

Use page.evaluate to inspect DOM state inside the page. Keep the wait bounded: a missing selector, failed API request or changed markup should produce a clear failure instead of an indefinitely running PhantomJS process.

Waiting for a JavaScript flag

function waitForAppReady(done) {
  var deadline = Date.now() + 30000;

  function poll() {
    var ready = page.evaluate(function () {
      return window.appReady === true;
    });

    if (ready) return done(true);
    if (Date.now() >= deadline) return done(false);
    setTimeout(poll, 250);
  }

  poll();
}

Using a fixed delay as a fallback

setTimeout(function () {
  page.render('delayed.png');
  phantom.exit(0);
}, 5000);

A delay can help with pages that expose no reliable signal, but it is less robust: fast pages waste time and slow pages may still be incomplete. Prefer a semantic condition and retain a maximum timeout.

3. Save an image, PDF, or a selected region

The file extension passed to page.render determines the format. Use a viewport for the visible browser area and clipRect to render a specific rectangle.

page.viewportSize = { width: 1280, height: 800 };
page.clipRect = { top: 100, left: 0, width: 1280, height: 600 };
page.render('content.png');

For a full-page capture, determine the document dimensions after the page is ready, then set the viewport accordingly:

var size = page.evaluate(function () {
  return {
    width: Math.max(document.body.scrollWidth, document.documentElement.scrollWidth),
    height: Math.max(document.body.scrollHeight, document.documentElement.scrollHeight)
  };
});
page.viewportSize = { width: size.width, height: size.height };
page.render('full-page.png');

PDF output is useful for print-oriented documents, but pagination and CSS support depend on PhantomJS’s legacy rendering engine. Inspect the resulting file rather than assuming screen CSS and print CSS will match.

4. Configuration options that affect dynamic captures

Option When to set it Practical use
javascriptEnabled Before page.open Keep enabled for application-rendered data; it is enabled by default.
resourceTimeout Before navigation Bound a stalled resource. It does not prove that required data loaded.
viewportSize Before rendering, or after measuring Choose desktop or narrow viewport dimensions.
clipRect Before rendering Capture one region instead of the whole viewport.
page.settings.userAgent Before navigation Use only when the target explicitly serves different markup by user agent.

Settings apply during the initial navigation. Set them before calling page.open, and create a new page when a later URL needs materially different settings.

5. Handle failures instead of saving incomplete files

The page.open callback reports success or fail. Treat fail as a failed capture. Log the URL, status and timeout reason in automation so a downstream job cannot mistake a partial file for valid output.

page.onResourceError = function (error) {
  console.error('Resource error: ' + error.url + ' - ' + error.errorString);
};

page.onResourceTimeout = function (request) {
  console.error('Resource timeout: ' + request.url);
};

6. Common errors and fixes

Symptom Likely cause Fix
Blank page or missing values Rendering occurred before asynchronous data arrived Check status, keep JavaScript enabled, and poll a data-bearing selector or application flag.
Process hangs Readiness selector never appears or a request stalls Use a maximum wait, set resourceTimeout before opening, and exit with a non-zero status.
Capture is cropped Viewport or clip rectangle is smaller than the required area Set viewportSize explicitly or measure document dimensions before rendering.
Expected element is absent Wrong selector, authentication redirect or changed page markup Inspect the DOM with page.evaluate, verify the final URL and handle login separately.
Modern page renders incorrectly PhantomJS is legacy software with suspended development Confirm the page’s browser requirements and consider maintained browser automation when newer features are required.
Script exits before injected code runs phantom.exit() was called before an includeJs callback Place the exit and render logic inside the callback, as described in the PhantomJS automation guide.

7. Reliability and performance practices

  • Use a page-specific readiness signal instead of a large arbitrary sleep.
  • Bound both resource waits and application waits.
  • Record the requested URL, final status, readiness outcome and output path.
  • Retry only transient navigation failures; repeated readiness failures usually indicate a selector, authentication or compatibility problem.
  • Reuse a PhantomJS process only when page isolation is understood; stale cookies and JavaScript state can affect later captures.
  • Keep viewport dimensions and output formats consistent when files are compared or processed downstream.

There is no universal PhantomJS timing or compatibility guarantee for modern sites. The upstream project states that development is suspended, and its repository is archived read-only as of May 30, 2023; the README identifies 2.1 as the latest stable release. Treat support for newer browser APIs as a compatibility risk rather than assuming every dynamic site will work.

8. Cost and operational considerations

Running PhantomJS yourself means managing the executable, scripts, page timeouts, output storage and failures. The main operational cost is the browser process and the time spent waiting for dynamic requests. A bounded readiness strategy prevents a small number of broken pages from consuming workers indefinitely. If you need large batches, add a queue, concurrency limit and retention policy for generated files.

9. Or skip the browser setup

ScreenshotNeo provides a single GET request for a PNG, JPEG, WebP or PDF. Its capture workflow accepts cookie and consent banners as a visitor, then removes more than 60 known consent platforms, newsletter popups and chat widgets before the shot; each step can be turned off. Bot checks, blank pages, timeouts, failed loads and cache hits are not billed, and response headers report the page verdict and billing result.

ScreenshotNeo removes common consent and overlay elements before capture.
ScreenshotNeo removes common consent and overlay elements before capture.

See the ScreenshotNeo API documentation for all options. This is a complete cURL example:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python:

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

Node.js:

const fs = require('node:fs/promises');
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot failed: ${res.status}`);
await fs.writeFile('shot.webp', Buffer.from(await res.arrayBuffer()));

ScreenshotNeo also includes an MCP server with take_screenshot, get_page_info and capture_pdf tools for Claude, Cursor and other MCP clients. The Free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Create a free ScreenshotNeo account.

10. FAQ

Does page.open wait for AJAX data?

It reports page-load completion and success or failure. Application code can continue updating the DOM afterward, so wait for the data-specific condition you need.

Can PhantomJS save PDFs?

Yes. Pass a filename ending in .pdf to page.render, subject to the formats supported by the Qt build.

Why is a fixed sleep unreliable?

Network and application time vary. A fixed delay can be too short for slow responses and unnecessarily long for fast ones; a bounded semantic poll is usually safer.

Is PhantomJS suitable for every modern website?

No guarantee exists. Its upstream development is suspended and the repository is archived, so pages requiring newer browser features may need maintained automation.

How do I capture only one component?

Measure the element’s rectangle in page.evaluate, assign that rectangle to page.clipRect, and render after the element’s data is ready.