How to Generate Dynamic Videos with an API

Short answer: dynamic video generation is a render pipeline. Your app validates a versioned JSON payload, makes every source asset publicly reachable, chooses a template or code composition, submits a render job, waits for a webhook or status change, validates the finished file, and copies it to durable storage. A hosted JSON API is usually fastest to ship; a React renderer such as Remotion gives you control over scene logic and the render runtime.

1. Model the pipeline before writing code

Keep request creation separate from rendering. A typical job moves through these states:

1. Prepare: validate data, assets, locale and output constraints.
2. Submit: send a template merge or a complete scene description with an idempotency key.
3. Render: the provider downloads assets and produces the media asynchronously.
4. Notify: receive a webhook, or poll a status endpoint when no callback is available.
5. Deliver: validate the artifact, copy it to durable storage, then publish your URL.

Store the provider job ID, your payload hash, schema version and output location. That record lets retries be safe and makes a failed render reproducible.

2. Define a versioned data contract

Use JSON-serializable values so the same contract works with hosted APIs and Remotion input props.

{
  "schemaVersion": 1,
  "id": "order-1842",
  "locale": "en-US",
  "title": "Your weekly report",
  "firstName": "Mina",
  "price": "$49",
  "media": {
    "hero": "https://cdn.example.com/assets/hero-v3.jpg",
    "voiceover": "https://cdn.example.com/audio/order-1842.mp3"
  },
  "scenes": [
    {"kind": "intro", "seconds": 3},
    {"kind": "product", "seconds": 7, "image": "https://cdn.example.com/assets/item-7.png"},
    {"kind": "outro", "seconds": 3}
  ],
  "output": {"width": 1920, "height": 1080, "fps": 30}
}

• Pin immutable asset URLs (for example, a content hash or version suffix).
• Validate MIME type, dimensions and duration before submission.
• Keep optional fields explicit and define what happens when a scene is absent.
• Include a locale and a safe-area policy if text will be reused on multiple platforms.

3. Choose a rendering model

Model	Use it when	Trade-off
Hosted JSON timeline (Shotstack)	Scenes follow repeatable tracks and non-developers need reusable templates.	Less infrastructure; you work within the service’s edit model.
Hosted template or RenderScript (Creatomate)	You need template substitutions, or scene count and order vary per record.	Fast delivery; template and script behavior are provider-specific.
React renderer (Remotion)	You need loops, conditional scenes, custom animation or full code ownership.	You operate a rendering runtime or Lambda deployment.

Shotstack describes a JSON and REST editing service: arrange an edit, POST it, then receive a file location after rendering. Its templates support merge fields such as {{ FIRST_NAME }} and callbacks. Creatomate accepts a template ID with text and media modifications, or a direct RenderScript for completely dynamic scenes, and supports success and failure webhooks. Remotion renders React components from frame numbers; a composition declares width, height, FPS and duration, and its CLI and server-side/Lambda APIs accept JSON props.

4. Hosted JSON render: request, callback and retry

Keep the provider URL in configuration because endpoints and API versions differ by account. The following payload shape is the important part: a deterministic edit, public assets and a callback URL.

{
  "timeline": {
    "soundtrack": "https://cdn.example.com/audio/order-1842.mp3",
    "tracks": [
      {"clips": [{"asset": {"type": "image", "src": "https://cdn.example.com/assets/hero-v3.jpg"}, "start": 0, "length": 3}]},
      {"clips": [{"asset": {"type": "title", "text": "Your weekly report"}, "start": 0, "length": 3}]}
    ]
  },
  "callback": "https://app.example.com/webhooks/video-render",
  "output": {"format": "mp4", "width": 1920, "height": 1080, "fps": 30}
}

cURL

export RENDER_API_URL='https://your-provider.example/render'
export RENDER_API_KEY='replace-me'
curl -sS -X POST "$RENDER_API_URL" \
  -H "Authorization: Bearer $RENDER_API_KEY" \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: order-1842-v1' \
  --data @render.json

Python

import json, os, requests

payload = json.load(open("render.json"))
r = requests.post(
    os.environ["RENDER_API_URL"],
    headers={
        "Authorization": f"Bearer {os.environ['RENDER_API_KEY']}",
        "Idempotency-Key": "order-1842-v1",
    },
    json=payload,
    timeout=30,
)
r.raise_for_status()
job = r.json()
print(job["id"])

Node.js

import fs from 'node:fs/promises';
const payload = JSON.parse(await fs.readFile('render.json', 'utf8'));
const res = await fetch(process.env.RENDER_API_URL, {
  method: 'POST',
  headers: {
    authorization: `Bearer ${process.env.RENDER_API_KEY}`,
    'content-type': 'application/json',
    'idempotency-key': 'order-1842-v1'
  },
  body: JSON.stringify(payload)
});
if (!res.ok) throw new Error(`${res.status}: ${await res.text()}`);
console.log((await res.json()).id);

Use the provider’s documented status endpoint to poll only when a webhook is unavailable. Exponential backoff (for example, 2, 4, 8, 16 seconds with a cap) prevents a polling storm. Verify webhook signatures if the provider supplies them, reject duplicate event IDs, and return a fast 2xx response before doing long processing.

5. Template merges and fully dynamic scenes

For a fixed layout, keep the composition in a template and send only substitutions such as FIRST_NAME, price and media URLs. For variable scene order, generate a RenderScript or timeline from your validated scenes array. Do not let user text become executable code; treat it as data and escape it at the template boundary.

6. Code-first rendering with Remotion

A minimal composition receives JSON props and maps scene durations to frame ranges.

import React from 'react';
import {AbsoluteFill, Sequence, interpolate, useCurrentFrame} from 'remotion';

export const DynamicVideo = ({title, scenes, fps = 30}) => {
  let offset = 0;
  return <AbsoluteFill style={{background: '#111', color: 'white', fontFamily: 'sans-serif'}}>
    {scenes.map((scene, i) => {
      const start = offset;
      const duration = Math.max(1, Math.round(scene.seconds * fps));
      offset += duration;
      return <Sequence key={i} from={start} durationInFrames={duration}>
        <Scene title={scene.kind === 'intro' ? title : scene.kind} />
      </Sequence>;
    })}
  </AbsoluteFill>;
};

const Scene = ({title}) => {
  const frame = useCurrentFrame();
  const opacity = interpolate(frame, [0, 15], [0, 1], {extrapolateRight: 'clamp'});
  return <AbsoluteFill style={{justifyContent: 'center', alignItems: 'center', opacity}}>{title}</AbsoluteFill>;
};

Declare the composition’s width, height, FPS and duration in your Remotion entry point. Pass the contract as input props to the CLI or server/Lambda render API. Keep rendering deterministic: avoid current time, random values and mutable remote data inside a frame.

7. Asset, audio and layout edge cases

• Private assets: issue short-lived signed URLs that remain valid for the entire preprocessing window.
• Redirects and hotlink protection: test the final URL from the renderer’s network, not only from your browser.
• Missing media: fail validation or choose an explicit fallback; never silently render a blank frame.
• Audio drift: make the soundtrack and scene timeline share one time base, then check the final duration.
• Long text: wrap, truncate with intent, or switch to a compact layout. Reserve safe areas for platform controls.
• Aspect ratios: render separate 16:9, 1:1 and 9:16 compositions when cropping would remove important content.
• Fonts: package exact font files in the render environment so line breaks do not change between machines.

8. Validate every output before delivery

• Confirm duration, dimensions, frame rate, container and codec.
• Confirm an audio stream exists when one is required and that captions are present when promised.
• Inspect the first and last frames for black frames, clipped text and unsafe crops.
• Copy the result to durable storage if the provider’s output URL can expire, and record its checksum.

9. Reliability, scaling and cost controls

• Queue jobs and cap concurrency to the provider quota and your own CPU, memory and bandwidth limits.
• Use idempotency keys and payload hashes so a timeout does not create duplicate renders.
• Retry transient network errors and provider 5xx responses with backoff; do not retry schema or asset validation failures unchanged.
• Measure queue wait, render duration, download time, failure reason and output size per composition.
• Batch only when the API supports it and when one failed record can be isolated.
• Estimate cost from scenes, duration, resolution and retry rate using the current plan documentation; the available source material does not establish comparable prices, quotas or latency benchmarks.

10. Troubleshooting

Symptom	Likely cause	Fix
Job fails before rendering	Malformed JSON, unsupported field or invalid duration.	Validate against the provider schema and log the exact payload version.
Image or video is missing	Asset is private, expired, blocked or has an unsupported MIME type.	Use a stable public or signed URL, verify headers and pin the asset.
Webhook never arrives	Callback URL is not publicly reachable, TLS fails, or response is non-2xx.	Expose HTTPS, allow provider IPs if required, return 2xx quickly and poll as a fallback.
Duplicate videos	Client retried after a submission timeout.	Reuse the same idempotency key and deduplicate webhook event IDs.
Text shifts between renders	Different fonts, locale or renderer versions.	Bundle fonts, pin versions and set locale explicitly.
Audio is cut off	Scene duration and soundtrack length disagree.	Choose a trim or pad policy and validate the final duration.
Renderer runs out of memory	Large images, long compositions or excessive concurrency.	Resize inputs, split long jobs and lower concurrency.

11. Or skip the browser setup

When a video scene needs a clean still of a live web page, ScreenshotNeo returns PNG, JPEG, WebP or PDF from one GET request. Cookie and consent banners, newsletter popups and chat widgets are removed before capture; bot checks, blank pages and failed loads are never billed. Its MCP server lets Claude, Cursor and other MCP clients take screenshots, inspect pages and capture PDFs. You get 1,000 screenshots a month free with no card; paid plans start at $5 for 3,000.

See the ScreenshotNeo API docs for options such as full-page or selector capture, dark mode, device and retina settings, custom CSS/JavaScript, waits, blocking rules, headers, cookies, geolocation, caching and signed webhooks.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Use the returned image as a deterministic visual input to your video job, and inspect the X-Page-Verdict and X-Billed headers when deciding whether to enqueue it. Create a free ScreenshotNeo account with 1,000 shots a month and no card.

12. FAQ

Should I use a template API or React?

Choose templates when layouts repeat and editors need to change them. Choose React when scene logic, animation or branching is the product.

How do I make retries safe?

Persist a payload hash and reuse an idempotency key for the same logical job; deduplicate callback event IDs.

Can the renderer fetch assets from my localhost?

No. Hosted renderers need publicly reachable HTTPS URLs or provider-supported uploads.

When should I poll?

Prefer a webhook. Poll with backoff only as a fallback, and stop after a bounded deadline.

How do I support multiple social formats?

Declare dimensions and safe areas in the contract, then render dedicated compositions for each aspect ratio.