ScreenshotNeo

BlogHow-to

Guide to Python’s requests POST Method

Learn how to send form data, JSON, files, and raw bodies with requests.post, then handle responses, timeouts, and errors safely.

By the ScreenshotNeo team30 September 202610 min read

Guide to Python’s requests POST Method

requests.post() sends an HTTP POST request and returns a Response. Choose json=payload for a JSON request body, data=fields for form data, and files=... for multipart uploads. Set a timeout, check the HTTP status with raise_for_status(), and parse the response according to the endpoint’s contract.

Install Requests if it is not already in your environment:

python -m pip install requests

Here is a complete JSON example. Replace the example URL and fields with the endpoint and payload your application requires.

import requests

url = "https://api.example.test/items"
payload = {"name": "Ada", "active": True}

try:
    response = requests.post(
        url,
        json=payload,
        timeout=(3.05, 20),
    )
    response.raise_for_status()
except requests.exceptions.Timeout as exc:
    print(f"The request timed out: {exc}")
except requests.exceptions.ConnectionError as exc:
    print(f"Could not connect: {exc}")
except requests.exceptions.HTTPError as exc:
    print(f"The server returned an HTTP error: {exc}")
else:
    # Only call .json() if this endpoint returns JSON.
    result = response.json()
    print(result)

The timeout values above are examples, not universal defaults. Choose them based on how quickly the service should accept a connection and how long it may take to return data.

1. Choose the right request body

The main choice is what the server expects in the body. Requests supports form fields, JSON, multipart uploads, and raw data. Match the endpoint’s documentation; sending valid data in the wrong encoding commonly produces a 400 or 415 response.

Choose the body encoding the endpoint expects: form fields, JSON, or multipart file data.
Choose the body encoding the endpoint expects: form fields, JSON, or multipart file data.
Endpoint expects Use Typical use
URL-encoded form fields data=dict HTML-style forms and many legacy APIs
JSON object or array json=payload Most JSON APIs
Multipart form with files files=..., optionally data=... File upload endpoints
Raw text or bytes data=body Custom protocols or pre-encoded content

Send form-encoded fields

When data is a dictionary, Requests encodes the fields as form data. The values are represented as strings on the wire, so API-specific boolean, date, and number conventions still matter.

import requests

response = requests.post(
    "https://api.example.test/submit",
    data={"name": "Ada", "active": "true"},
    timeout=(3.05, 20),
)
response.raise_for_status()

If a form allows repeated names, pass a list of pairs. A normal dictionary cannot represent duplicate keys.

response = requests.post(
    "https://api.example.test/form",
    data=[("tag", "python"), ("tag", "http")],
    timeout=(3.05, 20),
)
response.raise_for_status()

Send JSON

Use json= with a Python dictionary, list, or other JSON-serializable value. Requests serializes it and sets the JSON content type. This is usually safer than serializing the object yourself.

payload = {"name": "Ada", "roles": ["reader", "writer"]}

response = requests.post(
    "https://api.example.test/items",
    json=payload,
    timeout=(3.05, 20),
)
response.raise_for_status()
created_item = response.json()

A manually serialized string passed as data= does not automatically get Content-Type: application/json. If you have a special reason to serialize manually, set the header yourself. Do not combine json= with data= or files=: the JSON argument is ignored when either is supplied.

Send raw text or bytes

For a raw body, pass a string or byte sequence to data and set the media type the server expects. The body is not automatically interpreted as JSON just because its contents look like JSON.

body = b"custom binary payload"

response = requests.post(
    "https://api.example.test/ingest",
    data=body,
    headers={"Content-Type": "application/octet-stream"},
    timeout=(3.05, 30),
)
response.raise_for_status()

2. Upload files with multipart encoding

Use files= when the endpoint expects a multipart form upload. Open files in binary mode and keep the file open until the request finishes. Requests constructs multipart data for ordinary uploads; very large multipart requests are not streamed by default, so memory use can become a concern.

import requests

with open("report.csv", "rb") as file_obj:
    response = requests.post(
        "https://api.example.test/upload",
        files={"file": file_obj},
        data={"description": "Monthly report"},
        timeout=(3.05, 60),
    )
response.raise_for_status()

The field name file must match the endpoint’s expected multipart field. Some APIs also require a filename, MIME type, or additional metadata; consult that API’s contract and supply those fields in the format it documents.

3. Set timeouts so calls do not wait forever

Requests has no timeout by default. Its timeout controls how long it waits for socket data; it is not a total deadline for downloading the complete response. A server that continues sending data can therefore take longer than the read timeout duration overall. The Requests Quickstart says nearly all production code should use the timeout parameter.

A single number applies to the connect and read timeout. A tuple gives each phase a separate limit:

# One value for both connect and read waits
requests.post(url, json=payload, timeout=10)

# Separate connect and read limits
requests.post(url, json=payload, timeout=(3.05, 20))

Use a shorter connect timeout when an unreachable service should fail quickly. Use a longer read timeout for operations that legitimately take time, such as processing an upload. If the application needs a strict end-to-end deadline, enforce that at the application or task level as well; the Requests socket timeout alone is not a total wall-clock limit.

4. Check the response before interpreting it

Getting a Response object does not mean the operation succeeded. A server can return an error status with a perfectly valid JSON body. Call raise_for_status() to raise HTTPError for unsuccessful HTTP status codes, or explicitly check the status codes the endpoint documents.

response = requests.post(url, json=payload, timeout=(3.05, 20))
response.raise_for_status()

if response.status_code == 204:
    # No response body is expected.
    result = None
else:
    result = response.json()

Do not call response.json() unconditionally. An endpoint might return an empty body, plain text, or a different format. Also, successful JSON parsing is not proof of HTTP success: inspect the status independently.

When diagnosing an API error, inspect the status and, if safe, the body before deciding how to handle it:

response = requests.post(url, json=payload, timeout=(3.05, 20))

print(response.status_code)
print(response.headers.get("Content-Type"))
print(response.text[:1000])
response.raise_for_status()

Be careful not to log credentials, personal information, or full request bodies in production logs.

5. Reuse a Session for repeated requests

A Session persists cookies and uses connection pooling, which is useful when making multiple calls to the same service. It can also hold shared request configuration. Use a context manager when the group of requests has a clear lifetime.

import requests

with requests.Session() as session:
    session.headers.update({"Accept": "application/json"})
    first = session.post(
        "https://api.example.test/login",
        json={"user": "ada"},
        timeout=(3.05, 20),
    )
    first.raise_for_status()

    second = session.post(
        "https://api.example.test/items",
        json={"name": "Notebook"},
        timeout=(3.05, 20),
    )
    second.raise_for_status()

Use the same session when you want cookies from one response to be available to later calls. For concurrent work, do not assume that sharing mutable session state between threads is safe; use a session strategy that matches your concurrency model.

6. Add authentication and request headers

Follow the service’s documented authentication scheme. For a bearer token, set the Authorization header; keep secrets in environment variables or a secrets manager rather than committing them to source control.

import os
import requests

api_token = os.environ["API_TOKEN"]
response = requests.post(
    "https://api.example.test/items",
    json={"name": "Ada"},
    headers={
        "Authorization": f"Bearer {api_token}",
        "Accept": "application/json",
    },
    timeout=(3.05, 20),
)
response.raise_for_status()

Some APIs use HTTP Basic authentication or a custom header instead. Do not guess the scheme: use the endpoint documentation. If the API supports idempotency keys for writes, use the documented mechanism when retrying could create duplicate records.

7. cURL, Python, and Node.js equivalents

These examples send JSON to a placeholder endpoint. Replace the URL and request fields, and use the success conditions and response format documented by your service.

cURL

curl --request POST "https://api.example.test/items" \
  --header "Content-Type: application/json" \
  --data '{"name":"Ada","active":true}'

Python

import requests

response = requests.post(
    "https://api.example.test/items",
    json={"name": "Ada", "active": True},
    timeout=(3.05, 20),
)
response.raise_for_status()
print(response.json())

Node.js

const response = await fetch('https://api.example.test/items', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ name: 'Ada', active: true }),
  signal: AbortSignal.timeout(20000),
});

if (!response.ok) {
  throw new Error(`HTTP ${response.status}: ${await response.text()}`);
}

const result = await response.json();
console.log(result);

8. Retry carefully and make writes reliable

Requests documents ConnectionError, Timeout, TooManyRedirects, and HTTPError as common exceptions within its RequestException hierarchy. A connect timeout is documented as safe to retry at the library level, but that does not make every POST safe to repeat. The server may have completed a write even if the client never received the response.

  • Retry only when the endpoint’s semantics make repetition safe, or when it documents an idempotency mechanism.
  • Use a bounded number of attempts with a delay strategy rather than an unbounded loop.
  • Do not retry validation errors or authentication failures unchanged; fix the request first.
  • Record a correlation or request identifier when the API provides one, so an uncertain result can be investigated.

For critical operations, decide how to reconcile an ambiguous timeout: query the operation by a client-generated identifier if supported, or use the endpoint’s idempotency feature. A timeout means the client did not receive data in time; it does not necessarily prove the server did nothing.

9. Troubleshooting common POST failures

Symptom Likely cause What to check
Request appears to hang No timeout was set, or the chosen read timeout is too long for the application Set explicit connect and read limits; remember they are socket waits, not a total deadline.
400 Bad Request Missing, malformed, or incorrectly encoded fields Compare the exact body shape and field names with the endpoint contract.
415 Unsupported Media Type The server expects a different content type Use json= for JSON, data= for form encoding, or multipart files= as required.
401 or 403 Missing, invalid, expired, or insufficient credentials Check the authentication scheme, token scope, and whether the header reaches the service.
JSONDecodeError Empty or non-JSON response, including some error pages Check status and content type before parsing; handle documented empty responses such as 204.
File upload rejected Wrong multipart field, file opened incorrectly, or metadata missing Use binary mode, verify the expected field name and file constraints, and include required form fields.
Duplicate operation after retry The server processed the first POST but the response was lost Use a supported idempotency key or query the operation before retrying.
TooManyRedirects The endpoint redirects repeatedly or cycles Inspect the endpoint URL and redirect behavior; use the canonical API URL if documented.

For network failures, verify DNS, proxy settings, TLS certificates, firewall rules, and the service hostname. Avoid disabling TLS verification as a routine fix; correct the certificate or trust configuration instead.

10. Performance, reliability, and cost considerations

For repeated requests, connection pooling through a Session can avoid repeatedly establishing connections. Keep response timeouts appropriate to the operation and reuse connections where the application’s lifecycle allows. Large uploads can consume substantial memory because Requests does not stream large multipart requests by default; consider the API’s supported upload approach when payloads are large.

Screenshot cleanup can remove common consent banners, newsletter popups, and chat widgets before capture.
Screenshot cleanup can remove common consent banners, newsletter popups, and chat widgets before capture.

Reliability depends on the endpoint’s semantics as much as on the client. A timeout may leave the outcome unknown, and retries can duplicate side effects. Use documented idempotency support, bounded retries, and clear status handling. Cost is determined by the service you call: Requests itself is an HTTP client library, but the remote API may charge per operation, data volume, or account plan. Check that service’s pricing and quotas before scaling traffic.

If your task is specifically to capture a web page as an image or PDF, that is a different job from sending arbitrary JSON to an API. You can use a browser automation stack yourself, or call a screenshot service. ScreenshotNeo is a website screenshot API and MCP server for developers; its documented screenshot endpoint takes a URL and returns an image or PDF. See the ScreenshotNeo API documentation for request options and response handling.

11. Or skip the browser setup

If your goal is a screenshot, make one GET request with the page URL:

cURL

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Cookie banners are accepted before capture, and known consent platforms, newsletter popups, and chat widgets can be removed. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed; response headers identify the page verdict and billing status. An MCP server gives AI agents screenshot, page-info, and PDF capture tools. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Sign up for 1,000 free screenshots a month, with no card required.

12. Frequently asked questions

How do I send JSON with requests.post?

Pass a Python object using json=payload. Requests serializes the body and sets the JSON content type. Use raise_for_status() before interpreting a successful response.

How do I send form data with requests.post?

Pass a dictionary or a list of key-value pairs using data=. Use a list of pairs when the form has repeated field names.

Why does requests.post hang?

Without a timeout, Requests can wait indefinitely. Set a timeout, optionally as a connect/read tuple, and remember it limits socket waits rather than total download time.

Can I use data= and json= together?

Do not rely on both. If data or files is supplied, Requests ignores json. Choose the body encoding the endpoint expects.

Does a successful response.json() mean the request succeeded?

No. JSON parsing only means the body could be decoded as JSON. Check the HTTP status separately with raise_for_status() or the endpoint’s documented success codes.

Which Requests version and Python versions are covered?

The Requests documentation in the research source is for version 2.34.2 and states official support for Python 3.10 and later. Check the current project documentation when choosing versions for a new deployment.