Getting Started with Web Scraping in C#
Learn a responsible C# scraping workflow: fetch pages with HttpClient, parse HTML with AngleSharp, and use Playwright only when browser execution is needed.
Blog
Guides for capturing the web
How to take clean screenshots, turn HTML into images and PDFs, run headless browsers, and give AI agents eyes. Written by the team that builds ScreenshotNeo.
How to Capture Background Requests and Responses in Puppeteer
Capture fetch and XHR traffic in Puppeteer with page events, wait for a specific response, and troubleshoot failures without unnecessary interception.
How to Get Your Headlines Back into X/Twitter Link Previews
Learn why X link cards lose headlines, how to fix missing previews, and what to do when headline display is controlled by X.
How to Build an LLM Interface for Your Website
Build a website chat interface with a server-side LLM endpoint, streamed replies, practical security controls, and clear privacy choices.
How to Scrape beIN Sports Pages
A compliance-first guide to collecting public beIN Sports metadata with robots.txt, Python, cURL and Node.js—plus safer page capture options.
How to Build Reusable Element Libraries for Design Editors
Build a maintainable design-editor library with foundations, components, variants, documentation, publishing workflows, governance, and practical examples.
How to Find a Website’s Hosting Provider
Learn how to identify a website’s DNS provider, registrar, CDN and likely origin host with dig, WHOIS, IP lookups and practical checks.
Fit vs. Fill: Which Image Scaling Mode Should You Use?
Use fit when every pixel matters and fill when the frame matters. Learn CSS contain, cover and fill, responsive patterns, cropping control and API workflows.
How to Build a Custom MCP Client
Build a production-ready MCP client: choose transports, negotiate protocol eras, discover tools, route model calls, secure data, and clean up reliably.
How to Turn a Web Scraper into an RSS Feed
Convert scraped pages into a valid RSS 2.0 feed with stable IDs, safe XML, validation, scheduling, and reliable publishing.
Remote Browser Benchmarks: Comparing Performance and Reliability
Learn how to benchmark remote browsers fairly with lifecycle latency, failure rates, concurrency tests, and reproducible code.
How to Pass Data Between Scrapy Callbacks
Pass spider-owned values between Scrapy callbacks with cb_kwargs. Learn when to use meta or spider.state, how to handle errbacks and JOBDIR, and how to troubleshoot common mistakes.
Bulk Screenshot Export: A Complete Guide to Capturing Many URLs
Learn how to bulk export consistent screenshots with Playwright, deterministic filenames, full-page capture, batching, retries, and a hosted API option.
Axios Set Headers: The Complete Guide for 2026
Set Axios headers per request, per client, or with an interceptor. Learn precedence, CORS, FormData, token safety, and common fixes.
Making Concurrent Requests in C#
Use Task.WhenAll for a small batch of HTTP calls and Parallel.ForEachAsync for a collection with bounded concurrency. Learn how to reuse HttpClient and handle failures safely.
How to Build a No-Code Web Scraper in Make
Build a Make scenario that requests a public page, checks its HTML, extracts fields, and routes the results—with guidance for JavaScript-rendered sites.
How to Scrape Pages Behind a Login with Session Cookies
Use an authorized login flow, save Playwright state securely, reuse it for protected pages, and handle cookies, storage, expiry, and errors.
Playwright vs. Puppeteer: Which Is Better?
Choose Playwright for cross-browser testing, more language options, or a built-in test runner. Choose Puppeteer when its Node.js workflow and browser coverage fit your project.