How to Access Secured Pages in C#

Direct answer: use the authentication scheme required by the service. For a cookie-protected website, keep one CookieContainer and reuse the same HttpClientHandler for login and subsequent requests. For a bearer-protected API, obtain an OAuth/OIDC access token through the provider’s supported flow and send it as Authorization: Bearer <token>. A 401 means authentication is missing or invalid; a 403 means authentication succeeded but the caller lacks permission.

Do not try to bypass MFA, CSRF protection, consent screens, bot checks, certificate validation, or authorization policies. Reproduce the service’s documented protocol.

1. Identify the authentication scheme first

Inspect the service documentation and, when a request fails, the WWW-Authenticate response header. Common schemes are:

Scheme	Use it for	C# client state
Cookie session	Websites that log a user in and issue a session cookie	A persistent CookieContainer
Bearer token	Web APIs protected by OAuth 2.0 or OIDC	An access-token cache and refresh strategy
Basic	Legacy or explicitly documented services	HTTPS plus carefully protected credentials
Windows/Negotiate	Integrated corporate environments	Deployment-specific handler and credentials

For delegated interactive access, use authorization code with PKCE. For an unattended service with no user, use client credentials. Microsoft describes these flow choices in its authorization-code guidance and client-credentials guidance.

2. Access a cookie-protected page with HttpClient

A browser login normally ends by setting an authentication cookie. A non-browser client must retain that cookie and send it on the page request. ASP.NET Core’s documentation explains that after a successful cookie login, the authentication cookie is automatically sent with the request and the endpoint is authorized.

using System.Net;
using System.Net.Http;
using System.Collections.Generic;

var cookies = new CookieContainer();
using var handler = new HttpClientHandler
{
    CookieContainer = cookies,
    UseCookies = true,
    AllowAutoRedirect = true
};

using var client = new HttpClient(handler)
{
    BaseAddress = new Uri("https://example.com")
};

var loginValues = new Dictionary<string, string>
{
    ["username"] = Environment.GetEnvironmentVariable("SITE_USER")
        ?? throw new InvalidOperationException("SITE_USER is missing"),
    ["password"] = Environment.GetEnvironmentVariable("SITE_PASSWORD")
        ?? throw new InvalidOperationException("SITE_PASSWORD is missing")
};

using var login = await client.PostAsync(
    "/login",
    new FormUrlEncodedContent(loginValues));
login.EnsureSuccessStatusCode();

using var page = await client.GetAsync("/secure/page");
page.EnsureSuccessStatusCode();
var html = await page.Content.ReadAsStringAsync();
Console.WriteLine(html.Length);

Adapt the URL and field names to the site’s documented login endpoint. Keep the same handler and client for both calls. Creating a new handler loses the in-memory session cookie.

When the login form requires an antiforgery token

Many server-rendered forms require a token in addition to the username and password. Fetch the login page first, read the token from the HTML, then submit the token and credentials through the same cookie-enabled client. The token format and field name are site-specific; use the site’s supported integration rather than guessing.

using System.Net;
using System.Net.Http;
using System.Text.RegularExpressions;

var cookies = new CookieContainer();
using var handler = new HttpClientHandler { CookieContainer = cookies, UseCookies = true };
using var client = new HttpClient(handler) { BaseAddress = new Uri("https://example.com") };

var formHtml = await client.GetStringAsync("/login");
var match = Regex.Match(
    formHtml,
    "name=\\\"__RequestVerificationToken\\\"[^>]*value=\\\"([^\\\"]+)\\\"",
    RegexOptions.IgnoreCase);
if (!match.Success)
    throw new InvalidOperationException("The login page did not contain the expected antiforgery token.");

var values = new Dictionary<string, string>
{
    ["__RequestVerificationToken"] = match.Groups[1].Value,
    ["username"] = Environment.GetEnvironmentVariable("SITE_USER")!,
    ["password"] = Environment.GetEnvironmentVariable("SITE_PASSWORD")!
};

using var login = await client.PostAsync("/login", new FormUrlEncodedContent(values));
login.EnsureSuccessStatusCode();
var secureHtml = await client.GetStringAsync("/secure/page");

For production code, use an HTML parser suited to the page instead of a regular expression when markup can vary. Some applications place the token in a header or require a second cookie.

Redirects and login detection

A redirect to /login usually means the request was not authenticated. Temporarily disable automatic redirects to inspect the original response:

using var handler = new HttpClientHandler
{
    CookieContainer = new CookieContainer(),
    UseCookies = true,
    AllowAutoRedirect = false
};
using var client = new HttpClient(handler);

using var response = await client.GetAsync("https://example.com/secure/page");
Console.WriteLine((int)response.StatusCode);
Console.WriteLine(response.Headers.Location);

3. Call a bearer-token API

Acquire an access token using the API provider’s official library or flow, then put the access token in the authorization header. Microsoft identity platform examples use the same AuthenticationHeaderValue pattern.

using System.Net.Http.Headers;

using var client = new HttpClient();
var accessToken = Environment.GetEnvironmentVariable("API_ACCESS_TOKEN")
    ?? throw new InvalidOperationException("API_ACCESS_TOKEN is missing");

client.DefaultRequestHeaders.Authorization =
    new AuthenticationHeaderValue("Bearer", accessToken);

using var response = await client.GetAsync("https://api.example.com/secure-resource");
var body = await response.Content.ReadAsStringAsync();
if (!response.IsSuccessStatusCode)
    throw new HttpRequestException($"API returned {(int)response.StatusCode}: {body}");

Console.WriteLine(body);

Use an access token, not an ID token. Treat tokens as secrets: do not log them, commit them, put them in URLs, or ship confidential client secrets in desktop or browser-distributed binaries.

Token acquisition choices

• Authorization code plus PKCE: a user signs in interactively and grants delegated access. Use a provider-supported library such as MSAL.NET for token acquisition, caching, and refresh.
• Client credentials: a daemon or backend acts as itself, with no user. The API authorizes the application identity and its scopes or roles.
• On-behalf-of: a middle-tier API exchanges a validated user token for a downstream API token when the identity provider supports it.

The API validates the token and its claims. The client should not attempt to decide whether a token is valid by inspecting its payload.

4. Basic and Windows authentication

Only use these schemes when the server explicitly advertises and requires them. Always use HTTPS.

using System.Net.Http.Headers;
using System.Text;

using var client = new HttpClient();
var raw = Encoding.UTF8.GetBytes($"{userName}:{password}");
client.DefaultRequestHeaders.Authorization =
    new AuthenticationHeaderValue("Basic", Convert.ToBase64String(raw));

using var response = await client.GetAsync("https://example.com/secure-resource");
response.EnsureSuccessStatusCode();

For Windows/Negotiate authentication, configure the handler with the credentials required by your deployment and network policy. Do not enable credential forwarding to arbitrary hosts.

5. Diagnose 401, 403, redirects, and browser differences

Symptom	Likely cause	Fix
401 Unauthorized	Missing, expired, malformed, or wrong-audience credential	Check the scheme, token issuer, audience, scopes, expiry, and WWW-Authenticate challenge.
403 Forbidden	Identity is valid but lacks a role, scope, policy, or resource permission	Request the required permission or use an account authorized for that resource.
302 to login	No valid cookie session, lost cookie jar, or an interactive login is required	Reuse the same handler, inspect the redirect, and complete the documented login/OIDC flow.
Works in a browser only	Browser supplied cookies, CSRF tokens, redirects, headers, MFA, or JavaScript-generated state	Compare the supported protocol and reproduce each required step. Do not scrape around controls.
Token accepted by one endpoint but rejected by another	Wrong audience or scope	Request a token for the target API and its required scopes.
Certificate or TLS error	Untrusted or mismatched certificate	Fix the server certificate or trusted root. Never disable validation in production.

Log safely while troubleshooting

• Record status code, request host and path, elapsed time, redirect location, and response correlation IDs.
• Redact cookies, authorization headers, passwords, refresh tokens, and full response bodies when they may contain secrets.
• Check server time when tokens appear prematurely expired.
• Confirm that a proxy has not stripped the Authorization header.

6. Reliability, performance, and cost considerations

• Reuse clients: use IHttpClientFactory in ASP.NET Core or a long-lived client in a worker to avoid socket exhaustion and repeated connection setup.
• Set bounded timeouts: use a cancellation token and an explicit timeout appropriate to the service. Retry only transient network failures and selected 5xx responses; do not blindly retry 401 or 403.
• Cache tokens: use the identity library’s cache and refresh path. Requesting a new token for every call adds latency and can trigger provider throttling.
• Preserve cookies per session: do not share a user’s cookie container across users or unrelated jobs.
• Use least privilege: request only the scopes and roles required by the operation.
• Measure the full exchange: login, token acquisition, redirects, DNS, TLS, server processing, and response transfer can each dominate latency.

7. Or skip the browser setup

If your goal is a screenshot or PDF of an authenticated page, ScreenshotNeo accepts custom headers and cookies so you can send the session or authorization data with the capture request. It also supports user-agent, timezone, geolocation, JavaScript, custom CSS, waits, request blocking, full-page capture, element capture, PDF output, async jobs, and bulk capture. See the ScreenshotNeo API documentation for request options.

For a public page, the one-call shape is:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

ScreenshotNeo removes cookie banners, newsletter popups, and chat widgets before the shot. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers identify the page verdict and billing result. Its MCP server lets Claude, Cursor, and other MCP clients call take_screenshot, get_page_info, and capture_pdf. The Free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Create a free ScreenshotNeo account.

8. Security checklist

• Use HTTPS and validate certificates.
• Keep credentials in a secret store or environment supplied by your deployment.
• Never log authorization headers, passwords, cookies, or refresh tokens.
• Use authorization code with PKCE for interactive delegated access.
• Use client credentials only for app-only services.
• Rotate secrets and revoke sessions when a credential may be exposed.
• Honor MFA, CSRF, consent, rate limits, and service terms.

FAQ

Can I send a browser’s cookie string manually?

You can add cookies to a CookieContainer when the service permits it, but a session may also depend on CSRF tokens, device state, expiry, or MFA. Prefer the documented login flow.

Should I use an ID token to call an API?

No. APIs generally require an access token issued for that API and its scopes. ID tokens describe the sign-in event for the client application.

Why does a valid token still return 403?

The token proves identity, but the account may lack the endpoint’s scope, role, tenant membership, resource permission, or policy approval.

Is a login redirect proof that authentication failed?

It is a strong signal that the server did not recognize the session. Inspect the redirect and cookies, because some applications redirect for consent or other interactive steps.