Privacy and safety design
scrape-diagnose assumes that target URLs and API keys may contain secrets even when a user did not intend to expose them.
Run a free diagnosis
No account or API key is needed for a direct check. This pinned command tests a harmless public example:
npx --yes github:sanjayamaharjancodes/scrape-diagnose#abe305a38cee37a0287d5bb9d1097964759b8c72 -- https://example.com/ --expect-text "Example Domain"Data flow
Direct mode performs a DNS preflight, rejects any private or special-use result it observes, and sends one read-only request to the target. Only when that request returns 403, it sends one browser-header comparison to distinguish a free header fix from a proxy need. Each profile follows at most five redirects, preflighting every new destination before requesting it. The classifier reads at most 1 MiB of response bytes, then discards them.
Live mode sends the target URL and selected provider parameters to ScraperAPI. Before each paid request it calls the provider’s cost endpoint. It sends the paid request only if the estimate fits both the remaining total-credit budget and request-count ceiling.
Report allowlist
Reports contain:
- tool/schema version and mode
- target scheme and hostname only
- boolean configuration flags
- coarse HTTP/content classifications
- bounded byte and visible-text counts
- stable finding codes
- integer estimated and observed costs
- environment-variable-based snippets after success
Reports never contain the API key, target path/query/fragment, marker value, response body, redirect destination, raw header, provider account field, exception text, timing, request ID, or local absolute path.
Network boundaries
Literal and preflight-resolved loopback, private, link-local, multicast, documentation, metadata-service, .local, .internal, and localhost targets are rejected. Redirects are handled manually and revalidated. DNS can change between lookup and connection, so this is a defense-in-depth preflight rather than a guarantee against rebinding. Do not use pull_request_target or accept arbitrary target inputs from untrusted contributors.
Credential handling
The CLI reads only SCRAPERAPI_KEY; it does not search the filesystem or load .env. Command-line key options are rejected. The Action masks its key before parsing other inputs. Network and provider errors are mapped to stable generic codes instead of printing thrown errors or request URLs.
Explicit non-goals
No login, authenticated target, paywall, interactive CAPTCHA solving, local browser automation, POST, mutation, bulk URL input, response export, extraction, scheduling, retry storm, telemetry, referral tracking, or automatic redirect is supported by the CLI or Action. Provider-side redirect following is explicitly disabled on cost and paid-probe requests. If a user explicitly selects provider rendering, ScraperAPI may use remote browser infrastructure under its own terms.
The public README and Python guide contain static, clearly disclosed affiliate links. The executable never prints or opens them. Vendor attribution begins only when a reader deliberately follows one of those links; no user or session identifier is added by this project.