Skip to main content

Errors

URLpipe uses conventional HTTP status codes and a consistent JSON error shape, so failures are easy to detect and handle.

Status codes

Status
Meaning
200 OK
The request succeeded. Async requests return 200 with a token.
401 Unauthorized
Missing or invalid API key.
403 Forbidden
The key is valid, but the email address on the account hasn't been confirmed.
422 Unprocessable Entity
The operation failed or a parameter was invalid — see the error field.
429 Too Many Requests
Too many of your requests already running (concurrency_limit), sending too fast (rate_limited), or — Free plan only — out of credits (quota_exceeded). A paid plan is never refused for credits; it keeps serving at the overage rate.

Error shape

When an operation fails, the response is JSON with a single error field holding a human-readable message:

422 Unprocessable Entity
{
  "error": "The request timed out."
}

Three responses carry extra fields: the quota 429 (with limit, used, needed, resets_at), the parallel-requests 429 (with limit and running), and the invalid-parameter 422 shown below. Both 429s also carry next_plan and upgrade_url: the plan that lifts the limit and where to switch to it.

Authentication errors

A 401 Unauthorized means the Authorization header is missing or the token doesn't match an active project key; its code is invalid_api_key, and the message says which of the two it was. A 403 Forbidden with the code email_unverified means the key is fine but none of the organization's admins has confirmed their email address yet — resending the key or rotating it will not help; an admin clicking the link we emailed them will. See Authentication.

Validation errors

Bad parameters return a 422 with a machine-readable error code:

  • Name
    invalid_url
    Description
    The url is missing or not acceptable. It must be a valid http/https URL for a public domain — not an IP address, localhost, an internal hostname, or a custom (non-default) port. It must not carry credentials: https://user:pass@example.com is rejected.
  • Name
    invalid_max_age
    Description
    The max_age value couldn't be parsed. Use a number of seconds or a duration like "2 hours".
  • Name
    invalid_options
    Description
    A page option or screenshot option is out of range, of the wrong type or not one we know, and the message names which — "screenshot_options.viewport_width must be a whole number from 320 to 1920.", for example.
  • Name
    invalid_labels
    Description
    labels must be an object of up to 16 keys with string values, and the message names what to change — "labels.client must be a string.", for example.
  • Name
    invalid_auth_profile
    Description
    The auth_profile is not one of this project's, the URL is on none of its domains, or it was named on /lighthouse or on a scrape that includes lighthouse — and the message says which. POST /auth_profiles answers with the same code when the state uploaded is not one it can use.
  • Name
    invalid_idempotency_key
    Description
    The Idempotency-Key must be 1 to 255 printable ASCII characters, with no spaces.
  • Name
    idempotency_key_reused
    Description
    This Idempotency-Key came with a different request in the last 24 hours. Nothing ran: send a new key for a new request. See retries & duplicates.
  • Name
    report_to …
    Description
    A report_to was given that we will not deliver to, and the message names the reason — the same rules the URL being analyzed is held to, so an IP address, a localhost address, credentials in the URL or a custom port are all refused. Omitting it entirely is not an error: see where results go.

Common failure messages

Operational failures — network issues, unreachable or oversized pages — come back as a 422 with one of these messages:

Message
Cause
The request timed out.
The page took too long to load.
The connection to the server timed out.
A connection to the host could not be established in time.
The requested page was not found.
The host couldn't be resolved, the page returned HTTP 404, or it answered 200 with an error page saying it does not exist.
An internal server error occurred in the requested page.
The page returned an HTTP 5xx status.
The request was invalid.
The page returned another 4xx status (e.g. 401, 403, 410).
There was a problem with the SSL certificate.
The host's TLS certificate could not be validated.
The server refused the connection.
The host actively refused the connection.
The page is rate-limiting requests.
The page returned HTTP 429. We slow down and try again ourselves before you ever see this.
Too many redirects occurred while processing the request.
A redirect loop was detected.
No internet connection was detected.
A network connectivity problem occurred.
The request was blocked by the client.
The request was blocked (e.g. by ad-blocking rules).
The requested URL resolved to an address that is not publicly reachable.
The URL, or a redirect from it, pointed at a private or loopback address. Only public web addresses can be analyzed.
The requested URL is not a web page.
The URL returned something this endpoint cannot read: a file Chrome downloads rather than shows (a zip, a CSV), an image, or a document asked for what only a page has — /html or /console of a PDF. See Documents and files.
The document could not be read: it is encrypted, damaged, or has no text in it (a scanned PDF, for example).
The URL served a PDF or text document with no text to read. Scanned PDFs need OCR, which we do not run.
The page asked us to complete a bot check before it would load.
The site served an anti-bot challenge instead of the page. We work the challenge before you see this — waiting it out and retrying — and most of them clear; a handful of sites put a CAPTCHA in front of every visitor that is not a person. A block page served with a 200 is reported the same way.
The page asked us to sign in before it would show its content.
The visit was sent to a sign-in or registration form in place of the page: the page exists and is not public. Nothing is spent. A sign-in page you ask for by its own address is returned like any other.
The page could not be loaded.
The page never came up, or all it showed was a loading indicator or a notice that JavaScript is required, so there was nothing to analyze.
An unexpected error occurred while processing the request.
Something failed that we do not have a specific answer for. These are reported to us automatically; retrying is usually worthwhile.
The page is too big to be processed.
The HTML exceeds the 10 MB limit, or the page holds more content than an AI operation can return in one response.
No residential exit was free to load this page. Retry shortly, or send the same request without residential to use our standard network.
The request asked for a residential exit and none could take it. There is no fallback here on purpose — answering from a data center address is the one thing the request ruled out. Nothing is spent; retry in a moment, or drop the parameter.
The exit this auth_profile's session is used from was not free to load this page. Retry shortly.
Every visit made as an auth profile goes out from the one exit its session has been used from, because sites end a session that moves. That exit was busy or unreachable, and there is no other on purpose. Nothing is spent; retry in a moment. See Auth profiles.
Send the request without residential: this auth_profile's session is used from our standard network, so it cannot be fetched from a residential exit.
The request asked for a residential exit, and the auth profile's session had meanwhile been used from our standard network by another request made as it. A session that moved exit would be ended by the site, and the request ruled out every other one. Nothing is spent. See Auth profiles.
The auth_profile this request named was deleted before the page was loaded.
The auth profile was deleted between the request being accepted and its page being visited. Nothing is spent.
A selector in the request is not valid CSS.
A selector in page_options or screenshot_options could not be parsed as CSS.
No visible element on the page matched the selector.
A screenshot's selector matched nothing, or only an element with no size. Nothing is spent.
The element named by wait_for_selector did not appear on the page.
The page was given 10 seconds for the element and it never arrived. Nothing is spent.
The screenshot is too large to return. Lower full_page_max_height, or use format jpeg or webp.
The image came out larger than 20 MB — usually a very long page at device_scale_factor 2 or 3, as a PNG.
The PDF is too large to return. Turn off print_background, lower the scale, or use page_options to remove what you do not need.
The PDF came out larger than 20 MB — usually a very long page full of images or heavy backgrounds.
The site's robots.txt disallows this page, and this project is set to follow robots.txt.
The project follows robots.txt and a rule in the site's file covers this page, so it was not fetched. Nothing is spent. See robots.txt for how the rules are read and where the setting lives.

Busy pages are waited out for you

Three of those messages — a timeout, a 5xx and a 429 — mean the page is struggling rather than broken, and they are the ones a retry actually fixes. URLpipe does that part itself: when a page starts stalling, requests to that host are spaced further and further apart and the analysis is tried again, for up to ten minutes, before any failure is reported. A crawl of a site that cannot keep up therefore takes longer and still finishes, instead of failing every request after the first few.

What you see is a request that takes longer, not one that fails. A synchronous request may outlive its wait and answer processing_timeout with a token — collect the result from GET /result/:token as you would for any long analysis. An async request simply delivers later. The processing_time_ms we report covers the whole of it, waiting included.

Recommended handling

  • Check the HTTP status first: 401 → fix credentials, 403 → confirm the account's email address, 429 → back off, 422 → inspect the message.
  • Retry timeouts and connection errors with backoff; they're often transient — and by the time one reaches you we have already waited the page out without success.
  • Keep target pages under 10 MB of HTML to avoid The page is too big to be processed..