URLpipe API
URLpipe turns any public URL into clean, structured data. Point it at a page and get back Markdown, metadata, an AI summary, keywords, data in your own schema, a Lighthouse audit, a screenshot, a PDF, raw HTML or the JavaScript console, or the technologies it is built with — all through one simple, bearer-authenticated HTTP API.
Every endpoint takes the same shape: a POST request with a JSON body containing the url you want to process. There is nothing to install on the target site — URLpipe renders each page with headless Chrome from our own servers, so JavaScript-heavy and single-page apps work too.
A first request
Send a URL to the /markdown endpoint and get back clean Markdown, ready to drop into an LLM prompt or a RAG pipeline.
curl -X POST https://urlpipe.dev/markdown \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}'import requests
res = requests.post(
"https://urlpipe.dev/markdown",
headers={"Authorization": "Bearer YOUR_API_KEY"},
json={"url": "https://example.com"},
)
markdown = res.textconst res = await fetch("https://urlpipe.dev/markdown", {
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({ url: "https://example.com" }),
})
const markdown = await res.text()$ch = curl_init("https://urlpipe.dev/markdown");
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
"Authorization: Bearer YOUR_API_KEY",
"Content-Type: application/json",
],
CURLOPT_POSTFIELDS => json_encode(["url" => "https://example.com"]),
]);
$markdown = curl_exec($ch);Replace YOUR_API_KEY with your project's API key. Get one free, no card needed.
The endpoints
URLpipe has twelve endpoints. Eleven return one kind of data each, in two families, and /scrape runs any of them together from a single page visit. Web endpoints fetch and measure the page directly; AI endpoints use a language model to extract or transform its content. One allowance covers both, and each endpoint spends from it what it costs to run — a page fetch is one credit, and the AI endpoints are dearer because a model bills us by the token. See Credits.
Web endpoints
The fully rendered HTML of a page, after JavaScript has run.
The page's main content as clean Markdown, chrome stripped.
Title, description, authors, dates, share image, icons and feeds.
A full-page screenshot of the rendered page — PNG, JPEG or WebP, base64-encoded.
The whole rendered page as a PDF, on the paper you choose or as one long page.
A real Google Lighthouse audit — performance, SEO, accessibility.
JavaScript console errors, warnings and uncaught exceptions.
What a site is built with — CMS, frameworks, analytics, CDN — with versions.
AI endpoints
A concise AI summary of the page's main content.
5–15 relevant keywords, ordered by relevance.
Any data on the page, in a schema you define — null where the page has none.
The same endpoints are also an MCP server, so an AI agent in Claude, Cursor or any MCP client can call them itself — same arguments, same responses, same credits.
Core concepts
How bearer tokens and per-project API keys work.
Caching & freshnessReuse recent results for free with max_age.
Async & sync modesWebhook the result (the default), verify its signature, or get it inline with sync.
Residential exitsFetch a page from a home broadband address, for sites that serve a data center less.
Credits & rate limitsWhat each call costs, and why cache hits are free.
ErrorsStatus codes and error payloads to handle.
Base URL & conventions
- All requests go to
https://urlpipe.devover HTTPS. - Every endpoint is a POST with a JSON body.
- Authenticate with
Authorization: Bearer YOUR_API_KEY— see Authentication. - Web endpoints return
text/plainorapplication/json; failures return JSON with anerrorfield. - The whole API is described in OpenAPI 3.1 at /openapi.json, and as a public Postman workspace you can fork and send requests from.