x402-url-to-markdown
Advanced tools
URL to Markdown for AI agents and LLMs: turn any public web page into clean Markdown (HTML to Markdown, SSRF-safe, robots.txt-aware). MCP server (stdio + remote), zero-dependency library and x402 pay-per-call API ($0.002 USDC on Base).
Start here: copy-paste MCP configs for Claude, Cursor and Windsurf, x402 client code, AgentKit and LangChain wrappers: https://agentgate-source.pages.dev/start/
API: https://api.youraitools.co/webread · fallback (same Worker): https://webread.agentgate.workers.dev · npm:x402-url-to-markdown
Give your agent any web page as clean Markdown, for $0.002 per page. No account or API key needed.
WebRead fetches a public web page and returns only its main content as Markdown: no navigation, scripts or ads. Links come back absolute, with title, description, language and canonical metadata. Agents pay per successful read over x402 in USDC on Base, and failed reads are free. There is a free preview, and a remote MCP server for Claude, Cursor and other MCP clients.
GET https://api.youraitools.co/webread/v1/read?url=<page> costs $0.002 USDC on Base (x402 v2 and v1). It is listed on the x402 Bazaar (PayAI).GET /v1/read/preview?url=<page> returns the first 2,000 characters, 10 calls/min per IP.https://api.youraitools.co/webread/mcp with the tool read_url. It is listed in the official MCP Registry as io.github.agentgate-builder/x402-url-to-markdown.npx -y -p x402-url-to-markdown webread-mcp or docker build -t webread-mcp . && docker run -i --rm webread-mcp, tool read_url. Reads pages locally with the same SSRF-safe, robots.txt-respecting reader (free; no data goes to WebRead).x402-url-to-markdown, which ships the dependency-free HTML-to-Markdown converter, the SSRF-safe fetcher and a robots.txt parser.# 1. Free, no wallet
curl "https://api.youraitools.co/webread/v1/read/preview?url=https://example.com/"
# 2. Paid: an unpaid call answers 402 with the x402 terms (2000 = 0.002 USDC, Base, payTo)
curl -i "https://api.youraitools.co/webread/v1/read?url=https://example.com/"
{ "mcpServers": { "webread": { "type": "http", "url": "https://api.youraitools.co/webread/mcp" } } }
// 3. Pay from code with the official x402 client: npm i @x402/fetch @x402/evm viem
import { wrapFetchWithPaymentFromConfig } from '@x402/fetch';
import { ExactEvmScheme } from '@x402/evm';
import { privateKeyToAccount } from 'viem/accounts';
const pay = wrapFetchWithPaymentFromConfig(fetch, { schemes: [{ network: 'eip155:8453', client: new ExactEvmScheme(privateKeyToAccount(process.env.BUYER_KEY)) }] });
const { title, markdown } = await (await pay('https://api.youraitools.co/webread/v1/read?url=' + encodeURIComponent('https://example.com/'))).json();
// Or use just the converter (no network): npm i x402-url-to-markdown
import { htmlToMarkdown } from 'x402-url-to-markdown';
const { markdown, title } = htmlToMarkdown(html, { baseUrl: 'https://example.com/' });
{ "url": "https://example.com/", "finalUrl": "https://example.com/", "status": 200, "contentType": "text/html; charset=utf-8",
"title": "Example Domain", "description": null, "lang": "en", "canonical": null,
"markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.",
"chars": 156, "wordCount": 25, "links": 0, "truncated": false, "sourceBytes": 577, "redirects": 0, "fetchMs": 17,
"robots": "allowed", "fetchedAt": "2026-10-21T21:32:28.690Z" }
(Live output for https://example.com/ on 2026-10-21; the page has no heading or links today, so the Markdown is one paragraph. The free preview adds "tier": "free" and a full link.)
Query parameters: url (required) and maxChars (100–200000, default 200000).
Machine-readable discovery: /openapi.json (OpenAPI 3.1 with x-payment-info) and /.well-known/x402 (x402scan manifest).
| Situation | HTTP | Charged? |
|---|---|---|
| Success | 200 | yes, $0.002 |
| Invalid URL, blocked scheme/host/IP/port | 400 | no |
| Disallowed by robots.txt, or robots.txt unreachable (5xx/timeout) | 403 | no |
| Not HTML or text | 415 | no |
| Page returned 4xx/5xx, DNS failure, fetch error | 502 / 422 | no |
| Timeout (10 s) | 504 | no |
* and $). The product token is WebReadBot and the User-Agent links to this repo. robots.txt files are cached for 10 minutes. A 4xx robots.txt means allowed; a 5xx or timeout means disallowed.http:/https: on ports 80/443, and no credentials in URLs. It blocks localhost, internal TLDs and private or reserved IPv4/IPv6 literals (including IPv4-mapped forms and numeric IPv4 encodings). Hostnames are resolved over DNS-over-HTTPS and refused if they point at a private address. At most 5 redirects are followed, each one re-validated.| Route | Price |
|---|---|
GET /v1/read?url=&maxChars= | $0.002 USDC (x402, Base) |
GET /v1/read/preview?url= | free, 10/min per IP |
POST /mcp | free (remote MCP, Streamable HTTP, stateless JSON) |
GET /v1/stats | free, public usage counters |
GET /health, GET / | free (JSON for agents, HTML landing page for browsers) |
npm install
npm test # node:test, offline: fake web + fake facilitators, real x402 v2 and v1 clients
npx wrangler dev # Workers runtime locally
It deploys to Cloudflare Workers (free plan) with wrangler deploy. Sister project: AgentGate, which checks whether an AI agent is verified.
Built by an AI builder agent (agentgate-builder) with human oversight. MIT licensed. Tips: 0x3c1A1E01e790Cd326495631B3b38Bd7068F08d6F (Base).
Five small x402 tools for agents that act on outside input. Same pay-per-call USDC on Base, free tiers, remote MCP:
| Tool | What it does | Price |
|---|---|---|
| AgentGate | Is this AI agent who it says it is? Concordium Agent Registry (CIS-8004) check | $0.005 |
| WebRead | Any public web page as clean Markdown, robots.txt-aware, SSRF-safe | $0.002 |
| InjectScan | Scans text, pages and MCP tool definitions for prompt injection (repo) | $0.003 |
| DomainCheck | Is this domain or email real and trustworthy? DNS, SPF/DMARC, RDAP registration age, disposable and parked detection, 0-100 score (source) | $0.002 domain / $0.001 email |
| PayGuard | Is this crypto address on the US Treasury OFAC SDN list? Exact match with entity, programs and list date, plus chain and checksum (source) | $0.001 address / $0.01 batch of 50 |
Typical flow: verify the counterpart agent with AgentGate, check its domain or contact email with DomainCheck, read a page with WebRead, check it with InjectScan before following anything in it, and screen any wallet with PayGuard before paying it.
FAQs
URL to Markdown for AI agents and LLMs: turn any public web page into clean Markdown (HTML to Markdown, SSRF-safe, robots.txt-aware). MCP server (stdio + remote), zero-dependency library and x402 pay-per-call API ($0.002 USDC on Base).
The npm package x402-url-to-markdown receives a total of 0 weekly downloads. As such, x402-url-to-markdown popularity was classified as not popular.
We found that x402-url-to-markdown demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Threat Research
/Security News
Socket uncovered secret-stealing workflows that ran in LiteLLM’s GitHub repository 11 days before the supply chain attack on PyPI and Docker.
Security News
Cloudflare introduces tsrs, a Rust port of TypeScript’s compiler promising faster checks and lower memory use, with the TypeScript team’s help.
Security News
The PHP Foundation says Chainguard broke a security embargo ahead of critical RCE patches, as leaks disrupted the coordinated release.