Crawl a site and return clean page content. POST { url, limit?, maxDepth?, instructions? }. Follows links from the start URL (up to 10 pages, depth ≤2) and returns each page's extracted content. Optional natural-language instructions steer which pages to follow (e.g. "only pricing and docs pages"). For ingesting a small site/section in one call.
/api/search/crawlPAYMENT-SIGNATURE.The crawl API is a pay-per-call search endpoint built for AI agents and autonomous software. Crawl a site and return clean page content.
There is no signup and no API key. An agent (or any HTTP client) hits the endpoint, receives an x402 "402 Payment Required" challenge, signs a sub-cent USDC payment on Base or Solana, and retries — the data comes back on the paid request. That makes it a drop-in crawl data source for an agent tool-use loop, an MCP host, or a backend that needs search data on demand without onboarding to yet another vendor portal.
| Name | Type | Description |
|---|---|---|
urlrequired | string | Start URL to crawl. max 2048 chars · uri |
limit | integer | Max pages (≤10). min 1 · max 10 |
maxDepth | integer | min 1 · max 2 |
instructions | string | Natural-language steering for which pages to follow. max 300 chars |
# 1. Probe the endpoint with no auth — receive 402 with PaymentRequirements
curl -sS -X POST 'https://2s.io/api/search/crawl' \
-H 'Content-Type: application/json' \
-d '{"url":"https://example.com","limit":1,"maxDepth":1,"instructions":"example"}'
# 2. Sign the EIP-3009 transferWithAuthorization for the advertised price +
# payTo from the 402 envelope, then retry with PAYMENT-SIGNATURE:
curl -sS -X POST 'https://2s.io/api/search/crawl' \
-H 'Content-Type: application/json' \
-H 'PAYMENT-SIGNATURE: <base64-json-payload>' \
-d '{"url":"https://example.com","limit":1,"maxDepth":1,"instructions":"example"}'
# Or just use the canonical runner — it handles the whole loop:
# EVM_PRIVATE_KEY=0x... node --env-file=.env.local \
# --experimental-strip-types scripts/x402-pay.ts \
# 'https://2s.io/api/search/crawl'import { TwoS } from '@2sio/sdk'
const client = new TwoS({
privateKey: process.env.EVM_PRIVATE_KEY as `0x${string}`,
})
const result = await client.search.crawl({
"url": "https://example.com",
"limit": 1,
"maxDepth": 1,
"instructions": "example"
})
console.log('endpoint:', result.endpoint)
console.log('cost:', result.costUsd, 'USDC')
console.log('tx:', result.settlement?.txHash)
console.log('data:', result.data)import os
from twosio import TwoS
client = TwoS(private_key=os.environ["EVM_PRIVATE_KEY"])
result = client.search.crawl(url="https://example.com", limit=1, maxDepth=1, instructions="example")
print("endpoint:", result.endpoint)
print("cost:", result.cost_usd, "USDC")
print("tx:", (result.settlement or {}).get("tx_hash"))
print("data:", result.data)// 1. Add @2sio/mcp to your MCP host config (Claude Desktop example below).
// EVM_PRIVATE_KEY funds x402 payments per call.
// claude_desktop_config.json
{
"mcpServers": {
"2sio": {
"command": "npx",
"args": ["-y", "@2sio/mcp"],
"env": { "EVM_PRIVATE_KEY": "0x..." }
}
}
}
// 2. Once the server is running, agents call this tool via standard MCP:
{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "search.crawl",
"arguments": {
"url": "https://example.com",
"limit": 1,
"maxDepth": 1,
"instructions": "example"
}
}
}| Field | Type | Description |
|---|---|---|
ok | boolean | one of: true |
items | array | |
total | integer | Total matching rows upstream; null when unknown. |
source | object | |
meta | object |
{
"ok": true,
"items": [
{
"url": "example",
"content": "example"
}
],
"total": 1,
"source": {
"provider": "example",
"url": "example",
"license": "example"
},
"meta": {
"baseUrl": "example"
}
}