Cloudflare Browser Rendering

Browse and scrape the web via Cloudflare's Browser Rendering REST API. Every call is a single POST request — no browser setup, no Puppeteer scripts.

Prerequisites

Requires two env vars (confirm they're set before making calls):

• CF_ACCOUNT_ID — Cloudflare account ID
• CF_API_TOKEN — API token with Browser Rendering - Edit permission

Helper script

Use cfbr.sh for all API calls. It handles auth headers and the base URL:

# JSON endpoints
cfbr.sh <endpoint> '<json_body>'

# Screenshot (binary) — optional third arg for output filename
cfbr.sh screenshot '<json_body>' output.png

Choosing an endpoint

Goal	Endpoint	When to use
Read page content for analysis	markdown	Default choice — clean, token-efficient
Extract specific elements	scrape	Know the CSS selectors for what you need
Extract structured data with AI	json	Need typed objects, don't know exact selectors
Get full rendered DOM	content	Need raw HTML for parsing or debugging
Discover pages / crawl	links	Building a sitemap or finding subpages
Visual inspection	screenshot	Need to see the page layout or debug visually
DOM + visual in one shot	snapshot	Need both HTML and a screenshot

For full endpoint details and parameters, see api.md.

Scraping workflow

Follow this sequence when scraping a site for structured data (e.g. rental listings, product catalogs, job boards):

1. Reconnaissance — understand the page

Start with markdown to see what content is on the page and how it's structured:

cfbr.sh markdown '{"url":"https://target-site.com/listings", "gotoOptions":{"waitUntil":"networkidle0"}}'

If the page is an SPA or loads content dynamically, networkidle0 ensures JS finishes executing. If you know a specific element that signals content is ready, use waitForSelector instead — it's faster:

{"url":"...", "waitForSelector": ".listing-card"}

2. Discover structure — find the selectors

From the markdown/HTML, identify repeating patterns (listing cards, table rows, etc.) and their CSS selectors. If unclear from markdown alone, use screenshot to visually inspect:

cfbr.sh screenshot '{"url":"https://target-site.com/listings", "screenshotOptions":{"fullPage":true}, "gotoOptions":{"waitUntil":"networkidle0"}}' listings.png

3. Extract — pull structured data

Option A: CSS selectors (when you know the DOM structure)

cfbr.sh scrape '{
  "url": "https://target-site.com/listings",
  "gotoOptions": {"waitUntil": "networkidle0"},
  "elements": [
    {"selector": ".listing-card .title"},
    {"selector": ".listing-card .price"},
    {"selector": ".listing-card .address"},
    {"selector": ".listing-card a"}
  ]
}'

The scrape endpoint returns text, html, attributes (including href), and position/dimensions for each match. Correlate results across selectors by index (first title matches first price, etc.).

Option B: AI extraction (when structure is complex or unknown)

cfbr.sh json '{
  "url": "https://target-site.com/listings",
  "gotoOptions": {"waitUntil": "networkidle0"},
  "prompt": "Extract all rental listings with title, price, address, bedrooms, and link",
  "response_format": {
    "type": "json_schema",
    "schema": {
      "type": "object",
      "properties": {
        "listings": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "title": {"type": "string"},
              "price": {"type": "string"},
              "address": {"type": "string"},
              "bedrooms": {"type": "string"},
              "url": {"type": "string"}
            },
            "required": ["title", "price"]
          }
        }
      }
    }
  }
}'

Prefer scrape when selectors are clear — it's deterministic and free. Use json when the page structure is messy or you need semantic interpretation (incurs Workers AI charges).

4. Paginate — get all results

Use links to find pagination URLs:

cfbr.sh links '{"url":"https://target-site.com/listings"}'

Look for ?page=2, next, or load-more patterns. Repeat extraction for each page.

Infinite-scroll pages are a limitation — the API is stateless (one request = one browser session), so there's no way to scroll, wait for new content to load, and then extract in a single call. For these pages, look for an underlying API or URL parameters (e.g. ?page=2, ?offset=20) that serve paginated data directly.

5. Handle obstacles

SPA / empty results — Add "gotoOptions": {"waitUntil": "networkidle0"} or "waitForSelector": "<selector>".

Slow pages — Increase timeout: "gotoOptions": {"timeout": 60000}.

Heavy pages — Strip unnecessary resources:

{"rejectResourceTypes": ["image", "stylesheet", "font", "media"]}

Auth-gated pages — Pass session cookies:

{"cookies": [{"name": "session", "value": "abc123", "domain": "target-site.com", "path": "/"}]}

Bot detection — Cloudflare Browser Rendering is always identified as a bot. The userAgent field changes what the site sees but will not bypass bot protection. If a site blocks the request, there is no workaround via this API.

Tips

• markdown is the best default for content extraction — it's clean, compact, and LLM-ready.
• Always use networkidle0 or waitForSelector on any modern site. Without it you'll get incomplete content.
• rejectResourceTypes dramatically speeds up text-only operations. Always strip images/fonts/stylesheets when you only need text.
• scrape results are ordered by DOM position — correlate across selectors by array index.
• For large scraping jobs, process pages sequentially to stay within rate limits.