llms.txt Documentation Extraction Guide

llm.energy extracts documentation from websites implementing the llms.txt and install.md standards. It transforms raw documentation into structured, agent-ready formats optimized for LLMs.

Standards

Standard	Purpose	Specification
llms.txt	Machine-readable documentation	llmstxt.org
install.md	LLM-executable installation instructions	installmd.org

What It Does

Input: https://docs.chat.sperax.io
    │
    ▼
Auto-detects:
  ├── /llms.txt (summary)
  ├── /llms-full.txt (full docs)
  └── /install.md (setup instructions)
    │
    ▼
Output:
  ├── Individual markdown files per section
  ├── Single concatenated document
  ├── JSON structured output
  └── MCP-compatible tool responses

Quick Start

Web Interface

Visit llm.energy, paste a URL, and extract.

CLI

npx @llm-energy/cli extract https://docs.chat.sperax.io

MCP Server

{
  "mcpServers": {
    "llm-energy": {
      "command": "npx",
      "args": ["@llm-energy/mcp-server"]
    }
  }
}

MCP Tools

Tool	Description
extractDocs	Extract llms.txt from a URL
extractFull	Extract llms-full.txt (complete documentation)
extractInstall	Extract install.md (setup instructions)
detectStandards	Check which standards a site supports
batchExtract	Extract from multiple URLs at once

REST API

# Extract documentation
curl https://llm.energy/api/extract?url=https://docs.chat.sperax.io

# Detect standards support
curl https://llm.energy/api/detect?url=https://docs.chat.sperax.io

# Batch extract
curl -X POST https://llm.energy/api/batch \
  -d '{"urls": ["https://docs.chat.sperax.io", "https://docs.aave.com"]}'

Export Formats

Format	Best For
Markdown	Human reading, GitHub, documentation sites
JSON	Programmatic consumption, databases
Plain text	LLM context windows, RAG pipelines
Split files	Individual section storage

Use Cases

Build a Knowledge Base

Extract docs from multiple DeFi protocols to create a comprehensive AI knowledge base:

llm-energy batch \
  https://docs.chat.sperax.io \
  https://docs.aave.com \
  https://docs.uniswap.org \
  --output ./defi-knowledge/

Agent Documentation

Give AI agents up-to-date documentation for any tool or protocol they need to use.

SperaxOS Integration

llm.energy powers the documentation extraction in SperaxOS's knowledge base tool, enabling agents to ingest any protocol's docs on-demand.

Adding llms.txt to Your Site

To make your site AI-friendly, add these files:

/llms.txt (Summary)

# My Protocol

> One-line description

## Docs
- [Getting Started](https://docs.example.com/start)
- [API Reference](https://docs.example.com/api)

/llms-full.txt (Full Content)

Concatenation of all documentation pages in a single markdown file.

/install.md (Setup)

Step-by-step installation instructions that an LLM can execute.

Links

• Website: https://llm.energy
• GitHub: https://github.com/nirholas/extract-llms-docs
• npm: https://www.npmjs.com/package/@llm-energy/mcp-server
• llms.txt spec: https://llmstxt.org
• install.md spec: https://installmd.org