LLM-Ready Web Scraping & Markdown ExtractionOpen SourceOpen Source Crawler

Crawl4AI

Open-source, blazing fast asynchronous LLM web crawler

Crawl4AI is an open-source, speed-optimized asynchronous web crawler that generates structured Markdown and JSON graphs for AI models.

Officiel Hjemmeside Dokumentation GitHub Repository

.md

Kom hurtigt i gang

crawler.py

Kom i gang med Crawl4AI i dit projekt.

import asyncio
from crawl4ai import AsyncWebCrawler

async def main():
    async with AsyncWebCrawler() as crawler:
        result = await crawler.arun(url="https://www.retsinformation.dk")
        print(result.markdown[:500])

asyncio.run(main())

Om Crawl4AI

Crawl4AI is engineered specifically for AI engineers who want zero subscription fees and extreme throughput. It runs asynchronous crawls with smart caching, CSS-selector stripping, heuristic content scoring, and multi-URL batching, producing clean LLM markdown with metadata at zero cost.

Danske data- og produkt-guides

Hvordan udviklere og agenter bruger Crawl4AI til at tilgå dansk digital infrastruktur.

Batch Ingest Danish Legal Codes from Retsinformation.dk

Run Crawl4AI across Danish legal statues and minister orders on Retsinformation to generate an open-source vector store of Danish law.

Nøglefunktioner

100% open source and local execution

Asynchronous batch crawling with asyncio

Heuristic content extraction (removes headers, footers, and noise)

Extracts structured JSON using local models (Ollama) or cloud APIs

Screenshots and PDF extraction capabilities