# Crawl4AI

Open-source, blazing fast asynchronous LLM web crawler

Crawl4AI is an open-source, speed-optimized asynchronous web crawler that generates structured Markdown and JSON graphs for AI models.

## Overview

- **Website**: https://github.com/unclecode/crawl4ai
- **Docs**: https://crawl4ai.com
- **GitHub**: https://github.com/unclecode/crawl4ai
- **License / Pricing**: Open Source
- **Directory Entry**: https://agentaccess.dk/enablers/crawl4ai

## About

Crawl4AI is engineered specifically for AI engineers who want zero subscription fees and extreme throughput. It runs asynchronous crawls with smart caching, CSS-selector stripping, heuristic content scoring, and multi-URL batching, producing clean LLM markdown with metadata at zero cost.

## Key Features

- 100% open source and local execution
- Asynchronous batch crawling with asyncio
- Heuristic content extraction (removes headers, footers, and noise)
- Extracts structured JSON using local models (Ollama) or cloud APIs
- Screenshots and PDF extraction capabilities

## Danish System Application Recipes

### Batch Ingest Danish Legal Codes from Retsinformation.dk
Run Crawl4AI across Danish legal statues and minister orders on Retsinformation to generate an open-source vector store of Danish law.

## Quickstart

```python
import asyncio
from crawl4ai import AsyncWebCrawler

async def main():
    async with AsyncWebCrawler() as crawler:
        result = await crawler.arun(url="https://www.retsinformation.dk")
        print(result.markdown[:500])

asyncio.run(main())
```

---
Source: [AgentAccess Enablers](https://agentaccess.dk/enablers/crawl4ai)