crawlee-python

crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

github Automation Python free
★ 8,783Stars
708Forks
8,783Watchers
2Views
Apr 2026Last Update
View on GitHub →

About crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

Related Tools

n8n

Fair-code workflow automation platform with native AI capabilities. Combine visual building with custom code, self-host or cloud, 400+ integrations.

★ 184,253 typescript github

core

:house_with_garden: Open source home automation that puts local control and privacy first.

★ 86,045 python github

awesome-claude-skills

A curated list of awesome Claude Skills, resources, and tools for customizing Claude AI workflows

★ 54,112 python github

airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

★ 45,058 python github

phantomjs

Scriptable Headless Browser

★ 29,473 cpp github

oh-my-claudecode

Teams-first Multi-agent orchestration for Claude Code

★ 29,195 typescript github