crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
About crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
What you should know about crawlee
crawlee — Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.. It is categorized under Automation and primarily built with TypeScript. The project has gathered 22,808 stars and 1,316 forks on GitHub, indicating strong adoption among developers.
Pricing & licensing: This tool is offered free of charge , released under the Apache-2.0 license. The source code is openly available on GitHub, allowing engineers to audit, contribute, or fork as needed.
Use cases & topics: crawlee is associated with the following topics: apify, automation, crawler, crawling, headless, headless-chrome, javascript, nodejs. Teams working in apify / automation / crawler spaces typically evaluate this kind of tool when scoping new architecture decisions or replacing legacy components.
Getting started: Check out the official GitHub repository for installation steps, configuration examples, and the latest release notes. Most teams hit value within the first week if the tool aligns with their existing Automation stack.
Editor's note from Fanny Engriana (Founder, Wardigi Digital Agency): when evaluating tools in the Automation category for our agency clients, we look at three things first — license clarity, community size, and active maintenance. Tools with explicit license terms and ongoing commits tend to remain viable across multi-year projects.