#asyncio
5 guides
Requests vs HTTPX for Web Scraping: Sync, Async, Retries, and Throughput
A practical comparison of Requests and HTTPX for scraper teams choosing between simplicity, async growth, retry control, and throughput.
Web Scraping Rate Limiting: How to Throttle Requests Without Killing Throughput
Design rate limiting for scrapers that stays polite enough to reduce bans but fast enough for production, with practical token-bucket patterns, concurrency controls, and retry strategy.
Web Scraping with HTTPX: Async Fetching, Retries, and Timeouts
A practical guide to web scraping with HTTPX in Python: sane timeouts, bounded async fetching, explicit retries, and production-ready request patterns.
Async Web Scraping in Python: asyncio + aiohttp Guide (Patterns That Don’t Get You Banned)
A practical asyncio + aiohttp guide for web scraping: bounded concurrency, semaphores, retries with backoff, timeouts, per-host limits, and batch exporting. Includes a complete working template.
Async Web Scraping in Python: asyncio + aiohttp (Concurrency Without Getting Banned)
Learn production-grade async scraping in Python with asyncio + aiohttp: bounded concurrency, per-host limits, retry/backoff, timeouts, and proxy rotation patterns. Includes a complete working crawler template.