Web Crawler
A basic web crawler architecture showing the core crawl loop: seed URLs feed a frontier queue, a crawler fetches pages via DNS resolution, and a content parser extracts text for storage while feeding discovered URLs back into the frontier.
