Search Engines

2 articles in this category.

Written by me.

How Googlebot Discovers and Crawls Pages

An engineering breakdown of how Googlebot discovers and crawls URLs: crawl frontiers, internal link graphs, sitemap processing, and crawl demand dynamics.

November 13, 20267 min read

Read article

Technical SEOWeb InfrastructureArchitecture

Technical SEO Engineering: From Web Request to Search Index

An engineering blueprint connecting web requests to search indexing: DNS, TLS, edge caching, rendering pipelines, crawl mechanics, and observability.

November 13, 20268 min read

Read article