Topic

crawling

4 guides on crawling.

SEO

robots.txt mistakes that silently hide your site from Google

The robots.txt mistakes that quietly drop pages from search: leftover staging blocks, wildcard typos, blocked CSS/JS. Audit checklist plus a free tester inside.

2 min read

SEO

How Google discovers new pages — the four channels that matter

Google discovers new URLs via sitemaps, internal links, external links and IndexNow. Here is how to prioritise each channel for fast indexing of fresh content.

2 min read

SEO

IndexNow protocol for Indian publishers — adopt it for Bing share

IndexNow lets you ping Bing, Yandex and a few others when content changes — indexed within minutes. Google has not adopted it. When and how to set it up.

2 min read

SEO

Crawl budget for small Indian sites — when it actually matters

Crawl budget worries are usually overblown for small sites. Here is when it genuinely matters, how to diagnose, and the fixes that actually shift the dial.

2 min read