How Google discovers new pages — the four channels that matter

Quick frame: Google discovers new pages four ways: sitemap submission, internal links from already-indexed pages, external links / mentions, and IndexNow ping. Each has different latency and reliability. For fast indexing, combine all four.

Channel 1: sitemap submission

The most reliable for new pages on established sites. After Google fetches your sitemap (triggered by lastmod changes), it queues new URLs for crawl. Latency: hours to days.

Get the basics right:

• Generate with the sitemap generator.
• Reference in robots.txt via Sitemap: directive.
• Submit in Search Console.

Channel 2: internal links from already-indexed pages

When you publish a new post and link to it from your homepage / a recent post / a frequently-crawled hub page, Google discovers the new URL on the next crawl of the source page. Latency: hours to a week, depending on the source page's crawl frequency.

Tactical implication: place internal links to new content from your highest-traffic pages, not buried in the archive.

Channel 3: external links / social mentions

When external sites link to your new URL, Google discovers it via the linking site's crawl. Twitter/X and LinkedIn mentions can also drive discovery (though not link equity). Latency: highly variable — hours if the linking site is high-authority, weeks if not.

Channel 4: IndexNow

IndexNow is a protocol (supported by Bing, Yandex, Naver and some smaller engines) where you ping a URL when content changes. Google doesn't officially support IndexNow but Bing's adoption matters in some markets. Latency: minutes.

Indian publishers should adopt IndexNow for the Bing share alone, then expect Google to follow eventually. Background in IndexNow protocol for Indian publishers.

Crawl budget reality

Google doesn't crawl every page every day. Smaller sites (under a few thousand URLs) get full re-crawl roughly weekly. Larger sites get prioritised crawl based on:

• Authority (PageRank-like signals).
• Update frequency (sites that publish often get crawled more).
• Server response time (slow sites get less crawl).

For sites genuinely starved of crawl budget, the crawl budget for small Indian sites piece has the diagnostic.

When indexing stalls

If new pages aren't getting indexed after 2 weeks:

1. Search Console → URL Inspection → request indexing. (Works for ~10 URLs per day per property.)
2. Verify the page is crawlable — test in robots.txt tester.
3. Verify no noindex (check via robots meta generator reference).
4. Build internal links from indexed pages.

FAQ

Q. Does pinging Google Search Console help? A. Yes, but only ~10 URL inspections per day per property. Useful for time-sensitive content; not scalable.

Q. Should I include very new URLs in the sitemap immediately? A. Yes — the sitemap is the cleanest discovery channel.

Q. How do I get fast indexing for breaking news? A. Combine: NewsArticle schema, sitemap with fresh lastmod, internal link from homepage, external syndication, and URL Inspection request. The fastest combination indexes within 1–4 hours.