Skip to content
Free Indian Tools

SEO · Free tool

Robots Meta Tag Generator

Pick the indexing directives — get both the meta tag (for HTML) and the X-Robots-Tag header (for PDFs and other non-HTML files).

As meta tag (paste into <head>)

<meta name="robots" content="noindex, follow" />

As HTTP header (for PDFs, images, non-HTML files)

X-Robots-Tag: noindex, follow

Meta tag vs HTTP header

The two transports take identical directives. The meta tag goes inside the HTML <head>; the X-Robots-Tag header is sent in the HTTP response. Use the header for non-HTML files — PDFs, images, JSON downloads — since you can't inject a meta tag into them.

noindex versus Disallow

Disallow in robots.txt blocks crawling but the URL can still appear in results. noindex here prevents indexing — but Google must be able to crawl the page to see the tag. So the right pattern for removal is: noindex first, wait for re-crawl, then optionally Disallow. Use the robots.txt tester to confirm the page is crawlable. Background in noindex vs Disallow vs canonical.

Useful combinations

  • noindex, follow — keep link equity flowing but remove from index (e.g., thin tag pages).
  • max-snippet:0 — suppress the snippet entirely (rare; usually paywalled content).
  • max-image-preview:large — opt into large image previews on SERPs and Discover.
  • unavailable_after — auto-deindex time-limited pages (events, sales).

FAQ

What is the difference between robots meta and x-robots-tag header?

Same instructions, different transport. Meta tag goes in HTML head. x-robots-tag is an HTTP header - required for non-HTML files (PDFs, images) since they have no HTML to inject meta into.

Does noindex,follow still work?

Google has said noindex eventually behaves like noindex,nofollow because the page stops being crawled regularly. For temporary noindex (e.g., staging), it works; for long-term, expect link equity to fade.

What does max-snippet do?

Limits the character length of the SERP snippet Google generates from your page. max-snippet:-1 means no limit. max-snippet:0 effectively suppresses the snippet (rare use case - usually for paywalled content).