SEO

Robots.txt Generator

Create a clear robots.txt starter for websites, blogs and ecommerce stores before launch.

Allow crawling

Sitemap URL

Private paths

robots.txt

User-agent: *
Allow: /
Disallow: /admin
Disallow: /account
Disallow: /checkout
Disallow: /cart

Sitemap: https://example.com/sitemap.xml

How it works

Choose whether crawlers are allowed, add private paths and a sitemap URL, then copy the generated robots.txt content.

A simple robots.txt helps guide crawlers and keeps common private paths out of routine crawling.

No. It is a crawler instruction, not a security layer. Protect private areas with authentication.

Yes. A sitemap line helps crawlers discover your XML sitemap faster.