Free Robots.txt Generator for Websites

Free technical SEO tool

Build a Safe Starting Robots.txt File

Choose a conservative crawl setup, add your sitemap, and download a clean robots.txt draft. Review the output before publishing it on your domain.

1. Configure your file

Your entries stay in this browser and are not submitted to Avenzo.

Website URLUse the canonical HTTPS version of your homepage.

Website type

Additional paths to disallow (optional)Enter one URL path per line. Do not add pages that need to be crawled to process a noindex rule.

Add a sitemap referenceUses your domain plus the sitemap path below.

Sitemap path

Allow OpenAI search discoveryKeeps OAI-SearchBot eligible to surface public pages in ChatGPT search. Block GPTBot training crawlerSeparate from OAI-SearchBot, which is used for ChatGPT search discovery.

2. Review and export

Check every rule against your real site structure before uploading it.

Your generated robots.txt file will appear here.

Robots.txt manages crawler access; it does not secure private data and is not a reliable way to remove a page from search results. Use authentication or an appropriate noindex method when that is your goal.

Guidance checked against Google Search Central and OpenAI crawler documentation.

Need a developer to review the implementation?

We can check crawl controls, sitemaps, indexability, and the wider technical setup.

Discuss your website