# Don't allow web crawlers to index Craft User-agent: * Disallow: /craft/ # Hide Craft-generated "nodes" for the blog Disallow: /blog/author/ Disallow: /blog/all Disallow: /blog/tags/