Manage a robots.txt file

[CXL (CXL Playbooks)]

2 Playbooks

Audit your robots.txt file

Ensure that there’s no mistake in your robots.txt file restricting search engines from areas you want them to crawl.

Create a robots.txt file

Let search engines know which pages on the site to crawl and how they should interact with your site when crawling it.