Editing robots.txt.liquid

Search engines, such as Google, constantly crawl the internet in search of new data as a source for their search results. The robots.txt file tells search engine bots, known as crawlers, which pages to request to view from your online store. All Shopify stores have a default robots.txt file that's optimal for Search Engine Optimization (SEO).

Your sitemap is used by search engines to place your online store in the search engine results. Learn how to find and submit your sitemap.

Overview

The default robots.txt file works for most stores, but you can edit the file through the robots.txt.liquid theme template. You can make the following edits:

  • allow or disallow certain URLs from being crawled
  • add crawl-delay rules for certain crawlers
  • add extra sitemap URLs
  • block certain crawlers

The robots.txt file is located at the root directory of your Shopify store's primary domain name. For example: johns-apparel.com/robots.txt.

Edit robots.txt.liquid

If you want to edit the robots.txt.liquid file, then you should work with a Shopify Partner or have expertise in code edits and SEO.

You can use Liquid to add or remove directives from the robots.txt.liquid template. This method preserves Shopify’s ability to keep the file updated automatically in the future, and is recommended. For a full guide on editing this file, refer to Shopify's Developer page Customize robots.txt.liquid.

Remove any previous customizations or workarounds, such as using a third-party service such as Cloudflare, before you edit the robots.txt.liquid file.

Steps:

  1. From your Shopify admin, click Settings > Apps and sales channels.

  2. From the Apps and sales channels page, click Online store.

  3. Click Open sales channel.

  4. Click Themes.

  5. Click the ... button, and then click Edit Code.

  6. Click Add a new template, and then select robots.

  7. Click Create template.

  8. Make the changes that you want to make to the default template. For more information on liquid variables and common use cases, refer to Shopify's Developer page Customize robots.txt.liquid.

  9. Save changes to the robots.txt.liquid file in your published theme.

Changes are instant, but crawlers don't always react immediately. You can test changes using Google's robots.txt Tester.

You can also delete the contents of the template and replace it with plain text rules. This method is strongly not recommended, as rules may become out of date. If you choose this method, then Shopify can’t ensure that SEO best practices are applied to your robots.txt over time, or make changes to the file with future updates.

Delete robots.txt.liquid customizations

If you want to use the default robots.txt file again, then save a copy of your robots.txt.liquid template customizations as you need to delete the template, which can't be undone.

Steps:

  1. From your Shopify admin, click Settings > Apps and sales channels.

  2. From the Apps and sales channels page, click Online store.

  3. Click Open sales channel.

  4. Click Themes.

  5. Click the ... button, and then click Edit Code.

  6. Click robots.liquid, and then click Delete file.

  7. If you still want to delete robots.txt.liquid, then click Delete file.

You can always edit the robots.txt.liquid file in the future, or if you saved your past customizations, then you can redo your changes by copying over your saved previous customizations.

Ready to start selling with Shopify?Try it free