---
title: Setting robots.txt in elcomCMS
description: Setting the robots.txt file is important for your web site's health - Google uses it as a guide for what to index and what to exclude.
---

[XEN Digital Marketing and HubSpot Blog ](https://www.xen.com.au/blog)

# [Setting robots.txt in elcomCMS](https://www.xen.com.au/blog/setting-robots-txt-in-elcomcms)

 Written by [Craig Bailey](https://www.xen.com.au/blog/author/craig-bailey) | September 28, 2015

Setting the [robots.txt file](http://www.robotstxt.org/) is important for your web site's health - Google uses it as a guide for what to index and what to exclude.

I've previously included references to the robots.txt file in [this post on going live with a new elcomCMS site](https://xen.com.au/elcomcms-seo-checklist-new-site-go-live/) (see point 6) but it's worth covering again here.

In elcomCMS, setting the robots.txt file is really easy.

You can access it from the Admin menu:

and then edit it directly:

<https://www.xen.com.au/hubfs/Imported_Blog_Media/robots.png>

## Recommended robots.txt Settings

### Staging Site

If a site is in staging, you'll ideally want to block Google from crawling it. Use the following to block the entire site:

```
User-agent: *
Disallow: /
```

### Live site

Once the site is live, you only really want to block a few items eg blocking the login page is reasonably common. You can also block entire directories eg such as an FTP folder:

```
User-agent: *
Disallow: /cmlogin.aspx
Disallow: /ftp/
```

Be very careful when going live with a site - I often see sites go live (from staging) and keep their robots.txt file from staging - and in the process block their entire live site in Google! Make sure you check your robots.txt file when going live.

[View full post](https://www.xen.com.au/blog/setting-robots-txt-in-elcomcms)

```json
{
  "@context" : "http://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Craig Bailey"
  },
  "dateModified" : "2018-09-04T08:24:12.372Z",
  "datePublished" : "2015-09-28T04:32:58Z",
  "headline" : "Setting robots.txt in elcomCMS",
  "image" : {
    "@type" : "ImageObject",
    "height" : 60,
    "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png",
    "width" : 60
  },
  "mainEntityOfPage" : "https://www.xen.com.au/blog/setting-robots-txt-in-elcomcms",
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "height" : 60.0,
      "url" : "https://cdn2.hubspot.net/hubfs/306227/xen-b2b/logo/xen-logo.png",
      "width" : 145.2
    },
    "name" : "XEN Digital Marketing and HubSpot Blog"
  }
}
```