When using MMT on a dev store with the default config, I'm thinking I need to remove the password-protected folder status. So the question is, In order to prevent bots and spiders, etc from crawling the site I need to set a disallow all condition in the robots.txt?
Example:
Thanks,
Scott
Example:
Code:
User-agent: * Disallow: /
Scott
Comment