# All robots will spider the domain User-agent: * Disallow: /admin Disallow: /clients Disallow: /docs Disallow: /emails Disallow: /forms Disallow: /includes Disallow: /ioncube Disallow: /js Disallow: /rssnews Disallow: /samswhois Disallow: /templates Disallow: /tools Disallow: /whois Disallow: /xml-api