# This is the 2bet.co.uk robots.txt file # It states which directories should not be visited by robots # not because there is anything secret there, but just because we don't want random requests # that will not yield any useful results increasing the load on what is already a busy server. # BTW, the last line is there so we can see how many people/robots break the rules # if you want more info on robots go to http://info.webcrawler.com/mak/projects/robots/robots.html # The owner/administrator of this site is webmaster@2bet.co.uk # Have a nice day :-) User-agent: * Disallow: /includes/ Disallow: /cgi-bin/ Disallow: /admin/ Disallow: /check_for_bad_robots.html