# don't let search engines see the RSS feed, it's just confusing. User-agent: Googlebot User-agent: InfoNaviRobot User-agent: TV33_Mercator User-agent: AVSearch User-agent: Mercator User-agent: Scooter User-agent: Slurp User-agent: FAST-WebCrawler User-agent: Lycos_Spider Disallow: /searchtools-rss.xml User-agent: * Disallow: / Disallow: /images/ Disallow: /exec/ Disallow: /cgi-bin/ Disallow: /awstats/ Disallow: /_private/ Disallow: /_vti_txt/ Disallow: /_vti_pvt/ Disallow: /_vti_log/ Disallow: /_vti_cnf/ Disallow: /_vti_bin/ Disallow: /now/ Disallow: /forum/ Disallow: /.fp/ # updated 2002-03-22 (disallow rtestprob links)