#This robots.txt file has been created to prevent the user-agents from crawling the entire site User-agent: WebTrends Disallow: User-agent: * Disallow: /cgi/ Disallow: /clslink2005/ Disallow: /commercial/ Disallow: /ici_search/ Disallow: /images/ Disallow: /inprint/ Disallow: /listings/ Disallow: /newimages/ Disallow: /newsletter/ Disallow: /CLSLink/PropertySearch/ Disallow: /rebgv/ Disallow: /search/ Disallow: /sitemap/