I have a few forums that I want to restrict the google/yahoo/etc. spiders from crawling. What is the best way to do this? I have already removed them from the sitemap generation, but google still wants to index them (and I'm trying to funnel google toward other content). Should I just add the forums URLS to robots.txt, or is there a better way?


LinkBack URL
About LinkBacks





Reply With Quote