According to my Google/Yahoo Sitemap plugin logs, Yahoo has been hitting my site hard again the last few days, but...
Today alone I'm showing 3,123 hits to newreply.php by the Yahoo bot.![]()
This is a discussion on Well, Yahoo ignores rel="nofollow" I guess... within the General Discussion forums, part of the vBSEO SEO Plugin category; According to my Google/Yahoo Sitemap plugin logs, Yahoo has been hitting my site hard again the last few days, but... ...
According to my Google/Yahoo Sitemap plugin logs, Yahoo has been hitting my site hard again the last few days, but...
Today alone I'm showing 3,123 hits to newreply.php by the Yahoo bot.![]()
Yeah I noticed they do as well, that's why I added them to my robots.txt file and it took a couple of weeks before they stopped doing that.
Yes, it's a good idea to have also robots.txt file with these pages excluded.
Oleg Ignatiuk / Crawlability Inc.
Security vbulletin - Patch Level for all supported versions released!
Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!
Yep I removed my robots.txt after I turned on all the relnofollows via vbseo and yahoo continued to access these pages
Looks like the crawled -print pages ranks higher the mayority of the time, which means most of my yahoo visitors lands here. nofollow does not work, so I guess the only solution is to take away that functionality?
It would be great if one of you could attach a robots.txt file that specifically deals with congsoft`s problem and the rest of the files that shouldn`t be crawled, please?
Last edited by Haakon; 02-01-2006 at 03:40 AM.