Our robots.txt contains all the *.php files that you recommend to include. My question is if it says /forums/members.php - and then vbseo rewrites it as /forums/members/blah then is it still blocking? If not, then do we need to add all the rewritten urls if we want to block the spiders?



LinkBack URL
About LinkBacks





Reply With Quote

