What directories should I disallow in my robots.txt. Is there a general common list used for vb forums?
This is a discussion on Robots.txt Directories to Exclude? within the General Discussion forums, part of the vBulletin SEO Discussion category; What directories should I disallow in my robots.txt. Is there a general common list used for vb forums?...
What directories should I disallow in my robots.txt. Is there a general common list used for vb forums?
I found this somewhere else, not here, and am not sure if this is a good list. I would like an SEO Experts opinion for people running VBSEO
User-agent: *
Disallow: /attachment.php
Disallow: /avatar.php
Disallow: /editpost.php
Disallow: /misc.php
Disallow: /moderator.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /poll.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /report.php
Disallow: /sendtofriend.php
Disallow: /threadrate.php
Disallow: /usercp.php
Disallow: /admin/
Disallow: /modcp/
Disallow: /sendmessage.php
Disallow: /register.php
Disallow: /subscription.php
i re-write everything i want indexed with no extention (/folder/page-123/) and then i disallow all *.html, *.php, etc etc