Results 1 to 2 of 2

Robots.txt and the new url formats...

This is a discussion on Robots.txt and the new url formats... within the General Discussion forums, part of the vBSEO SEO Plugin category; Do we still do it like this: User-agent: * Disallow: /forum/editpost.php Disallow: /forum/moderator.php Disallow: /forum/newreply.php Disallow: /forum/newthread.php Disallow: /forum/online.php Disallow: ...

  1. #1
    Senior Member BamaStangGuy's Avatar
    Real Name
    Brent Wilson
    Join Date
    Aug 2005
    Location
    Huntsville, Alabama
    Posts
    2,484
    Liked
    1 times

    Robots.txt and the new url formats...

    Do we still do it like this:

    User-agent: *
    Disallow: /forum/editpost.php
    Disallow: /forum/moderator.php
    Disallow: /forum/newreply.php
    Disallow: /forum/newthread.php
    Disallow: /forum/online.php
    Disallow: /forum/poll.php
    Disallow: /forum/postings.php
    Disallow: /forum/printthread.php
    Disallow: /forum/private.php
    Disallow: /forum/report.php
    Disallow: /forum/sendtofriend.php
    Disallow: /forum/threadrate.php
    Disallow: /forum/usercp.php
    Disallow: /forum/admincp/
    Disallow: /forum/modcp/
    Disallow: /forum/sendmessage.php
    Disallow: /forum/subscription.php
    Even though the URLs now do not end in php?

    Also can you please tell me if the above files are good to block and if any additonal ones should be added?

  2. #2
    Senior Member
    Real Name
    Keith Cohen
    Join Date
    Jul 2005
    Location
    Raleigh, NC USA
    Posts
    6,147
    Liked
    13 times
    Check out Michael's post here: "should" I use a robots.txt file?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •