Results 1 to 3 of 3

Robots.txt query

This is a discussion on Robots.txt query within the General Discussion forums, part of the vBulletin SEO Discussion category; Hi all ( again lol ) Having moved my site to root, im just wanting to make sure my robots.txt ...

  1. #1
    Member magz's Avatar
    Real Name
    Tony
    Join Date
    Dec 2006
    Posts
    60
    Liked
    2 times

    Robots.txt query

    Hi all ( again lol )

    Having moved my site to root, im just wanting to make sure my robots.txt file is accurate.

    Could someone advise on the following please

    Code:
    User-agent: Mediapartners-Google*
    Disallow: /cgi-bin/
    Disallow: /clientscript/
    Disallow: /includes/
    Disallow: /install/
    Disallow: /photopost/
    Disallow: /printthread.php
    Disallow: /subscription.php
    Disallow: /payments.php
    Disallow: /profile.php
    Disallow: /faq.php
    Disallow: /calendar.php
    Disallow: /private.php
    Disallow: /sendmessage.php
    Disallow: /sendmessage.php?do=
    Disallow: /showgroups.php
    Disallow: /reputation.php
    Disallow: /report.php
    Disallow: /threadrate.php
    Disallow: /postings.php
    Disallow: /newthread.php
    Disallow: /newreply.php
    Disallow: /register.php
    Disallow: /login.php
    Disallow: /faq.php
    Disallow: /image.php
    Disallow: /cron.php
    Disallow: /joinrequests.php
    Disallow: /usercp.php
    Disallow: /sitemap/
    Disallow: /archive/
    Should any mention of admin and mod cp be in here?

    tia

  2. #2
    Senior Member
    Real Name
    Ceri May
    Join Date
    Jul 2009
    Location
    United Kingdom
    Posts
    1,726
    Liked
    15 times
    Blog Entries
    1
    Hi,

    It is believed that including admincp and modcp in the robots.txt file is a security issue as it is globaly readable by anyone. You don't want to be telling people what you don't want to be seen just what you don't want indexed.

    As long as there are no links to your admincp / modcp panel globaly viewable and considering there is no content on the page it is very unlikely that it will be indexed anyway.

    Ceri

  3. #3
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,782
    Liked
    648 times
    Blog Entries
    2
    Correct. NEVER publish the location of anything 'secret'. It's actually a pci violation.

    The rest of your settings look fine.
    Brian Cummiskey / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


Similar Threads

  1. Replies: 6
    Last Post: 10-19-2009, 11:55 AM
  2. Redirecting /forums/robots.txt to /robots.txt - Is it good?
    By MadK in forum Custom Rewrite Rules
    Replies: 6
    Last Post: 08-22-2008, 07:29 PM
  3. Bad Query
    By KirbyDE in forum Bug Reporting
    Replies: 4
    Last Post: 04-04-2008, 07:41 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •