Results 1 to 2 of 2

The best robots.txt

This is a discussion on The best robots.txt within the General Discussion forums, part of the vBSEO SEO Plugin category; What would you say the best robots.txt config (values) are for use with vbulletin and vbSEO ? I say we ...

  1. #1
    Senior Member MaestroX's Avatar
    Real Name
    Chris A.
    Join Date
    Jan 2007
    Location
    UK
    Posts
    125
    Liked
    0 times
    Blog Entries
    1

    The best robots.txt

    What would you say the best robots.txt config (values) are for use with vbulletin and vbSEO?

    I say we come up with a definative solution if there isn't one :P

  2. #2
    Junior Member dtv100's Avatar
    Real Name
    julie
    Join Date
    May 2006
    Posts
    22
    Liked
    0 times
    this is the one I use :


    Code:
     
    #ALL BOTS
    User-agent: *
    Disallow: /forums/admincp/
    Disallow: /forums/ajax.php
    Disallow: /forums/attachments/
    Disallow: /forums/clientscript/
    Disallow: /forums/cpstyles/
    Disallow: /forums/images/
    Disallow: /forums/includes/
    Disallow: /forums/install/
    Disallow: /forums/modcp/
    Disallow: /forums/subscriptions/
    Disallow: /forums/customavatars/
    Disallow: /forums/customprofilepics/
    Disallow: /forums/announcement.php
    Disallow: /forums/attachment.php
    Disallow: /forums/calendar.php
    Disallow: /forums/cron.php
    Disallow: /forums/editpost.php
    Disallow: /forums/external.php
    Disallow: /forums/faq.php 
    Disallow: /forums/frm_attach
    Disallow: /forums/image.php
    Disallow: /forums/inlinemod.php 
    Disallow: /forums/joinrequests.php
    Disallow: /forums/login.php 
    Disallow: /forums/member.php? 
    Disallow: /forums/memberlist.php
    Disallow: /forums/misc.php
    Disallow: /forums/moderator.php
    Disallow: /forums/newattachment.php
    Disallow: /forums/online.php
    Disallow: /forums/payment_gateway.php
    Disallow: /forums/payments.php
    Disallow: /forums/poll.php
    Disallow: /forums/postings.php 
    Disallow: /forums/register.php
    Disallow: /forums/report.php
    Disallow: /forums/reputation.php
    Disallow: /forums/search.php
    Disallow: /forums/sendmessage.php
    Disallow: /forums/showgroups.php
    Disallow: /forums/subscription.php
    Disallow: /forums/usercp.php
    Disallow: /forums/threadrate.php
    Disallow: /forums/usercp.php
    Disallow: /forums/usernote.php
    Disallow: /forums/flashchat.php
    Disallow: /forums/printthread.php
    Disallow: /forums/Order_Page.php
    Disallow: /forums/chat/
    Disallow: /forums/downloads/
    Disallow: /forums/season/
    Disallow: /forums/arcade/
    Disallow: /forums/private.php

Similar Threads

  1. robots.txt
    By Zenith in forum General Discussion
    Replies: 64
    Last Post: 12-01-2010, 08:52 PM
  2. MSNbot - robots.txt
    By curriertech in forum General Discussion
    Replies: 1
    Last Post: 10-05-2006, 09:29 AM
  3. Temp robots.txt Brand New Forum?
    By rmjvol in forum Pre-Sales Questions
    Replies: 7
    Last Post: 08-26-2006, 02:53 AM
  4. robots.txt entries
    By shaochun in forum General Discussion
    Replies: 5
    Last Post: 12-10-2005, 08:18 PM
  5. Possible to use Disallow: /*?pp=10 in robots.txt?
    By PageUp in forum General Discussion
    Replies: 3
    Last Post: 11-03-2005, 10:01 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •