Results 1 to 5 of 5

Is my robots.tx file ok?

This is a discussion on Is my robots.tx file ok? within the General Discussion forums, part of the vBSEO SEO Plugin category; I'm following a vb optimization article I found on vbulletin.org and one of the suggestions is to update the robots.txt ...

  1. #1
    Member swiftor's Avatar
    Real Name
    Joseph Alminawi
    Join Date
    Jun 2009
    Posts
    60
    Liked
    0 times

    Is my robots.tx file ok?

    I'm following a vb optimization article I found on vbulletin.org and one of the suggestions is to update the robots.txt with the below. Any thoughts on whether this will hurt seo or not?

    User-agent: *

    Disallow: /admincp/
    Disallow: /clientscript/
    Disallow: /cpstyles/
    Disallow: /customavatars/
    Disallow: /customprofilepics/
    Disallow: /images/
    Disallow: /modcp/
    Disallow: /ajax.php
    Disallow: /attachment.php
    Disallow: /calendar.php
    Disallow: /cron.php
    Disallow: /editpost.php
    Disallow: /global.php
    Disallow: /image.php
    Disallow: /inlinemod.php
    Disallow: /joinrequests.php
    Disallow: /login.php
    Disallow: /member.php
    Disallow: /memberlist.php
    Disallow: /misc.php
    Disallow: /moderator.php
    Disallow: /newattachment.php
    Disallow: /newreply.php
    Disallow: /newthread.php
    Disallow: /online.php
    Disallow: /poll.php
    Disallow: /postings.php
    Disallow: /printthread.php
    Disallow: /private.php
    Disallow: /profile.php
    Disallow: /register.php
    Disallow: /report.php
    Disallow: /reputation.php
    Disallow: /search.php
    Disallow: /sendmessage.php
    Disallow: /showgroups.php
    Disallow: /subscription.php
    Disallow: /threadrate.php
    Disallow: /usercp.php
    Disallow: /usernote.php

  2. #2
    Junior Member
    Real Name
    Carlos Pastor
    Join Date
    Aug 2010
    Posts
    26
    Liked
    0 times
    hello:

    add:

    calendar.php
    /archive/
    /sitemap/

  3. #3
    vBSEO Staff Andrés Durán Hewitt's Avatar
    Real Name
    Andrés Durán
    Join Date
    Jul 2009
    Location
    Costa Rica
    Posts
    3,393
    Liked
    411 times
    Blog Entries
    1
    Hello Joseph,

    You never should include your "admincp" and "modcp" directories for security purposes:

    Easy Security Tips for vBSEO customers

    For the rest, your .htaccess file looks fine.
    Andrés Durán / Crawlability Inc.
    ˇvBSEO 3.6.0 GOLD Liberado!
    Inaugurando el NUEVO vBSEO Sitemap Generator 3.0. - ˇAHORA disponible para Clientes de vBSEO!

    Síguenos en: Facebook | Síguenos en: Twitter


  4. #4
    vBSEO Staff Marco Mamdouh's Avatar
    Real Name
    Marco Mamdouh
    Join Date
    May 2010
    Location
    Egypt
    Posts
    3,143
    Liked
    110 times
    Here's some good tips about robots.txt file : Tutorial how to make a good robots.txt file

  5. #5
    Member
    Real Name
    Darren
    Join Date
    Oct 2009
    Posts
    70
    Liked
    3 times
    it's also a good idea to rename your "admincp" to another name.. you will need to edit the config.php to do this, but yes your robots.txt file looks relativity good.

Similar Threads

  1. how to configure the robots.txt file
    By universo in forum General Discussion
    Replies: 2
    Last Post: 12-23-2009, 05:30 PM
  2. Robots.txt file changes
    By Blind Dragon in forum General Discussion
    Replies: 3
    Last Post: 09-03-2009, 01:00 PM
  3. Robots.txt now support the sitemap file
    By Mohamed Mostafa in forum General Discussion
    Replies: 35
    Last Post: 08-24-2007, 09:33 AM
  4. Please review my robots.txt file...
    By Sonnie in forum General Discussion
    Replies: 11
    Last Post: 09-08-2006, 03:30 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •