vBulletin 4, the most powerful community software + vBSEO 3.5, the ultimate SEO solution = Your ultimate platform for 2010 and beyond. Click below to learn more.

Results 1 to 3 of 3

Robots.txt help

This is a discussion on Robots.txt help within the General Discussion forums, part of the vBulletin SEO Discussion category; I want to block the memberlist pages from being indexed as they are all junk but keep member profiles indexed. ...

  1. 05-28-2007 07:27 PM #1
    Dave Hybrid
    Dave Hybrid is offline
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    369

    Robots.txt help

    I want to block the memberlist pages from being indexed as they are all junk but keep member profiles indexed.

    Is this correct?

    Disallow: /members/list/

    Thanks guys.

    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  2. 05-29-2007 08:36 AM #2
    Dave Hybrid
    Dave Hybrid is offline
    Senior Member Dave Hybrid's Avatar
    Real Name
    Dave Hybrid
    Join Date
    Mar 2007
    Location
    UK
    Posts
    369
    Anyone?

    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  3. 05-29-2007 10:16 AM #3
    Oleg Ignatiuk
    Oleg Ignatiuk is offline
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    24,527
    Hello,

    yes, your code is correct (assuming that your forum is in domain root).
    Oleg Ignatiuk / Crawlability Inc.
    Sneek Preview Video of the new Control Panel

    vBSEO 3.5 RC2 (Final)- Released for your Evaluation
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
« Article Script to work with vbulletin + vbseo | Weird entry in Google webmaster tools. »

Similar Threads

  1. Robots.txt now support the sitemap file
    By Mohamed Mostafa in forum General Discussion
    Replies: 35
    Last Post: 08-24-2007, 09:33 AM
  2. googlebot ignoring robots.txt
    By Doug Nelson in forum General Discussion
    Replies: 16
    Last Post: 05-08-2007, 10:30 AM
  3. add sitemap to robots.txt for autodiscovery
    By GrendelKhan{TSU} in forum General Discussion
    Replies: 2
    Last Post: 04-13-2007, 02:21 PM
  4. MSNbot - robots.txt
    By curriertech in forum General Discussion
    Replies: 1
    Last Post: 10-05-2006, 08:29 AM
  5. Temp robots.txt Brand New Forum?
    By rmjvol in forum Pre-Sales Questions
    Replies: 7
    Last Post: 08-26-2006, 01:53 AM