Results 1 to 3 of 3

Should I prohibit these pages from being crawled/indexed?

This is a discussion on Should I prohibit these pages from being crawled/indexed? within the General Discussion forums, part of the vBulletin SEO Discussion category; Should I prohibit these pages from being crawled in the robots file? http://creditcardforum.com/credit-ca...k-tips-tricks/ http://creditcardforum.com/credit-ca...tions-issuers/ http://creditcardforum.com/off-topic/ Also, if I were to ...

  1. 01-27-2009 09:45 PM #1
    MTD
    MTD is offline
    Senior Member
    Real Name
    Mike
    Join Date
    Apr 2008
    Posts
    141
    Liked
    0 times

    Question Should I prohibit these pages from being crawled/indexed?

    Should I prohibit these pages from being crawled in the robots file?

    http://creditcardforum.com/credit-ca...k-tips-tricks/
    http://creditcardforum.com/credit-ca...tions-issuers/
    http://creditcardforum.com/off-topic/

    Also, if I were to change the category titles, of say, this one:
    http://creditcardforum.com/credit-ca...k-tips-tricks/
    To something different, would that change the old URL above?
    Reply With Quote Reply With Quote  
  2. 01-27-2009 11:03 PM #2
    Brandon_R
    Brandon_R is offline
    Senior Member
    Real Name
    Brandon
    Join Date
    Oct 2008
    Posts
    124
    Liked
    0 times
    You should leave it open to bots, seems like you have some valuable content there ex - http://creditcardforum.com/finance-c...dit-cards.html
    Reply With Quote Reply With Quote  
  3. 01-27-2009 11:11 PM #3
    MTD
    MTD is offline
    Senior Member
    Real Name
    Mike
    Join Date
    Apr 2008
    Posts
    141
    Liked
    0 times
    Hmm.... so if I block those folders, it blocks anything inside them. Good point I blanked on that very important detail!


    Reply With Quote Reply With Quote  
« Come Back Later Notice | Have social bookmarks helped your site? »

Similar Threads

  1. http:// and https:// pages being crawled & indexed
    By Coop1979 in forum Troubleshooting
    Replies: 8
    Last Post: 02-13-2012, 08:00 PM
  2. 00,000 Yahoo pages and 50,000 Google pages indexed
    By len backus in forum Analysis: Traffic & SERPS
    Replies: 6
    Last Post: 02-03-2010, 01:20 AM
  3. Indexed Pages?
    By ijob in forum General Discussion
    Replies: 8
    Last Post: 10-21-2008, 12:39 PM
  4. Too many pages indexed?
    By Corey in forum General Discussion
    Replies: 8
    Last Post: 07-07-2008, 07:45 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Forum Rules