Results 1 to 3 of 3

should we disable profile members from SE Bots?

This is a discussion on should we disable profile members from SE Bots? within the General Discussion forums, part of the vBSEO Google/Yahoo Sitemap category; Hi There i upgraded to vbseo 3.6 RC2 and to the new VBseo sitemap generator in the SE Bots logs ...

  1. #1
    Senior Member
    Real Name
    Seleno
    Join Date
    Mar 2007
    Posts
    255
    Liked
    0 times
    Blog Entries
    1

    Lightbulb should we disable profile members from SE Bots?

    Hi There
    i upgraded to vbseo 3.6 RC2 and to the new VBseo sitemap generator

    in the SE Bots logs i see that
    Google Bots check 13,971 members profile !
    showthread.php: 5,875
    forumdisplay.php: 377
    tags.php: 447
    converse.php: 304

    should i change anyting or disable the bots for members profile?
    so that google check the threads better than member profile??
    and how to do this
    need yr advice
    thanx

  2. #2
    vBSEO Staff Brian Cummiskey's Avatar
    Real Name
    Brian Cummiskey
    Join Date
    Jul 2009
    Location
    btwn NYC and Boston
    Posts
    12,789
    Liked
    657 times
    Blog Entries
    2
    I don't feel it is necessary to block user pages (ie, with robots.txt), but I also don't feel you should include them in your sitemap.

  3. #3
    Senior Member
    Real Name
    Seleno
    Join Date
    Mar 2007
    Posts
    255
    Liked
    0 times
    Blog Entries
    1
    Hi There
    i was looking at Google Bot on Whoi's online for forum i noticed that 10 Google Bots watching members profiles
    while only 1 Google Bot watching old thread and its very slow !
    while Msn Bots watching much threads !

    what should i do?
    should i stop crawl for members profile? this will give more chance for threads?
    thanx

Similar Threads

  1. Disable Relevant Replacements for Members
    By Sadik in forum Relevant Replacements
    Replies: 2
    Last Post: 11-22-2010, 01:35 AM
  2. Members VS Bots
    By DieselMinded in forum Analysis: Traffic & SERPS
    Replies: 3
    Last Post: 05-20-2008, 05:45 PM
  3. Mod: Allow bots to act as members
    By glorify in forum General Discussion
    Replies: 1
    Last Post: 02-17-2006, 11:37 AM
  4. Deleted members profile results in /members/.html
    By Ace Shattock in forum Bug Reporting
    Replies: 1
    Last Post: 10-13-2005, 07:48 AM
  5. Members Profile members/[user_id].html 3 BUGS
    By TopAs in forum Bug Reporting
    Replies: 2
    Last Post: 07-29-2005, 01:28 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •