vBulletin 4, the most powerful community software + vBSEO 3.5, the ultimate SEO solution = Your ultimate platform for 2010 and beyond. Click below to learn more.

Results 1 to 12 of 12

Google Webmaster tools- crawl rate

This is a discussion on Google Webmaster tools- crawl rate within the General Discussion forums, part of the vBulletin SEO Discussion category; I have just started using Google Webmaster tools...I stumbled upon "Set crawl rate." It seemed to make sense to set ...

  1. #1
    JADM
    JADM is offline
    Member JADM's Avatar
    Real Name
    Jason
    Join Date
    Dec 2007
    Posts
    72
    Blog Entries
    18

    Google Webmaster tools- crawl rate

    I have just started using Google Webmaster tools...I stumbled upon "Set crawl rate." It seemed to make sense to set to faster crawl. Any downside (other than server load) to this? I assume the server load effect is not that great on a smaller forum like mine. Anything I am missing here?
    Jason

    Physical Evaluation Board Forum
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  2. #2
    briansol
    briansol is offline
    Senior Member briansol's Avatar
    Real Name
    Brian
    Join Date
    Apr 2006
    Location
    Central CT, USA
    Posts
    6,983
    usually you have medium and slow as options, not fast.

    unless you're doing 5000 posts a day, i would leave it at medium.
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  3. #3
    JADM
    JADM is offline
    Member JADM's Avatar
    Real Name
    Jason
    Join Date
    Dec 2007
    Posts
    72
    Blog Entries
    18
    briansol,

    Thanks for the reply. I am looking at Dashboard >> tools >> Set crawl rate. It says "We've detected that Googlebot is limiting the rate at which it crawls pages on your site to ensure it doesn't use too much of your server's resources. If your server can handle additional Googlebot traffic, we recommend that you choose Faster below." I am WAYYYY below 5,000 post/day . But is there a reason NOT to do this? Or do you think it just doesn't matter?
    Jason

    Physical Evaluation Board Forum
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  4. #4
    briansol
    briansol is offline
    Senior Member briansol's Avatar
    Real Name
    Brian
    Join Date
    Apr 2006
    Location
    Central CT, USA
    Posts
    6,983
    it will just eat more processor on your server. I see no reason to change it from normal. Ive had mine there since day 1, and the faster option isn't even available to me.

    Faster At this time, crawl rate is not a factor in your site's crawl.
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  5. #5
    seometer
    seometer is offline
    Junior Member
    Real Name
    SEO Meter
    Join Date
    Jan 2008
    Posts
    3
    If you choose to use that option, you have to re-set the crawl rate once every 90 days.
    Otherwise, the crawl rate will be back to its default rate.

    More about on Google's crawl rate:
    What Determines Search Engine’s Crawl Rate? - Search Engine Optimization Blog
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  6. #6
    JADM
    JADM is offline
    Member JADM's Avatar
    Real Name
    Jason
    Join Date
    Dec 2007
    Posts
    72
    Blog Entries
    18
    I'll take a look at the blog, but do you have an idea of why this might be important. If it is the same info sent the same day, I would think this is negligible/not important. But then, I am stumped on why they would offer this option if it has no signifigance.
    Jason

    Physical Evaluation Board Forum
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  7. #7
    Notorious
    Notorious is offline
    Senior Member
    Real Name
    Dhillon
    Join Date
    Apr 2006
    Posts
    327
    When I enabled fast option, googlebot hit rate increased by 15-20%.
    If your server can handle that much load,Enable it
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  8. #8
    class101
    class101 is offline
    Senior Member
    Real Name
    A.D
    Join Date
    Oct 2007
    Location
    France
    Posts
    159
    Quote Originally Posted by seometer View Post
    If you choose to use that option, you have to re-set the crawl rate once every 90 days.
    Otherwise, the crawl rate will be back to its default rate.

    More about on Google's crawl rate:
    What Determines Search Engine’s Crawl Rate? - Search Engine Optimization Blog
    Hey good job on your site SEOmaster and thanks for the add
    security community | free directory
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  9. #9
    JADM
    JADM is offline
    Member JADM's Avatar
    Real Name
    Jason
    Join Date
    Dec 2007
    Posts
    72
    Blog Entries
    18
    I am on shared hosting at GoDaddy...not really sure how to tell if it can handle it....so, I am going to set it for fast crawl and see what happens! (Probably not the preferred method, but hey, nothing ventured, nothing gained)!
    Jason

    Physical Evaluation Board Forum
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  10. #10
    class101
    class101 is offline
    Senior Member
    Real Name
    A.D
    Join Date
    Oct 2007
    Location
    France
    Posts
    159
    no you can JADM, even with a fast google crawl rate , I see far much bots from msn live or yahoo.
    security community | free directory
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  11. #11
    seometer
    seometer is offline
    Junior Member
    Real Name
    SEO Meter
    Join Date
    Jan 2008
    Posts
    3
    Quote Originally Posted by JADM View Post
    I am stumped on why they would offer this option if it has no signifigance.
    I guess they wanted to have some sort of human input on their crawling algo, esp in case crawling rate got unduly influenced by temporary screwy hosting setup.
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
  12. #12
    seometer
    seometer is offline
    Junior Member
    Real Name
    SEO Meter
    Join Date
    Jan 2008
    Posts
    3
    Quote Originally Posted by class101 View Post
    Hey good job on your site SEOmaster and thanks for the add
    Thanks. My pleasure.
    Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
    Reply With Quote Reply With Quote  
« hwo to check indexes | Load issue »

Similar Threads

  1. Google webmaster tools - Network Uncreachable
    By OnR in forum Türkçe
    Replies: 1
    Last Post: 08-22-2007, 06:15 AM
  2. Google Webmaster Tools
    By OnR in forum Türkçe
    Replies: 2
    Last Post: 07-12-2007, 12:42 PM
  3. Weird entry in Google webmaster tools.
    By Dave Hybrid in forum General Discussion
    Replies: 5
    Last Post: 05-30-2007, 11:45 AM
  4. New SEO Tools: Robots.txt Builder, Term Targeting & Crawl Test
    By Eros in forum General Discussion
    Replies: 0
    Last Post: 04-21-2007, 06:30 AM
  5. Google Webmaster Tools - Matt Cutts Video
    By Joe Ward in forum General Discussion
    Replies: 2
    Last Post: 11-01-2006, 02:33 AM

Tags for this Thread