Results 1 to 7 of 7

Crawl rate - Google webmaster tools

This is a discussion on Crawl rate - Google webmaster tools within the Analysis: Traffic & SERPS forums, part of the vBulletin SEO Discussion category; Had a look in Google webmaster tools today (been a while since I last looked) and found 'Crawl rate' under ...

  1. #1
    Senior Member woostar's Avatar
    Real Name
    woostar
    Join Date
    Apr 2007
    Posts
    786
    Liked
    5 times

    Crawl rate - Google webmaster tools

    Had a look in Google webmaster tools today (been a while since I last looked) and found 'Crawl rate' under 'Setting' (last time I looked this was grayed out).

    So I've put the crawl rate up 100% too 1 request every second (rather than 1 every 2 seconds which is default). I'm guesssing results should start appearing twice as fast now, right?

    Obviously I'll be keeping an eye on server resources, but if it is negligiable would a higher crawl rate be preferable?
    2011 Average: 1 post every 5 seconds... 24/7

  2. #2
    Senior Member briansol's Avatar
    Real Name
    Brian
    Join Date
    Apr 2006
    Location
    Central CT, USA
    Posts
    6,981
    Liked
    8 times
    neato. mine has the new tool too.

    i was default at 1/1 when i opened it up.

    i bumped mine to 2.0/.5 and we'll see what happens.

  3. #3
    nfn
    nfn is offline
    Senior Member
    Real Name
    Nuno
    Join Date
    Feb 2008
    Location
    Portugal
    Posts
    276
    Liked
    1 times
    Hi,

    I have my forum at domain.com/forum and my sitemap at domain.com/forum/sitemap_index.xml.gz

    Because of this i can't use this feature since the sitemap isn't in the domain root (domain.com)

    What should I do? Should I rewite domain.com/forum/sitemap_index.xml.gz to domain.com/sitemap_index.xml.gz?

    I test it with this rule and this is the site map index:

    Code:
    RewriteRule ^((urllist|sitemap_).*\.(xml|txt)(\.gz)?)$ forum/vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]
    The site maps are shown as follow:

    Code:
    <sitemapindex xsi:schemaLocation="             http://www.sitemaps.org/schemas/sitemap/0.9             http://www.sitemaps.org/schemas/sitemap/09/siteindex.xsd">
    −
    <sitemap>
    −
    <loc>
    http://www.portaldasviagens.com/forum/sitemap_1.xml.gz
    </loc>
    <lastmod>2009-01-06T02:08:28+00:00</lastmod>
    </sitemap>
    </sitemapindex>
    Can this be a problem? some help will be appreciated

  4. #4
    Senior Member woostar's Avatar
    Real Name
    woostar
    Join Date
    Apr 2007
    Posts
    786
    Liked
    5 times
    don't think this has anything to do with sitemaps
    2011 Average: 1 post every 5 seconds... 24/7

  5. #5
    nfn
    nfn is offline
    Senior Member
    Real Name
    Nuno
    Join Date
    Feb 2008
    Location
    Portugal
    Posts
    276
    Liked
    1 times
    Well, If I add a site "domain.com" it tells me that sitemap can't be located at "domain.com/forum/sitemap_index.xml.gz" only at "domain.com/sitemap_index.xml.gz"

    If i submit "domain.com/forum/" everything is ok, but then it says "Crawl rate Setting is restricted to root level domains only"

    What should I do?

  6. #6
    nfn
    nfn is offline
    Senior Member
    Real Name
    Nuno
    Join Date
    Feb 2008
    Location
    Portugal
    Posts
    276
    Liked
    1 times
    Ok .. I'v tested again and it's working.
    Maybe my fault.

    Should I delete the old one?

  7. #7
    Member
    Real Name
    Luke McCoy
    Join Date
    Oct 2007
    Posts
    57
    Liked
    0 times
    I'd like to hear more about this and what it does before messing with any of the settings but I did see it in there yesterday.
    USA Carry - Open and Concealed Carry Information

Similar Threads

  1. Google Webmaster Tools reporting that 404 is a 200
    By Kanustep in forum Troubleshooting
    Replies: 2
    Last Post: 05-15-2008, 11:38 PM
  2. Sitemap 2.1 - google webmaster tools
    By mljask in forum General Discussion
    Replies: 10
    Last Post: 01-31-2008, 02:36 PM
  3. Google Webmaster tools- crawl rate
    By JADM in forum General Discussion
    Replies: 11
    Last Post: 01-11-2008, 06:26 PM
  4. Google Webmaster Tools
    By OnR in forum Türkçe
    Replies: 2
    Last Post: 07-12-2007, 12:42 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •