Results 1 to 6 of 6

How severely can Bots hamper a site?

This is a discussion on How severely can Bots hamper a site? within the General Discussion forums, part of the vBSEO Google/Yahoo Sitemap category; I was wondering why my new server yesterday was running at a CPU load average of 16.0. Could Google Bots ...

  1. #1
    Senior Member
    Real Name
    Mike
    Join Date
    Aug 2006
    Posts
    209
    Liked
    3 times

    How severely can Bots hamper a site?

    I was wondering why my new server yesterday was running at a CPU load average of 16.0.

    Could Google Bots really hamper a site so severely like this? It was going crazy on my site yesterday as I saw in the logs. And as you can see here -

    (16,000 google bots in one day)
    Attached Thumbnails Attached Thumbnails googlebot.jpg  

  2. #2
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    25,689
    Liked
    157 times
    Do you have some cron job scheduled at the time when server load jumped?
    Oleg Ignatiuk / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


  3. #3
    Senior Member
    Real Name
    Mike
    Join Date
    Aug 2006
    Posts
    209
    Liked
    3 times
    Quote Originally Posted by Oleg Ignatiuk View Post
    Do you have some cron job scheduled at the time when server load jumped?
    Negative...why?

    you dont think the bots could of caused this? It was a large spike that lasted for like 4 minutes..and then everything returned back to normal.

  4. #4
    Senior Member briansol's Avatar
    Real Name
    Brian
    Join Date
    Apr 2006
    Location
    Central CT, USA
    Posts
    6,981
    Liked
    8 times
    16? holy crap. most servers crash aroun 5. lol

  5. #5
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    25,689
    Liked
    157 times
    It is unlikely that all these hits happened in a short timespan. You can check this in your access log though.
    Oleg Ignatiuk / Crawlability Inc.
    vBSEO 3.6.0 GOLD Released!
    Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!


  6. #6
    Senior Member
    Real Name
    Matt
    Join Date
    May 2006
    Posts
    973
    Liked
    3 times
    offtopic: My old sever never crashed hardly it was a really crap sever aswell and it could go up to 70 and would be so slow but wouldnt crash, but my current sever which is a lot better gets to around 8 and crashes, so its crashing alot more then my old cheaper sever which in return is quite annoying lol.


    It could have been a short dos attack from a script kiddy if it only went to 16, the best way to check would be the logs.

Similar Threads

  1. Search Engine Bots
    By Mike in forum General Discussion
    Replies: 1
    Last Post: 11-16-2005, 04:02 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •