I've been noticing a ton of bots spidering my sites lately and it's really starting to drive up the web server load. These are not just search engine spiders, but also ad spiders that scan pages to determine the best ad to put on the page. Some of these spiders have over 100 connections each to apache.

My question is - should I limit the amount of connections each IP has to apache? Are search engine rankings going to drop if the spiders can't index the pages fast enough?