Multiple IP's per BOT.

Is it normal for search engine bots to have numerous different IP addresses?

For instance, im currently getting hundreds of more people "online" then is normal.

I have a screen shot of my bot activity below. Is 40,000 google bots a day normal?

This is weird because in my Urchin stats, it says that its Yahoo that is slamming me.

per my urchin stats yesterday :

1. Mozilla Compatible Agent|5.0 - (unknown) 42,548 Hits
2. Internet Explorer 6.0 - Windows XP 17,201 Hits
3. Internet Explorer 7.0 - Windows XP 11,496 Hits

so im guessing all these hits are bots? I just dont know why their showin up as Guests, as I have that option disabled. (bots showing up as guests)

maybe I have to block the bots manually? But they all have different IP's.

so confusing.

Hello Mike ,

If you block bots , your site will not get crawled at all. Also i advise you to not to include showpost to your sitemaps and block showpost pages via robots.txt since they will only create duplicate content.

I know how to block showpost.php from robots.txt. Isnt it controversial whether or not to blot showpost.php? as were not suppose to block showthread.php but can block showpost.php? how does the engines read the content of the page?

And im not sure what you mean by not including them in the sitemaps? Im guessing blocking it through robots.txt does the job.

vBSEO Sitemap Generator has a setting : Include Show Post pages . You shall disable that option and also block showpost pages from robots.txt . As showthread pages will have your main content you want bots to focus on showthread pages.

I have archive disabled though. Still ok to block showpost ?

Can someone explain how the post still shows up correctly for the bots by only using showthread ? Like how does showthread have all that data in it to read the content of each showpost? It has to click on a post url to view everything, right? please explain how this is done specifically, as im very hesitant to blocking showpost.php completely.