JADM

Awesome!

Hey, does the MSN bot seem to be the slowest to kick in for you too? I always notice Google seems to hit the majority of my content first, then Yahoo, and like a week later MSN finally bombards my site.

Joking, but what is really going on is msn hit me on the third and fourth day with 98 and 93 crawls...since then only 2 or 3 times a day. So, I am waiting for them to catch up, I guess. I have only been running both products for 20 days, so it is early.

I really wish I understood more about the Sitemap generator. I suspect part is that with the algorithms and such being closely held secrets and things changing, it may be hard to figure this stuff out. But from the vBSEO side, I wish they would explain the products typical performane a little better. I can only figure they are busy with other stuff or else don't want to give out too much info for fear of getting their program ripped off. For me, knowing and understanding things a little more would help me in figuring out if my results are as to be expected. My big fear, after reading different reviews on the forums is that I end being one of those guys who wait 7 months and then is like, "hey, this isn't working!" and come to find out I am doing something dumb. Still, I am happy / confident things will go well. Except for today with my traffic off by 2/3 (I am blaming football games).

where are you getting all these bot stats from?

Goto AdminCP >> VBSEO >> VBSEO Sitemap Reports >> Show SE Bots Activity Log .

That's how I do it, though, you could go to server logs and see, too (I guess...but I think it is not broken down as well, segregated from the other visitors).

I'm going to go ahead and give vBSEO and vBSiteMap Generator credit for getting pages indexed within 1-4 hours upon creation (with a little help from feedburner.) Yay PingShot

All my tags were indexed within 24 hours too when i upgraded to 3.7.

Since your crawling was about 50% member profiles vs. threads, that is one of the reasons I (personally) prefer blogging those member profile pages. I'd rather have the bots prioritize on the threads.

Joe,

I have been trying to figure that out. If there are a 100 bots crawling your site on a given day, and 50% crawl profile, if you had it to do over again and you block profiles, would the same 100 bots come to your site? Or would the SE say, "hey, there is no profile to crawl, let's only send 50 bots"?