sitemap & spiders

Can somebody please tell me what this setting should be set at?

Max URLs in Sitemap Files - I currently have it set at 40000 although default is set at 50000 (Google limit)

Also I set this up just under a week ago but I havent seen vast amount of spiders being on the site, don't get me wrong they are coming on but not in silly numbers like some people mention.

Is this still ok?

Oh also can you let me know if this url looks ok to you? I just want to make sure that the urls are set correctly.

http://www.chatfootballforum.com/pre...html#post11162

We break up your sitemap, because of the huge amount of pages forums have, into smaller files. Google and the others read a file at a time. You should be fine with the default setting.

You should verify your sitemap:
Verifying your vBSEO Google/Yahoo Sitemap Generator is Installed Correctly

Also check your Google Webmaster Tools to make sure there are no errors:
Google Webmaster Tools - Recommended for All vBSEO Customers

Beyond that, you need to give it more time so that we can assess your growth. Stick with it.

I upped mine by 10000 from 30000 to 40000, should I go back to default or is it ok at 40?

Looks like it's ok? http://www.chatfootballforum.com/sitemap_index.xml.gz

Google webmaster last downloaded 8hrs ago status was ok and 30844 urls submitted so looks good I think.

It should be fine. Leave it and see how it goes for about 2 weeks.

Ok thanks Joe, is there any reason as to why my threads are .html and yours aren't?

We selected another option that tests threads as virtual directories instead.

Yours is the default.

i run 10,000 with a 30 sec pause between files.

i get sql locks/high load if i just let it go.

Perhaps I should change it back to 30,000 then Brian. I was thinking the more the better but guess it isn't the case.

Originally Posted by briansol

i run 10,000 with a 30 sec pause between files.

i get sql locks/high load if i just let it go.

Brian,

Have you done any testing to discover if Google will index more URLs when the sitemap files are larger or smaller? The reason I ask is that I originally set my file size at 48k. That was a couple years ago and let everything run on auto pilot. Now I'm starting to hyper-focus on getting better indexing. I submit about 2.5-million pages and Google seems to grab about 1200 to 1500 URLs from each sitemap file. If I reduced the file size to 24,000 URLs would Google still grab 1200 from each file? I'm going to give it a try and see what happens.

~ David