vBulletin SEO Forums

SEO

vBulletin Search Engine Optimization

Buy vBSEO Now! HACKER SAFE certified sites prevent over 99.9% of hacker crime.
ne nw
vBSEO Total Support Team Launches DeskPro New vBSEO Discount Level for Network Builders vBSEO 3.2.0 GOLD Has Landed Success with vBSEO = 600ore Web Visitors + $1400 in a Day! Crawlability Inc. Files for SEO Technology Patent
se sw

sitemap & spiders

This is a discussion on sitemap & spiders within the General Discussion forums, part of the vBulletin SEO Discussion category; Can somebody please tell me what this setting should be set at? Max URLs in Sitemap Files - I currently ...

Go Back   vBulletin SEO Forums > vBulletin SEO Discussion > General Discussion

Enhancing 80 million pages.

Register FAQ Members List Social Groups Calendar Search Today's Posts Mark Forums Read
  #1  
Old 06-09-2008, 04:31 AM
Member
 
Real Name: Chat Football
Join Date: Jun 2008
Posts: 70
Send a message via MSN to Chat Football
sitemap & spiders

Can somebody please tell me what this setting should be set at?

Max URLs in Sitemap Files - I currently have it set at 40000 although default is set at 50000 (Google limit)

Also I set this up just under a week ago but I havent seen vast amount of spiders being on the site, don't get me wrong they are coming on but not in silly numbers like some people mention.

Is this still ok?

Oh also can you let me know if this url looks ok to you? I just want to make sure that the urls are set correctly.

http://www.chatfootballforum.com/pre...html#post11162
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #2  
Old 06-09-2008, 12:36 PM
Joe Ward's Avatar
vBSEO Staff
vBSEO Total Customer SupportvBSEO Documenter
 
Real Name: Joseph Ward
Join Date: Jun 2005
Location: Puerto Rico
Posts: 19,748
Blog Entries: 7
We break up your sitemap, because of the huge amount of pages forums have, into smaller files. Google and the others read a file at a time. You should be fine with the default setting.

You should verify your sitemap:
Verifying your vBSEO Google/Yahoo Sitemap Generator is Installed Correctly

Also check your Google Webmaster Tools to make sure there are no errors:
Google Webmaster Tools - Recommended for All vBSEO Customers

Beyond that, you need to give it more time so that we can assess your growth. Stick with it.
__________________
Joe Ward / Crawlability Inc.
Support Team Launches New DeskPro Powered Tool Enhanced Support at Your Service

vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #3  
Old 06-09-2008, 12:46 PM
Member
 
Real Name: Chat Football
Join Date: Jun 2008
Posts: 70
Send a message via MSN to Chat Football
I upped mine by 10000 from 30000 to 40000, should I go back to default or is it ok at 40?

Looks like it's ok? http://www.chatfootballforum.com/sitemap_index.xml.gz

Google webmaster last downloaded 8hrs ago status was ok and 30844 urls submitted so looks good I think.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #4  
Old 06-09-2008, 12:57 PM
Joe Ward's Avatar
vBSEO Staff
vBSEO Total Customer SupportvBSEO Documenter
 
Real Name: Joseph Ward
Join Date: Jun 2005
Location: Puerto Rico
Posts: 19,748
Blog Entries: 7
It should be fine. Leave it and see how it goes for about 2 weeks.
__________________
Joe Ward / Crawlability Inc.
Support Team Launches New DeskPro Powered Tool Enhanced Support at Your Service

vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #5  
Old 06-09-2008, 01:04 PM
Member
 
Real Name: Chat Football
Join Date: Jun 2008
Posts: 70
Send a message via MSN to Chat Football
Ok thanks Joe, is there any reason as to why my threads are .html and yours aren't?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #6  
Old 06-09-2008, 02:49 PM
Joe Ward's Avatar
vBSEO Staff
vBSEO Total Customer SupportvBSEO Documenter
 
Real Name: Joseph Ward
Join Date: Jun 2005
Location: Puerto Rico
Posts: 19,748
Blog Entries: 7
We selected another option that tests threads as virtual directories instead.

Yours is the default.
__________________
Joe Ward / Crawlability Inc.
Support Team Launches New DeskPro Powered Tool Enhanced Support at Your Service

vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #7  
Old 06-09-2008, 02:59 PM
briansol's Avatar
Senior Member
vBSEO Pre-Release TeamDesign for SEOBig Board Administrator
 
Real Name: Brian
Join Date: Apr 2006
Location: Central CT, USA
Posts: 5,542
i run 10,000 with a 30 sec pause between files.

i get sql locks/high load if i just let it go.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #8  
Old 06-09-2008, 05:56 PM
Member
 
Real Name: Chat Football
Join Date: Jun 2008
Posts: 70
Send a message via MSN to Chat Football
Perhaps I should change it back to 30,000 then Brian. I was thinking the more the better but guess it isn't the case.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #9  
Old 07-26-2008, 02:41 PM
Junior Member
 
Real Name: David
Join Date: Oct 2006
Posts: 21
Quote:
Originally Posted by briansol View Post
i run 10,000 with a 30 sec pause between files.

i get sql locks/high load if i just let it go.
Brian,

Have you done any testing to discover if Google will index more URLs when the sitemap files are larger or smaller? The reason I ask is that I originally set my file size at 48k. That was a couple years ago and let everything run on auto pilot. Now I'm starting to hyper-focus on getting better indexing. I submit about 2.5-million pages and Google seems to grab about 1200 to 1500 URLs from each sitemap file. If I reduced the file size to 24,000 URLs would Google still grab 1200 from each file? I'm going to give it a try and see what happens.

~ David
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads

Thread Thread Starter Forum Replies Last Post
Vbseo Sitemap and Spiders... gotlinks General Discussion 3 09-22-2007 05:14 PM
where are the spiders? obx11 General Discussion 8 03-04-2007 12:17 AM
Spiders! sohan vBSEO Reviews 5 11-11-2006 07:40 PM
Understanding Spiders, vBSEO, and Sitemap Sonnie General Discussion 13 05-19-2006 05:14 AM


All times are GMT -4. The time now is 12:32 AM.


Powered by vBulletin Version 3.8.0 Beta 4
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.5 ©2008, Crawlability, Inc.