404 error Google

I don't get it, did everything according to the readme and Google keeps showing this 404 error:

Your sitemap index doesn't exist.

http://forum.dutchbodybuilding.com/sitemap_index.xml.gz

Plus, those are sitemap 2.6 and lower style urls.

new format in 3.0 is sitemap_SECTIONNAME_X.xml.gz

So, it looks like your sitemap is timing out, and has never completed writing out the index file. Google is still trying to load your last good indexd file, which points to files that don't exist.


If you keep timing out, you may need to set up a true cron job on your server instead of through vb for a large site.
or, remove some sections.

vBSEO Sitemap Generator : Quality vs Quantity - vBulletin SEO Forums

I don't understand, I'm using version 3.0 and the URL http://forum.dutchbodybuilding.com/sitemap_index.xml.gz is shown in the upper right corner of the Sitemap Reports.

I did everything in the manual so why doesn't it work?

As stated above, it hasn't run correctly. it's timing out (the index file writes last).

Run it manually and see what happens. You may need to increase server resouces or remove sections to make it run faster.

Originally Posted by Brian Cummiskey

As stated above, it hasn't run correctly. it's timing out (the index file writes last).

Run it manually and see what happens. You may need to increase server resouces or remove sections to make it run faster.

It seems to me it does run correctly?

[SECTION START] showthread [6,633.2Kb mem used] [310s (+32s)]
[create sitemap file] filename: sitemap_forum_14.xml.gz, number of urls: 20000 [10,369.6Kb mem used] [323s (+13s)]
[create sitemap file] filename: sitemap_forum_15.xml.gz, number of urls: 20000 [10,429.3Kb mem used] [341s (+18s)]
[create sitemap file] filename: sitemap_forum_16.xml.gz, number of urls: 20000 [10,395.4Kb mem used] [359s (+18s)]
[create sitemap file] filename: sitemap_forum_17.xml.gz, number of urls: 20000 [10,389.1Kb mem used] [376s (+17s)]
[create sitemap file] filename: sitemap_forum_18.xml.gz, number of urls: 20000 [10,381.2Kb mem used] [396s (+19s)]
[create sitemap file] filename: sitemap_forum_19.xml.gz, number of urls: 20000 [10,572.6Kb mem used] [418s (+23s)]
[create sitemap file] filename: sitemap_forum_20.xml.gz, number of urls: 20000 [10,535.1Kb mem used] [438s (+19s)]
[create sitemap file] filename: sitemap_forum_21.xml.gz, number of urls: 20000 [10,539.4Kb mem used] [457s (+20s)]
[SECTION START] polls [9,886.1Kb mem used] [473s (+15s)]
[create sitemap file] filename: sitemap_forum_22.xml.gz, number of urls: 19158 [10,790.5Kb mem used] [478s (+5s)]
[archive homepage] [7,127.0Kb mem used] [478s (+0s)]
[SECTION START] forumdisplay archived [7,128.8Kb mem used] [478s (+0s)]
[SECTION START] showthread archived [9,589.4Kb mem used] [479s (+1s)]
[create sitemap file] filename: sitemap_archive_1.xml.gz, number of urls: 20000 [11,628.3Kb mem used] [481s (+2s)]
[create sitemap file] filename: sitemap_archive_2.xml.gz, number of urls: 20000 [10,772.7Kb mem used] [485s (+4s)]
[create sitemap file] filename: sitemap_archive_3.xml.gz, number of urls: 20000 [10,750.6Kb mem used] [488s (+3s)]
[create sitemap file] filename: sitemap_archive_4.xml.gz, number of urls: 20000 [10,763.6Kb mem used] [493s (+5s)]
[create sitemap file] filename: sitemap_archive_5.xml.gz, number of urls: 20000 [10,771.6Kb mem used] [500s (+7s)]
[create sitemap file] filename: sitemap_archive_6.xml.gz, number of urls: 20000 [10,766.9Kb mem used] [508s (+8s)]
[create sitemap file] filename: sitemap_archive_7.xml.gz, number of urls: 20000 [10,752.4Kb mem used] [513s (+6s)]
[create sitemap file] filename: sitemap_archive_8.xml.gz, number of urls: 20000 [10,743.8Kb mem used] [517s (+4s)]
[create sitemap file] filename: sitemap_archive_9.xml.gz, number of urls: 20000 [10,769.6Kb mem used] [526s (+9s)]
[create sitemap file] filename: sitemap_archive_10.xml.gz, number of urls: 20000 [10,763.6Kb mem used] [533s (+7s)]
[create sitemap file] filename: sitemap_archive_11.xml.gz, number of urls: 20000 [10,757.1Kb mem used] [540s (+7s)]
[create sitemap file] filename: sitemap_archive_12.xml.gz, number of urls: 20000 [10,756.4Kb mem used] [545s (+5s)]
[create sitemap file] filename: sitemap_archive_13.xml.gz, number of urls: 20000 [10,755.0Kb mem used] [549s (+4s)]
[create sitemap file] filename: sitemap_archive_14.xml.gz, number of urls: 20000 [10,759.7Kb mem used] [555s (+6s)]
[create sitemap file] filename: sitemap_archive_15.xml.gz, number of urls: 20000 [10,767.3Kb mem used] [560s (+5s)]
[create sitemap file] filename: sitemap_archive_16.xml.gz, number of urls: 20000 [10,769.0Kb mem used] [566s (+6s)]
[create sitemap file] filename: sitemap_archive_17.xml.gz, number of urls: 20000 [10,780.3Kb mem used] [572s (+7s)]
[create sitemap file] filename: sitemap_archive_18.xml.gz, number of urls: 20000 [10,764.0Kb mem used] [577s (+5s)]
[create sitemap file] filename: sitemap_archive_19.xml.gz, number of urls: 20000 [10,786.4Kb mem used] [585s (+7s)]
[create sitemap file] filename: sitemap_archive_20.xml.gz, number of urls: 20000 [10,790.0Kb mem used] [595s (+10s)]
[create sitemap file] filename: sitemap_archive_21.xml.gz, number of urls: 20000 [10,789.8Kb mem used] [603s (+8s)]
[create sitemap file] filename: sitemap_archive_22.xml.gz, number of urls: 15200 [10,017.6Kb mem used] [609s (+6s)]
[SECTION START] tags [7,152.9Kb mem used] [609s (+0s)]
[create sitemap file] filename: sitemap_tags_1.xml.gz, number of urls: 20000 [10,806.7Kb mem used] [659s (+50s)]
[create sitemap file] filename: sitemap_tags_2.xml.gz, number of urls: 20000 [10,783.7Kb mem used] [714s (+54s)]
[create sitemap file] filename: sitemap_tags_3.xml.gz, number of urls: 14069 [9,725.5Kb mem used] [752s (+38s)]
[SECTION START] member [7,157.8Kb mem used] [752s (+0s)]
[create sitemap file] filename: sitemap_member_1.xml.gz, number of urls: 20000 [10,725.6Kb mem used] [759s (+7s)]
[create sitemap file] filename: sitemap_member_2.xml.gz, number of urls: 20000 [10,725.1Kb mem used] [769s (+10s)]
[create sitemap file] filename: sitemap_member_3.xml.gz, number of urls: 20000 [10,727.5Kb mem used] [778s (+9s)]
[create sitemap file] filename: sitemap_member_4.xml.gz, number of urls: 20000 [10,727.9Kb mem used] [789s (+11s)]
[create sitemap file] filename: sitemap_member_5.xml.gz, number of urls: 4941 [8,996.6Kb mem used] [793s (+4s)]
[SECTION START] albums [7,163.6Kb mem used] [793s (+0s)]
[create sitemap file] filename: sitemap_album_1.xml.gz, number of urls: 4185 [8,114.2Kb mem used] [803s (+10s)]
[SECTION START] blog tags [7,164.6Kb mem used] [803s (+0s)]
[SECTION START] groups [7,165.0Kb mem used] [803s (+0s)]
[create sitemap file] filename: sitemap_group_1.xml.gz, number of urls: 338 [7,581.5Kb mem used] [803s (+0s)]
[create sitemap index] filename: sitemap_index.xml.gz, number of sitemaps: 54 [7,183.6Kb mem used] [803s (+0s)]
Sitemap has been created. Click here to return [7,157.0Kb mem used] [804s (+2s)]

What URL should I submit in Google WMT?

As you can see, this started with sitemap 14, which means the time it ran before it did not complete fully. So, you still have wait_timeout or memory_limit issues to address. Frankly, i'd look into ditching the archive and all the rest from the sitemap....

http://forum.dutchbodybuilding.com/sitemap_index.xml.gz SHOULD load (and is what you should submit to google), but it does not.

The longhand method DOES load.
http://forum.dutchbodybuilding.com/v...p_index.xml.gz
so, the sitemap HAS in fact run

What webserver software are you on (my toolbar shows NOYB which i've never heard of before) and what are your htaccess (or otherwise) rules in place as it appears that the rewrite is the item that is failing at this point.

Originally Posted by Brian Cummiskey

As you can see, this started with sitemap 14, which means the time it ran before it did not complete fully. So, you still have wait_timeout or memory_limit issues to address. Frankly, i'd look into ditching the archive and all the rest from the sitemap....

http://forum.dutchbodybuilding.com/sitemap_index.xml.gz SHOULD load (and is what you should submit to google), but it does not.

The longhand method DOES load.
http://forum.dutchbodybuilding.com/v...p_index.xml.gz
so, the sitemap HAS in fact run

What webserver software are you on (my toolbar shows NOYB which i've never heard of before) and what are your htaccess (or otherwise) rules in place as it appears that the rewrite is the item that is failing at this point.

Ok.

Webserver is running DirectAdmin 1.33.6 & Apache 2.2.9.

This is my htaccess content:

Code:

# Comment the following line (add '#' at the beginning)
# to disable mod_rewrite functions.
# Please note: you still need to disable the hack in
# the vBSEO control panel to stop url rewrites.
RewriteEngine On

RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://(www\.)?(forum\.|wiki\.|shop\.)?dutchbodybuilding.com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?dutchbodybuilding.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?offtopic.nl(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?off-topic.nl(/)?.*$ [NC]

# Some servers require the Rewritebase directive to be
# enabled (remove '#' at the beginning to activate)
# Please note: when enabled, you must include the path
# to your root vB folder (i.e. RewriteBase /forums/)
#RewriteBase /

RewriteCond %{HTTP_HOST} !^forum\.dutchbodybuilding\.com
RewriteRule (.*) http://forum.dutchbodybuilding.com/$1 [L,R=301]

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|cron|vbseo_sitemap)
RewriteRule ^((archive/)?(.*\.php(/.*)?))$ vbseo.php [L,QSA]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !/(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule ^(.+)$ vbseo.php [L,QSA]

<IfModule mod_security.c>
SecFilterEngine Off
SecFilterScanPOST Off
</ifmodule>

RewriteEngine On
RewriteRule ^((urllist|sitemap).*\.(xml|txt)(\.gz)?)$ vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

There's the problem...


Please use this instead

Code:

<IfModule mod_security.c>
 SecFilterEngine Off
 SecFilterScanPOST Off
 </ifmodule>

# Comment the following line (add '#' at the beginning)
# to disable mod_rewrite functions.
# Please note: you still need to disable the hack in
# the vBSEO control panel to stop url rewrites.
RewriteEngine On
 
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://(www\.)?(forum\.|wiki\.|shop\.)?dutchbodybuilding.  com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?dutchbodybuilding.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?offtopic.nl(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?off-topic.nl(/)?.*$ [NC]
 
# Some servers require the Rewritebase directive to be
# enabled (remove '#' at the beginning to activate)
# Please note: when enabled, you must include the path
# to your root vB folder (i.e. RewriteBase /forums/)
#RewriteBase /
 
RewriteCond %{HTTP_HOST} !^forum\.dutchbodybuilding\.com$
RewriteRule (.*) http://forum.dutchbodybuilding.com/$1 [L,R=301]
 
RewriteRule ^((urllist|sitemap).*\.(xml|txt)(\.gz)?)$ vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]                      

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|cron|vbseo_sitemap)
RewriteRule ^((archive/)?(.*\.php(/.*)?))$ vbseo.php [L,QSA]
 
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !/(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule ^(.+)$ vbseo.php [L,QSA]

Thanks. But I get an Internal Server Error with that code?

try this:

Code:

# Comment the following line (add '#' at the beginning)
# to disable mod_rewrite functions.
# Please note: you still need to disable the hack in
# the vBSEO control panel to stop url rewrites.
RewriteEngine On



<IfModule mod_security.c>
 SecFilterEngine Off
 SecFilterScanPOST Off
</ifmodule>

 
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://(www\.)?(forum\.|wiki\.|shop\.)?dutchbodybuilding.  com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?dutchbodybuilding.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?offtopic.nl(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www.)?off-topic.nl(/)?.*$ [NC]
 
# Some servers require the Rewritebase directive to be
# enabled (remove '#' at the beginning to activate)
# Please note: when enabled, you must include the path
# to your root vB folder (i.e. RewriteBase /forums/)
#RewriteBase /
 
RewriteCond %{HTTP_HOST} !^forum\.dutchbodybuilding\.com$
RewriteRule (.*) http://forum.dutchbodybuilding.com/$1 [L,R=301]
 
RewriteRule ^((urllist|sitemap).*\.(xml|txt)(\.gz)?)$ vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]                      

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|cron|vbseo_sitemap)
RewriteRule ^((archive/)?(.*\.php(/.*)?))$ vbseo.php [L,QSA]
 
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !/(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule ^(.+)$ vbseo.php [L,QSA]

Ok I replaced it with that and ran the sitemaps generator. But it seems the problem persists.

The longhand way doesn't work right now, which tells me the script timed out again. (it deletes all xml.gz files, then runs to create the new ones).

http://forum.dutchbodybuilding.com/v...p_index.xml.gz

it never got to the index.

If you have a progress.dat file in the vbseo_sitemap folder, the task did not complete successfully.

I see a sitemap_index.xml.gz file but no progress.dat file in /data. This is strange :/

The files are all CHMOD 666 is this ok?

please open a ticket with ftp access, your vbseo cp password and an admin account for further troubleshooting.