SEO

I recently updated the sitemap to the newest version and replaced the functions file per this forum and now I am getting a strange result:

Before the update:

What could be causing this? There are 86,000 seconds in a day. Is the script really running all day long?

Vbulletin 3.7.0
vBSEO 3.2.0 RC5
vBSEO Sitemap 2.1 w/vbseo_sitemap_functions.php fix applied
PHP 5
SQL 5

Its a permissions issue perhaps? I had exactly the same a few months ago. Allowing the folder the correct permissions sorted it out.

forums/vseo_sitemap = 755
forums/vseo_sitemap/addons = 755
forums/vseo_sitemap/data = 777
forums/vseo_sitemap/data/hits = 777

odd... none of thew stats match up at all.. not even forum count.

It means that sitemap generation was interrupted and resumed on next day with the next scheduled task. The total time was counted from "start time" to "finish time", i.e. one day, although it was not running in between.
You should increase max_execution_time setting in PHP config to avoid interruptions.

Forum pages include forumdisplay pagination and possibly "old URLs included", so perhaps it matches.

__________________
Oleg Ignatiuk / Crawlability Inc.
vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

I noticed the data folder had over 1200 files in it - so many that they woudln't all display in SmartFTP. Some of the log files were over three years old. I deleted the ones pre-08 and now the directory is displaying properly. Is it possible this had something to do with it?

From today:
Still, this script ran for an hour - I'm afraid my host will give me the boot if they see scripts running that long. They pulled the plug on me about a year ago when they thought vBSEO was causing some problems. (Hostgator)

That should not be related.
Did you try to generate sitemap manually?

__________________
Oleg Ignatiuk / Crawlability Inc.
vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

OKay- did that. Took about 1.5 min total.

Looks to me like the problem started around 5-6-08 (see attachment)

Thats shortly after upgrade to vb 3.7.0, vBSEO 3.2 and Sitemap 2.1. Also my host chnaged a setting in PHP to fix some database errors we were getting. They upgraded the PHP and SQL on or around 5-3-08.

My email to Hostgator support:

Quote:

I have noticed since you updated the SQL and PHP on my box two days ago, I have been receiving occasional errors from vBulletin. "MySQL server has gone away" I am noticing a pattern on the times I receive these errors. It appears that the errors occur at precisely 6:33 pm and 2:02 am every day. I have no crons or routines running at that time. Is it possible that the server is running a routine on the SQL at this time? I have presented the issue to vBulletin and they say it is a hosting issue, more specifically, with SQL. Here are the errors: ------------------------------------------------------------------------- Database error in vBulletin 3.7.0: Invalid SQL: SELECT p.dateline,p.postid,p.threadid FROM post AS p WHERE p.threadid = 4158 ORDER BY p.dateline; MySQL Error : MySQL server has gone away Error Number : 2006 Request Date : Thursday, May 1st 2008 @ 02:02:56 AM Error Date : Thursday, May 1st 2008 @ 02:05:21 AM Script : http://www.dodgeboard.com/forums/cron.php?rand=61088 Referrer : http://www.dodgeboard.com/forums/showthread.php?p=70361#post70361 IP Address : 71.218.37.132 Username : Unregistered Classname : vB_Database MySQL Version :

Reply from Hostgator:

From Me:

From HostGator:

EDIT: I still get the error "MySQL server has gone away" but not as frequent. I get about two each morning instead of 10-20.

Attachment shows my sitemap generator history which indicates the date that the problems started. Why would it run ok manually?

Attached Thumbnails

 

Just now noticing that I have two scheduled tasks:

vBSEO Sitemap Generator 2:00 am
Generate sitemaps

vBSEO - Create Google/Yahoo Sitemap 4:00 am

Is this normal?

No, you should remove one of them, keep this one: "vBSEO Sitemap Generator".

__________________
Oleg Ignatiuk / Crawlability Inc.
vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

OK- so I have removed the extra one but I am still having the same problem. Runs for 80,000+ seconds.

Ran manually and got:

[SECTION START] showthread [5,370.1Kb mem used] [168,229s (+80,927s)]
[create sitemap file] filename: sitemap_3.xml.gz, number of urls: 25000 [5,488.8Kb mem used] [168,233s (+4s)]
[create sitemap in text format] part #4 [5,488.7Kb mem used] [168,233s (+0s)]


Database error

The DodgeBoard.com - Forums database has encountered a problem.Please try the following:

• Load the page again by clicking the Refresh button in your web browser.
• Open the www.dodgeboard.com home page, then try to open another page.
• Click the Back button to try another link.

The www.dodgeboard.com forum technical staff have been notified of the error, though you may contact them if the problem persists.
We apologise for any inconvenience.

-------------email--------------------------------------------------------------------------

Database error in vBulletin 3.7.1:

Invalid SQL:

SELECT p.dateline,p.postid,p.threadid
FROM post AS p
WHERE p.threadid = 419
ORDER BY p.dateline;

MySQL Error : MySQL server has gone away
Error Number : 2006
Request Date : Monday, June 2nd 2008 @ 11:44:31 PM
Error Date : Monday, June 2nd 2008 @ 11:45:21 PM
Script : http://www.dodgeboard.com/forums/vbseo_sitemap/vbseo_sitemap.php
Referrer : http://www.dodgeboard.com/forums/vbseo_sitemap/index.php
IP Address : 70.167.xxx.xxx
Username : Unregistered
Classname : vB_Database
MySQL Version :

-----------2nd attempt----------------------------------------------

Ran it a second time and it completed with no errors :

[SECTION START] showthread [5,369.7Kb mem used] [168,393s (+160s)]
[archive homepage] [5,778.7Kb mem used] [168,395s (+1s)]
[SECTION START] forumdisplay archived [5,779.2Kb mem used] [168,395s (+0s)]
[SECTION START] member [5,801.7Kb mem used] [168,395s (+0s)]
[SECTION START] polls [5,590.0Kb mem used] [168,395s (+0s)]
[SECTION START] blogs [5,619.0Kb mem used] [168,395s (+0s)]
[addon module] vbseo_sm_vba.php [5,653.1Kb mem used] [168,395s (+0s)]
[create sitemap file] filename: sitemap_4.xml.gz, number of urls: 10461 [5,410.2Kb mem used] [168,395s (+0s)]
[create sitemap in text format] part #5 [5,410.2Kb mem used] [168,395s (+0s)]
[create sitemap index] filename: sitemap_index.xml.gz, number of sitemaps: 4 [5,410.5Kb mem used] [168,395s (+0s)]

--------------------Email results---------------------------------

Hello!

The vBSEO Google/Yahoo Sitemap has been successfully generated for your vBulletin forums at:
DodgeBoard.com - Forums (http://www.dodgeboard.com/forums)

Report:
============================
Click the following link for your vBSEO Google/Yahoo Sitemap Report:
http://www.dodgeboard.com/forums/vbseo_sitemap/

Summary:
============================
Forum Display: 319
Show Thread: 10849
Show Post: 73549
Member Profiles: 612
Poll Results: 48
Archive: 40

Total Indexed URLs: 85461
Total Processing Time: 168,396.57 seconds

Google ping: Successful.
Yahoo ping: Successful.
Ask ping: Successful.
Moreover ping: Successful.
Msn ping: Successful.
---------------------------------------------------------------

Looks like this one has been trying to complete for a few days.....

Any ideas on how I can cure this?

if you're losing mysql server (sql has gone away error) you're probably maxing out connections/concurrent processes to it.

MySQL :: MySQL 5.0 Reference Manual :: B.1.2.9 MySQL server has gone away


I get about 40 error reports a day on it from my site as well, usually when my rss feeds run.

This problem seems to have cropped up about the time the host upgraded the sql.

Is there anything that can be done to slow the sitemap generation, or to force the script to go easier on the server?

try setting your max URLs per sitemap to a lower figure (will make more files)
and add a pause of 10 secs or so

there's an option for this in the vb options panel for the sitemap