vBulletin Search Engine Optimization
This is a discussion on Googlebot activity in the vbSEO log within the Bug Reporting forums, part of the vBSEO SEO Plugin category; I'm seeing a discrepancy between my Who's Online and the vbSEO log for bots. I can see Googlebot hitting pages ...
| |||||||
Enhancing 80 million pages. | Register | FAQ | Members List | Social Groups | Calendar | Search | Today's Posts | Mark Forums Read |
|
#1
| |||
| |||
| Googlebot activity in the vbSEO log
I'm seeing a discrepancy between my Who's Online and the vbSEO log for bots. I can see Googlebot hitting pages every minute or so throughout the day... appearing to use the same session throughout (unlike Yahoo it doesn't spawn tons of sessions). So I know and can observe that Googlebot is hitting showthread.php quite heavily. However when I look in my log: /vbseo_sitemap/index.php?hitdetails=20051201.log I'm seeing reports that Googlebot *NEVER* hit showthread, just forumdisplay. This is clearly incorrect, as I can observe the bots but the stats never reflect what I observe. How are you identifying Googlebots? As an additional question... on top of the above I also see Google AdSense spiders with the user-agent "Mediapartners-Google/2.1", are you also including the Google AdSense spider in the Google totals? If not, could you add them to the bot list (unsure how you would do this, as your code is encoded). |
|
#2
| ||||
| ||||
| Re: Googlebot activity in the vbSEO log
Hello, hm. we never had this problem - showthread pages are shown in SE Bots Activity Log (as well as other pages). Do you have this problem for a long time or is it just a one day logged issue? PM sent. Quote:
Quote:
__________________ Oleg Ignatiuk / Crawlability Inc. vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD 6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You. |
|
#3
| ||||
| ||||
| Re: Googlebot activity in the vbSEO log Quote:
|
|
#4
| ||||
| ||||
| Re: Googlebot activity in the vbSEO log
It is supposed to be in a separate column and doesn't mixed with googlebot stats!
__________________ Oleg Ignatiuk / Crawlability Inc. vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD 6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You. |
|
#5
| ||||
| ||||
| Re: Googlebot activity in the vbSEO log
That is o.K. |
|
#6
| |||
| |||
| Re: Googlebot activity in the vbSEO log Quote:
You still have the password I gave you for my stats/log files, and you'll see in there that the value on showthread.php for Googlebot is still zero. I took the screenshot with the user agents in if it helps, not sure if it will. As I've remarked elsewhere, I would usually track down and debug a bug like this myself, but your encoded code makes this difficult to do (not worth the effort), so I'll just help how I can and leave the work to you |
|
#7
| |||
| |||
| Re: Googlebot activity in the vbSEO log
OK, you can ignore me now. It finally says 1!
|
|
#8
| ||||
| ||||
| Re: Googlebot activity in the vbSEO log
It also shows 1 for 2005-12-01 and 6 for 2005-11-28! Let's see how it will go further. Do you have some site statistics analyzer software installed to get googlebot hits numbers and compare it to the numbers reported by SE Bots Activity Log? (are they in nearly the same range?)
__________________ Oleg Ignatiuk / Crawlability Inc. vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD 6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You. |
|
#9
| |||
| |||
| Re: Googlebot activity in the vbSEO log
No, I've turned off all logging. I'm a performance freak and Google Analytics already do all my stats (but not bots unfortunately). I might add something to copy sessions into a second table where the googlebot useragent is detected... that would give me a pretty complete log of it. I'll think about doing that tomorrow if I haven't seen the number rise inline with the spiders I'm seeing through Who's Online. |
|
#10
| |||
| |||
| Re: Googlebot activity in the vbSEO log
OK, I have about 30 Googlebots on my site, nearly all of them looking at threads. Yet the stats show nothing. So my next question: Are the stats updated instantly? Or do they accumulate elsewhere (where) only to be updated hourly (or something)? If they're instant, then there is a problem, and I'll go and code a second session table to keep track of all bots for you to use for debugging. |
|
#11
| ||||
| ||||
| Re: Googlebot activity in the vbSEO log
Yes, results are updated instantly (in real-time). And if you can prepare a session table with date/time, page url, bot user-agent that would be helpful for troublehooting, thanks!
__________________ Oleg Ignatiuk / Crawlability Inc. vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD 6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You. |
|
#12
| |||
| |||
| Re: Googlebot activity in the vbSEO log
I've had a look at the class_core.php file of vBulletin where the session stuff is, and can't find a trivial way of adding what I would want so that I can log just Googlebot visits. I've also thought about it and come to the conclusion that it doesn't matter much So I'm actually going to drop this in priority. I didn't have stats on bots before, so I'm not going to miss them much Thanks anyway, I apologise for not being able to figure out class_core.php so we could resolve the bug. |
|
#13
| ||||
| ||||
| Re: Googlebot activity in the vbSEO log
Sure, I see. Hopefully we'll be able to find this issue somehow
__________________ Oleg Ignatiuk / Crawlability Inc. vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD 6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You. |
|
#14
| |||
| |||
| Re: Googlebot activity in the vbSEO log
The Hits are not counted in my Bots activity log - what am I doing wrong ? I set this option to yes : Log Spider Activity in your Forums? I am using VBSEO 2.3.2 and 1.5 Sitemap |
|
#15
| ||||
| ||||
| Re: Googlebot activity in the vbSEO log
Please check if the "vbseo_sitemap/data/hits/" folder exists at your host (and if not, try to create it manually and set 0666 permissions to it).
__________________ Oleg Ignatiuk / Crawlability Inc. vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD 6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You. |
| Thread Tools | |
| |
Similar Threads | ||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| vBSEO 2.4.0 Released - Includes Google AdSense Targeting Feature! | Juan Muriente | vBSEO Announcements | 74 | 05-20-2006 10:29 PM |
| vBSEO 2.0 RC7 Released | Juan Muriente | vBSEO Announcements | 17 | 09-09-2005 12:00 AM |