vBulletin SEO Forums

SEO

vBulletin Search Engine Optimization

Buy vBSEO Now! HACKER SAFE certified sites prevent over 99.9% of hacker crime.
ne nw
vBSEO Team: Hot or Not? Ready for Traffic Explosion? vBSEO 3.2.0 GOLD Has Landed Success with vBSEO = 600ore Web Visitors + $1400 in a Day! vBSEO Helps Forum Earn $100/day from Google AdSense Discover the Power of "Long Tail Search" Crawlability Inc. Files for SEO Technology Patent
se sw

Google Crawling and Indexing

This is a discussion on Google Crawling and Indexing within the Pre-Sales Questions forums, part of the Announcements & Pre-Sales category; Hello, I have been running a forum for about 6 years. Some months ago i upgraded the board into Vbulletin ...

Go Back   vBulletin SEO Forums > Announcements & Pre-Sales > Pre-Sales Questions

Enhancing 80 million pages.

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read

Reply

 

LinkBack Thread Tools
  #1  
Old 07-04-2006, 01:31 PM
Junior Member
 
Real Name: GTX2
Join Date: Jul 2006
Posts: 11
Google Crawling and Indexing

Hello,

I have been running a forum for about 6 years.
Some months ago i upgraded the board into Vbulletin software.
I have almost 1 milion page views per month and about 90.000 posts.

What is happening is that i only have about 2 or 3 pages indexed by google.
My board is open to guest's and i rarely see googlebots online crawling. Other like yahoo are always crawling my board.

Will VBSEO help me regarding this problem?
Thank you.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2  
Old 07-04-2006, 02:22 PM
Joe Ward's Avatar
vBSEO Staff
vBSEO Total Customer SupportvBSEO Documenter
 
Real Name: Joseph Ward
Join Date: Jun 2005
Location: Puerto Rico
Posts: 17,606
Blog Entries: 7
Re: Google Crawling and Indexing

Well - it sounds like Google dropped you from the index after noticing the massive changes. This was to be expected. I suspect they will eventually find your vBulletin content again, but I wouldn't leave anything up to chance.

I believe vBSEO along with the vBSEO Google/Yahoo Sitemap will help you to finally get Googlebot to return to your site.

The best time to do this is now. Since Google does not currently have your forum indexed, it will not notice another change from vBulletin dynamic URLs to vBSEO static URLs. This is similar to the advantage that new forums have when they install vBSEO prior to launch.

vBSEO will help you to achieve link consensus, and therefore eliminate alot of unnecessary paths to the same (or duplicate) content. This will allow Google PR to be distributed better, helping to get you a higher crawling priority.
__________________
Joe Ward / Crawlability Inc.
vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3  
Old 07-05-2006, 09:06 AM
Michael's Avatar
Senior Member
vBSEO Pre-Release Team
 
Real Name: Michael Benson
Join Date: Sep 2005
Location: United Kingdom
Posts: 778
Send a message via AIM to Michael Send a message via MSN to Michael
Re: Google Crawling and Indexing

A good idea is to partake in link exchanges with other related and popular websites and message boards. Preferably those with a high indexation rate, this is because Google and other search engine robots will take note of outbound links on these pages, and index these. Thus decreasing the time required to get indexed in comparison the usual method of the Add URL submission form.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4  
Old 07-05-2006, 05:12 PM
Juan Muriente's Avatar
vBSEO Staff
vBSEO Total Customer SupportvBSEO Documenter
 
Real Name: Juan Carlos Muriente
Join Date: Jun 2005
Location: Puerto Rico
Posts: 12,515
Blog Entries: 2
Re: Google Crawling and Indexing

Right, link building is critical when it comes to SEO. Make sure you work on gaining links-in while having your on-page/site SEO optimized with vBSEO.

Q: What is your forum-root Google PR? We've definitely been able to identify a faster/deeper indexing trend associated to higher forum-root PR....

ps- Welcome to vBSEO
__________________
Juan Muriente / Crawlability Inc.
vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5  
Old 07-13-2006, 12:14 PM
Junior Member
 
Real Name: GTX2
Join Date: Jul 2006
Posts: 11
Re: Google Crawling and Indexing

thank you for your replies.
I have been using google sitemap now and after making a search at google "link: www.myurl.com" i get 195 results of pages linking to my forums.

I cannot understand why google doesn't even want to look at my forums.

I'm always looking at who's online members, and i never see googlebot at my board (i only saw it once looking at my forum homepage and never more). Yahoo glurp spider is always at my board with 3/4 bots.

how can i guarantee google indexes my board? how can i make sure vbseo will help me?

I'm using also linkexchange.
thank you for your help.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6  
Old 07-13-2006, 01:05 PM
Mert Gökçeimam's Avatar
vBSEO Moderator
vBSEO Total Customer SupportDesign for SEO
 
Real Name: Lizard King
Join Date: Oct 2005
Location: Istanbul
Posts: 6,344
Blog Entries: 2
Send a message via MSN to Mert Gökçeimam
Re: Google Crawling and Indexing

You can never be sure about what google does but vBSeo is the best tool you can use

Also are you using spiders.xml ? If you are not maybe you don't know if google is crawling your site.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7  
Old 07-13-2006, 03:28 PM
briansol's Avatar
Senior Member
vBSEO Pre-Release TeamDesign for SEOBig Board Administrator
 
Real Name: Brian
Join Date: Apr 2006
Location: Central CT, USA
Posts: 4,580
Re: Google Crawling and Indexing

the link: command is unreliable, and it is not the tool for what you want to look up. link: shows sites that link to you, IF THEY are indexed. I wouldn't use this as a judge of your site. I have 0 link: results on some sites that have every page indexed.

the site: command is what you want to use to see how many SERP's you have.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8  
Old 07-13-2006, 03:43 PM
Junior Member
 
Real Name: GTX2
Join Date: Jul 2006
Posts: 11
Re: Google Crawling and Indexing

didn't know about the spiders.xml file... installing now.

What happens with googlebot is that when it comes to the forum, it gets stucked in only one thread for 10 minutes and then i only see the bot a couple of weeks later doing the same thing: crawling one thread and going away.

I will install spiders.xml and probably install vbseo also and see results.
thank you all for helping
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9  
Old 07-16-2006, 07:43 AM
Junior Member
 
Real Name: GTX2
Join Date: Jul 2006
Posts: 11
Re: Google Crawling and Indexing

after installing spiders hack i can see googlebot is coming sometimes to my forums but never following links.

What is the best command search to list all urls of my domain at google?
allinurl: myurl.com ?
allinurl: site:myurl.com ?


It's strange because google has in cache pages crawled in May 2005 and never updated those pages. They are still in google's cache.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10  
Old 07-17-2006, 02:50 PM
Junior Member
 
Real Name: GTX2
Join Date: Jul 2006
Posts: 11
Re: Google Crawling and Indexing

sorry being boring with all this questions, but i've been worried with this :s
Why googlebot never follow the links on my forums? Why it gets stucked in a thread/forum when it starts crawling the board? i see other forums and googlebot is always following the links....
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #11  
Old 07-17-2006, 03:02 PM
briansol's Avatar
Senior Member
vBSEO Pre-Release TeamDesign for SEOBig Board Administrator
 
Real Name: Brian
Join Date: Apr 2006
Location: Central CT, USA
Posts: 4,580
Re: Google Crawling and Indexing

do you have a robots.txt file?
a sitemap submitted to google daily?
vbseo installed?

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #12  
Old 07-17-2006, 06:27 PM
Junior Member
 
Real Name: GTX2
Join Date: Jul 2006
Posts: 11
Re: Google Crawling and Indexing

1) robots.txt: yes (although, many sites say robots.txt is not necessary because normal procedure of searchbots is to follow all links)

2) Sitemaps submitted daily: yes (although many sites say sitemaps should be submitted weekly or even monthly avoiding being "banned" from search engines)

3) vbSEO: no (not yet, thinking if it would be worth to buy it or not...)


I remembered one thing now: could this problem be because using vbadvanced in homepage as www.mydomain.com/index.php and using www.mydomain.com/forum/ as my forum directory and not in the root?

Last edited by gtx2; 07-17-2006 at 07:18 PM.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #13  
Old 07-17-2006, 08:55 PM
briansol's Avatar
Senior Member
vBSEO Pre-Release TeamDesign for SEOBig Board Administrator
 
Real Name: Brian
Join Date: Apr 2006
Location: Central CT, USA
Posts: 4,580
Re: Google Crawling and Indexing

1- right. but there are things you DONT want SE's to index to "pollute" your listings. if you do a search on here for robots.txt you will find many examples of files various members are excluding.

2- Google will take a new sitemap ever second if you wanted to. It won't get you very far, but it won't get you banned either. I run mine nightly at 4am EST.

3- what are you waiting for see my sig. it's been a great month.

3b- no, vbadvanced won't have ill effects, nor will 1 level deep. many vbseo users run vba as well
__________________
My Blog | Available for paid mods/addons. PM me your RFP.

My Free Mods/Articles: Easy Thread Prefixes for the 50-states | Who Tagged? | New Users with no Visitor Messages

the Ultimate Guide to vBSEO
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #14  
Old 07-30-2006, 07:37 AM
Junior Member
 
Real Name: GTX2
Join Date: Jul 2006
Posts: 11
I'm getting really bored with this problem about google.
I think my domain should be banned from google, because pages never appear at google. I think VBSEO can't help me, because the problem should be with google itself.

Google only indexes my homepage and updates it almost daily.
Then, never follow all other links and also there are only 3 or 4 pages cached in may2005 at google search engine.

Should i change my domain name to another one?
I'm getting desperate with this...

I see other boards in my country with 50 or 100 search engine bots online at the same time, including googlebot always online...
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #15  
Old 07-31-2006, 11:41 AM
Joe Ward's Avatar
vBSEO Staff
vBSEO Total Customer SupportvBSEO Documenter
 
Real Name: Joseph Ward
Join Date: Jun 2005
Location: Puerto Rico
Posts: 17,606
Blog Entries: 7
Well, if you have not achieved any significant level of current indexing, then changing your domain might be an option. It would be good if you could get an after market domain that already has some history, rather than a brand new domain.

However, have you done anything in the past that might have gotten you penalized in Google? When was your domain first purchased and launched (could you be sandboxed?). Have you tried a Google re-inclusion request?
__________________
Joe Ward / Crawlability Inc.
vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

vBSEO Google Sitemap Generator - Version 2.2 Released Mandatory Upgrade for vBSEO 3.2.0 GOLD

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump

Similar Threads

Thread Thread Starter Forum Replies Last Post
“Crawl Caching Proxy” (BigDaddy Update) Discussed by Matt Cutts Joe Ward SEO Buzz 1 04-24-2006 07:25 PM


All times are GMT -4. The time now is 12:55 PM.


Powered by vBulletin Version 3.7.3
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0 ©2008, Crawlability, Inc.