vBSEO Sitemap Generator : Quality vs Quantity

StarBuG has asked very nice question in this post why we should include only showthreads and forumdisplay pages in our sitemaps ?

I want to explain why you should include only showthreads and forumdisplay pages in your sitemap, That's from the Most important tips on vBSEO Sitemap generator we need the bots focus on our forum content not on our tags pages or social groups which will not send any visitors to your site.

Quality vs Quantity :
Here is the trick, If you include showthreads and forumdisplay pages only or if you include members profiles, showposts, poll results, archive, social groups ....etc, Here is an example :

Someone have 2 forums and he need to know what's the best ?

In the first forum
include "showposts, tags, archive, social groups, members profiles, poll results" So the total number is 90,000 page
Showthreads & Forumdisplay : 6000
So his sitemap has 96,000 page and he submitted it to Google.

After 2 Months he have 60,000 pages indexed on Google with only 1200 visitor, But he sure There is some threads is not indexed in Google yet and he also sure if they indexed they will send to me more traffic.

In the second forum
include "showthreads and forumdisplay pages" So the total number is : 5000
So his sitemap has 5000 page and he submitted it to Google.

After 2 Months he have 4000 pages indexed in Google and he sure the most of his threads is indexed in Google and Google send 1500 visitor

So he asked "why ?" And the answer is Quality over Quantity.

The second forum include only the important pages showthreads and foumdisplay, The pages if they indexed in Google he sure they will give him traffic.
But the first included all pages in his forum So Google index tags and members profile and few pages from showthreads and give him only 1400 visitor because Google will never show the tags pages and the social groups pages in his first search results. Ask yourself did you when you try to make search in Google have you see any tags pages or any other page appear in the search results ? Or only showthread pages ? Sure showthreads pages.

The Conclusion :
The quality is in the showthreads pages and I don't want the quantity in tags and members profiles ....etc, I only want Google to focus in showthread.
And I'm recommended these settings for vBSEO Sitemap :

• Include Show Post Pages : NO And replace it with permalinks : [How To] Replace Showpost Links with Permalinks
• Include Member Profile Pages : NO
• Include Archive Pages : NO And Completely disable your vBulletin Archive : [How To] Completely Turn Off the vBulletin Archive
• Include Show Thread Page : YES
• Include Forum Display Pages : YES
• Include Poll Results Pages : NO
• Include Blog Pages : If you have BLOGS Set it to YES and if not set it to NO.
• Include Blog Tag Pages : NO
• Include Member Profile Album Pages : NO
• Include Social Groups Pages : NO
• Include Social Groups Discussion Pages : NO
• Include Social Groups Picture Pages : NO
• Include Tag Pages : NO
• Smart Frequency for Show Thread Pages : YES

Leave the other options as the default they are the best settings to any forum.

Thank you

Is this proven or just a theory?

Google crawls around 2000-8000 pages a day average on my forum.
I don't think it will miss new content just because I have 180.000 links included into my sitemap.

I also checked my google analytics.
In the last 30 days I had 84 visitors via my tag pages so they are in the search results.
This is not much and of course dismissible but to say they don't get listed or found is wrong.

I removed all inclusions except the ones that you suggested.
I'll see if it improves anything

Originally Posted by StarBuG

Thank you

Is this proven or just a theory?

Google crawls around 2000-8000 pages a day average on my forum.
I don't think it will miss new content just because I have 180.000 links included into my sitemap.

I also checked my google analytics.
In the last 30 days I had 84 visitors via my tag pages so they are in the search results.
This is not much and of course dismissible but to say they don't get listed or found is wrong.

I removed all inclusions except the ones that you suggested.
I'll see if it improves anything

I tried now to make a quick search for "vBSEO" word in Google, There is many forums use auto tagger so I should found more than 20 tag page show on my results.
I Found tag in the page number 12

So You don't need to include it in your sitemap for 2 reason :
1- The huge number : In every thread you may found more than 4 tags and maybe more maybe less, one of my client is big board admin and he asked me to setting up his vBSEO and vBSEO Sitemap and I found he include tags so I found he have 19 sitemap 16 sitemap have only the tags pages.
2- The position : Why I subbmit this number of pages that's I sure they will not appear in the top 10 results ? the tags is appear in 10-12 pages which is very very very few people who view the 10 page of his search result, The people if they don't found what they want in the first or the second page they close the search or try another keyword.

Originally Posted by StarBuG

Thank you

Is this proven or just a theory?

Google crawls around 2000-8000 pages a day average on my forum.
I don't think it will miss new content just because I have 180.000 links included into my sitemap.

I also checked my google analytics.
In the last 30 days I had 84 visitors via my tag pages so they are in the search results.
This is not much and of course dismissible but to say they don't get listed or found is wrong.

I removed all inclusions except the ones that you suggested.
I'll see if it improves anything

The key to remember here is just because it's not in your sitemap doesn't mean it won't get indexed.

I agree with marco and do the same thing-- forumdisplay and showthread are the only 2 url sets I include in my sitemaps.
My meber pages, tag pages, blog pages, etc ARE indexed, I just don't care to put focus on any of them.

Originally Posted by marco1

I tried now to make a quick search for "vBSEO" word in Google, There is many forums use auto tagger so I should found more than 20 tag page show on my results.
I Found tag in the page number 12

I did the same search on google.de and it turns out that the first tags page is #9
which means first page

Originally Posted by Brian Cummiskey

The key to remember here is just because it's not in your sitemap doesn't mean it won't get indexed.

I agree with marco and do the same thing-- forumdisplay and showthread are the only 2 url sets I include in my sitemaps.
My meber pages, tag pages, blog pages, etc ARE indexed, I just don't care to put focus on any of them.

Valid point.
And I already turned it off myself.

But still, the argument of "Tag pages don't appear in any usefull position on SERPS" is not valid

StarBuG

Originally Posted by StarBuG

I did the same search on google.de and it turns out that the first tags page is #9
which means first page

Valid point.
And I already turned it off myself.

But still, the argument of "Tag pages don't appear in any usefull position on SERPS" is not valid

StarBuG

First page ? First page in the search results ? I don't think so.

http://www.google.de/#hl=de&source=h...seo&fp=1&cad=b

vBulletin Enterprise Translator (vBET) - Threads Tagged vBSEO

Position #9 on Google.de

Why should I lie?

I Don't said that , Google datacenters change the results as you country I use Google.com directly, Google.com not the same as google.de not the same as google.com.eg, So Does the tags pages give you the "good" traffic from search engines ? Check it via Google analytics if yes then index it and if no don't index it but it's in general doesn't send traffic or rarely send visitors so if you have more than 50,000 tags pages and they send only 20 or 14 visitor per day then don't index them.

Like I said in post #2, it is not much.

And I also deactivated it.
I just wanted to note, that tags pages sometimes are listed high in the rankings.

But also like Brian pointed out, not included does not mean not indexed.

StarBuG

But also like Brian pointed out, not included does not mean not indexed.

Sure, You doesn't disallow them you simply don't add them to Google bots directly via sitemaps.

And I also deactivated it.

That's nice

thanks for the post, I changed everything but I did select yes for the group discussion, why wouldn't we want that in the sitemap?

thanks again

Not all sites have an all too great group discussions, in-fact I think most done. I know on one of my personal sites, blogs have really started to kick off but I don't think I have seen a single decent group discussion.

Really it is personal preference but the idea is to only include the most important areas in your sitemap.

Agreed with you Ceri, Not all sites have groups discussion and I want to ask you question, Did you get any traffic from groups pages ? Personally I've never get any group discussion page in any of my search results, When you disable include something in your sitemap for example you disabled groups then that's don't mean tags pages will not index, NO Google can still index it but you simply don't include it in your sitemap it have less important than your forum threads and forum display pages.

i think that the SM 2.6 is a bit different or not?

It's a bit different on CMS pages only, But you can still use the same settings above, Posted it and updated it in the CMS now.