vBulletin SEO Forums

SEO

vBulletin Search Engine Optimization

Buy vBSEO Now! HACKER SAFE certified sites prevent over 99.9% of hacker crime.
ne nw
New vBSEO Discount Level for Network Builders Meet vBSEO Team in New York (Nov. 3rd & 4th) vBSEO 3.2.0 GOLD Has Landed Success with vBSEO = 600ore Web Visitors + $1400 in a Day! Crawlability Inc. Files for SEO Technology Patent
se sw

What not to include in Sitemap (and what to include in Robots.txt)

This is a discussion on What not to include in Sitemap (and what to include in Robots.txt) within the General Discussion forums, part of the vBSEO Google/Yahoo Sitemap category; At first: Total Indexed URLs: 2361 Then I excluded Show Post pages from Sitemap and it resulted in: Total Indexed ...

Go Back   vBulletin SEO Forums > vBSEO Google/Yahoo Sitemap > General Discussion

Enhancing 80 million pages.

Register FAQ Members List Social Groups Calendar Search Today's Posts Mark Forums Read
  #1  
Old 03-21-2008, 10:09 AM
Member
 
Real Name: Marvin Hlavac
Join Date: Oct 2007
Posts: 44
What not to include in Sitemap (and what to include in Robots.txt)

At first: Total Indexed URLs: 2361

Then I excluded Show Post pages from Sitemap

and it resulted in: Total Indexed URLs: 1169

Then I excluded Member Profile pages

and it resulted in: Total Indexed URLs: 630


My questions:
1) Would including Member Profile pages make any sense?
2) Should I worry that the number of indexed URL's dropped from over 2300 to just 600?
3) What else should I exclude?
4) The above was done in vBSEO Sitemate Settings. In addition to that, should I also add the above to Robots.txt?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2  
Old 03-21-2008, 10:40 AM
Mert Gökçeimam's Avatar
vBSEO.com Webmaster
vBSEO Total Customer SupportvBulletin HackerBig Board Administrator
 
Real Name: Lizard King
Join Date: Oct 2005
Location: Istanbul
Posts: 7,503
Blog Entries: 3
Send a message via MSN to Mert Gökçeimam
Hello Marvin ,

I personally donot add showpost and member profile urls within sitemaps cause i want bots to focus on my threads instead of member profiles. Indexed number of urls are not important. Important part is your SE rankings.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3  
Old 03-21-2008, 11:35 AM
Member
 
Real Name: Marvin Hlavac
Join Date: Oct 2007
Posts: 44
Thanks much, Mert. Other than the two items, what else would be recommended to exclude from Sitemap, and what items would be recommended to place into Robots.txt?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads

Thread Thread Starter Forum Replies Last Post
How do I include custom pages in the sitemap? amnesia623 Troubleshooting 2 02-03-2008 11:02 AM
Sitemap generator prefs - Include Old URLs - Yes or No? ehmax General Discussion 5 04-20-2007 09:43 PM
function.include !!! jacki Troubleshooting 4 09-21-2006 06:47 AM


All times are GMT -4. The time now is 06:47 PM.


Powered by vBulletin Version 3.8.0 Beta 3
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.5 ©2008, Crawlability, Inc.