robots.txt + sitemap + rss feed?

Hello,

I read mert post about blocking rss feed with robots.txt file and i was wondering why block rss feed?

http://www.vbseo.com/f84/vbseo-terri...tml#post216008

Hello Julio ,

Rss feed is a content which is already created and 99% indexed by SE's . Do you really want your board filled with duplicate content thread ? These have no meaning for SE's as they are already indexed urls and what you do is only help the original content owner. You should focus on adding your own content instead of content via RSS feed.

Thanks Mert,

Can you post what i have to do so that is not included in my sitemap and what i need to add to my robots.txt file?

Hello Julio ,

You can exclude rss feed forum id's within vBSEO Sitemap Generator settings Excluded Forums List . You can also block these urls within robots.txt file , an example format to do it is :

Code:

Disallow: /rss-feed-forum-name/

So i would exclude all forums?

You should exclude forums that you are adding RSS feed

I guess that having "Disallow: *.php" in the robots.txt would block also rss feed if the rss URL is not vBSEO re written, right?

Hello Stavros ,

This threads subject is related to content that is added to your forum automatically within RSS feeds. Disallow. php will not help on this as thread urls are rewritten format.

Originally Posted by Mert Gökçeimam

This threads subject is related to content that is added to your forum automatically within RSS feeds.

Oh I got it now, sorry.

Thanks Mert

You're welcome