Mydomain.com/robots.txt | Mydomain.com/forum/robots.txt : How to config ?

[setubal]

Hello,

I know, we need only 1 robots.txt on a website.

But I've got 2 robots.
1 Root (http://www.mysite.com/robots.txt)
1 Forum folder (http://www.mysite.com/forum/robots.txt)

Question :
How can I move my robots.txt Forum folder TO Mydomain.com/robots.txt :

My http://www.mydomain.com/forum/robotx.tx looks like that :
# Comment the following line (add '#' at the beginning)
# to disable mod_rewrite functions.
# Please note: you still need to disable the hack in
# the vBSEO control panel to stop url rewrites.
RewriteEngine On

# Some servers require the Rewritebase directive to be
# enabled (remove '#' at the beginning to activate)
# Please note: when enabled, you must include the path
# to your root vB folder (i.e. RewriteBase /forum/)
#RewriteBase /

RewriteCond %{HTTP_HOST} !^www\.mydomain\.com
RewriteRule (.*) http://www.mydomain.com/forum/$1 [R=301,L]

RewriteRule ^((urllist|sitemap_).*\.(xml|txt)(\.gz)?)$ vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|cron)
RewriteRule ^((archive/)?(.*\.php(/.*)?)?)$ vbseo.php [L,QSA]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !^(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule ^(.+)$ vbseo.php [L,QSA]

What should it looks like If I move it to my http://www.mydomain.com/robots.txt ?


----------- Like that ? It is correct please ? ------------------
# Comment the following line (add '#' at the beginning)
# to disable mod_rewrite functions.
# Please note: you still need to disable the hack in
# the vBSEO control panel to stop url rewrites.
RewriteEngine On

# Some servers require the Rewritebase directive to be
# enabled (remove '#' at the beginning to activate)
# Please note: when enabled, you must include the path
# to your root vB folder (i.e. RewriteBase /forum/)
#RewriteBase /

RewriteCond %{HTTP_HOST} !^www\.mydomain\.com
RewriteRule (.*) http://www.mydomain.com/forum/$1 [R=301,L]

RewriteRule ^((urllist|sitemap_).*\.(xml|txt)(\.gz)?)$ forum/vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|cron)
RewriteRule ^((archive/)?(.*\.php(/.*)?)?)$ vbseo.php [L,QSA]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !^(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule ^(.+)$ forum/vbseo.php [L,QSA]

Thank you.

[Mert Gökçeimam]

Hello ,

Robots.txt file should only reside on your root folder.

[setubal]

Yes,
My root folder is www.mydomain.com
But my Forum is www.mydomain.com/forum/

My robots.txt should be in www.mydomain.com/robots.txt

The question is :
This robots.txt it is correct ? (the paths in purple color) ?

Thx !

# Comment the following line (add '#' at the beginning)
# to disable mod_rewrite functions.
# Please note: you still need to disable the hack in
# the vBSEO control panel to stop url rewrites.
RewriteEngine On

# Some servers require the Rewritebase directive to be
# enabled (remove '#' at the beginning to activate)
# Please note: when enabled, you must include the path
# to your root vB folder (i.e. RewriteBase /forum/)
#RewriteBase /

RewriteCond %{HTTP_HOST} !^www\.mydomain\.com
RewriteRule (.*) http://www.mydomain.com/forum/$1 [R=301,L]

RewriteRule ^((urllist|sitemap_).*\.(xml|txt)(\.gz)?)$ forum/vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

RewriteCond %{REQUEST_URI} !(admincp/|modcp/|cron)
RewriteRule ^((archive/)?(.*\.php(/.*)?)?)$ vbseo.php [L,QSA]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !^(admincp|modcp|clientscript|cpstyles|images)/
RewriteRule ^(.+)$ forum/vbseo.php [L,QSA]

[Mert Gökçeimam]

You should use your sitemap url in your robots.txt file

User-agent: *
Disallow: 
Sitemap: http://www.mydomain.com/forum/sitemap_index.xml.gz

You need to upload default vBSEO .htaccess file to your forum folder and should not add the purple parts in .htaccess file.

[setubal]

I confused between robots and htaccess.. I'm tired sorry.

[Brian Cummiskey]

htaccess is a file your SERVER uses as instructions for apache/etc.

robots.txt is a file (that only works in root) that provides instructions for OUTSIDE services/bots.

[setubal]

Thanks you all.
It's time for me to go bed ^^ lol

Thank you team