LinkBack URL
About LinkBacksIs this robots.txt file correct: http://www.amdforum.se/robots.txt
I have checked with Google webmaster tools and when I test some extensions in the robots.txt that should be blocked (for example calendar.php) it says that it is allowed by Google bot.
Results 1 to 2 of 2
Robots.txt
This is a discussion on Robots.txt within the General Discussion forums, part of the vBSEO SEO Plugin category; Is this robots.txt file correct: http://www.amdforum.se/robots.txt I have checked with Google webmaster tools and when I test some extensions in ...
-
06-27-2010 06:08 PM #1Senior Member Array
- Real Name
- Daniel
- Join Date
- Dec 2009
- Location
- Sweden
- Posts
- 304
- Liked
- 4 times
Robots.txt
-
06-27-2010 06:32 PM #2vBSEO Staff Array
- Real Name
- Brian Cummiskey
- Join Date
- Jul 2009
- Location
- btwn NYC and Boston
- Posts
- 12,789
- Liked
- 675 times
- Blog Entries
- 2
It looks fine. Your calendar is disabled for guests anyway, so at most google is going to see is the login box.
Remember, the robots file is simply a SUGGESTION of areas to avoid crawling. It does not necessarily BLOCK them.Brian Cummiskey / Crawlability Inc.
Security bulletin - Patch Level for all supported versions released
Unveiling the NEW vBSEO Sitemap Generator 3.0. - available NOW for vBSEO Customers!
Reply With QuoteSimilar Threads
-
Mydomain.com/robots.txt | Mydomain.com/forum/robots.txt : How to config ?
By setubal in forum General DiscussionReplies: 6Last Post: 10-19-2009, 10:55 AM -
robots.txt
By zems in forum General DiscussionReplies: 0Last Post: 05-03-2009, 03:15 PM -
Redirecting /forums/robots.txt to /robots.txt - Is it good?
By MadK in forum Custom Rewrite RulesReplies: 6Last Post: 08-22-2008, 06:29 PM -
About robots.txt
By Diver in forum General DiscussionReplies: 1Last Post: 02-18-2008, 07:18 PM -
No Robots.txt
By friendly in forum TroubleshootingReplies: 1Last Post: 10-26-2006, 10:30 AM
