Results 1 to 3 of 3

Google and robots.txt

This is a discussion on Google and robots.txt within the General Discussion forums, part of the vBSEO Google/Yahoo Sitemap category; When I inspect the Google Webmaster Tools page under robots.txt analysis , I see this message: This site is not ...

  1. #1
    Senior Member
    Real Name
    jdougher
    Join Date
    Feb 2007
    Posts
    118
    Liked
    0 times

    Google and robots.txt

    When I inspect the Google Webmaster Tools page under robots.txt analysis, I see this message:

    This site is not located at the top level for the domain. A robots.txt file is only valid when located in the highest-level directory and applies to all directories within the domain. The robots.txt file that applies to your site (if one exists) is located at http://www.photocamel.com/robots.txt. This page provides information on that file.

    What am I doing wrong here?

    Thanks.

  2. #2
    Senior Member
    Real Name
    Michael
    Join Date
    Oct 2005
    Posts
    1,759
    Liked
    1 times
    Blog Entries
    1
    In your case, since you are redirecting the root to /forum, your highest level is /forum.

    Google's spiders don't recognize photocamel.com as an existing page.

    Rather than moving your robots.txt file, I would recommend turning off that redirect and putting in a homepage, even if it is something like vbadvanced.

  3. #3
    Senior Member
    Real Name
    jdougher
    Join Date
    Feb 2007
    Posts
    118
    Liked
    0 times
    Quote Originally Posted by Mike View Post
    In your case, since you are redirecting the root to /forum, your highest level is /forum.

    Google's spiders don't recognize photocamel.com as an existing page.

    Rather than moving your robots.txt file, I would recommend turning off that redirect and putting in a homepage, even if it is something like vbadvanced.

    I suspect that Google should be able to cope with /forum - the question is whether having a redirect in .htaccess causes Google to ignore the redirected link

    It would be worth getting some more understanding of the effects of redirects - both in .htaccess and httpd.conf



Similar Threads

  1. Replies: 1
    Last Post: 11-26-2006, 03:48 PM
  2. Google Webmaster Tools - Matt Cutts Video
    By Joe Ward in forum General Discussion
    Replies: 2
    Last Post: 11-01-2006, 01:33 AM
  3. Google Sitemap-Listing of old url's option-from 1.7
    By T2DMan in forum Member Articles
    Replies: 0
    Last Post: 04-02-2006, 08:46 AM
  4. Google Spidering: Too much of a coincidence to not be true...
    By Keith Cohen in forum Analysis: Traffic & SERPS
    Replies: 7
    Last Post: 12-29-2005, 12:41 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •