Results 1 to 8 of 8

Sitemap errors/robots.txt

This is a discussion on Sitemap errors/robots.txt within the General Discussion forums, part of the vBSEO Google/Yahoo Sitemap category; I have this as an explanation for sitemap_1.xml/gz. A status of a red X, with 96 URL's submitted and 85 ...

  1. #1
    Senior Member
    Real Name
    Robert
    Join Date
    Jul 2009
    Posts
    140
    Liked
    0 times

    Smile Sitemap errors/robots.txt

    I have this as an explanation for sitemap_1.xml/gz. A status of a red X, with 96 URL's submitted and 85 indexed URL's

    Network unreachable: robots.txt unreachable
    We were unable to crawl your Sitemap because we found a robots.txt file at the root of your site but were unable to download it. Please ensure that it is accessible or remove it completely.

    I have 55 crawl errors "In Sitemaps" and 277 "Unreachable"

    Is all this stuff because my site is still be indexed by Google? Or, is this something I need to fix and how do I fix it?

    Thanks!!

  2. #2
    Senior Member
    Real Name
    Marco Mamdouh
    Join Date
    May 2008
    Location
    Egypt
    Posts
    2,627
    Liked
    1 times
    Blog Entries
    1

  3. #3
    Senior Member
    Real Name
    Robert
    Join Date
    Jul 2009
    Posts
    140
    Liked
    0 times
    What would be the url to check my sitemap?

    Thanks!

  4. #4
    Senior Member
    Real Name
    Marco Mamdouh
    Join Date
    May 2008
    Location
    Egypt
    Posts
    2,627
    Liked
    1 times
    Blog Entries
    1
    After your forum URL add
    Code:
    /sitemap_index.xml.gz

  5. #5
    Senior Member
    Real Name
    Robert
    Join Date
    Jul 2009
    Posts
    140
    Liked
    0 times
    Ok, thanks.

    I did that and I get an internal server error

  6. #6
    Senior Member
    Real Name
    Marco Mamdouh
    Join Date
    May 2008
    Location
    Egypt
    Posts
    2,627
    Liked
    1 times
    Blog Entries
    1

  7. #7
    Senior Member
    Real Name
    Robert
    Join Date
    Jul 2009
    Posts
    140
    Liked
    0 times
    OK, thanks. I guess I don't have a sitemap for my frontpage?

    I don't think I have a robots file for the /forums

    What should I include/exclude when I do one?

  8. #8
    Senior Member
    Real Name
    Marco Mamdouh
    Join Date
    May 2008
    Location
    Egypt
    Posts
    2,627
    Liked
    1 times
    Blog Entries
    1
    You didn't need to have your robots in your /forums
    robots file should be in your site root not in your forum root
    Your site is correct i didn't recommended to make any changes at this time

Similar Threads

  1. robots.txt + sitemap + rss feed?
    By Julio in forum General Discussion
    Replies: 9
    Last Post: 06-02-2009, 11:00 AM
  2. robots.txt errors
    By AdamFL in forum Troubleshooting
    Replies: 4
    Last Post: 01-31-2009, 08:34 PM
  3. Replies: 10
    Last Post: 07-21-2008, 07:04 PM
  4. Robots.txt now support the sitemap file
    By Mohamed Mostafa in forum General Discussion
    Replies: 35
    Last Post: 08-24-2007, 10:33 AM
  5. add sitemap to robots.txt for autodiscovery
    By GrendelKhan{TSU} in forum General Discussion
    Replies: 2
    Last Post: 04-13-2007, 03:21 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •