Results 1 to 11 of 11

Google webmaster tools - Errors for URLs in Sitemaps 4094URLs restricted by robots.txt 5201

This is a discussion on Google webmaster tools - Errors for URLs in Sitemaps 4094URLs restricted by robots.txt 5201 within the Analysis: Traffic & SERPS forums, part of the vBulletin SEO Discussion category; Errors for URLs in Sitemaps 4094 Don't like the sound of that, I've looked into this and I've changed my ...

  1. #1
    Member
    Real Name
    squeak
    Join Date
    Jun 2008
    Location
    UK
    Posts
    44
    Liked
    0 times

    Thumbs down Google webmaster tools - Errors for URLs in Sitemaps 4094

    Errors for URLs in Sitemaps 4094

    Don't like the sound of that, I've looked into this and I've changed my permalinks to how it's meant to be set up now, so hopefully the errors will start to decrease?

    URLs restricted by robots.txt 5201 - is there anyway I can stop the robots from looking at private forums? would this decrease the amount of errors in the sitemap URLS?

    Also the amount of pages indexed has gone down by about 2000 recently? does anyone know what would cause this when no settings had been changed?


    thanks if you can help
    Last edited by squeak; 07-19-2008 at 02:27 PM.

  2. #2
    Member
    Real Name
    squeak
    Join Date
    Jun 2008
    Location
    UK
    Posts
    44
    Liked
    0 times
    anyone know?

  3. #3
    Senior Member briansol's Avatar
    Real Name
    Brian
    Join Date
    Apr 2006
    Location
    Central CT, USA
    Posts
    6,981
    Liked
    8 times
    its ok. a 404 or a robots restriction is not ALWAYS a bad thing. it simply means google isn't going to index it. If it's truely not there, or not accessible, this is exactly what you want to see.


    NEVER judge your site by # of indexed pages. Juge by measuring your search engine traffic.

  4. #4
    Member
    Real Name
    squeak
    Join Date
    Jun 2008
    Location
    UK
    Posts
    44
    Liked
    0 times
    thats the thing, past few days the search engine traffic has suddenly dropped? Not sure why, maybe it'll pick back up soon

    it's gone from like 6000 to 200? (google)

  5. #5
    Senior Member briansol's Avatar
    Real Name
    Brian
    Join Date
    Apr 2006
    Location
    Central CT, USA
    Posts
    6,981
    Liked
    8 times
    did you sell links in your footer?

  6. #6
    Member
    Real Name
    squeak
    Join Date
    Jun 2008
    Location
    UK
    Posts
    44
    Liked
    0 times
    nope, never have done. it's gone down to 80 today so far from google?

  7. #7
    Senior Member
    Real Name
    Joseph Ward
    Join Date
    Jun 2005
    Posts
    23,847
    Liked
    32 times
    Blog Entries
    9
    Have you made any changes? It would seem like you were penalized or something is wrong. What is the contents of your robots.txt?

  8. #8
    Member
    Real Name
    squeak
    Join Date
    Jun 2008
    Location
    UK
    Posts
    44
    Liked
    0 times
    User-agent: *
    Disallow: /monoslideshow/
    Disallow: /vsadmin/
    Disallow: /shop/
    Disallow: /members/
    Disallow: /forums/admincp/
    Disallow: /forums/clientscript/
    Disallow: /forums/includes/
    Disallow: /forums/install/
    Disallow: /forums/modcp/
    Disallow: /forums/subscription.php
    Disallow: /forums/payments.php
    Disallow: /forums/profile.php
    Disallow: /forums/faq.php
    Disallow: /forums/calendar.php
    Disallow: /forums/search.php
    Disallow: /forums/private.php
    Disallow: /forums/online.php
    Disallow: /forums/sendmessage.php
    Disallow: /forums/sendmessage.php?do=
    Disallow: /forums/showgroups.php
    Disallow: /forums/reputation.php
    Disallow: /forums/report.php
    Disallow: /forums/threadrate.php
    Disallow: /forums/postings.php
    Disallow: /forums/newthread.php
    Disallow: /forums/newreply.php
    Disallow: /forums/register.php
    Disallow: /forums/login.php
    Disallow: /forums/faq.php
    Disallow: /forums/image.php
    Disallow: /forums/cron.php
    Disallow: /forums/joinrequests.php
    Disallow: /forums/printthread.php
    Disallow: /forums/showpost.php
    Disallow: /forums/archive/
    Sitemap: http://www.skyline-owners-club.com/f...p_index.xml.gz

  9. #9
    Member
    Real Name
    squeak
    Join Date
    Jun 2008
    Location
    UK
    Posts
    44
    Liked
    0 times
    I hope this is correct to how it should be

  10. #10
    Senior Member Lee G's Avatar
    Real Name
    Lee
    Join Date
    Sep 2006
    Location
    Costa Blanca
    Posts
    683
    Liked
    40 times
    Blog Entries
    4
    On your sitemaps over view page are you showing any errors there?

  11. #11
    Member
    Real Name
    squeak
    Join Date
    Jun 2008
    Location
    UK
    Posts
    44
    Liked
    0 times
    No errors at all, the numbers have started to rise though, not sure what was causing it but if they continue to rise I'll let you guys know. Thank you so far for you time

Similar Threads

  1. Indexed URLs in Sitemap (Google Webmaster Tools)
    By MikeWarner in forum General Discussion
    Replies: 0
    Last Post: 05-01-2008, 04:27 AM
  2. Multiple sitemaps in Google Webmaster Tools
    By serphog in forum General Discussion
    Replies: 2
    Last Post: 03-14-2008, 04:34 PM
  3. Google Webmaster Tools
    By OnR in forum Türkçe
    Replies: 2
    Last Post: 07-12-2007, 12:42 PM
  4. URLs restricted by robots.txt= 27187 :(
    By Bydigi in forum Türkçe
    Replies: 3
    Last Post: 12-03-2006, 06:53 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •