Hey people,
This morning, (after altering my sitemap settings to those seen in Briansol's Ultimate Guide to vBSEO), this morning I logged into google webmaster tools and noticed I have a sitemap warning of:
Google then pointed out the URL's in question, which are:When we tested a sample of the URLs from your Sitemap, we found that the site's robots.txt file was blocking access to some of the URLs.
If you don't intend to block some of the URLs contained in the Sitemap, please use our robots.txt analysis tool to verify that the URLs you submitted in your Sitemap are accessible by Googlebot. All accessible URLs will still be submitted
www . stackleaders.com/forum/archive/index.php/f-15.html
www . stackleaders.com/forum/members/royalflushrob/
www . stackleaders.com/forum/members/josh/
Does anyone have any idea whats happening?
It looks to me like the sitemap is telling google to crawl some pages which are blocked by my robots.txt file. For SEO purposes, I do not wish to have the archive crawled (as stated in the top seo guide link), and nor would I like to have the members profiles crawled either.
What's weird though, is I had a look in my sitemap file and i couldn't see any of the above urls listed - so i'm presuming it's some kind of cross over between me doing those changes and submitting the new sitemap, and google crawling the new sitemap?


LinkBack URL
About LinkBacks





Reply With Quote
