vBulletin Search Engine Optimization
This is a discussion on Robot.txt within the Pre-Sales Questions forums, part of the Announcements & Pre-Sales category; hi when i add my site map in Google webmaster i receive an error : Code: Network unreachable: robots.txt unreachable ...
| |||||||
Enhancing 80 million pages. | Register | FAQ | Members List | Social Groups | Calendar | Search | Today's Posts | Mark Forums Read |
|
#1
| |||
| |||
|
hi when i add my site map in Google webmaster i receive an error : Code: Network unreachable: robots.txt unreachable We were unable to crawl your Sitemap because we found a robots.txt file at the root of your site but were unable to download it. Please ensure that it is accessible or remove it completely. ![]() ![]() i delete this file but Google say again to me u have this File !! please help me |
|
#2
| |||
| |||
|
Hello !!
|
|
#3
| |||
| |||
|
Whats the permission of robots.txt file and whats the content do you have in it?
|
|
#4
| |||
| |||
| Code: # Allow Archiver User-agent: ia_archiver Allow: / User-agent: Slurp Crawl-delay: 60 User-agent: * Disallow: *.php Disallow: *.js Disallow: *.jsp Disallow: *.cfm Disallow: *.asp Disallow: *.html Disallow: *.htm Disallow: *.aspx Disallow: *.cgi Disallow: /includes/ Disallow: /install/ Disallow: /customavatars/ Disallow: /archive/ Disallow: /sitemap/ Sitemap: http://forums.parsforums.net/sitemap_index.xml.gz |
|
#5
| |||
| |||
|
is this ok ? or have a problem ?
|
|
#6
| |||
| |||
|
you need to wait maybe few hours or a day for the next attempt to crawl your site by google
|
|
#7
| |||
| |||
|
Where have you placed robots.txt? check it using New Robots.txt Syntax Checker: a validator for robots.txt files |
|
#8
| |||
| |||
|
thanks , its about 777 and 666 !!
|
|
#9
| ||||
| ||||
|
644 is generally the setting you want for regular files.
|
| Thread Tools | |
| |
Similar Threads | ||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Robot Text Do I need it? | photochimper | General Discussion | 2 | 05-21-2008 12:31 AM |
| vbseo and robot.txt? | Julio | General Discussion | 4 | 12-31-2007 06:25 PM |
| robot.txt dosyasında ne yazmalı | bendensin890 | Türkçe | 9 | 02-28-2007 04:29 PM |
| Robot.txt file | BamaStangGuy | General Discussion | 3 | 06-17-2006 01:09 PM |
| I need some help - concerned about robot activity | Mike | Off-Topic & Chit Chat | 10 | 01-17-2006 10:31 AM |