vBulletin SEO Forums

SEO

vBulletin Search Engine Optimization

Buy vBSEO Now! HACKER SAFE certified sites prevent over 99.9% of hacker crime.
ne nw
vBSEO Total Support Team Launches DeskPro New vBSEO Discount Level for Network Builders vBSEO 3.2.0 GOLD Has Landed Success with vBSEO = 600ore Web Visitors + $1400 in a Day! Crawlability Inc. Files for SEO Technology Patent
se sw

Robot.txt

This is a discussion on Robot.txt within the Pre-Sales Questions forums, part of the Announcements & Pre-Sales category; hi when i add my site map in Google webmaster i receive an error : Code: Network unreachable: robots.txt unreachable ...

Go Back   vBulletin SEO Forums > Announcements & Pre-Sales > Pre-Sales Questions

Robot.txt Robot.txt

Enhancing 80 million pages.

 Register FAQ Members List Social Groups Calendar Search Today's Posts Mark Forums Read

Reply

 

LinkBack Thread Tools
  #1  
Old 09-02-2008, 07:58 AM
Junior Member
  
Real Name: susan
Join Date: Sep 2008
Location: IRAN
Posts: 23
Send a message via Yahoo to susanarend3
Question Robot.txt
hi

when i add my site map in Google webmaster i receive an error :


Code:
Network unreachable: robots.txt unreachable
We were unable to crawl your Sitemap because we found a robots.txt file at the root of your site but were unable to download it. Please ensure that it is accessible or remove it completely.









i delete this file but Google say again to me u have this File !!

please help me
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #2  
Old 09-02-2008, 12:18 PM
Junior Member
  
Real Name: susan
Join Date: Sep 2008
Location: IRAN
Posts: 23
Send a message via Yahoo to susanarend3
Hello !!
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #3  
Old 09-02-2008, 12:32 PM
Junior Member
  
Real Name: Krish
Join Date: Sep 2008
Posts: 9
Whats the permission of robots.txt file and whats the content do you have in it?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #4  
Old 09-02-2008, 12:58 PM
Junior Member
  
Real Name: susan
Join Date: Sep 2008
Location: IRAN
Posts: 23
Send a message via Yahoo to susanarend3
Code:
# Allow Archiver
User-agent: ia_archiver
Allow: /


User-agent: Slurp
Crawl-delay: 60


User-agent: *

Disallow: *.php
Disallow: *.js
Disallow: *.jsp
Disallow: *.cfm
Disallow: *.asp
Disallow: *.html
Disallow: *.htm
Disallow: *.aspx
Disallow: *.cgi

Disallow: /includes/
Disallow: /install/
Disallow: /customavatars/
Disallow: /archive/
Disallow: /sitemap/

Sitemap: http://forums.parsforums.net/sitemap_index.xml.gz
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #5  
Old 09-02-2008, 01:00 PM
Junior Member
  
Real Name: susan
Join Date: Sep 2008
Location: IRAN
Posts: 23
Send a message via Yahoo to susanarend3
is this ok ? or have a problem ?
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #6  
Old 09-02-2008, 01:11 PM
Senior Member
Big Board Administrator
  
Real Name: Alex
Join Date: Mar 2007
Location: Italy
Posts: 163
you need to wait maybe few hours or a day for the next attempt to crawl your site by google
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #7  
Old 09-02-2008, 01:20 PM
Junior Member
  
Real Name: Krish
Join Date: Sep 2008
Posts: 9
Where have you placed robots.txt?
check it using New Robots.txt Syntax Checker: a validator for robots.txt files
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #8  
Old 09-02-2008, 05:59 PM
Junior Member
  
Real Name: susan
Join Date: Sep 2008
Location: IRAN
Posts: 23
Send a message via Yahoo to susanarend3
thanks , its about 777 and 666 !!
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #9  
Old 09-02-2008, 06:01 PM
briansol's Avatar
Senior Member
vBSEO Pre-Release TeamDesign for SEOBig Board Administrator
  
Real Name: Brian
Join Date: Apr 2006
Location: Central CT, USA
Posts: 5,532
644 is generally the setting you want for regular files.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
Reply

« I have installed TfSEO, but now want to have vBSEO. Will it affect? | did vbseo v3 support arabic url? (charset=windows-1256) »
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

Similar Threads

Thread Thread Starter Forum Replies Last Post
Robot Text Do I need it? photochimper General Discussion 2 05-21-2008 12:31 AM
vbseo and robot.txt? Julio General Discussion 4 12-31-2007 06:25 PM
robot.txt dosyasında ne yazmalı bendensin890 Türkçe 9 02-28-2007 04:29 PM
Robot.txt file BamaStangGuy General Discussion 3 06-17-2006 01:09 PM
I need some help - concerned about robot activity Mike Off-Topic & Chit Chat 10 01-17-2006 10:31 AM


All times are GMT -4. The time now is 08:46 AM.

Contact Us - vBSEO Home - Site Map - Top

Powered by vBulletin Version 3.8.0 Beta 4
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.5 ©2008, Crawlability, Inc.