vBulletin SEO Forums

SEO

vBulletin Search Engine Optimization

Buy vBSEO Now! HACKER SAFE certified sites prevent over 99.9% of hacker crime.
ne nw
New vBSEO Discount Level for Network Builders Meet vBSEO Team in New York (Nov. 3rd & 4th) vBSEO 3.2.0 GOLD Has Landed Success with vBSEO = 600ore Web Visitors + $1400 in a Day! Crawlability Inc. Files for SEO Technology Patent
se sw

New sitemap problem...

This is a discussion on New sitemap problem... within the Troubleshooting forums, part of the vBSEO Google/Yahoo Sitemap category; I'm getting this error... URL restricted by robots.txt We encountered an error while trying to access your Sitemap. Please ensure ...

Go Back   vBulletin SEO Forums > vBSEO Google/Yahoo Sitemap > Troubleshooting

Enhancing 80 million pages.

Register FAQ Members List Social Groups Calendar Search Today's Posts Mark Forums Read
  #1  
Old 09-20-2008, 04:07 PM
SIINSI's Avatar
Member
Big Board Administrator
 
Real Name: Miguel Diaz
Join Date: Sep 2008
Location: Palm Coast, FL
Posts: 74
Send a message via AIM to SIINSI
Unhappy New sitemap problem...

I'm getting this error...

URL restricted by robots.txt
We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit.

I resubmitted and got the same error.

This is the sitemap URL I submitted:
http://www.jdmuniverse.com/forums/sitemap_index.xml.gz

Here is my robots.txt:

User-agent: Mediapartners-Google*
Disallow:
# Allow Archiver
User-agent: ia_archiver
Allow: /
User-agent: Slurp
Crawl-delay: 60

User-agent: *

Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/faq.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/payments.php
Disallow: /forums/showpost.php
Disallow: /forums/showgroups.php
Disallow: /forums/spiders.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/admincp/
Disallow: /cgi-bin/
Disallow: /forums/includes/
Disallow: /forums/install/
Disallow: /forums/modcp/
Disallow: /customavatars/
Disallow: /forums/xmlrpc.php
Disallow: /forums/archive/
Disallow: /forums/sitemap/
Disallow: /forums/gallery/member.php
Disallow: /forums/gallery/showgallery.php?cat=
Disallow: /forums/gallery/showgallery.php?limit=
Disallow: /forums/gallery/comshow.php
Disallow: /forums/gallery/search.php
Disallow: /forums/gallery/*search.php
Disallow: /forums/gallery/showframe.php
Disallow: /forums/gallery/slideshow.php
Disallow: /forums/gallery/*limit
Disallow: /forums/gallery/*ppuser
Disallow: /forums/gallery/*views
Disallow: /forums/gallery/*date
Disallow: /forums/gallery/*recent
Disallow: /forums/gallery/*in
Sitemap: http://www.jdmuniverse.com/forums/sitemap_index.xml.gz


Any idea what's wrong? :(

Miguel
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2  
Old 09-20-2008, 04:34 PM
SIINSI's Avatar
Member
Big Board Administrator
 
Real Name: Miguel Diaz
Join Date: Sep 2008
Location: Palm Coast, FL
Posts: 74
Send a message via AIM to SIINSI
Double Post.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3  
Old 09-20-2008, 04:34 PM
SIINSI's Avatar
Member
Big Board Administrator
 
Real Name: Miguel Diaz
Join Date: Sep 2008
Location: Palm Coast, FL
Posts: 74
Send a message via AIM to SIINSI
Wow, I just used a robots.txt generator to allow all engines and I still get the error :(

Here's my new robots.txt

# robots.txt generated at International SEO | SEM | PPC - McAnerin International Inc.
User-agent: *
Disallow:
Disallow: /cgi-bin/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/faq.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/payments.php
Disallow: /forums/showpost.php
Disallow: /forums/showgroups.php
Disallow: /forums/spiders.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/admincp/
Disallow: /cgi-bin/
Disallow: /forums/includes/
Disallow: /forums/install/
Disallow: /forums/modcp/
Disallow: /customavatars/
Disallow: /forums/xmlrpc.php
Disallow: /forums/archive/
Disallow: /forums/sitemap/
Disallow: /forums/gallery/member.php
Disallow: /forums/gallery/showgallery.php?cat=
Disallow: /forums/gallery/showgallery.php?limit=
Disallow: /forums/gallery/comshow.php
Disallow: /forums/gallery/search.php
Disallow: /forums/gallery/*search.php
Disallow: /forums/gallery/showframe.php
Disallow: /forums/gallery/slideshow.php
Disallow: /forums/gallery/*limit
Disallow: /forums/gallery/*ppuser
Disallow: /forums/gallery/*views
Disallow: /forums/gallery/*date
Disallow: /forums/gallery/*recent
Disallow: /forums/gallery/*in
Sitemap: http://www.jdmuniverse.com/forums/sitemap_index.xml.gz


What on Earth could be the problem?

Miguel
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4  
Old 09-20-2008, 05:54 PM
SIINSI's Avatar
Member
Big Board Administrator
 
Real Name: Miguel Diaz
Join Date: Sep 2008
Location: Palm Coast, FL
Posts: 74
Send a message via AIM to SIINSI
I just completely deleted robots.txt and I still get this error when trying to submit my sitemap!

URL restricted by robots.txt
We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit.

Miguel
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5  
Old 09-20-2008, 06:10 PM
briansol's Avatar
Senior Member
vBSEO Pre-Release TeamDesign for SEOBig Board Administrator
 
Real Name: Brian
Join Date: Apr 2006
Location: Central CT, USA
Posts: 5,473
get rid of the blank disallow at the top.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6  
Old 09-20-2008, 06:17 PM
SIINSI's Avatar
Member
Big Board Administrator
 
Real Name: Miguel Diaz
Join Date: Sep 2008
Location: Palm Coast, FL
Posts: 74
Send a message via AIM to SIINSI
Now I used Google's robots.txt generator and made a file with only this:

User-Agent: *
Allow: /


I still get the same error. This is killing me.

Miguel
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7  
Old 09-20-2008, 06:28 PM
SIINSI's Avatar
Member
Big Board Administrator
 
Real Name: Miguel Diaz
Join Date: Sep 2008
Location: Palm Coast, FL
Posts: 74
Send a message via AIM to SIINSI
I even added allows for all googles bots... :( Same error.

# robots.txt generated at International SEO | SEM | PPC - McAnerin International Inc.
User-agent: *
User-Agent: *
Allow: /
User-Agent: Mediapartners-Google
Allow: /
User-Agent: Googlebot
Allow: /
User-Agent: Adsbot-Google
Allow: /
User-Agent: Googlebot-Image
Allow: /
User-Agent: Googlebot-Mobile
Allow: /
Disallow: /cgi-bin/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/faq.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/payments.php
Disallow: /forums/showpost.php
Disallow: /forums/showgroups.php
Disallow: /forums/spiders.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
Disallow: /forums/admincp/
Disallow: /forums/includes/
Disallow: /forums/install/
Disallow: /forums/modcp/
Disallow: /customavatars/
Disallow: /forums/xmlrpc.php
Disallow: /forums/archive/
Disallow: /forums/sitemap/
Disallow: /forums/gallery/member.php
Disallow: /forums/gallery/showgallery.php?cat=
Disallow: /forums/gallery/showgallery.php?limit=
Disallow: /forums/gallery/comshow.php
Disallow: /forums/gallery/search.php
Disallow: /forums/gallery/*search.php
Disallow: /forums/gallery/showframe.php
Disallow: /forums/gallery/slideshow.php
Disallow: /forums/gallery/*limit
Disallow: /forums/gallery/*ppuser
Disallow: /forums/gallery/*views
Disallow: /forums/gallery/*date
Disallow: /forums/gallery/*recent
Disallow: /forums/gallery/*in
Sitemap: http://www.jdmuniverse.com/forums/sitemap_index.xml.gz
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8  
Old 09-20-2008, 07:16 PM
SIINSI's Avatar
Member
Big Board Administrator
 
Real Name: Miguel Diaz
Join Date: Sep 2008
Location: Palm Coast, FL
Posts: 74
Send a message via AIM to SIINSI
OK, when I go into tools of G's site and click analyze robots.txt, it still shows the file in the first post. It says last downloaded 14 hours ago. I guess I have to wait until it DL's it again. Hopefully then I can submit my sitemap.

Miguel
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads

Thread Thread Starter Forum Replies Last Post
Sitemap Problem... Please help me... MorYeL Troubleshooting 3 04-19-2008 07:02 PM
Sitemap Problem sokorelis Troubleshooting 13 02-22-2008 01:49 PM
Sitemap-Problem fide Deutsch 28 12-21-2007 06:36 AM
Problem with Sitemap! MUAHHH Troubleshooting 7 05-03-2007 05:36 AM
Sitemap problem vargrider General Discussion 3 05-17-2006 07:38 AM


All times are GMT -4. The time now is 05:12 PM.


Powered by vBulletin Version 3.8.0 Beta 3
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.5 ©2008, Crawlability, Inc.