robots.txt denied indexing of my forum

[m0rgulvale]

hi, i noticed lots of site map errors from google webmaster tools,
for example...

http://laserpointerforums.com/f37/hi-39604.html
URL restricted by robots.txt
Jul 2, 2009
http://laserpointerforums.com/f37/hi-39790.html
URL restricted by robots.txt
Jul 1, 2009
http://laserpointerforums.com/f37/hi-40279.html
URL restricted by robots.txt
Jul 1, 2009
http://laserpointerforums.com/f37/hi-all-something-new-39376.html
URL restricted by robots.txt
Jul 4, 2009
http://laserpointerforums.com/f37/hi-d-39772.html
URL restricted by robots.txt
Jul 6, 2009
http://laserpointerforums.com/f37/hi-everybody-can-we-become-friends-39768.html
URL restricted by robots.txt
Jul 6, 2009
http://laserpointerforums.com/f37/hi-kansas-u-s-39551.html
URL restricted by robots.txt
Jul 3, 2009
http://laserpointerforums.com/f37/hi-laz0r-lovers-39434.html
URL restricted by robots.txt
Jul 3, 2009
http://laserpointerforums.com/f37/hi-vancouver-island-41660.html
URL restricted by robots.txt
Jul 2, 2009
http://laserpointerforums.com/f37/how-1-rep-someone-39581.html
URL restricted by robots.txt
Jul 2, 2009
http://laserpointerforums.com/f37/how-rep-someone-39447.html
URL restricted by robots.txt
Jul 4, 2009
http://laserpointerforums.com/f37/howdy-41547.html
URL restricted by robots.txt
Jun 28, 2009
http://laserpointerforums.com/f37/hoy-all-40720.html
URL restricted by robots.txt
Jul 9, 2009
http://laserpointerforums.com/f37/i14.html
URL restricted by robots.txt
Jun 30, 2009
http://laserpointerforums.com/f37/i25.html
URL restricted by robots.txt
Jul 1, 2009
http://laserpointerforums.com/f37/im-back-40834.html
URL restricted by robots.txt
Jun 28, 2009
http://laserpointerforums.com/f37/introduction-39798.html
URL restricted by robots.txt
Jul 1, 2009
http://laserpointerforums.com/f37/itsss-tarek-39714.html
URL restricted by robots.txt
Jul 6, 2009
http://laserpointerforums.com/f37/joining-buy-lasers-40266.html

at the time, my robots.txt was

# Allow Archiver
User-agent: ia_archiver
Allow: /


User-agent: Slurp
Crawl-delay: 60


User-agent: *

Disallow: *.js
Disallow: *.jsp
Disallow: *.cfm
Disallow: *.asp
Disallow: *.htm
Disallow: *.aspx
Disallow: *.cgi

Disallow: /includes/
Disallow: /install/
Disallow: /customavatars/
Disallow: /archive/
Disallow: /vbseo_sitemap/
Disallow: /referrers.php
Disallow: /ajax_cron.php
Disallow: /ajax.php
Disallow: /attachment.php
Disallow: /calendar.php
Disallow: /cron.php
Disallow: /editpost.php
Disallow: /global.php
Disallow: /image.php
Disallow: /inlinemod.php
Disallow: /joinrequests.php
Disallow: /login.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /poll.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /showgroups.php
Disallow: /subscription.php
Disallow: /threadrate.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /member/
Disallow: /members/
Disallow: /tags/

Sitemap: http://laserpointerforums.com/sitemap_index.xml.gz

what was wrong?


my new robots.txt is now like vbseo.com's

User-agent: *
Disallow: /*-print/
Disallow: /vbseocp.php
Disallow: /login.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /newthread.php
Disallow: /newreply.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /search.php
Disallow: /sendmessage.php
Sitemap: http://laserpointerforums.com/sitemap_index.xml.gz

have i solved the problem? what was the issue before?


thanks for any suggestions.. i need to get this right once and for all

peace

[Michael Biddle]

I do not know what it was showing that. I do not know if blocking .htm also blocks .html.

You can still block the *.php files though.

[Joe Ward]

Blocking htm does not block html. Both must be specified individually.