Results 1 to 3 of 3

Google ignores my robots.txt

This is a discussion on Google ignores my robots.txt within the General Discussion forums, part of the vBulletin SEO Discussion category; Hello, it is now a month ago i uploadet this robots.txt, but google still visits the member.php and calendar.php. Is ...

  1. #1
    Junior Member
    Real Name
    Test
    Join Date
    Apr 2007
    Posts
    7
    Liked
    0 times

    Question Google ignores my robots.txt

    Hello, it is now a month ago i uploadet this robots.txt, but google still visits the member.php and calendar.php. Is my robots.txt correct? Does it make a difference if I save the robots.txt with wordpad instead of editor?

    Code:
    User-agent: *
    Disallow: /admincp/
    Disallow: /arcade/
    Disallow: /archive/
    Disallow: /geek/
    Disallow: /clientscript/
    Disallow: /cpstyles/
    Disallow: /customavatars/
    Disallow: /images/
    Disallow: /includes/
    Disallow: /install/
    Disallow: /modcp/
    Disallow: /sitemap/
    Disallow: /web967h/
    Disallow: /web97f1/
    Disallow: /web98qJ/
    Disallow: /web99l2/
    Disallow: /template_cache/
    
    User-agent: *
    Disallow: /ajax.php
    Disallow: /announcement.php
    Disallow: /arcade.php
    Disallow: /attachment.php
    Disallow: /autolink.php
    Disallow: /calendar.php
    Disallow: /credits.php
    Disallow: /cron.php
    Disallow: /editpost.php
    Disallow: /external.php
    Disallow: /faq.php
    Disallow: /global.php
    Disallow: /glossary.php
    Disallow: /holdsession.php
    Disallow: /image.php
    Disallow: /infraction.php
    Disallow: /inlinemod.php
    Disallow: /joinrequests.php
    Disallow: /login.php
    Disallow: /member.php
    Disallow: /memberlist.php
    Disallow: /mh_mv_membersvisited.php
    Disallow: /misc.php
    Disallow: /moderation.php
    Disallow: /moderator.php
    Disallow: /newattachment.php
    Disallow: /newreply.php
    Disallow: /newthread.php
    Disallow: /online.php
    Disallow: /payment_gateway.php
    Disallow: /payments.php
    Disallow: /post_thanks.php
    Disallow: /postage.php
    Disallow: /postings.php
    Disallow: /printthread.php
    Disallow: /private.php
    Disallow: /profile.php
    Disallow: /register.php
    Disallow: /report.php
    Disallow: /reputation.php
    Disallow: /search.php
    Disallow: /sendmessage.php
    Disallow: /showgroups.php
    Disallow: /subscription.php
    Disallow: /threadrate.php
    Disallow: /threadprefix.php
    Disallow: /usercp.php
    Disallow: /usernote.php
    Disallow: /vbpost_ajax.php
    Disallow: /vbseo.php
    Disallow: /vbseocp.php
    Disallow: /vbseocpform.php
    Disallow: /warn.php
    
    Sitemap: http://www.shokmuzik.net/sitemap_index.xml.gz

  2. #2
    vBSEO Staff Oleg Ignatiuk's Avatar
    Real Name
    Oleg Ignatiuk
    Join Date
    Jun 2005
    Location
    Belarus
    Posts
    25,744
    Liked
    168 times
    Hello,

    if you have your member profile URLs rewritten, you should add this in robots.txt:
    Disallow: /members/

  3. #3
    Senior Member BamaStangGuy's Avatar
    Real Name
    Brent Wilson
    Join Date
    Aug 2005
    Location
    Huntsville, Alabama
    Posts
    2,484
    Liked
    1 times
    They visit your members.php because you rewrite your username urls.

Similar Threads

  1. Google Now Reporting Anchor Text Phrases
    By Eros in forum General Discussion
    Replies: 4
    Last Post: 02-11-2009, 03:42 PM
  2. Replies: 2
    Last Post: 03-16-2007, 12:58 PM
  3. Matt Cutts - SEO Videos on Google Video
    By Joe Ward in forum General Discussion
    Replies: 9
    Last Post: 08-08-2006, 07:05 PM
  4. Google is All About Search: 4 New Products
    By Joe Ward in forum General Discussion
    Replies: 0
    Last Post: 05-16-2006, 01:52 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •