vBulletin SEO Forums

SEO

vBulletin Search Engine Optimization

Buy vBSEO Now! HACKER SAFE certified sites prevent over 99.9% of hacker crime.
ne nw
vBSEO Total Support Team Launches DeskPro New vBSEO Discount Level for Network Builders vBSEO 3.2.0 GOLD Has Landed Success with vBSEO = 600ore Web Visitors + $1400 in a Day! Crawlability Inc. Files for SEO Technology Patent
se sw

Google ignores my robots.txt

This is a discussion on Google ignores my robots.txt within the General Discussion forums, part of the vBulletin SEO Discussion category; Hello, it is now a month ago i uploadet this robots.txt, but google still visits the member.php and calendar.php. Is ...

Go Back   vBulletin SEO Forums > vBulletin SEO Discussion > General Discussion

Enhancing 80 million pages.

Register FAQ Members List Social Groups Calendar Search Today's Posts Mark Forums Read
  #1  
Old 10-25-2007, 08:02 PM
Junior Member
 
Real Name: shokmuzik.com
Join Date: Apr 2007
Posts: 6
Question Google ignores my robots.txt

Hello, it is now a month ago i uploadet this robots.txt, but google still visits the member.php and calendar.php. Is my robots.txt correct? Does it make a difference if I save the robots.txt with wordpad instead of editor?

Code:
User-agent: *
Disallow: /admincp/
Disallow: /arcade/
Disallow: /archive/
Disallow: /geek/
Disallow: /clientscript/
Disallow: /cpstyles/
Disallow: /customavatars/
Disallow: /images/
Disallow: /includes/
Disallow: /install/
Disallow: /modcp/
Disallow: /sitemap/
Disallow: /web967h/
Disallow: /web97f1/
Disallow: /web98qJ/
Disallow: /web99l2/
Disallow: /template_cache/

User-agent: *
Disallow: /ajax.php
Disallow: /announcement.php
Disallow: /arcade.php
Disallow: /attachment.php
Disallow: /autolink.php
Disallow: /calendar.php
Disallow: /credits.php
Disallow: /cron.php
Disallow: /editpost.php
Disallow: /external.php
Disallow: /faq.php
Disallow: /global.php
Disallow: /glossary.php
Disallow: /holdsession.php
Disallow: /image.php
Disallow: /infraction.php
Disallow: /inlinemod.php
Disallow: /joinrequests.php
Disallow: /login.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /mh_mv_membersvisited.php
Disallow: /misc.php
Disallow: /moderation.php
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /payment_gateway.php
Disallow: /payments.php
Disallow: /post_thanks.php
Disallow: /postage.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /showgroups.php
Disallow: /subscription.php
Disallow: /threadrate.php
Disallow: /threadprefix.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /vbpost_ajax.php
Disallow: /vbseo.php
Disallow: /vbseocp.php
Disallow: /vbseocpform.php
Disallow: /warn.php

Sitemap: http://www.shokmuzik.net/sitemap_index.xml.gz
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #2  
Old 10-25-2007, 08:06 PM
Oleg Ignatiuk's Avatar
vBSEO Staff
vBSEO Total Customer SupportvBSEO Documenter
 
Real Name: Oleg Ignatiuk
Join Date: Jun 2005
Location: Belarus
Posts: 21,597
Hello,

if you have your member profile URLs rewritten, you should add this in robots.txt:
Disallow: /members/
__________________
Oleg Ignatiuk / Crawlability Inc.
Support Team Launches New DeskPro Powered Tool Enhanced Support at Your Service

vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations

6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You.

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
  #3  
Old 10-25-2007, 08:09 PM
BamaStangGuy's Avatar
Senior Member
vBSEO Pre-Release TeamBig Board Administrator
 
Real Name: Brent Wilson
Join Date: Aug 2005
Location: Huntsville, Alabama
Posts: 2,347
Send a message via AIM to BamaStangGuy Send a message via MSN to BamaStangGuy
They visit your members.php because you rewrite your username urls.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Share on Facebook!
Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads

Thread Thread Starter Forum Replies Last Post
Google Now Reporting Anchor Text Phrases Eros SEO Buzz 2 03-22-2007 04:09 AM
Interview with Google's Matt Cutts about Next-Generation Search Eros SEO Buzz 2 03-16-2007 01:58 PM
Matt Cutts - SEO Videos on Google Video Joe Ward SEO Buzz 9 08-08-2006 08:05 PM
Google is All About Search: 4 New Products Joe Ward SEO Buzz 0 05-16-2006 02:52 PM


All times are GMT -4. The time now is 01:40 AM.


Powered by vBulletin Version 3.8.0 Beta 4
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.5 ©2008, Crawlability, Inc.