vBulletin Search Engine Optimization
This is a discussion on Google ignores my robots.txt within the General Discussion forums, part of the vBulletin SEO Discussion category; Hello, it is now a month ago i uploadet this robots.txt, but google still visits the member.php and calendar.php. Is ...
| |||||||
Enhancing 80 million pages. | Register | FAQ | Members List | Social Groups | Calendar | Search | Today's Posts | Mark Forums Read |
|
#1
| |||
| |||
|
Hello, it is now a month ago i uploadet this robots.txt, but google still visits the member.php and calendar.php. Is my robots.txt correct? Does it make a difference if I save the robots.txt with wordpad instead of editor? Code: User-agent: * Disallow: /admincp/ Disallow: /arcade/ Disallow: /archive/ Disallow: /geek/ Disallow: /clientscript/ Disallow: /cpstyles/ Disallow: /customavatars/ Disallow: /images/ Disallow: /includes/ Disallow: /install/ Disallow: /modcp/ Disallow: /sitemap/ Disallow: /web967h/ Disallow: /web97f1/ Disallow: /web98qJ/ Disallow: /web99l2/ Disallow: /template_cache/ User-agent: * Disallow: /ajax.php Disallow: /announcement.php Disallow: /arcade.php Disallow: /attachment.php Disallow: /autolink.php Disallow: /calendar.php Disallow: /credits.php Disallow: /cron.php Disallow: /editpost.php Disallow: /external.php Disallow: /faq.php Disallow: /global.php Disallow: /glossary.php Disallow: /holdsession.php Disallow: /image.php Disallow: /infraction.php Disallow: /inlinemod.php Disallow: /joinrequests.php Disallow: /login.php Disallow: /member.php Disallow: /memberlist.php Disallow: /mh_mv_membersvisited.php Disallow: /misc.php Disallow: /moderation.php Disallow: /moderator.php Disallow: /newattachment.php Disallow: /newreply.php Disallow: /newthread.php Disallow: /online.php Disallow: /payment_gateway.php Disallow: /payments.php Disallow: /post_thanks.php Disallow: /postage.php Disallow: /postings.php Disallow: /printthread.php Disallow: /private.php Disallow: /profile.php Disallow: /register.php Disallow: /report.php Disallow: /reputation.php Disallow: /search.php Disallow: /sendmessage.php Disallow: /showgroups.php Disallow: /subscription.php Disallow: /threadrate.php Disallow: /threadprefix.php Disallow: /usercp.php Disallow: /usernote.php Disallow: /vbpost_ajax.php Disallow: /vbseo.php Disallow: /vbseocp.php Disallow: /vbseocpform.php Disallow: /warn.php Sitemap: http://www.shokmuzik.net/sitemap_index.xml.gz |
|
#2
| ||||
| ||||
|
Hello, if you have your member profile URLs rewritten, you should add this in robots.txt: Disallow: /members/
__________________ Oleg Ignatiuk / Crawlability Inc. Support Team Launches New DeskPro Powered Tool Enhanced Support at Your Service vBSEO 3.2.0 Launched - Maximum Overdrive for Your Web Traffic! Over 100 Instant SEO Optimizations 6X Traffic - $1400 in One Day with vBSEO! Imagine What the vBSEO Patent Pending Technology Can Do For You. |
|
#3
| ||||
| ||||
|
They visit your members.php because you rewrite your username urls.
|
| Thread Tools | |
| |
Similar Threads | ||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Google Now Reporting Anchor Text Phrases | Eros | SEO Buzz | 2 | 03-22-2007 04:09 AM |
| Interview with Google's Matt Cutts about Next-Generation Search | Eros | SEO Buzz | 2 | 03-16-2007 01:58 PM |
| Matt Cutts - SEO Videos on Google Video | Joe Ward | SEO Buzz | 9 | 08-08-2006 08:05 PM |
| Google is All About Search: 4 New Products | Joe Ward | SEO Buzz | 0 | 05-16-2006 02:52 PM |