How to stop Google spiders and others from going to members.php?

Hello,

I've just noticed that google and other SE bots visits members pages as much as my view thread which i guess is bad because they aren't focusing on my content? if so how can i stop them from doing so?

should i just let them view "view forum" and "view thread"? and how would i do that?

Thanks

You can add any page to robots.txt

Tutorial how to make a good robots.txt file - vBulletin SEO Forums

Originally Posted by Brian Cummiskey

You can add any page to robots.txt

Tutorial how to make a good robots.txt file - vBulletin SEO Forums

Thanks, i didn't notice about the root path before, just to make sure, if let say my archive folder is in forums root so i should write:

Disallow : forums/archive ?

Thanks

Disallow: /forums/archive/

is the correct syntax

Thanks for sharing. It is very helpful!!

If you placed these member profiles in sitemaps, make sure remove them. Also if you change access setting from "user groups" generally it's 1, the member profiles. Google bot will never index it. It saves bantwidth and blocks the potential profil spam.