Can robots read if forum is blocked?

[majordude]

What I mean is, if the basic level of our site is non-registered users can read forum headlines but not read individual posts, etc... can a search engine? They aren't "registered"... unless there is an IP or "bot" pass I don't know about.

[Ace Shattock]

Hi majordude,

Quick answer: "No, they can't."

Bots are treated as Guests/Unregistered with regards to Forum Permissions by default vB.

Even if you hacked it to allow specific UserAgents to read posts, your more savvy Guests will be able to by spoofing those UserAgents in their browser.

They will also be able to read the cached versions on the SEs, defeating the Privacy purpose.

[majordude]

Sigh.

Thought so.

I wonder if there is a way to create a "user" that can see everything you want them too but can't post in case it is discovered by hackers. All "bots" would be given this user level.

Sort of a reverse cloak.

[eJM]

That's against the rules and you could get banned by Google for doing it.

[Noodles]

Well yes and no. Google doesn't like you serving up different content to spiders and users, it prefers you treat everyone the same, but some sites like the nytimes have google spider pages and then users have to login to get to those pages (the pages aren't cached by google).

I think as long as you're not doing it as an SEO trick, i.e. making the page that spiders get like "viagra xenical weight loss pills etc" and the one the users get something else, you should be ok.