is it good idea to stop crawling some forums?

[Seleno]

Hi There
i have some fun forums that members playing on it,in my google analytics i notice that this forum didn't bring any traffic by google
and my members used to bring these threads from another forums
any way

do you think its good idea to add this forum to robots.txt so google will stop crawling it! and crawling the better threads!?
and how can i add this form to stop googlebot from crawling it?
vb/f112 is right?

waiting your reply
seleno

[Seleno]

Joe ward why u didn't answer my thread?
u answered 2 but mine no...why?

[Joe Ward]

What's the content of forum? Is it on target with your forum theme?

If it's off-topic, then you can consider blocking it.

[Seleno]

the content is not good,just members playing like: write number 4 and 5 and we see who when!
in my google analytics i didn't notice much people found this forum by google, they found only the main forum like vb/f15 but not the threads!
so if google crawl the oither forums it will be better for my forum!
but how can i block it?
can i block it by googlebot?....or close it from visitors?

[Joe Ward]

Just block that particular forum using robots.txt.

[Seleno]

thanx for your help
this is my robots::::
User-agent: *
Disallow: /register.php
Disallow: /faq.php
Disallow: /login.php
Disallow: /payments.php
Disallow: /sendmessage.php
Disallow: /moderator.php
Disallow: /member.php
Disallow: /cgi-bin/
Disallow: /tmp/

lebnights.net/vb/f38/

so my robots will be like this?
User-agent: *
Disallow: /register.php
Disallow: /faq.php
Disallow: /login.php
Disallow: /payments.php
Disallow: /sendmessage.php
Disallow: /moderator.php
Disallow: /member.php
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /vb/f38/

i have sub forums on it also,do you this this will effect the sub forums and the pagerank?

[Joe Ward]

That should work. It will only effect content found below /vb/f38/ folder.

[Zagis]

What's the content of forum? Is it on target with your forum theme?

If it's off-topic, then you can consider blocking it.

Is the Member's Introduction Forum on target with the forum theme, If you have for example a car related forum? Should that forum be disallowed in robots.txt?

[Joe Ward]

It depends. It is actually possible for you to get traffic to this content. It's kind of the same idea of member profiles.

However, I tend to think of such pages (including member profiles) as low value content. I'd (personally) rather boost crawling efficiency and keep the search engines "view" of the site as being more strongly related to the niche, and better overall content.

By keeping out low value content, the proportionate value of high quality content to low quality content on your site grows.

[Zagis]

Thats great, thanks.

[Seleno]

thanx joe ward,but this can effect the subforums and pagerank also?

[Joe Ward]

thanx joe ward,but this can effect the subforums and pagerank also?

By eliminating pages, PageRank or link value should be more concentrated on your higher value pages.

Sub-forums won't be affected by the robots.txt line. You need to add a separate line for each sub-forum to be blocked as well.

[Seleno]

i did it,thanx Joe ward, but GoogleBot still crawling faq and members,while i blocked them in robots since long time!

[Zagis]

Wait a couple of days until your robots.txt file to be downloaded by Google.

[Seleno]

dunno why googlebot still crawling faq.php and members.php ,while i added them to robots since long time!