BB Code page getting crawled/indexed

I submitted my sitemap to bing and its putting my sites bbcode link as # 3, this mysite.com/misc.php?do=bbcode

How can I fix this?

You can simply disallow misc.php from your robots.txt

Originally Posted by marco1

You can simply disallow misc.php from your robots.txt

Thanks, but I already have that disallowed and its still doing it.

So that's will not crawl it again the robots.txt is search engines rules (allow and disallow)

Originally Posted by marco1

So that's will not crawl it again the robots.txt is search engines rules (allow and disallow)

I have that disallowed already and its doing it. I also have a lot of robots looking at the same /[img] all the time.

What's your site URL please so I can check your robots.txt file

www.site.com/robots.txt

That's seems you are disable disallow it

Code:

# Disallow: /misc.php

You can use this robots.txt

Code:

User-agent: *

Disallow: /calendar.php
Disallow: /login.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /newthread.php
Disallow: /newreply.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /member.php
Disallow: /archive/
Disallow: /misc.php

Sitemap: http://www.sitename.com/sitemap_index.xml.gz

That's the best robots.txt file

Thank you for your help.

You are welcome

Okay I added new robot file as you said. Can you remove the link please?

No problem removed your site link

I put my site map as you say, but its still showing a LOT of bots crawling images. How can I stop this?

Unknown Location
/ [ IMG ] www.photobucket.com/blahblahblah... [IMG ] code

Originally Posted by doubler0204

I put my site map as you say, but its still showing a LOT of bots crawling images. How can I stop this?

Unknown Location
/ [ IMG ] www.photobucket.com/blahblahblah... [IMG ] code

Can you please take snapshot of these bots

Okay first is a bot registering and not getting no permission:

And 2nd is the IMG getting viewed.

Its also viewing /favicon.