No more spiders visiting my forum!

[fpattberg]

Hello.

I have had the situation that suddenly (since yesterday) somehow there are now spiders visiting my site and I have no idea why.

My site: http://www.sustainabilityforum.com/forum

Has anyone an idea why that could be the case? Have I missed something?

I am using 3.6.7 PL1 but that should not be a problem right?

Please help.

Thanks.

Fabian

[Oleg Ignatiuk]

You have disallowed them in your robots.txt: http://www.sustainabilityforum.com/robots.txt
You should remove those lines from robots file.

[fpattberg]

Okay.

Stupid question but how do I do that?

Fabian

[Shadab]

If you don't want to put some other rules, in that robots.txt file;
then simply delete it (/robots.txt) from your forum root using a FTP program.

[fpattberg]

Okay.

But I cant find a robots.txt on my root anywhere.

Can I change it somewhere else?

Fabian

[fpattberg]

Is there a way that someone can put a robots.txt on via ftp without me knowing?

[Oleg Ignatiuk]

Don't you have "robots.txt" folder there?

[fpattberg]

Hi.

No. There is no file or folder robots.txt on my root or anywhere else. I have look through every folder and there is nothing!

Have I been hacked or so?

Fabian

[Joe Ward]

Hi.

No. There is no file or folder robots.txt on my root or anywhere else. I have look through every folder and there is nothing!

Have I been hacked or so?

Fabian

You should check this directly with your web hosting provider. We have heard instances of some hosts actually blocking search crawling activity.