Banning Cuil Bot also known as Twiceler bot via robots txt and htaccess
by
, 05-11-2010 at 06:33 AM (3480 Views)
Cuil Bot also known as Twiceler bot, can bleed your server resources.
The likes of google has a simple user interface to slow it down via google webmasters, Cuil Bot also known as Twiceler bot, you dont have that pleasure.
This one single search engine is a menace on eating resources and bringing servers to their knees
Having found my DDS at almost a standstill, with dire page loads times, I had to investigate and found Cuil Bot also known as Twiceler bot, hitting my forums on average about five times a second, with fifteen of their bots all doing the same
Two simple ways to do it
Robots txt file. Add the following.
This can take seven days to take effect
Or, an all out ban on the ipīs it uses, via htaccess.PHP Code:User-agent: twiceler
Disallow: /
Instant and effective banning of the search bots
The list of ipīs used below, are the present list they use, taken from their website
Just add the following code to your htaccess file to ban all Cuil Bot also known as Twiceler bot
My forums The Spain Forums where i use the above code to great effectPHP Code:order allow,deny
deny from 67.218.116.130
deny from 216.129.119.10
deny from 216.129.119.40
deny from 67.218.116.131
deny from 216.129.119.11
deny from 216.129.119.41
deny from 67.218.116.132
deny from 216.129.119.12
deny from 216.129.119.42
deny from 67.218.116.133
deny from 216.129.119.13
deny from 216.129.119.43
deny from 67.218.116.134
deny from 216.129.119.14
deny from 216.129.119.44
deny from 216.129.119.15
deny from 216.129.119.45
deny from 67.218.116.162
deny from 216.129.119.16
deny from 216.129.119.46
deny from 67.218.116.163
deny from 216.129.119.17
deny from 216.129.119.47
deny from 67.218.116.164
deny from 216.129.119.18
deny from 216.129.119.48
deny from 67.218.116.165
deny from 216.129.119.19
deny from 216.129.119.49
deny from 67.218.116.166
deny from 216.129.119.50
deny from 216.129.119.51
deny from 216.129.119.81
allow from all









