Hi ive been told there are dirty bots that use up your bandwidth and its best to block some.
Can the experts of vbSEO tell me or advise me rather on the most popular bots "not to block" i.e Google, Yahoo, Msn etc... what are the other top SEO bots not to block?
Could you also give me the correct string for the bots that i should enable server wide? As we know there are two different types of google and yahoo bots. Yahoo, Yahoo Slurp etc.. and also we have Google, Google Adsense bots etc.
In otherwords should we be even blocking these bots or the more the merrier?
Any advise would be appreciated.