I really do not know why, but Yahoo’s search bots have been hammering my server for the last 24 hours, so much so, pointlessly, and taking no notice at all of the robot.txt that i have just added the who class c to iptable’s and banned the lot of them.
Why the hell Yahoo seem to ignore the robot text file when every other crawl bot seems to work ok, totally defeats me
Anyway, it was less hassle to just ban them, if it means getting f=dropped from the yahoo search system, then that’s no big deal, because most of the sites I have on the server get 95% of their hits from Google searched results