Dan Kegel wrote:
canned query links like those on http://kegel.com/openoffice without regard for the robot exclusion protocol.
- Dan
Personally, I think setting up robot traps is better than blocking everyone. You know the drill - a 1x1 picture link with a "on click" that prevents it from being ever clicked. Anyone accessing that link is marked as a robot, and no queries from that IP are honored for half an hour. All dynamic pages return a caned response "Sorry, no robots allowed past this point".
Shachar