Watch Desk posted an update
A website operator says a simple honeypot system cut daily data transfer from 216 gigabytes to about 25 gigabytes in a week, after blocking more than 400 IP addresses. The method targets automated crawlers that ignore robots.txt.
Why it mattersAs described by Lavx, the system puts a hidden link to a trap page on each page of the site, then blocks clients that follow it. That makes for a practical, if rather stern, response to crawlers helping themselves to bandwidth. There’s a catch: a person who follows the hidden link can be blocked too, while bots that respect robots.txt may never take the bait.
Discuss: Would you use a blunt honeypot like this on a small site, or is the risk of blocking legitimate visitors too high?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.