Max bandwidth options
First of all how does this normally work? Will linode do any sort of limiting? Also, what are some of the solutions that others have put in place to safeguard themselves from bankrupcy?
im not sure if iptables has the ability to generate emails, but u could write a cron job to check the iptables count, and if it is getting high report it
It hadn't, but my usage was unusual, and they noticed.
I don't want to tell you that you could count on linode letting you know, because I don't know if you can. But they did notice when my traffic spiked up, and they did ask me about it.
Not all crawlers honors robots.txt since it's voluntary, so keep that in mind. Still, it's quick and easy to put up and doesn't hurt anything.