Backups cause disk I/O warning email
> …exceeded the notification threshold (1300) for disk io rate by averaging 1329.47 for the last 2 hours
This is what triggered it:
Pretty obvious it's the daily backup:
Is the right thing to do to raise the warning threshold? The default is 1000 so I think I already raised it to 1300. I would like to know about excessive IO, but I'd like backups excluded. If I get an email every day, I would have to either signore it and assume it's a backup (bad!) or check it every day (annoying!).~~~~
I have some cron scripts that run every night, but I just ran them manually and they don't cause a spike. I was just assuming it was the Linode backup service, since it happens every day in the backup time window. Are you sure the Linode backup service won't cause an IO spike?
Edit: actually it seems the cron jobs do cause a spike, just took a while for the chart to update. Seems the mystery is solved. Maybe I shouldn't optimize a multi million row table every night.
In any case, the default notifications are set pretty low and in general I agree that raising the threshold is probably the way to go.
P.S. To clarify, Linode's backup service doesn't create disk IO that's metered in the manager. I'm not trying to say that it pulls the data magically, but it shouldn't affect your performance or be visible in your graphs.