I've checked my logs and see regular attempts to SSH into my linode around the same time that the linode becomes unresponsive. I've installed fail2ban, which is doing its job, but my understanding is that these attempts wouldn't be enough to cause a bottleneck.
My graphs are good - CPU average of ~4%, and I/O rate average 6.55.
Is there something else I should be looking for here? Or another way I can diagnose this issue?
(Check the linode IO graphs, there should be a separate line for swap IO)