how investigate a past spike in activity?
I got 2 emails 2 days ago, about
- exceeded the notification threshold (23) for CPU Usage by averaging 147.6% for the last 2 hours
- exceeded the notification threshold (5) for outbound traffic rate by averaging 33.52 Mb/s for the last 2 hours
I just went in, and see a huge spike in those things, way unlike any day (e.g. my typical monthly CPU% spike is to 8%!).
- since it's over, should I care?
- if I should care, how do I dig in to something that isn't happening now? Browsing a log doesn't smell promising, is there some sort of analysis tool built in?
1 Reply
eruzanski
Linode Staff
Unexpected CPU utilization can arise from various factors, not all inherently negative. For instance, if you operate a web server, a sudden surge in traffic may result from a rapid influx of hits within a short timeframe, stemming from human visitors or web bots. Another potential cause for the spike could be recent software installations or background tasks executed on your behalf.
However, if you observe sustained outbound traffic exceeding 33.52Mb/s for over two hours, combined with the elevated CPU activity, and this activity cannot be attributed to any normal services running on your Linode, there is a distinct possibility that your Linode may have been compromised. Additionally, the inability to promptly identify the source of the higher-than-normal utilization, despite your services running at a typical 8% utilization, suggests a compromised state for your Linode.
For guidance on recovering from a system compromise, please refer to the following resource: Recovering from a System Compromise.
To proactively scan your system for vulnerabilities, consider using ClamAV with the instructions provided here: How to Scan Your System for Vulnerabilities with ClamAV.
Despite the incident having concluded, it is imperative to remain vigilant and take immediate action to cleanse and secure your system.