The Xen security team recently made public three security advisories regarding the Xen Hypervisor. Linode customers are not affected by the issues outlined in the advisories due to proactive maintenance performed by Linode over the past few weeks.
- XSA-7 – 64-bit PV guest privilege escalation vulnerability
- XSA-8 – guest denial of service on syscall/sysenter exception generation
- XSA-9 – PV guest host Denial of Service (AMD erratum #121)
The Xen blog has a really nice writeup on the issue.
Having to deal with advisories like these is just part of our industry. One of our challenges in just about everything is our scale. Suddenly a required update means wrangling thousands of machines and causing a huge disruption for our customers.
These specific advisories had the potential to affect our entire fleet, however we were able to devise a clever plan which put the number of affected Linodes into the minority. The plan combined: 1) A rush to deploy additional capacity reserves across all facilities 2) a reboot/upgrade of only the hosts that would recover the most capacity, and 3) an automated migration queue of only the remaining affected Linodes onto the good capacity. As a result, the majority of customers were unaffected by this maintenance.
Almost everyone in the entire company had a hand in this effort – kudos to the entire team for making this as seamless and streamlined as possible.