In his January blog post, Alex Forster articulated that we have a plan in motion to upgrade the network across all of our data centers. We would like to share with everyone what has been done so far, and what still needs doing.
Staffing up
Since December 2015, we’ve added two new members to our Network Operations department with a total of 24 years of experience between them. Our new Network Operations Manager, Dan Spataro, brings with him a wealth of experience in the maintenance of backbone and data center networks. Owen Conway has also come over from LINX, one of the largest peering exchanges in the world. They have been working tirelessly along with our existing Network Operations crew to make substantial improvements to our network infrastructure, and have proven to be invaluable additions to our team.
Infrastructure Improvements
Status: In progress
Expected Completion: Mid May
Phase one of our plan is to install new higher capacity networking hardware in every one of our locations. These new devices will greatly increase our capacity to the internet and allow us to multi-home to many internet peers at once. Our Network Operations team has been rolling out this new hardware as quickly as possible and is currently working on many simultaneous turn-ups with our providers. By mid-May we hope to have these installations complete.
Dark Fiber Turn-ups
Status: In progress
Expected Completion: July
Due to infrastructure variations between geographic locations, we have had to get creative in our mission to turn up large amounts of additional bandwidth capacity at our various data centers in order to reach the tier 1 providers in each region. Our first plan involved the procurement of 200 Gbps of lit services from our data center locations to the well connected carrier hotels. The provider estimated a late June install date for the first location. That timeline was unacceptable to us, so we decided to lease our own dark fiber and light it ourselves. The time to market is much quicker and the potential capacity is much greater. Instead of the fixed 200 Gbps of capacity per DC we can turn up many terabits of capacity per DC on demand. We expect to begin rolling out the optical gear in the next few weeks and hope to have the first locations up in mid May.
Internet Capacity
Status: In progress
Expected Completion: July
Internet capacity is a large part of our DDoS mitigation strategy – the more capacity, the better. We are currently working on turning up terabits of capacity to the Internet and have started to achieve this by peering with hundreds of Internet peers over many high capacity links. We have also reached agreements with three more tier 1 providers to provide us more connectivity to the Internet across all of our data center locations. We will begin installation of the first batch of connectivity from those agreements next week in Frankfurt.
Our entire Network Operations team has been working really hard to bring everyone these upgrades as quickly as possible. Your patience and understanding over the last few months have been greatly appreciated, and we look forward to sharing the benefits of these upgrades with everyone soon.
Comments (12)
I’m very pleased to see some progress on this. It’s been a rough several months, I know, and I’m happy to see follow-through at long last. Can’t wait to see what it’s like after this is all rolled out in full!
When will you be detailing the recent changes to the DNS service? Anybody with half a networking brain (which is presumably all of your customers) can see what you did, but we’re waiting for you to tell us the how and why of it.
Most of the DNS-related changes have been gone over here:
https://status.linode.com/incidents/smh59r6tyw7j
Not sure there’s going to be a full blog post dedicated to the CloudFlare Nameserver switch.
I realise you can’t do everything at once but do you have any plans regarding block storage in the pipeline? I’ve been at my instance’s storage limit for some time and given the expansion options at Linode it’s very hard to ignore the new block options from your competitors.
No doubt Linode’s infrastructure is always the best in VPS, but from software service aspect, it ‘s kind of outdated: the dashboard looks like Web 1.0, docker support – docker machine drive, elastic IP, etc. Please put more effort on your software service. Now, software industry is started transfer into container based microservice, even serverless, the infrastructure layer should be expose less and less to the developers.
I disagree with Docker support. It’s too early to bet on it IMO. I like VPS because it’s a bare-metal solution.
Elastic IP would be nice in theory, but not in the way AWS does it, please. One of my apps needs to bind multiple IPs to a socket, which is simply impossible with AWS. I hated it when I had to learn their proprietary way of doing on everything.
If you want an encapsulated infrastructure layer, go with PaaS like Heroku.
If Web 1.0 dashboard is working fine and secure, I see no reason to change that; since anytime you could write your favorite fancy Web-UI with Linode API.
Time-to-time DDoS packet-drop and Tokyo-sold-out(KVM announced for ~1 year) shall be resolved ASAP.
Nice! 🙂 I can’t believe I missed this announcement. Credit where due.
Glad you’re beefing up your network capacity. But this doesn’t change a thing for us: customers of yours that get DDOSed will still be booted off your platform. That is pretty harsh. Yes, there are services such as Cloudflare which protects websites cheaply but what about FTP, SSH + other random ports applications use? You really should offer some kind of paid DDOS protection service with all that bandwidth you’re buying, which can be turned on with a flick of the switch. We’re running our external firewalls with a DDOS protected provider then routing traffic to your DC and considering just making a switch to have everything under one roof.
There isn’t anything in the line at the time, but it’s certainly something that is always considered! Linode has always been trying to provide more for the same price – cause they love you. Promise.
Touching up the look and infrastructure is something always considered as well, and our engineers have been working on this, actually! You can find their progress on it (as well as a github to the project) here:
https://engineering.linode.com/2016/06/03/Building-a-comfy-frontend-stack.html
“The provider estimated a late June install date…”.
Should’ve gone with “the provider”. It’s almost September and you’re still single-homed in Atlanta.
Which data centers are complete on this now? Is there a schedule that we can see?