Why are my transfers slow only from Dallas data center, and only from some ISPs?

I have several Linode VPS, some in the Dallas data center and some in the Atlanta data center. I have customers that are transferring files from these servers in many different locations, and found that certain connections are always slow, and others are always fast.

When I test file transfer via FTP or HTTP from two different locations myself, I get consistently slow downloads only from one download location, and only from the servers in Dallas.

I ran mtr from the slow connection, and not seeing any issues like dropped packets or slow times, but TCP connections seem impacted 100% of the time.

In a table format to try and be clear:

Endpoint   Server   Speed
ISP1       Dallas   Slow
ISP2       Dallas   Fast
ISP1       Atlanta  Fast
ISP2       Atlanta  Fast

This was tested on three different Linode servers in each data center, and the test was run with FTP and HTTP download, and the result is consistent. The downloads are just very slow - the transfers work, but take 10x-20x longer than usual.

So rebooting servers or restarting processes would not have any impact, especially since the speed is always fast from some locations even to Dallas.

Another important note is that using a VPN over ISP1 can resolve the issue.

All of this suggests that there is something in the route from ISP1 to Dallas dropping packets and causing slowness that doesn't impact the other connections.

I've got many customers transferring large files, and suggesting to all of them to use a VPN as a workaround is not viable.

Any ideas or suggestions on what I could do here?

5 Replies

I noticed the exact same issue on a VM located in Dallas - extremely slow transfer speeds to multiple servers located on completely different networks.

Like you, MTR shows nothing out of the ordinary but when I packet traced one of the transfers it seems that the congestion window never grows much beyond the initial starting value. Something definitely strange going on in Dallas.

I'm seeing what sounds like the same issue. I have logged an issue with Linode about it, but we're having trouble pinpointing the issue. Can you provide an MTR in both directions so that we can see whether there are any common paths that are having an issue?

mtr -rwbzc 100 $REMOTE_IP

https://www.linode.com/docs/guides/diagnosing-network-issues-with-mtr/

@jd123 I continued doing mtr probing and eventually narrowed down what I believe to be the culprit and opened a ticket with relevant details yesterday. Please see ticket 16777296.

My (naive) view of things looks like everything is jamming up in the dal01/d1 exchange. Reverse paths avoided equinix and were clean so I didn't bother including them. I can still update the ticket with them if necessary.

Looks to me like Linode has changed routes to avoid equinix/dal01 so my outbound transfer speeds appear to be back to normal.

Can confirm after Linode changed routes, I'm no longer seeing any transfer slowness out of the Dallas data center.

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct