Linode.com Forum Forum Index Linode.com Forum
Linode Community Forums
 


Emergency Router Maintenance: Atlanta

Click here to go to the original topic
Goto page Previous  1, 2, 3, 4  Next
 
       Linode.com Forum Forum Index -> System and Network Status
Author Message
kiomava



Joined: 13 Dec 2003
Posts: 87

Posted: Fri Dec 07, 2007 9:32 pm    Post subject:  

the network status forum thread says things are better, but i'm not seeing an improvment.

Hey, what tool are you guys using that generates those nifty % packet lose stats in the text output?

Code:
  4    28 ms    28 ms    28 ms  200.ge-0-1-0.cr2.sea1.speakeasy.net [69.17.83.250]
  5    29 ms    29 ms    28 ms  ge3-0.cr02.sea01.pccwbtn.net [198.32.180.13]
  6   101 ms    90 ms    91 ms  gnax.ge2-13.br01.atl01.pccwbtn.net [63.216.31.130]
  7    90 ms   165 ms    89 ms  63.247.77.6
  8     *        *        *     Request timed out.
  9     *        *        *     Request timed out.
 10     *        *        *     Request timed out.
 11     *        *        *     Request timed out.
Back to top  
Funky



Joined: 07 Dec 2007
Posts: 4

Posted: Fri Dec 07, 2007 9:33 pm    Post subject:  

I am seeing a slight improvement. But it's still loosing packets.
Back to top  
caker



Joined: 15 Apr 2003
Posts: 2392
Location: Galloway, NJ

Posted: Fri Dec 07, 2007 9:39 pm    Post subject:  

kiomava wrote: Hey, what tool are you guys using that generates those nifty % packet lose stats in the text output?
mtr, with the --report option.

-Chris
Back to top  
vca



Joined: 25 Sep 2007
Posts: 17

Posted: Fri Dec 07, 2007 10:04 pm    Post subject:  

We're getting rather slow response from atlanta23.linode.com this evening (and this morning around 0700MST it was also slow).
Back to top  
kiomava



Joined: 13 Dec 2003
Posts: 87

Posted: Fri Dec 07, 2007 10:10 pm    Post subject:  

I don't know if this helps, but I it seems the routing between my test point and atlanta22 is different than to my linode on atlanta22:

I'm realiably getting 100% loss to my linode on atlanta22:

Code:
 10. pr1-so-0-3-0.atlantapaix.sav  8.3%   120   27.1  31.0  22.6 254.8  31.8
 11. bpr1-so-1-2-0.AtlantaPaix.sa  0.0%   120   22.7  22.9  21.8  50.8   3.5
 12. 216.89.82.82                  0.0%   120   22.1  26.9  21.7 202.3  25.5
 13. 216.89.82.82                  2.5%   120   22.2  26.8  22.0 130.6  17.3
 14. 63.247.77.6                  53.3%   120   23.0  23.8  23.0  45.9   3.1
 15. ???                          100.0   120    0.0   0.0   0.0   0.0   0.0


but not to atlanta22 host linode

Code:
  8. sl-bb22-fw-12-0.sprintlink.n  0.0%   120    1.6  11.5   1.5 203.2  33.9
  9. sl-bb25-atl-6-0.sprintlink.n  0.8%   120    1.8  23.2   1.6 255.4  44.5
 10. sl-st21-atl-0-0-0.sprintlink  0.0%   120   20.0  21.5  18.0 157.9  18.0
 11. 207.13.126.74                 0.0%   120   20.7  24.4  18.1 189.6  25.5
 12. l3-atl-18.gnax.net            0.0%   120   20.1  25.8  19.9 213.7  26.1
 13. 63.247.71.217                 0.0%   120   20.6  20.8  20.0  41.1   2.3


I've repeated this several times, same results. I have several linodes on atlanta22, same results -- can't reach the linode IP but can reach the host IP.
Back to top  
caker



Joined: 15 Apr 2003
Posts: 2392
Location: Galloway, NJ

Posted: Fri Dec 07, 2007 10:27 pm    Post subject:  

kiomava wrote: I'm realiably getting 100% loss to my linode on atlanta22:
What is your Linode's IP?

-Chris
Back to top  
kiomava



Joined: 13 Dec 2003
Posts: 87

Posted: Fri Dec 07, 2007 10:32 pm    Post subject:  

my linode IPs on atlanta22:

64.22.124.38 - 64.22.124.47

(10 linodes on atlanta22, sequentally numbered IPs cuz I ordered em within seconds of each otehr)
Back to top  
caker



Joined: 15 Apr 2003
Posts: 2392
Location: Galloway, NJ

Posted: Fri Dec 07, 2007 10:45 pm    Post subject:  

kiomava wrote: my linode IPs on atlanta22: 64.22.124.38 - 64.22.124.47
Are you guys doing some funky firewalling? Because I can get to .37 just fine, which is someone else's Linode but would be the same route, and I'm seeing my packets hit your Linode's interface.

-Chris
Back to top  
vca



Joined: 25 Sep 2007
Posts: 17

Posted: Fri Dec 07, 2007 11:49 pm    Post subject:  

I'm on 64.22.124.245, the dash board shows the host load as low, but http requests are being served dead slow, pings and tracert from my end (137.186.192.1) are timing out.
Back to top  
vca



Joined: 25 Sep 2007
Posts: 17

Posted: Fri Dec 07, 2007 11:51 pm    Post subject:  

and I don't think I've got any complex firewall settings, just a firehol configuration to allow SSH, HTTP and HTTPS through which has not been modified in a few weeks.
Back to top  
kiomava



Joined: 13 Dec 2003
Posts: 87

Posted: Sat Dec 08, 2007 12:29 am    Post subject:  

I do have some weird firewall rules, but they've been up for months. But that explains the dropped packets at the host. But I'm still experiencing the strangeness reflected by seemingly dropped packets at the upstream router, that .6 router.
Back to top  
mbbx6spp



Joined: 08 Dec 2007
Posts: 2

Posted: Sat Dec 08, 2007 12:41 am    Post subject: Issues on atlanta21.linode.com  

I am having major network woes on atlanta21.linode.com.
Back to top  
astrashe3



Joined: 05 May 2007
Posts: 29

Posted: Sat Dec 08, 2007 1:48 am    Post subject:  

I'm seeing flaky connectivity from host77 @ atlanta at 12:44a central time on dec 8.

My linode isn't mission critical, and I don't want to be one of those whiny people, but it's a little discouraging that we don't seem to be getting close to a diagnosis.

It seems to me that this isn't a linode problem, but rather a data center problem. So maybe the linode guys could start yelling louder at the data center guys?

Thanks...
Back to top  
kiomava



Joined: 13 Dec 2003
Posts: 87

Posted: Sat Dec 08, 2007 2:18 am    Post subject:  

Yeah what I worry about is that the ATL data center people think it's fixed when it's really not fixed. It's good that we keep an eye on this I think, especially since it's hard to diagnose this particular routing problem.
Back to top  
OverlordQ



Joined: 04 Jun 2004
Posts: 200

Posted: Sat Dec 08, 2007 3:58 am    Post subject:  

Can't be the DC, I mean I can ping atlanta21.linode.com from my Freemont linode, and from my linode in Dallas.

Code: thedarkcitadel.com (0.0.0.0)                                                                Sat Dec  8 02:52:54 2007
                                                                            Packets               Pings
 Host                                                                     Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. gateway-he-li3.linode.com                                              0.0%    90    3.9   3.8   0.5  24.6   5.0
 2. 10gigabitethernet1-2.core1.sjc2.he.net                                 0.0%    89    8.2   2.4   0.7  58.9   6.6
 3. ge13-1.br02.sjo01.pccwbtn.net                                          0.0%    89    1.1   7.8   1.1 161.4  27.3
 4. gnax.ge2-13.br01.atl01.pccwbtn.net                                     0.0%    89   76.3  85.0  76.2 308.6  37.9
 5. atl-core-a-g3-1.gnax.net                                               0.0%    89   76.5  86.1  76.2 294.6  40.6
 6. l3-atl-18.gnax.net                                                     0.0%    89   76.6  77.6  76.5  98.4   3.7
 7. 63.247.71.216                                                          0.0%    89   76.3  78.5  76.2 227.3  16.0


Code: cuppa.remnetworks.org (0.0.0.0)                                           Sat Dec  8 02:54:25 2007
                                                          Packets               Pings
 Host                                                   Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. up3.linode.com                                       0.0%    33   63.1   3.4   0.4  63.1  11.0
 2. vl2.dsr02.dllstx2.theplanet.com                      0.0%    33    1.0   9.4   0.4 173.7  32.8
 3. 7d.fd.5746.static.theplanet.com                      0.0%    33    2.2   0.9   0.5   2.5   0.5
 4. 5.7f.5546.static.theplanet.com                       0.0%    33    0.9   0.8   0.5   3.5   0.6
 5. ge5-3.br02.dal01.pccwbtn.net                         0.0%    33   38.2   8.0   0.6 145.2  26.8
 6. gnax.ge2-13.br01.atl01.pccwbtn.net                   0.0%    33   21.1  21.5  20.9  24.0   0.7
 7. atl-core-a-g3-1.gnax.net                             0.0%    33   21.6  24.6  20.8 118.8  17.0
 8. l3-atl-18.gnax.net                                   0.0%    33   23.4  22.1  20.7  40.4   3.4
 9. 63.247.71.216                                        0.0%    32   20.7  21.2  20.2  24.0   0.8


But using your IP,

Code:                                                                             Packets               Pings
 Host                                                                     Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. gateway-he-li3.linode.com                                              0.0%    10    3.2   2.2   0.5   6.2   2.0
 2. 10gigabitethernet1-2.core1.sjc2.he.net                                 0.0%    10    0.8   3.9   0.8  11.1   4.2
 3. ge13-1.br02.sjo01.pccwbtn.net                                          0.0%    10    1.4   1.3   1.1   1.5   0.1
 4. gnax.ge2-13.br01.atl01.pccwbtn.net                                     0.0%    10   75.1  75.5  74.7  81.3   2.0
 5. atl-core-a-g3-1.gnax.net                                               0.0%     9   74.8  74.9  74.8  75.1   0.1
 6. l3-atl-18.gnax.net                                                     0.0%     9   75.3  75.3  75.0  76.0   0.4
 7. ???


The only difference is the last hop which is your linode.

As well though the IPs are in different blocks, so maybe when they swapped out maybe the routing tables got FUBARd.

*shrug*
Back to top  
 
       Linode.com Forum Forum Index -> System and Network Status Goto page Previous  1, 2, 3, 4  Next
Page 3 of 4