SSH timeout

So I host a two websites on my server and I also run tmux with weechat-curses inside. Normally (well up until about 8PM EDT), I would SSH into my linode, then issue 'tmux attach -d' and open up my tmux session and watch IRC while doing other things. However, now when I do this after roughly 3 minutes (even if I am active typing during the 3 minutes) I get disconnected from the Linode and issued this error:

"Timeout, server not responding."

When I SSH back in everything is as it was right before I got disconnect, tmux is still running and weechat is still connected to the IRC network.

So here is what I have tried: I change my ~/.ssh/config to have the TCPKeepAlive and ServerAliveInterval lines. This did nothing.

Host idle

TCPKeepAlive yes

ServerAliveInterval 20

Hostname 173.255.236.80

User yano

I even tried adding "ServerAliveInterval 60" to /etc/ssh/ssh_config and then did "sudo /etc/init.d/sshd restart" and reconnected.

I even stopped MySQL (which I don't even use and it was only running from an incomplete mail server setup), Apache, and NTP. I was thinking maybe there was some high traffic coming on my server.

I've also tried connecting from a different ISP.

I even reverted my SSH config file back to the original and restart SSHD, no luck.

I contacted support but they didn't mention anything about possible network problems on there end, so I'm left to assume this problem is either with my VPS SSH configuration on computer I'm connecting from, or the ISPs I'm testing from.

Any ideas?