Server executes script on login and won't let me do anything else
I was trying to update my Let's Encrypt SSL certs using their certbot script. When I noticed that all mail is bouncing and nginx is down I tried logging on via SSH. The server acts as if it's executing certbot and asks for a sudo password. When I give it one, it acts as if it's going through the installation but doesn't spin mail or web servers back up. Not possible to quit out of the script without the server ending my session. Rebooting doesn't solve the problem.
I'm not an experienced Linux person so this is way over my head. I'd restore from an older backup (I don't care what mail I lose), but the oldest I have is still cheesed.
Any help would be greatly appreciated. I'm sure I'm missing something obvious to more experienced people.
Here's what a session looks like:
jason @ ~ > ssh jason@xxxxxx
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.18.16-x86_64-linode118
x86_64)
* Documentation: https://help.ubuntu.com/
Last login: Wed Jan 16 17:51:03 2019 from
xxxxxx.hsd1.il.comcast.net
Requesting to rerun /bin/bash with root privileges...
[sudo] password for jason:
6 Replies
Hi Jason,
That's a weird one. It almost sounds like your shell is set to run a script rather than … just be a shell. This would explain why killing the process immediately logs you out of the system. I don't know why a user would be setup in this manner, but you might want to check it out anyway --
You can login to your Linode as root via Lish:
https://www.linode.com/docs/networking/using-the-linode-shell-lish/
and then review the contents of your /etc/passwd file. I think you might find that the jason user has something other than /bin/bash at the end of their entry.
If your passwd file looks good, you'll probably be best off just adding another non-root user to your Linode and using that user going forward.
If you need to reset your root password, you can follow this guide:
https://www.linode.com/docs/quick-answers/linode-platform/reset-the-root-password-on-your-linode/
And if you need assistance adding a non-root user, please see our docs here:
https://www.linode.com/docs/tools-reference/linux-users-and-groups/
I hope this is helpful.
Warm regards,
Benjamin
Linode Support Team
Thanks for the reply, Benjamin.
I've tried logging in via the LISH interface and faced the same issue. I disabled root logins by SSH and only have one regular user set up.
I think you're right that somehow it's executing a script instead of invoking a shell, but it also looks like I'm locked out if I can't SSH in as root or any other user.
I don't know what to do if I can't SSH in as root.
Update. I can login as root using the LISH console. It does the same thing whether I log in as jason or root: invokes the script and then kicks me out.
Hi there,
Thanks for getting back to us. It sounds like you might need to boot into rescue mode, mount the disk, and investigate from there.
Booting into rescue mode:
https://www.linode.com/docs/troubleshooting/rescue-and-rebuild/#booting-into-rescue-mode
Mounting the disk:
https://www.linode.com/docs/troubleshooting/rescue-and-rebuild/#mounting-disks
You'll want to check the following files: ~/.bashrc, ~/.profile, ~/.bash_profile, /etc/passwd, /etc/profile and /etc/bashrc
I hope this is helpful.
Warm regards,
Benjamin
Linode Support Team
Those files look normal. /bin/bash was generated by certbot and is what's controlling the session.
root@ttyS0: grep -rnw '/media/sdb' -e 'has-root'
/media/sdb/bin/bash:193:if [ "$1" = "--cb-auto-has-root" ]; then
/media/sdb/bin/bash:199: $SUDO "$0" --cb-auto-has-root "$@"
/media/sdb/bin/bash:204:# Runs this script again with the given
arguments. --cb-
auto-has-root is added
/media/sdb/bin/bash:208: "$0" --cb-auto-has-root "$@"
/media/sdb/var/log/auth.log.1:19002:Jan 12 12:10:34 mx sudo:
jason : TTY=pts/
7 ; PWD=/home/jason ; USER=root ; COMMAND=/bin/bash --cb-auto-has-
root
This is getting really demoralizing.
Hi there,
Thanks for checking back. Don't get too demoralized yet, it looks like we found the issue here.
/bin/bash should not have been replaced with a script. Since binary compatibility between the rescue system and your Linode are not guaranteed or likely, I think the best course of action here is to change the shell for both users in /etc/passwd .
You can do this by editing your Linode's /etc/passwd from the rescue system so that the shell specified for each user is /bin/sh rather than /bin/bash. This will get you access to your system, but you're going to have a broken bash installation.
In order to fix bash, you'll need to do the following. As root or with sudo --
download the .deb package:
apt-get download bash
extract the package to /tmp:
dpkg-deb -x ./bash*deb /tmp
copy the new bash to /bin/:
cp /tmp/bin/bash /bin/bash
At this point you should have a working bash installation, and you can run chsh
to change your shell back to /bin/bash.
I'm very hopeful that this will work for you. Please keep us updated. Thanks!
Warm regards,
Benjamin
Linode Support Team