Filesystem corruption on cPanel/WHM CentOS7
Linode
Linode Staff
My Linode has been periodically crashing and rebooting with a filesytem read-only mode. I've following instructions to run a filesystem check according to the Rescue and Rebuild guide, but my Linode is still showing filesystem issues in its boot log:
[ 3.095292] systemd-fsck[584]: /dev/sda: Inodes that were part of a corrupted orphan linked list found.
[ 3.095760] systemd-fsck[584]: /dev/sda: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
[ 3.096020] systemd-fsck[584]: (i.e., without -a or -p options)
[FAILED] Failed to start File System Check on Root Device.
See 'systemctl status systemd-fsck-root.service' for details.
Starting Remount Root and Kernel File Systems...
[ 3.442180] EXT4-fs (sda): warning: mounting fs with errors, running e2fsck is recommended
...
[FAILED] Failed to start Create Volatile Files and Directories.
See 'systemctl status systemd-tmpfiles-setup.service' for details.
Any idea how to fix this issue?
1 Reply
bbigger
Linode Staff
First, you might want to try rebooting into Rescue Mode and running a filesystem check again, making sure you carefully following through the instructions in the Rescue and Rebuild guide.
Make sure you reboot your Linode from Cloud Manager (rather than from terminal command) after you've run the filesystem check. Feel free to post the output of e2fsck -f /dev/sda to confirm any issues.
You should also run the suggested command systemctl status systemd-tmpfiles-setup.service to investigate the issues with starting that service, as they could be related. Here's the man page for systemd-tempfiles in case it helps.
Apart from that, I found this thread regarding /tmp issues with cPanel that seemed relevant to what you're seeing: https://forums.cpanel.net/threads/problem-with-tmp-partition.76122/
From that thread, you may want to try following these instructions:
Check open /tmp files:
lsof /tmp
Then umount /tmp and /var/tmp:
umount -l /tmp
umount -l /var/tmp
Then remove the corrupt partition file:
rm -fv /usr/tmpDSK
Then create a nice new one:
/scripts/securetmp
Ultimately, we're looking to make sure that you can boot your Linode cleanly without indications of filesystem errors — if they recur later, then we can focus on indications of issues preceding such recurrence to narrow down the cause.