I can't boot from my restored backup! Seeing 'UUID does not exist`

Linode Staff

I'm getting a message similar to this one:

mdadm: No arrays found in config file or automatically
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  UUID=<UUID> does not exist. Dropping to !

6 Replies

Judging by the error messages shown above, it's possible that your /etc/fstab or /boot/grub/grub.cfg files contain a disk UUID that doesn't exist (it's most likely the UUID of your old disk before this restore). You'll likely want to update those files so that they are able to use your current root disk.

The first step towards accomplishing this would be to boot into Rescue Mode. Once you've done so, you'll want to switch your working directory to /dev/sda disk so that you can make any necessary changes by running the following commands:

mount -o exec,barrier=0 /dev/sda
cd /media/sda
mount -t proc proc proc/
mount -t sysfs sys sys/
mount -o bind /dev dev/
mount -t devpts pts dev/pts/
chroot /media/sda /bin/bash
mdadm --detail --scan >> /etc/mdadm.conf
update-initramfs -u
update-grub

You'll then want to check your /etc/fstab file for any references to disk UUIDs:

less /etc/fstab

If this file does indeed reference UUIDs for your disks, you'll want to update them to your current disk UUIDs, which can be found by running the blkid command.

Once you've tackled everything above, you'll want to try issuing a normal boot job to your Linode to see if these changes resolved this issue. While Backups should certainly work straight out of the box, there are some cases where hardcoded UUIDs can cause issues when restoring Backups because your restored disks will be using different UUIDs than the disks that were captured in the Backup process.

One thing to note is that after running this sequence of commands. You need to exit the chroot and issue the reboot command from the terminal (not from the Linode web interface).

update-grub
exit
reboot

We recently made changes to the Finnix image we use for Rescue Mode. Because of that, you will likely see the following error message:

mount: /dev/sda: can't find in /etc/fstab.

You can resolve it by running:

mkdir /media/sda

Then you can proceed with the rest of the steps listed above.

Quick follow up. After running mkdir /media/sda

You'll want to use this command to mount the disk:

mount -o barrier=0 /dev/sda /media/sda

I have a number of devices listed in blkid:

/dev/sr0
/dev/loop0
/dev/sdb
/dev/sda
/dev/zram0

Should I be adding them all to /etc/fstab? Just /dev/sda and /deb/sdb? Or just /dev/sda?

In my restore I get my directories back but the system is in read-only mode and no services will start.

@adamatengaging writes:

I have a number of devices listed in blkid:
 
/dev/sr0
/dev/loop0
/dev/sdb
/dev/sda
/dev/zram0

From the man(1) page for fstab(5):

The  file  fstab contains descriptive information about the filesystems
the system can mount.  fstab is only read by programs, and not written;
it is the duty of the system administrator to properly create and main-
tain this file.  The order of records in  fstab  is  important  because
fsck(8), mount(8), and umount(8) sequentially iterate through fstab do-
ing their thing.

fstab(5) is only for file systems/dev/sr0 is a (virtual) SCSI controller. /dev/loop0 is a special block device used for creating/mounting ISO or CD disc images. See:

https://stackoverflow.com/questions/10379897/how-to-use-dev-loop0-to-create-and-mount-image

/dev/zram0 is a special compressed, RAM-based block device typically used to speed up swapping:

https://www.kernel.org/doc/html/latest/admin-guide/blockdev/zram.html

That leaves /dev/sda and /dev/sdb

Should I be adding them all to /etc/fstab? Just /dev/sda and /dev/sdb? Or just /dev/sda?

You should be adding the block devices that you want automatically-mounted to a filesystem at boot time in fstab(5).

In my restore I get my directories back but the system is in read-only mode and no services will start.

This is because, since you're devices are not present in fstab(5), the system is mounting them as read only to protect them. For a description of the contents/format of fstab(5), see here:

https://linux.die.net/man/5/fstab

One of the parameters in the 4th field of an fstab(5) entry tells the system not to do this (I forget which one at the moment). FWIW, here's the fstab(5) entries for my two discs on my Linode:

# <device> <mount point>   <type>  <options>       <dump>  <pass>
/dev/sda        /           ext4    errors=remount-ro 0       1
/dev/sdb        none        swap    defaults          0       0

This should be pretty self-explanatory. Note that /dev/sdb is a swap device…YMMV… sudo /sbin/blkid should tell you the UUID of each device as well so the above example can become:

# <device>                               <mount> <type>  <options>       <dump>  <pass>
UUID=64218522-a39b-496a-a666-91584ddd78e9  /     ext4    errors=remount-ro 0      1
UUID=e32e7a5d-49b4-4ae4-a43a-bcafcefa12f0  none  swap    defaults          0      0

Doing it this way is especially useful for USB devices so that you don't have to use the /dev/sdX designation (because it can change from one mount to the next depending on the characteristics of the USB controller…especially true on single-board systems that are widely available today). I have an armbian system with a USB drive I use for automatic backups at home that operates this way.

You can read more here:

https://wiki.debian.org/Part-UUID

This applies to Debian/Ubuntu…and is probably applicable to most other Linuxen as well…

Since I'm not familiar with what you're trying to do, I can't really say any more.

-- sw

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct