Ubuntu 10.04: corrupted symlinks
At first glance everything seems to be in order but when I check untarred files, there are many symlinks invalid (i.e. zero length files with 000 permissions). Strange enough, those invalid symlinks are not same on multiple tries - as far as I noticed it's like random 70 % of symlinks are invalid. I tried many different kernels assuming it was fs issue without any success.
When I swapped to Ubuntu 9.10 (with same kernel) and performed exactly same actions, extracted symlinks were correct.
Reproducible - always:
Deploy Ubuntu 10.04/9.10 x86 (512M + 256 swap, latest paravirt)
apt-get update && apt-get upgrade && apt-get install lvm2
mke2fs -t ext4 /mnt/gentoo && mount /mnt/lvm/gentoo
wget stage3.tbz2 && md5sum OK
tar xf stage3.tbz2
On Ubuntu 10.04 symlinks are invalid, on 9.10 they are correct.
Any ideas or experience? I am not stuck but I would rather have rescue system LTS (but working one).
4 Replies
@jed:
Hm. Does any stage3 file exhibit this behavior, or can you point to a specific URL? I trust any stage3 does same thing since I encountered this bug few month ago with same result (I didn't report then).
Yesterday I used this one:
If you are gonna test this, it should be enough to look in /bin and /etc directories for 000-perm files.
One more thing - when I encountered this few months ago, I tried various kernels (linode and pv_opts >2.6.28) and several tar packages without any difference. And I didn't use lvm too so it shouldn't be cause.
I didn't use LVM (I have no idea what you're doing), but I made an ext4 filesystem on /dev/xvdc, mounted it to /mnt, and extracted the same tarball you did under 10.04 LTS:
root@li167-124:/mnt/bin# ls -l
total 6116
-rwxr-xr-x 1 root root 9388 Sep 15 03:00 attr
lrwxrwxrwx 1 root root 10 Sep 17 16:57 awk -> gawk-3.1.6
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 basename
-rwxr-xr-x 1 root root 862428 Sep 15 02:28 bash
lrwxrwxrwx 1 root root 7 Sep 17 16:57 bb -> busybox
lrwxrwxrwx 1 root root 5 Sep 17 16:57 bunzip2 -> bzip2
-rwxr-xr-x 1 root root 984624 Sep 15 04:01 busybox
lrwxrwxrwx 1 root root 2 Sep 17 16:57 busybox.static -> bb
lrwxrwxrwx 1 root root 5 Sep 17 16:57 bzcat -> bzip2
-rwxr-xr-x 1 root root 33996 Sep 15 01:25 bzip2
-rwxr-xr-x 1 root root 42260 Sep 15 03:35 cat
-rwxr-xr-x 1 root root 13484 Sep 15 03:01 chacl
-rwxr-xr-x 1 root root 2277 Sep 15 02:51 checkpath
-rwxr-xr-x 1 root root 50476 Sep 15 03:35 chgrp
-rwxr-xr-x 1 root root 46384 Sep 15 03:35 chmod
-rwxr-xr-x 1 root root 54572 Sep 15 03:35 chown
-rwxr-xr-x 1 root root 29996 Sep 15 03:35 chroot
-rwxr-xr-x 1 root root 95544 Sep 15 03:35 cp
-rwxr-xr-x 1 root root 138392 Sep 15 01:29 cpio
-rwxr-xr-x 1 root root 38136 Sep 15 03:35 cut
-rwxr-xr-x 1 root root 58672 Sep 15 03:35 date
-rwxr-xr-x 1 root root 50500 Sep 15 03:35 dd
-rwxr-xr-x 1 root root 66864 Sep 15 03:35 df
-rwxr-xr-x 1 root root 103956 Sep 15 03:35 dir
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 dirname
-rwxr-xr-x 1 root root 5292 Sep 15 02:48 dmesg
lrwxrwxrwx 1 root root 8 Sep 17 16:57 dnsdomainname -> hostname
lrwxrwxrwx 1 root root 8 Sep 17 16:57 domainname -> hostname
-rwxr-xr-x 1 root root 95540 Sep 15 03:35 du
-rwxr-xr-x 1 root root 25848 Sep 15 03:35 echo
-rwxr-xr-x 1 root root 83480 Sep 15 03:31 egrep
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 env
-rwxr-xr-x 1 root root 34088 Sep 15 03:35 expr
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 false
-rwxr-xr-x 1 root root 50708 Sep 15 03:31 fgrep
-rwxr-xr-x 1 root root 26120 Sep 15 05:06 fuser
lrwxrwxrwx 1 root root 10 Sep 17 16:57 gawk -> gawk-3.1.6
-rwxr-xr-x 1 root root 345984 Sep 15 03:14 gawk-3.1.6
-rwxr-xr-x 1 root root 22060 Sep 15 03:01 getfacl
-rwxr-xr-x 1 root root 17920 Sep 15 03:00 getfattr
-rwxr-xr-x 1 root root 87576 Sep 15 03:31 grep
-rwxr-xr-x 1 root root 9388 Sep 15 03:57 groups
-rwxr-xr-x 1 root root 2249 Sep 15 03:12 gunzip
-rwxr-xr-x 1 root root 89668 Sep 15 03:12 gzip
-rwxr-xr-x 1 root root 34096 Sep 15 03:35 head
-rwxr-xr-x 1 root root 13544 Sep 15 03:13 hostname
lrwxrwxrwx 1 root root 20 Sep 17 16:57 igawk -> /usr/bin/igawk-3.1.6
-r-xr-xr-x 1 root root 13484 Sep 15 05:07 kill
-rwxr-xr-x 1 root root 42296 Sep 15 03:35 ln
-rwxr-xr-x 1 root root 38572 Sep 15 03:57 login
-rwxr-xr-x 1 root root 103956 Sep 15 03:35 ls
-rwxr-xr-x 1 root root 9388 Sep 15 02:50 lsmod
-rwxr-xr-x 1 root root 42232 Sep 15 03:35 mkdir
-rwxr-xr-x 1 root root 25848 Sep 15 03:35 mkfifo
-rwxr-xr-x 1 root root 29996 Sep 15 03:35 mknod
-rwxr-xr-x 1 root root 34092 Sep 15 03:35 mktemp
-rwxr-xr-x 1 root root 38156 Sep 15 02:48 more
-rws--x--x 1 root root 67404 Sep 15 02:48 mount
-rwxr-xr-x 1 root root 9388 Sep 15 01:35 mountpoint
-rwxr-xr-x 1 root root 87352 Sep 15 03:35 mv
-rwxr-xr-x 1 root root 169312 Sep 15 04:03 nano
-rwxr-xr-x 1 root root 130092 Sep 15 03:13 netstat
lrwxrwxrwx 1 root root 8 Sep 17 16:57 nisdomainname -> hostname
-rws--x--x 1 root root 40648 Sep 15 03:57 passwd
lrwxrwxrwx 1 root root 20 Sep 17 16:57 pgawk -> /usr/bin/pgawk-3.1.6
lrwxrwxrwx 1 root root 14 Sep 17 16:57 pidof -> /sbin/killall5
-rws--x--x 1 root root 38460 Sep 15 02:59 ping
-r-xr-xr-x 1 root root 87492 Sep 15 05:07 ps
-rwxr-xr-x 1 root root 25848 Sep 15 03:35 pwd
lrwxrwxrwx 1 root root 4 Sep 17 16:57 rbash -> bash
-rwxr-xr-x 1 root root 9296 Sep 15 02:51 rc-status
-rwxr-xr-x 1 root root 34092 Sep 15 03:35 readlink
-rwxr-xr-x 1 root root 50480 Sep 15 03:35 rm
-rwxr-xr-x 1 root root 34040 Sep 15 03:35 rmdir
lrwxrwxrwx 1 root root 4 Sep 17 16:57 rnano -> nano
-rwxr-xr-x 1 root root 13732 Sep 15 01:35 run-parts
-rwxr-xr-x 1 root root 63112 Sep 15 03:11 sed
-rwxr-xr-x 1 root root 38136 Sep 15 03:35 seq
-rwxr-xr-x 1 root root 30252 Sep 15 03:01 setfacl
-rwxr-xr-x 1 root root 13636 Sep 15 03:00 setfattr
lrwxrwxrwx 1 root root 4 Sep 17 16:57 sh -> bash
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 sleep
-rwxr-xr-x 1 root root 83404 Sep 15 03:35 sort
-rwxr-xr-x 1 root root 58616 Sep 15 03:35 stty
-rws--x--x 1 root root 34572 Sep 15 03:57 su
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 sync
-rwxr-xr-x 1 root root 58696 Sep 15 03:35 tail
-rwxr-xr-x 1 root root 362432 Sep 15 03:16 tar
-rwxr-xr-x 1 root root 9440 Sep 15 01:35 tempfile
-rwxr-xr-x 1 root root 46332 Sep 15 03:35 touch
-rwxr-xr-x 1 root root 38188 Sep 15 03:35 tr
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 true
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 tty
-rws--x--x 1 root root 46512 Sep 15 02:48 umount
-rwxr-xr-x 1 root root 25852 Sep 15 03:35 uname
lrwxrwxrwx 1 root root 6 Sep 17 16:57 uncompress -> gunzip
-rwxr-xr-x 1 root root 103956 Sep 15 03:35 vdir
-rwxr-xr-x 1 root root 34092 Sep 15 03:35 wc
-rwxr-xr-x 1 root root 21752 Sep 15 03:35 yes
lrwxrwxrwx 1 root root 8 Sep 17 16:57 ypdomainname -> hostname
-rwxr-xr-x 1 root root 1941 Sep 15 03:12 zcat
Fine for me.
The issue is certainly unrelated to LVM and more than likely unrelated to ext4.
Since disk addressing should be unrelated to ext4 symlinks, I am puzzled of cause - it might be some rarely fired bug in Xen translating disk operations. Thus I swapped to partition disk using sectors and now it seems to be OK. Hopefully this may help someone when he encounter same issue as I did.
So I am ceasing torturing london126 I/O and continue with regular installation. Sorry guys, hope you don't hate me too much now.