Linode.com Forum

[jimflip]

my server basically stopped working Ubuntu 8.4 server edition. When i try and boot it I get the following.

NET: Registered protocol family 17
NET: Registered protocol family 15
Bridge firewalling registered
Ebtables v2.0 registered
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
SCTP: Hash tables configured (established 16384 bind 16384)
Starting balanced_irq
Using IPI Shortcut mode
unknown partition table
XENBUS: Device with no driver: device/console/0
md: Autodetecting RAID arrays.
md: Scanned 0 and added 0 devices.
md: autorun ...
md: ... autorun DONE.
kjournald starting. Commit interval 5 seconds
EXT3 FS on xvda, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem).
Freeing unused kernel memory: 332k freed
Warning: unable to open an initial console.


When I mount it via Finnix my /dev folder is missing....HELP?
From what i can gather off the interweb is that my console device is not created thus I cannot open an initial console.

Any help would be really appreciated.

Jim.

[piglet]

If the dev directory is missing, does creating it make any difference? On my Ubuntu 8.04 system, /dev is of type tmpfs:
udev on /dev type tmpfs (rw,mode=0755)

So I would think that the contents of /dev don't survive a reboot, so maybe just having the directory there to serve as a mount point is enough.

Alternately, Ubuntu has a /sbin/MAKEDEV shell script (at least, over here it does), which could be used to fill up an empty /dev if you were so inclined. I think you cd to the new dev directory, and then run 'MAKEDEV generic'.

[jimflip]

I'm unable to cd into the dev folder :)

inspection of my daemon.log shows this about the time of the crash

Jun 30 05:42:05 doughnut modprobe: FATAL: Could not load /lib/modules/2.6.18.8-domU-linode7/modules.dep: No such file or directory
Jun 30 05:42:05 doughnut modprobe: FATAL: Could not load /lib/modules/2.6.18.8-domU-linode7/modules.dep: No such file or directory
Jun 30 05:48:58 doughnut init: tty1 main process (2062) killed by TERM signal
Jun 30 05:48:58 doughnut mysqld[29849]: 080630 5:48:58 [Note] /usr/sbin/mysqld:
Normal shutdown
Jun 30 05:48:58 doughnut mysqld[29849]:
Jun 30 05:48:58 doughnut mysqld[29849]: 080630 5:48:58 InnoDB: Starting shutdown...
Jun 30 05:49:00 doughnut mysqld[29849]: 080630 5:49:00 InnoDB: Shutdown completed; log sequence number 0 4947393


udev and modprobe are a mystery to me made even more difficult by having the server on a VPS.

[gsf]

Was your /etc/fstab changed in some way so that /dev is no longer mounted?

[NeonNero]

gsf wrote: Was your /etc/fstab changed in some way so that /dev is no longer mounted?
I don't think fstab has anything to do with that, though. I've checked on 3 different machines now, and neither references /dev in the /etc/fstab file:

One of my Linodes (Debian Linux): Code: /dev/ubda /     ext3    defaults 0 0
/dev/ubdb swap  swap    defaults 0 0
proc    /proc   proc defaults 0 0
The firewall computer at work (also Debian Linux, not virtualized in any way): Code: proc            /proc           proc    defaults        0       0
/dev/hda1       /               ext2    defaults,errors=remount-ro 0       1
/dev/hda5       none            swap    sw              0       0
/dev/hdd        /media/cdrom0   udf,iso9660 user,noauto     0       0
/dev/fd0        /media/floppy0  auto    rw,user,noauto  0       0
The primary e-mail server at work (Gentoo Linux, also not virtualized in any way): Code: /dev/hda1               /boot           ext2            defaults,noatime        1 2
/dev/hda2               none            swap            sw              0 0
/dev/hda3               /               ext3            noatime         0 1
/dev/hdb1               /brukere        ext3            noatime         0 1
/dev/hdb2               /home/vmail     ext3            noatime         0 1
/dev/cdroms/cdrom0      /mnt/cdrom      auto            noauto,ro,user  0 0
proc                    /proc           proc            defaults        0 0
none                    /dev/shm        tmpfs           nodev,nosuid,noexec     0 0
For now, I think we can eliminate /etc/fstab as being the "culprit" in this situation.

[jimflip]

/proc /proc proc defaults 0 0
/sys /sys sysfs defaults 0 0
/dev/xvda / ext3 defaults 0 1
/dev/xvdb none swap defaults 0 0

I don't think it to do with fstab, but thanks for the help.
It would appear as if udev isn't being called on start up or perhaps its not being called early enough.

[pclissold]

jimflip wrote: I'm unable to cd into the dev folder :)
With your disk image mounted under Finnix, you should be able to cd into the dev folder - it normally contains a default set of device nodes. udev mounts a tempfs on /dev at boot, making the defaults invisible and replacing them with the device nodes it has created in the tempfs. Check /dev permissions: Code: drwxr-xr-x  4 root root 12288 Jul  4 09:03 dev
If necessary, remove and recreate /dev I shouldn't matter that the defaults are missing since udev will override them anyway.

Check the contents of /etc/udev/rules.d. It should look something like this: Code: 05-options.rules                  70-persistent-cd.rules
05-udev-early.rules               70-persistent-net.rules
20-names.rules                    75-cd-aliases-generator.rules
30-cdrom_id.rules                 75-persistent-net-generator.rules
40-basic-permissions.rules        80-programs.rules
40-permissions.rules              85-hdparm.rules
45-fuse.rules                     85-hwclock.rules
60-persistent-input.rules         85-ifupdown.rules
60-persistent-storage.rules       85-pcmcia.rules
60-persistent-storage-tape.rules  90-modprobe.rules
60-symlinks.rules                 95-udev-late.rules
61-persistent-storage-edd.rules   README

Check that /etc/udev/udev.conf is not corrupted. Contents on a fresh install: Code: # udev.conf

# The initial syslog(3) priority: "err", "info", "debug" or its
# numerical equivalent. For runtime debugging, the daemons internal
# state can be changed with: "udevcontrol log_priority=<value>".
udev_log="err"

[jimflip]

Thanks all for your help, it's working again now, which is a big relief.

I created a new /dev folder and ran 'MAKEDEV generic' in it. Did a reboot and bingo...it works.

Udev seems a bit fragile to me and found lots of people having problems with it on the internet..it does all seem a bit odd, I will investigate it further.

Again many thanks all.

[Jay]

I know you had problems with it... but the truth is that udev is pretty standard at this point.

Perhaps some rogue app deleted your "base" dev nodes and caused things to break? It seems like the deeper problem is that your /dev/ got cleared out.

[jimflip]

Quote: Perhaps some rogue app deleted your "base" dev nodes and caused things to break? It seems like the deeper problem is that your /dev/ got cleared out.

I agree, this all happened after I did a apt-get upgrade followed by a manual reboot.

My server is 7.10 server upgraded to 8.04 server Ubuntu, I have nothing too funky installed and everything is from standard apt repositiories.

I have had problems previously with udev on doing upgrades with ubuntu on a desktop machine.