Linode.com Forum

[wazdog]

Okay, so in searching the Gentoo forums and bugzilla for a solution to (and reason for) this problem:

http://www.linode.com/forums/viewtopic.php?t=557

(i.e. having to comment out auth_digest_module in apache2 in order to get apache2 to spawn child processes)


I came across the following info:

http://bugs.gentoo.org/show_bug.cgi?id=37039

In effect, apache2 needs to read some data from /dev/random in order to generate keys when starting up auth_digest, and can't because there is no entropy in /dev/random, so the startup fails. One solution is to comment out the auth_digest_module, and the other is to populate /dev/random.

This is where I am having trouble. Suggestions on the Gentoo forums include running du /usr in order to seed the random with the disk access... but this fails on my Linode. I have a feeling it's because our kernel doesn't have direct hardware access because it is running under UML. Is this a reasonable assumption?

And if so, any suggestions on how to populate /dev/random and /dev/urandom ? This can be a problem for many other programs other than just apache2, because any program that relies on random data may fail...

So does anyone else have any info in /dev/random, or is this just a Gentoo problem. If my above assumptions are correct, it may be a Linode problem in general...


(One can go to /proc/sys/kernel/random and read entropy_avail to see if they have any info in random. If it says "0", you don't...)


thanks
- j

[pclissold]

I am running Gentoo on my Linode with Apache 2.0.48 and auth_digest_module working OK. My /dev/random and /dev/urandom are both populated and entropy_avail = 343(without any specific action on my part) so this does not seem to be problem that is specific to Linodes.

[wazdog]

Okay, so hmmmmm.

Wonder why I can't get /dev/random to populate? Any suggestions anyone?

From what I've read, disk access should populate /dev/random, that's why I read "du /usr" should work. But it doesn't for me.

In further researching, I've found these threads, one guy runs under UML and has no problems:

http://www.securityfocus.com/archive/1/330526/2003-07-21/2003-07-27/0

http://www.mail-archive.com/uug-list@uug.byu.edu/msg07982.html

Anyone else having problem with /dev/random? How else does /dev/random get populated? Shouldn't the kernel populate it, as well as just about all other disk access, etc?

pclissold, which kernel are you running under? From what I've read, this may be a kernel issue...

Also, what bad things can happen with no entropy? I know apparently apache has problems with auth_digest, and Ive read ssl over apache wont work right. Also, ssh uses /dev/random, so what effect should this have on my ssh session Im using now?

Sorry for all the questions, I'm just at a loss for what to do, and cant' even find any more info out there...

thanks
- j

p.s. on last question, how does one find out which processes are calling /dev/random for entropy?

[wazdog]

Okay, so I know where the dilemma may be. Something is sucking my /dev/random dry real quick.

No sooner do i post that long-ass reply above, but i reboot my linode again, and immediately go off to "/proc/sys/kernel/random/entropy_avail" and now I have some. I run "du /usr" to kick it up, and it does.

But immediately starts plummeting, and goes completely empty in about 60-120 seconds.

So what may be sucking it dry so fast? What would be constantly accessing it? Any ideas? How do I find out what processes are calling it?

At least now, running "du /usr" will repopulate it, at least temporarily...

[pclissold]

wazdog wrote: pclissold, which kernel are you running under? From what I've read, this may be a kernel issue...
Currently running 2.4.24-linode20-1um

[mikegrb]

2.4.24-linode19-1um here. Perhaps we should compare stuff running at startup?

michael@orion michael $ ls /etc/runlevels/default/
apache2 courier domainname hostname iptables local mysql net.eth0 net.lo netmount ntpd pdnsd sshd sysklogd vcron

nothing there would strike me as being the problem right off. As for adding entropy lots of stuff do it. Disk and other i/o and cpu usage are the big ones though. This seems odd. What kernel are you using and what do you have starting up at boot wazdog?

pclissold could you also paste your list of applications starting up?

wazdog: sorry for not replying sooner, was out of town ;)

[pclissold]

mikegrb wrote: pclissold could you also paste your list of applications starting up?
peter@fremont peter $ ls /etc/runlevels/default
apache2 courier-imapd courier-imapd-ssl courier-pop3d courier-pop3d-ssl domainname hostname local mysql net.eth0 net.lo netmount postfix saslauthd sshd sysklogd

[wazdog]

hey thanks for chiming in mike...

my defaults are

hobbes root # ls /etc/runlevels/default/
domainname hostname local mysql net.eth0 net.lo netmount sshd sysklogd webmin


mike, are you showing 0 entropy as well? or are you stocked up?

i'm running 2.4.24-linode20-1um as well...

[wazdog]

i've only recently installed this gentoo system.

Here is what i've done so far (been keeping a log, so hey, guess it'll come in handy!):

emerge sync && emerge screen && emerge -u system && emerge -u world && emerge gentoolkit

emerge apache2 php mod_php mysql postfix bind bind_tools webmin

and that's about it. I've had some other stuff pulled in by php, but nothing major. only started apache2 up once, and got only one child, so went searching on the gentoo forums...

--

okay, i just checked my other gentoo host (on another linode host) and the entropy seems fine now, BUT I know for a fact that I had the apache2 problem on that host as well, but in that case, I've commented out auth_digest....

and no, apache2 is not currently running on the host that has the entropy problem (and it's back down to zero and not rising whatever i do...)

weird.

- j

[mikegrb]

wazdog, how frequently do you use screen? You mention having it instaled... I use it two but wouldn't expect it to cause this problem, who knows.

pclissold, do you use screen?

Here is the output of pstree, listing all running processes:
Code: michael@orion michael $ pstree
init-+-agetty
     |-apache2-+-13*[apache2]
     |         `-apache2---nph-irc.cgi
     |-authdaemond.pla---5*[authdaemond.pla]
     |-bdflush
     |-courierd---courierd-+-courierdsn
     |                     |-courieresmtp---courieresmtp
     |                     |-courierfax
     |                     |-courierlocal
     |                     `-courieruucp
     |-courierfilter
     |-3*[courierlogger]
     |-couriertcpd---couriertls---imapd
     |-couriertcpd
     |-couriertls
     |-cron---cron---startserver.sh---sleep
     |-devfsd
     |-gim.pl
     |-infobot
     |-ircd---4*[servlink]
     |-jfsCommit
     |-jfsIO
     |-jfsSync
     |-keventd
     |-kjournald
     |-klogd
     |-ksoftirqd_CPU0
     |-kswapd
     |-kupdated
     |-mdrecoveryd
     |-mysqld_safe---mysqld---mysqld---6*[mysqld]
     |-ntpd
     |-pdnsd---pdnsd---2*[pdnsd]
     |-screen-+-bash---irssi
     |        |-bash---mutt
     |        `-bash---pstree
     |-screen---bash---irssi
     |-screen-+-bash---irssi
     |        `-bash
     |-sshd---3*[sshd---sshd---bash---screen]
     `-syslogd

[wazdog]

i only use screen when emerging stuff, so not too often really!

Okay, I just stopped all services, except those that run by default on caker's setup... then ran "emerge sync" to get some entropy, and got up to 600 (the highest Ive ever noticed, which isnt good, eh, cuz I should have up to 4k, right?)

But it's slowly fading away. Not as fast as before, but its constantly decreasing...

this is all i got going now

Code:
hobbes root # pstree
init-+-agetty
     |-bdflush
     |-devfsd
     |-dhcpcd
     |-jfsCommit
     |-jfsIO
     |-jfsSync
     |-keventd
     |-kjournald
     |-klogd
     |-ksoftirqd_CPU0
     |-kswapd
     |-kupdated
     |-mdrecoveryd
     |-sshd---sshd---bash---pstree
     `-syslogd


Should my entropy be seeping away? Is ssh or something else constantly requesting it? I was under the impression it was only really used to create keys, etc...

thanks for helping
-j

p.s. sorry for asking again, but mike are you having the entropy problem too, or just the apache2 startup problem?

[pclissold]

mikegrb wrote: pclissold, do you use screen?
Never on this Linode.

[mikegrb]

yes, I'm having 0 entropy too.

pclissold, do you mind posting pstree output?

entropy could be being used by ssh. I know the server has a key that is regenerated every hour or some such. I don't know if it continues to use entropy, I wouldn't imagine so. Do you normally have an ssh session open pclissold and wazdog? I normally have at least one normally two or three.

Edit:
doing du -sh /usr and checking during and after show no entropy available. What does a normal pstree look like for you wazdog? With normal stuff running do you still see some entropy available after du /usr or do you see none?

[wazdog]

yeah, Im always connected via ssh.

At the rate mine entropy decreases, ssh would have to be regenerating keys every minute...

i don't know, weird.

I just went through my log files, nothing suspect, but hey, how would I know really? This one's got me stumped...

(though i did verify that this IS the reason auth_digest wont work... in my apache error_log, apache choked on: "[Sun Jan 25 13:49:43 2004] [notice] Digest: generating secret for digest authentication ...")

- j

okay, off to bed, gotta hit the slopes tomorrow morning. i'll check back in a day... thanks all, take care

[blahrus]

I want to hit the slopes