--- Day changed --- Log opened Tue Mar 28 23:59:04 2006 01:15 -!- kevin [~46867f42@webuser.linode.com] has joined #linode-xenbeta 01:15 < caker> Did you read the gotchas and known issues> 01:15 < caker> ? 01:16 < caker> kevin: ^-- 01:16 < kevin> ops sorry... yes, I did 01:16 < caker> ok, yeah, that's what I'm fighting with now 01:16 < caker> trying to get your node to boot with all its disk visible 01:16 < kevin> ok, I will keep trying 01:17 < caker> all right .. I'll turn it over to you then 01:17 * caker closes lish 01:18 < kevin> thx 01:22 -!- TheFirst [gaveup@CPE-70-92-72-102.new.res.rr.com] has joined #linode-xenbeta 01:29 < kevin> caker: I am now able to login from Lish.... but if I try to ssh to my linode directly, the server is not responding.... I verified in lish that sshd and apache are both running. 01:29 < TheFirst> how long does transferring the partitions generally take? 01:29 < Spads> it's quick 01:29 < Spads> quicker if you shrink first 01:30 < TheFirst> yah ... i forgot too ... but only had like 100 i could shrink by anyway so .... 01:44 < kevin> hi, can you please help on my previous message? 01:44 -!- kevin [~46867f42@webuser.linode.com] has quit [Quit: CGI:IRC 0.5.7 (2005/06/19)] 01:45 -!- kevin [~46867f42@webuser.linode.com] has joined #linode-xenbeta 01:51 < caker> kevin: networking setup correctly? 01:51 < kevin> yes, I just did a reboot. and see if that help 01:53 < caker> kevin: Starting DHCP server: dhcpd failed to start - check syslog for diagnostics. 01:54 < kevin> I might just remove it, since i don't really need it. 01:54 < caker> kevin: grep dhcp /var/log/messages 01:54 < caker> you could set things up manually in /etc/network/interfaces 01:55 < caker> kevin: a hah .. was set statically with your old IP address 01:55 < caker> kevin: after you fix that, just do /etc/initd./network restart .. that should fix things 01:55 < kevin> ok. thx 01:56 < caker> you need to fix /etc/network/intefaces before running network restart 01:57 < caker> sorry, it's /etc/init.d/networking restart 01:58 < caker> it pings! :) 01:58 < kevin> ok, cool. now I can login.... thx a lot for your help!! I really learn a lot from linode :) 01:59 < caker> :) 01:59 < kevin> btw, what is the process to revert the network setup back to DHCP from my manual setting? so I don't lock myself out again next time? 02:09 -!- kevin [~46867f42@webuser.linode.com] has quit [Quit: CGI:IRC (Ping timeout)] 02:46 < TheFirst> hmm noticed one odd thing about lish...but maybe it's always been this way...some things don't resize to the window, like top 02:50 < caker> what things resize your window? 02:50 < TheFirst> text output ... ps and such 02:50 < caker> console is just weird .. it always wants to default back to 80x24 or whatever it is 02:50 < TheFirst> yah 02:50 < caker> I'll often use the "reset" command 02:50 < TheFirst> ok first thing that isn't working 02:50 < TheFirst> cryptsetup 02:51 < caker> which depends on what? 02:51 < TheFirst> couldn't tell ya...i'll google and see what i find out 02:51 < caker> did that work under UML? 02:51 < TheFirst> yah 02:51 < caker> hmm 02:51 < TheFirst> worked fine under uml ... complains of invalid arguments in xen 02:52 < TheFirst> couple of kernel things it depends on ... http://deb.riseup.net/storage/encryption/dmcrypt/ lists them 02:53 < caker> # CONFIG_CRYPTO_AES is not set 02:53 < caker> # CONFIG_CRYPTO_AES_586 is not set 02:53 < caker> the two I'm missing 02:53 * caker recompiles 02:53 * Spads watches his UML node chug chug chug 02:53 < Spads> grrrr 02:53 < TheFirst> that's the only thing i've found so far ... so thats good ... well beside a apache2 issue that i'm scratching my head with but that's not xen related 02:53 < TheFirst> at least i haven't shown it is yet :P 02:56 < TheFirst> hmm ok well i'm out of ideas...apache2 init script just hangs on start and stop but it does start the thing, doesn't stop... gotta ctrlc or it hangs bootup 03:03 < caker> TheFirst: ok, kernel with requested support is out there 03:03 < caker> (just reboot to get it) 03:03 < Spads> support for what? 03:03 < TheFirst> k 03:04 < Spads> oh 03:04 < caker> crypt-dm and friends 03:04 < Spads> yeah 03:04 < TheFirst> so cool looks like its just those two issues 03:04 < TheFirst> not bad 03:04 < caker> those were to be expected .. the big one I'm fighting now is the missing devices on reboot bug 03:05 < TheFirst> i saw that one in the forums....thought wtf..that's odd 03:07 < caker> Starting web server: Apache2[Wed Mar 29 03:07:22 2006] [error] (EAI 3)Temporary failure in name resolution: Cannot resolve host name webmail.vidphiles.com --- ignoring! 03:09 < TheFirst> hmm why didn't that one get spit out to me 03:10 < caker> !dns host9.linode.com 03:10 < linbot> caker: 64.62.190.9 03:10 < caker> the rest of the dns servers for that datacentert are listed on the network info page 03:11 < TheFirst> ah i changed that prior to reboot and it got changed back at some point 03:12 < TheFirst> no i take that back...i was using that nameserver 03:12 < caker> networking up correctly? ifconfig; route -n ? 03:13 < TheFirst> i think it's a djb issue 03:13 < TheFirst> set the nameserver to 127.0.0.1 and it will resolve two domains just not vidphiles 03:14 < TheFirst> ah found the problem... 03:14 < TheFirst> damn godaddy didn't change the ips on that one 03:16 < TheFirst> ok so everything solved now 03:16 < TheFirst> just kinda wondering why the apache error didnt show up on the lish console 03:17 < caker> oh, it did -- that's where I nabbed it from 03:18 < caker> back out to Lish, and run logview 03:18 < caker> (control-a, d) 03:18 < TheFirst> ah went to that log instead of the stdout 03:18 < TheFirst> that explains it 03:18 < caker> no no .. logview in Lish *is* what went into the screen session that you're viewing 03:18 < TheFirst> oh well all i can see is it did not show up there 03:19 < caker> maybe you didnt wait long enoigh 03:19 * caker shrugs 03:19 < caker> ok, off to be for me .. 03:19 < TheFirst> if it showed up in logview it's fine for me 03:19 < TheFirst> i know to look there now 03:19 < TheFirst> yah goto bed...then i can instantly find more problems :P 03:19 < caker> hah 03:19 < caker> cya 03:20 < TheFirst> good riddens 03:20 < TheFirst> err i mean cya 03:20 < TheFirst> ;) 04:02 -!- ElectricElf [~dbharris@electricelf.noc.oftc.net] has quit [Read error: Connection timed out] 06:14 -!- TheFirst [gaveup@CPE-70-92-72-102.new.res.rr.com] has quit [Quit: You're a bloody puppet!] 06:15 -!- TheFirst [gaveup@li4-83.members.linode.com] has joined #linode-xenbeta 08:05 -!- petert_ [~peter@130.57.22.69] has joined #linode-xenbeta 08:08 -!- petert [~peter@snuvv.teichman.org] has quit [Quit: reboot] 09:32 -!- petert_ is now known as petert 09:32 < petert> caker: looks like a perfect cross-site migration for me, thanks 09:45 < petert> is it a bug or a policy decision that my authorized_keys wasn't migrated to the new node? 09:59 < mikegrb> not exactly either ;) 10:00 < mikegrb> will probably be done in the future 10:00 < petert> either is acceptable to me, just wanted to make sure you knew :) 10:00 < mikegrb> yeah 10:41 -!- JasonF [~jay@cialis.oldos.org] has quit [Ping timeout: 480 seconds] 12:33 -!- TheFirst [gaveup@li4-83.members.linode.com] has quit [Quit: Reconnecting] 12:33 -!- TheFirst [gaveup@li4-83.members.linode.com] has joined #linode-xenbeta 12:38 -!- bendy24_ is now known as scott 13:08 -!- vodka_ [~knarf@ip-81-11-200-79.dsl.scarlet.be] has joined #linode-xenbeta 13:09 -!- vodka [~knarf@ip-81-11-205-187.dsl.scarlet.be] has quit [Ping timeout: 480 seconds] 13:12 -!- TheFirst [gaveup@li4-83.members.linode.com] has quit [Quit: Reconnecting] 13:12 -!- TheFirst [gaveup@your.friendly.neighborhood.hellmouth.info] has joined #linode-xenbeta 13:23 -!- JasonF [~jay@cialis.oldos.org] has joined #linode-xenbeta 13:32 -!- vodka_ [~knarf@ip-81-11-200-79.dsl.scarlet.be] has quit [Read error: Connection reset by peer] 13:32 -!- vodka [~knarf@ip-81-11-226-166.dsl.scarlet.be] has joined #linode-xenbeta 14:04 < Spads> man, I can't wait until I can move off uml and onto xen 14:05 < caker> heh 14:05 < caker> problems? 14:08 < Spads> just load 14:08 < caker> which host? 14:08 < Spads> I've got one user who's got a huge Maildir 14:08 < caker> ah 14:08 < Spads> and he can't seem to keep mutt from re-polling it every five minutes 14:08 < Spads> I had him set timeout=0 14:08 < Spads> but... 14:09 < scott> you think xen will solve that problem?! 14:09 < scott> :) 14:09 < Spads> I think xen will make it so that his mutt doesn't throw the rest of the node into crisis 14:09 < Spads> I'd love to be able to just ionice him to oblivion 14:09 < scott> yes 14:09 < scott> that would be my suggestion 14:10 < caker> Spads: we could do that now 14:10 < Spads> oh yeah? 14:10 < Spads> ionice is supported? 14:10 < caker> yup.. running 2.6? 14:10 < Spads> yeah 14:10 < Spads> Linux frotz 2.6.12.3-linode14 #1 Tue Jul 26 23:44:29 EDT 2005 i686 GNU/Linux 14:10 < caker> hmm 14:10 < caker> not sure if cfq-ts is in that one 14:10 < Spads> Did I get bumped off of "latest" at some point? 14:11 < caker> not sure 14:11 < Spads> lemme fix it 14:11 < Spads> damb I was 14:11 < Spads> ok 14:11 < Spads> I thin it was during debugging 14:12 < caker> after you reboot: echo cfq > /sys/block/ubda/queue/scheduler (ubda, and any other devices besides swap) 14:12 < Spads> hmmm 14:12 < Spads> do scheduled reboots work yet? 14:13 < caker> heh, no 14:13 < Spads> :< 14:13 < Spads> in xen? 14:13 < caker> nope 14:13 < Spads> :< 14:16 < caker> Spads: anyway, whatever is causing that IO -- is it a persistent process or something spawned each time? 14:16 < Spads> it's a persistant process 14:16 < caker> groovey 14:16 < Spads> every few minutes it checks for mail 14:17 < Spads> and this guy never reads the mail he gets 14:17 < Spads> so Maildir/new/ has a zillion entries 14:20 -!- oneman [~oneman@ip68-230-208-113.rd.hr.cox.net] has quit [Remote host closed the connection] 14:31 < Spads> so what's the user-space tool for ionice? 14:32 < caker> google for Documentation/block/ioprio.txt 14:34 < caker> I haven't looked at the cfq-ts code recently, but I tink it 14:34 < caker> it's tied to nice levels 14:34 < caker> so just renice-ing the hell out of something should turn town its ioprio 14:34 * caker shrugs 14:39 * caker adds debug code to xebus_watch.c .. 17:20 -!- OvrLrd-Q [erwin@ip70-178-83-96.ks.ks.cox.net] has joined #linode-xenbeta 17:41 * caker is getting closer with this bug 17:43 < Spads> oh yeah? 17:43 < Spads> what's the trail now? 17:44 < caker> deadlock 17:45 < caker> I'll have some trace output in a sec 17:45 < Spads> ohhhh 17:51 < caker> ?? 17:51 < caker> # scp -P 22200 /linodes/linode3/screenlog.last caker@theshore.net:~/public_html/xen/InitWait/dmesg-not_working.txt 17:51 < caker> ssh: connect to host theshore.net port 22200: Connection refused 17:51 < caker> yet: 17:51 < caker> lost connection 17:51 < caker> # ssh -p 22000 caker@theshore.net 17:51 < caker> caker@theshore.net's password: 17:52 * caker is baffled 17:54 < mikegrb> you are giving a sekrits mr cake! 18:15 * caker fires off an email to xen-devel 18:16 < caker> http://www.theshore.net/~caker/xen/InitWait/dmesg-working.txt 18:16 < caker> http://www.theshore.net/~caker/xen/InitWait/dmesg-not_working.txt 18:34 * Spads notes the port 18:37 < OvrLrd-Q> what's the big P option for? 18:37 < caker> port 18:38 < OvrLrd-Q> thought that was little p 18:38 < caker> scp it's P 18:38 < OvrLrd-Q> oh doh 18:38 < caker> p for scp is preserve 18:38 < OvrLrd-Q> saw the s :) 18:38 < OvrLrd-Q> brain filled in sh 19:17 -!- warewolf [warewolf@warewolf.org] has joined #linode-xenbeta 19:19 < Spads> [nick@frotz(/var/www/zork.net/galleries/teh-japan)] cat /sys/block/ubda/queue/scheduler 19:19 < Spads> noop [anticipatory] deadline cfq 19:19 < Spads> I haven't even rebooted yet 19:19 < Spads> huh? 19:19 < Spads> I didn't touch it! 19:20 < Spads> is that the default now? 19:21 < OvrLrd-Q> urwha 19:21 < Spads> Ur-Quan Masters 19:26 < OvrLrd-Q> hurm 56 go down or did I fark up my node 19:26 < Spads> I think caker's playing games 19:26 < OvrLrd-Q> ah 19:27 < Spads> my xenode just came back up 19:27 < OvrLrd-Q> i take it he.net fell over again? 19:27 < OvrLrd-Q> heh or maybe it's he again 19:27 -!- TheFirst [gaveup@your.friendly.neighborhood.hellmouth.info] has quit [Remote host closed the connection] 19:27 -!- TheFirst [gaveup@your.friendly.neighborhood.hellmouth.info] has joined #linode-xenbeta 19:27 < Spads> yeah 19:27 < Spads> I think so 19:32 -!- SupaZubon [~crack@frotz.zork.net] has quit [Remote host closed the connection] 19:39 < Spads> [nick@frotz(~)] cat /sys/block/ubda/queue/scheduler 19:39 < Spads> noop [anticipatory] deadline cfq 19:39 < Spads> huh 19:39 -!- SupaZubon [~crack@frotz.zork.net] has joined #linode-xenbeta 19:54 -!- ElectricElf [~dbharris@electricelf.noc.oftc.net] has joined #linode-xenbeta 19:56 < Spads> ahhh, just figured that out 19:56 < Spads> noop anticipatory deadline [cfq] 19:57 < Spads> common ground, eh 19:58 < OvrLrd-Q> i'll just pretend like I know what each of those settings do :) 19:59 < Spads> heh 19:59 < Spads> well 19:59 < Spads> lots of folks say CFQ is a great I/O scheduler for high load situations 20:24 -!- OvrLrd-Q [erwin@ip70-178-83-96.ks.ks.cox.net] has quit [Quit: ] 21:59 < caker> I think I hacked my way through this bug 22:00 < caker> no clue as to the root cause, but I think it's working 22:02 < caker> wow... 22:03 < caker> four reboots, worked every time 22:04 < TheFirst> the proc bug? 22:04 < caker> proc bug? 22:04 < caker> No, the missing devices bug 22:04 < caker> what's the proc bug? 22:04 < caker> six reboots... 22:05 < TheFirst> that's the one i meant... 22:06 < TheFirst> or maybe not...i was thinking the /proc/devices bug... 22:06 < caker> aye .. /proc/partitions, rather 22:07 < Spads> well at least you can show your hack to the xen folks 22:07 < caker> not much of one .. I comment out part of their code 22:08 < Spads> ! 22:08 < Spads> still, that's bound to get someone's attention 22:08 < Spads> perhaps the person who wrote that line :) 22:08 < caker> their code is actually pretty resilient -- they're making secondary attempts to update the "status" field, but then they're skipping over actually write out the status, because they think it's already been set 22:09 < caker> but the question remains: why the first (true) writeout didn't stick 22:09 < caker> ok, 12 reboots, all four devices have appeared each time 22:09 < caker> I 22:09 < caker> I'm considering that fixed 22:09 < caker> now to just undo all my printks :) 22:09 < caker> man, this is awesome ... 22:10 < caker> I had the feeling like this was going to loom over me for months 22:10 < caker> I had that "pre-token-limiter-hell" feeling 22:10 < Spads> haha 22:10 < Spads> oh man 22:11 < Spads> did you open for business without the token limiter? 22:11 < caker> YES 22:11 < Spads> Doom. 22:11 < Spads> how long did that last? 22:11 < Eman> that calls for lolz 22:11 < caker> I spent about a week straight, no sleep, once I finally got the idea 22:11 < caker> <-- not much of a C guy, and def. not much of a kernel hacker 22:11 < Spads> yeah, I feel ya there 22:11 < caker> lemme see ... 22:12 < caker> http://www.linode.com/forums/viewtopic.php?t=790 <-- 22:12 < caker> Linode.com Status Update 04/06/04 22:12 < TheFirst> not much? hell i like to consider myself one and i'd not touch what you mess with 22:12 < Spads> haha yeah 22:12 < Spads> like, hacking with drivers is one thing 22:12 < caker> I launched June 16th, 2003 22:12 < Spads> but poking at the VM and stuff... 22:13 < caker> fortunately, the part the token-limiter messes with was in the U part of UML (userland, not the kernel port side) 22:14 < caker> TheFirst: mind pasting "uname -a" from your xenode, please? 22:15 < caker> pretty sure I upped the .version a bunch doing all these builds 22:15 < TheFirst> sure...why 22:15 < caker> host56:~/xen-unstable.hg/linux-2.6.16-xenU# cat .version 22:15 < caker> 17 22:15 < TheFirst> Linux hellmouth 2.6.16-domU-linode1 #5 SMP Wed Mar 29 03:02:56 EST 2006 i686 GNU/Linux 22:16 < caker> ok, build #5, thanks 22:16 < TheFirst> that'll be $50 for labor ;P 22:22 < caker> # cat /proc/partitions 22:22 < caker> major minor #blocks name 22:22 < caker> 3 1 1024000 hda1 22:22 < caker> 3 2 266240 hda2 22:22 < caker> 3 3 102400 hda3 22:22 < caker> 3 4 217088 hda4 22:22 * caker snickers 22:23 < Spads> heh 22:25 < caker> li3-242:~# uname -a 22:25 < caker> Linux li3-242 2.6.16-domU-linode1 #6 SMP Wed Mar 29 22:21:16 EST 2006 i686 GNU/Linux 22:25 * caker moves that kernel into place 22:32 < TheFirst> there are still outstanding issues ... you just don't know about them yet ;P 22:37 < caker> works for me 22:40 < TheFirst> bah! no rest for the admins...that's the rule dag nabit ;P --- Log closed Wed Mar 29 23:59:00 2006