IRC Logs for #crux-devel Sunday, 2007-08-12

prologicwot ?00:19
prologicyes ccze is cool!00:19
prologicand yes by default it'll use curses00:19
prologicccze is also plugable00:20
*** jaeger has quit IRC00:36
Romsteryeah it needs a regex for gcc compile output i've googled and not found anything.00:43
Romsterso i guess i'll do that i've learned regex enougth (i think)00:43
*** maro has joined #crux-devel01:54
*** treach has joined #crux-devel02:47
tilman1.) write a neutral mail02:56
tilman2.) get shit back02:56
tilman3.) ???02:56
tilmanthe usual fourth step can't be applied here i think02:56
treachyeah, he's incredible.02:57
treachor rather "unbelivable"02:57
treachsour grapes in spades. :p02:57
treach"you're using ad-hoc methods" wth is that supposed to mean? "You came up with a smarter way than I, damn you"?03:02
tilmannot sure03:30
*** cptn has joined #crux-devel04:40
*** ChanServ sets mode: +o cptn04:41
cptnRomster: I've implemented a file cache for fsearch04:42
cptnRomster: I'd appreciate if you could check whether it's "fast enough" :-)04:43
treachnah, it's probably pretty useless. :>04:53
treachhi, btw :)04:53
* Romster looks05:18
cptnthere's a bug when searching for files which are symlinks05:22
cptnbut I'm about to fix that05:22
Romsterk minor thing i'm about to test it.05:27
cptnjust updated it, should be fixed now05:28
Romsterwas it you i emailed about the infinity loop?05:28
Romsterages ago..05:28
cptnthat's fixed in git05:28
cptnit's just that it was after the 5.15 release05:29
Romsterah sadly.05:29
Romsterdidn't realsie it was you lol.05:29
Romsterwish i could program decently :)05:30
Romsterok lets see how we go..05:30
cptnI guess the endless loop fix and the fsearch cache could justify a new release...05:31
Romsterits doing all in /usr/ports/* arn't it so it dosn't matter on prtdir in prt-get.conf05:31
Romsterone other thing05:31
Romsterregarding the kernel05:31
cptnno, this version only uses the prtdir's specified in prt-get.conf05:31
Romsterbut it could be in pkgutils too.. whos maintianing that?05:31
Romsterah k.05:32
cptnwell, tilman did most of the pkgutils work05:32
cptnso I'd ask him first05:32
cptnhe'll delegate it if needed05:32
Romsterhmm reasonable speed it seems better than it used to be.05:34
cptn"reasonable" doesn't sound like a major win though05:35
Romsteris it possable to have anothe entry to list ports for fsearch and not have them installable by the prtdir entry like searchdir05:35
treachreasonable..? Have you tried without the chache?05:35
Romsteri haven't tested it alot yet.05:35
Romsternot yet hench i haven't tested alot yet..05:35
Romster-rw-r--r--  1 root root  40M Aug 12 20:34 prt-get.filecache05:38
Romster40 megabytes lol...05:38
cptnmmmh, maybe the file cache should have an additional configuration key to enable/disable it05:39
Romstercptn, not related to speed but gziping it sounds like an idea.05:39
cptnnot sure if that's really worth the effort05:40
Romsterok why did fsearch work without the cache O_o05:41
Romsterfall back method?05:41
cptnit parses the footprints05:41
cptnlike it used to do since 200305:42
Romsterprt-get cache makes the prt-get.filecache file of all the fiels from the .footprint files.05:42
treachcptn: what about lzo? At least it's supposedly faster than gzip.05:42
cptnmmmh, yeah05:43
Romsterlzma :P lzo is quite fast. even distcc can use lzo.05:43
treachthat said, it's not exactly like *I'm* suffering, it's 13MB here. :)05:43
cptnOTOH zlib is in core05:43
cptn18 MB here05:43
cptnI'd mainly like to avoid the external dependency05:44
treachugly.. but there's the static way.. :/05:44
Romsterem i doing it right i remove both cache files and then do a fsearch and the speed seems the same..05:47
cptnbut you used 'prt-cache' or 'prt-get --cache' before?05:48
treachem, you're screwing something up.05:48
Romsterprt-get cache05:48
cptnand then prt-get --cache fsearch05:48
cptnor prt-cache fsearch05:49
Romstershush treach05:49
Romsterok one search05:50
Romsterwith --cache05:50
Romsterreal    0m0.683s05:50
Romsteruser    0m0.451s05:50
Romstersys     0m0.156s05:50
Romsterwithout on same search05:51
Romsterreal    0m1.633s05:51
Romsteruser    0m1.108s05:51
Romstersys     0m0.340s05:51
cptn--cache probably also benefits more form disk cache05:51
Romsteri've had searches that took longer than that though...05:51
cptnso repeated searches should be faster05:51
Romstermemory buffer.05:51
Romsteri noticed doing prt-get cache took less time the second time to index.05:52
Romsterafter i had deleated the cache files.05:53
Romsterits about 1 second faster for me. why its running quicker now than othe times it takes a /few/ seconds like 4 or so hmm.05:54
Romstercptn, what's the point of this /usr/bin/prt-cache -> prt-get06:00
Romsterrun prt-get with no args says to try prt-get help06:01
Romsterso running prt-cache will do what now :P06:01
cptnit's called a symbolic link06:01
cptnso you can call prt-get using 'prt-cache'06:02
Romsteri know that06:02
cptnin this case, all arguments you pass to prt-cache will be passed to prt-get06:02
Romsterbut it dosn't have a poupose..06:02
Romsteri was expecting prt-cahe to do prt-get cache06:02
cptnwhat about ls -l /usr/bin/pkgrm /usr/bin/pkginfo06:02
cptnto, it'll do 'prt-get --cache'06:03
Romsterand i noticed a time change on disk cache as you said too. second time ran faster.06:06
Romsterwithout --cache06:06
Romsteri'd imagein the --cache option would be betetr over I/O performance.06:07
Romsterbetter* sorry about my typing..06:07
treachthe cache is pretty much consistently twice as fast for me.06:09
Romstercould it be posable to generate the cache if it dosn't exist instead of saying it failed to open the cache?06:10
Romster1.1seconds down to 0.6 but 1.1 is not a true number i seen over 2 seconds, i'm wondering where the slowness is, i think its from no .footprint files not cached at all on the I/O buffer.06:12
Romsterfor the first run.06:12
Romsterthe cache file would give more consistant results.06:13
Romsterbeing 1 file.06:13
Romsterwhat do you think cptn.06:13
Romsteri would prefer being able to fearch all /usr/ports as i do't always list them all on prtdir entrys. as i don't wanna install then with prt-get but i do monitor them for changes and search them using grep to find some obsecure file. and having fsearch use all of /usr/ports would be ideal or a alternate prt-get entry for only fsearch but not able to install with prt-get.06:15
* Romster shrugs06:15
Romsterkinda the fsearch that the old web site had..06:16
cptnthat doesn't quite fit with prt-get's ideas06:17
cptnalthough I can see why you'd want that06:17
cptni.e. ports can be everywhere06:17
cptnbut fsearch_path might be okay06:18
cptnthat's just more code I'm interested in myself at this point in time06:18
Romsterhmm pointing fsearch_path to /usr/ports/ and it'll go though all .footprints it finds.06:23
Romsterthat does sound like an idea..06:23
*** Romster has quit IRC06:39
*** Romster has joined #crux-devel06:43
Romsterif its not too much trouble of course it'll work as it is now if fsearch_path isn't set. although all ports are in /usr/ports now so a option to turn it on, default off, would be enougth. kills off using grep/find/locate... but then there could be that XML interface to the ports repo on the crux web site too :/ really hard to decide on keeping things simple and still have the functionality of what one/many wants.06:52
Romstercptn, ^^ i pingedout saw what i typed that got mised from the online log file.06:52
Romsterand now i find the fakeroot howto for crux... in my bookmarks..06:53
*** pedja has quit IRC07:13
Romstercptn, when you do read this one other issue i can't use ccze in prt-get.conf07:26
Romstermakecommand      nice -n10 pkgmk |ccze -A07:28
Romsterdoes weird stuff07:28
Romster<good>checking</good><default> </default><default>build</default><default>07:28
Romsterworks with pkgmk |ccze -A07:28
tilmandoesn't surprise me07:30
tilmanyou cannot expect prt-get to handle that pipe07:31
tilmanprepend it with 'sh' maybe07:31
Romsternever tryed.. but i'l give it a go.07:34
Romster/usr/bin/ccze: /usr/bin/ccze: cannot execute binary file07:36
Romsterwell thats gonna make the plugin i make for ccze to colour gcc output less fun..07:37
cptnwhy don't you use CC=yourscript in pkgmk.conf?07:37
Romsteryeah thats an idea atm it points to a masked gcc for distcc.07:38
cptn"makecommand      nice -n10 pkgmk |ccze -A" is quite wrong BTW07:39
cptnif you run 'prt-get update samba -fr' it'll run nice -n10 pkgmk |ccze -A -fr07:39
Romsterso i found out.07:39
cptnnote that -fr which is appended to ccze07:39
cptnjust put it in a script, and it'll work fine07:39
Romsterah.. ok i'll make a script to do that, i wasn't sure how to do that..07:41
Romsteri'm always doing silly things...07:41
cptn"nice -n10 pkgmk $* |ccze -A"07:41
cptnthis way, the arguments to the script will be passed to pkgmk07:42
*** cptn has quit IRC07:42
Romsterand yes i know the clouring isn't right /yet/ if you do try ccze, i'm gonna attempt to make a module.07:44
Romsterhighlight includes, links, flags etc.07:45
Romsterforgot about $*07:45
Romsternope don't work..07:45
Romsterback to a quick shell wrapper or even a alias maybe..07:45
*** treach has quit IRC08:52
*** treach has joined #crux-devel08:53
*** aon has quit IRC08:55
*** deus_ex has joined #crux-devel08:55
*** deus_ex is now known as pedja08:56
tilmani want to have some unit tests for pkgutils609:10
tilmanwonder whether to mimic the git approach (shell scripts that test stdout and stderr etc)09:11
tilmanto test the library, you'd need c code anyway09:11
*** jjpk has joined #crux-devel09:12
*** aon has joined #crux-devel09:31
tilmanneed guinea pigs!10:03
tilmanor maybe not yet10:04
jjpktilman: what for?10:04
tilmani want to gather some statistics about the package database10:04
tilmanbut i think i'll some more info than i can extract atm10:04
*** jaeger has joined #crux-devel10:28
*** alancio has joined #crux-devel10:46
*** alancio has quit IRC10:58
*** aon has quit IRC11:55
*** aon has joined #crux-devel11:55
*** aon has quit IRC11:56
*** pedja has quit IRC13:02
jjpkmeester alsa. :s14:15
*** sepen has joined #crux-devel14:35
*** sepen has quit IRC15:31
tilmanbleh, i think using tries to store lists of files per package won't work at all16:04
tilmanso sad, it's a neat data structure16:04
*** jjpk has quit IRC16:10
treachI wonder why this whole ports thing is file based, really.16:38
treachI mean, it's easy to sync, and easy to fix with an editor, but from any other point of view, it's incredibly stupid.16:38
treachOr at least it seems so to me.16:39
treachand btw, this goes not only for our ports, freebsd has the same problems, only in spades.16:40
tilmanwhat are you proposing?16:42
tilmanif anything16:42
treachwell, it's all just metadata, isn't it?16:42
tilmandon't think you can call build() metadata :P16:43
treachI know it's a really wild idea, and probably not feasable, but putting it all in a database instead would make a lot more sense.16:43
treachwhy couldn't you make pkgmk fetch the buildinfo from a db?16:43
tilmanthat way i wouldn't have to waste 44MB for ports, either16:44
treachyeah, it would be much more effective.16:44
tilmantreach: could make it so that it uses $PWD/{Pkgfile,.md5sum,.footprint} _if_present_16:44
tilmanso you could still override stuff16:44
tilmanknow what i mean?16:45
treachno, not really.16:45
tilmani forgot to mention idea 116:45
tilman1.) keep the dir hierachy in /usr/ports/repo/portname, but with empty directories o_O16:45
tilmanso old-time crux users aren't confused and can keep their sanity16:45
tilmanbut pkgmk just slurps data from that db16:46
treachbah, sanity is overrated. :P16:46
tilman*but* if /usr/ports/foo/bar/Pkgfile is present, use that instead16:46
tilmanand if /usr/ports/foo/bar/.md5sum is present, use that instead16:46
treachkind of integrating the overlaying in prt-get within pkgmk, you mean?16:47
tilmanum, maybe16:48
tilmanyeah, probably16:48
treachI think it's an interesting approach, at least.16:49
treachbecause wasting space and inodes on heaps of metadata doesn't make much sense to me.16:49
tilmanthink of the chicken and egg problem though16:49
tilmanpkgmk shouldn't *require* sqlite3 or the local ports db16:50
tilman(not too hard since it's a shell script)16:50
tilmantreach: but i agree, it might be a good idea16:50
treachok, maybe it should have a different nanme then.16:50
tilmanneed sleep now. lectures again tomorrow \o/16:51
treachbecause I don't see anything wrong with requiring sqlite (or whatever backend is used), as long as it's for such a purpose16:51
treachgood night tilman.16:51
tilmanports using sqlite seems ok16:51
treachthe problem, as cptn pointed out is syncing.16:52
treachSince the current approach is entirely file based.16:52
*** Romster has quit IRC18:40
*** Romster has joined #crux-devel19:09
*** treach_ has joined #crux-devel19:15
*** treach has quit IRC19:16
*** treach_ is now known as treach19:16
*** treach has quit IRC21:08

Generated by 2.11.0 by Marius Gedminas - find it at!