IRC Logs for #crux-devel Friday, 2019-03-29

ryuothis should be helpful... reading the whole package metadata into memory first shuld make the overall footprint code faster.02:30
ryuowhile i'm at it, i'm also sorting the resulting array.02:31
ryuoit should make the sorting pkgmk does obsolete.02:31
ryuough. found a memory leak in my code. hm.02:40
ryuothere. i just needed to check that the copy operation succeeds or not.02:44
ryuoi'll do more vigorous testing of my error handling once I get something closer to usability02:46
ryuothere we go. patched up the leak.02:57
ryuogoto... it sure helps me implement my own clean up code with ease.02:57
ryuook, now i can try to write the footprint code.02:58
ryuowow. integrating functional approaches into my C code is rather fun.05:34
ryuoand... done. time to test the footprint mode.06:57
ryuo... insane07:03
ryuoit's already outperforming the reference implementation.07:03
ryuoit's performing 47% faster07:04
ryuothis is definitely a keeper07:06
frinnsthow much more ram does it use?09:39
ryuofrinnst: i'll do a check.09:41
ryuobut it's relative to the package size.09:41
ryuodo you have a suitable tarball?09:42
ryuoi was using firefox09:42
ryuoguess i'll try a kernel source09:43
ryuofrinnst: i'm only caching the metadata, so it doesn't amount to as much...09:46
frinnstjust worried about small VPS instances, arm devices etc :-)09:46
ryuo... wow09:47
ryuois it really that high...09:47
ryuolet's see how much is consumed just to read the archive09:50
ryuointeresting. 94M09:51
ryuothat's without actually caching anything.09:52
ryuook... 67M is already in use.09:53
ryuojust from going through the normal read proces09:53
ryuolet's see how much it jumps to when i turn caching back on09:53
ryuoIs this right?09:54
ryuoOnly 2 megabytes?09:54
ryuomaybe address sanitizer is exaggerating09:55
ryuofrinnst: what's the typical RAM of arm devices that CRUX is supported on?09:56
ryuoor lowest anyway09:56
ryuoi was expecting at least 256MB09:56
ryuothere's also the fact that my data structures will use less RAM on ARM too due to smaller pointers09:59
ryuoguess i'll test this new feature out in a container. i can limit its ram usage for testing purposes.10:00
ryuothat should give me some idea of how much RAM it requires.10:00
ryuoi'll turn off address sanitizer for this... it may interfere with the test.10:01
ryuofrinnst: true, but even they usually have 256M or more.10:02
ryuoCRUX was never intended for openwrt style embedded...10:02
ryuoi think it's a good idea to use RAM to speed things up if the RAM usage isn't likely to grow too much as a result.10:03
ryuoright now compiling git in my test container10:04
ryuoi ended up not using hash tables... binary search is fast enough.10:06
ryuoboth run with 128MB...10:20
ryuo(old and new)10:20
ryuoboth fail at 64MB10:20
ryuotrying 96MB10:21
ryuofound their approximate limits10:28
ryuofrinnst: existing pkgutils required about 75MiB of system RAM to avoid being killed by the OOM for the footprint operation10:30
ryuofrinnst: the new one requried 82MiB10:30
ryuoso about 8MiB more usage on amd6410:30
ryuoit might be a good idea to consider CRUX as requiring a minimum of 128MB...10:31
ryuolol... the bulk of that memory usage is from the stat structure...10:36
ryuoi can cut it down if i don't save the whole struct.10:36
ryuo46% less RAM usage if i discard the extraneous metadata10:43
ryuoshould I bother though? i may regret not having it later.10:43
ryuoguess i won't bother for now, but i'll need to field test this sucker when it's more complete10:46
ryuoapparently it's considered rude to ask someone asking for support on a soon to be unsupported flavor of Ubuntu why they haven't upgraded.10:56
ryuoholy cow.11:55
ryuoi reduced the amount of metadata waste and the new version is now on par with the RAM usage of the original...11:55
ryuolet's see if it can go lower.11:56
ryuonope. lol11:56
ryuobut i might as well tune the RAM usage...11:57
ryuopkgutils has to work on every port of CRUX11:57
ryuook. time to do an i386 build test.12:09
ryuoi'll see if i can benefit from reordering my fields.12:09
ryuoseems to use about 2x the file size of the text database12:20
ryuowith the memory cleanup12:21
ryuoi think that's the right balance...12:21
ryuo... seriously? the new pkginfo's installed mode is a tad faster than the reference?12:50
ryuo0.08s for old pkginfo right now... 0.06s for new pkginfo12:52
ryuopkginfo -i is working...13:11
juejaeger: did an update with test2 yesterday, no problems13:26
ryuostarting to think this channel is my blog... lmao13:31
Romstergeez the backlog of mostly ryuo only...13:44
Romsterit's never been this chatty before :D13:44
frinnstryuo: so not a noticable difference then13:50
ryuofrinnst: slightly faster, but not enough for anyone to notice outside benchmarks.13:50
ryuostill, a faster footprint mode can shave some time off builds, albeit not much.13:52
jaegerjue: good to hear, thanks13:59
juejaeger: np, thanks for the ISO, have you seen what I've written to #crux by mistake?14:03
jaegeryeah, I see it. Will add them14:09
*** Romster has joined #crux-devel15:25
*** onodera has joined #crux-devel16:56
*** kyaaa has joined #crux-devel17:44
jueoops, my update of TODO35 breaks our timeline, just as expected ;)17:59
*** onodera has quit IRC18:48
*** prologic_ has joined #crux-devel18:57
*** onodera has joined #crux-devel19:24
jaegeryuck. If you don't beat me to it I'll try to figure it out later20:30
*** onodera has quit IRC23:35

Generated by 2.14.0 by Marius Gedminas - find it at!