IRC Logs for #crux-devel Sunday, 2019-03-03

frinnst huh, nvidia security fixes00:01
frinnstfun times00:01
frinnstseems we are up to date00:02
jaegerwhat's with the packages.boot file?00:03
jaeger(in the iso tree)00:04
frinnstI didnt touch that, did I?00:04
jaegeryou added it :)00:04
jaegerin the commit that removed xorg-mkfontdir from packages.xorg00:04
jaegerI just wanted to check if that was intended before removing it, hehe00:04
jaegersounds like not00:04
frinnstah, yeah. I think i started to look around on how to handle the bootloader selection thingy a few months ago00:05
frinnstmust have pulled that branch in somehow00:06
frinnstcleaned up00:06
jaegeradding a setup-helper stanza for it00:10
jaeger(mkfontscale/mkfontdir change)00:10
*** groovy2shoes has joined #crux-devel01:10
ryuojaeger: what does it take to get contrib access?01:41
jaegerisn't there a wiki page about contrib?05:44
ryuomaybe. i guess i'll take a look.05:44
jaegeryeah, found it.
jaegeryou've been around plenty of time, have you ever hosted a repo somewhere?05:45
ryuonot yet, but i can figure it out pretty easily i'm sure.05:46
ryuoi manage a web server already.05:46
ryuoi've already contributed to crux programs.05:47
ryuoi think i'd like to return to using CRUX, but focus on containers.05:48
ryuoit's hard to find any projects worth using C for.05:48
ryuoCRUX has plenty.05:48
ryuook, so first stop... getting a ports repo or so.05:49
ryuoRomster was wanting byobu packaged... that seems a reasonable place to start.05:50
ryuoi enabled it on his shell on our shared server and he seems to be hooked now... heh05:51
Romsterwe need more maintainers :)07:33
jaegerI wonder, do we need bin86 in core anymore now that lilo has moved to opt?07:53
jaegergoing to sleep, good night07:53
*** pedja has joined #crux-devel10:29
jueFYI, I'll be off until Thursday, take care :)10:46
*** jue has quit IRC10:46
*** onodera has joined #crux-devel14:39
jaegerok, catch you later15:41
ryuoACTION delivers jaeger's order of poke balls.18:28
ryuoNow you really can catch them later.18:29
frinnstjaeger: doubt its needed. I dont have it installed and i've rebuilt everything countless times :-)18:33
ryuofrinnst: i saw that db got patched for some gcc issue.18:44
ryuofrinnst: db-32 also needs patching. it failed during one of Romster's build tests.18:44
ryuolet's see what the diffs are...18:45
jaegeroops. somehow I've screwed up dependency addition on the test ISO18:48
ryuooh. already editted by Romster.18:48
jaegerprobably something dumb like overwriting the list file18:48
jaegerryuo: what do you mostly use now?18:51
ryuojaeger: use? in what sense?18:52
ryuojaeger: Linux wise?18:52
jaegerin terms of linux. Sorry, I should have specified to which comment I was referring/responding. You said you might like to return to crux18:52
ryuoMostly Ubuntu derived hosts right now.18:53
jaegercool, was just curious18:53
ryuobut there's a lot that can be ported to CRUX like lxc.18:53
ryuoit needs kernel support but it's rather easy to start containers for major distros.18:54
jaegerWhen I worked for the brain research institute we used ubuntu server for everything... now at F5 it's all centos18:54
ryuoCRUX would need to be packaged for it.18:54
ryuoi heard some say that ubuntu server is taking off due to familiarity with ubuntu desktop for their developers.18:54
ryuono clue myself.18:55
jaegerNo idea here. This was years ago... but it worked fine for me18:55
ryuobut, i guess it makes sense. it can arguably lower deployment costs.18:55
ryuoand apparently canonical now offers 10 year support, but only for paying customers.18:56
ryuoLTS is otherwise uneffected.18:56
ryuostill capped at 5.18:56
jaegerwhich is generally plenty in my experience18:56
ryuoyea... i use it for ease of administration.18:57
ryuono offense to CRUX, but it's more work to maintain a series of them.18:57
ryuobut, a container is easy to start up. no kernel to configure.18:57
ryuoi was thinking of bringing some stuff that makes CRUX easier to get into while retaining the old approach for those that still care for it.18:58
ryuomaybe it can be in core or opt at some point.18:58
ryuoa system for managing kernels for people that don't want to mess with it would be a good addition i think.18:59
jaegerinteresting... one of the recent changes to prt-get seems to have broken the ISO build19:50
ryuoit might be one of the patches i did to "fix" the compile warnings.19:51
jaegerI've been chasing this for a while now thinking I fucked up the setup script on my new test ISO but it's exactly the same as setup on the test0 one19:51
jaegerso I figured it had to be a problem with some file setup uses or a package19:52
jaegerturns out it's setup.dependencies19:52
jaegerbefore, a line looked like this:19:52
jaegeracl: attr acl19:52
ryuoOh wait. It might be related to the warning i fixed that involved this line:19:52
jaegernow, it's:19:52
jaegeracl: attr19:52
jaegernewlines are getting added19:52
jaegerwhich is fine, I can work around that in the Makefile or script... just annoys me that it took me so long to track this down19:53
ryuo+           InstallTransaction::InstallResult result __attribute__((unused)) = trans.calcDependencies();19:53
ryuonone of the patching i did involved direct use of output...19:54
ryuoisatty patch in git does change how IO is performed.19:55
ryuoi wonder if that's related.19:55
ryuoi mean, it only outputs newlines to ttys now19:56
ryuojaeger: what generates setup.dependencies?19:56
jaegerThey're generated in the Makefile by something like this (for opt packages):20:01
jaegerfor P in $(cat packages.opt); do echo -n "${P}: "; prt-get --no-std-config --config-append="prtdir /usr/ports/core" --config-append="prtdir /usr/ports/opt" --config-append="prtdir /usr/ports/xorg" quickdep ${P}; done20:01
ryuoi'll see if i can do a backtrace.20:02
ryuofind which commit did it.20:02
jaegerdefinitely seems like an IO/TTY handling thing20:02
jaegerbecause if I run the above it works... but if I pipe it through less it results in the extra newlines20:02
ryuoi'll take a look.20:02
jaegeror if I generate it using the Makefile20:02
ryuoand it incidently applies to quickdep20:03
ryuolmao. they broke one of the uses of it.20:03
ryuothe logic...20:03
ryuocout << *it << (tty ? " " : "\n");20:03
ryuoit needs to be reverted.20:04
ryuoit may fix one person's problems but makes another.20:04
ryuoi mean, it's either that or patch how the Makefile works.20:05
jaegerwho's they?20:05
ryuoi guess whoever filed the original bug report.20:06
jaegeroh, it's frinnst :D20:06
jaegeryeah, already there20:07
ryuofrinnst is just the applier20:07
jaegerNot sure how I feel about the change but I can easily modify the Makefile to deal with it20:07
ryuoi have an idea...20:07
ryuocould modify it to treat output to a file or tty the same, but treat pipes special.20:08
ryuofstat can be used to know what's attached to stdout, iirc.20:08
ryuoi'll do a quick test to see how programs will behave.20:09
jaegerThe more I think about it the more I think FS#593 was maybe a mistake20:14
ryuoperhaps so.20:14
jaeger'prt-get quickdep' functionality changing just so someone can be a bit lazier seems bad (tm)20:14
ryuoi was just thinking of a workaround.20:14
jaegerI'm willing to have my mind changed if you guys disagree20:14
jaegerAnyway, building another test ISO with workaround20:15
ryuoseems the idea worked.20:15
ryuoecho `./test`20:15
ryuotty: no file: no pipe: yes20:15
ryuotty: yes20:16
ryuofile: no20:16
ryuopipe: no20:16
ryuofor ./test20:16
ryuotty: no20:16
ryuofile: yes20:16
ryuopipe: no20:16
ryuofor ./test > x20:16
ryuoso, as i thought.20:16
ryuoif it was intended to be kept, it could just make a special case for pipes, but then that breaks pipelines if prt-get quickdep isn't at the end.20:16
*** onodera has quit IRC20:17
jaegerso this change sat in the git master branch for a long time before a release, I take it20:19
jaegerbased on the dates in the flyspray ticket20:23
ryuonearly 2 years20:26
ryuoit should probably be reverted...20:27
ryuoideally prt-get would be able to take options to enable such behavior.20:27
jaegerYeah, I don't like this change and would prefer to see it reverted. If someone actually NEEDS the functionality a switch/option would be my preference20:28
ryuoit sat there for 2 years and no one else has been using it.20:28
ryuoprobably best to just revert it old behavior.20:28
jaegerAgreed. I can't imagine it was important if it sat that long20:28
jaegeralso, *reported* 8+ years ago20:29
jaegerfrinnst: any objections?20:29
ryuoare they even still a member of crux?20:29
jaegerI know jue is gone for now20:29
jaegerNo idea who reported it, it says anonymous20:29
ryuoi doubt jue even knows about it.20:29
jaegerThe comment by fun is a much better way to go about it, in my opinion20:34
ryuoi suppose so.20:34
ryuojaeger: question though. why have you been hiding your work so far on a new pkgutils?20:35
jaegerbecause I don't want to publish crappy WIP code20:35
ryuoI see.20:36
ryuoFair enough.20:36
jaegerAnd because it's not been a priority for anyone other than me20:36
ryuowell, when you're ready, i can get involved to help push it towards something better perhaps.20:36
ryuoi've been itching to write some PM code anyway.20:37
jaegerMainly I just need to comment the code and add the db file locking20:37
ryuoi see.20:38
jaegerBeen too busy with random little stuff to get to it so far20:38
jaegerlike the ISO tests :D20:38
ryuoi'd like to go through it when you're ready. i may have some redesigns to do, to make it more library friendly.20:39
ryuoideally pkgadd and such would be a shell and the real logic in the library.20:39
ryuoallowing users to write other kinds of frontends20:39
ryuothat's what i set out to do before, but then i heard you were doing something similar.20:40
ryuoso, i stopped until i could see your work.20:40
ryuoit seemed wasteful to duplicate the effort.20:40
jaegerAnother question I haven't really asked is if folks in general would be willing to move to a new pkgutils if I or you or someone rewrote it20:40
jaegertek__ also had done some work in this area in the past, I think20:40
ryuoprobably, if we can preserve existing behavior.20:41
jaegerSorry, not intending to keep you from doing anything20:41
ryuooriginally that's all i intended to do to start with.20:41
ryuoreplicate the existing system in a library format.20:41
jaegerThat's more or less what I intended, except my goal was replace c++ with c. A library would be nice too20:42
ryuoindeed. C is better for libraries.20:42
ryuoC++ is harder to bind.20:42
ryuoit's great for applications but sucks for libraries.20:42
jaegerI just don't enjoy reading it or using it20:43
ryuoi don't mind C++, but it has legendary complexity for a reason.20:43
jaegeralso, for this purpose C is turning out to be faster. I don't know if that's a general rule but I would guess so based on C++ being more featureful, etc.20:43
ryuoprobably for the best to integrate prt-get with the whole shebang.20:45
ryuobut to do that... we're going to need to use something other than getopt.20:45
ryuogetopt sucks beyond single use applications.20:45
ryuois popt in core?20:45
jaegerI'm using getopt so far... I think it  could be made to do what we want20:45
ryuojaeger: getopt's problem is that it uses global state...20:46
ryuonot sure it can be reset after the first use.20:46
ryuolike, if we had pkgadd in the same library prt-get uses...20:46
ryuocan prt-get repeatedly invoke that functionality rather than using fork+exec?20:47
ryuowhich has its own overhead.20:47
ryuoor... does pkgmk install the files itself?20:47
ryuonot sure.20:47
jaegerpkgadd does the installation20:47
ryuowho calls it?20:47
ryuoprt-get or pkgmk?20:47
jaegerprt-get, probably20:47
ryuoif so, then we could have it run the library directly20:48
ryuowhich might pose problems...20:48
ryuooh, i know.20:48
ryuowe can make the getopt crap into a frontend problem.20:48
ryuoit'll just map to library flags.20:48
jaegerin my current code I have some getopt logic in two places20:48
jaegerin main and then in pkginfo20:49
jaegerin main it only handles --help and --version20:49
ryuoi see.20:49
ryuowell, i imagine it'll need a lot of rework to be a library.20:50
ryuoif nothing else i can repack it into an API that's more suitable.20:50
ryuothat also makes it possible to separate backend and frontend20:50
ryuoprt-get would also have more options for implementation if it has direct access to the same code.20:51
ryuoand i could revise revdep to have less waste.20:52
ryuowhile we're at it, i might as well write a vector API. it'll be useful for more than just pkgutils.20:53
ryuobasically a dynamic array, using C++ name.20:53
jaegergimme a few, I'll post what I have... because if you're gonna end up rewriting it all anyway, no reason for me to spend a ton of time on it21:05
frinnstgo nuts21:08
ryuojaeger: thanks.21:15
ryuojaeger: i've been wanting a project to find new uses for C11.21:15
ryuowhat few new features it has.21:16
ryuoalso taking some ideas i've gotten from higher level languages.21:16
ryuousing some extern inline functions to boot.21:16
ryuoallows me to make an inline function that has a link time form as well.21:16
ryuouseful for libraries.21:17
ryuonot bad.21:18
jaegerIt's very much a work in progress. only pkginfo implemented so far and it still needs DB locking21:18
ryuojaeger: if you'd like, we can coordinate on github, or i can work on this project alone.21:19
jaegeralso, I've not put a lot of thought into which functions go into pkgutil.{c,h} vs. their own {c,h} file yet21:19
jaegerJust sort of starting and moving as I go21:20
ryuofair enough. i think that becomes more obvious if you design with a library in mind though...21:20
jaegeryes, and when more than just one util is implemented21:20
ryuoi also studied libpacman when i worked on frugalware.21:20
jaegerI'm sure there will be some overlap and those things will make sense to abstract21:20
ryuoi was thinking of using a similar design, but modified to fit what crux currently does.21:20
ryuowhich means a lot can be purged.21:21
ryuoonly prt-get deals in depends for example.21:21
jaegeryeah. My intent as mentioned was to first implement pkgutils in C, ignoring prt-get21:21
jaegerthen to make my own prt-get replacement (hence cpkg)21:21
ryuoAs for your new age database...21:21
ryuoi was thinking something that can still be processed by AWK would be a good idea.21:22
jaegerNo immediate plan to change the DB format21:22
ryuoDebian's format is pretty nice and easily concatenable.21:22
jaegerMaybe in the future, but for now, in order to preserve the functionality of the original tools, I was going to leave it alone and add my own metadata files elsewhere21:22
ryuoyes... i was thinking of adding explicit field names.21:22
ryuoso it can be easily expanded as needed.21:23
ryuono more positional crap.21:23
ryuosomething like.21:23
ryuoName: foo21:23
ryuoVersion: 0.321:23
ryuoFile: ...21:23
ryuo<empty line>21:23
ryuothat would be one record21:23
ryuorepeated fields could be used to build an array type.21:24
jaegerfrinnst: ok21:24
ryuomore verbose but a lot more flexible format.21:24
ryuoit could then be expanded as we discovered need.21:25
ryuobut would require a deprecation of the old database file.21:25
ryuoeither way, not something to deal with immediately.21:25
jaegerYeah. My intent at first was not to break any tools that use the database as it is now21:25
ryuoi think it best to discuss the matter with the general user base.21:25
ryuowe should pick a new format that most can live with.21:25
ryuoDebian's approach is a major improvement and still parseable by AWK.21:26
ryuoif people really want to bypass pkgutils still.21:26
ryuosqlite may be faster but it's also binary only...21:26
ryuoi'd rather use a text file you can fix by hand if it gets slightly corrupted.21:26
jaegersqlite's also easy to corrupt without meaning to, unfortunately21:27
jaegerI like using a text-based format, even if it's a complicated one like json, as long as it's parseable21:28
ryuojson... easier to parse for languages, but tools like AWK can't comprehend it.21:28
jaegeryeah. If we went that direction I'd push for tools that use language bindings or API21:29
ryuojaeger: you familiar with function pointers? i plan on using those a lot.21:30
ryuoi think they're the most useful thing in C.21:30
jaegerI haven't used them extensively but I know the general gist. You can see I used one in main21:31
ryuojaeger: how do you think we should handle malloc failures? most languages crash the program if they can't allocate memory. not sure if that's a good idea in a critical system component.21:33
jaegerhow big a problem is that in C? I would expect that you could try a malloc and check the pointer it returns, then error out if something went wrong, it's still NULL, etc.... - are you saying that's not how it usually works? Does C not handle that gracefully?21:34
ryuojaeger: well... basically there's two approaches to it in C.21:34
ryuojaeger: malloc rarely fails unless you request ridicilous amounts.21:34
ryuoand, most programs assume malloc succeeds.21:35
ryuoso, if it fails, they get a segfault.21:35
ryuoif you check for it and abort if it does, not much difference.21:35
ryuoif it happens at a bad time, it could leave your system in an inconsistent state in the case of pkgutils21:35
jaegerWell, we're in control here, not using some third-party programs of which  we don't know the inner workings21:36
ryuoif we handle it everywhere we allocate memory, we can at least exit gracefully.21:36
jaegerMaybe I'm missing the point21:36
ryuoit's a choice.21:36
jaegerIsn't it up to the software author to handle it properly? If you don't, well, that'21:36
jaegers  doing it wrong in my opinion21:36
ryuoindeed, but most higher level languages just crash and burn.21:37
ryuoThat's the default in C++.21:37
ryuoit throws an exception most don't catch.21:37
ryuowhatever your program was working on grinds to a halt21:37
jaegerI don't intend to use anything but C for this21:37
ryuobut the C default isn't much better. if you don't check for NULL, same outcome for all effective purposes.21:38
jaegerright... so maybe I should ask this, then: why would you ever intentionally NOT check?21:38
jaegerobvious you might forget to or something, but that would be a bug21:39
ryuoit can get pretty hairy.21:39
ryuoevery malloc allocation, no matter how small, would need to be checked.21:39
ryuoi've tried it before.21:39
ryuoit's easier to ignore it.21:39
ryuoit's very rarely an issue.21:40
ryuohell, it's the same principle glib software uses.21:40
ryuog_malloc is the usual allocation method they use.21:40
jaegerI make sure to check with valgrind21:40
jaegerfor what that's worth21:40
ryuothat's fine. this isn't about memory errors, but how we deal with a rare failure.21:40
jaegerOK, I'm still missing your point21:41
jaegerIf there is a bug causing failures, we fix it21:41
ryuojaeger: well.. it's an exceptional condition more than anything.21:41
ryuomalloc only fails when there's not enough memory to fill a request.21:41
ryuonot really a bug... in the end most just abort in some way.21:42
ryuojust it's usually ungracefully.21:42
ryuoglib2 for example, the usual allocation API, just aborts if it fails.21:42
ryuobut it provides ones that act like regular malloc.21:43
jaegerOh, so you're talking about how to deal with it if we use something like glib rather than manage it ourselves?21:43
jaegeralso a thing I wasn't planning to do, for what that's worth21:43
jaegerjust to keep dependencies minimal21:43
ryuoNot at all. i'm using it as an example of how others deal with malloc failure.21:43
ryuomalloc works 99.999% of the time unless you have a giant leak problem or have massive allocations.21:44
ryuoit's very rare for it to fail.21:44
ryuoso, should it still be handled? it's somewhat like a situation where...21:45
ryuothe block device you write down suddenly fails21:45
ryuoof course, we can't do anything if that happens.21:45
ryuoshould we think of an allocation failure the same way?21:46
jaegerSo are you asking if it's worth checking the result of your malloc when most of the time (tm) it'll be fine?21:46
ryuoyes, plus the best we can do is exit gracefully.21:46
ryuowhatever that means21:46
jaegerthen I ask again, why would you ever intentionally NOT check? I see no benefit21:46
ryuopeople are lazy.21:47
ryuothat's all I can say.21:47
jaegerYes, definitely. How does that tie into what we're discussing? I'm seriously not trying to be an asshole, I can't figure out the point you're making21:47
jaegerIf you're saying I'm lazy and my code sucks, that's fine, heh. I'm not a professional programmer21:48
ryuojaeger: No, i wasn't speaking about your code at all...21:48
jaegerI didn't really think you were, but I'm confused21:48
ryuoi was thinking implementation/design wise...21:49
ryuois this something we want to try to support? graceful exit from rare but usually catatrophic failures?21:49
jaegerRight... so you're saying people are lazy, which can totally be true... but there are no unknowns here... so far it's you and me... so we can choose not to be lazy, right?21:49
ryuoRAM availability largely varies...21:50
jaegerPersonally I think if you want to produce quality software, no matter how simple or complicated, you explicitly try to avoid being lazy21:50
ryuowell then, i guess we should just check it.21:50
jaegeryeah, that's true, but we're talking about quite small amounts of data here for the most part, I think21:50
ryuowe'll figure out how to do a graceful exit later.21:50
ryuo... uh...21:51
ryuoi just looked at pkgutil.c21:51
ryuo... malloc is being used incorrectly.21:51
ryuopkgname = malloc(sizeof(char *));21:51
ryuothis will be an allocation of 8 bytes on x86-6421:51
ryuowhich can handle a string of up to 7 characters...21:52
ryuobut anything more will get a buffer overfow21:52
ryuousually to allocate a string, you do this:21:52
ryuochar* p = malloc(sizeof(char) * len)21:52
ryuosizeof the underlying type, not the pointer.21:53
ryuosome people use this as shorthand:21:53
ryuop = malloc(sizeof(*p) * len)21:53
ryuojaeger: not bad, though. i've seen a lot worse C before.21:55
jaegerI've no doubt there are mistakes21:55
jaegerI'm not an expert by any means21:56
ryuome either, i just know a lot more about correctly using C.21:56
ryuoit's amusing how hard it can be to get your integer calculations exactly right.21:56
ryuothat's something i've learned from C, and why i frequently desire to put repetive and error prone bits in a reusable form.21:57
jaegerI'll fix those mallocs next time I work on it22:06
ryuojaeger: ok. i'm going to work on designing a library then.22:10
ryuohm. i think i'll start with utility functions.22:13
ryuojaeger, frinnst: what i have so far. i'm going to design the API before i write an implementation of much.23:30
ryuoright now just moved the macros used to define certain defaults to a setup where they can be changed at runtime if the client wants to.23:31

Generated by 2.14.0 by Marius Gedminas - find it at!