IRC Logs for #crux-devel Tuesday, 2018-02-20

j_vjaeger, with those builds that vary on different machines... are you using any kind of build isolation tool, like docker/qemu/chroots?00:19
fun_The footprint can be mangled by overriding a pkgmk function from go/Pkgfile, like it is done for kernel modules.01:37
fun_And the footprint helps whe  searching for files on not installed ports01:38
jaegerj_v: varied results aren't limited to "unsafe" environments, it can happen in docker as well as clean VMs02:18
pitillojaeger: different umask on these machines?02:24
jaegernope. that was the first thing I checked :)02:45
j_vok, just curious. one other thing, is it random which ports it happens to, only certian one(s)?03:05
j_vs/, only/, or only/03:05
jaegerIt's only go.03:13
j_vdo you mind me looking in to it? i have some ideas what could be causing it, but won't have time to test that out for a few hours, though.03:15
jaegerFeel free03:15
j_vok, thanks. will let you know.03:15
pitilloprobably the most obvious thing to check... but I found overlays over the system umask, for users with custom umask which messed up a bit all things03:19
*** __18VADBHIX has quit IRC03:44
j_vi think that one issue has to do with build systems like that of go and rust and probably some others, that download stuff during the build. it makes code difficult to track and monitor. and worrisome, because who will ever have time to audit something that changes out from under you so often.03:45
*** ___18VADBHIX has joined #crux-devel03:45
j_vi know for sure that rust's cargo package manager is specifically difficult to nail down.03:47
jaegeryeah, I'm not a big fan of systems like that04:00
j_vunfortunately, getting the go build to turn verbose is a bit of black magic that i haven't found the info for, but i don't give up easily04:01
j_vhas anyone concidered using gcc's go frontend instead of google's go?04:04
j_vi've haven't messed with go much yet, so i'm not sure how compatible they are to each other04:06
*** j_v has quit IRC06:05
*** j_v has joined #crux-devel06:07
*** chinarulezzz_ has quit IRC07:01
*** chinarulezzz_ has joined #crux-devel10:04
*** chinarulezzz_ has quit IRC10:13
*** chinarulezzz_ has joined #crux-devel10:15
*** Workster has quit IRC13:29
*** opuvxcephstsmirq has joined #crux-devel13:29
*** Workster has joined #crux-devel13:29
pedjadidn't gcc already try that with gcj, or whatever it's java frontend was until 7.x?13:35
pedjaor is Google behind gcc go frontend?13:37
pedjaindeed they are. gcc8 will have a full go-1.10 implementation. nice13:46
j_vman, do i feel silly, i've built go several times, various vm's, containers, etc, etc... but i forget to pull the recent changes so i was building the previous one.14:23
j_vthough, on a side note, i think the issue is where we're doing: 'cp -r $SRC/go $PKG/usr/lib'14:24
j_vi think usr/lib/go/obj/go-build looks like it needs to trimmed out14:25
j_vactually, i think whole usr/lib/go/obj directory is a build artifact14:41
jaegerCould be, yeah14:56
j_vit's weird, i'm looking at arch's PKGBUILD, and i don't see anything that would remove that dir, but when i look at their file list for the package, i don't see any of that dir15:03
jaegerhrmm, odd15:07
pedjafrom Gentoo ebuild 'cp -R api bin doc lib pkg misc src test "${ED}"/usr/lib/go'15:07
j_vi'm still looking... might have to do with the fact that they depend on go itself, instead of bootstrapping it like we do... so the go compiler might be storing its intermediate files differently15:08
j_vi think bootstrapping makes a lot more sense (i hate circular dependencies) but perhaps as a consequence of bootstrapping go like we do, we need to remove that directory, perhaps before copying the directory (saves moving them and removing them)15:11
*** dlcusa has quit IRC15:14
j_vanother directory that should probably go is the usr/lib/go/pkg/bootstrap15:14
j_vs/go/go away/15:14
j_vsorry, i should look at what i've typed before i hit enter15:15
pedjawhy? I never do15:15
pedjathen again, I am a slooow typist, so :)15:16
jaegerRunning a manual build now to see what artifacts it generates15:16
j_vpedja: well, perhaps i try to type too fast15:17
jaegerI type pretty quickly but that's probably balanced out by the number of typos I make. :D15:17
jaegerThere's a README in the go-build dir that says: This directory holds cached build artifacts from the Go build system.15:19
j_vi think there are more directories that could/need to be pruned, but i think that one will at least get a start at making the build predictable15:20
jaegerMaybe so. Worth a try, at least. Will have to test it for a little while to make sure15:21
j_vi suggest looking at arch's and gentoo's build scripts... they are doing something that generates so libraries in usr/lib/go/pkg/linux_amd64* that i don't see in ours15:23
j_vs/ so / static /15:24
j_vthe difference for arch's PKGBUILD seems to where they run 'go install -v -buildmode=shared std' and 'go install -v -race std'15:26
j_vi can dig deeper if you like, but i think that's a good start15:27
jaegerThose aren't in the official compile docs and a quick search seems to indicate that needing to do that is a bug15:29
jaeger for example15:29
pedjabtw, jaeger, have you rolled any 3.4 iso yet? I'd like to check if consolekit2 and few others compile with gcc715:32
jaegerI've started to but keep getting sidetracked with other stuff. I'll try to get one made for real (tm) in the next couple days15:33
pedjano worries, I was just curious :)15:33
pedjaI have a feeling spidermonkey will be a PITA. Linux desktop is such a mess15:35
pedja'new' spidermonkey, not the opt one15:36
j_vwhat if you use the same version for the binary bootstrapper? i note that the build instructions only list using 1.4 if you are bootstrapping all the way from C sources, since that was the last version to be written in C15:37
jaegerI doubt that it makes much of a difference but it wouldn't be hard to test15:40
j_vheh, I know it's a long shot, but trying it now just to see what happens15:42
j_vdamn, still getting that build cache... maybe it's a bug15:43
*** nomius has quit IRC15:46
jaegeryeah.... different list of files but still there15:46
j_vi wonder if it's possible to make to bootstrap go to put it's temporary files elsewhere15:50
j_valso, that bug report you linked to was 2-3 years old, so much could have changed since then. i'm under the impression that golang is a moving target (though, maybe i'm confusing go with rust)15:52
jaegerCould be. I'm certainly no expert, I'm just the maintainer because someone better hadn't done it first :P15:56
j_vthere is a GOCACHE environment variable introduced with this release... the cache stuff is apparently new. i testing a build with that variable set to ${SRC}/tmp15:58
jaegerI do see the go-build dir in the arch package but without any contents15:58
j_vafter setting GOCACHE in the Pkgfile, i can already see that the build is using it at least16:00
j_vdamn, very frustrating. but at least it's a new feature, so this it seems feasible that quirks may exist or adjustments will need to be made16:01
pedjaafter removing go-build from go, I rebuilt docker, packer and hugo. no issues, afaict16:02
pedjayou are probably correct that it is some kind of intermediate build artefact16:03
j_vyeah, need to take a break from this for a bit... too close to it now that i need to back up from it to see it clearly again16:04
pedjaoff topic, i see that 20+ US states are fighting against NN repeal. how likely are they to succeed?16:07
j_vvery good question... my opinion, not very. but i'm a  pessimist/cynic when it comes anything government related.16:09
pedja'the optimist sees the world as it should be. the pessimist sees the world as it is'16:13
pedja(apologies to Bierce for quoting from memory)16:14
j_vand the cynic sees both, and realizes the futility of wishful thinking16:14
pedjanice :)16:15
j_vi never should have taken philosophy in college... life would be so much simpler... i can argue myself around a corner16:16
*** john_cephalopoda has joined #crux-devel16:26
j_vgo's idea of a verbose build is pathetic17:10
j_vok, progress: using 'export GOCACHE="off"' in the go Pkfile stops it from caching it's intermediate files in the pkg dir17:35
j_vat first, i thought the build slowed down, but some weird reason it changed the verbosity by lowering it even more17:37
j_vbut the resultant build only has an empty usr/lib/go/pkg/obj/go-build directory17:39
j_vthere are a few other files gone from usr/lib/go/pkg/bootstrap/pkg/linux_amd64/bootstrap/, but i'm not sure that whole heir sitting at usr/lib/go/pkg/bootstrap is necessary anyways17:41
j_vi miss the days when 'make' was the king of build automation. it may not be perfect, but most of it's obscurity was only compounded by autotools...17:45
j_vnow days, every mother's cousin is creating a new build tool, or their own. waf, meson, scons, cmake, qmake, the list gets longer by every other day17:46
jaegerHere's another one that happens on rare occasions:18:42
jaegercp: error reading '/usr/ports/work/go/src/go/pkg/linux_amd64/mime': Is a directory18:42
jaegerrm: cannot remove '/usr/ports/work/go/src/go/pkg/linux_amd64/mime': Is a directory18:42
jaeger=======> ERROR: Building '/usr/ports/pkg/go#1.10-1.pkg.tar.xz' failed.18:42
*** onodera has joined #crux-devel18:53
jaegerso far removing the build cache doesn't seem to be causing any problems in my tests19:02
j_vthat other error... on first blush, that looks to me like a symlink/directory type problem, though not sure. that 'error reading' part has me thinking maybe something of that ilk19:16
jaegerthe file about which it complains also varies when that happens :D19:16
jaegerwhy? Because fuck you, packagers, that's why!19:17
j_vi totally get your frustration19:17
j_vdoes happen during the make.bash call or while cp'ing to $PKG?19:18
j_vand does it happen on all machines? because another issue could be bad sectors, couldn't it?19:19
jaegerIt's so rare I don't have useful stats on it. If it happens again I'll capture the output19:20
jaegerThat particular build was in a VM19:21
*** Henschi has joined #crux-devel19:22
j_vcould be VM issue... i know right now i'm tracking down an issue with using my container issue i'm having in a VM. i'm fairly sure it's a kernel config issue... but tracking it down is monotonous19:22
jaegerCould be. There are too many factors to make it simple to narrow19:23
j_vyeah... a pain19:23
jaeger+ cp -r /usr/ports/work/go/src/go/api /usr/ports/work/go/src/go/bin /usr/ports/work/go/src/go/doc /usr/ports/work/go/src/go/lib /usr/ports/work/go/src/go/misc /usr/ports/work/go/src/go/pkg /usr/ports/work/go/src/go/src /usr/ports/work/go/src/go/test /usr/ports/work/go/pkg/usr/lib/go/19:24
jaegercp: error reading '/usr/ports/work/go/src/go/pkg/linux_amd64/vendor/golang_org/x/net': Is a directory19:24
jaegerrm: cannot remove '/usr/ports/work/go/src/go/pkg/linux_amd64/vendor/golang_org/x/net': Is a directory19:24
jaeger=======> ERROR: Building '/usr/ports/pkg/go#1.10-1.pkg.tar.xz' failed.19:24
jaegerso not in the build phase but in the package phase19:25
jaegerI do see an old ext4 error in the logs on that VM, maybe it needs a filesystem check19:29
j_vmight be worth a check before getting too far in depth looking for something else19:29
jaegerwon't hurt anything, at least19:30
jaegerbuilt 10 times in a row without issues on the laptop, heh19:41
jaeger[    9.371170] EXT4-fs (sda2): Delayed block allocation failed for inode 529733 at logical offset 2 with max blocks 1 with error 12119:44
jaeger[    9.371369] EXT4-fs (sda2): This should not happen!! Data will be lost19:44
jaegergood times19:44
jaegerfsck doesn't find anything wrong with it, though19:44
jaegergoing to try a different disk controller on the VM19:46
j_vhow old is the kernel? maybe it's a known bug in that version19:46
*** Workster has quit IRC19:48
j_vseeing as it's a ex4fs error and not an ata error, makes me suspect fs driver issue rather than hardware, but i'm no expert on that19:48
*** Workster has joined #crux-devel19:48
jaegerYeah, that was just an easy thing to rule out. Same error with a different controller19:51
jaegerPerhaps it's time to rebuild that VM, it's one of my oldest ones19:55
jaegerwas created back on esxi 5.1, I think, moved all the way through to 6.520:01
frinnstwe have a few old debian vms built on esx 3.5 still running :-)20:32
frinnsthave you powered it off? we noticed some odd performance issues with a couple of VMs recently. hotadding cpu's apparantly can cause issues. numa crap i guess20:33
frinnstthe Vms have rebooted numerous times but never powered off20:34
frinnstnever had any disk issues tho20:34
jaegerI did, yeah. I even moved it between storage locations as a test20:35
jaegerwell, after a rebuild (backup and restore with clonezilla) it doesn't throw that error anymore20:40
jaegertime to rebuild go a bunch of times and see how it behaves20:45
*** Henschi has quit IRC21:19
*** onodera has quit IRC21:30
*** frinnst has quit IRC21:45
*** frinnst has joined #crux-devel21:51
*** dlcusa has joined #crux-devel22:12
jaegeron the laptop, 25 go builds resulted in 25 successes22:19
jaegerstill going on the VM and in a container22:19
j_vsounds like a good start22:22
jaeger$ grep failed builds.log | wc -l22:22
jaeger$ grep succeeded builds.log | wc -l22:23
jaegerNot looking reat on the VM so far :D22:23
j_vsame failures as before?22:23
jaegergrr, I wish sprunge weren't still broken22:24
j_vtry http://ix.io22:24
jaegerI'll give a look, thanks22:24
jaegerGotta love that inconsistency, though22:26
jaegerin the container, 9 successful so far, 0 failed22:27
jaegerSo while backing up and restoring the VM's filesystem may have fixed that one error it seems like the VM may still be the problem.22:27
jaegerIn the container builds finish without issue, I'll rebuild the VM from scratch22:27
j_vi wish i hadn't misplaced all my old KVM kernel configs, having issues getting a working one... which is strange, i've never had issues like this before. i think i'm over thinking it. going to take another break and come back at it fresh22:29
j_vi'll be afk a while... going to catch up on watching Travellers on Netflix for a few episodes22:30
*** chinarulezzz_ has quit IRC22:47
jaeger16/25 succeeded on the first VM23:02
pedjaI should really get this soundtrack
jaegerIt does have good music23:06
pedjasoundtrack is on Spotify23:10
pedjayet another service I don't have an account on :)23:11
pedjaRe:genesis series also has an excellent soundtrack. their podcasts with the composer talking about how and why were awesome23:15
pedjadamn Canadians and their quality TV :)23:16
*** john_cephalopoda has quit IRC23:30
j_vfor me, Travellers rates up there with Godless, River, and Hinterland... or at least close... Skarsgard in River was phenomenal23:36

Generated by 2.14.0 by Marius Gedminas - find it at!