IRC Logs for #crux-devel Saturday, 2016-10-01

*** heroux has quit IRC00:07
*** heroux has joined #crux-devel00:08
*** heroux has quit IRC01:38
Romsterwhy don't we have a /etc/profile.d/ setup yet?01:39
Romsterit would be much easier to export variable's than to stick a README stating you need to set this for it to work.01:40
*** heroux has joined #crux-devel01:47
*** onodera has quit IRC01:54
*** _________mavric6 has quit IRC02:44
*** _________mavric6 has joined #crux-devel02:45
*** darfo has joined #crux-devel02:55
darfo"<frinnst2> has anybody tried the download-as patches?"03:00
darfowhere are they and can I check them out on CRUX 3.203:00
Romsteri haven't yet but i was gonna take a look this weekend.03:09
Romsterhmm i think it's on the todo crux 3.3 page?03:09
darfoyeah, FS#923. I was reading that yesterday.03:13
Romsteri don't know if i wanna do that.... it makes more issues.03:13
Romsterya know it's not a problem with the default settings, as each file is saved in $PWD of the pkgfiles location. no file conflicts there.03:14
Romsterthe issue is the mirrors and setting PKGMK_SOURCE_DIR03:14
darfoYes, but I like having all my source files in one directory.03:16
Romsteri have another solution i just do /hash/<filename>/<sha256sum>/03:16
darfoIt makes it easy to share them with other machines via HTTP03:16
darfoAnd to clean out old versions.03:16
darfoI'll take a look at that.03:16
darfoIt seems there isn't a concensus on how to address the problem03:17
Romsteri made them with hardlinks and i got md5 sha256 sums on all of them, i use a few scripts for it. i need to polish it off more.03:17
darfobtw, thanks for restarting the irc logger the other day03:17
Romsterwell we could set the filename int he source but it's still going to break other scripts.03:18
Romsterno problem i just poked prologic i know his hangouts :D03:18
darfoyes, it is a tough problem to fix elegantly. I've been trying to come03:18
Romster like that 2 versions of the same file03:18
darfoup with an elegant solution but no luck03:19
Romsterto share the source with more than one Pkgfile read the .md5sum file find the source in the url/directory03:20
Romsterthat is my take on it no need to change any file names.03:20
darfoaah, that is neat03:20
Romsterupside is if .md5sum changes you don't get the stale youtube-dl in this case you download a fresh copy03:20
Romsterthat also works for any say v1.2.3.tar.xz files matching.03:21
darfothose are ones I'm having problems with03:21
Romsterand it's easy to remove files that none of the .md5sums don't list.03:22
Romsterfor old files that are not used anymore.03:22
darfoooh, I like the sound of that. I end up doing a mass purge03:22
darfoevery six months or so.03:22
Romsterthis might be more complex than renaming files though03:22
darfomaybe. but renaming makes things non-obvious too.03:23
Romsterpurge but keep last distfiles that are still listed in source/.md5sums03:23
darfoIt seems like it's gonna take a lot of changes no matter which solution03:23
Romsterit does the non-obvious part is what i don't like, it breaks other scripts thinking that is the name, sure that can be fixed.03:23
Romsterbut that still does not fix a changed hash on the same name file?03:24
darfoI am going to do an analysis of how many ports I have that use the same source but built differently.03:24
Romstermy method by design will ignore the old file03:24
Romsterwell some i have on with a $name-$version because of that too.03:25
darfothat's why it seems like a better solution if a bit more complex03:25
Romsterthat is another option but it obscures where the file is got from originally.03:25
darfoagreed. I've been doing that with some git sources so at least the source03:26
darfocan be reproduced03:26
Romsterand introduces a possability of host being unreachable and can't get the file and help where is the original name so i can get it off another mirror say gentoo?03:26
darfotrue, that will be an issue. argh. this is a tough nut to crack.03:27
Romstersure we have some mirrors now but if is down what was them mirrors again?03:27
darfosame in my case with putting git tarballs on a server03:28
darfobut I've always been willing to wait a while (days) to get source03:28
Romsterand we could argue why we arn't doing git+$version or something?03:28
Romsterfor a git branch03:28
darfobut I'm not responsible for anything security critical like ssl03:28
Romsterwe don't want git in core i'd imagine.03:29
darfoi learned how to cut a tarball in gitlab by tagging a version03:29
darfoso it has a real name just like the good old days03:29
Romstergit archive or something?03:30
darfosince it is compressed it doesn't eat a lot of space03:30
Romsteryeah and self host it for source=03:30
Romsterstill that obscures things i think. but is the only solution if no archives are released03:31
Romsterwhole thing is a mess, i personally like my method /var/ports/distfiles/hash/${file}/md5/${hash5}/${file}03:31
Romsterreason why i use md5 directory is i also do sha256, but i don't see crux ever switching. we have this signature ting for crux 3.3 that i haven't even done anything with at all.03:32
Romsterreally not sure on the solution other than github should fix there crap, which won't happen.03:36
Romsterand supporting the http header filename isn't always reliable and it also obscures the real filename unless you do a header lookup and save that result and that introduces a problem if it times out or no network connection at that time.03:38
Romsteri've thought this though long and hard darfo03:38
darfoi know you have. i've been reading what everybody has been saying03:40
darfobut I don't know if they've really tried to live with some of the proposed03:40
Romsterand i know crux wants a kiss solution03:40
darfosadly they custom of properly named tarballs has fallen away03:41
darfoI don't want multiple copies of the same source as one proposal would cause03:43
Romsterto be honet it's simple enoughto read the .md5sum file and set $md5sum from that and wget$file/$md5sum/$file or even$md5sum&file=$file but that later way is ugly in my opinion03:43
Romstermy proposal (i should add this to the bug report) only keeps one copy03:44
darfobut that puts all the source load on where now the downloads get spread all over.03:44
Romsteranother thing i do is grab the souces and save them in my hash and some ocasions i download them all again and rehash and see if the hash hasn't changed. if it does i end up with multiple copies i can diff them.03:45
darfoYes, that would catch what happened to one of my ports recently. Upstream changed the tarball contents but not the version number.03:45
Romsternot really tek has a redirect on to that picks a mirror like and a few others03:46 is just a kick off spot, but it's a single point of failure. we would be better with a subdomain with a round robbin dns for these other mirrors. unless the dns also goes down on crux.03:48
Romsterbut that is another topic03:48
darfowhat if you added a file next to your source file with the origin URL? in the distfiles hash tree I mean.03:49
Romsteri'm working on another solution to archive downloading (but i don't think crux would be interested in it's complexity)03:49
darfono, jaeger and other already work a lot at explaining how crux works to newbies03:49
Romsteri often make a url file in my{url,file,MD%SUM}03:50
Romsterso i know where the source url file came from03:50
Romstercrux isn't for newbies through03:50
Romsterit's for experienced linux users but then i did start out with only knowing a little.03:51
darfoi don't mean newbies like that, I mean arch and ubuntu convertees03:51
darfowith url, file, and MD%SUM every thing is auditable03:52
Romstermost if not all distros hide the glory details and i origionally found crux hard to start with but i learned so much i couldn't go to some hand holding distro now.03:52
Romsternot unless i got that old that i didn't care to tinker anymore. (not likely to happen)03:52
darfothe older I get the more I want to know what's actually going on03:53
darfoto many snoopers and shysters want to take advantage of disinterest03:53
Romsteranyways my little project is to use aria2 download manager and meta4 files that describe the mirrors and ways to get that file, and that also handles the sums for me.03:53
Romsteri like knowing what everything does, windows 10 scares me, google scares me... all data collection scares me03:54
Romsterputting adds in everything also scares me.03:54
Romsterhell firmware... it could be phoning home03:55
darfoyes, that is not what the internet was started for03:55
Romsteri might be paranoid but this shit happens03:55
darfoit was supposed to be about sharing and all raising up technology03:55
Romsteri don't mind paid services for movies and such but information should be free.03:55
Romsterthis is why i use crux i like knowing what goes on and tinker.03:56
darfoI'm just as paranoid but I've kinda surrendered to being under surveillance all the time in the city because of all the cameras and the state of biometrics.03:56
Romsterfinger prints that shit is bad03:56
darfome too. I had to support RHEL 6 & 7 at work.03:57
Romsternot only can they clone it, but you can't just change your finger prints like a password or gpg key now03:57
Romsterand if someone is serious enough just cut your finger off and take it to get into top secret facility03:58
darfoI was reading the other day they had to up their number of match points on finger prints because there are so many people in the world now.03:58
Romsterand the chances of any two people having the similar print03:59
darfoahh, the last one of those I worked in they had to take your both your eyeballs and weigh the same to get in.03:59
darfothey weren't top secret just very concerned about corporate espionage04:05
Romsterweight hmm interesting carries a briefcase with extra weight beeep nope no entry04:09
Romsteranyways this source archive issue, not sure what the solution will be04:10
Romsteri am biased but i like my hashed directory idea04:11
Romsterit adds some complexity behind the scenes, but the pkgmk command changes are minimal to tack on the url path04:11
Romsterit can do this for just the MIRRORS in pkgmk.conf04:12
darfohad to give your briefcase or test gear to security and they passed it through to make sure no exploses of strong magnets04:15
Romsterdarfo, you probably know of my stuff and it's been stagnating for ages, and recently been working on how to get the job done (like revision 20 of it...), is to make meta4 files that describe all mirrors crux ones and project mirros and it's main download site and that handles the hashing and checks, but more importantly it'll download in segments from a group of servers no more waiting.... for slow downloads. (unless your last04:16
Romstermile is the weakest link :P)04:16
Romsteron top of this, i want to make a pkgdl daemon pkgdld. that you send RPC json files to from prt-get it'll pre-fetch all files as it's building the first package.04:17
darfowoohoo. i'd see get hopping but I know you're a busy fellow. I've been waiting for versionsort. But like I said: I'm patient.04:17
Romsteri got a friend helping me when he isn't busy04:18
darfoI do a manual form of that with prt-get --margs="do" sysup followed by prt-get sysup04:18
Romsterto the end use pre-caching files and fast downloads, to the packager semi-automated grabbing new files and building packages.04:19
darfoNot quite as multitasking as your method but it gets my local source up-to-date so I can start the sysup on my other CRUX boxen and use the source04:19
Romstercron fetching new versions from data. (when that is operational)04:19
Romsterif source changes or server goes down well not stuck waiting for the source04:20
darfoalas, until I get a fatter pipe I will get rate-shaped on that and end up at 50kbps as usual. Canada internet sux.04:20
Romsterand if it's changed can see what has changed.04:20
Romsterewwww i got 700kbs to home in australia04:21
Romsterwhen iw as on dialup i was messing with xdelta on large 100MB plus source files and made small bianry diff files to get over dialup.04:21
darfoyou live in a real country that won't put up with ISP BS obviously04:21
Romsterit would make a 100MB or so file for a new version turn out to be 10MB if you had the previous version04:22
Romsteri'd like to do that again but the meta4 xml specification doesn't have that in the specs04:22
darfowhen I was on dialup I was also using the base kernel and then only applying patches to get to the next point release.04:23
darfoi like xdelta. that was the only thing I liked about OpenSUSE and RHEL 7.04:24
darfobut they were still binary distros04:24
Romsteri'd like to add patch/rsync/zsync/xdelta some method to make pkgdl fetch a diff if it finds a older version of the source on the system to use.04:25
darfothat would be kind to both the Internet and my ISP bill04:28
Romster ugly but this did work in the day04:28
darfowouldn't want you slowing down all the important pr0n or e-business :)04:28
Romsterit wold look for the old version find the patches needed to bring it up to the new version...04:29
Romsterif no old source or patches found or too many are needed to make size too big skip them and just grab the archive directly.04:29
Romsterbut this time it will be written in a sane language.04:30
Romsteri did some crash shit back then...04:30
darfoah, but brute force is still force. that's a long read.04:31
Romsterlong nights that was04:33
Romsteri'll get there even if it takes another 5 years to get this done04:34
Romstereasy solution would be to offer rsync but i find it doens't work well with different file names.04:34
Romsterand it doesn't work well with compressed archives, which is why i did xdelta at the .tar level04:35
Romsterqt5 is alost done, lets hope this is the last change in a long while.04:36
darfothey need to modularize qt5 like they did xorg. The compile is a ridiculous hog.04:37
Romsteri've built it like 15 times now04:38
Romsterthank goodness for ccahce04:38
Romsteri haven't seen jue in awhile is he on holidays?04:39
darfodon't know04:39
darfobut I hope so. I had a holiday recently and it did me a world of good.04:40
Romsteri could do with one too, but i did get friday off this long weekend.04:41
darfoI'll be glad when winter finally (maybe) gets here so I can spend more time at the keyboard.04:41
darfodo anything special?04:42
Romsternah went to the beach walked like 4 killometers on sand, spend today sleepng in to 10am and at the computer since04:42
Romsterand it's nearly 3pm04:43
darfothat beats the hell out of work04:43
Romsterovercast and patches of rain today, yesterday was perfect sunny04:43
darfoI quit my regular job in May but I've been so busy with projects on the farm and house. Thinking about getting another job so I can rest :p04:44
darfoThat's our usual statutory 3-day weekends. Sunny during the work week, then crap weather on the long weekend. Not got for morale.04:45
darfoI miss the old IRC bot where you could ask when last time someone was seen.04:49
darfoI better quit talking and get some sleep. I've already said more today than in all the years I've been lurking here.04:50
Romsterlol yeah you don't say much usually and yeah i miss that bog04:51
Romstergood night darfo :)04:51
Romstermaking coffee and working on stuff.04:51
darfohave a good afternoon romster :)04:51
Romsterat least this is relaxing :)04:51
*** onodera has joined #crux-devel10:06
*** onodera has quit IRC10:21
*** onodera has joined #crux-devel12:18
*** heroux has quit IRC18:21
*** heroux has joined #crux-devel18:22
*** groovy2shoes has joined #crux-devel18:24

Generated by 2.14.0 by Marius Gedminas - find it at!