IRC Logs for #crux-devel Thursday, 2015-01-15

*** leo-unglaub has quit IRC00:03
diversesigh, curl bindings has a lot of dependencies00:03
diverseand because cargo keeps faulting on certificates I have to pull each repo manually for quick-n-dirty testing00:18
diversesigh, got thru the dependency hell00:46
diversewell at least now I figured out an easier way01:04
diverseso after compiling all the libs, I decided to move them to the central area where I needed to compile the curl lib, so all I need is just one -L arg instead of so many01:07
diverseand boom, got my libcurl binding01:08
diverseat first I thought -L would search recursively but it didn't01:09
diverseI hope some of those libs move to the std so it's less ports to manage01:10
diverseanyway I hope I wasn't being too offtopic, it's about development and related to pkgmk after all01:12
diverseprologic: I looked into every toml file for dependencies, went to their respective repos and had to git clone them01:17
diverseso much for quick and dirty01:17
diverseprologic: I envy the fact python has a better pkg manager at least, pip01:22
prologicit sure does :)01:25
prologicand with Python wheels installing a large list of compled dependneices01:25
prologiceven ones with C bindings is01:25
prologiclightning fast01:25
prologicfor example our Django software here at work with about ~50-60 deps01:25
diversewell and without that "central repo only" non-sense01:26
prologicinstalls in ~10s01:26
prologic$ wc -l requirements.txt01:26
prologic90 requirements.txt01:26
prologicThere's many reasons I like Python :)01:26
diverseI know, I know, but I'm a statically typed, hardcore to the metal, type of programmer.01:27
diverse(I know it's not really to the metal because of microcode, but low level I mean)01:28
diverseprologic: although when assigning, type inference is really nice ;)01:36
diverseprologic: takes away the extra verbosity like with Java ex: SomeObject foo = new SomeObject(bar, baz);01:41
prologicyou should just code in RPython then01:43
diversewhat file should I try downloading?01:46
diversemaybe a pkgfile01:48
prologicOh RPython?01:54
prologicRPython is a restricted subset of Python01:54
prologicwhich basically means static typing01:54
prologicbut it's all inferred01:54
prologicand you "compile" your app into a binary01:54
jaegershould have called it "ython"01:54
prologicThe way you get RPython and play with it is by clonging the pypy repo01:54
prologicI have a branch of it with easier so you can use it a bit nicer01:55
prologicjeager: :)01:55
diversepython has a lot of dialects01:57
prologictar jxvf tip.tar.bz201:57
prologic(cd prologic-pypy-* && python develop && python develop)01:57
prologicpretty easy to setup01:57
prologicripped that off my miolang experimental language01:58
prologicOne of the nice things about working with RPython is that because the language is a subset of Python itsaelf you can run/test your code against regular CPython before compiling01:59
prologicit's the compiling step that you get all your nice type errors and have to fix up your nasty uses of dynamic typing :)02:00
prologicIf it helps you play with it I can build a docker image of it so you don't have to install anything02:00
*** kori has joined #crux-devel02:01
*** irclogger___ has joined #crux-devel02:55
prologicjeager: re notify the ml of iso updates?03:18
*** mavrick61 has joined #crux-devel03:36
*** pitillo has joined #crux-devel06:24
*** Phonester has joined #crux-devel06:55
*** Phonester has quit IRC07:03
*** Workster_ has joined #crux-devel14:59
*** Workster has joined #crux-devel14:59
*** Feksclaus has joined #crux-devel17:44
*** Roosevelt has joined #crux-devel19:46
*** Roosevelt has quit IRC20:23
*** Feksclaus has quit IRC21:06
*** Feksclaus has joined #crux-devel21:07
*** diverse has joined #crux-devel21:30
diverseso do we have to wait for a bug fix release from curl?21:52
diverseprologic: why is there 3 irclogger bots?21:53
koriextra logging21:53
diversefrom netsplits?21:53
koridiverse: I actually don't know, but probably21:54
teK_who of you set up distfile mirrors? Romster, jaeger?21:55
prologicwe haven't yet :)21:58
prologicI think we're planning on?21:58
prologica communict driven cdn for distfiles?21:58
diverse(I guess I assumed correctly)21:59
prologicworking on it :)22:00
teK_When you're downloading Linux, you are downloading COMMUNISM!!!22:02
jaegerI played with (in VMs) a CDN built with bind, geoip, and nginx22:02
jaegerhaven't put up anything official yet22:02
prologicDoes that work?22:03
diverseI hate it when people can differentiate the difference between socialism and communism22:03
teK_diverse: I quoted a meme22:04
teK_oooh so jaeger you might like that one..22:04
teK_in your pkgmk.conf22:04
teK_redownload some ports, watch and enjoy22:04
teK_it's not geo-location-based, though. But there's even an apache module (with hostname lookup etc.)22:05
prologicoh you've already set this up?22:05
teK_cost me ~30 minutes22:06
teK_most time I spent with apache's config file (again)22:06
diverseteK_: sorry, it just brought back unwanted memories :(22:07
teK_sadly I could not figure out how to _really_ transparently map /distfiles to my PHP script. Lighttpd can do that without a HTTP 30222:07
teK_diverse: no need to apologize :)22:07
prologicso how does it work the way you've set it up?22:09
teK_it's a HTTP redirect from /distfiles/$1 to ~tek/dist.php?f=$1. The script does round-robin (two hosts atm) and another redirect22:11
jaegerDid you figure out a way to clean it automatically?22:12
teK_to avoid one of the two redirects, on could set in pkgmk.conf, too22:12
teK_jaeger: my mirror directory?22:13
teK_nah, doing it by hand atm, it's not that much of an issue atm22:13
jaegerI'll set up a mirror on my VPS as well if you'll add it to the list22:13
teK_I will figure that out if this topic gets traction22:13
prologicso it doesn't do any checks to make sure the file being asked for is a valid source for some port?22:14
teK_this can be made more complex in an arbitrary manner.. imagine crux-specifiz source-file names (think: fucking github n.n.n.tar.gz files)22:14
teK_triggerable on git-hooks22:14
teK_prologic: nope22:14
teK_not atm22:14
jaegerwe'll need to add the support for renaming the destination file in source=() at some point22:14
jaegerto work around the github stupid22:15
prologicso it in theory will let me download anything :)22:15
teK_yes, or github finds some sanity22:15
teK_did anyone poke them about this?22:15
jaegerI think the former is more likely22:15
teK_prologic: give it a try22:15
prologicjeager: yeah my suggestion is using #filename=22:15
diverseoh, we're changing pkgmk a little?22:16
teK_gotta fix things22:17
prologicseems to work22:18
prologichow does it know where to download the file from?22:18
prologicReferrer header?22:18
teK_prologic: almost22:19
prologicexplain :)22:20
teK_header('Location: ' . $mirrors[0] . $_GET['f']);22:20
teK_that's about it. I did not investigate security further22:20
prologicand $mirrors is?22:21
teK_you can try requesting ?f= ;)22:21
teK_$mirrors = array('', '');22:21
prologicahh I see22:21
prologicso pkgmk issues such a request to the configured mirror?22:21
teK_works both ways22:22
teK_the first does not work afaik22:22
prologicI don't get how this works :)22:23
prologiclooking at /usr/bin/pkgmk22:23
prologicI see no ?f=22:23
teK_nope doesn't work, you have to use
teK_that's magic done on :)22:23
teK_RewriteRule ^/distfiles/(.*)$1 [NC,R,L]22:24
teK_the only ugly thing are the two redirects,, it can be done in one; I will investigate some time this weekeend22:27
teK_lighty CAN do that22:27
prologicOh I see22:27
prologicyes but I still don't see how you know where to get the filwe from in the first place22:27
prologicsince all your script gets is a filename22:27
teK_pkgmk issues wget
jaegerprologic: from source=() of the original port, he's already downloaded them22:29
prologicI see :)22:29
prologicso the only clever part to this is22:30
prologica) you've downloaded hpas of source files22:30
prologicand b) it's one url :)22:30
teK_it's clever ALLTOGETHER :p22:31
jaegeranother option would be to use a squid server as the distfiles cache. I do that for ubuntu here at work but it isn't perfect22:31
diverseso, we are collecting all the non-local stuff in ?22:31
jaegerthey won't be on at all22:32
jaegerteK_: how difficult would it be to add geoip support to the php script?22:34
prologicyeah I was thinking more along the lines of a cdn22:34
prologicso doesn't have to bear the grunt of the workload and/or bandwidth22:34 does neither in this case22:34
diverseso a decentralized approach22:35
teK_jaeger: that's a pit I'm a bit concerned with, tbh22:35
prologicso it just redirect to a known source url for the given tarball?22:35
teK_there's a module requiring pecl22:35
teK_never heard of that22:35
jaegerIf you look up a bit teK_ explained exactly how it works22:35
teK_prologic: yes22:35
teK_also it refers to maxmind,.. dunno if they do things for free22:36
prologicoh hmm22:37
teK_I'm not sure how much work this would include22:37
prologicI'm stupid :)22:37
prologicI think I completely missed one key aspect to this22:37
prologicthe $missors array22:37
diverseprologic: that's how I feel right now22:37
teK_jaeger: how much I/O do you get from, just out of curiosity22:37
diverseprologic: but I only had 4 hours of sleep, so I'm not at full thinking capacity22:38
prologicI haven't had coffee yet :/22:40
prologictek, jeager: how large are your distfile collections atm?22:41
prologicI'd like to become a 3rd mirror :)22:41
jaegerteK_: I got about 6.7MB/s on a 250MB file22:42
jaegermine's still downloading but teK_ said it was 6.x GB if I remember correctly22:42
*** `c0x has joined #crux-devel22:44
*** atharic has joined #crux-devel22:45
*** teK__ has joined #crux-devel22:46
teK__23:42 < teK_> 6.8GB22:47
teK__23:42 < teK_> but don't be surprise if fake b****es be hotlinking your stuff for other purporses ;)22:47
teK__23:43 < teK_> had that with my ISO mirror and softpedia (or some site)22:47
teK__23:43 < teK_> they do advertising, that's why this is not ok22:47
teK__just in case..22:47
*** jaeger- has joined #crux-devel22:49
*** erdic_ has joined #crux-devel22:51
*** diverse has quit IRC22:51
*** Workster has quit IRC22:51
*** heroux_ has joined #crux-devel22:53
*** diverse has joined #crux-devel22:53
*** Workster has joined #crux-devel22:53
*** heroux_ has quit IRC22:54
*** heroux_ has joined #crux-devel22:54
*** prologic has joined #crux-devel22:54
*** kori has joined #crux-devel22:54
*** frinnst has joined #crux-devel22:54
*** erdic_ has quit IRC23:02
*** irclogger___ has quit IRC23:02

Generated by 2.11.0 by Marius Gedminas - find it at!