IRC Logs for #circuits Friday, 2017-03-10

*** chilli0 has joined #circuits01:20
chilli0Hello guys :)01:20
chilli0how has things been prologic01:20
*** chilli0 has quit IRC03:53
*** xMopxShell has quit IRC03:55
*** xMopxShell has joined #circuits03:56
*** pombreda has joined #circuits09:12
pombredaHi: Would circuits be the cure to replace all or some of this? https://github.com/nexB/scancode-toolkit/blob/develop/src/scancode/interrupt.py09:13
pombredaand and this https://github.com/nexB/scancode-toolkit/blob/develop/src/scancode/cli.py#L37309:13
pombredae.g. I have an outer loop that uses today a multiprocessing pool and an inner loop in a process that spawns a thread running the actual callable09:14
pombredamultiprocessing is really problematic. and fails randomly some python versions on some OSes.09:14
pombredaI really need on only spawn a limited number of processes in //.  And I need a sentinel for max memory and timeout on these sub processes. And this is a CLI that runs on Linux/Windows/Mac09:15
pombredaspaceone: what would be your take?09:20
*** pdurbin has quit IRC10:02
*** apollo13 has quit IRC11:07
*** apollo13 has joined #circuits11:08
pombredaapollo13: hey :P11:45
pombredado you think circuits is for me?11:45
*** chrismay has joined #circuits13:25
riotpombreda: easiest way to find out yourself is to just try it :>14:06
spaceonewe use multiprocessing in the underlying components, too.14:09
spaceonee.g. if you start subprocesses with our components14:10
pombredaspaceone: thx. no go then for me. pickle is too capricious across common Python versions14:18
pombredariot: it sure is. there are just 30 ways to do thing in 30 libraries, so asking first for some opinion does not hurt14:18
spaceoneyes14:19
spaceone:)14:19
pombredathanks again14:20
spaceonepombreda: but you might want to use circuits.node which uses json in the underlying implementation14:22
*** pdurbin has joined #circuits14:25
pombredaspaceone: interesting. thx :)14:30
riotspaceone: ever thought of combining the node-stuff with zmq?15:16
apollo13pombreda: how would I know :)15:17
apollo13zmq is a nasty beast :)15:17
apollo13pombreda: I am currently still in the planning phase of using, to many other things got in between15:17
apollo13which is kinda annoying but I guess that is work15:20
apollo13on the other hand I am nowadays building cpp11 apps on centos6 and shipping to other systems which don't support cpp11 yet, that is a win…15:20
pombredacongrats15:21
apollo13pombreda: what do you wanna use circuits for?15:23
spaceone+pombreda | Hi: Would circuits be the cure to replace all or some of this? https://github.com/nexB/scancode-toolkit/blob/develop/src/scancode/interrupt.py15:24
spaceone +pombreda | and and this https://github.com/nexB/scancode-toolkit/blob/develop/src/scancode/cli.py#L37315:24
spaceone +pombreda | e.g. I have an outer loop that uses today a multiprocessing pool and an inner loop in a process that spawns a thread running the actual callable15:24
pombredaapollo13: do you have the scroll log? See also my posts in http://irclogs.shortcircuit.net.au/%23circuits/2017-03-10.log.html15:24
spaceone  +pombreda | multiprocessing is really problematic. and fails randomly some python versions on some OSes.15:24
spaceone  +pombreda | I really need on only spawn a limited number of processes in //.  And I need a sentinel for max memory and timeout on these sub processes. And this is a CLI that runs on Linux/Windows/Mac15:25
apollo13mhm, looks like you want that thing running locally, cause it certainly looks like a perfect job for a task queue15:28
pombredaapollo13: yes, except that this is a self-contained CLI package for Linux/Windows/macOS. sounds like most task queue demand an external broker15:34
pombredaapollo13: unless you would know of one that would not?15:39
pombredae.g. a task queue for use in a CLI15:40
apollo13pombreda: yeah, that was what I feared :D16:01
*** pombreda has left #circuits ("Leaving")20:15
*** chrismay has quit IRC21:43

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!