IRC Logs for #circuits Thursday, 2016-09-29

*** ke4roh has joined #circuits00:43
*** Workster has quit IRC02:28
*** wclwyobtwvdypkwg has joined #circuits02:28
*** Workster has joined #circuits02:28
*** ke4roh has quit IRC04:38
*** braoru has joined #circuits06:59
braoruhello :) Is someone here for a couple of question ? I try to revwrite one of our software with circuits but I still have some trouble at some edge :-) (It's probably a chair-keyboard problem I admit)07:01
Worksterbraoru, state the questions and hang around for prologic or someone else to reply07:05
Worksterdifferent time zones and all that.07:05
braoruok ;)  I will try to make it a whole :)07:06
riotprologic: did not yet come around to further testing.07:27
*** Workster has quit IRC07:30
*** robert_ has quit IRC08:13
prologicbraoru: hey; I'm around. What seems to be the problem?08:22
prologicRomster: thanks :)08:22
prologicriot: kk let me know :) if you can repro the problem with a simplified test case/example I'll definitely have a closer look! :)08:23
Romsterno problem prologic08:30
braoruOk the 2 main question 1 : Is it possible to know when start(process=true) is done (to be able to fire event) and 2 What is the best way to fire a event within a compoenent from outside the component. It's seem to work if I register everything to manager and then use manger.fire + manager.flush but it's not async :/08:50
braoruthe idea is to migrate some part of an app to circuits (modules are completely uncoupled) it's a tornado webbapp and I try to rewrite some modules on a event-base systems and interact between my  component and my handler through an eventbus (as I would do in scala+akka)08:52
braoruthe current main workflow client <- HTTP CRUD -> tornado <- handler -> (Task + futures) <- modules -> external platform interaction08:54
prologic1) not builtin; but make the sub-process/component fire an event on started (which will fire when *that* process/component is started via .start(process=True)08:55
braoruBut it will work if the main app "is" the main component right ?08:55
prologicshould do08:56
prologic2) I'm not sure what you're asking here08:56
prologicif you have a component graph (a bunch of components registered together) just call .fire() on any one of them08:56
prologiccircuits *is*/*should* be thread safe and you can call "into" the system from outside08:57
braorubut can I call fire from outside of the first component ?08:57
braorubut then it's blocking until the call is done08:57
prologicyou can call any of the API(s) from any part of the component graph - yes08:57
prologiccircuits is effectively an event bus :)08:58
braoruYeah it's the main reason I trying to use it :)08:58
braorubut I think I get too formated by akka08:58
prologicalso funny you are porting tornado apps to circuits08:58
prologicwe have a repo that lets you run twisted apps on top of circuits :)08:58
braorudo you have a link :)08:58
braoru?08:59
prologicsure https://bitbucket.org/circuits/twistedintegration08:59
prologicwe didn't migrate this repo to github unfortunately and it may be oldoutdated08:59
prologic*could* be make to work again if someone cares :)09:00
braoruBut will help to get an Idiea09:00
prologicthat's why I mentioned :)09:00
braoruIn my dram I would like to develops news modules as set of component registering on some event (channel here)09:00
prologicsounds dosable09:00
prologicdoable*09:01
braoruinstead of having a  giant polymorphic factory trying to create driver instance based on black oracle magic and one of the 80 implementations09:01
prologicyou should also check out https://github.com/prologic/autodock https://github.com/prologic/kdb and https://github.com/prologic/sahriswiki for other ideas :)09:01
braoruget to the handler wrap a call inside of a future with the good event09:01
braoruthanks :)09:02
prologicnps :)09:02
prologicif you haven't already seen09:02
prologiccircuits is all about events and components09:02
prologicand composition of components09:02
prologic@graph09:03
braoruyes I spoted that, now I try to better understand how event are distirbuted and "handled by handlers". Sometime I get event fired as "<boum[*] ( )>" but the handler never execure09:03
braoruor call that never resolve :)09:04
braoruevent for a simple "yeild 4"09:04
braoruI may still missing some basics09:04
prologicthe various examples should help09:04
prologicas well as the doco09:04
prologicbut in general "component registration" is really only a design feature09:05
prologicusually used for visualization of your system via circuits.utils.graph() and circuits.utils.inspect()09:05
prologicevents are bubbled up to a "core" manager (usuaully the first component)09:05
prologicand we cache handlers for performance09:06
prologicso things are quite fast09:06
prologicand of course there are two primary ways of separating events09:06
braoruthen why when I do a start(process=true) the ready event fire like 3-4s later ?09:06
prologicevent names themselves and channels09:06
prologicbraoru: example?09:07
braoruI try to make one09:10
braorucleaning my test file09:10
prologic$ time ./examples/hello_bridge.py09:18
prologicHello from parent with pid 755209:18
prologicHello from child with pid 755309:18
prologicreal   0m1.037s09:18
prologicuser   0m0.156s09:18
prologicsys    0m0.070s09:18
braoruprologic, morning haze I think https://gist.github.com/braoru/fa4fdedf2fa51c9db2fb62ad5a62894d09:21
braoruit's fast, sorry .)09:21
braorubut still non fired event (but I should try harder)09:21
prologiclemme see09:22
prologicoooh09:24
prologicthis is interesting09:24
prologicI'm not sure you can do what you're doing here :)09:24
prologic.start(process=True) forks a subprocess09:25
prologicand link=self creates a Bridge between the child and parent process via UNIX sockets09:25
prologicso you can transmit events between parent<->child09:25
prologicL84 and L8709:26
prologicyou have no "main" component/system09:26
prologicif you don't have a system that "runs" nothing will really work very well09:27
braoruthat what I was thinking09:27
braorubecause all my test run well if I use run :)09:28
prologicyeap :)09:28
braoruand everything quiet wired when I use start()09:28
braorubut run block09:28
prologicyou're racing in this code09:28
prologicthe main process will die too quickly for anything to really work09:28
braorustart call run no ?09:28
prologicyou can do what you're doing here09:29
braorubut the second call09:29
braoruafter 5s09:29
prologicbut your main process has to block or do something09:29
prologicotherwise the forked subprocess via .start(process=True) won't have a chance to run09:29
braoruis there a way to know that everything started ?09:30
braoruright you said not builtin09:30
prologicno09:30
prologicand more importantly09:30
prologicnot without linking the child component/process to your main process09:30
prologicand firing an event09:30
prologicbut may I ask... why this design?09:31
prologicare you doing some kind of integration with Tornado?09:31
braoruthe idea is to make our tornado handling fireing event09:31
prologicOh I see09:31
braoruand re-do our design on a event.based approach instead of our current massive callback internal systems09:32
prologicso from your Tornado web handlers09:32
prologicyou want to fire events into a circuits system?09:32
braoruexactly09:32
prologicahh09:32
prologicthat's easy :)09:32
prologicuse circuits.node instead09:32
braoruthe main idea would be to wrap the fire within a await :)09:32
prologicthis will let you fire events into your circuits system via JSON over HTTP09:32
braoruthen the handler can keep being async09:32
prologicoh :)09:33
braorucan try09:33
prologicSo you're also using Python 3.5+ with Tornado and await/async?09:33
braorucurrently handler just await giant task09:33
prologicahh09:33
braoruand the giant task load everthing  she need, register 100/150 callback and run subtask calling back themselv09:34
braoru..09:34
braoruhuge crapy things to debug09:34
prologicwell tbqh we haven't really gotten into the whole Python 3.5+ async/await integration (yet)09:34
prologichah09:34
braoruso we pile futures and resolve togerther but it's a big mess and an event bus would be easier09:34
prologiccallbacks are a nightmare to debug :)09:34
braorulike akka :-) (we have serval services written in akka/scala)09:34
braoruthe system is in production and it work fine but it's messy09:35
braorunext step would be to link grpc call to event :-)09:35
prologicahh nice09:35
prologicso yeah no matter how you do this really09:35
prologicI would use circuits.node (or a modified version of)09:36
prologicbecause you're really doing distributed event processing09:36
prologicI have a custom circuits.node in autodock (linked earlier)09:36
*** ke4roh has joined #circuits09:36
braorutaht mabye my first error ... thinking of circuit as a classical event bus where you start the bus then you just register actors on it09:37
prologicif it was pure circuits - yes :)09:37
prologicbut playing nice with other systems is different09:37
braorubut it's not09:37
prologicthat is not to say you can't achieve what you want though with a little effort09:38
braoruclearly09:38
prologicbut the "event bus" will be too low level for your needs I think09:38
braorumixing async/away with tornado + grpc was already quite edgy :)09:38
prologicI'll bed09:38
prologicbet*09:38
braoruand there is not realy high level eventbus as I saw .. ./09:39
braorufor python09:39
prologicbut look into circuits.node; simple events API with a TCP+JSON transport09:39
braoruI will look at it09:39
braorummh I may need to wrap the run() in something09:41
braorubut spawning a thread/sub process just to hold the .run() .. there should be something cleaner .)09:42
braoruprologic, what is the particularity of your customized version of autodock ?09:48
prologicjust a better implemtnation09:49
prologicwith better serialization09:49
prologicalso run just basically does:09:50
prologicwhile True:09:50
prologic    self.tick()09:50
prologic    self.flush()09:50
prologicmore or less :)09:50
prologichave a look though09:50
braoruit's within you git repos ?09:53
braoruyours09:53
prologichttps://github.com/prologic/autodock09:58
prologicman comcast is really play'n up tonight :/10:12
braoruit's morning here :)10:59
braoruyou get huge lag ?11:00
prologicdropouts :/11:01
prologicit's late :) I should go to bed!11:01
braoruthen thanks again for the help :)11:25
braoruI will come back laster with other question !11:25
*** ke4roh has quit IRC12:24
*** ke4roh has joined #circuits12:33
braorunot better with circuits.node :-( debugger show a connection .. but fire just fire nothing in to empty void :-(13:29
braoruok will look tomorrow .. if I run with python -i and then fire manual it's work .. if I just executte it does not work :-/14:33
*** braoru has quit IRC14:43
*** ke4roh has quit IRC19:46
*** FSX has joined #circuits19:47
*** Guest65549 has quit IRC19:50

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!