IRC Logs for #circuits Tuesday, 2015-03-10

*** techdragon has quit IRC00:28
*** techdragon has joined #circuits00:30
prologicrogeriobastos: I don't think it'll make much difference00:32
prologicbecause the implemtnation of redis's async api is likely to not cause any blocking much00:32
prologicthat's the whole point of async :)00:32
prologicso it'll be fine00:32
prologicgenerally speaking you want to avoid any kidn of threading in your app00:32
prologicif you do CPU bound work - use a Worker process pool00:32
prologicusing the Worker() component00:32
prologicotherwise try to do everything async :)00:32
kwmiebach"from circuits import listener" is in the examples, but fails here: cannot import name 'listener'02:06
prologicumm02:16
prologicwhat example(s)?02:16
prologicbecause that's quite old and deprecated long ago02:16
prologicI hope we didn't miss updating something :)02:16
prologicand hi :)02:16
kwmiebachhi :) ok i will not use those thank you02:17
prologicgot a link to this?02:17
prologicI'm kinda curious wher eyou've found such old examples/docs :)02:17
prologicI hope you're using the latest circuits 3.x?02:17
prologiccircuits-3.102:18
prologicso it'd be:02:18
prologicfrom circuits import handler02:18
prologicbut in general @handler is not really necessary for the simplest of caes02:18
prologicjust define method son a Component subclass02:18
prologic:)02:18
prologicthey are automagically decorated with @handler02:18
prologic@handler is reall yonly useful if you want ot be more explicit02:19
prologicand change some properties of the event handler02:19
prologicnames, channel, priority, etc02:19
kwmiebachI found it here https://raw.githubusercontent.com/circuits/circuits/master/examples/timers.py02:22
kwmiebachwait my fault02:25
kwmiebachok it was really old, I just pasted this example http://jessenoller.com/blog/2009/01/31/circuits-event-driven-components into my code - later I forgot to remove the import02:27
prologichmm02:34
prologicthat github link is right :)02:34
prologicand looks correct02:34
prologichaha02:34
prologicahhh02:34
prologicJessie Noller02:34
prologicgotcha :)02:34
prologicyeah blog posts can get a bit old and outdated sometimes02:34
prologickwmiebach: you're clearing evaluating circuits for something -- may I ask what? :)02:40
kwmiebachprologic: I am writing a service that needs to spawn long running activities after it detects some events that trigger these activities02:49
kwmiebachand also time based activities similar to cron02:49
prologicgreat02:49
kwmiebachthose are in different python scripts but I want them all in one process02:49
prologic*nods*02:50
kwmiebachI wonder what happens if I have several unrelated components and on "crashes" meaning an unexpexted exception. can I momitor the component and restart it?02:52
prologicis the component running in a separate process?02:53
prologicotherwise generally speaking compoents make up a graph of features of your application02:53
prologicgiving you a loosely coupled architecture02:53
prologicthey don't "run" as such02:53
prologicthe whole system does -- the top-most compoent in the graph02:53
prologicif an event handler throws an error that goes uncaughtt02:54
prologicthe event loop will capture that and fire an exception() event02:54
kwmiebachit would be best if I have only one process that contains all the components. It is easier do monitor only one process and not 10 (like now)02:55
prologichttps://gist.github.com/6932f10add56db6bac1402:56
prologicyeah you'll be fine :)02:56
kwmiebachso I could catch all known exceptions - and in case of an uncaught exception there is an event exception() which I could use to restart the component?02:58
prologicthere's no notion of a'restarting a componet"02:59
prologicit'll keep working just fine :)02:59
prologicjust don't inadvertently cause an event loop02:59
kwmiebachI see. Lets say I have a component that does something like many network activities that take 1 hour or so. If after half an hour an exception occurs it should wait 5 minutes and then start from the beginning of the 1 hour activity - that was what my "restarting" should do03:02
prologicahh03:03
prologicyeah no problems03:03
prologicyou'll have to manage the flow yourself I imagine03:03
prologicthere's no "magic" in circuits really03:03
prologicso sounds like you'll want a task queue of sorts03:03
prologicand some tcp client components03:03
prologicand some timers03:03
prologicand you'll want to coordinate several or more events of varying types03:03
kwmiebachsounds good03:04
prologiccircuits provides you the basic application framework and structure to create main table and scalable applications03:04
prologicso if in your project you come up with more generic components and other useful things we'd definitely love to see those contributed back :)03:05
prologicsome examples03:05
prologicwe don't yet have a multiplexed tcp client component03:05
prologicso you'd be spinning up multiple TCPClient(s) on different channels in your case I htink03:05
kwmiebachyes03:05
prologica generic task queue / worker framework builtin to circuits would be awesome - perhaps with integration(s) with things like celery and others03:06
prologicall definitely possible03:06
prologicI was working on integrating circuits and rabbitmq a while back but never finished it03:06
kwmiebachfor example in one component that is planned there are 1000 mysql operations that need to be done on a remote machine by triggering an http command. not more than 5 should run in parallel, but sometimes the connection goes away for some minutes - in this case the component should wait and later continue03:14
prologicsure03:15
kwmiebachbut sometimes the mysql operations "hang" and if one or more of the 5 parallel workers are hanging they should advance to the next operation (and log the failed operation)03:15
prologicyou'll have to implement the wait part as a imed event03:15
prologicor make user of the call/wait primitives in circuits03:15
prologiccall/wait give you some level of synchronous calls to events03:15
prologicbut operate asynchronously underneath03:15
kwmiebachand yes that code could be shared03:15
prologicI'd plan out your "event flows" now :)03:16
prologicyour system doens't sound trivial :)03:16
kwmiebachit could start simpler and improve over time - it is already in production with simple manual scripts that have only one worker03:17
kwmiebachso I would wrap the scripts that exist in a component and log it03:18
kwmiebachand log the failures and improve over time03:18
prologic*nods*03:19
kwmiebachI will be idling here. need to go now it is late. Thanks a lot I am happy that someone helps and this is a very cool project03:20
kwmiebachi looked at the varous options but many python concurrency projects are old03:21
kwmiebachon the other hand it is strange there are not more people using ther technology03:23
kwmiebachit seems many people prefer callback programming with node03:24
kwmiebachok see you later, bye and thanks again03:24
prologicnps :)03:27
prologicyeah most of the Python community tend to favour Twisted03:27
prologicbut on the other hand most of the Python community can't stand using Twisted03:27
prologicso *shrugs*03:27
prologicThe only reason circuits isn't more widely used is because I'm a Software Engineering and don't have the marketting expertise to gain more traction than what it already has over the past 10 years03:28
*** Osso has joined #circuits07:54
*** Osso has quit IRC07:54
rogeriobastosthanks prologic11:30
rogeriobastosyour tips were useful and I am having fun with circuits11:32
rogeriobastosI hope talk in this channel more times and make some contributions to the project11:32
rogeriobastosI am curious to know how many users in this channel are bots because you are the only one that answer questions :-p11:34
prologichaha11:40
prologicno they're all users11:40
prologicexcept the irclogger11:40
prologic:)11:40
pdurbina bunch of lurkers11:46
prologicsee11:50
prologicthere ya go :)11:50
prologichi pdurbin11:50
pdurbinhi. I added async to my java ee project the other day11:51
prologicoh?11:52
pdurbinmake "index dataset" async #702 · 22e5c1a · IQSS/dataverse - https://github.com/IQSS/dataverse/commit/22e5c1a99935c000c4f76cb8c728227719779cec11:52
prologichmmm11:54
prologicvery good :)11:54
pdurbin:)11:54
rogeriobastoshaha...was a joke. I watch your presentation and there you say that likes write ircbots11:54
prologiclol11:57
prologicyeah11:57
prologicgawd I was nervous :)11:57
prologicI might present at PyCOnAU this year too11:58
prologicprobably something about Django though11:58
prologicpdurbin, oh btw11:58
prologichttps://vallinux.github.io/11:58
prologic:)11:58
pdurbinprologic: cool. you like pelican?12:29
prologicit's ok :)12:33
pdurbin:)12:34
*** my_chiguai has joined #circuits22:40
*** my_chiguai has quit IRC23:33

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!