IRC Logs for #circuits Sunday, 2013-05-05

prologicand added unit test ensuring that doesn't break in future00:06
EasyMac_Yeah, I noticed there weren't any relevant unit tests for those00:10
EasyMac_you probably already figured it out, but I'm on python 3.3 with circuits-dev cloned from the latest00:10
prologicexcellent00:23
prologicthen do a hg pull -u00:23
prologic:)00:23
prologicI fixed that issue you found00:23
EasyMac_Yep, already pulled down and working00:24
EasyMac_Thanks00:24
prologicno problems00:25
EasyMac_Are there any circumstances in which a component's "stopped" handler wouldn't get called by the time the program has exited or been killed?  Other than kill -9 [pid]?00:56
prologicim out atm01:46
prologicbut yeah it should get called01:46
prologichowever in the non-running state there is limited time to do things01:46
prologici cant remember the exact semantics we put in place...01:47
prologicill have a look and test it when i get home01:47
*** koobs has joined #circuits02:57
*** koobs has quit IRC03:02
*** koobs has joined #circuits03:02
prologicEasyMac_, ping?03:29
EasyMac_pong03:29
prologichttp://codepad.org/50E05LKq03:29
prologicThis demonstrates (I think) what you were asking03:29
prologicOutput: http://codepad.org/1JeTDXZ603:30
EasyMac_yeah, I've been using the debugger component to see those events getting sent around03:34
prologicahh cool03:34
prologicDid I answer your questino?03:35
EasyMac_although I've been usually ending the program with a ctrl+c signal03:35
prologicI feel like I haven't :)03:35
prologicahh yeap03:35
prologic^C is always caught in the same way03:35
prologic$ ./foo.py03:35
prologic<Registered[*.registered] (<Debugger/* 11963:MainThread (queued=0) [S]>, <App/* 11963:MainThread (queued=2) [R]> )>03:35
prologic<Started[*.started] (<App/* 11963:MainThread (queued=1) [R]> )>03:35
prologic<App/* 11963:MainThread (queued=1) [R]> started03:35
prologic^C<Signal[*.signal] (2, <frame object at 0x82f38e4> )>03:35
prologic<Stopped[*.stopped] (<App/* 11963:MainThread (queued=0) [S]> )>03:35
prologic<App/* 11963:MainThread (queued=0) [S]> stopped03:35
EasyMac_Well, is "stopped" the place to handle object cleanup?  Or is there some better convention?03:36
EasyMac_I guess that is the heart of my question03:36
prologicahh03:36
prologictbh I don't know :)03:36
prologicthe only kind of cleanup we've done is to close sockets out properly03:36
prologicahh and to fire off CLose() events03:37
prologic$ grin "on_stopped" circuits03:37
prologiccircuits/core/workers.py:03:37
prologic   76 :     def _on_stopped(self, event, *args):03:37
prologiccircuits/io/file.py:03:37
prologic  158 :     def _on_stopped(self, component):03:37
prologiccircuits/net/sockets.py:03:37
prologic  292 :     def _on_stopped(self, component):03:37
prologic  601 :     def _on_stopped(self, component):03:37
prologicso I guess we do use the Stopped event internally :)03:37
prologicThe way the semantic work is you have at most 3 ticks to complete any cleanup03:38
prologicand a 4th final one at the end03:38
EasyMac_good to know03:39
prologica tick is a complete event-loop cycle03:39
EasyMac_Is there some kind of timeout on tick handlers?03:39
prologicumm03:39
EasyMac_If my component has a tick handler that is taking a long time, what happens to it?03:39
prologicthere is no such thing as tick handlers anymore03:39
prologicwe got rid of that concept in favor of just plain ol handlers and GenerateEvents03:40
EasyMac_ahh, okay03:40
prologicwe use that internally everywhere we need something to occur every tick03:40
prologicor a way to control idling03:40
prologicfor examples see circuits/core/pollers.py and circuits/io/file.py03:40
EasyMac_but I should use a poller instead of a tick handler03:40
prologicsearch for generate_events03:40
prologicalso circuits/core/timers.py uses it as well03:40
prologicideally if you're waiting on file descriptors - yes you should use a poller03:41
prologicthat's what they're designed for03:41
prologicof course you can just write your own using generate events too03:41
prologicbut pollers are the general abstraction of "waiting for read/write readyiness on fds"03:41
EasyMac_And for some general task that needs to be checked on?03:42
prologic<EasyMac_> If my component has a tick handler that is taking a long time, what happens to it? <-- anything that can potentially block should really be done in a Worker (in process mode, process=True)03:42
EasyMac_okay03:42
EasyMac_I think that answers that for me03:43
prologiccool03:43
prologicWorker wraps multiprocessing basically03:43
prologicasynchronously03:43
EasyMac_yeah, I was using multiprocessing pools before I got here to circuits03:44
EasyMac_Because in reality, not everything needs to be a process, I wanted to have some kind of event driven framework instead03:44
EasyMac_But sending out a few spawned processes would probably still be useful03:45
prologicyeah it is03:45
prologicespecially if they are heavily cpu bound03:45
prologicif they simply block a little (and can't be done asynchronosuly) just use a separate Worker in thread mode03:46
prologice.g: Worker(channel="worker_threads").register(...)03:46
prologicand another03:46
prologicWorker(channel="worker_processes", process=True).register(...)03:46
prologicand do things like:03:46
prologicself.fire(Task(some_func), "worker_threads")03:47
prologicor03:47
prologicself.fire(Task(some_func), "worker_processes")03:47
prologicalso circuits supports coroutines as well03:47
prologiccheck the examples in examples/primitives/03:47
prologiccall/wait03:47
prologicwhere any event handler can yield03:47
EasyMac_with "block a little" meaning for a short period of time? or happening not very often?03:47
prologiccircuits will treat it as a coroutine and keep "checking on it"03:48
prologicwell it just depends how concurrent you want your system :)03:48
prologicthe more you block in the main process03:48
prologicthe less concurrent requests you can handle03:48
prologicblocking for 0.1s means you can only handle 10 req/s concurrently03:49
EasyMac_got it03:49
EasyMac_I'm going to take a look at the process wrapping code for a bit03:49
EasyMac_And get a feel for how the result comes back in03:50
EasyMac_So, "while not result.ready(): yield" is more efficient in the main process than simply blocking with the original task in the main process?03:53
EasyMac_Or is the benefit really only coming to cpu-bound tasks?03:53
prologicwell03:56
prologicbasically that returns a coroutines to the circuits core/manager03:56
prologicso yes03:56
prologicrather tha wait/block03:56
prologicwe wait but yield to other handlers03:56
prologicyou know all the fancy yield from03:57
prologicand promises stuff by Guildo03:57
prologicbbs03:57
EasyMac_so, looking at timers.py, it does seem like "generate_events" is what I was trying to use "tick" to do.04:01
EasyMac_Although you said tick handlers didn't exist anymore, earlier today I was successfully having code execute from a component method named "tick"04:02
EasyMac_So it is good to know that I shouldn't be doing that04:03
*** EasyMac_ has quit IRC04:34
*** EasyMac_ has joined #circuits04:37
*** EasyMac_ has quit IRC06:01
*** koobs` has joined #circuits06:18
*** koobs` has quit IRC06:19
*** koobs` has joined #circuits06:19
*** koobs has quit IRC06:19
*** spaceone has quit IRC10:27
*** jgiorgi has quit IRC10:27
*** koobs has joined #circuits10:28
*** spaceone has joined #circuits10:28
*** jgiorgi has joined #circuits10:28
*** prologic has joined #circuits10:28
*** spaceone has quit IRC10:28
*** koobs has quit IRC10:33
*** jgiorgi has quit IRC10:33
*** prologic has quit IRC10:33
*** spaceone has joined #circuits10:33
*** spaceone has quit IRC10:34
*** black_rez has quit IRC10:34
*** koobs has joined #circuits10:34
*** jgiorgi has joined #circuits10:34
*** prologic has joined #circuits10:34
*** spaceone has joined #circuits10:41
*** FSX has joined #circuits10:41
*** black_rez has joined #circuits10:41
*** ChanServ has joined #circuits10:41
*** ronny_ has joined #circuits10:44
*** ronny has quit IRC10:48
*** EasyMac_ has joined #circuits14:29
*** EasyMac_ has quit IRC16:43
*** EasyMac_ has joined #circuits17:17
*** EasyMac_ has quit IRC18:46
*** EasyMac_ has joined #circuits19:58
prologicEasyMac_, hey20:34
prologicregarding the tick handlers stuff20:34
prologicto clarify, in previous versions we used to have a decorator called @tick20:34
prologicwhich created a special event handler that got called on every event loop cycle (tick)20:34
prologicwe removed this in favor of listening to and intercepting GenerateEvents isntead20:35
prologicI general, it's probably not a good idea to define a method called tick20:35
prologice.g:20:35
prologicdef tick(self):20:35
prologic   ...20:35
prologicReason being. Can't guarantee that your method will be called (if at all) because tick() is an internal method of the running Manager - the Component/Manager responsible for running a component graph (a set of components registered in some combination)20:36
prologicHope this makes sense :)20:37
*** koobs has quit IRC21:14
*** koobs has joined #circuits21:14
*** koobs has quit IRC21:14
*** koobs has joined #circuits21:14
*** jgiorgi has quit IRC22:59
*** jgiorgi has joined #circuits23:00
*** EasyMac_ has quit IRC23:06
*** jgiorgi has quit IRC23:52
*** jgiorgi has joined #circuits23:52
*** jgiorgi has quit IRC23:52

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!