IRC Logs for #circuits Friday, 2013-01-04

*** platzhirsch has joined #circuits00:04
prologichowdy again00:32
prologicI added three examples in examples/primitives/00:43
prologicdemonstrating the three circuits' primitives00:43 .call(...) and .wait(...)00:43
platzhirschprologic: perfect!00:45
prologicI assume you're up to speed with the dev repo?00:45
prologicI mean - found it?00:45
prologicLet me know if you have questions00:48
prologicGlad to help00:48
*** platzhirsch has quit IRC01:11
*** Osso has joined #circuits01:15
prologichey Osso01:17
prologicdid you check out that stuff I posted last night?01:17
Ossonot yet01:17
prologicyou aware we have irc logs now?01:17
prologiclink is in #circuits-dev topic01:18
prologic:: Logs: ...01:18
prologiccatch up with that, then say hi :)01:18
prologicwait lol01:20
prologicwhich bit(s) ? :)01:20
Ossoit's great that we have logs01:23
OssoI have to go out right now but I can check the content of the logs later01:23
Ossoand you got the handlers stuff done too!01:25
prologicI was playing with that a bit01:26
prologic.handles("...) seems nice to have01:26
prologicie: you can query a Component's "itnerface" before it's registered01:26
prologicor after01:26
*** Osso has quit IRC02:50
*** Osso has joined #circuits04:10
OssoI am back04:23
prologicI can see that :)04:23
prologicI've pushed
prologica wrapper for subprocess.Popen04:23
prologicseems to work well04:23
prologicexcept for one thing04:23
prologicthe test passes 9/1004:23
prologicbut every now adn again it fails04:23
prologicsomething seems to get stuck04:24
prologicalso component instance targeting04:24
prologicI think we should remove it04:24
Ossothere must be some race condition04:25
prologicthat's what I'm thinking04:26
prologicyou seem to be better at hunting those down04:26
prologicso :)04:26
prologicI've been slowing rewriting tests (as it demands so) using the new manger/watcher fixtures btw04:27
prologiccomponent instance targeting?04:27
prologicremove it?04:27
Ossonot sure04:27
Ossoit does not play nicely but it is really useful04:28
Ossoif we generate a unique channel for each component04:30
Ossowe can replace it I suppose04:30
Ossobut we have to look up in a crazy dictionary04:30
prologicI'm unsure also04:32
prologicit does add unnecessary compilcations though04:33
Osso"test" is always failing here04:33
prologicfor test_process ?04:33
prologicit passes here 9/1004:33
prologicsee if it works by hand on your env?04:33
prologicsample code you can run04:33
OssoI get '' as return value from stdout04:36
OssoI see04:37
prologicthat sample works for me04:37
OssoI am missing hello.py04:38
prologiccopy it from examples/04:38
prologicdid you hg fetch?04:38
prologicand btw04:38
prologicthere is hg pull --rebase04:38
Ossooh cool04:39
prologicexactly the same as git's rebase04:39
prologicthought I'd let you know :)04:39
Ossodoes always work ?04:39
prologicgood question04:40
prologicit ought to unless there's something wrong with our shutdown sequence04:40
prologicalso I notice I can't raise SystemExit(0) inside a handler anymore04:41
prologicI had to use self.stop()04:41
prologicwhich is odd04:41
Ossook yeah it does work04:42
prologicyeah it should04:43
Ossothere's only 1 generate events happening04:50
Ossomore precisely GenerateEvent is fired but04:51
Osso_on_generate_events is not called04:51
Ossopriority is -9 which is conflicting04:55
Ossonot conflicting but no racing against a poller04:55
Ossobut I don't see any poller registered04:56
prologicmy bad?04:56
prologicwhat should the priority be?04:56
Ossojust 0 is good and remove the filtering04:57
prologicthere most certainly should be a poller04:57
prologicbecause of the 3 File components04:57
Ossonow it always fails with the same error which is good!04:58
prologicspeaking of race conditions04:58
prologicI'll let you finish :)04:58
prologicI'm seeing how badly removing component instance targeting will be04:59
Ossoyeah it is fine the race condition is fixing as soon as you remove priority/filter04:59
Ossother reason it is failling here is that it can't import circuits05:00
Ossocause it is not in python path05:00
prologicI give up05:20
prologicremoving component instance targeting is not easy05:20
prologicso we should fix it :)05:20
prologicwhat? :)05:23
Ossowhere are we using it ?05:23
prologicgod only knows05:23
prologicI tried to remove it05:23
prologicand remove any places I could see that was using it05:23
prologicbut alas too many tests failing for no good reason05:24
prologicwhich seem to have nothing to do with component targeting05:24
prologicsomehow we've done something special with it :)05:24
prologicie: it sort of half works05:24
prologicall tests pass here05:24
prologiconly test_component_targeting fails here05:24
OssoI am failing to have circuits imported by hello.py05:25
prologicfor test_process?05:26
Ossoit is using the wrong python for some reason05:26
Ossomy virtualenv seems lost05:28
prologicin the test I make it use sys.executable05:29
Ossosys.executable is wrong here05:31
Ossoor wait05:31
Ossono it is right05:31
Ossowait it works now ?!05:32
Ossoanyway it is passing here now05:33
Ossocomponent targetting should be easy to fix anyway05:33
prologiccan you fix it?05:37
prologicI tried and failed :/05:37
prologicthen I thought about removing it and fail :)05:37
*** Osso has quit IRC05:40
*** Osso has joined #circuits05:40
*** Osso has quit IRC06:34
*** Osso has joined #circuits07:09
*** Osso has quit IRC07:51
*** Osso has joined #circuits07:51
*** Osso has quit IRC09:54
*** Osso has joined #circuits12:05
*** Osso has quit IRC13:12
*** Osso has joined #circuits13:33
*** Osso has quit IRC13:38
*** Osso has joined #circuits13:39
*** Osso has quit IRC14:38
*** Osso has joined #circuits18:34
*** Osso has quit IRC19:34
*** Osso has joined #circuits08:09
*** Osso has quit IRC09:16
*** Osso has joined #circuits09:24
*** Osso has quit IRC10:25
*** Osso has joined #circuits11:32
*** Osso has quit IRC11:32
*** Osso has joined #circuits11:33
*** Osso has quit IRC12:33
*** Osso has joined #circuits12:58
*** Osso has quit IRC17:34
*** Osso has joined #circuits07:23
*** Osso has quit IRC07:24
*** Osso has joined #circuits08:17
*** Osso has quit IRC10:05
*** Osso has joined #circuits10:46
*** Osso_ has joined #circuits11:45
*** Osso has quit IRC11:47
*** Irinix_Away has joined #circuits11:56
*** Osso has joined #circuits13:26
*** Osso has quit IRC13:28
*** Osso has joined #circuits13:29
*** Osso_ has joined #circuits13:36
*** Osso_ has quit IRC14:37
jgiorgiprologic: circuits.web file upload, is file.value a file-like object (ie is it iterable) or is it just a str?16:22
prologicit is a file-like object16:33
prologicit comes from cgi.FieldStorage16:33
jgiorgicool, life is easy16:44
prologicof course it is :)17:09
prologicwe try to wrap as much standard stuff as we can17:09
prologicand make things as simple as possible :)17:09
*** Osso has quit IRC17:24
jgiorgii appreciate it, it takes a lot less time to implement anything that way18:17
jgiorgiprologic: i remember the old serve_file, is there an equivalent to serve from a file-like object?18:38
jgiorgii'm serving from GridFS so i'd prefer not to load it, save it to file and serve it18:39
prologicjust return the file like object in your response handler18:39
prologiccircuits.web will deal with that18:39
prologicad automatically stream it18:39
prologicit'll stream is as an octer-stream18:40
prologicas expected18:40
prologicpretty sure anyway :)18:40
prologiclet me know how you go18:40
jgiorgithen i just need to set the self.response.headers["Content-Type"] and i'm all set18:40
prologicI'm pretty sure you don't even have to do that18:44
prologicit'll handle all that for you18:44
prologic*I think*18:44
prologiccheck the unit tests18:44
jgiorgireally, how? don't tell me you analyze the data for headers?18:44
prologicyeah we do18:45
prologicwe look at the file object18:45
prologicand determine a few sensible things about it18:45
prologicset response headers appropriately18:46
prologicthen start streaming it's data18:46
prologicI'm almost positive we do support that18:46
jgiorgiwell cool, that saves me some time18:46
prologicok on18:49
prologicwe don't set any response headers like that18:49
prologicI don't think18:49
prologicby looking at the code18:49
prologicbut we do support automatic streaming of file like objects18:49
prologicbut do double check the unit tests18:49
prologicand test it yourself :)18:49
prologicmaybe that could be a new feature!18:49
prologicI just did a quick test18:51
prologicand yeah you do have to set Content-Type and optionally Content-Dispotition if you need that tooo18:51
prologicbut yeah it totally works18:51
jgiorgisweet, i'll set the content type manually, that's one line19:46
jgiorgiprologic: got an unexpected keyword argument error with a web form but the keyword is expected in index ( index(self, user=None, passwd=None, redirct="/") )21:50
jgiorgiare passwords treated differently or something?21:51
prologiccheck what you are actually sending it21:51
prologicthis is well unit tested21:51
prologicand not based on code we per say wrote21:52
prologicyou might be getting something else you're not expecting21:52
jgiorgior it might be going somewhere im not expecting21:53
jgiorgithink i found it21:53
jgiorgiit was me being stupid, wrong action22:14
jgiorgiso same error trying to upload a file, unexpected kwarg csv, but the traceback says it's going to index()22:55
jgiorgiprologic: the action is /expert/bulkresult so i dont get it22:56
jgiorgithere is a method bulkresult in /expert22:56
jgiorgiok and i have one file upload working but it's nothing like that of the file upload example23:06
jgiorgii accept the argument csv which should be an uploaded file, all i get is the file name in str format, no data23:06
jgiorgiok second problem fixed, man i gotta get my head around circuits.web again23:38
prologicjgiorgi, figured it out?00:42
prologiccsv won't contain the file00:43
prologiconly the filename00:43
prologicthe file will be in the FieldStorage00:43
prologicjgiorgi, please remind us to document this a bit better00:45
prologicso it's more obvious00:45
prologicbut the resulted file will be in self.body00:45
prologicand all the fields in **kwargs00:45
*** zsolt has joined #circuits02:19
*** prologic has joined #circuits02:19
*** prologic has quit IRC02:19
*** prologic has joined #circuits02:19
zsoltgot a cardbus expresscard converter and a cardbus tv tuner card.probably my tv tuner doesn't get enough energy.what to do?02:20
*** Osso has joined #circuits02:30
*** Osso has quit IRC02:34
*** Osso has joined #circuits02:35
*** Osso has quit IRC02:35
*** Osso has joined #circuits02:36
*** zsolt has quit IRC02:37
*** Osso has quit IRC03:39
*** Osso has joined #circuits05:34
*** Osso has quit IRC05:39
*** Osso has joined #circuits05:39
*** Osso_ has joined #circuits05:40
*** Osso_ has quit IRC06:11
*** Osso_ has joined #circuits06:12
*** circuits has joined #circuits06:17
*** circuits_ has joined #circuits06:17
*** circuits has quit IRC06:20
*** circuits_ has quit IRC06:20
*** Osso_ has quit IRC06:40
*** Osso has quit IRC07:47
*** Osso has joined #circuits07:48
*** Osso has quit IRC08:38
jgiorgiprologic: the problem with just getting the filename was not setting enctype in the form, i have a few errors but it looks like i've got the file and it's just parsing incorrectly09:04
jgiorgiprologic: still cant figure out the unexpected keyword going to index09:05
*** Osso has joined #circuits09:26
*** Osso_ has joined #circuits09:32
*** Osso has quit IRC09:36
*** Osso has joined #circuits13:46
*** Osso_ has joined #circuits13:47
*** Osso has quit IRC13:48
*** Osso_ has joined #circuits13:49
*** Osso_ has quit IRC13:59
prologicjgiorgi:  can you paste a working sample that shows this problem?14:43
jgiorgican't seem to accurately reproduce the error14:47 was in template inheritance... f-ing mako14:48
jgiorgiprologic: hows multiprocess support in circuits.web nowadays?18:00
prologicshould work fine18:57
prologicjust shares a listening socket18:57
jgiorgiis there an example and is there anything i need to know? is the entire system available to each process as if it was a single process?19:13
prologicsystem = Server(8000) + Root()19:15
prologicI forgot process=True19:15
prologicNB: The last .run()19:15
prologicthis is ay if you had a 4-core server19:15
*** qwebirc4073 has joined #circuits19:17
marky1991prologic:  Would you happen to be on by any chance?19:18
prologicI'm always on :)19:19
prologichi Mark!19:19
marky1991o have a question19:19
marky1991where is the __tick__ method called from?19:19
prologicit isn't anymore19:19
prologicit's gone19:19
prologicforget about it :)19:19
marky1991Crap. I'll have to update my repo then19:19
prologicyeap :)19:19
marky1991I've almost got asynchronous io working on windows19:19
prologicI think we're trying to sort out some race condition issues to do with multithreading19:20
marky1991turns out seect doesn't work with file o=bjects in windows19:20
prologicnot that circuits is multithreaded19:20
prologicbut a lot of our tests access a threaded system externally19:20
marky1991are you okay with having ctypes used on windows?19:20
prologicand we're trying to reintroduce a feature we broke in dev where19:20
prologicthe system would sleep if there was nothing to do19:20
prologicthus saving many cpu cycles19:21
prologicctpyes on windows?19:21
prologicif you can make File work on windows asynchonrously19:21
prologickudos to you :)19:21
marky1991i already have file working mostly19:21
prologicI'm impressed :)19:22
marky1991i'm just getting strane errors when making the write event work properly19:22
prologiclike I said, I'm no windows dev19:22
marky1991it starts the write handler, then just stops after a couple of lines19:22
prologicI suggest updating your repo19:22
prologicI made change to the File component too19:22
marky1991i'll update and see if it fixes it19:22
prologicso it's in line with the rest of the I/O components in circuits19:22
prologicie: uses the poller components19:22
prologicbefore it did it's own polling19:22
prologicwell updating may not fix your problem19:23
prologicbut update so you're up-to-date with us :)19:23
marky1991do you have any idea what could cause my problem?19:23
marky1991it literally just stops while doing nothing significant19:23
marky1991i was thinking it was related to a timeout somehow, but that doesn't make much sense19:24
marky1991since the write handler itself doesn't havea  timeout19:24
prologictbh no I'm not sure19:24
prologicif you do update though19:24
marky1991I'll update and see if that somehow fixes it19:24
prologicand adapt your changes to the new File componnet19:24
prologicyou should see that it automatically now uses the system poller19:24
prologicor if it doesn't have one will create the default one19:24
prologicjust like
marky1991not seeing a new version19:27
marky1991did you seay you updated
marky1991the newest versionlooks like it's from march19:28
marky1991i'm not seeing any new coimmits since 27 days ago...19:29
prologicyou're not using the dev repo are you? :)19:29
marky1991ooh, i think i saw that earleir19:30
marky1991no, i was not.19:30
marky1991how far backis the other one?19:30
prologicwell it's not that different pe rsay19:30
prologicbut it's many revisions old now (stable)19:30
prologicyou're best taking what you've done so far and putting them into a patch queue or something19:31
prologicand reapplying to dev19:31
jgiorgiis it that easy to multiprocess any circuits application?20:12
prologicwell umm20:12
prologicsort of20:12
prologicit's a property of most POSIX systems I guess20:12
prologicin that you can share a listening socket between multiple proesese20:12
prologicI'm still working on better bridging and worker processes20:13
jgiorgiok so does circuits just clone itself across the processes? is data available cross process?20:14
jgiorgiright now i use an environment object to track some basic data across components/controllers, will that break?20:14
prologicit's effectively a fork20:29
prologicusing multiprocessing.Process20:29
prologicdon't assume anything :)20:29
prologiconly the sharing listening socket is special20:29
prologicI'm going to talk to Osso and mehere20:29
prologicbut I'm advocating for bringing back the Bridge20:30
prologicto enable one to bridge processes together20:30
prologic-however- the only thing going across the bridge would be events of course20:30
prologicI'd also like to keep circuits.node20:30
prologicbut as a different use-case20:30
prologicfor connecting remote nodes to a system20:31
prologicsimilar in behavior to a Bridge20:31
prologicbut for remote systems20:31
jgiorgifair enough, i'll implement a pre-loading system20:31
prologicI'm also advocating for a complete rewrite of circuits.core.workers and circuits.core.pools20:31
prologicwhereby the new Worker is simply a wrapper around multiprocessing.Proces for doing work20:32
jgiorgiarent you the commander in chief of this project? advocating... hahah20:32
prologicie: CPU bound "work"20:32
prologicand Pool managers several or more workers20:32
prologicbut they processes they fork do not participate in events20:32
prologicthey simple perform "work"20:32
prologiccommander in chief me ass20:33
jgiorgilast i checked your name's in the url20:33
prologicmy knowledge of threading and muteness suck compared to Osso and mehere20:33
prologicthey both help greatly20:33
prologicand are awesome debs :)20:33
prologicthat's the list of changes I' wanting to see for Worker, Pool, Bridge and Node20:34
jgiorgicool, for now i'll just treat it as a shared listening socket20:34
*** qwebirc10668 has joined #circuits20:35
jgiorgii figure if i initialize a web server, fork and then have each fork initialize the rest of the system20:35
jgiorgii can build my environment independently from the same sources and they would all be identical, in theory20:35
marky1991I don't mean to interrupt, but in circuits-dev's manager, what exactly is the purpose fo the bridge? In any case, circuits-dev is currently unrepairably broken for windows. It's start now relies on Pipe, which isn't available on windows. Can this be changed somehow?20:39
prologichang on20:41
prologicthat would only happen if you started in process mode?20:41
marky1991ah, you're right20:41
marky1991missed the if clause.20:41
marky1991Now just need ot see why the tests are triggering process mode20:42
marky1991and disable those tests for windows20:42
marky1991or modify them20:42
prologicideally we need to build a Pipe-like component for Windows20:47
prologicir: an equivilent somehow20:47
prologicthat uses tcp underneath20:47
prologicor similar20:47
prologicso that has some cod elike20:47
prologicif sys.platform == "win32":20:47
prologic   Pipe = WindowsPipe20:48
prologicthat would be ideal I think20:48
prologicthat way birding works in Windows too20:48
*** qwebirc31450 has joined #circuits20:48
prologiceven if it's slower (because of tcp overhead)20:48
*** qwebirc31450 has quit IRC21:04
*** qwebirc58603 has joined #circuits21:04
prologicmarky1991:  did you get what I suggested regarding Pipe on Windows?21:11
prologicThat we build an equivalent component for windows using underlying tcp21:11
marky1991oh, yay21:14
marky1991that's what I wanted to do earlier21:14
marky1991it's not going to be as fast, of course, but it makes it so all of the API is cross-platform21:14
marky1991in a retarded way on some platforms : )21:14
marky1991The more I program with windows, the more I wish it were more unix-like in some places21:16
prologicI agree21:21
prologicbut yeah21:22
prologicif you feel up to the challenge21:22
prologicplease do :)21:22
jgiorgii agree with the winblows hating21:23
*** Osso has joined #circuits21:39
*** Osso has quit IRC22:39
marky1991For the pollers, what is the event parameter in the _generate_events(self, event), call? Am I correct in reading that the generate_events is where the actual polling takes place?23:07
*** Osso has joined #circuits23:42
*** qwebirc41477 has joined #circuits00:12
*** Osso has quit IRC00:42
*** Osso has joined #circuits02:29
*** Osso has quit IRC03:28

Generated by 2.11.0 by Marius Gedminas - find it at!