IRC Logs for #circuits Tuesday, 2013-03-19

prologickoobs, whenever you're ready00:11
prologicspaceone, I cannot pull in and merge your branch still - tests fail. Tests hang at tests.web.test_unicode on Python 2.600:14
prologicI would pull/merge anyway and try to fix them, but you've made far too many changes to the header handling that I can't follow it all that well00:15
prologicThe headers implementation in circuits.web was directly borrowed and integrated from wsgiref (I could/should have just improted from wsgiref as it's std. lib - but anyway)00:15
prologicThe fact you want to re-implement it all has a big impact on every part of circuits.web00:16
prologicI'm okay with that, but you'll have to make sure every test passes before I merge into mine :) 2.6/2.7/3.2/3.3/pypy (use tox)00:16
prologicTBH though, I'm not exactly sure what re-implementing header classes buys us exactly - I'm not sure you're particularly improving the API per say , only internals?00:17
koobsprologic; go login00:18
prologicwhat was the ip again?00:18
prologicon my desktop :)00:18
*** Osso has joined #circuits00:52
prologicAny good with BSD Osso?01:03
prologicOur KQueue poller has a few bugs apparently01:03
Ossooh ?01:04
Ossoit's working fine here01:04
prologicthrowing 22 Inavlid Arguments errors01:04
prologictry running siege against a circuits.web server with KQueue01:04
prologicthen try starting it multiple times in process mode01:04
prologicthis throws 22 Invalid Arguments errors01:05
prologicand you uncomment out the multi-process bits, it also does01:06
prologicjust a lot quicker01:06
prologicI'm seeing these errors atm:01:15
prologic(single process)01:15
prologiceven EPoll is broken :/01:18
prologicare we not testing this stuff01:18
prologicOsso have a look at
prologicwhy are we doing:01:20
prologic   from circuits.core.pollers import EPoll01:20
prologic   EPoll()01:20
prologic   metafunc.addcall(funcarg={"Poller": EPoll})01:21
prologicexcept AttributeError:01:21
prologic   pass01:21
prologicwhy are we doing EPoll()01:21
prologicif there's a bug with intialization (which I'm seeing)01:21
prologicwe'll never perform the tests01:21
prologicit was totally my fault01:23
OssoI am checking kpoll01:24
Ossoit's working fine here01:26
prologicok that's odd01:29
prologicI'm seeing errors on koob's bsd box01:29
OssoI am wondering how _ctrl_recv is an int in your case01:29
prologicI pushed a change that means EPoll will now fail it's tests01:33
prologicyes so am i01:33
koobshey ossy01:36
koobswe could be hitting an fd limit now01:36
koobsim isolating01:36
koobsbe nice to know which kqueue call is failing01:36
koobsmax files and maxfilesper proc are relatively high01:39
koobskern.maxfiles: 32000001:39
koobskern.maxfilesperproc: 28800001:39
koobshave a read.01:51
*** koobs has quit IRC03:48
*** koobs_ has joined #circuits03:48
*** Osso has quit IRC03:50
*** Osso has joined #circuits03:56
*** ronny_ has quit IRC04:30
*** ronny_ has joined #circuits04:47
*** ronny_ has quit IRC04:47
*** ronny_ has joined #circuits04:47
koobs_Lifting the server siege..      done.                                                                                                                             Transactions:                  491720 hits05:27
koobs_Availability:                 100.00 %05:27
koobs_Elapsed time:                  36.51 secs05:27
koobs_Data transferred:           19638.81 MB05:27
koobs_Response time:                  0.01 secs05:27
koobs_Transaction rate:           13468.90 trans/sec05:27
koobs_Throughput:                   537.93 MB/sec05:27
koobs_Concurrency:                   96.0005:27
koobs_Successful transactions:      49172105:27
koobs_Failed transactions:               005:27
koobs_Longest transaction:            0.0805:27
koobs_Shortest transaction:           0.0005:27
koobs_just making sure the box is tuned for you05:28
koobs_and by box i mean my widdle VM05:28
koobs_lol i think i broke it on core dump05:32
koobs_Concurrency Level:      100005:55
koobs_Time taken for tests:   2.078 seconds05:55
koobs_Complete requests:      5000005:55
koobs_Failed requests:        005:55
koobs_Write errors:           005:55
koobs_Keep-Alive requests:    5000005:55
koobs_Total transferred:      2145916806 bytes05:55
koobs_HTML transferred:       2133578436 bytes05:55
koobs_Requests per second:    24061.59 [#/sec] (mean)05:55
koobs_Time per request:       41.560 [ms] (mean)05:55
koobs_Time per request:       0.042 [ms] (mean, across all concurrent requests)05:55
koobs_Transfer rate:          1008479.73 [Kbytes/sec] received05:55
koobs_ok, i think its ready for circuits.05:55
*** Osso has quit IRC06:15
*** Osso has joined #circuits07:22
*** ronny_ has quit IRC07:27
*** ronny has joined #circuits07:44
*** ronny has quit IRC07:44
*** ronny has joined #circuits07:44
*** Osso has quit IRC09:25
*** millerti has joined #circuits09:35
jgiorgiwait, has priority been reversed at some point?10:20
jgiorgididnt lower used to mean run first, now it's higher?10:20
*** ronny has quit IRC10:36
*** ronny has joined #circuits10:49
jgiorgiprologic: about? i doing something wrong with Stream10:56
jgiorgiseems to be two problems, shouldnt that HTTP header be at the beginning of the document, not the end11:07
jgiorgisecond the content-length is 999 on the second header11:08
jgiorgi that's the entire debugger output of the session, there's a few errors i can't figure out at the end11:10
spaceoneand your code?11:13
jgiorgiis massive, the only relevant parts or setting response.Stream = True and firing the two Stream events11:14
jgiorgi(first codepad)11:14
spaceoneis this the complete output of curl -i ?11:14
jgiorgicurl, didnt use the -i flag11:14
spaceoneadd it11:15
spaceoneamd show what is returned then11:15
jgiorginote that the curl session doesnt exit, it hangs waiting for more data11:16
spaceoneof course11:16
*** Osso has joined #circuits11:16
jgiorgiusing the tip (rev 3665:42e62ab9f8dd) of circuits (not circuits-dev)11:17
spaceonejgiorgi: is it circuit-dev ?11:17
spaceoneah ok11:17
jgiorgiis this fixed in circuits-dev? i can switch to that environment but my headaches always tend to multiply11:17
spaceonei don't think so11:18
jgiorgiwell if anyone has any idea what i'm doing wrong i'd appreciate it11:20
jgiorgithis Component worked in a very very old version of circuits.web11:20
*** Osso has quit IRC12:17
*** Osso has joined #circuits14:43
*** Osso has quit IRC14:43
prologicjgiorgi: its always been igher no. the higher the priority14:53
prologicjgiorgi: yopu have errors14:55
prologicso youre seeing streamed output followed by an error response14:56
jgiorgii cant figure out what those errors mean, they dont appear to involve my code14:57
*** cpt_mojo has joined #circuits15:01
*** ronny has quit IRC15:08
jgiorgiprologic: can you tell me what's causing those errors?15:08
prologicjgiorgi: somewhere youre returning a list ir something15:08
prologicunder normal circumsyances streaming works perfectly fine15:09
prologicif you find an edge case lemme know!15:09
jgiorgii'm not returning a list anywhere15:09
jgiorgisequence is15:10
prologicmultiple tesponse?15:10
jgiorgii'm firing multiple Streams, one with data one with None15:10
jgiorgifirst) set request.fc_vset, return None (from a filter on Request)15:11
prologicwhat about the inital request handler?15:11
jgiorgisecond) set = True, fire a couple events and return None (another Request filter)15:11
jgiorgifinally) render the template, stream it, stream None15:12
jgiorgino Controllers involved anywhere, just a couple Request filters and a Component that fire's Stream events15:12
prologicyou have two resquest handlers doing something with the request hefore you stream?15:13
jgiorgiactually 3 if you count Sessions15:13
prologichmmm ok i think i know whats going on15:13
prologiceach request handler is adding to the resquests value15:14
prologicso the value becomes [None, None, None]15:14
prologica condition is failong in this case and an error is being raised15:15
prologica corner case we did not take into acvount15:15
jgiorgihrmm i didnt consider that circuits.web would care about the value of the Request event15:15
prologicill look into it15:16
prologicand let you know15:16
jgiorgiweird, because i'm writing even more filters for the Request object15:16
jgiorgitemporary workaround? can i set that value within the Request handler?15:16
prologiclet me explain better when i get to work?15:16
prologicim in the car atm going to the doctors15:17
jgiorgiit's no rush just debugging FrankenCircuits15:17
*** koobs_ has quit IRC16:09
*** koobs_ has joined #circuits16:09
*** koobs has joined #circuits16:10
prologicspaceone:  ping?17:01
*** millerti has quit IRC19:06
*** ronny has joined #circuits21:17
jgiorgiso i'm doing a bit of reimplementation on my bridge due to some flaws in the event routing, while i'm at it i'm redesigning the protocol a bit and supporting multiple encapsulation methos21:25
jgiorgibson, json and pickle/cpickle, in that order of preference, reencapsulating if the next link doesnt support the existing encapsulation even if it's not the best available21:26
jgiorgiit's a bigger change than i thought but it's a good idea21:27
prologicwell I think the serialization should be explicit21:36
prologici.e: you setup a bridge node on X and explicitly say you want to use BSON21:37
jgiorgithat option exists21:37
jgiorgiyou can set an individual node to only support one or two encapsulations and to change the default preference21:38
jgiorgionce it leaves that node the encapsulation will only change if it needs to traverse a link that doesnt support the current encapsulation21:38
jgiorgiof course, provided it's available it would be best to support all encapsulation modes to prevent having to reencapsulate multiple times, encapsulation is fairly procesor intensive21:39
prologicI would just make it explicit and only use one serialization all the way through21:40
jgiorgithe intention is for the developer to have as much control over the network as possible but let a lot of the nitty gritty to be handled automatically unless configured otherwise21:40
prologicwhere json_serializer is a function that takes an event object as input and returns binary data21:41
jgiorgiyou have the option to configure all of your nodes to only use one serializer21:41
jgiorgibut that's a choice you make, not a mandate by the system21:41
prologicI think in terms of implementation and maintainence21:41
prologicthat's simpler :)21:41
prologicyou don't generally want to mix serializations I don't htink21:42
jgiorgiconsider a network where some nodes can only support one serializer or another21:42
prologicyeah what I mean is keep the Bridge implementation simple21:42
prologicif such a case occurs21:43
prologicthe application developer can worry about it21:43
jgiorgiit doesnt make sense to force the rest of the nodes to use an inefficient method due to the needs of one segment21:43
prologicalso it would be nice to build a Javascript version21:43
prologicthat uses websockets21:43
jgiorgihaving the ability to mix and match serialization methods makes alternate implementations easier21:43
prologicAll nodes should use the same serialization no?21:44
prologicI would21:44
jgiorginot necessarily21:44
prologicI think you're making this more plicated than it needs tone :)21:44
prologicbut ok :)21:44
prologicCould you test some of your apps/code on the latest circuits-dev branch?21:45
prologicI've merged circuits-experimental into it today21:45
prologicand all tests are passing on all platforms and environments21:45
jgiorgitbh, i dont use your bridge because it dumps too much in my lap, the entire reason i started this project was to create an overcomplicated message passing architecture that handled a wide range of circumstances without troubling me with the details21:45
jgiorgii can spin up a couple things on circuits-dev sure, it's just a quick environment swap21:46
prologicappreciate it21:47
prologiclet me know how it goes21:47
prologicand if anything breaks for you21:47
jgiorginp running a few tests now21:48
jgiorgiany word on a workaround for that streaming issue21:48
prologicsweet build passed on shining panda21:48
prologicall 1438 tests21:48
prologicI haven't looked at it yet - sorry21:49
jgiorgii'd like to have it working even if i just have to hack something21:49
prologicI will try to tonight21:49
jgiorgi1438, that's a lot of tests lol21:49
jgiorgiWell all my tests pass, everything seems function21:50
jgiorgivery different behavior from stream21:50
jgiorgistill broken but appears better, looks like the errors did get streamed21:51
jgiorgiheader is still after the html document (i think that's wrong but maybe i'm confused)21:52
jgiorgiand the connection actually closes properly instead of hanging21:52
jgiorgiwell i'm off to bed22:02
prologicg'night :)22:02
prologicspaceone:  I've fixed the ssl handshake problem for SSLv322:19
prologicI just need to figure out how to detect SSLv2 now22:19
prologicWe just close the connection if we detect an ssl handshake to a non-ssl server22:19
prologicI found trying to respond with 400 Bad Request wasn't understood by any clients22:19
spaceoneprologic: yes, closing the connection is right,,,22:37
spaceoneprologic: the header sorting thing: yes, the order does not matter, but it is good to send the server related headers first22:38
spaceone(Date, Server)22:38
prologicyeah well22:38
prologicwe're not going to do that :)22:38
prologiclet's just leave it as a CaselessDict22:39
prologicI've merged your changes in btw22:39
spaceonethis would not be hard to imlpement22:39
prologicand fixed everything up22:39
prologicI've also merged experimental inot dev22:39
spaceonei have to go to work now22:39
prologicand all tests are passing on all platforms and enviornments22:39
prologicI plan to delete experimental soon22:39
prologicI also fixed the ssl thing22:39
prologicbut I still don't have a test for SSLv222:39
prologiconly SSLv322:39
prologicbut aside from that I think we're good22:40
prologicok cya l8r22:40
prologichave a nice day at work22:40
koobsprologic; see my stats from last night?23:13
*** millerti has joined #circuits23:22

Generated by 2.11.0 by Marius Gedminas - find it at!