IRC Logs for #circuits Saturday, 2013-03-16

*** ronny has joined #circuits01:11
*** ronny has quit IRC01:11
*** ronny has joined #circuits01:11
prologichey spaceone02:11
prologichey ronny02:11
prologicI'm almost finished with the set of changes I'm working on02:11
prologicI'm integrating:02:12
spaceoneprologic: ;)02:12
prologicactually I'm using right now02:12
spaceonei hope these libs are good02:12
prologicbecause there's a bug with the other one02:12
prologicthat's why I pasted them02:12
prologicsee for yourself :)02:12
prologicbut they all have good support for 2.6, 2.7, 3.2 and 3.302:13
prologicso that's been the factor in my choices02:13
prologicthat and libs that just do one thing02:13
prologicand I'm taking the cherrypy approach to processing request bodies02:13
prologicscrapping cgi.FieldStorage as it's broken in Python 3.x02:13
prologicthey completely f'd it up02:13
prologicAll 99 web tests pass on 2.6, 3.2 and 3.302:15
prologictook me the better part of 2 days02:15
prologicand one very late night02:15
spaceoneprologic: !!! the pyquerystring is cool1!!02:15
spaceoneit supports ?foo[bar]=baz02:15
prologicI thought you'd like that :)02:16
prologicI didn't particularly care too much02:16
prologicbut hey it seemed to do query string aprsing02:16
prologicand nothing more02:16
prologicMy decision making has been:02:16
prologicQ: Does it already support 2.x and 3.x and have tests that pass02:16
ronnyi think flatland can do something semilar02:17
prologicQ: Does it just do the one thing I need it to?02:17
prologicif it does more than one part of this, I probably won't use it02:17
prologicI'm integrating these libs directly into circuits.web's namespace02:17
spaceone'directly' ?02:18
ronnyprologic, flatland is for shema validation and mapping things02:18
ronnyi get the feeling you are aiming for feature-creep :P02:18
spaceoneprologic: what do y<ou mean by02:18
prologictrying to avoid feature-creep02:19
prologicspaceone, I'm committing the modules into circuits.web02:19
spaceoneprologic: ok, but don't foreget to add the version and revison number this time!!!02:20
spaceonenot that have the same problem like the problem with cherrypy02:20
ronnyprologic, why is depending on them not ok ?02:21
prologicwe never have before02:21
ronnyprologic, by now im strongly opposed to anything that does code inclusion, so much mess02:21
prologicso we're not going to start02:21
prologicpip install circuits02:21
prologiconly installs circuits02:22
prologicno dependencies02:22
prologicthere are optional ones however02:22
prologiclike pyinotify02:22
ronnyi see02:23
prologicronny, it's called code reuse and not reinventing the wheel. these libs are all MIT licensed (as is circuits), so I'm leveraging off the hard work and knowledge of others02:23
prologicOsso, mehere or myself cannot possibly be experts in everything - we just continue circuits development, improve it, add new features, etc02:23
ronnyprologic, its basically creating custom forks, there is a certain guarantee of upstream divergence02:24
prologicI imagine we won't ever need to refer to the upstream versions02:24
prologicwe have never done so when we borrowed code from cherrypy and BaseHTTPServer02:24
prologicit just meant we didn't have to write thousands of lines of code from scratch02:25
spaceoneprologic: but in cherrypy there are fixews which would be useful for circuits02:25
prologicwell there might very well be02:25
spaceonein circuits is a verys old cherrypy verison02:25
prologicbut circuits.web and cherrypy are nothing alike anymore02:25
prologicso it would be impossible to say the least02:25
prologicby borrow - I mean we borrowed "some code"02:26
prologicnot entire modules from cherrypy02:26
prologiceg: (a lot of that was taken from cherrypy)02:26
prologicbut not a lot else02:26
prologicthe original circuits.web.http parsing code was taken from cherrypy.wsgiserver initially02:26
prologicat least the logic of parsing http02:26
prologicdispatchers, exception handling, events02:27
prologicwere all written by us02:27
prologicas well as the servers (on top of
prologicso you can see why we can't really adopt cherrypy changes/fixes anymore02:28
prologicI really do want to use:
prologicbut the later is the one that works right now02:33
prologicI've contacted aventurella already and he says he's going to push some changesets that will fix the bug I found02:34
prologicbut I want to use his because it's simpler and less code to maintain02:34
prologicplus I'm not sure about all this fancy stuff in defnull's version02:34
ronnythe bottle guy?02:34
prologicI have no idea :)02:34
ronnyah, yeah he is02:35
prologicwhich one?02:35
ronnyi avoid his stuff02:35
prologicahh k02:35
prologichis version (although does work) is a lot of code02:35
prologicand does a lot of fancy sh*t02:35
prologicI want to use Adam's version02:35
prologicwhich is a simple state machine02:35
prologicand only does multipart parsing02:36
prologichopefully he'll have a fixed version for me which I can integrate02:36
prologicI just didn't want to fix his code too soon before I understood how it worked deeply02:36
ronnyanyway, back later02:39
spaceonei am also for the avanturella version!02:39
prologicthat's my feeling too02:40
prologicI'm going to push what I've done so far to02:40
prologicWould you mind testing things?02:40
spaceonei will integrate the latest things in SF, so i will also test02:43
prologicI'll let you know when I've pushed it up02:43
prologicthe url as per above02:43
prologicI like seeing all tests pass on all environments :)02:46
prologicjust rerunning tox again02:47
prologicbefore I push02:47
prologicAlso this major change02:47
prologicremoves the automatic 100 Continue response as well02:47
prologicso your application will have to handle that now02:47
prologicNB: There is no sensible way for the server to determine if your application does handle it or not and provide a response from the server if the app doesn't02:48
prologicSo it's entirely up to the application being run now02:48
spaceonehm, yep02:48
prologicmake sense?02:48
spaceonenot at all02:48
spaceoneyou can test it02:49
spaceoneif '02:49
spaceoneif header['Expect'] == '100-continue' and response.status not in (100, 417): response.status = 41702:50
spaceonebefore responding02:50
prologichmm hang on02:50
spaceonethe app must not execute the method02:50
prologichang on02:51
prologicso what you're saying is:02:51
prologicif the app does not respond with the proper status for a Expect header02:51
prologicthe server should respond with a 417 ?02:51
prologictoo easy02:51
spaceonethat would be wrong, because if the app does not handle this, the app MUST not be executed02:51
spaceoneSF makes it easy to handle this :D02:52
prologicyeah ok02:52
prologicbu in general we can't do that02:52
prologican app is an app is an app02:52
prologicif you write your own custom dispatcher (as I've seen you do)02:52
prologicthat's fine02:52
prologicbut in normal circutis.web the Request event will be fired regardless02:53
prologicbut the logic you presented earlier I can do02:53
spaceonewhat about adding a event 'Expect' ?02:53
spaceonewhich is handled by default from the server02:53
spaceonewith responding 41702:53
prologicand the default behavior being?02:53
prologicnah I can already handle that in the response handler02:53
spaceonebut which is overwritable by dispatcher / etc02:53
prologicbe checking the request Expect header02:53
prologicand checking the response status02:53
prologicit's a simple 2-liner02:53
prologicWill this work?02:56
prologicOr do you need an event you can filter?02:56
prologicI'm not 100% sure how firing an Expect event would work exactly02:56
prologicit's kind of a specialized request of sorts02:56
prologicI'm just going to commit and push this for now02:56
spaceonei will do code review then02:59
prologicIt's been pushed.03:00
prologicTry it out with your stuff03:02
prologicAnd let me know03:02
prologicI'll be up for a few more hours yet03:02
prologicBut not as late as last night :)03:02
prologicThere _should_ be no breaking changes03:02
prologicNo unit test had to be changed for all this.03:02
spaceonewhat is the time?03:02
prologicThat was my other criteria too03:02
prologicTo not break backwards compatibility03:02
prologicSat Mar 16 21:01:21 EST 201303:03
prologicAlso NB: circuits.web will now respond to invalid requests03:03
prologici.e: If an invalid request is given and it blows up somehow, you'll still get a response03:03
prologicI modified the circuits.core to allow picking up of "Failed Events" in error events03:04
prologicSo your secure/inscure server in one should work03:05
prologicas expected03:05
prologicjust be sure to use a different channel for each server03:05
prologicI added a unit test to ensure this does in fact work as expected03:06
prologicbbs, gonna take the MB to bed03:06
prologicok I'm here03:11
prologicWhat about this for a URL Object, parsing and general URL handling?03:20
prologicSupports all Python versions we need03:20
prologicAnd is Unlicensed so integration is not a problem03:21
spaceonemaybe too bloatet03:22
spaceonebut i will look at it03:23
prologicI think it's along the same goals as url.py03:24
prologicYes slightly larger implementation I agree03:24
prologicBut not overly03:24 unfortunately doesn't work very well with Python 303:25
spaceoneyes, i would prefer it when the alternative would be ''03:25
prologicon 2nd thoughts it is a little larger than we need03:27
prologicwe only need url parsing/handling03:28
ronnyurlobject is a complete insane mess :)03:28
ronnyit inherits from text-type03:28
prologicI'm more linclinded to go with url.py03:28
prologicif only the code were clearner03:28
prologicand worked on Python 303:28
prologicMaybe I could write my own03:28
ronnyprologic, for werkzeug i need to make a new uri/iri lib anyway03:28
prologichow long?03:29
spaceonewe dont need to parse things like  user:pass@host03:29
prologicwill it only handle url parsing and handling?03:29
ronnyprologic, and it seems like its necessary ot include urllib.parse from python3, due to the need to handle bytes vs text03:29
spaceoneor #fragments03:29
ronnyso you only need basic things like join?03:29
spaceoneoh hmm, wait03:29
spaceonewe could use these features to build an uri03:30
spaceonethat would be nice03:30
prologicwe don't need to parse user:pass@host?03:30
spaceoneprologic: because we don't get such uri03:30
prologicronny: just url parsing and handling (e.g.: escape, encode/deocde, relative/absolute and basic properties)03:30
spaceonewe are only getting host from Host/X-FORWARDED-FOR header or from the GET line03:31
spaceoneand we only get the path without fragment in the HTTP line03:31
prologicthis is true03:31
prologicit's always03:31
prologicGET / ...03:31
spaceoneGET http://foo/bar03:31
ronnyprologic, so host, path, join/split encoding03:32
prologichow would you do http://foo:bar@localhost:8000/admin03:32
prologicif we didn't parse user:pass@host ?03:32
ronnyprologic, thig is, urls are bytes, iris are unicode, propper handling/transform is tricky with the stdlib03:32
prologicronny: yeah03:32
spaceoneprologic: that is an invalid request03:32
spaceonethis is only for browsers03:33
spaceoneto set the Authorization header03:33
prologicoh :)03:33
prologicso the browser translates it into authorisation headers?03:33
prologicI never knew this :)03:33
prologicmostly because I don't do it )03:33
spaceonethat are basics03:33
prologicI cannot remember the last time I did http://foo:bad@blah03:34
spaceonei am using it every day03:34
prologicmaybe 12+ years ago :)03:34
prologicthere's nothing else besides and urlobject?03:34
prologicThis one is ok03:48
prologicbut missing joining new urls (relative/absolute)03:48
prologicWhat about this one?04:00
spaceonewe already had this04:00
spaceoneprologic: maybe it is a step to far... let us go slower... let us first use the new nice features, if we really need an URI class we will find one in future04:01
prologicyeah :)04:02
prologicFrom looking at various implementations though I'd like something of a cross between, furl, purl and hurl :)04:03
prologicbut anyway enough of that now04:03
spaceonewhich things are on TODO now?04:03
prologicOh still the same04:03
prologiccheck the PT list04:03
spaceoneprologic: can you add it to /topic ?04:04
prologicYour 4 bugs you reported via BB prompted me to do all this (in experimental) over the weekend04:04
prologicI can't topic is full :)04:04
prologicIt's on the website though04:04
prologicUnder Links04:04
spaceoneah ok04:04
prologicTwo clicks away :)04:05
spaceonei can't see release dates on the pival tracker04:06
spaceoneare there any?04:06
prologicno we're not setting dates04:07
prologicbut we will release after every sprint04:07
spaceoneah ok04:07
prologicbut I'm releasing early this sprint04:07
prologic2.1.1 (bug fixes)04:07
prologicspaceone: I'm going to go to bed04:21
prologicYou'll let me know how the experimental branch goes for you?04:21
prologicIf it goes well and we're happy with this, I'll merge it into dev and perform a bunch of cleanup removing dead code and modules04:21
*** ronny has quit IRC05:43
*** ronny has joined #circuits06:55
spaceoneprologic: i begin to understand circuits11:00
spaceonei am not used to that 'controller' registering themself at their childs11:01
spaceonenormally i used patterns which doing the other way round11:02
spaceonebut it is a nice pattern, i think it allows better modularity11:06
spaceoneit replaces also a little bit inheritance11:07
spaceonei don't know if it makes sense to use self.parent... maybe the server could also drop a Event ServerInitialized with these values12:10
spaceonebut these informations are normally not required12:11
*** ronny has quit IRC13:12
*** ronny has joined #circuits14:00
*** ronny has quit IRC14:17
prologicspaceone, to be honest14:44
prologicit doesn't matter where you register components14:44
prologichowever it's nice to have some kind of visual representation of your system and some kind of design14:45
prologicand yes you're quite right, it does lend itself towards better modularity and reuse14:45
prologicin circuits we walk the component graph and cache all event handlers into a dict so that we get a O(1) performance for event handler lookups14:45
prologicor near to O(1)14:45
prologicit's probably more like O(log n) actually14:46
prologicspaceone, What do you need a ServerInitialized event for?14:46
prologicI/O and Socket components already fire a Ready event to indicate they're ready to start processing14:46
jgiorgicurrently only one module, working on the auth plugin next14:50
prologicFrankenCircuits? :)14:50
prologicooo hang on14:51
jgiorgiSuperBridge is still there, i thought i deleted that14:51
prologichmm no commits/source yet14:51
jgiorgithat was reincarnated as Nathifa and is in closed beta atm14:52
prologicyou never implemented it?14:52
prologicbetter than circuits.core.bridge and circuits.node ?14:52
jgiorgiextraordinarily fault tolerant, capable of mix and matching multiple network topologies14:53
prologicI want to see that :)14:55
jgiorgionly pushes events to nodes that actually want them in the first place, supports multiple queue types and failover schemes, multiple event types (send and forget ie best-effort delivery, verified delivery and verified execution)14:55
prologicthat's something I definately want to integrate nito circuits14:55
jgiorgiand the underlying protocol isn't dependent on circuits (although it's implemented in circuits)14:56
jgiorgitheoretically a driver could be written for any language or framework14:56
prologicyou must publish and show me this :)14:58
prologicif you can open source it14:58
prologicspaceone, ahh I see what you mean by Request.server now15:00
prologicwhat if you have two15:00
prologicgood point15:00
jgiorgithere are some faults, it doesn't really serialize the event, just a series of args and kwargs using BSON (slower than pickle but faster than json and there's a module for most languages)15:01
jgiorgii'm working some bugs out and cleaning it up and then i'll be publishing it15:01
prologicit would be good to provide other serialization format sas well15:03
jgiorgii was considering it but that adds some complication to de-serializing15:04
jgiorgii could probably just use an addition couple bits to denote the format15:05
prologicanway bbs15:06
spaceoneprologic: i am tired, i might sleep on my computer :D15:15
jgiorgithe keyboard doesnt make a good pillow, trust me15:24
*** christopher has joined #circuits15:24
prologicspaceone, go to bed :)15:34
prologicI'll pull your changes and make some more improvements today15:34
jgiorgiprologic: is there a repo for i can contribute to?15:47
prologicfor the website?15:48
prologicyeah it's in the circuits repos15:52
prologicfork, contribute and submit a pull request :)15:52
jgiorgisounds good, first thing i'm getting rid of that invalid code >:|15:53
prologiceh which invalid code?15:53
jgiorgirun your site through a validator my friend16:19
jgiorgiit's not bad when compared with the rest of the internet but still, it invalid code makes me twitch16:19
prologicoh hmm16:20
prologicit was valid at one point16:20
prologicthe templates must have changed over time16:20
prologicif you fix those templates16:20
prologicI'll pull that back into sahriswiki16:20
prologicit's the default theme/templates16:20
prologicindividual changes please :)16:22
prologicand submit one pull request containing all changesets :)16:22
jgiorgiok sounds good16:26
spaceoneprologic: hmm, experimental does not work for me: the request is always the same19:57
*** christopher has joined #circuits21:52
*** anunnaki has joined #circuits22:42
*** ronny has joined #circuits23:38

Generated by 2.11.0 by Marius Gedminas - find it at!