IRC Logs for #circuits Saturday, 2017-01-21

*** markb1 has quit IRC00:15
prologicriot: listen to the signal event01:22
prologicriot: or stopped event01:22
prologicriot: not sure what you're doing per se but hitting ^C in a circuits apps that uses *any* socket component will incur a slight delay before terminating01:24
prologicto give a chance for things to flush/clear01:24
prologicACTION flying back to the US tomorrow01:26
prologicACTION was in Dublin all of this week01:26
prologicapollo13: spaceone WoW! you guys have gotten circuits master to "Build Passing", "78% Coverage" and "90% Health"01:28
prologicI'm impressed! WoW01:28
prologicCan we cut circuits 3.3 soon?01:29
prologicMaybe turn on strict branch/quality controls again?01:29
*** markb1 has joined #circuits03:56
*** Coldblackice_ has joined #circuits04:56
*** Coldblackice has quit IRC04:59
*** habs has quit IRC05:04
apollo13prologic: well, there is still the issue with pypy (which I allow as failure, which is the reason why the tests are green in the first place)09:53
apollo13prologic: but to fix that, I'd have to know which methods are allowed to be called from other threads, .fire() seems to be one… wondering if we can use a deque in .fire() to push the events into an extra queue and then the handling thread does heappush if that one is not empty before popping/handling events. That is a bit out of my comfort area though09:54
GitHub153[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMHpm10:01
GitHub153circuits/master 2989419 Florian Apolloner: Fixed #202 -- Removed the (unused) internal cache from EventType.10:01
spaceoneapollo13: I marked you at the specific lines10:09
spaceonegrr, python3 api changes:10:11
spaceone    path = re.sub(ur'\/{2,}', u'/', self.path)  # remove //10:11
spaceone                           ^10:11
spaceoneSyntaxError: invalid syntax10:11
spaceoneb'' % '' → TypeError ...10:24
spaceone{'': ''}[b''] → IndexError10:24
spaceoneb'%s' % (0.0,) → TypeError10:25
spaceonefuck rb'' is a syntax error in python210:32
spaceonegr, __metaclass__ not evaluated anymore10:41
GitHub12[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMHhC10:42
GitHub12circuits/master 5797ea3 Florian Apolloner: Fixed manager.join()10:42
apollo13spaceone: https://github.com/circuits/circuits/pull/200/files#diff-97df3ebde961fe3946cdf76057b4047c doesn't show me anything, neither does the github notification stuff :(10:44
spaceoneapollo13: refresh without cache? https://github.com/circuits/circuits/pull/200 → there are all comments as well as:10:45
spaceonehttps://github.com/circuits/circuits/pull/200/files10:45
apollo13spaceone: https://github.com/benjaminp/six/blob/master/six.py#L815 regarding your metaclass issue10:45
spaceonehm, then i need to include six :/10:45
spaceoneseems no way arround this10:46
apollo13spaceone: six is already in circuits10:46
apollo13circuits.six ;)10:46
apollo13https://pageshot.net/3DcoJjyu59EfrF4b/github.com10:46
apollo13that is what I see from your comments10:46
spaceonescroll up to the review10:47
spaceoneand you should have received email with every comment10:47
spaceonewhere a direct link points to this10:47
apollo13spaceone: did you actually finish the review? otherwise github won't show anything10:47
spaceoneoh? i thought they are released immediately10:48
apollo13no, if you start a review you have to finish it10:48
apollo13go to the pr files and on the right you should see approve/deny/comment to finish your review10:48
apollo13https://pageshot.net/9ddlpAfWZ0ZxJCxp/github.com10:48
spaceoneapollo13: now?10:49
apollo13yes10:49
apollo13I was going crazy already10:49
spaceonehmm, thats not good, some comments are immediately shown and some are not10:50
spaceonei want them to be always displayed directly10:50
spaceonebut maybe i have to be cautious about which button i click10:51
apollo13yes10:52
apollo13you have to click "add single comment"10:52
apollo13the other one starts a review10:52
spaceonei hope i didn't start a review somewhere else...10:54
spaceoneis there some site which shows all pending reviews?10:54
apollo13no idea10:55
apollo13anyways, gotta finish a few other things and then take a look10:57
apollo13I find my most recent commit to master particulary funny :D10:57
*** travis-ci has joined #circuits11:02
travis-cicircuits/circuits#589 (master - 5797ea3 : Florian Apolloner): The build was fixed.11:02
travis-ciChange view : https://github.com/circuits/circuits/compare/2989419cfc29...5797ea3769d811:02
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19398587411:02
*** travis-ci has left #circuits ()11:02
apollo13spaceone: ok, I think I got all your comments now :D11:22
spaceoneapollo13: yes11:29
apollo13spaceone: is it important that event classes have names or is it okay if just the instances have it?11:33
apollo13cause from the looks of it I can just delete EventType -- __init__ sets name anyways if it is not set11:33
spaceoneapollo13: as in python3 they don't have names it seems okay11:35
apollo13jupp, will yank it11:35
GitHub76[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMQew11:36
GitHub76circuits/master 07d8e43 Florian Apolloner: Removed EventType metaclass completely, refs #20211:36
apollo13spaceone: "Alright. Then it is wrong on python3." -- but out of scope for the PR -- do you think it is merge ready? if yes I'll merge and hope for the best :þ11:37
spaceoneapollo13: yes11:38
apollo13nice11:38
spaceoneapollo13: but wait11:38
spaceoneplease squash some commits11:38
apollo13for?11:38
spaceoneso that the unnecessary changes which got reverted don't appear11:39
apollo13spaceone: I'll squash them all, but github can do that during the merge automatically11:39
spaceonewell, but github appends all commit messages11:39
apollo13ah indeed, will do manually11:40
spaceoneis there a chr() function in python3 which returns bytes ?11:40
spaceoneb''.join(map(chr, range(0x80, 0xFF)))11:40
spaceone→ fail in python311:40
spaceonenice, this could be: bytes(range(0x80, 0xFF))11:41
spaceoneif python2 would support this11:41
spaceone...11:41
apollo13bytes(bytearray(range(0x80, 0xFF)))11:42
apollo13should work on py2 too11:42
spaceoneb''.join([bytearray(range(0x80, 0xFF))])11:44
spaceoneah yes11:44
apollo13yeah, but no need for the join in that form11:44
spaceonenice :)11:44
GitHub103[circuits] apollo13 closed pull request #200: WIP: Flake8 & isort (master...flake8_isort) https://git.io/vM97V11:45
GitHub170[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMQeM11:45
GitHub170circuits/master f2798d0 Florian Apolloner: Flake8 & isort. Also updated six.py while on it.11:45
apollo13spaceone: I agree that http code on py2/3 is annoying -- but it is relatively easily doable and if you were on py3.5+ it is actually going to be somewhat nice :D11:46
*** travis-ci has joined #circuits11:46
travis-cicircuits/circuits#590 (master - 07d8e43 : Florian Apolloner): The build passed.11:46
travis-ciChange view : https://github.com/circuits/circuits/compare/5797ea3769d8...07d8e4304bc711:46
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19399232211:46
*** travis-ci has left #circuits ()11:46
spaceoneapollo13: yes, that's why i am doing the step forward11:47
spaceoneapollo13: s/xrange/range/ ?!11:49
apollo13yes, unless you need a list ;)11:49
apollo13rather py2 range is a list and py3 range is a generator11:49
apollo13so when you reuse them make sure to put list around it11:50
apollo13but xrange -> range should be fine11:50
apollo13or you use six.moves.range11:50
apollo13https://github.com/circuits/circuits/blob/master/circuits/six.py#L24411:50
spaceonehmm, the tests are awfully slow on python3.12:03
spaceoneand wtf am i getting so much coveralls mails?!12:04
apollo13cause coverage is annoying12:05
apollo13they have that bug since a while and not fixing it12:05
spaceonewe should change to https://codecov.io/12:05
apollo13yeah, I do not have the rights to do so I think12:06
spaceonewhic my projects already are at12:06
GitHub65[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMQvV12:07
GitHub65circuits/master 331da4f Florian Apolloner: Simplified coveralls integration.12:07
GitHub171[circuits] apollo13 closed pull request #203: Simplified coveralls integration. (master...coveralls) https://git.io/vMHuA12:07
apollo13I just pushed the simplified coveralls integration -- if you could switch us to codecov that would be nice12:07
spaceonehm12:08
spaceoneyou remove the tests/main.py12:08
spaceonenow i don't know how to run the tests anymore12:08
apollo13oh :D12:08
apollo13py.test :D12:08
spaceonealso -r", "fsxX", "--durations=10", "--ignore=tmp" has been removed12:08
apollo13ok, will readd that to the configs12:09
apollo13I think we can do that via config files without hardcoding that12:10
apollo13spaceone: I was trying to get down to one way to run the tests, we could readd a python setup.py test hook, but in general tox -e py35 (or whatever env you want) is a good way to run the tests12:13
apollo13or just py.test using your existing system/setup12:13
spaceoneokay12:14
apollo13if you'd like another way to run things, tell me -- I just got confused by the like 10 possible entry points12:15
apollo13also if you use tox you can use "--" and everything after that is passed to py.test12:15
spaceonefor me it was always clear12:15
spaceonei opened main.py12:15
spaceonecopied -r", "fsxX", "--durations=10", "--ignore=tmp"12:15
spaceoneands that12:15
spaceone:d12:15
apollo13so tox -e py26 -- -x would fail on the first failed test12:16
spaceonei have no tox installed12:16
apollo13just py.test then, I am adding those opts to the config files so they should get picked up automatically12:16
*** travis-ci has joined #circuits12:16
travis-cicircuits/circuits#595 (master - 331da4f : Florian Apolloner): The build passed.12:16
travis-ciChange view : https://github.com/circuits/circuits/compare/f2798d076816...331da4fb0c6112:16
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19399676612:16
*** travis-ci has left #circuits ()12:16
GitHub183[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMQfm12:19
GitHub183circuits/master 4666b85 Florian Apolloner: Readded default py.test options.12:19
apollo13spaceone: okay, so with the newest commit if you run "py.test tests/web/test_websockets.py" you should get the same behavior as before12:19
apollo13ie slowest then tests etc12:19
apollo13please tell me if you miss something else12:19
spaceonenice, thanks12:20
apollo13if we want it as python setup.py test too we can add http://doc.pytest.org/en/latest/goodpractices.html#integrating-with-setuptools-python-setup-py-test-pytest-runner12:21
apollo13spaceone: oh wow, codecov is easily enable by just replacing coveralls with codecov everywhere and we are done XD + prologic probably needs to setup codecov.io since he is the admin and disable coveralls12:25
spaceoneyes12:26
*** travis-ci has joined #circuits12:26
travis-cicircuits/circuits#596 (master - 4666b85 : Florian Apolloner): The build passed.12:26
travis-ciChange view : https://github.com/circuits/circuits/compare/331da4fb0c61...4666b8535e9f12:26
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19399830712:26
*** travis-ci has left #circuits ()12:26
spaceoneWTF my tests are passing in one minute under py2 and failing in 30 minutes under py312:27
apollo13got a link to the travis build?12:29
spaceoneit's basically one test which runs very long12:31
spaceonehttps://travis-ci.org/spaceone/httoop/jobs/19399522212:31
apollo13do you know which one?12:32
spaceoneyes12:33
spaceonegr, if one test fails to import all the following are skipped12:39
apollo13that is weird :D12:40
spaceone749 failed, 71 passed, 11016 skipped, 26 xfailed, 221 error in 63.49 seconds12:43
apollo13better :)12:44
apollo13but you have really plenty of tets12:44
spaceoneit will take days to fix all of the unicode-byets issues12:46
spaceonefrom __past__ import bytes_literals12:46
spaceonehm, i would like to know some sed script to transform every string literal into b''12:51
spaceone707 failed, 109 passed, 11016 skipped, 26 xfailed, 220 error in 56.88 seconds12:53
spaceoneone string literal changed :D12:53
spaceoneNameError: name 'unicode' is not defined12:53
prologicapollo13: do we want to use codecov.io instead? happy to do so; file an issue and assign it to me with whatever details I need13:02
prologicI already have an account setup from the other day13:02
apollo13prologic: I have no idea what you need to do on the codecov side; if you can just disable coveralls and enable codecov I can open a PR and see :D13:03
apollo13spaceone might know what you actually have to do since he used it13:03
apollo13basically go to https://codecov.io/gh/circuits/circuits/settings and add a webhook13:04
apollo13then you are done I think13:04
apollo13and we maybe need a codecov.yml13:05
prologicI see13:09
prologicthis is good enough info; I'll do it as soon as I'm home13:09
prologicon/in my flight back home to the US atm :)13:09
prologicwith that shitty ~800ms latency internet :)13:09
prologicAlso I see ~12hrs of unread backlog here; what did I miss? can anyone summarize?13:10
GitHub178[circuits] apollo13 opened pull request #204: Changed coveralls to codecov. (master...codecov) https://git.io/vMQUt13:10
apollo13prologic: I've updated six, flake8 & isorted the whole codebase. Made py.test the canonical entry point for tests -- should be the tl;dr13:11
apollo13https://github.com/circuits/circuits/pull/204 lets see if it manages to submit to codecov without a repo really setup :D13:11
prologicre codecov.io13:12
prologicI think that's done; I created the Webhook13:12
prologiclet's see if that's all we need?13:12
apollo13jupp, should see in a minute13:12
prologiccool13:13
prologicand also; awesome work you've been doing! really impressive13:13
apollo13https://codecov.io/gh/circuits/circuits seems to be doing some things13:14
prologicI saw a comment in some issue/thread somewhere between you and spaceone about circuits.web vs circuits.http (spaceone's derivative) and I wanted to point out a few things13:14
prologicactually you could always override the error page by simply listening to the response_failure event I believe13:14
prologicand a few other things were also already there13:14
prologic-but- circuits.web is a mess of cobbled together bits of BaseHTTPServer and CherryPy shoved into what circuits is all about; components and events13:15
prologicso -- I'm all for rewriting circuits.web entirely!13:15
prologicas long as it's still all event-driven and has nice components (ala the circuits way)13:15
apollo13what about backwards compat? or is it not that much in use anways?13:16
riotprologic: i think i have that running. I'm asking the thread that handles zmq to stop when the circuits component that spawns it gets a signal that is SIG_TERM/INT - works most of the time. I think it does not when i have a bug in the thread - when the stop handler is not able to respond because the thread threw a hidden exception.13:16
spaceonecodecov works13:17
prologicI think we can completely rewrite circuits.web without breaking backwards compat too much13:17
spaceoneprologic: it's not circuits.web VS circuits.http13:17
prologicIHMO most of the interface (the events themselves) should remain the same13:18
GitHub52[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMQUE13:18
GitHub52circuits/master 623ea8c Florian Apolloner: Changed coveralls to codecov.13:18
GitHub25[circuits] apollo13 closed pull request #204: Changed coveralls to codecov. (master...codecov) https://git.io/vMQUt13:18
prologicwhat will probably break a lot is the req/res objects (the containers that actually hold http req/res data/attributes)13:18
spaceoneprologic: in circuist.web there are parameter: (request, response) in circuits.http there is only client.{request,response,socket,server,…}13:19
prologicriot: interesting; do you or can you provide us a minimal repro case where the signal handler doesn't behave correctly?13:19
prologicspaceone: no I know :) but I sympathize with your original needs for writing your own web framework on top of circuits13:20
prologicspaceone: circuits.web (I admit) kind of sucks but it works :)13:20
prologicin any case circuits is not about a web framework per se but it should have a good one :)13:21
prologichopefully between apollo13's experience from Django we can make it better overall :)13:21
spaceoneIt's actually Client.{request,response,socket,server,user,domain,resource,method,local,remote,ssl,session}13:22
prologicspaceone: yeah that's also the approach some golang frameworks take; just passing around a context object that containers everything13:22
apollo13how much is .web used? what about just introducing newweb and then deprecate web?13:22
spaceoneusing names like 'new' and '2' is bullshit13:23
spaceoneSet-Cookie2: blah13:23
apollo13bullshit is relative, it allowed us in django to have newforms and forms and at some point just drop forms and rename everything -- that is imo a sensible choice if you are going to have to many differences13:24
apollo13either way, personally for web I have django, so I am not going to use circuits.web/http anyways :D13:25
spaceoneanother difference: circuits.web: request.kwargs = parse_qs() + parse_body() (for all http features some functions are called), circuits.http: client.request.body.{query,query_string} and client.request.body.data → HTTP is the interface, there is only one way to parse the data and this is defined by the http headers13:27
spaceonethe parsing should be done once. not everywhere where it might be needed13:27
spaceonein circuits.web every method in BaseComponent can be reached in different ways: 1. GET /foo/bar 2. GET /foo?bar=1 3. GET /foo/ with body bar=baz etc.13:29
spaceonein circuits.http the uri is bound to a resource, which calls the function defined for the HTTP method13:30
riotprologic: nah, i don't think it is the signal handler. I did not yet come around to adding thread safe exceptions to the threading.Thread component i have, so i assume it is the (threaded) method the handler executes that is misbehaving (sometimes. depending on the state of the thread)13:32
*** travis-ci has joined #circuits13:32
travis-cicircuits/circuits#599 (master - 623ea8c : Florian Apolloner): The build passed.13:32
travis-ciChange view : https://github.com/circuits/circuits/compare/4666b8535e9f...623ea8c0265a13:32
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19400625213:32
*** travis-ci has left #circuits ()13:32
spaceonein circuits.web you need to do: response.headers['Content-Type'] = 'application/json'; response.body = json.dump(data). in circuits.http you return some data and add some codecs for the accepted mime types. the client chooses the preferred mimetype, not the server13:32
spaceone(well it's a contract of both, client and server) → content negotiation13:32
GitHub127[circuits] apollo13 created codecov (+2 new commits): https://git.io/vMQT713:48
GitHub127circuits/codecov d5f0340 Florian Apolloner: Changed coveralls to codecov.13:48
GitHub127circuits/codecov 2c6eec5 Florian Apolloner: Removed six.py from coverage.13:48
GitHub103[circuits] apollo13 deleted codecov at 2c6eec5: https://git.io/vMQT513:48
*** travis-ci has joined #circuits13:55
travis-cicircuits/circuits#600 (codecov - 2c6eec5 : Florian Apolloner): The build has errored.13:55
travis-ciChange view : https://github.com/circuits/circuits/compare/d5f03404a876^...2c6eec5b447313:55
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19401076013:55
*** travis-ci has left #circuits ()13:55
GitHub66[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMQky14:07
GitHub66circuits/master bb7e71e Florian Apolloner: Put everything (except six) under coverage.14:07
*** travis-ci has joined #circuits14:16
travis-cicircuits/circuits#601 (master - bb7e71e : Florian Apolloner): The build passed.14:16
travis-ciChange view : https://github.com/circuits/circuits/compare/623ea8c0265a...bb7e71e7f73114:16
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19401402314:16
*** travis-ci has left #circuits ()14:16
spaceonewtf. why does json.dumps() return unicode in py314:26
prologicbecause dumps is string?14:41
prologicmaybe there's a dumpb()?14:41
apollo13spaceone: the stdlib is a mess :D14:47
spaceone719 failed, 322 passed, 11016 skipped, 26 xfailed in 56.65 seconds14:48
spaceonethis is soo much work :(14:48
spaceone>           return b''.join(b'%%%X' % (ord(d),) if d not in charset else d for d in data)14:52
spaceoneE           TypeError: sequence item 0: expected a bytes-like object, int found14:52
GitHub113[circuits] prologic created issue#179 (+1 new commit): https://git.io/vMQLM15:01
GitHub113circuits/issue#179 0c5578f James Mills: [docs]: Fixed documentation for circuits.core.Worker...15:01
GitHub148[circuits] prologic opened pull request #205: [docs]: Fixes documentation for circuits.core.Worker (master...issue#179) https://git.io/vMQLd15:05
*** travis-ci has joined #circuits15:09
travis-cicircuits/circuits#602 (issue#179 - 0c5578f : James Mills): The build passed.15:09
travis-ciChange view : https://github.com/circuits/circuits/commit/0c5578f92bc215:09
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19402327715:09
*** travis-ci has left #circuits ()15:09
*** markb1 has quit IRC15:20
spaceonegive up for now...15:25
spaceonehave a nice weekend15:25
apollo13spaceone: you too15:25
prologicspaceone: have a nice weekend!15:29
prologicI'm trying to work on https://github.com/circuits/circuits/issues/8615:29
apollo13prologic: btw one thing was hidden in that backlog -- with regards to threading. would it be possible for .fire() to just push events into a dequeue (which is guranteed to be thread-safe) and then have the code that consumes manager._queue (ie the handling thread) pull stuff out of the dequeue and heappush into its _queue15:31
apollo13if we could get that working somehow, it should be possible to use .fire from another thread (for tests) without to much performance loss15:31
apollo13otherwise we will have to really lock-guard all access to _queue (aside from len() maybe)15:31
prologicactually funny thing15:33
prologiccircuits.core.managers.Manager used to use a deque once :)15:34
prologicbut honestly I feel like no-one really uses the whole event priority really15:34
prologici.e: self.fire(foo(), priority=N)15:34
apollo13wait events have a priority?15:34
apollo13I thought handlers have one15:34
prologicboth :)15:35
apollo13what does a priority on an event even mean?15:35
prologicyou can in fact ire events at a higher priority than others15:35
prologicand they'll be dequeed first15:35
apollo13mhm15:35
prologicI think there was one person that used this15:35
apollo13any usecase?15:35
prologicfor a circuits app that did some realtime stuff with audio15:36
apollo13lolwut?15:36
prologicso maybe removing it is not such a good idea :)15:36
apollo13realtime audio in python?15:36
prologicyes!15:36
prologicwell real-time-ish15:36
apollo13well, who is that person? :D couldn't we get away with two queues then?!15:36
apollo13ie normal priority and high priority15:36
prologicI think he managed to get circuits to perform with enough perf to get down to ~10Hz timing ir something15:36
prologicI don't honestly remember -- it was a while back15:37
apollo13would still be easier than the heap queue15:37
prologicand used the circuits.io.serial.Serial component heavily15:37
prologictwo queues eh? hmm15:37
apollo13anyways, I managed to once get the pypy failures even on py3.something15:37
apollo13so the issue seems to be very real and not a pypy problem :D15:37
prologicofc15:38
prologicwanna have a crack at what you suggested?15:38
prologicAFAIK there should be tests for event priority15:38
prologicas well as handler priority15:38
apollo13well I can try. but I am not sure how15:39
apollo13or rather .fire() should be threadsafe, anything else?15:40
apollo13.tick()? probably not I think, that should only get called in that one thread I assume15:40
prologic.call() and .wait() rely on fire15:40
prologicso not sure if they need anthing special15:40
prologicyeah .tick() is called peridocially inside the manager (the managing component/thread)15:41
prologicused to tick over all coroutines/tasks15:41
apollo13gotta see if I understand it -- last time I got stuck at https://github.com/circuits/circuits/blob/master/circuits/core/manager.py#L45215:42
prologicoh that stuff haha15:44
prologicdon't blame me for that code15:44
apollo13na, just trying to understand15:44
prologicit's all to do with blocking if there's nothing to do instead of spin-looping15:45
apollo13I at least found https://github.com/circuits/circuits/commit/6800fbc87d58a469f68d6c7f1a281db0cc0019e5  now15:45
prologicand also how all I/O components work by firing generate_events()15:45
apollo13yeah, I guess I only understand half of that currently15:48
prologicthat's 1/2 more then me :)15:49
apollo13grml :(15:49
prologicheh :)15:52
riotrealtime audio in python? YESS. I could finally build me my own modular soft synth19:00
riotalso: a little pet/side-project i started some time ago: github.com/ri0t/avio19:01
GitHub35[circuits] apollo13 opened pull request #206: Removed a bit of whitespace. (master...whitespace) https://git.io/vMQWP19:07
GitHub98[circuits] apollo13 pushed 1 new commit to master: https://git.io/vMQlt19:20
GitHub98circuits/master 4c55575 Florian Apolloner: Removed a bit of whitespace.19:20
GitHub103[circuits] apollo13 closed pull request #206: Removed a bit of whitespace. (master...whitespace) https://git.io/vMQWP19:20
apollo13prologic: https://github.com/circuits/circuits/blob/0c5578f92bc26c45956790eab0e33d23cbc486e5/circuits/core/workers.py#L86-L87 how much "stress" does that produce on the master process? ie assume it is not doing anything aside from waiting for that to finish, how often would that get called?19:30
*** travis-ci has joined #circuits19:39
travis-cicircuits/circuits#605 (master - 4c55575 : Florian Apolloner): The build passed.19:39
travis-ciChange view : https://github.com/circuits/circuits/compare/bb7e71e7f731...4c555758fc1819:39
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19406671119:39
*** travis-ci has left #circuits ()19:39
prologicapollo13: good questin; I do not know :/20:10
prologicbut you know what -- we should build support for counters/metrics in circuits20:10
prologicall the big companies have metrics driven software systems why shouldn't circuits :)20:11
*** Coldblackice_ has quit IRC20:15
*** Coldblackice has joined #circuits20:15
apollo13prologic: regarding the whitespace, it was mainly to normalize and only at the start of the file20:17
prologicokay :)20:17
apollo13prologic: if that causes a problem I can revert and add one or two lines…20:17
prologicsetting up branch prot again20:17
prologicno it's okay leave it :)20:17
apollo13regarding metrics: prometheus ist the cool new thing ;)20:18
apollo13prologic: with branch protection you get a chance to stop me before committing now :D20:19
prologicPlease review https://github.com/circuits/circuits/pull/20520:22
prologic:)20:22
prologicand vice versa if I try to commit stupid crap :)20:23
prologicyeah I'm not super familiar with how applications ca publish metrics20:23
prologicwe should see if there at least one or two standardish ways20:24
prologiclike collectd or something20:24
apollo13collectd/statsd etc is already old and boring20:27
apollo13prometheus :D20:27
prologicsure20:28
prologicbut what does an app/system need to expose?20:28
prologictightly integrating with prometheus here should not be the goal20:28
apollo13memory usage, uptime, cputime20:29
prologicbuilding builtin metrics publishers into circuits should that agents can pickup20:29
prologicyeah exactly20:29
prologicand...20:29
prologicn_events, n_exceptions, n_tasks, etc20:30
apollo13mhm, that sounds interesting and a good idea for me to learn circuits20:30
prologicand decorators to support custom counters20:30
prologickdb: events20:30
kdbEvents: 8808178720:30
prologickdb (a highly pulggable IRC bot written in circuits ofc) has some internal counters20:31
kdbUnknown Command: (a20:31
apollo13I'll see if I can come up with a Prometheus Metrics component, afaik you just need to expose an http endpoint20:31
prologicbut no way to publish/collect them anywhere useful20:31
prologickdb: errors20:31
kdbNo errors20:31
prologickdb: uptime20:31
kdbUptime: 99+12:33:45 (CPU: 67771.48s 0.79%)20:31
prologickdb: mstats20:31
kdbMemory Usage: 30.11MB20:31
prologicif that's all it takes is a HTTP endpoint; that's perfect!20:32
apollo13yeah, prometheus is pull based20:32
apollo13which has benefits20:32
prologicthen we just need an internal structure to store and bump counters, reset counters, etc20:32
prologicand bam we're done20:32
apollo13ie if prometheus doesn't scape every second or so, you just loose resolution, but not the numbers per se20:32
prologic*then* I can accurately answer your question on Worker from before :)20:33
apollo13even better, we can easily graph memory usage then etc…20:33
prologicwe should also support push based systems too20:33
prologicthe one I'm very familiar with is InfluxDB20:33
prologicI can write one for that as well20:34
apollo13well, for me it is first and foremost a learning system, so I'll do something for prometheus and then follow your guidance on how to make it plugable/reusable20:34
apollo13btw how strongly are the pollers modified from twisted?20:34
apollo13ie would twisted be possible as dependency instead?20:35
prologicahh no :)20:36
prologicand heavily borrowed from Twisted20:36
prologicbut no; circuits cannot depend on Twisted without much change in architecture :)20:37
prologic-but-!20:37
prologiccircuits *can* run Twisted apps/protocols :) lol20:37
prologicthere's a circuits.twisted repo I believe20:37
prologicbut very old and probably needs work to make it work again20:37
prologichttps://bitbucket.org/circuits/twistedintegration20:38
prologicI never migrated it to GH I don't think20:39
prologicfor fear Twisted devs would shoot me :)20:39
prologicGlyph (the original author of Twisted) has for years and years tried to convince me to make circuits run atop Twisted20:39
apollo13oh, amber would not shoot you20:39
prologicbut it as easier going the other way around :)20:39
apollo13mhm, maybe we can bribe amber to try rewriting it onto twisted XD20:40
apollo13https://github.com/prometheus/client_python#twisted20:40
apollo13seems easy enough XD20:40
prologicoh!20:40
prologicbut writing components is easy :)20:40
apollo13yes, I will write that one http component20:40
prologictbh I'm not sure of the benefits of running circuits atop Twisted20:40
apollo13hehe, probably none20:41
prologicbut there is a definite benefit in integration the other way aroud; reusing Twisted protocols and libraries20:41
apollo13but if I managed that I'd certainly understand how circuits works ;)20:41
prologicsomeone needs to do circuits+Tornado integration and circuits+asyncio integration too20:41
prologicso we can just reuse and wrap into components anything out there20:41
apollo13hehe20:42
apollo13how do I set a content-type on circuits.web?20:46
apollo13ah there we go20:47
GitHub182[circuits] prologic force-pushed issue#179 from 0c5578f to 27345bc: https://git.io/vMQ0s20:48
GitHub182circuits/issue#179 27345bc James Mills: [docs]: Fixed documentation for circuits.core.Worker...20:48
*** robert_ has quit IRC20:50
*** robert_ has joined #circuits20:51
*** robert_ has quit IRC20:51
*** robert_ has joined #circuits20:51
apollo13prologic: https://pageshot.net/wZPwQNIg2JvBTPHz/localhost :D20:58
apollo13already monitoring https://pageshot.net/2DWyvCg9sIGwYr9o/localhost20:58
*** travis-ci has joined #circuits20:59
travis-cicircuits/circuits#606 (issue#179 - 27345bc : James Mills): The build passed.20:59
travis-ciChange view : https://github.com/circuits/circuits/compare/0c5578f92bc2...27345bcf524b20:59
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19408051120:59
*** travis-ci has left #circuits ()20:59
GitHub189[circuits] prologic closed pull request #205: [docs]: Fixes documentation for circuits.core.Worker (master...issue#179) https://git.io/vMQLd21:06
GitHub64[circuits] prologic pushed 1 new commit to master: https://git.io/vMQEs21:06
GitHub64circuits/master 70080ac James Mills: [docs]: Fixed documentation for circuits.core.Worker (#205)...21:06
prologicapollo13: WoW!21:12
prologicThat was fast :)21:12
prologicI'm impressed21:12
prologicnow let's make the code modular and circuits style :)21:12
prologicand add a bunch more counters for cpu and internal counters21:12
apollo13https://dpaste.de/W2bB/raw21:12
apollo13that is all that is needed :D21:12
apollo13not exactly sure how to abstract that21:13
apollo13prometheus is so simple and nice, let me start up grafana to show you something nice21:13
prologicoh fuck wow21:13
prologicthat's it?! :)21:13
apollo13yes21:14
apollo13well you register that on a web.BAseServer and you are done21:14
prologicof course21:14
prologicwow21:14
apollo13now you can obviously get fancy by grouping on certain channels etc… but… ;)21:14
prologicokay when I do an InfluxDB one I'll have to see about ways of making the API the same for both :)21:14
prologicor maybe Prometheus is the bees knees? :)21:15
apollo13;) let me put that into grafana and you will cry21:15
prologiclol21:16
prologicgo fer it :)21:16
prologicThere you go #208 is yours :)21:20
*** travis-ci has joined #circuits21:23
travis-cicircuits/circuits#608 (master - 70080ac : James Mills): The build passed.21:23
travis-ciChange view : https://github.com/circuits/circuits/compare/4c555758fc18...70080ac77dce21:23
travis-ciBuild details : https://travis-ci.org/circuits/circuits/builds/19408342121:23
*** travis-ci has left #circuits ()21:23
prologiclanding soon21:28
prologiccya later21:28
apollo13prologic: https://pageshot.net/1LpKJWRO0BEVOuAA/localhost taken the total counter and graphed them as rates per minute :D21:36
apollo13or lovlier https://pageshot.net/nHV6595WjbYBgawh/localhost21:39
*** Coldblackice has quit IRC22:00
apollo13mhm weird, sometimes I get a simple echoserver which I cannot kill via ctrl+c22:19
apollo13gotta see if I can debug that22:20
apollo13(tomorrow)22:20
prologicapollo13: wow23:54
prologicnice graphs :)23:54
prologicme wants!23:54
apollo13it is literally just the few lines I showed and clicking it together in grafana :D23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!