IRC Logs for #circuits Monday, 2014-03-10

*** Osso has quit IRC00:07
prologicspaceone: Merged PR #5 -- But note my comment on the use of conditional expressions00:24
prologicI do find them more readable in certain situations such as these :)00:24
prologicBut otherwise it doesn't matter too much00:24
*** Osso has joined #circuits00:38
*** flikker has quit IRC01:46
*** Osso has quit IRC02:04
*** yestreen has joined #circuits02:10
prologicspaceone: I added Email to circuits-dev@googlegroups.com, Twitter to python circuits and IRC (circuitsbot, hasn't joined yet?) to the github mirror of ciruits02:16
prologicTry it out with another PR :)02:16
*** yestreen has quit IRC02:17
*** ninkotech has quit IRC03:13
*** ninkotech has joined #circuits03:13
*** ninkotech__ has joined #circuits03:51
*** ninkotech has quit IRC03:52
prologickoobs: know anyone that can confirm some socket behavior for me on BSD?05:12
koobsprologic: i can :)05:25
*** mediamagnat has joined #circuits06:08
*** mediamagnat has quit IRC06:30
prologickoobs, awesome :)06:44
prologicI want to know how BSD sockets behave in thhe folloinw conditions06:45
prologica tcp client trying to connect to a tcp server06:45
prologicwhoose listening socket is closed06:45
prologicin python you would06:45
prologiccreate the listening socket06:45
prologicbind it to an interface06:45
prologiccall .listen()06:45
prologicthen immediately close the socket06:45
prologicthen try to connect a client to this06:45
prologickoobs, to be clear07:28
prologicbasically what I'm seeing on OS X07:28
prologicwhich is basically BSD07:28
prologicis that I get no socket error back07:28
prologicjust a bad file desctiptor07:28
prologici.e: -107:28
prologicbut on Linux(es) I get a socket error07:28
*** Osso has joined #circuits08:02
*** Ossoleil has joined #circuits09:40
koobsprologic: as in conn refused from the clients point of view?12:47
prologickoobs, yeah13:02
prologicI'm seeing no such socket error on OS X13:03
prologicjust a -1 file descriptor13:03
koobssysctl -a |Grep blackhole13:03
koobsgrep13:03
koobsare any set to 1 ?13:03
koobsif so, http://www.freebsd.org/cgi/man.cgi?query=blackhole13:03
prologickoobs, http://codepad.org/I7Nw4reh13:05
prologic^^^ for example13:05
prologicnotice the last failing test because of a failed assertion13:05
koobsthe assert is to wide13:06
koobsSocketErrorOrTimeout?13:06
koobsare you testing the clients behaviour here, or the servers?13:07
prologicLinux (same tests): http://codepad.org/D24DF40v13:08
prologictesting the client's behavior13:08
koobs<error[client] (error(111, 'Connection refused') )>13:08
prologicthat connecting to a closed listening port fails13:09
koobsyour test relies on the behaviour of the remote end13:09
koobsa response from the remote end is not guaranteed13:09
prologichave a look at the 2nd paste13:09
prologicand compare it with the first13:09
koobsam looking)13:09
prologicon Linux I get back a socket error13:09
prologic111 connection refused13:09
prologicon bsd I don't seem to get this13:09
koobsbecause the remote end 'conn refused'13:10
koobswhich means you get a RST back13:10
prologicthis is my whole point :)13:10
koobswhich leads to your socketerror exception13:10
prologicam I missing something13:10
prologicor does bsd behave slightly differently ?13:10
koobsyes, the codepath in your client/connect code that describes 'what i do when i dont hea rback from remote end'13:10
koobsyou could be firewalled, the endpoint could be blackhole'ing you13:11
prologicso you think our circuits/net/sockets.py connect event handler needs some work?13:11
koobsyou could be losing packets, your link may be saturated13:11
prologicwe know it does13:11
prologicbecause it doesn't actually know when the conenction was successful or not13:11
prologicit "assumes" it is13:11
prologicdon't ask13:11
koobsnot strictly work, just that the current test, relies on the fact that the servers *responds*13:11
prologicbased off Twisted code :(13:11
koobswhich is only true in some cases (not an OS question)13:11
koobsi mean, if the bsd endpoint doesnt have blackholing enabled, you'll get a rst just like linux13:12
prologicI think the thing we have to look at is:13:12
prologic(let me look it up)13:12
prologichttps://bitbucket.org/circuits/circuits/src/tip/circuits/net/sockets.py#cl-23513:13
prologica) this code assumes a successfully connection13:13
koobsprologic: if the socket doesnt respond, youll get a python socket timeout exception (its undefined by default)13:13
prologicwrongly so and I'd like to fix that at some point as soon as someone can tell me how to asynchornous detremine a conenction was in fact successfuly13:13
prologicand b) this code may not be quite as cross platform as it should be in terms of client behavior13:13
koobsprologic: the easy part is the success condition (3 way handshake)13:14
koobsthe challenge is the failure condition(s)13:14
koobs assert pytest.wait_for(client, "connected")13:14
koobshow long does it wait for?13:15
prologica default timeout13:15
prologicit's part of the test fixtures13:15
koobsand what does the client do in that case (no response)13:15
prologicoh nothing13:15
prologictest just fails13:15
koobsthere you go13:15
prologicthe client isn't the thing that's wiating13:15
prologicjust the test13:15
koobsyeh i know13:15
koobs exception socket.timeout¶13:16
prologichmm13:16
koobshttp://docs.python.org/2.7/library/socket.html#socket.timeout13:16
koobsyou only cover socketerror13:16
prologicso you're saying on BSD13:16
koobswhich you wont get in timeout cases13:16
prologicwe may in fact get a timeout?13:16
koobsno, not just bsd13:16
koobsin the !success case, you may timeout13:16
koobsprologic: how does your test behave if you try to connect to nonexistent.hostname.tld ?13:17
prologicfails immediately with a lookup failure13:17
prologicalthough you raise a good point13:17
prologicshould write a test specifically for dns lookup failures13:17
koobssocket.herror socket.gaierror13:17
prologicand test this against 2.6, 2.7, 3.2, 3.3, pypy on all supported platforms we support :)13:17
koobsunfortauntely test_connect is not as unit'ey as you think it is13:18
prologicno :)13:18
koobsor fortunately as the case may be13:18
koobsthe assert relies (incorrectly) on remote endpoint behaviour13:18
koobsits an integration level test masquerading as a unit test :)13:18
prologichttps://bitbucket.org/circuits/circuits/issue/76/missing-unit-test-for-dns-lookup-failures13:19
koobsyou could mock a response13:19
koobsthen it would be correct13:19
prologicyeah13:19
prologic:)13:19
prologicwe actually have very few unit tests13:19
prologicmost are integration tests13:19
koobsget a coveralls badge13:19
prologicand testing against itself13:19
prologichehe13:19
koobsthen make sure it only goes up13:19
prologicin any case you raise good points13:19
prologicone thing though I'd like to sound out13:20
prologicis there a sensible way to:13:20
prologicdetermine asynchronously if a client connection was actually made successfully or not13:20
prologicright now I hate that the connect event handler "assumes" this13:20
prologic(borrowed from Twisted)13:20
koobswhen you say async what do you mean?13:21
prologicbecause this will likely inadvertently fix any so-called timeout behavior too ihmo13:21
prologicwell so the calls in the connect handler I linked to baove13:21
prologicdon't block and don't wait13:21
koobsi assume the completion of the handshake would trigger the event/signal no?13:21
prologicso a connection attempt is made13:21
prologicwe assume it's successfuly13:21
prologicbut don't actually find out for real13:22
prologicuntil we try to read or write to the socket13:22
prologicafaik I'm not aware of a handshake/completion signal that can be listedn to?13:22
koobshttps://stackoverflow.com/questions/17695388/proper-use-of-a-client-and-deferred-with-twisted13:22
koobs?13:22
prologicperhaps some special i/o polling event?13:22
koobssorry, i not code'ey13:22
koobsbut my expectation is that the async event handler should let the caller know13:23
koobsif not it sounds like its not doing its job13:23
koobshttps://twistedmatrix.com/documents/12.0.0/api/twisted.internet.protocol.BaseProtocol.connectionMade.html13:24
prologicI need to check if they've somehow fixed this in some way13:25
prologicspeaking of which13:28
prologicTwisted is only just today finally getting "some" support for Python3 and pip installable13:28
koobsprologic: yup13:31
*** Osso has quit IRC13:52
prologickoobs, I'm just trawled and grepped through the twisted code base again13:55
prologicand they haven't fixed or changed this behavior at all13:55
prologicthey rely on a predetermined/default timeout13:55
prologicand fail the connection if not connected in some period of time13:55
prologicwhich tbh is ridicoulous13:56
prologichttps://github.com/twisted/twisted/blob/trunk/twisted/internet/tcp.py#L54213:56
prologicYou'll notice many similarities with our codebase: https://bitbucket.org/circuits/circuits/src/tip/circuits/net/sockets.py#cl-23513:57
prologicbut unlike Twisted you can probably follows ours and reason about it a bit better! :)13:57
prologicYou have to carefully follow at least 4 modules to fully understand what Twisted is or isn't doing13:57
prologicBut in short13:58
prologicTwisted does not determine if the connection was actually successfully or not13:58
prologicin any way - it just assumes it has connected13:58
prologicand times out if it hasn't13:58
prologicI assume there's more code later to actually cancel the .callLater() call13:58
koobsprologic:  theres no other way to do it except on timeouts14:34
prologicthat's what I thought15:09
prologicpity really15:10
prologicOr is it a limitation of TCP?15:10
prologichmm15:10
prologicyou'd think you could get an I/O signal from your OS when the tcp handshake is complete15:10
*** Osso has joined #circuits17:57
*** zleap has joined #circuits18:09
*** zleap has quit IRC18:53
*** zleap has joined #circuits19:01
*** oteroma has joined #circuits22:29
*** oteroma has left #circuits ()22:30
prologicmorn'n all22:43
*** ninkotech__ has quit IRC23:19
*** ninkotech__ has joined #circuits23:19
*** zleap has quit IRC23:45

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!