IRC Logs for #circuits Monday, 2015-08-24

*** Workster has joined #circuits00:55
kdbWelcome back workster :)00:55
*** metanosulfonico has joined #circuits02:27
kdbWelcome back metanosulfonico :)02:27
metanosulfonicohahaha, hello :)02:28
*** Workster has quit IRC04:43
*** Workster has joined #circuits04:47
*** Workster has quit IRC04:47
*** Workster has joined #circuits04:47
*** Workster has quit IRC05:33
*** Workster has joined #circuits05:43
*** metanosulfonico has quit IRC06:10
*** koobs has quit IRC06:49
*** Coldblackice_ has joined #circuits06:52
kdbHey coldblackice_06:52
*** Coldblackice has quit IRC06:54
*** ninkotech_ has joined #circuits07:29
kdbHey ninkotech_07:29
*** ninkotech_ has quit IRC07:29
spaceone                 │10:58:22 @prologic | I saw nothing?                                                                                                                                                                                                                 │08:59
spaceone                 │10:58:35  spaceone | │10:57:11  spaceone | python -c 'import sys; sys.stdout.write("USER foo x x :foo\r\nNICK foo%s%s\r\n" % (u"→".encode("utf8"), u"ä".encode("latin1")))' | nc 7000                                     │08:59
spaceone                 │10:58:43  @Romster | <Romster> did spaceone break it                                                                                                                                                                                                │08:59
spaceone                 │10:58:44  @Romster | <spaceone> maybe because my mixed utf-latin1 input?                                                                                                                                                                            │08:59
spaceone                 │10:58:44  @Romster | <prologic> spaceone, what are you doing to it? :)08:59
*** techdragon has quit IRC10:50
*** y0no has quit IRC10:51
*** LyndsySimon has quit IRC10:55
*** techdragon has joined #circuits10:56
kdbHello techdragon10:56
*** LyndsySimon has joined #circuits10:58
kdbHeya lyndsysimon10:58
*** y0no has joined #circuits11:01
kdbWelcome back y0no :)11:01
*** travis-ci has joined #circuits11:05
travis-cicircuits/circuits#316 (fix-debugger-event-repr-errors#124 - 11f14d1 : James Mills): The build passed.11:05
travis-ciChange view :
travis-ciBuild details :
*** travis-ci has left #circuits ()11:05
spaceoneprologic: FYI: you can pass a tuple of types to isinstance()11:31
spaceone(you implemented a any(ininstance(...) for .. in ...) in that IRC server11:31
prologicI did :)11:40
prologicand I know11:40
prologicand thanks I'll remove it :)11:40
prologicgone :)11:41
Romster wow zfs some mature age woman is smart12:16
prologic>>> str(message)12:21
prologicTraceback (most recent call last):12:21
prologic  File "<console>", line 1, in <module>12:21
prologic  File "/home/prologic/circuits/circuits/protocols/irc/", line 52, in __str__12:21
prologic    " ".join(args)12:21
prologicUnicodeEncodeError: 'ascii' codec can't encode characters in position 4-7: ordinal not in range(128)12:21
prologicspaceone, can you help me understand this?12:21
prologicI really get sick of Unicode encodings :)12:21
spaceonehm. what is so hard?12:22
spaceoneyou have python212:22
spaceonethat means " " is a bytestring12:22
spaceonethe default encoding of python2 is ascii12:22
spaceoneso if args contains a string with non-ascii chars it will fail12:22
prologicwhich in this case it does12:23
prologicit contains some unicode objects12:23
spaceonethat is the problem12:23
prologicthe intention here is that they are joined by a u" "12:23
prologicwe later do a s.encode(utf-8") on the result12:23
spaceoneyou could this line to:12:23
spaceonereturn ' '.join(s.encode('utf-8', 'replace') for s in args)12:24
spaceone(if it is always unicode)12:24
prologicAhh I see12:24
spaceonebut this will fail in python312:24
spaceone(i hate python3)12:24
prologicme too12:24
prologicbut okay12:24
prologicI don't really want to do the above per se (yet)12:24
prologicI just want to join the args by a space12:24
prologicand not yet encode them to bytes12:25
spaceonewell, __repr__ expects bytes to be returned in py212:25
prologicand __str__ ?12:26
spaceoneotherwise it will just call internally bytes(your_returned_unicode_thing) around it12:26
spaceone__str__ expects bytes, too in py212:26
prologicis that the problem all along12:26
spaceoneof course12:26
prologicso implementing __bytes__ is useless here?12:27
spaceonethat's why __unicode__ exists in py212:27
prologicI should just get rid of it12:27
spaceonethere is no __bytes__ in py212:27
spaceoneprologic: you can have a look how i solved this in httoop12:27
spaceonei implemented __bytes__() and __unicode__() and differentiate in __str__ if is it py2 or py312:28
spaceoneit is*12:28
spaceonebut well, that have to be done similar for __repr__ of course12:29
prologic$ python -c 'import sys; sys.stdout.write("USER foo x x :foo\r\nNICK foo%s%s\r\n" % (u"→".encode("utf8"), u"ä".encode("latin1")))' | nc 700012:29
prologic:localhost NOTICE * :*** Looking up your hostname...12:29
prologic:localhost NOTICE * :*** Found your hostname12:29
prologic:localhost 432  foo→ä :Erroneous nickname12:29
prologicmuch better!12:29
prologicThank you!12:29
prologicSo I've been doing this wrong the whole time12:29
prologicWe need to implement __unicode__ instead12:29
spaceonewell, both imho12:30
prologiclook right now?12:30
prologicyeah both __unicode__ and __str__12:30
prologic__str__ does the .encode()12:30
spaceoneno, wait12:30
prologicShould I be fixing up circuits.web too like this?12:30
prologicsince it does this wrong as well12:30
prologicwhat did I foobar up? :)12:31
spaceoneI will comment there12:31
spaceonenice that you understand now :)12:35
prologic>>> from circuits.protocols.irc import Message12:39
prologic>>> args = ("NICK", u"→", u"ä",)12:39
prologic>>> m = Message(*args)12:39
prologic>>> m12:39
prologic"NICK → ä"12:39
prologic>>> print m12:39
prologicNICK → ä12:39
prologicthis stuff has always confused me tbqh12:39
prologicnot my strength :)12:39
prologicUpdated diff:
prologicSo should we do this also for circuits.web.wrappers?12:39
prologicAlso I saw your comment re __bytes__  # this still might raise UnicodeEncodeError with invalid user input!12:40
prologicIs there any practical thing we can do about that?12:40
prologicor is this a case of bad input; bad output12:40
prologicand the app has to handle it somehow?12:40
spaceoneyou can .encode(self.encoding, 'replace')12:42
prologicahh yeah12:42
prologicshould we?12:42
prologicbetter not to fail?12:43
spaceonedepends who is using it for what12:43
spaceoneif debugger uses it for printing it would be good12:43
spaceoneif it is used internally to compose the message back it would be broken12:43
prologicwell in this case it's Message() objects are written out to sockets12:43
spaceonein httoop it would fail12:43
prologicso that cannot/should-not fail12:44
prologicfor example12:44
spaceoneif it is written to sockets it should raise UnicodeEncodeError!12:44
prologicsomeone provides invalid input for NICK12:44
prologicand we cannot respond with ERR_ERRONESOUSNICK12:44
prologicbecause .encode() fails12:44
prologicso that sucks :)12:44
spaceonewell, that is better than sending broken data12:45
spaceoneDebugger should handle UnicodeError12:45
spaceonewell, the best would be to use repr() arround these strings when printing debug output!12:46
spaceonebecause repr will always return ascii12:46
spaceoneit escapes to \xff or \ufooo notation12:47
prologicif you note in my fix for Debugger12:47
prologicrepr() can fail with UnicodeEncodeError too!12:47
prologicbut maybe that's because of bad input12:48
spaceoneprologic: no, that is because the __repr__ implementations are broken12:49
spaceone 57     def __repr__(self):12:49
spaceone 58         return "\"{0:s}\"".format(str(self)[:-2])12:50
spaceonewhy not return repr(unicode(self))12:50
*** travis-ci has joined #circuits12:50
prologicahh :)12:50
travis-cicircuits/circuits#318 (fix-debugger-event-repr-errors#124 - 761559c : James Mills): The build was broken.12:50
travis-ciChange view :
travis-ciBuild details :
*** travis-ci has left #circuits ()12:50
prologic>>> from circuits.protocols.irc import Message12:52
prologic>>> args = ("NICK", u"→", u"ä",)12:52
prologic>>> m = Message(*args)12:52
prologic>>> m12:52
prologicu'NICK \u2192 \xe4'12:52
prologic>>> print m12:52
prologicNICK → ä12:52
prologiclike this?12:52
*** travis-ci has joined #circuits12:54
travis-cicircuits/circuits#320 (master - 761559c : James Mills): The build was broken.12:54
travis-ciChange view :
travis-ciBuild details :
*** travis-ci has left #circuits ()12:54
prologicI'm happy with that12:55
prologicre saferepr()12:55
prologicshould we still do this?12:55
prologicbut catch specific exceptions?12:55
spaceoneno, we should only use the first except-block which you already added12:55
prologicbecause without it Debugger can blow up12:55
spaceonethere is no need to call safererp()12:55
spaceoneyou are handling it already here:
prologicahh yes12:57
prologicwe may be handilng it there12:57
prologicbut that throws another exception too12:57
spaceoneyes it does12:58
prologicyou are right12:59
prologicit's not needed now that we've fixed the Message.__bytes__, Message.__str__ ?12:59
*** travis-ci has joined #circuits13:00
travis-cicircuits/circuits#321 (master - 489a019 : James Mills): The build is still failing.13:00
travis-ciChange view :
travis-ciBuild details :
*** travis-ci has left #circuits ()13:00
spaceoneyes. Well, we need to fix all the broken __repr__'s in circuits13:01
prologic2015-08-24 23:03:42,259 - charla.main - ERROR - ERROR <handler[*][join] (Commands.join)> (<join[commands] (<socket._socketobject object at 0x7fa12c64bfa0>, (u'foo\xe2\x86\x92\xe4', u'foo', u'::ffff:'), '#circuits' )>) (<type 'exceptions.UnicodeEncodeError'>): UnicodeEncodeError('ascii', u'foo\xe2\x86\x92\xe4', 3, 7, 'ordinal not in range(128)')13:01
prologicTraceback (most recent call last):13:01
prologic  File "/home/prologic/circuits/circuits/core/", line 681, in _dispatcher13:01
prologic    value = event_handler(*eargs, **ekwargs)13:01
prologic  File "/home/prologic/work/charla/charla/plugins/", line 17, in join13:01
prologic    replies = [JOIN(name, prefix=user.prefix)]13:01
prologic  File "/home/prologic/work/charla/charla/", line 76, in prefix13:01
prologic    return joinprefix(*self.source)13:01
prologic  File "/home/prologic/circuits/circuits/protocols/irc/", line 47, in joinprefix13:01
prologic    return "{0:s}!{1:s}@{2:s}".format(nick or "", user or "", host or "")13:01
prologicUnicodeEncodeError: 'ascii' codec can't encode characters in position 3-6: ordinal not in range(128)13:01
prologicstill get some exceptions though13:01
prologicI'm wondering whether circuits/protocols/irc/ is a bit broken too13:01
spaceonewell yes, because there is mixed-unicode-bytes everywhere13:02
spaceoneuse unicode everywhere wheere you expose a API or displaying things. Use bytes everywhere where you write to sockets / databases / files.13:03
spaceonenever mix these things13:03
prologicthis is easier in Python313:04
prologicbecause string literals are unicode13:04
prologicbut in Python2 you have to u""13:04
prologicI'm not sure which is more painful :)13:05
spaceoneor from __future__ import unicode_literals13:05
prologicwhich does?13:05
spaceoneyou don't need u'' then13:05
spaceoneit is always unicode then13:05
prologicshould we just put that at the top of every moduel then?13:05
spaceonewe could. but we have to check every string then.13:06
spaceonei am doing this in httoop13:06
*** travis-ci has joined #circuits13:06
travis-cicircuits/circuits#322 (master - 5fbbeb6 : James Mills): The build is still failing.13:06
travis-ciChange view :
travis-ciBuild details :
*** travis-ci has left #circuits ()13:06
prologicusing u"" everywhere?13:06
prologicwhere you expect unicode strings13:06
prologicand explicitely encoding to bytes for files/sockets13:06
spaceone(i am doing this in circuits.http but not yet in httoop)13:06
prologicI'll fix up what I can13:07
spaceoneand i'll review what I can13:07
prologicthanks :)13:07
spaceoneit can't be done really blindly. Python3 at least raises TypeError for unicode/bytes mixes13:08
spaceoneand py2 expects __str__ to return bytes (str is bytes) so with that future import one should not use: def __str__(self): return 'foo'13:09
spaceonemaybe the future-thing is only best-practice for py2 only projects13:11
spaceonethe gist looks okay. but it is a big API change.13:11
spaceonebut well, not really because there was no real API. it was broken in either py3 or py2 since beginning.13:12
prologicprecisely :)13:13
spaceonein py2 it used to have bytes as API and in py3 it used to have unicode as API13:13
prologicI don't really see "fixing things" as breaking API :)13:13
spaceoneokay ;)13:13
prologicThe API *is* meant to be Unicode IHMO13:13
prologicAs you said (and U did know; but never quite got it right)13:13
spaceoneok, that seems very fine13:13
prologicwe only encode to bytes for files/sockets13:14
prologicye ahyeah13:14
prologicI think I'll try to remember this one13:14
prologicUnicode everything!13:14
spaceonei need to work now :)13:14
prologic.encode() explicitly :)13:14
prologicThanks for helping silly 'ol me :)13:14
*** y0no has quit IRC13:22
*** y0no has joined #circuits13:27
*** Coldblackice_ has quit IRC13:29
*** Workster has quit IRC15:37
*** kdb has quit IRC15:53
*** metanosulfonico has joined #circuits18:52
*** Coldblackice_ has joined #circuits20:10
prologiclet's chat on /server 7000 today21:57
prologicgive it a good test21:57
prologicI implemented a bunch more features/commands last night21:57
prologicas well as some resource restrictions21:57
prologicspaceone, re unicode NameError and Py3; is that a six import tp fix that?21:58
*** metanosulfonico has quit IRC21:59
spaceoneprologic: text_type as unicode21:59
prologichow butt ugly :)21:59
prologicis there an alternative?21:59
prologicthat from __future__ thing21:59
prologicbut hmm21:59
prologicoh wait22:00
prologicsorry I'm tired :)22:00
prologicall good22:00
prologiconly had 3hrs sleep :)22:00
*** travis-ci has joined #circuits22:06
travis-cicircuits/circuits#323 (master - 3ab7681 : James Mills): The build is still failing.22:06
travis-ciChange view :
travis-ciBuild details :
*** travis-ci has left #circuits ()22:06
prologicstill failgine H?22:10
*** Workster has joined #circuits22:59
prologicso this sux23:27
prologicin Python 3.2 they remove the u literal23:27
prologicthen put it back in 3.323:27
prologicSo you can't really use u"foo" anywhere23:27
prologicif you want to support Python 3.223:27
prologicShould we even bother supporting 3.2?23:27
prologicIt's kind of a fucked up version of Python anyweay IHMO23:27
prologicSo as done with previous code23:30
prologicWe have to use circuits.six.u everywhere23:30
prologicfor where we define an API of Unicode strings23:30
prologice.g: u("foo")23:30
Worksterwhat relies on 3.2 that can't be done on 3.3 or 3.4?23:41

Generated by 2.11.0 by Marius Gedminas - find it at!