IRC Logs for #circuits-dev Tuesday, 2013-04-09

prologichey Osso00:01
Ossohello00:01
Ossoso do you think we should optimise tasks?00:02
Ossoas in waitEvent00:02
prologicIs it actually slow?00:03
Ossodepends on what you define slow00:04
prologicwell what I was noticing was00:04
Ossoit's called all the time to check that the status is not done00:04
prologicif you had nothing but tasks left00:04
prologicit iterated over the tasks quite slowly00:05
Ossoit is going through all the tasks at each tick00:06
Ossosay you have a 1000 tasks waiting00:06
Ossoit's 1k function calls per tick00:06
Ossothat's not nice00:06
prologichmm00:13
prologicdoesn't the FallbackGenerator kick in though?00:13
Ossowell yes and no00:18
Ossoit'll do the 1k function calls then sleep for TIMEOUT :)00:18
prologichmm00:42
prologicyeah I guess my point is we need the tasks to happen quickly00:42
prologicand perhaps concurrently00:42
prologicas well as normal I/O as well00:42
Ossothey do00:45
OssoI checked there's no delay in the event happening and the task being triggered00:46
prologichmm ok00:47
prologicwhat was that suggested change to the crawler?00:47
prologicAlso removing singeton checks sped things up quite a lot00:48
prologicbut also I think we need to add support for multiple-outgoing connections to our Client sockets00:48
prologicso we're not creating and destroying components just to do web craweling00:48
*** Osso has quit IRC02:44
*** Osso has joined #circuits-dev02:45
Ossosorry had a meeting then I went to eat02:48
OssoI implemented the depth of links handling02:49
Ossoalso it's commenting the reduce_time to 0 which prevents sleeping02:50
*** Osso has quit IRC08:24
*** logix812 has joined #circuits-dev09:12
*** Osso has joined #circuits-dev10:48
Ossothe watcher will never see itself unregistered13:40
Ossocause it is unregistered :p13:41
circuits6f2e04d5e328 by osso: Optimizes waitEvent13:54
*** Osso has quit IRC14:10
circuitsd0afc5d78fcf by prologic: Fixed some Python 3 compatibility issues with bytes vs. text16:48
circuits966980bdd8f9 by prologic: Fixed condition checking against string types16:48
*** logix812 has quit IRC23:16
*** Osso has joined #circuits-dev23:21
*** Osso has quit IRC23:33

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!