IRC Logs for #circuits Monday, 2014-04-14

prologicpdurbin: lol00:01
prologicis it annoying? :)00:01
pdurbinmeh. I used to like slashdot. back in 1999. it's fine00:02
pdurbinwent to see Taco when he came to town00:03
prologicfair enough00:04
prologicwell feel free to remove the feed00:04
prologicand add a better one00:04
prologicor better one(s) :)00:04
prologickdb’s rss plugin is also contextually aware00:04
prologicmeaining that if you asked it to join your channel you could have separate feeds for that channel00:05
prologicor even your own private nick if you /msg kdb radd …00:05
kdbRSS: Slashdot (http://slashdot.org/)00:05
kdb * Mr. Schmidt Goes To Washington: A Look Inside Google's Lobbying Behemoth: barlevg (2111272) writes "In May 2012, in the midst of an FTC investigation <http://rss.slashdot.org/~r/Slashdot/slashdot/~3/Ijn9UzZY_W4/story01.htm>00:05
pdurbingood to know. maybe I'll have it join a channel or two00:05
pdurbinit? him? her?00:06
prologicit :)00:07
prologicand if you do00:07
prologicbe sure to channels add <channel>00:07
prologicso it adds the channel to it’s startup join list00:07
prologicalthough I have actually tested sub commands00:07
prologicbetter quickly test it now :)00:08
prologic@help channels00:08
kdbChannel Management00:08
prologic@commands channels00:08
kdbAvailable commands for channels: channels part join00:08
prologichmm00:08
prologicbit sucky there00:08
prologiclet’s try it anyway00:08
prologic@channels add #kdb00:08
prologichmm nah I’d say it errored out00:09
prologic@errors00:09
c45yprologic: pong00:09
kdbErrors: 1700:09
prologic@channels add #kdb00:09
prologic@errors00:09
kdbErrors: 1800:09
prologicyeah00:09
prologicI’ll have to fix that tonight00:09
prologicc45y: heya :)00:09
prologicc45y: what we doing for lunch?00:09
prologicI didn’t bring anything today :(00:09
prologicread: my wife didn’t make me anything :)00:09
c45yumm I have no solid plans00:10
prologic@die00:10
*** kdb has quit IRC00:10
prologicsweet00:10
prologicpizza?00:10
c45yi'm down00:10
prologicawesome00:10
prologicI have a BCCVL standup at 1100-113000:11
prologicRossa at 11.35?00:11
prologicI’m not staying over the time allocated for anything :)00:11
c45yheh ok00:11
c45yI have to write terms and conditions :(00:11
*** kdb has joined #circuits00:11
prologicawesome!00:12
prologicwhat the legan team isn’t doing this for you?00:12
prologiclike wtf, what kind of sysadmin/devloper/networkadmin writes T&C?00:12
prologicwow :)00:12
prologicdo we ever read them? :)00:12
prologic@channels00:12
kdbNo command specified.00:12
prologic@channels list00:12
c45yI'm apparently supposed to do it00:12
c45yI write it, legal team approves it00:13
c45yor so I'm told by management00:13
c45yI'll make a note to get my job title modified00:13
c45yLegal systems engineer00:13
c45yitc00:13
prologicnah that’s bs00:15
prologicas if we’re qualified to write terms and conditions00:15
prologicseriously00:15
prologicwe’re not00:15
prologicI know of no developer, sysadmin or network admin taht does this00:15
prologicor has done this00:15
prologiclol00:15
prologicI reckon00:15
prologicand get a pay rise too!00:15
c45ygood plan!00:16
*** kdb has quit IRC00:18
*** kdb has joined #circuits00:19
prologic@channels list00:20
*** kdb has quit IRC00:23
*** kdb has joined #circuits00:23
prologic@channels list00:23
*** kdb has quit IRC00:24
*** kdb has joined #circuits00:26
prologic@channels list00:27
prologicreally?00:27
prologic@uptime00:27
kdbUptime: 0+0:1:33 (CPU: 0.36s 0.38%)00:27
prologic@channels add #kdb00:27
prologic@reload channels00:29
kdbUnloaded plugin: channels00:29
kdbLoaded plugin: Channels00:29
prologic@channels add #kdb00:29
prologic@channels list00:29
prologic@reload channels00:30
kdbUnloaded plugin: channels00:30
kdbLoaded plugin: Channels00:30
prologic@channels list00:30
kdbERROR: 'generator' object has no attribute 'value'00:30
prologic@reload channels00:30
kdbUnloaded plugin: channels00:30
kdbLoaded plugin: Channels00:30
prologic@channels list00:30
kdbStartup Join List: #circuits #kdb00:30
prologicyay00:31
prologic@channels del #kdb00:31
prologic@channels remove #kdb00:31
kdb#kdb removed from startup join list00:31
prologic@channels list00:31
kdbStartup Join List: #circuits00:31
*** kdb has quit IRC00:31
*** kdb has joined #circuits00:32
ircnotifier57c6e8940b57 by prologic: Fixed sub-commands with the Channels Plugin00:33
prologicpdurbin: fixed :)00:33
pdurbinso I see00:33
c45yprologic: where do you get your T&C from?00:51
c45yI've decided against going to jail00:51
prologicI don’t00:55
prologicthat’s the thing00:55
prologicit is completely outside of my scope of knowledge00:55
prologicand expertise00:55
prologicand quite frankly I could get into a lot of troulbe00:55
prologicI am NOT a lawyer not konw anything about many aspects of law00:55
prologicexcept odn’t murder, steal, etc :)00:55
prologicSo in effect, anything I have built and produced00:56
prologichas no T&C00:56
prologicand usually I license things as MIT00:56
prologici.e: the most permissive liecnese next to BSD00:56
prologicthere are absoultely no f’n warrenties or guarentees with anything I do, produce or offer up as  a service00:56
prologicecplitely so00:56
prologicif a project or employee wants different00:57
prologic_they_ have to take that responsiblity00:57
prologicand take _appropriate action_00:57
prologic_it is not_ apprioraite for a developer to do to IHMO00:57
prologicit’s like asking a carpenter to draw you up a legal document for the work he/she is about to do on your home00:57
prologicusually the company that employs him/her takes care of this00:58
prologicand might have their own legal team or have previously hired such a team to wrote contracts, t&c, service policies, etc00:58
c45ywhere do you get yours from is what I was asking00:58
c45ydo you have someone in legal to email?00:58
prologicmy advice00:58
prologicdon’t do it00:58
prologickinldly state the legal issues of you doing this00:58
prologicand walk away00:58
c45yI am...00:58
c45yI00:58
prologichold on00:58
prologiclet me ask :)00:58
c45yI'm looking for who to contact to write them for me lol00:59
prologicok01:01
prologicdid milo ask you to do this?01:01
prologicif so, my project manager will talk to him01:01
prologicso in effect, tehre is a process to go through01:01
prologicand you should not be the one doing this :)01:01
prologicaside from technically putting the page up somewhere, etc01:01
prologiclunch at 11.35 ok :)01:04
prologic@eval 60*3001:07
kdb180001:07
prologic@timer 1800 Lunch Time01:08
kdbTimer set for 1800s01:08
c45ytoo slow01:20
c45ygoing now01:20
c45yhurry along :P01:20
c45yping prologic01:20
prologicpong?01:21
prologicok let’s go :)01:21
c45ylunch now01:21
c45yyay01:21
c45ysee you up there01:21
kdbLunch Time01:38
prologiclol02:15
prologicfunny as02:15
prologicanyways :)02:15
prologic@rlist02:15
kdbNo feeds available.02:15
prologicoh damnit02:15
prologicit didn’t save the feeds02:15
prologichaha that’s because I didn’t configure a file path to save them to02:15
prologic@rehash02:21
kdbConfiguration reloaded.02:21
c45ys/.+/test/02:29
c45yprologic: make it sed pls02:29
prologichow shall it work?02:30
prologicdescribe the functional behaior :)02:30
prologicor better yet, write the plugin yourself :)02:30
prologicbeen thinkig of writing a plugin that allows users to write and load new plugins via it’s web itnerface :)02:30
prologiclol02:30
c45yterrible idea02:30
c45yopen('/etc/passwd').read()02:31
prologicdon’t be so quick to judge :)02:32
prologicsuch a plugin would have to have scrutinty02:32
prologic@spell scrutinty02:32
kdbscrutinty ? Try: scrutiny scrutiny's scrutineer scrutinize scrutineers secreting scrutinized02:32
prologicscrutiny* :)02:32
prologicprobably a process whereby it’s open access to all02:33
prologicbut once you write or moidfy a plugin02:33
prologicit has to go through a review process02:33
prologicor maybe just a plugin that can hot load plugins from codepad or pastebin or github/mercurial hosted services02:34
prologicbut where the loading of aush c aplugin is reviewed and scrutenized02:34
prologicI think I like the 2nd idea better because it means you can write the plugin however you want :)02:34
prologicin whatever tool method you want02:34
prologic@radd http://shortcircuit.net.au/~prologic/blog/feeds/tags/circuits.atom.xml02:43
kdbRSS: James Mills's Blog (http://shortcircuit.net.au/~prologic/blog/)02:43
kdb * Getting started with circuits: Here's a quick [ASCIInema](http://asciinema.org) tutorial I put together to <http://shortcircuit.net.au/~prologic/blog/article/2014/03/27/getting-started-with-circuits/>02:43
kdb * circuits + twisted: Well I've finally gone and done it! It's still alpha at this stage but twisted <http://shortcircuit.net.au/~prologic/blog/article/2013/08/24/circuits-twisted/>02:43
kdb * circuits-2.1.0 is out!: I'm pleased to announce the release of circuits 2.1.0 [] <http://shortcircuit.net.au/~prologic/blog/article/2013/02/27/circuits-210-is-out/>02:43
c45yprologic: I can write bad python that will clear your review02:44
prologicI very much doubt it :)02:46
prologictell you what, I’ll write the plugin02:46
prologicand I’ll see if you can abuse the system :)02:46
Romsterjust use gpg signatures02:47
prologicbut to be on the safe side02:47
prologicI’ll first Dockerize kdb02:47
c45ydeal02:47
prologicso you don’t destroy my Desktop :)02:47
Romsterlol02:47
prologicRomster: no02:47
prologicsignatures != security02:47
Romsterheartblleed bug :D02:47
prologiche could sign his work02:47
prologicand automatically be accepted02:47
prologicand then what?02:47
prologiche still nukes my file system?02:47
prologicno thanks02:47
c45yI could mitm your workstation :P02:47
Romsteronly if you accept his key02:47
prologicsignatures are not a form of security or scruteny02:48
prologicdoesn’t matter02:48
prologicI could accept his key02:48
prologicand he could still crew me over02:48
Romsterhe man in the middle i don't doubt that stupid openssl02:48
prologicthe only way to prevent bad code or malcious code02:48
prologicis to REVIEW REVIEW REVIEW02:48
prologicand run in ISOLATIOn02:48
c45yRomster: I'm also a sysadmin at his work :P02:48
Romsterminimal command set like each plugin neds it's own sandbox02:48
Romsterc45y, ya i know that but different department.02:49
c45ybyt python is hard to lock down02:49
prologicno the solutin is reather easy and simple02:49
Romsters/neds/needs/02:49
c45yunless you compile your own without stuff02:49
prologicI’m 99.9% sure c45y will not find a way to break it02:49
prologicDockerize kdb02:49
c45yor walk the ast02:49
prologicso it runs in a container02:49
prologicand 3rd-party plugins require review before being loaded02:49
prologicsimple02:49
c45yI'm game, sounds like fun02:49
Romsterc45y, is a programmer if anyone can find a way in he can02:49
prologicI accept your challange :)02:49
prologicit’s a done deal02:50
c45yprologic is also a programmer, by nature he can secure it as well as I can pop it02:50
prologicI’ll have the plugin ready tonight02:50
c45yhaha ok02:50
prologicthis is going to be oh so much fun :)02:50
prologicI might run this inside a virtualbox vm as well as containerize it02:50
c45yI'm not going to escape docker or even try02:51
prologicjust to be doubly sure you can’t break my desktop :)02:51
prologicyes but just in case you find a way to break out of the jail(s0 :)02:51
c45yI wouldn't do you any harm02:51
c45yI'm just planning on touching a file somewhere02:51
prologiceventually I would run kdb like this anyway02:51
prologicin isolation02:51
c45ykk02:51
c45ybest idea for most programs02:51
prologic*nods*02:51
prologicabsolutely02:51
prologicyou’ll have a hard time though02:51
prologicbecause you’ll have about 4 layers to get thorugh02:52
prologica) chroot’ed/jail02:52
prologicb) container02:52
prologicc) virtual machine02:52
prologicd) review process for plugins02:52
c45yand most of it will be blind02:52
prologicthat aren’t part of kdb itself02:52
prologic*nods*02:52
c45yI hate blind exploits02:52
prologicthis is why I’m 99.9% confident :)02:52
prologicbecause of the sheer no. of layers to get through02:52
c45yI'm not going to break out of docker, I'll tell you that now02:52
prologicyou have to firstly fool me :)02:52
prologiconce you get thorugh me02:52
c45ythat's easy02:52
c45yjust use small letters02:52
c45y:P02:52
prologicthen you have to find a way to circumvent chroot02:53
prologicif you manage to get rhogugh that02:53
prologicyou’ have to find a way out of the container to do anything harmful to the host02:53
prologicwhich will be a vm anyway02:53
prologicand to get past that to the actual metal host02:53
c45yI'm only going to be getting past kdb02:53
prologicwell you’re probably a better sysadmin tahn me :)02:53
prologiclol02:53
prologicuse small letters02:53
prologicyou’re so funny :)02:53
c45y^_^02:53
prologicooer that’s a new one02:54
prologicmy irc licnet even rendered that02:54
prologiclooks like a bird/chicne/ :)02:54
prologicor some kind of face02:54
c45yits just ^ and an underscore02:54
c45ycarrot?02:54
c45ycarret02:54
c45yIdk02:54
prologicahh02:54
prologic^_02:54
prologic*shrugs*02:54
prologicwerid02:54
c45y¯\_(ツ)_/¯02:55
prologicoh man this is going to be to easy02:55
prologichere’s my idea so far02:55
prologicmodify core to load external plugins02:55
prologicinitially just supporting raw content from a given uri02:55
prologice.g: codepad.org’s raw uri02:55
prologicor any raw capable uri that spits out plain text02:56
prologicget a sha hash of the enteire code02:56
prologicand put the hash, uri into a verification process02:56
prologicthe only trick is to only allow trusted persons to accept the verification02:56
prologiconce accepted02:57
prologicload the code, compile it and load the plugin as per normal02:57
c45yoh you're manually inspecting code?03:01
c45yI thought this was all an automated process03:01
prologicno no lol03:03
prologicI said it would go through a review03:03
c45yI thought it was something like a lexical review03:03
prologicright now I’m thinking the review process itseof would be password authenticated03:03
c45yI'm going to be hard pressed to get code past you03:04
c45yI give up now then03:04
prologicso it would be something like /msg kdb plugins review03:04
prologicfollowed by a /msg kdb plugins auth key password03:04
prologicwhere key is a unique hash of the plugin you want to enable after looking at it’s code03:04
prologicI thought you might say that03:05
prologicsorry for the confusion :)03:05
c45yI could get past a lexical review and even keyword blacklisting, but not manual inspection03:05
prologicthat’s what I meant by peer reviewed03:05
c45yWell, it would be hard03:05
c45ymaybe03:05
c45ylets still try03:05
prologicokay sure :)03:05
prologicI mean you may find other ways to break the system03:05
c45yjust ignore my strange os.system calls :P03:05
prologicmaybe03:05
prologiclol03:05
c45yI can think of a few03:05
prologicof course and obfuscation you attempt03:05
prologicI will throughly decode03:05
c45yit will be an off by one kinda thing03:06
prologic*nods*03:06
prologicplus also03:06
prologicwhilst the plugin load will “import” the module03:06
prologicso things in the module scope will “run"03:06
prologicit will only register subclasses of kdb.plugin.BasePlugin03:06
prologicso anything that isn’t a function in the module scope Il will go “hmmm"03:07
c45yseems reasonable03:07
c45ythis is going to be messy code03:07
c45yhow do you feel about space indenting :P03:07
prologicdon’t care03:08
prologiclike I said, I’ll take what you’ve written and psoted somewhere03:08
prologiccopy it to my editor03:08
prologicapply pep8 formatting to it03:08
c45yheh ok03:08
prologicmake it look ptetty in my eyes03:08
prologicand look at _every single lline_03:08
prologicso you kind ahave to get past 15 years of experience with Python :)03:09
c45yI'm game03:09
prologicsounds hard when you put it that way huh :)03:09
c45ynot so much03:09
c45ythat many years gets you used to conventions03:09
prologicI mena it’s not 100% fool proof03:09
prologicI may get lazy03:09
prologicor falter03:09
c45yexactly03:09
prologic*nods*03:09
prologicnow what would be good however03:09
prologicis an automagic way of analyzing the code03:09
prologic:)03:09
c45ythats what I was expecting03:09
prologicApple and Google and the likes have tried to develop such tools03:09
prologicwith varying degrees of success03:10
prologicyeah03:10
c45ythey have for the most part03:10
c45yappengine did well03:10
prologicI am not experienced with such tools03:10
prologicso I would not risk the dstrictuion of my desktop03:10
prologic:)03:10
c45ymost just pulled parts out of python and recompiled03:10
prologichuman review is more fool proof ihmo03:10
c45yyes and no03:10
prologicof course03:10
c45ylook at nearly every major exploit in the last however long :P03:10
prologicno system is completely fool proof03:10
c45yhuman review is always in the process03:10
c45yit just falls down03:11
prologicbut I doubt I’m smart enough to write such an analyzer03:11
c45ywalk the ast03:11
prologicit falls down because of the boave reasona03:11
prologiclaziness, mistakes and complexity03:11
prologicreviewing C would be hard03:11
prologicor C++, C#, Java, etc03:11
c45yC is easier for pointer failures03:11
prologicthe harder and more complex the tools and language03:11
c45ypython protects a lot of that03:11
prologicthe more the review process will fall down03:11
prologicprecisely03:11
prologicand despite that fact you can probably write really bad Python code03:12
prologicit is probably easier to reason about than say C#03:12
prologicor C++03:12
c45yeh03:12
c45yI dunno about htat03:12
c45ythat*03:12
prologicI do :)03:12
prologichaving come from those backgrounds too03:12
c45yC you would do bytecode analysis03:12
prologicand delving in such langauges and ecosystems03:12
prologicno I mean human review03:13
prologicit’s potentially harder and more complex to review C/C++/C#/Java etc03:13
c45yoh yea03:13
prologicbecause they are more complex and harder to reason about than say Python03:13
c45ythis looks legit right? https://gist.github.com/stevommmm/2801da9f90dc77f522d403:22
prologicwha’ts this for?03:24
prologicpython already caches module iimports in sys.modules03:24
c45yhow else am I going to sneak in exports?03:25
c45yn = imports()03:25
c45yprint n['time'].time()03:25
c45ys/exports/imports/03:25
prologicsure okay03:25
prologicbut this is kinda pointless if you’re trying to evade my measures03:25
prologicI’ll gladly accpe that as a valid plugin03:26
c45yits all part of the larger plan03:26
prologicadmitedly it does nothing03:26
prologicsure03:26
prologickeep going :)03:26
prologicI like where this is (or may be) going03:26
c45yreally sad python has no pointer parsing03:26
c45yso I have to get creative with strings03:26
prologicsure it does03:28
prologicyou could start using ctypes03:29
prologicor try to import cffi03:29
prologicwhich would not be necessarily available03:29
prologicunless kdb’s setup.py or requirements.txt installed cffi03:29
prologicwhich I’m also very happy to entertain03:29
c45yI can do it with the stdlib03:29
prologicwell there’s ctypes in the std lib03:30
prologickdb runs on Python 2.7 currently03:30
kdbUnknown Command: runs03:30
c45ylel03:31
prologicwhen I have the plugin ready, it’ll be in a vm in a docker container03:31
c45ykdb runs ls03:31
kdbUnknown Command: runs03:31
c45ykdb ls03:31
kdbUnknown Command: ls03:31
c45ywhats the plugin framework like?03:31
prologickdb: commands03:31
kdbAll available commands: load rehash plugins reload unload google uptime errors mstats graph inspect version nstats events cstats info commands help status spell rlog channels part join host resolve weather timer sum eval beat itime status quit die ircinfo jump nick say hello read rdel radd rlist03:31
prologicdefine like?03:31
c45ywhere can i see it :P03:31
prologichttps://bitbucekt.org/prologic/kdb/src/tip/kdb/plugins/__init__.py03:31
c45ykdb part irc.freenode.org03:31
prologichttps://bitbucket.org/prologic/kdb/src/tip/kdb/plugins/__init__.py03:32
c45ysafe__import__ confuses me03:34
c45yI still get to execute code03:34
c45yas long as I don't raise an error03:34
prologicit’s not the safe kind of import you think03:34
prologicit just guards against import errors03:34
c45yo i c03:34
prologic@load foobar03:34
kdbCould not load plugin: foobar Error: No module named foobar03:34
prologicsee03:34
c45y@load hashlib03:34
kdbLoaded plugin: hashlib03:35
c45ykdb hashlib03:35
kdbUnknown Command: hashlib03:35
prologichaha03:35
prologicgood find :)03:35
c45y;)03:35
c45y@load hashlib03:35
prologicin theory you could load anything in the std. lib03:35
kdbCould not load plugin: hashlib Error: reload() argument must be module03:35
prologicor part of kdb’s virtualenv03:35
c45y@load 'os.system(03:35
kdbCould not load plugin: 'os.system( Error: No module named 'os.system(03:35
c45yhmm03:35
prologicnice try03:35
prologic:)03:35
prologicunfortunately loading plain ol modules03:36
prologicwill not get you any accessible commands03:36
c45yYea03:36
prologicthe modules themselves would have to be malicous03:36
prologicso you could load up the entire std. lib if you wanted03:36
prologicyou’d probably blow kdb’s memory usage out a bit03:36
c45y@load n03:36
kdbCould not load plugin: n Error: No module named n03:36
prologicbut that’s probably the worst you could do03:36
prologic@load re03:36
kdbCould not load plugin: re Error: reload() argument must be module03:36
prologic@load sys03:36
kdbCould not load plugin: sys Error: reload() argument must be module03:36
prologichaha03:36
prologic@reload stats03:37
kdbUnloaded plugin: stats03:37
kdbLoaded plugin: Stats03:37
prologicinteresting03:37
prologic@load socket03:37
kdbCould not load plugin: socket Error: reload() argument must be module03:37
prologic@load wsgiref03:37
kdbLoaded plugin: wsgiref03:37
prologicahh03:37
prologicthere we go03:37
prologicapparently you can’t load stuff that’s alredy in sys.modules03:37
c45y@eval log03:37
kdbERROR: 'log' is not defined for expression 'log'03:37
c45y@eval log + 203:38
kdbERROR: 'log' is not defined for expression 'log + 2'03:38
c45y@eval 2*303:38
kdb603:38
prologiceval pulgin uses simpleeval03:38
c45y@eval 9*9*9*9*9*9*903:38
prologichttps://pypi.python.org/pypi/simpleeval03:38
kdb478296903:38
c45y@eval n+1 for n in range(9)03:39
kdbERROR: invalid syntax (<unknown>, line 1)03:39
c45y@eval self.config['autoid']['password'] + ""03:41
kdbERROR: Sorry, Attribute is not available in this evaluator03:41
c45y@rlog03:41
kdbLast 5 Remote Messages:03:41
c45y@jump irc.gamesurge.com03:43
*** kdb has quit IRC03:43
c45ylol03:43
*** kdb has joined #circuits03:43
prologicwell that’s not exactly an explit now is it :)03:44
prologicit’s designed to be allowed to jump to another server03:44
prologicor another network even03:44
c45ymmhmm03:44
c45y@timer 10 test03:47
kdbTimer set for 10s03:47
kdbtest03:47
c45y@timer 10 @timer 10 @timer 10 ?03:48
kdbTimer set for 10s03:48
kdb@timer 10 @timer 10 ?03:48
c45yhmm doesn't listen to its own chat lines03:48
c45y@eval self.auth['nick'] + ""03:49
kdbERROR: Sorry, Attribute is not available in this evaluator03:49
c45yI should get back to work03:49
c45y@timer 3 !ping03:50
kdbTimer set for 3s03:50
kdb!ping03:50
prologiclol03:53
prologicthis is kinda funny03:53
prologicnow if we have a 2nd instance of this up03:53
prologicin teh same channel03:53
prologicyou could abuse timer03:53
prologicand get the bots to trigger each other’s actions03:53
prologicthat could get messy and possibly Klined :)03:53
prologic@timer 0.5 foo03:54
kdbInvalid duration specified!03:54
prologic@timer 1 foo03:54
kdbTimer set for 1s03:54
kdbfoo03:54
prologicbut of course at worst it would be 1s apart03:54
prologicyou’d cause unncessary network traffic is all I think03:54
prologic:)03:57
c45y@load os sys json sqlite3 time socket SimpleHTTPServer SocketServer03:57
kdbCould not load plugin: os Error: reload() argument must be module03:57
c45y:/03:57
c45ymore heap size03:57
c45y!!!03:57
*** kdb2 has joined #circuits04:00
kdbHello kdb204:00
prologicthere you go04:01
prologicgo nuts :)04:01
c45y@timer 10 @timer 10 @timer 10 ?04:01
kdbTimer set for 10s04:01
c45ycan't think of a way for it to go forever04:01
kdb@timer 10 @timer 10 ?04:01
prologicit doesn’t have any puligns of interest loaded04:01
prologickdb2: plugins04:01
kdbUnknown Command: 2:04:01
kdb2Plugins: channels core irc help04:01
c45y@load timer04:01
kdbCould not load plugin: timer Error: No module named timer04:01
c45y@timer 10 @timer 10 @timer 10 ?04:02
kdbTimer set for 10s04:02
kdb@timer 10 @timer 10 ?04:02
prologickdb2: nick foobarbaz04:02
kdbUnknown Command: 2:04:02
c45y@nick lalalala04:02
prologickdb2_: nick kljhasdf04:02
lalalalaUnknown Command: 2_:04:02
c45yhaha04:02
prologiclalalala: load timer04:02
prologiclalalala: plugins04:02
prologicerr04:02
prologicit’s intenral nick state didn’t change04:02
prologicbug!04:03
prologicdamn :)04:03
prologickdb2: plugins04:03
lalalalaUnknown Command: 2:04:03
prologicbah04:03
c45y@load os04:03
lalalalaCould not load plugin: os Error: reload() argument must be module04:03
prologicthis isn’t going to work sorry :)04:03
prologichave to fix that internal state04:03
prologicI’ll fix that tonight too04:03
prologicit’ll be funny to see two bots hammer each other04:04
prologicand how bad it _could_ get04:04
*** marky1991 has quit IRC04:33
prologicHome time05:45
*** kdb has joined #circuits07:46
prologickdb, nick kjashlkjashdf07:46
Romsterif you made the bots say the delay command to the other bot07:59
*** Ossoleil has joined #circuits08:22
prologicRomster, sorry what?08:59
prologichi Ossoleil08:59
Ossoleilhi08:59
*** kdb has joined #circuits09:34
prologic@nick test09:34
prologictest, uptime09:34
testUptime: 0+0:0:33 (CPU: 0.31s 0.93%)09:34
prologickdb, uptime09:34
prologic@nick kdb09:34
prologic@ircinfo09:34
kdbI am kdb!kdb@daisy on the Unknown IRC Network. Connected to leguin.freenode.net Running version ircd-seven-1.1.309:35
prologic@nick test09:35
prologic@ircinfo09:35
testI am test!kdb@daisy on the Unknown IRC Network. Connected to leguin.freenode.net Running version ircd-seven-1.1.309:35
prologic@nick kdb09:35
ircnotifier9b0b9c683fd7 by prologic: Configure the RSS Plugin in the supplied sample configuration file.09:35
ircnotifier6372d1487e48 by prologic: Fixed state management09:35
prologic@load plugineditor10:12
kdbCould not load plugin: plugineditor Error: No module named plugin10:12
prologic@load plugineditor10:13
kdbLoaded plugin: PluginEditor10:13
prologic@reload pluginedito10:15
kdbPlugin pluginedito is not loaded!10:15
prologic@reload plugineditor10:15
kdbUnloaded plugin: plugineditor10:15
kdbLoaded plugin: PluginEditor10:15
prologic@reload plugineditor10:17
kdbUnloaded plugin: plugineditor10:17
kdbLoaded plugin: PluginEditor10:17
*** kdb has quit IRC10:18
*** kdb has joined #circuits10:19
prologicc45y, http://arrow.shortcircuit.net.au:9000/plugineditor/10:37
prologicahh man I really suck at UI :)10:37
prologicthis is going to take me longer than I had hoped for10:37
prologicjust because of a UI10:37
prologicc45y, help :)10:39
pdurbin@ircinfo10:57
kdbI am kdb!kdb@daisy on the Unknown IRC Network. Connected to verne.freenode.net Running version ircd-seven-1.1.310:57
prologichmm10:57
prologicI suck so much at ui10:57
prologictakes me too long10:57
prologictoo fiddly10:58
prologicI'm just going to do this the waqy I originally though10:58
prologicallow loading of remote plugins10:58
prologicvia url10:58
*** kdb has quit IRC10:58
*** kdb has joined #circuits10:59
Romster<a href="/projects/xine/files/xine-lib/1.2.3/" title="Click to enter 1.2.3" class="name">11:10
Romsterprologic, nope no java and it don't follow them.11:10
Romsteri eman no javascript11:10
Romstermean*11:10
Romstercurl -s http://sourceforge.net/projects/xine/files/xine-lib/1.2.4/ |grep /download returns files even11:12
Romsterhref="http://sourceforge.net/projects/xine/files/xine-lib/1.2.4/xine-lib-1.2.4.tar.xz/download"11:13
Romstercrawl --max_depth=3 --whitelist="^.*gnome\.org.*$"  http://ftp.gnome.org/pub/GNOME/sources/gobject-introspection/11:20
Romsterthat even fails to list the file sin /1.2.3/ directories.11:21
Romstereither i'm doing something wrong or it's got a parsing issue prologic11:21
Romstercrawl --max_depth=3 --whitelist="^.*gnome\.org.*$"  http://ftp.gnome.org/pub/GNOME/sources/gobject-introspection/1.40/11:22
Romsterthen i see gobject-introspection-1.40.0.tar.xz11:22
Romsterit's like you follow urls to other html pages but never transverse just sub directories.11:29
prologicRomster, try with -v/--verbose mode12:18
prologicand try to determine why it isn't following12:18
*** kdb has quit IRC12:48
*** jpenny_ has joined #circuits12:50
prologichi jpenny_12:54
*** kdb has joined #circuits12:57
prologic@load rplugins12:58
prologickdb, load broadcast12:58
kdbLoaded plugin: Broadcast12:58
prologic@load rplugins12:58
kdbCould not load plugin: rplugins Error: cannot import name mkdir12:58
*** kdb_ has joined #circuits12:59
*** kdb_ has quit IRC13:00
*** kdb has quit IRC13:00
*** kdb has joined #circuits13:01
prologic@rplugins load http://codepad.org/HTfQb9uG/raw.py13:01
*** kdb has quit IRC13:01
*** kdb has joined #circuits13:03
prologic@rplugins load http://codepad.org/HTfQb9uG/raw.py13:03
kdbRemote Plugin http://codepad.org/HTfQb9uG/raw.py (<built-in function hash>) not authorized.13:03
*** jpenny_ has quit IRC13:04
prologic@reload rplugins13:05
kdbUnloaded plugin: rplugins13:05
kdbLoaded plugin: RPlugins13:05
prologic@rplugins load http://codepad.org/HTfQb9uG/raw.py13:05
kdbRemote Plugin http://codepad.org/HTfQb9uG/raw.py (None) not authorized.13:05
prologic@reload rplugins13:05
kdbUnloaded plugin: rplugins13:05
kdbLoaded plugin: RPlugins13:05
prologic@rplugins load http://codepad.org/HTfQb9uG/raw.py13:06
kdbUnknown Command: rplugins13:06
prologic@rplugins load http://codepad.org/HTfQb9uG/raw.py13:06
kdbUnknown Command: rplugins13:06
prologic@reload rplugins13:06
kdbUnloaded plugin: rplugins13:06
kdbLoaded plugin: RPlugins13:06
prologic@rplugins load http://codepad.org/HTfQb9uG/raw.py13:06
kdbRemote Plugin http://codepad.org/HTfQb9uG/raw.py (9013e13aeccd3b43166a179539343ee59d8ce791) not authorized.13:06
*** kdb has quit IRC13:16
*** kdb has joined #circuits13:17
prologic@rplugins add http://codepad.org/HTfQb9uG/raw.py13:17
kdbRemote Plugin http://codepad.org/HTfQb9uG/raw.py (9013e13aeccd3b43166a179539343ee59d8ce791) not authorized.13:17
Romster (F): http://sourceforge.net/projects/xine/files/xine-lib/1.2.5/13:17
Romsterdoes the F mean they fail?13:18
prologicread the docs13:18
prologicF == Followed13:18
prologichttp://spyda.readthedocs.org/en/latest/13:18
Romsterman crawl13:18
RomsterNo manual entry for crawl13:18
Romsterright13:18
prologichttp://spyda.readthedocs.org/en/latest/api/spyda.html#module-spyda.crawler13:18
prologicI don't usually write man pages13:19
prologicsorry :)13:19
prologicI should get into the habit for cli apps13:19
Romsterok so it follows those urls and does not parse what's in those directories?13:19
RomsterF should mean i see what's in them directories.13:21
prologic@reload rplugins13:22
kdbUnloaded plugin: rplugins13:22
kdbLoaded plugin: RPlugins13:22
prologicwhat did it find after the (F) ?13:22
prologicany errors?13:22
prologicany urls founds?13:23
prologic@rplugins pending13:23
Romsterhttp://pastebin.com/DvA1Z0wJ13:25
Romsterhttp://pastebin.com/GZnRjqQi notice in tis versionsed directory see files, where the paste above should list files in all them versioned directories.13:26
Romsterand this same behavior happens on other sites.13:27
Romsteri need to be able to collect all them file names.13:28
Romsteri could damn near write a loop to regex match them directories and curl the contents of each. i did that but it was very unreliable due to html on sites.13:31
prologicsorry what are you not getting?13:31
prologicwhy are you still using a max_depth?13:32
Romsterhttp://sourceforge.net/projects/xine/files/xine-lib/1.1.18/xine-lib-1.1.18.tar.xz/download13:32
Romstersee that line ^13:32
Romsteri should have a ton of them i do not.13:32
Romsterfor each versined directory13:32
prologicand what paste is that meant to come from?13:32
prologicpage*13:32
Romstersee the top of the paste it has the command.13:33
Romsteris this hard to explain13:33
prologicstop using max_depth13:34
Romsteri had it off it ran indeffently13:34
Romsterjesus christ13:34
Romster2 levels should be more than enough13:34
prologicthen increase it until you get the results you want13:34
prologiccrawling is HARD13:34
prologicobviously it isn't?13:34
Romster20 levels then...13:35
prologicthis is not easy by any means13:36
prologicit is hard to get right13:37
Romsterthat's why i went looking for a spider my methods are limited.13:37
Romster20 levels deep and nope f it i'll do it with no limit13:37
prologicdon't get frustrated13:38
prologicpatience :)13:38
prologicGoogle spent billions of $$$ on their indexing and crawling13:39
Romsterhttp://pastebin.com/GfPjnrN4 that is how i am grabbing urls now.13:40
Romsterand yet i had a google bot stuck on my site could do nothign but hammer my site every hour because i never had any urls to outside sites for it to transverse years ago13:41
Romsterwhen i added one the google bot left me alone.13:41
prologicalso you should learn to read docs more carefully13:41
prologicand don't trust my word :)13:41
prologicF actually means (F)ound a valid URL13:41
prologic$ crawl -b ".*" -w "^http\:\/\/sourceforge\.net\/projects\/xine\/.*$" http://sourceforge.net/projects/xine/files/xine-lib/1.1.18/ -p "^.*download.*$" > urls13:42
Romsteri always look at the  -h of commands and man pages if they exist13:42
prologicthis is finding valid urls of interest13:42
prologicit's found 6 so far13:42
Romsterhmm13:42
prologicbut may take a while to complete13:42
Romsteri never escaped my dots...13:42
prologic*sigh*13:42
prologicdid I not say valid python regex?13:43
prologicthat means you must properly escape special characters13:43
prologic: / . etc13:43
Romsterno i'm just used to regular regex.13:43
Romsterand forgot toescape \.13:43
prologiceven regular regex you have to escape special chacrs13:44
prologic@reload rplugins13:44
kdbUnloaded plugin: rplugins13:44
kdbLoaded plugin: RPlugins13:44
prologic@rplugins pending13:44
kdbUnknown Command: rplugins13:44
*** kdb has quit IRC13:46
*** kdb has joined #circuits13:46
prologic@rplugins pending13:47
kdbNo Remote Plugins pending authorization.13:47
prologic@rplugins add http://codepad.org/HTfQb9uG/raw.py13:47
kdbRemote Plugin http://codepad.org/HTfQb9uG/raw.py (9013e13aeccd3b43166a179539343ee59d8ce791) pending authorization.13:47
prologic@rplugins pending13:47
kdbRemote Plugins Pending Authorization:13:47
kdb 9013e13aeccd3b43166a179539343ee59d8ce791 (http://codepad.org/HTfQb9uG/raw.py)13:47
prologic@rplugins auth13:47
kdbNo plugin specified.13:47
kdbAuthorization failed.13:47
kdbRemote Plugin  not found.13:47
prologic@reload rplugins13:49
kdbUnloaded plugin: rplugins13:49
kdbLoaded plugin: RPlugins13:49
prologic@rplugins auth13:49
kdbUnknown Command: rplugins13:49
*** kdb has quit IRC13:50
*** kdb has joined #circuits13:50
prologic@rplugins auth13:50
kdbNo plugin specified.13:50
prologic@rplugins auth asdf13:50
kdbAuthorization failed.13:50
kdbRemote Plugin asdf not found.13:50
*** kdb has quit IRC13:51
*** kdb has joined #circuits13:51
prologic@rplugins auth asdf13:51
kdbAuthorization failed.13:51
prologic@rplugins pending13:51
kdbNo Remote Plugins pending authorization.13:51
prologic@rplugins add http://codepad.org/HTfQb9uG/raw.py13:51
kdbRemote Plugin http://codepad.org/HTfQb9uG/raw.py (9013e13aeccd3b43166a179539343ee59d8ce791) pending authorization.13:51
prologic@rplugins pending13:51
Romsterhttp://pastebin.com/CwTz0ru0 still running but caught an excemption13:51
kdbRemote Plugins Pending Authorization:13:51
kdb 9013e13aeccd3b43166a179539343ee59d8ce791 (http://codepad.org/HTfQb9uG/raw.py)13:51
prologicRomster, no all pages parse correctly13:52
prologicbad html13:52
Romsterah13:52
Romsternot to fussed but isn't beatuelsoup meant to handle that13:52
Romsterman if it's gonna take this long to parse i'd be better writing more in lsurl to get subdirectories13:54
*** kdb has quit IRC13:55
*** kdb has joined #circuits13:55
prologiccrawling is not quick no13:56
prologicespecially since spyda is not doing any kind of async i?o right now13:56
prologicor multiprocessing13:56
Romsterneeds theading.13:56
*** kdb has quit IRC13:56
prologicno13:57
prologicneeds async i/o13:57
*** kdb has joined #circuits13:58
Romstermaybe if i could restrict it to look for directories it would be faster in searching then listing files13:58
prologicand multiprocessing for the cpu bound parsing13:58
prologic@rplugins add http://codepad.org/HTfQb9uG/raw.py13:58
Romsterthis might be a start if i can get it too actually list what i'm interested in seeing13:58
*** kdb has quit IRC13:59
Romsteri know spiders are complex beasts13:59
*** kdb has joined #circuits13:59
prologicI'm not sure why you don't just simply use requests13:59
prologicand lxml13:59
prologicto parse a specific page's content13:59
prologicrather than crawl all over the web for what you want13:59
prologic@rplugins add http://codepad.org/HTfQb9uG/raw.py14:00
kdbRemote Plugin http://codepad.org/HTfQb9uG/raw.py (9013e13aeccd3b43166a179539343ee59d8ce791) pending authorization.14:00
prologic@rplugins pending14:00
kdbRemote Plugins Pending Authorization:14:00
kdb 9013e13aeccd3b43166a179539343ee59d8ce791 (http://codepad.org/HTfQb9uG/raw.py)14:00
Romsteri have in lsurl but.... i want to expand this to search for files in said areas of said sites.14:00
prologic<kdb> Remote Plugin http://codepad.org/HTfQb9uG/raw.py (9013e13aeccd3b43166a179539343ee59d8ce791) successfully authorized.14:00
Romsterfrom multiple starting locations.14:00
prologic:)14:00
prologicof course I'm going to show anyone the password14:00
prologic:)14:00
prologic@load 9013e13aeccd3b43166a179539343ee59d8ce79114:00
kdbCould not load plugin: 9013e13aeccd3b43166a179539343ee59d8ce791 Error: Attempted relative import in non-package14:00
*** kdb has quit IRC14:01
Romstergood one change the pass.14:01
*** kdb has joined #circuits14:02
prologiceh?14:02
prologicwhat password?14:02
prologic@load 9013e13aeccd3b43166a179539343ee59d8ce79114:02
kdbLoaded plugin: Hello14:02
prologicsweet14:02
prologic@hello14:02
kdbHello World!14:02
kdbHello World!14:02
prologicawesome :)14:02
prologichaha14:02
prologicthe concept works!14:02
prologicman I'm brilliant14:02
prologicnow to Dockerize this14:02
*** kdb has quit IRC14:03
ircnotifier75e2179986a2 by prologic: Fixed sub-commands for Channels Plugin14:16
ircnotifierde08ea179b47 by prologic: Implemented Remote Plugins and Docker image14:16
ircnotifier7b4d12d99af3 by prologic: Fixed Dockerfile14:17
prologicc45y, oh boy are you gonna have some fun tomorrow :)14:18
prologicremote plugins ftw14:18
prologicanyone can add em14:18
prologicbut only the person with the authorization can authorize them :)14:18
ircnotifier48e226c38af5 by prologic: Use --allow-external in RUN pip in Dockerfile14:31
*** Ossoleil has quit IRC14:32
*** jpenny_ has joined #circuits14:34
ircnotifier3370a8d9b6d1 by prologic: Use --allow-all-external in RUN pip in Dockerfile14:47
*** kdb has joined #circuits14:47
*** kdb has quit IRC14:48
*** kdb has joined #circuits14:49
*** kdb has quit IRC14:49
ircnotifierd4123675eb6c by prologic: Use os.path.exists to check for config= file.14:51
*** kdb has joined #circuits15:02
*** kdb has quit IRC15:03
*** kdb has joined #circuits15:10
prologickdb, load broadcast15:11
kdbLoaded plugin: Broadcast15:11
prologic@uptime15:11
kdbUnknown Command: uptime15:11
prologic@load stats15:11
kdbLoaded plugin: Stats15:11
prologic@uptime15:11
kdbUptime: 0+0:0:2 (CPU: 0.21s 7.87%)15:11
prologic@uptime15:11
kdbUptime: 0+0:0:17 (CPU: 0.23s 1.30%)15:11
prologic@load rplugins15:11
kdbCould not load plugin: RPlugins Error: 'rplugins'15:11
*** kdb has quit IRC15:13
*** kdb has joined #circuits15:13
prologickdb, load rplugins15:13
kdbCould not load plugin: RPlugins Error: 'rplugins'15:13
*** kdb has quit IRC15:13
*** kdb has joined #circuits15:25
prologickdb, load rplugins15:25
kdbCould not load plugin: RPlugins Error: 'rplugins'15:25
*** kdb has quit IRC15:26
*** kdb has joined #circuits15:27
prologic@load 9013e13aeccd3b43166a179539343ee59d8ce79115:27
kdbLoaded plugin: Hello15:27
prologic@hello15:27
kdbHello World!15:27
kdbHello World!15:27
*** kdb has quit IRC15:27
*** kdb has joined #circuits15:28
prologickdb, load rplugins15:28
kdbCould not load plugin: RPlugins Error: 'rplugins'15:28
prologickdb, load broadcast15:28
kdbLoaded plugin: Broadcast15:28
prologic@load rplugins15:28
kdbCould not load plugin: RPlugins Error: 'rplugins'15:28
*** kdb has quit IRC15:29
*** kdb has joined #circuits15:29
prologickdb, load rplugins15:29
kdbCould not load plugin: RPlugins Error: 'rplugins'15:29
*** kdb has quit IRC15:30
*** kdb has joined #circuits15:31
prologickdb, load rplugins15:31
kdbCould not load plugin: RPlugins Error: 'rplugins'15:32
prologickdb, load rplugins15:32
kdbCould not load plugin: RPlugins Error:15:33
*** kdb has quit IRC15:33
ircnotifierc11248fb5caf by prologic: Print a traceback on Plugin load failure15:36
jpenny_Tried to pip install circuits.16:08
jpenny_Ran into a couple of problems.16:08
jpenny_in web/apps/memorymonitor/reftree.py, lines 178 and 180 print statements are being used.16:09
jpenny_web/apps/webconsole/__init__.py at line 61 is syntactically incorrect, not sure what is should be.16:10
jpenny_in examples/factorial.py, Task is spelled task.18:24
*** jpenny_ has quit IRC20:24
prologicHow is it that pip install circuits20:40
prologicstill gets you 2.1.0?20:40
prologicis it because of the 3.0.0.dev on pypi?20:40
*** kdb has joined #circuits20:56
ircnotifierd255789678f3 by prologic: Move example config into etc/20:57
ircnotifier52d383cc64aa by prologic: Move example config into etc/20:57
ircnotifierbaffd5a7df93 by prologic: Updated README with Docker instructions20:59
prologic@load 9013e13aeccd3b43166a179539343ee59d8ce79120:59
kdbCould not load plugin: 9013e13aeccd3b43166a179539343ee59d8ce791 Error: No module named 9013e13aeccd3b43166a179539343ee59d8ce79120:59
prologic@plugins21:00
kdbPlugins: channels google stats help xmlrpc core ctcp greeting rmessage broadcast dnstools weather swatch web eval timers rplugins irc hello21:00
*** kdb has quit IRC21:00
*** kdb has joined #circuits21:01
prologic@uptime21:01
kdbUptime: 0+0:0:47 (CPU: 0.29s 0.61%)21:01
prologicc45y: good morning22:49
prologicc45y: it’s ready for you to test :)22:49
prologic@rplugins add http://codepad.org/NtkPvwDj/raw.py23:59
kdbRemote Plugin http://codepad.org/NtkPvwDj/raw.py (9013e13aeccd3b43166a179539343ee59d8ce791) pending authorization.23:59

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!