IRC Logs for #crux Sunday, 2017-06-04

*** irclogger_do has joined #crux16:54
prologicokay and we're all back up16:54
prologicnow let's add monitoring and alarming16:54
prologicand fix that disk space leakage (leaking container runs)16:54
*** Guest1309 has joined #crux17:15
*** abenz_ has joined #crux17:27
*** Guest1309 has left #crux ("Leaving")17:35
prologicsent 876,989 bytes  received 169,525,086 bytes  3,070,307.66 bytes/sec18:14
prologictotal size is 852,024,123  speedup is 5.0018:14
prologicroot@vz2:/data#18:14
prologicheh I kind of expected more data than this18:14
prologicI feel like a 50GB volume is overkill :D18:14
frinnst:>18:21
frinnstcheers again18:21
*** john_cephalopoda has joined #crux18:26
*** ubuuu has joined #crux18:29
prologicfrinnst: btw, familiar much with synchronization with shared volumes?18:29
prologicthinking of attaching this volume to two instances in different regions18:29
prologicbut writing to the same file from two bot instances of irclogger seems like a bad idea (corruption/conflict)18:30
frinnstnot really, no18:30
prologicrats :)18:31
prologicI'll go ask in #sourcefu -- there are smart (developer) cookies over there :P18:31
frinnstwhat are you running it on?18:32
prologicirclogger?18:32
frinnstyeah18:32
prologicPython app (using circuits / async stuff) -> runs in Docker container -> Runs on an Ubuntu/Docker host -> on a DO Droplet18:33
frinnstdocker with lvm backend?18:33
prologicumm nfi tbh18:33
prologicits the one-click app version of Docker18:33
prologicvery old on this particular host though18:33
prologichence my spinning up vz2 (a new replacement) with separate storage volume18:34
frinnstah, so the shared volume has nothing to do with the ircbot then i guess :)18:34
prologicnope18:34
prologicbut it will :)18:34
prologicI just rsync'd (above) all the data over18:34
prologicbasically the way I see this; I need to add/modify irclogger to be "distributed" aware18:35
prologicwhatever that means18:35
frinnststore in a db maybe?18:35
prologicor (probably more complex) write irc logs into a stream and have another process do the actual work of processing them into the actual logs18:35
frinnstmysql master/slave/slave ?18:35
prologicoh god wow18:36
frinnst:) well it would work18:36
prologiccomplex much? :P18:36
prologicsure18:36
prologicbut seriously :)18:36
prologicI have enough complex issues to deal with with our own mysql clusters at work :D18:36
frinnstwe run our dns servers just as mysql ro slaves18:36
prologichaha18:36
frinnstits awesome. but makes it hard to play nice with others if we want to do dns slave syncs18:37
prologicI rely on DO externally nowadays18:37
prologicand udns (my own) internally18:37
prologicredis backed18:37
prologictbh the stream idea is probably the best and most robust18:38
prologicit would also solve the split network problem (depending)18:38
prologicbut you'd have failover/redundancy too either way18:38
prologic*meh* irclogger is only ~350 LOC (https://github.com/circuits/irclogger/blob/master/irclogger/main.py)18:39
prologicI'll do it :D18:39
*** TheCephalopod has joined #crux18:43
*** ileach has joined #crux18:43
*** john_cephalopoda has quit IRC18:45
jaegerput log messages into a redis queue and pull them out on the other sides18:53
jaegers/redis/<insert your favorite queue software here>/18:54
rmullteK__: New libreoffice available, for when you have a moment. Thanks19:00
prologicjaeger: you're funny :D19:07
jaeger:D19:07
prologicI was just talking to bear in #sourcefu about this19:07
prologiche suggested the same thing too19:07
prologicbut then realized after looking at the code (https://github.com/circuits/irclogger) that I'm already using the file system as a queue :D19:08
jaegerAh. I haven't looked at the code19:08
prologicmy plan now is to just let multiple instances write/append to the same raw log files and adapt the processing to merge results, ordered by timestamp, hashed to skip identical lines and detect torn writes (missing \n)19:08
prologicit's only ~350 sloc :D19:09
prologicthat's why I don't want to make this more complex like adding redis/mysql/whatever into the mix :D19:09
*** john_cephalopoda has joined #crux19:44
*** ubuuu has quit IRC19:57
*** snux has joined #crux20:02
*** snux has quit IRC20:25
*** teK___ has joined #crux20:39
*** cruxbot has joined #crux20:47
*** darthlukan has joined #crux20:51
*** moth has joined #crux21:17
*** john_cephalopoda has left #crux ("Trees can see into your soul.")21:59
*** chinarulezzz has joined #crux22:03
*** abenz_ has joined #crux23:40
*** onodera_ has joined #crux23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!