IRC Logs for #crimsonfu Friday, 2018-12-07

benei'm pretty sure you can generate a letsencrypt cert for an amazon default dns entry for an ec2 instance, e.g. ec2-18-232-90-63.compute-1.amazonaws.com01:11
beneunless there is some specific prohibition against doing it on amazon01:12
benei just spun up a throwaway linode instance to test:01:12
benePlease enter in your domain name(s) (comma and/or space separated)  (Enter 'c'01:12
beneto cancel): li422-77.members.linode.com01:12
beneObtaining a new certificate01:12
benePerforming the following challenges:01:13
benehttp-01 challenge for li422-77.members.linode.com01:13
beneWaiting for verification...01:13
beneCleaning up challenges01:13
bene - Congratulations! Your certificate and chain have been saved at:01:13
bene   /etc/letsencrypt/live/
beneroot@localhost:~# openssl x509 -noout -subject -in /etc/letsencrypt/live/
benesubject=CN = li422-77.members.linode.com01:14
benehere's the thing01:20
beneall of these technologies we use are in some sense fungible01:21
beneyou can get an ssl cert on your ec2 instance via terraform or cfn or ansible or letsencrypt or via a bash script01:22
benesome will be easier to build/support than others, but the end result is largely the same01:22
pdurbinInteresting. Do you *don't* need control of DNS for the domain for Let's Encrypt. Thanks for testing this, bene01:24
benethe harder/more important questions in my mind are not the "how can i do X?" ones, but "should i be doing X?" or "how does doing X further my longer term strategic goals?" type questions01:25
beneit's the peter drucker quote01:25
beneefficiency is doing things right; effectiveness is doing the right things.01:26
pdurbinmy wife is reading one if his books right now... "the effective executive" or something... she's enjoying it01:26
pdurbinI want hear more about 35 installations vs 350 vs 3500 and wasting time. What were you saying?01:27
beneif you want to generate a cert for foo.yourcooldomain.tld, then you _will_ need control to create that A record/CNAME01:27
benebut for an existing dns entry, you can just roll with what you got01:27
beneso that's 35 installations over... the last 5 years?01:27
benelike are you up to 10 new installs a year?01:28
pdurbin12 years01:28
benebut presumably there's a growth curve there? :-)01:28
beneso it was 1-1-1-0-1-2-2-3-5-7... ?01:28
pdurbinbene: here's the curve:
benethat's what i'm talking about01:29
benein some way01:29
beneso you're up to 35 ish and your end goal is... 100s? 1000s?01:29
pdurbinworld domination01:29
beneyou want every higher ed institution in the world to run one of these things?01:29
pdurbinbuh. sure01:30
pdurbinhow many are there?01:30
beneno idea01:30
beneat some point i spitballed there being around 3500 structural biology labs in the world with maybe 2500 in countries that could actually pay for sbgrid services01:31
benebut i don't remember my methodology01:31
pdurbin26,000 according to
beneanyway, my point was mostly that you should be spending time optimizing/operationalizing things that your users do regularly01:32
pdurbinfixing bugs and adding features?01:33
beneand prod installs are probably not that common an occurrence01:33
benesure, but also spending a lot of time on ensuring the upgrade process is smooth and painless01:33
pdurbinoh, upgrades01:33
pdurbinwe're working on that. adding flyway01:33
benebuilding things is frequently rather simple01:33
pdurbin(flyway is a java database migration thingie)01:34
benemaintaining/supporting them is usually much harder01:34
beneand you have the updates on your server within 24 hours and all your users updated within 72 more hours01:35
pdurbinwell, you can lead a horse to water but you can't make him upgrade glassfish... but sure, I hear you01:35
benewhat are the maintenance requirements for these things?01:35
pdurbinwell, we put out maybe 6 releases a year01:36
benebut security updates are a real thing that all your sysadmin/supporting users will have to do on a somewhat regular basis01:36
pdurbinpeople can choose to upgrade or not01:36
pdurbinsome upgrade more frequenty than others01:37
pdurbinof course we want everyone to stay with us, to get on the latest version01:37
pdurbinwe don't want to answer questions about old versions01:37
benenods, older versions in the wild usually increase the support burden01:37
benedo these things federate with each other in any way?01:38
pdurbinthey "harvest" metadata from each other via the OAI-PMH protocol01:38
pdurbinmetadata about datasets, not the files themselves01:38
pdurbinexcept for the sbgrid installation, which mirrors files all over the world with globus, but those globus sites don't need to be running Dataverse01:39
pdurbinBut I'm still confused. You don't like Oliver's suggestion to have a meeting to talk about running Dataverse on Kubernetes? Why not?01:40
benehis suggestion is fine01:40
beneand kubernetes is fine01:40
beneor openshift or aws or whatever01:40
pdurbinSo what are you objecting to?01:42
benebut if you're going to turn this into something more like a "product" then you probably need to pick the 2-3 most common installation methods/use cases and then officially build and support those01:42
beneand everyone else gets to roll their own01:43
benei'm not objecting to anything exactly01:43
pdurbinI guess I would need to understand what the "methods" could be. I guess you're saying "run it on Kuburnetes" is a method. We definitely don't support that now.01:44
pdurbintoo fancy01:44
benebut i was browsing through that dataverse-ansible repo and it all looks pretty half baked to me01:44
beneACTION shrugs01:45
pdurbinto be honest, I have barely looked at it. from my perspective it works well enough01:45
benenot the first time people have objected to my assessments of technology :-)01:45
pdurbinthe developer is very responsive01:45
pdurbinrough consensus and working code, right?01:46
pdurbinI feel like we only support one installation method: do a bunch of manual setup. Then run our Perl script.01:47
beneyou can build things in an ad hoc way for quite a long time, but it will eventually bite you :-)01:47
pdurbinah, maybe you don't know that dataverse-ansible is entirely community contributed and community supported01:48
beneyou going to abcd tomorrow?01:48
pdurbinyeah, it's noted in the bottom of the readme01:49
pdurbin"This is a community effort"01:49
pdurbinyeah, I'm thinking I will01:49
pdurbinleaning toward it01:49
beneyeah, i read that01:49
beneit's going to be a squishy talk01:50
pdurbinsounds meta, I guess?01:50
beneheh, something like that01:50
pdurbinabout abcd itself?01:50
pdurbinI guess it's been a year since I gave a talk. Did you come? Sorry that I can't remember.01:51
benei did01:51
beneyou were prepping for javaone?01:51
pdurbinvery nice of you01:51
benei'm a nice guy :-)01:51
pdurbinyeah, it was my practice talk01:51
bene"Facebook will provide privacy-preserving data and access (through Dataverse)"01:52
pdurbinI had more fun giving it at abcd than javaone. Bigger crowd, maybe because of the free food.01:52
benecolor me skeptical01:52
pdurbinis that in those slides?01:53
benei was having a beer with ijstokes a few years ago back when he was working for SEAS01:53
beneand he told me the story of an undergrad who had gotten his compute account locked because he downloaded ~700 TB of data into his home directory on one of the SANs01:54
pdurbinah, slide 26. huh. I missed this talk, actually01:54
beneso he walked into the office to get his account unlocked and ian asked him what all that data was01:54
beneand it turned out the kid had signed up for a facebook developer account and just downloaded SCADS of user profile data and then cached it all locally01:54
benewhich was against the TOS but not really possible for facebook to police01:55
pdurbingood times01:55
pdurbinremind me to tell you about the level 4 data01:55
benethis was like 2012?01:55
beneback when 700 TB was actually a lot of data01:55
pdurbinbene: this is the new facebook thing, if you're interested: https://socialscience.one01:58
pdurbinbut not related to those slides, I don't think. separate thing01:59
benethat's a cool library01:59
pdurbinthis new thing is about democracy02:00
benenot very practical layout for that library, but it *looks* pretty awesome02:00
benei'm pretty sure facebook as a for-profit company is not compatible with democracy02:00
pdurbinwhy can't I vote through facebook?02:01
benenow you're just trolling :P02:02
benei'm putting some loud children to bed02:02
pdurbinme too02:02
pdurbinlook how hard we're working on getting them to bed02:02
pdurbinI better get in there. It's getting louder.02:02
pdurbinactually, they're talking about bras. no thanks. not my department but my wife has a work event tonight02:04
pdurbinactually, I guess that slide is related02:07
pdurbinI don't know. I'm just trying to make the open source project I work on the best it can be. It's a long road.02:08
beneno doubt02:22
pdurbinI'm actually surprised we're talking about Kubernetes at all.02:23
pdurbinOur app is pretty old school.02:23
beneit's the new hotness for sure02:23
pdurbinThe interest in Kubernetes is coming from the outside, from the community.02:23
pdurbinIt's nice. It's healthy for our community to push us a bit.02:24
pdurbinYou can lead a J2EE app to Kubernetes but you can't make it jump in a pod.02:25
bene <- some grist for the mill03:14
*** jri has joined #crimsonfu08:22
*** jri has quit IRC16:38

Generated by 2.14.0 by Marius Gedminas - find it at!