IRC Logs for #cmvt Thursday, 2013-05-16

ircnotifier999f1321a7d0 by prologic: Added, deploy.test convenience subtasks00:07
ircnotifier56da2e55f3ec by prologic: Fixed deploy convenience subtasks00:09
prologiclet's talk more about it when you get back from you're meeting (which I can hear) :)00:12
prologicre-testing full provisioning and deployment locally00:46
prologicthere are some gotchas with CentOS00:46
prologicthat I had to take care of in the playbooks00:46
prologicspecifically around allowing webdevs to use sudo without a tty00:46
prologicand adding /usr/local/bin to the secure_path00:46
prologicmight get you to run a test too soon00:46
prologicI'll see if it all works here first00:46
prologicI added those convenience subtasks00:47
prologicnow "fab deploy" will by default deploy to with the leafs code00:47
prologicand there's also00:47
prologicfab deploy.test00:47
prologicfab deploy.prod00:47
DanielBairdgood idea, the short / least explicit one does the safest thing00:48
prologicbbs - lunch01:55
ircnotifier2c6afe9550b9 by prologic: Use a file instead of a template for this02:00
ircnotifier512ada4bd9ed by prologic: Fixed up sudoers config02:00
ircnotifierc8aac0091814 by prologic: Fixed typo02:00
ircnotifier88dca7236181 by prologic: Fixed initialization of virtualenv for webapps user02:00
ircnotifierbe52298ffb1d by prologic: Fixed apache NameVirtualHost config02:00
DanielBairdI don't know if that's the format the final set of 250m data will be in, or just a random testing output02:56
prologicmy understanding is that we need it in the same format as the other dataesets02:58
prologicI especially would like it to be so02:58
prologicless work that I have to do :)02:58
prologicCould you test the playbook against a blank vm?02:58
prologicI believe they're ready to re-test02:58
ircnotifierd1b82ac561af by prologic: Fixed apache NameVirtualHost config02:58
ircnotifier1a7163a489e0 by prologic: Fixed apache NameVirtualHost config02:58
ircnotifieraa9a09cf8561 by prologic: Fixed creation of test virtualenv02:58
ircnotifier920ba72a616f by prologic: Fixed creation of test virtualenv02:58
ircnotifier3c1f42b7d535 by prologic: Don't install node dependencies globally03:05
prologicI'm having problems with nom now03:15
prologicI can't seem to get it to install something locally03:15
prologicand actually install the coffee entry point to ~/bin03:15
DanielBairdi had something in the vagrantfile back when i was installing node.. stand by03:16
prologicthe problem is I don't want to install globally03:18
prologicI want to install locally03:18
DanielBairdas in, the npm binary?03:18
prologicnpm install coffee-script03:19
prologicbut it doesn't add anything to ~/bin03:19
prologicno, nom is there goobally03:19
prologicI'm trying to install nom modules/packages locally though03:19
prologicand it's fucking up03:19
DanielBairdyeah so you want the npm-installed things to be local.. yes03:19
prologicI do not want fabric tasks to have to do anything as root03:19
prologiceverything I've read says this should just work03:20
prologicbut it bloody doesn't03:20
DanielBairdi think npm tries to install into the local project dir, doesn't it?03:20
prologicbut I don't want that behavior either03:21
prologicI just want to install foo03:21
prologicand expect foo in ~/bin03:21
DanielBairdyeah i think npm only supports global or project, not user03:21
prologicoh great03:22
prologicyou're kidding me?03:22
DanielBairdso if you want non-global, you have to make a ./node_modules dir, and npm will install stuff there.  then you need to add node_modules/bin to the path or whtaevber03:22
prologicoh ffs03:22
prologicit's too bad there isn't a coffee-script compiler written in Python03:23
prologicthere's a less compiler written in python03:23
DanielBairdi wonder if you made a node_modules folder in your user dir03:23
DanielBairdis the project dir inside the user home?03:23
DanielBairdi think npm walks up the dir chain looking for the node_modules dir03:24
prologicyeah no03:25
DanielBairdthe easiest solution is to install coffeescript globally.  every other compiler on the system is global03:25
prologicthe problem is it just doesn't install the binary anywhere03:25
prologicit's stupid03:25
prologicnpm is a pos03:25
prologicyes this is true03:26
prologicI should probably just do this then03:26
prologicbut gah03:26
prologicI'll have to weave it back in to the playbooks03:26
prologicI'll do that03:26
prologicbut still03:26
prologicffs :)03:26
prologicwtf were they thinking seriously03:26
prologicevery other package manager does the right thing03:26
prologicpip, easy_install (python)03:27
DanielBairdor, you could curl the source out to some online coffeescript compiler and save the returned output :)03:27
prologicgem (ruby)03:27
prologichang on03:27
DanielBairdyeah the js culture enjoys being different03:27
DanielBairdthere's npm configs, might be able to set the install path.  Still think just use a global coffee install, though03:31
ircnotifier7b7a5c6b82fc by prologic: Install commonly used NodeJS tools03:40
ircnotifier5e517ac90fd2 by prologic: Fixed typo03:40
ircnotifier5e8540a79ba2 by prologic: Removed the use of npm and the npm function and the installation of coffee-script/ess (done by playbooks now)03:43
DanielBairdi'm using these settings, i think they're correct but i suspect i'll also need to fiddle the "ask for sudo password" or something also.03:48
DanielBaird        ansible.playbook = "../playbooks/site.yml"03:48
DanielBaird        ansible.inventory_file = "../playbooks/hosts"03:48
DanielBaird        ansible.limit = "vagrantvm"03:48
prologicI'll need to add in an entry into the hosts file then03:53
prologicyou'll want to add a limit option03:53
prologicthe CLI version of this is -l <pattern>03:53
DanielBairdi have a vagrantvm group which the limit line is picking out03:53
prologicahh k03:53
prologicgood good03:53
prologicis that committed?03:53
DanielBairdyeah i think so.. you mightn't notice coz you'd be limiting to your own group03:54
DanielBairdoh i mean the group in the hosts is committed.  the change to the vagrantfile isn't committed yet03:54
DanielBairdhmm.. i brew-installed something and just noticed that brew has little beer mugs in its output.03:57
ircnotifier18cd8edfea67 by prologic: Code cleanup04:06
ircnotifier7bcad9bd4b5f by prologic: Code cleanup04:06
ircnotifiera94b769bf77d by prologic: Fixed a really really broken @requires(...) decorator :/ Bad James :/04:06
DanielBaird_finally_ my vm is starting04:07
ircnotifierb49fc49164d9 by prologic: Added graceful kwarg to restart_apache()04:08
DanielBairdwhat's your normal ansible command line?04:08
ircnotifier71e2c1f0ba44 by prologic: Fixed typos04:09
DanielBairdping, ahoy hoy04:10
prologicnormally it's:04:10
prologicansible-playbook -k -u root -l test_local_proxmox site.yml04:10
prologicfor local testing04:11
prologicansible-playbook -s -u ec2-user -l test_jmils_nectar site.yml04:11
prologicfor nectar deploy/testing04:11
DanielBairdand -k is "ask for password", right?04:11
prologicand -s is use sudo04:11
prologicnectar cents vm has ec2-user to ssh in as04:11
prologicand must sudo up04:11
prologicmy own vm locally I can just ssh in as root04:11
DanielBairdhmm didn't get ansible inside vagrant working yet, but i'm running playbooks manually and they're working04:21
DanielBairdi get this: fatal: [] => ssh connection closed waiting for password prompt04:21
DanielBairdimmediately after entering the vagrant root user password.  after this i'll try running ansible in verbose mode04:22
prologicpaste what your terminal output looks like?04:37
prologicfrom when you entered the command04:37
prologicoh wait04:38
prologicyou're running this within the vagrant script rigt?04:38
prologicI'm not sure how prompt stuff and what not would work04:38
prologicif you can run it manually04:39
prologicI'm sure you can figure out how to get vagrant to run it automatically04:39
prologicon vm creation04:39
DanielBairdyeah agree04:43
DanielBairdjust gotta work it out04:43
prologiclet me know though if you run into any bugs with the plays04:44
prologicdid you see I integrated the version into the css04:44
prologicand the title tag of the index.html04:44
prologicI made this a simple template04:44
DanielBairdi'm running a slightly older version according to the irc logs of the last hour or so..04:44
prologicthat juste gets compiled into the real thing04:44
DanielBairdlooking through teh fabfiles now, while i wait for ansible and IEVMs to install (figured i'd finally sort out IE testing VMs on this macbook)04:50
DanielBairdi might try out r.js.  ig i can get it working i'll add it to the non-dev build04:51
DanielBaird* if04:51
DanielBairdon reflection, perhaps i shouldn't be downloading ievms to my external hdd at the same time as i run a virtual machine from that same drive04:57
prologicoh well :)05:00
prologicyes if you could work on r.js compilation that would be great!05:01
prologicin fact awesome :)05:01
DanielBairdthere's something weird about the require.js docs.  dunno what, i just get annoyed while reading them05:02
prologiclet's make the test/prod deploys build the javascript/css into  a single bundle05:03
prologicand the dev not05:03
prologicawesome right :)05:03
prologicsomeone so hard to get right05:03
prologicthere is something else I need to work on too05:03
DanielBairdyes agree.  getting all your errors as "error in app.js on line 12,543,123" isn't much help05:03
prologicright now I'm using
prologicto daemonize the application backend05:03
prologicI currently have no way to recover from a complete system failure05:04
prologici.e: a reboot of the server05:04
prologicSo my current thinking is that I should write some plays (ansible) for installing and configuring supervisor/supervisord05:04
prologicand setting it up in much the same way as I've done with apache05:04
prologicso the webdevs/webapps users can write application configs to /etc/supervisor.d/05:04
DanielBairdsounds workable.. at present supervisord starts apache?05:05
prologicno apache is different05:05
prologicit starts on it's own05:05
prologicit has service startup scripts, etc05:05
prologicwhat I'm saying is I have nothing or the ccav app05:05
prologicfor simplicity I believe I will leave it run as a separate app05:05
prologicand proxies through apache via mod_proxy05:06
DanielBairdoh i see, you have apache starting via init.d or whatever as normal05:06
prologicanyway supervisor is the thing I'm thinking of setting up05:06
prologicpretty standard apache config05:06
prologicI did not modify the stock CentOS install much05:06
prologicjust enable Named Virtual Hosts05:06
prologicand Include of /etc/httpd/sites.d/05:06
DanielBairdso what's the advantage of supervisord vs. scripts in init.d?05:07
DanielBairdmagical restarting when crashed?05:07
prologicnot much else really from what I can see05:07
prologicI mean I could just write my own init.d startup script for ccav05:07
prologicand continue to use anyway05:07
prologicthe thing is any circuits app is pretty stable anyway05:07
prologicit WILL NOT crash05:07
prologicunless the system kills it05:07
prologicor Python itself segfaults05:08
prologicbut then I have another problem05:08
DanielBairdi've not used supervisord before, i guess whether it's worth using it depends on how much hassle it is to set up.05:08
prologichow do I manage multiple instnaces?05:08
DanielBairdhmm yes instances.05:08
prologicand I don't really want fabric to have to do anything as root05:08
prologicin fact it cannot anyway05:08
prologiconly the ansible playbooks can05:08
prologicso it leads me to supervisor05:08
prologicwhere I configure it carefully via playbboks05:08
prologicin such a way that webapps/webdevs can write configs to /etc/supervisor.d/ for example05:10
prologicmuch like the way I've configured and setup apache with /etc/httpd/sites.d/05:10
prologictbh I don't think it's going to be much "hassle"05:10
prologicpip install supervisor05:10
prologicmkdir -p /etc/supervisor.d05:10
prologicchown -R webapps:webapps /etc/supervisor.d05:10
DanielBairdyeah that's my opinion too, i think supervisord is the usual option becuase it's low cost and reliable05:11
prologicchmod 1770 /etc/supervidor.d05:11
prologicsetup a /etc/init.d/supervidord init script05:11
prologicwhich I can rip off from others05:11
prologicwell I'll do it then05:13
prologicI'll leave the daemon option on ccav though05:13
prologicin case anyone wants to run it stand alone05:13
prologicwithout supervisord05:13
prologicbut I'll work on this now05:13
prologicand test a server reboot05:13
prologickind of want websites/webservices to stay nailed up :)05:14
DanielBairdsupervisord even has a web interface, apparently05:14
DanielBairdah vagrant is only up to installing gdal05:15
DanielBairdievms is up to IE9, so that'll finish soon05:16
prologicyeah it does05:19
prologicI'm not going to enable it however05:19
DanielBairdexample of annoying require.js docs:05:23
DanielBaird"For a list of all options, see"05:23
DanielBaird(i follow the link...)05:23
DanielBairdThere is an file in the requirejs/build directory that details all of the allowed optimizer configuration options."05:23
DanielBaird(.. a section with a heading and ONE LINE of content, that links me to a raw source file in the repo.)05:23
DanielBairdi need to compile the coffeescript into js somewhere, before i feed it to r.js.. i'll just make a temp dir for now that I hg-ignore, lemme know if you have a better idea05:37
DanielBairdthere's a coffeescript compiling plugin for r.js05:44
DanielBairdmeh i'll come back to that, for now i'll do the coffee compilation separately05:46
ircnotifier64190d4a0f9b by prologic: Added roles and plays for setting up supervisord in a sane way that lets webapps define their own application procesess05:56
prologicyeah no05:57
prologicjust do it in a tempdir05:57
prologicuse the python tempfile module to handle this05:58
prologicor use the localpath.mkdtemp() function05:58
prologicyou should see I import that in several places05:58
prologicfrom py.path import local as localpath05:58
prologicyou can use:05:59
prologicpath = localpath.mkdtemp()05:59
prologicfor example05:59
DanielBairdalright cool05:59
prologicbe sure you convert the path to str(path) first05:59
prologicbefore you use it in things that don't understand LocalPath objects05:59
prologicand only accept string paths05:59
prologicso I'd probably do06:00
prologicpath = str(localpath.mkdtemp())06:00
prologicand pass that to your call to coffeescript06:00
prologicabout to test this supervisor thing06:00
prologic*fingers croseed*06:00
DanielBairdawesome, i was hoping there would be a way to get a temp dir like that06:00
ircnotifierc8a4a63f6b2e by prologic: Forgot an action: here06:01
prologicdoes that count? :)06:02
prologici guess not06:02
prologicI screwed up the play06:02
prologicforgot the action:06:02
DanielBairdfiguring out config is okay, it's the dicking about with obscure dependencies or changing libraries coz the os has and older one or whatever that I'd love to avoid just one friggin time for this project06:04
DanielBairdawesome, r.js's output was 66 lines of comments which are the licences from all the js files, and the 67th line is 643,974 characters of javascript.06:13
prologiccan we strip all the comments too?06:18
prologicDanielBaird: figuring out config is okay, it's the dicking about with obscure dependencies or changing libraries coz the os has and older one or whatever that I'd love to avoid just one friggin time for this project <-- say what?06:20
ircnotifierbc0553c9eed6 by prologic: Create /var/log/supervisor dir and fix permissions06:20
ircnotifiera6bb6f11d58e by prologic: Make paths to supervisor binaries /usr/local/bin because we installed with pip which runs on python-2.7 which gets installed to /usr/local/bin06:20
prologicsupervisor is in and working06:20
DanielBairdi mean, it would be nice if todays things, as in r.js and supervisord, just installed and did the right thing without and fiddling about.06:20
prologicjust have to test running up ccav instances through it now06:20
prologicI'll do that in the morning - just about to leave06:20
DanielBairdlooks like r.js is on track too06:20
prologicI might even do it tonight at home06:21
prologicyeah well they pretty much have done the right thing so far06:21
DanielBairdand = any06:21
prologicat least I can say that with certainty with the supervisor stuff06:21
prologicit was all pretty straight forward06:21
prologicI tend to pick "sane" solutions to things :)06:21
DanielBairdi'm still scarred from getting python to work on my machine, and node to work on a centos vm06:22
DanielBairdbut anyway, everything's been working fine today.  so the fiddling-about-demon has probably moved on.06:23
DanielBairdi can ask r.js to not include licence text, but maybe we're supposed to06:24
prologicyeah no don't06:24
prologicnot in production running code06:25
prologicthere is no requirement to do so06:25
prologicexcept that the source code be available06:25
prologicwhich it is :)06:25
prologicI haven't really been told specifically what to do regarding licensing and copyright tec06:25
prologicso I've just been licensing projects here under MIT06:25
prologicand CopyRight
prologicso *meh*06:25
prologicdoes anyone ever look at source code anyway?06:26
prologicexcept other developers? :)06:26
DanielBairdhmm true, every licence in the file is MIT.  so it's open slather06:26
prologicyeah well exactly06:27
prologicI'm sticking with what I do06:27
prologicunless explicitly told otherwise06:27
prologicif you wrote it put your name :)06:27
prologicyour JCU email that is and name06:28
DanielBairdyeah and it's a transient product anyway, the original licenced files are available from our repo.06:28
prologicI never ever modify the original author's name of anything I modify or improve06:29
prologicat the top of all your coffeescript/les files06:29
prologicI'd put something similar to what I've been putting in all my python files06:29
prologicthe Date is usually the date it was oringally created06:29
DanielBairdyeah i should sort out a standard licence header.  does griffith own the code, or jcu, or ands?  actually i guess you can leave it vague and just include author's name and MIT licence06:31
prologicok I'm outta here06:31
prologiccya Monday06:31
DanielBairdnp see ya06:31
prologicfor the moment06:31
prologicYou and I are going to own the codebase06:31
prologicuntil someone wants that changed06:31
prologicI don't like the fact that an Organization says they want to "own" the code06:32
prologicbut then never do anything with it - ever06:32
prologicit seems utterly pointless06:32
prologicespecially when said Organization wants to do OSS06:32
prologicIf it's truly OSS it should be owned by the community06:32
prologicby people that care06:32
prologicbut that's my 2c :)06:32
prologicafaik none of this is meant to be proprietary or commercial in any way shape or form06:33
prologicso I'm doing the "right" thing by OSS standards :)06:33
prologicso essentially I own all the python code06:33
prologicand you own all the coffee/less code06:33
DanielBairdyeah the govt is paying, i.e. taxpayers06:34
prologicwith attributions to Griffith and JCI via our Griffith/JCU Email Addreses06:34
prologicHowever I may also write up in the docs06:34
prologiccreated at Griffith and JCU Universities in collaboation06:34
prologicor something06:34
prologicanyway cya Mon06:34
DanielBairdoh yeah i have to add a logo farm into it somewhere06:34
*** DanielBaird has quit IRC07:15
*** ircnotifier has quit IRC20:08

Generated by 2.11.0 by Marius Gedminas - find it at!