IRC Logs for #cmvt Wednesday, 2013-06-26

*** DanielBaird has joined #cmvt01:43
DanielBairdi'm currently wrapping my init stuff up in callbacks.. I should have made my get-data-by-randomly-generating-it asynchronous like an AJAX fetch.  I'll be ready to actually try an api call or two today i hope01:56
prologicI'm not :)01:59
prologicI have however completed the first stab at generating the summary data02:00
prologicI just have to get it to your client now02:00
DanielBairdcool.  will you be able to make the api just serve static json?02:00
DanielBairdstatic json files, i mean02:01
prologicSure I could do that02:01
prologicLet me throw some things by you though02:01
prologicI was thinking of utilizing JSON Schema02:01
prologicread about it?02:02
prologicAnd JSON Hyper Schema (link relations)02:02
prologicfor documentation of the API02:02
prologicand discoverability02:02
prologicAs well as trying to maintain a RESTful design02:02
prologicBut naively I could just serve up static JSOn right now02:02
DanielBairdis it just for describing the returned json of an api call?02:02
prologicbut that would be neither RESTful nor discoverable02:02
prologicbut also for describing links you can follow02:03
prologicala JSON Hyper Schema02:03
prologicsince we want to use JSON for all this02:03
prologicthere are standards being drafted for this02:03
prologicI just finished watching this presentation on it:02:04
prologic(not JSON Schema specific, the presentation is a bit old for that)02:04
prologicAnd then there's Fielding's original research paper on REST02:04
prologicIt's worth noting that many popular so-called RESTful API(s) on the web are in fact not RESTful at all02:05
prologicReading also this:
DanielBairdso if we had a schema description of the api, is there some tool that makes it easy to serve correct json, or something?  or is it just for documenting the api.02:06
DanielBairdi like the idea of a uri template02:16
DanielBairdthere's a bit of tension about the right way to have links to other API objects.. in theory the id of the other thing should be enough, but that requires the reader to have enough pre-existing knowledge to be able to turn the id into a correct url.  alternatively you could list the full URL to the other object, so the user can just hit it directly, but then the url includes irrelevant stuff like the hostname and protocol.02:18
DanielBairdan id and a template for turning ids into urls seems cool02:18
DanielBairdcoffee time for me.. back in a little while02:59
prologicsorry went and had lunch03:11
prologicback now :)03:11
prologicre what you were saying about uris and links etc03:27
prologicthe idea (I think) is to use the JSON Hyper Schema staandard03:27
prologicas well as the JSON Schema standard03:27
prologicso that you are doing 3 things:03:28
prologica) describing the resource you are serving03:28
prologicb) also allowing discoverability of where to go next, etc03:28
prologicc) providing basic documentation on the API(s)03:28
prologicthere are two known tools that allow you to traverse and read documentation for APIs that use JSON Schema and JSON Hyper Schema03:29
prologicIO Docs:
prologicWe can in theory just simply ship these along side our API (I think)03:29
prologicSo they provide the traversal and documentation tooling for JSON Schema / JSON Hyper Schema based API(s)03:30
prologicFrom everything I've read and understood so far03:30
prologicI think the general idea is to a) provide schemas for the resources you are serving03:30
prologicwith link relations (JSON Hyper Schema)03:30
prologicand obviously to use HTTP as the application protocol03:31
prologicso using JSON Hyper Schema essentially give us the missing bit of RESTful -- the Hyper Media stuff (HATEOS)03:31
DanielBairdiodocs looks cool03:39
prologicNB: I haven't done any of this yet03:39
prologicBut what do you think?03:39
prologicAlso note the commit above03:39
prologicYou can try out some of the generated summary data if you wish03:39
prologicNot in the nicest way, but hty it's there03:39
prologicSo to be truly RESTful you not only have to use HTTP as your application protocol (using appropriate methods and response status codes, etc) but you also have to maintain application state via the use of hyper media (this is where JSON Schema and JSON Hyper Schema come into play for JSON I think).03:45
DanielBairdi'm not sure how much time you should put into making the api better than the minimum necessary for my web ui.  if you're supposed to be building a data api for everything in terra nova or something, then it's worth spending the time including json schema in the api docs, supplying tools for browsing it, etc etc.  otherwise having a clean restful api that covers just what it needs to is sufficient. i'm very awayre that soon i'll get sucked awa03:46
DanielBairdhow much longer are you supposed to be working on this hub project?03:46
prologicI don't know03:49
prologicBut yes you are probably right03:49
prologicCan you check out the /test/api/ and fab data stuff03:50
DanielBairdit's still totally the right thing to do to make stuff restful and clean03:50
prologicAnd see if I've met your data requirements so far?03:50
prologic*nods* of course03:50
prologicand I'm aware I may be spending too much time on this and over engineering it03:50
prologicbut we'll see if time permits it or not03:50
prologicotherwise you'll get what you get :)03:50
prologicIf you update all your stuff now though03:50
prologicyou should be able to do:03:51
DanielBairdthe pattern at the startup i was at before eresearch was 1. think about it, 2. work out the full and complete solution, 3. implement only enough of the full and complete solution to get to the next bit.03:51
prologicfab data03:51
prologicactually I think this is included in fab deploy as well now03:51
DanielBairdthen later, 4. come back and take another step toward the fuill and complete solution as necessary03:51
prologicbut I also wrote a new script in ./scripts/ called gensummaries03:51
prologicbut it's not run automatically with fab data/deploy yet03:51
DanielBairdI ran up a VM on thursday, i'll re-run ansible on that03:51
prologicand I just committed a static serving api /test/api/ that serves up the data directory03:52
prologicif ansible ran cleanly - no need03:52
prologicjust fab deploy03:52
prologicthen login and run the gensummaries script03:52
prologic$ gensummaries /path/to/data03:52
prologicI think the default is /webapps/var/<instnace>/data03:52
prologicdev if you do fab deploy.dev03:53
prologicand yeah don't worry :)03:53
prologicI follow the same pattern03:53
prologicI've just been reading up on this stuff and thinking about it as well03:53
prologicand wanted to share03:53
prologicright now I've implement enough I think to get stuff going03:54
prologicI hope :)03:54
prologicneed you to test :)03:54
DanielBairdit seems possible that iodocs makes api doccing quicker.. that's worth pursuing :)03:54
prologicwell yes03:59
prologicI will try to pursuer this (time permitting)03:59
prologicbut let's play with what we have so far03:59
prologicyou all good with the current state of things?03:59
prologicI mean do you have enough to go on and test?03:59
prologicgensummaries and /testapi/ endpoint04:00
DanielBairdoh that's right, fab deploy doesn't work in the vm04:01
prologicno :)04:02
prologicit's designed to work remotely04:02
prologici.e: from your own workstation04:02
prologicor whatever04:02
DanielBairdC02JKVMWDR53:ui-ccav pvrdwb$ fab -u vagrant04:02
DanielBairdDeploying dev instance to ... [] Login password for 'vagrant':04:02
DanielBairdFatal error: put() encountered an exception while uploading '<StringIO.StringIO instance at 0x103d7da70>'04:02
DanielBairdUnderlying exception:04:02
DanielBaird    Permission denied04:02
DanielBairdDisconnecting from done.04:02
prologicif your ansible playbooks ran successfully04:02
prologicwith the updated key I put in04:02
prologicyou should be able to just do:04:03
prologicfab deploy.dev04:03
prologicand nothing more04:03
DanielBairdahh i probably need to re-ansible04:03
prologicdon't -u vagrant though04:03
prologicjust fab deploy.dev04:03
DanielBairdthis was last playbooked on thursday before i sent you my key, i think04:03
prologicit should ssh in with your local user/key04:03
prologicwhich ansible playbooks should have put on the server04:03
prologicand it'll sudo up to web apps user04:03
prologicre-play the playbooks then04:04
prologicI did add you key and account04:04
prologicI assume your local account name was daniel04:04
prologicas specified by your key04:04
prologicif this all works fairly well04:04
prologicI'll put the gensummaries into the deeply task as well04:05
DanielBairdokay updated playbooks, ansible is applying them to my vm04:06
DanielBairdmy local user isn't daniel it's pvrdwb (uni issued login from the 90s)04:07
DanielBairdcan i use -u daniel to fix it04:07
DanielBairdlooks like re-playbooking doesn't work, it skipped a step and then errored with a:04:08
DanielBairdfatal: [] => Conditional expression must evaluate to True or False: {% if last_result.changed %} True {% else %} False {% endif %}04:08
DanielBairdi guess last_result isn't set if the previous step is skipped04:09
DanielBairdTASK: [Install virtualenvwrapper] *********************************************04:09
DanielBairdskipping: []04:09
DanielBairdTASK: [Initialize virtualenvwrapper] ******************************************04:09
DanielBairdfatal: [] => Conditional expression must evaluate to True or False: {% if last_result.changed %} True {% else %} False {% endif %}04:09
prologicI've never had that issue since I fixed it04:11
prologicyou just tried to run an updated set of playbooks against an existing vm?04:12
prologicand your local user account is pvrdwb ?04:12
prologicFixed your account name.04:13
DanielBairdsorry back.. yes i'd just hg-fetched the playbook repo just before04:14
prologicPLAY RECAP ********************************************************************04:14
prologic10.0.0.2                   : ok=64   changed=19   unreachable=0    failed=004:14
prologicmine did not error out04:15
DanielBairddid it skip some steps?04:15
DanielBairdoh hang on.. i'm not in the ccav virtualenv, it might be an old ansible or something04:15
prologicTASK: [Install virtualenvwrapper] *********************************************04:16
prologicskipping: []04:16
prologicTASK: [Initialize virtualenvwrapper] ******************************************04:16
prologicskipping: []04:16
prologicTASK: [Update postmkvirtualenv for "webapps"] *********************************04:16
prologicchanged: []04:16
prologicwell that's my first though04:16
prologicwhat version of ansible are you using?04:16
prologicIs it the latest or something?04:17
prologicor something vagrant pulled down/04:17
prologicI git cloned the ansible repo (locally on my mac) at some working known revision04:17
prologicand have been using that ever since04:17
prologicI haven't updated it at all04:17
prologiclike 1.2 installed how?04:17
prologicpip install ansible ?04:17
DanielBairdprobably.. i only have it because you told me to install it, i guess it was pip04:18
prologiclet me see if I pip install ansible04:19
prologicI get the same error04:19
prologicif I do, I'll use that version04:19
prologicand try to fix the problem04:19
DanielBairdthere's a couple of people getting that error on google but hard to tell if it's the same thing04:21
DanielBairdi'll wait and see if your ansible 1.2 does04:21
DanielBaird* dies04:21
prologicok running ansible 1.204:24
DanielBairdi'll run ansible in verbose mode04:24
prologicmine did not die04:25
prologicI'm out of ideas04:25
prologichard to fix something I can't reproduce :)04:25
prologicany ideas?04:25
DanielBairdmy command line is:04:25
DanielBairdansible-playbook -u vagrant -ksv site.yml04:25
DanielBairdare you using sudo to run your playbook?04:25
prologicthat's a good point04:26
prologicI do use root04:27
prologicas I'm not using vagrant04:27
prologichow is ansible being run under vagrant again?04:27
prologiclocally or on the vm itself?04:27
prologicdoes vagrant pull down and install a local copy of ansible on the vm?04:27
DanielBairdsay that being applied, so i do have that playbook version04:27
DanielBairdnvm answering something from earlier in the stream)04:28
DanielBairdi'm just using vagrant to run a VM at it has a user 'vagrant'.  then i'm running ansible from my host machine with that comnand line.  so it should be the same as your VM setup, pretty much.04:29
DanielBairdare you using -u root ?04:29
prologicI'm going to try and remove that register last_changed and when stuff04:30
prologicI'm not sure that I need that condition04:30
DanielBairdi'll try -u root and see if that gets through04:30
DanielBairdnope didnt' get there04:30
DanielBairdpretty freaking weird04:31
prologicre-creating my vm04:33
DanielBairdi suppose it's possible a previous install on my vm failed at a weird spot, but it looks like an ansible error rather than some permissions issue or anything.  in -vvv verbose mode it still just goes from the TASK: header line to the fatal error line, there's no additional logging between them04:33
prologiccould be04:33
prologicanyway I'll see if this works04:33
prologicand push the changes up04:33
prologicgimme a min04:33
DanielBairdfixed two months ago, wonder if it's in my ansible 1.3204:35
prologicI'm using the git version04:36
prologicso no wonder I didn't see this04:36
prologicI did a fresh pip install ansible04:36
prologicand did not see the error occur04:36
prologicperhaps try04:37
prologicpip uninstall ansible04:37
prologicpip install ansible04:37
prologicpip install -U ansible04:37
prologicmaybe this is fixed already04:37
DanielBairdupdated ansible, same problem, frickit.  should i try a git install?  or do you want to fiddle the playbook to work around it.04:42
DanielBairdfrom the google group in Feb:04:45
DanielBaird$last_result is no longer automatically registered in Ansible04:45
DanielBairdso it looks like you just need to do a04:45
DanielBairdregister: $last_result04:45
DanielBairdon the task04:45
prologicI'll try this work around04:49
prologicby not using last_result04:49
prologicsorry :)04:49
DanielBairdi'm occasionally fetching the playbooks.. lemme know when i can try again05:02
prologicone moment05:08
prologicsorry got side tracked by a request from AB05:09
prologichopefully this works for you05:16
DanielBairdrunning now..05:28
DanielBairdyep it got past that bit.  looks good so far05:34
DanielBairdfinished successfully.  what was i doing ,again?05:41
DanielBairdoh yeah fab deploy etc05:41
DanielBairdfab deploy wants an instance name, what's that?05:44
DanielBairdoh dev05:51
prologicdo fab deploy.dev05:53
prologicsee you tomorrow05:53
prologicI'm outta here :)05:53
DanielBairdsee you.05:56
DanielBairdfab deploy is running05:56
DanielBairdwill update you tomoz05:56
*** DanielBaird has quit IRC08:01
*** DanielBaird has joined #cmvt08:32
*** DanielBaird has quit IRC08:41
*** DanielBaird has joined #cmvt09:37
*** DanielBaird has quit IRC09:41
*** DanielBaird has joined #cmvt10:37
*** DanielBaird has quit IRC10:41
*** DanielBaird has joined #cmvt11:38
*** DanielBaird has quit IRC11:43
*** DanielBaird has joined #cmvt12:38
*** DanielBaird has quit IRC12:43
*** DanielBaird has joined #cmvt13:39
*** DanielBaird has quit IRC13:43
*** DanielBaird has joined #cmvt13:58
*** DanielBaird has quit IRC14:24
*** DanielBaird has joined #cmvt14:54
*** DanielBaird has quit IRC15:03
*** DanielBaird has joined #cmvt15:29
*** DanielBaird has quit IRC15:34
*** DanielBaird has joined #cmvt16:31
*** DanielBaird has quit IRC16:35
*** DanielBaird has joined #cmvt17:31
*** DanielBaird has quit IRC17:36
*** DanielBaird has joined #cmvt18:32
*** DanielBaird has quit IRC18:36
*** DanielBaird has joined #cmvt19:32
*** DanielBaird has quit IRC19:36
*** DanielBaird has joined #cmvt20:32
*** DanielBaird has quit IRC20:37
*** DanielBaird has joined #cmvt21:33
*** DanielBaird has quit IRC21:37
*** DanielBaird has joined #cmvt22:33
*** DanielBaird has quit IRC22:37
*** DanielBaird has joined #cmvt22:41
DanielBairdi went looking for your new script, should it be in /scripts ?22:54
prologicIt is in ./scripts/22:55
prologicAnd should be in your bin directory22:56
prologicif you re-installed ccav22:56
prologicOr at least it should be on the server already22:56
prologicsudo -i -u webapps22:56
prologicgensummaries /webapps/var/data/dev22:56
DanielBairdwhere's the source live on my vm?22:58
prologicin /webapps/work/dev22:59
prologicbut there should be a /webapps/bin/gensummaries22:59
DanielBairdvagrant gives me access to the host dir but that's not gonna help22:59
prologiclet me know if there isn't22:59
prologicas I said yesterday I haven't integrated gensummaries into the deployment yet23:00
DanielBairdah it's all root owned.23:00
prologicit should all be owned by the web apps user23:01
prologicI don't believe you :)23:13
prologicEverything in /webapps is owned by webapps23:13
prologicI'm puzzled as to your statement23:13
DanielBairdno you're right it's webapps (sry was talking locally)23:13
prologicAhh and of course:23:13
prologic[webapps@ccav ~]$ ls ~/bin/23:13
prologicactivate     easy_install      pip      python   python2.7   virtualenv-2.7    virtualenvwrapper_lazy.sh23:13
prologicactivate.csh  easy_install-2.7  pip-2.7  python2  virtualenv  virtualenv-clone  virtualenvwrapper.sh23:14
prologic[webapps@ccav ~]$ workon dev23:14
prologic(dev)[webapps@ccav ~]$ which gensummaries23:14
prologic(dev)[webapps@ccav ~]$23:14
prologicSorry my bad23:14
prologiceverything is installed in a virtual environment23:14
prologicYou would have to23:14
prologicsudo -i -u webapps23:14
prologicworkon dev23:14
DanielBairdah workon dev alright23:14
prologicThen you can run gensummaries23:14
prologicforgot about environments :)23:14
prologicThat's how I'm able to run multiple instnaces23:14
prologicwithout conflicts23:14
prologicand different versions even23:14
prologic(dev)[webapps@ccav ~]$ gensummaries ~/var/data/dev/23:15
prologicthis should work23:15
prologicI'm testing myself on my vm now23:15
prologicShould take a few mins to generate23:15
DanielBairdokay so i've run gensummaries, it returns immediately23:15
DanielBairdoops forgot the arg23:16
prologicarg is the path to the data dir23:16
prologicfor the instnace23:16
prologicso ~/var/data/dev23:16
DanielBairdso your server setup supports multiple deployments of ccav by using a virtualenv for each one23:17
DanielBairdit returned.  so there's data somewhere now.. do i need to run ccav or is it all init-ed23:23
prologicit should pick it up automatically I'm pretty sure23:58
prologictry the /testapi uri23:58
prologicI have to cleanup some data for AB23:58
prologicbut then I'll be back on it23:58

Generated by 2.11.0 by Marius Gedminas - find it at!