IRC Logs for #cmvt Wednesday, 2013-04-17

*** DanielBaird has joined #cmvt00:15
*** DanielBaird has quit IRC00:20
*** DanielBaird has joined #cmvt00:40
circuitsef88fdc5d0cb by prologic: Implemented utils.read_raster and other helper functions and associated CLI tools and clipmask.py00:50
circuits4219a267dec7 by prologic: Try to import Image and ImageDraw from PIL first then directly01:40
circuits3afc6b332d2f by prologic: Added missing dependencies01:40
DanielBairdi forgot my entire morning was allocated to meetings.  finally at midday i get to do some actual work01:48
prologicaren't meetings fun and productive? :)01:48
DanielBaird*rolls eyes* first one was workplace safety.. zomg the web tools in that area are traumatically bad01:49
prologicnice web standards there :)01:51
prologicawesome :)01:51
DanielBairdactually.. can you get to this site:
DanielBairdback.. that's jcu's chem ahndling site.  it's awful.. resize the width, gah!03:42
DanielBairdthe search panel (which is the gold-ish bit across the top right of the screen) has a help button on the far right that you can only see if you have a very wide browser window03:43
DanielBairdi'm about to start the fetch-the-right-layer part of my thingy but i think i'll commit all the code first03:48
DanielBairdcurrently my working dir looks like this:03:53
DanielBairdin the current project dir there's a ccav/htdocs dir that i guess is the website root03:53
DanielBairdwhat do you recon will be easiest, if i move my src dir to root/websitesrc or something, and change my compile scripts to suit?03:56
DanielBairdhmm actually that's very sensible.03:57
prologicback was tired up with reading some material :)03:57
prologichang on03:57
prologicwhat's in site?03:57
DanielBairdnp.  it's the website root, similar to htdocs i guess03:58
prologiccan it be generated from src?03:58
prologicin it's entirety?03:58
DanielBairdsome stuff can, but there's static html in there that isn't03:58
prologicahh k03:58
prologicperhaps we should use a tempting engine not he backend?03:59
prologicfor static html (which we can shove site-specific configuration into, etc)03:59
prologicin any case I don't mind :)03:59
DanielBairdi guess the main qn is should we have a separate dir for the website stuff *including* the source, or just have the web site src alongside your back end stuff03:59
prologicwe just need a way to build the site I guess03:59
prologicI think putting it into it's own dir is okay04:00
prologichg rm ccav/htdocs04:00
prologicmaybe /site ?04:00
DanielBairdat least in dev mode04:00
prologiceven in production I'm looking at doing so as well04:01
DanielBairdso how about a /webapp dir, inside that is my src, and an htdocs dir04:01
prologicand at a later date moving some "static" parts to apache/nginx/etc04:01
prologic /web ? :)04:02
DanielBairdall my stuff is static04:02
prologicI could restructure the repo a bit if you like04:02
prologic /04:02
prologic /src04:02
prologic /src/ccav04:02
prologic /src/css04:02
prologic /src/htdocs04:02
prologicI have no particular issues with this04:02
DanielBairdwhat's in src/ccav?04:03
DanielBairdthe python for the site?04:03
prologicthe python libraries and web backend04:03
prologicit's called "ccav" for lack of a better name :)04:03
DanielBairdyep that seems okay.. then htdocs will have css, js, etc04:03
prologicthe entire package might get renamed at some point04:04
DanielBairdcould we call it something egeneric like "web"04:04
prologicwant mt to restructure it that way first?04:04
prologicand nuke the current htdocs?04:04
prologicnah cause it gets installed in an environment in some way04:04
DanielBairdyeah i guess so, when i add my stuff i'll add build scripts to the makefile etc04:04
prologicand affects imports04:04
prologice.g: from ccav.utils import read_raster04:05
DanielBairdahh so we need a unique name04:05
prologicok brb04:05
prologicI'll restructure the repo04:05
DanielBairdhmm is it still called terra nova?04:05
prologicdon't commit anything yet04:05
prologicI suggest you re-clone a 2nd copy of the repo04:05
DanielBairdi found a logo image from the terra nova tv show and used that04:05
prologicso you can see what I've done04:05
DanielBairdyep np.. i'll wait for the nod before i start copying stuff in04:05
prologicyes terra nova is the domainname/site for the climate hub repository04:06
prologicthis is different and will be linked from there04:06
prologicbut a separate tool04:06
prologicyou'll see the commit notifications here :)04:06
DanielBairdOoohhh.. so this is (one of potentially several) web front ends for the terra nova system?04:06
DanielBaird(hmm should have spelled it "Ohhhhhhh" so it sounded less orgasmic)04:07
DanielBairdSo maybe something like TNMaps or something.. instead of ccav04:08
DanielBairdare we doing mandatory user docs/ install guides etc?  if those need to be in the repo, we should have:04:09
DanielBaird /04:09
DanielBaird /docs04:09
DanielBaird /mapserver04:09
DanielBaird /web04:09
DanielBairdor something04:09
prologicwe already do have docs04:10
prologicbut not a terrible lot there atm04:10
prologicI recommend:04:10
prologicassume a developer audience04:10
DanielBairdso we do..04:10
prologicand I'd like to utilize fabric for dpeloys04:10
prologiceven development ones04:10
prologicbut I haven't spent a lot of time on that yet04:11
DanielBairdthis is an ANDS project, right.. is there a compulsory blogging element?04:11
prologicnot that I've been told, no04:11
prologicthe only compulsory thing is that it be OSS04:11
prologicand on their f'n sourcforge thingy04:11
prologicwhich I've so far refused to do04:11
DanielBairdthey were happy with github for edgar etc04:12
DanielBairdthey should be okay with bitbucket too04:12
prologicand they're going to have to be happy with bitbucket too04:12
prologicsorry ANDS :)04:12
DanielBairdhow easy is it to write a fabric script to install all the deps for the project from scratch (on a minimal centos install i mean)?04:13
prologicright now?04:14
prologiceven without fabric04:14
prologicit's basically a 2-step process04:14
DanielBairdwhenever, it was a long process get this macbook setup with everything04:14
prologicwell 3 or 404:14
prologicdepending on if Python ir on the system or not04:14
prologicand mercurial04:14
prologicI've ensured I don't break dependency stuff now04:15
prologicI will get you to test setting this up from scratch too I think04:16
DanielBairdi've just noticed the fabfile dir, should i use fab for my makefile-type stuff?  we also have a makefile in there...04:16
prologicI might test it myself as well :)04:16
prologicyeah it would be good04:16
prologicyou don't mind learning a bit of Python no?04:16
prologicI quite like fabric and what it offers04:16
prologicI will probably nuke the Makefile at some point04:16
prologicwhen I've written fabric tasks for what the targets of the Makefile do04:17
prologicnamely clean, test and docs04:17
prologicactually IIRC I already have I think :)04:17
DanielBairdi'm okay with Python.  i dislike make, this norming i wrote a cake file for compiling all my css nad stuff04:17
DanielBairdanyway it's easy to write a vagrantfile that starts a blank centos system, installs python, and runs fab install-all or whatever04:19
DanielBairdbut first: fresh new repo layout for strong make greater nation future!04:20
prologicsorry restructure taking longer than expected04:23
prologicDanielBaird: anyway it's easy to write a vagrantfile that starts a blank centos system, installs python, and runs fab install-all or whatever <-- yes this would be good04:24
prologicand ideal04:24
prologictreat vagrant like you would puppet, chef, ansible04:24
DanielBairdyeah and vagrant is easy to ignore too04:24
prologicspool up vm, install bootstrap requirements04:25
prologicrun deployment script04:25
circuits2f1c297cdd19 by prologic: Removed old htdocs from earlier prototyping of web frontend04:41
circuits4194b98cfa23 by prologic: Major restructure to accomondate future development04:41
circuits8516713baf80 by prologic: Re-implemented part of the old Makefile clean target04:41
prologicdoing a few more test runs of deploy04:45
prologicwell really installing deps04:45
DanielBairdthis is not a huge deal, we can install a cold water tap under the sink, but it's annoying04:54
DanielBairdand if we get a plumber to do it, it makes it more expensive to end up with a new dishwaster04:55
DanielBairdlol wrong window, my wife is trying to buy a dishwasher04:58
DanielBairdcloning the new thing now..05:04
DanielBairdokay so i have the new repo, with a src dir that includes a ccav dir. i'll put my coffeescript and less dirs directly in the src dir.  where should i put my static html?  in something like src/static, and we have a "fab buildall" thing that copies or compiles everything into a /htdocs dir?05:41
DanielBairddoes the webroot dir have to be inside /src/ccav/ to work>05:42
prologicbumpy has some issues06:14
prologichave to install it by hand06:14
prologicgah :/06:14
prologicsorry dude06:15
prologicyeah put all your "sources" in /src06:15
prologicthen I think we should build a fabric task06:15
prologicthat lets us do a quick 'n dirty deploy of the web app (fronted)06:16
prologicproviding an optional argument for location06:16
prologicotherwise defaulting to e.g: current_directory + htdocs06:16
prologicor something06:16
DanielBairdoops,  back06:17
DanielBairdbut we can't make a totally static site, right.. we still need to serve up the api responses.  so would the frontend deploy copy python files to the deploy spot?06:18
DanielBairdbut anyway i'll dump my static stuff into a 'static' dir06:19
DanielBairdthat our deploy thingy can just copy directly06:19
prologicit'll be fine06:22
prologicassume a Less compile06:23
prologicand more or less a copy of any other static files06:23
prologicinto a target directory06:23
prologicwith a default location06:23
prologicI will write options for the ccav binary/script06:23
prologicthat points it to a "theme" as such06:23
prologic$ ccav -t /path/to/theme -d /path/to/data06:23
DanielBairdah okay so the dynamic stuff will run from wherever06:24
DanielBairdthat seems cool06:24
prologicand on the deployment side of things06:24
prologicApache/Nginx can be configured to do reverse proxy to our app06:24
prologicand optionally service up the status resources directly as well06:24
prologicmake ense?06:24
prologicit means configuration is minimal06:24
prologicand deployment is minimal06:25
prologicI'll also be adding optional configuration file support06:25
prologic$ ccav --config=/path/to/config06:25
prologicone sec06:27
prologicthe tests task accepts a list of optional arguments06:27
prologicand they're passed to the function from "fab" like this:06:28
prologicfab deploy:path=/path/to/output/theme06:29
prologicalthough this perhaps should be considered a separate step06:30
prologicpart of a "build" task?06:30
prologicwhere build_theme depends on build_css and build_html06:30
prologicyou can create a new module for this:06:31
DanielBairdthe build results have to go somewhere, so you'll need the path for any of them.  but yeah i imagine making a build_coffee and build_css and i guess an build_html06:31
prologicwith tasks build_css, build_html and build_theme06:31
DanielBairdtheme is both?06:31
DanielBairdcss and html06:31
prologicand build_theme would internally just call both functions directly06:31
prologicwith the arguments passed06:31
prologici.e: a wrapper task06:31
prologicshould be pretty easy to do I think06:32
prologicfabric provides shell wrappers, etc to execute arbitrary commands and check the results of their execution and optionally their output, etc06:32
prologicso it should all be very nice and Pythonic with minimal code06:32
prologicI like Fabric :)06:33
DanielBairdtask 'coffee', 'concatenate and compile coffeescript files from the src/coffee/ dir', (options) ->06:33
DanielBaird    exec 'coffee --compile --bare --output site/js/ --join all_0_1 src/coffee/'06:33
DanielBairdtask 'less', 'compile app.less from the src/less/ dir', (options) ->06:33
DanielBaird    exec 'lessc src/less/app.less site/css/app.css'06:33
DanielBairdtask 'compile', 'compile coffeescript and less', (options) ->06:33
DanielBaird    invoke 'coffee'06:33
DanielBaird    invoke 'less'06:33
prologicit's very explicit and has a nice API06:33
DanielBaird(took a minute to paste that, perhaps the conversation moved on, but i'm showing a task made of two other tasks)06:33
prologicthese days I kinda like things like Fabric06:34
prologicas opposed to cake, Makefile, puppet, etc06:34
prologicI don't have a great desire for "yet another DSL" :)06:34
prologicI wonder if there's a decent CSS compiler that is basically a Python library :)06:35
prologici.e: you write your CSS in Python06:35
DanielBairdmakefiles are definitely bad.. these days the cool ones (rake, jake, cake..) all use their own languages for the makefile06:35
prologicthen run a tool to generate .css :)06:35
DanielBairdyou're thinking of sass06:35
prologicahhh :)06:35
DanielBairdnot exactly python06:35
prologicyeah see I don't like rake, jake, cake, etc06:35
DanielBairdbut indented, clean06:35
prologicbecause they basically just redefine a new DSL06:35
prologicit's just like a frigg'n Makefile06:35
DanielBairdyou like the build tool that uses your language06:35
prologicof course!06:36
prologicit means I have to think less :)06:36
prologicand remember less06:36
prologicI can speak Python almost fluently :)06:36
prologicthey all have their uses I guess :)06:37
prologicbut usually I prefer to use something that is a library/tool06:37
prologicthat does not define a new DSL06:37
prologicand is in Python with a Pythonic API06:37
prologicI don't ask for much do i :)06:37
DanielBairdi try and use the tool that common to the language i'm using06:37
prologichehe :)06:38
prologiccommon != good (in many cases)06:38
prologicbut yeah I know what yo umean06:38
DanielBairdso how do i exec something from python / a fab task?06:38
DanielBairdexec("command here")?06:38
prologicAt it's simplest:06:39
prologicfrom fabric.api import local06:39
prologiclocal("some command")06:39
prologicit only gets slight y more complex when you want to capture exit status's and capture stdout and/or stderr06:40
DanielBairdhmm suggests a "remote('command')" might exist06:40
prologicbut otherwise it's just Python06:40
prologicyou could write any normal Python in the task06:40
prologica remote function does exist indeed06:40
prologicwhich means we can also write the fable package06:41
prologicto also perform remote dpeloyments06:41
DanielBairdlocal is a fabric API call!  you don't get to use that and simultaneously complain about DSLs :)06:41
prologicwhich I intend to use06:41
prologicyes I do :)06:41
prologicit's an API06:41
prologicnot a DSL :)06:41
prologica DSL implies a new Domain Specific Lnguage06:41
prologicwhich involves (the author of said DSL and tool)06:41
prologicto write a new lexer and parser06:41
prologicand an interpreter for said DSL06:41
DanielBairdyes a build tool that stretches to remote deployment as well as handling basic compilation is cool06:41
prologicit's overkill :)06:41
prologicwhen good API(s) are often better06:42
prologicand simpler to use and understand06:42
DanielBairdahh well i n that case rake and cake are both fine, neither of them do DSLs.  they both just have a couple of things defined locally in the real language.  just like local()06:42
prologicread the parts on Failure Handling too:06:42
DanielBairdi thknk jake is the same, not sure06:42
prologicahh ok :)06:43
prologicI didn't know that :)06:43
DanielBairdi might print out that tutorial page and take it home06:43
prologicso with rake you are writing actual ruby?06:43
DanielBairdacutally i've never written a rakefile, but yeah it's ruby06:44
DanielBairdit looks like a DSL coz Ruby is good at being readable.06:44
DanielBairdwhich mainly boils down to optional brackets06:44
DanielBairdbut, you already have python, don't spend too much time on Ruby when you should be learning node and coffeescript.06:45
DanielBairdoptional brackets (ruby and coffescript share that feature) means this ugly code:06:47
DanielBairdtask('mytask', 'this is the thingy task') {06:47
DanielBairdtask 'mytask', 'this is the thingy task'06:47
DanielBaird(and coffeescript doesn't need the "end" coz it's indent powered like python)06:47
circuits71ace72d9b0b by prologic: Completed clean taskk using fabric.api.local06:49
DanielBairdare we using the latest fabric, 1.6 or whetaver?06:49
DanielBairdyou linked me to the 1.4 docs06:49
prologicI tend to use the latest of everything06:50
prologicassume "yes" to this question always06:50
prologicunless otherwise said06:50
DanielBairdalright cool06:50
prologicI don't believe in legacy crap (to a point)06:50
prologiccircuits still supports Python 2.6 for example06:50
prologicbut I want to drop that soon-ish06:51
prologicoh btw06:51
prologicI played around with trying to improve the rendering of the circuits component graph of apps06:51
prologicI tried to use networkx06:51
prologicwhich seemed to draw nicer looking raphs right out of the box06:51
prologicI could not get it to look nice on larger more complex aps06:51
prologicI don't know enough about rendering pretty graphs06:52
prologicI have to go shortly, but I can chat from my iPad06:52
DanielBairdnp heading soon myself.  networkx have some pretty examples06:53
DanielBairdoh i see, networkx uses graphviz or matlab or whatever to draw the diagrams06:54
prologicmaybe you kef for home already ;)07:18
prologicahh you have ;)07:18
*** DanielBaird has joined #cmvt08:46
*** DanielBaird has quit IRC08:55
*** DanielBaird has joined #cmvt11:10
*** DanielBaird has quit IRC11:14
*** DanielBaird has joined #cmvt12:10
*** DanielBaird has quit IRC12:15
*** DanielBaird has joined #cmvt13:10
*** DanielBaird has quit IRC13:15
*** DanielBaird has joined #cmvt13:58
*** DanielBaird has quit IRC14:24
*** DanielBaird has joined #cmvt14:55
*** DanielBaird has quit IRC15:04
*** DanielBaird has joined #cmvt15:56
*** DanielBaird has quit IRC16:01
*** DanielBaird has joined #cmvt16:26
*** DanielBaird has quit IRC16:31
*** botton has joined #cmvt16:41
*** DanielBaird has joined #cmvt16:57
*** DanielBaird has quit IRC17:02
*** DanielBaird has joined #cmvt17:27
*** DanielBaird has quit IRC17:32
*** DanielBaird has joined #cmvt17:58
*** DanielBaird has quit IRC18:03
*** DanielBaird has joined #cmvt18:28
*** DanielBaird has quit IRC18:33
*** DanielBaird has joined #cmvt18:59
*** DanielBaird has quit IRC19:04
*** DanielBaird has joined #cmvt19:29
*** DanielBaird has quit IRC19:33
*** DanielBaird has joined #cmvt20:00
*** DanielBaird has quit IRC20:04
*** DanielBaird has joined #cmvt20:30
*** DanielBaird has quit IRC20:35
*** DanielBaird has joined #cmvt21:01
*** DanielBaird has quit IRC21:05
*** DanielBaird has joined #cmvt21:31
*** DanielBaird has quit IRC21:35
*** DanielBaird has joined #cmvt22:01
*** DanielBaird has quit IRC22:06
*** DanielBaird has joined #cmvt22:32
*** DanielBaird has quit IRC22:36
*** DanielBaird has joined #cmvt22:48
DanielBairdhmm, looks like i get disconnected once every hour.  wonder what that's sabout22:58
DanielBairdand if we ever care enough to build our own vagrant centos base system:
prologicjust a standard CentOS base is fine23:42
prologicmorning btw :)23:42

Generated by 2.11.0 by Marius Gedminas - find it at!