IRC Logs for #cmvt Tuesday, 2013-05-21

prologichmm00:04
prologicjust reading your webapi docs00:05
prologicI'm not sure about the .<format>00:05
prologicas part of the uri00:05
prologicI've seen uses of00:05
prologic&f=json (optional) before00:05
prologicdefaulting to "json"00:05
prologicmaking it part of the URI00:05
prologicmeans I have to parse the URI00:05
prologicjust to get the requested format00:05
prologicunless of course all this data was precmputed00:06
prologicand just regular files on disk00:06
DanielBairdyeah that's okay.  the idea of having the format appear like a file extension helps for human perception and expectation, and also get the filename right when you wget the url.  but it's not a big deal, feel free to change the doc, i think just altering the prototype URL to say "[&f=<format>]" instead of ".<format>" is all that's necessary.00:14
DanielBairdjavascript doesn't have ruby's spaceship operator.00:28
DanielBairdpython had cmp() up until python3.. how do you do three-way comparison in python 3?00:30
DanielBairdbbl meeting elsewhere.00:30
prologic3-way comparisions?00:38
DanielBairdor the other way around, whichever.  you use it when you're doing sorting callbacks.01:11
prologicyes no01:27
prologicPython 3 got rid of this01:27
prologicin favor of __lt__, __eq__ and __gt__01:27
prologicrather than mixing comparison implementations into a single method01:27
prologicit's enough to just implement __lt__ and __eq__01:28
DanielBairdterrible idea.. having an operator that does it is super handy01:28
DanielBairda function that does it is almost as good01:28
prologicno01:29
prologicnot a terrible idea at all01:29
prologicthey separate it out for simplicity01:29
prologicand separation of conerns01:29
prologicmany new languages do not implement a comparison operator/function01:30
prologicmost modern implement ions of languages now adopt implementing them separately01:31
prologicI'm upgrading the public test/prod server02:45
prologicmy jmills nectar vm02:46
DanielBairdjust committed sortable table columns.03:05
DanielBairdhave you tried out the r.js compiled thing?  if it's working i can delete the separate repo-clone i made to work on it.03:07
DanielBairdoh i also realised another field we need in the region data: state.03:08
prologicI've made a few modifications to the web api docs03:10
prologicI'm going to bring down the test/prod instances now03:11
prologicand re-deploy03:11
DanielBairdusing "current" means the model and emission scenario aren't required03:12
DanielBaird(sorry i mean, in teh API docs..)03:12
prologicagh yes of course03:13
prologicyou are quite right03:13
prologichang on though03:13
prologicgoing to redeploy to the nectar vm03:13
DanielBairdgo ahead03:14
prologic*fingers crossed* :)03:16
prologicI'd almost want a separate endpoint btw03:16
prologicfor current03:16
prologicvs. future models03:16
DanielBairdyeah it's annoying.03:16
DanielBairdthe data will look the same, but it's quite different even qualitatively03:17
prologicexactly03:17
prologicbasically you're accessing statistics of a raster layer03:17
prologicwith pixel values representing some kind of data point03:17
prologicthen you ant to intersect that with region boundaries03:17
prologicyou essentially want to do the same thing with future models as wlel03:17
DanielBairdthe future projections are also that though.03:18
prologicbut I think they should be different end points03:18
DanielBairdyeah one is "show me how it is"03:18
prologicthe future models are indxes03:18
prologicwhereas the current is just straight single layer03:18
prologici.e: no time dimension03:18
DanielBairdthe other is "show me what certain researchers projections say it might be"03:18
prologicplus there is also the delta layers don't forget03:18
prologicof "what it looks like compared to current"03:19
prologicfor every model + scenario + year03:19
prologicthere is a delta for it compared with current03:19
prologicfor each bioclim03:19
DanielBairdi've been slightly troubled by saying current, really it's some year, and it would be cool to have earlier / historical data too03:19
prologicyes it would be03:19
prologiccan we generalize it?03:19
prologicif we can generalize it and get access to appropriate data03:20
prologicthen we could just have the one endpoint03:20
prologicwhere we can go back and forth in time03:20
prologicthe default of course being 201303:20
DanielBairdi don't think we can call it a year without the data source researchers agreeing03:20
prologicwhat od you mean?03:20
DanielBairdwell, we don't have data for 201303:21
DanielBairdso we can't call it that.03:21
prologicright03:21
prologicwe only have current data for 2009?03:21
DanielBairdi think the projections for future years, e.g. 2020, are really for something like "the 30 year period around the specified year"03:22
DanielBairddunno what exactly "current" means03:22
prologichrmm03:22
prologicthis could be a problem03:22
prologiccurrent to me so far has meant 200903:22
prologicbecause that's the only "current" data we have03:22
DanielBairdi bet the source data is from various projects, all with different data collection times03:22
prologicof what things look like "now" roughly speaking03:22
DanielBairdbut, "current" is accepted (afaict) in the industry as jargon for "not a future projection"03:23
DanielBairdso we're okay to jsue "curent", doing anuything different would require a researcher to agree that it's sensible03:24
DanielBairdjsue = use03:24
prologicjust my luck :)03:24
DanielBairdthe other api strategy is to have an endpoint for "current", and an endpoint for "projected"03:24
DanielBairdand pass in scenario, year, etc as args to the "projected"03:24
DanielBairdor even more restful, /climateprojections/ gets a list of all the available projections and their IDs03:26
DanielBairdthen /vlimareprojections/<<projectionid>> to get a specific one03:26
DanielBairdzomg my typing is terrible today03:26
DanielBairdthen, i could fetch the climateprojections/ list, use the dropdowns etc to let the user choose one of the specific projections they're interested in, and just hit that URL to get the data.03:28
prologichmm03:28
DanielBairdif you really don't like the long URL pattern, the next best would be something like:03:29
DanielBaird.../climatesummaries/current03:29
prologic /models/current03:29
prologic /models/<model>/<scenario>/<year>03:29
DanielBaird.../climatesummaries/projection?model=csiro1&year=2050&scenario=RCP8.503:30
DanielBairdthe current summary isn't a model though03:30
prologichmm03:30
prologicok03:31
prologic /current/03:31
prologic /current/<var>03:31
prologic /models/<model>/<scenario>/<year>03:31
prologic /models/<model>/<scenario>/<year>/<var>03:32
prologic2 endpoints, 4 URI(s)03:32
prologicand ofc /wms03:32
DanielBairdwhat's var though?03:32
prologicok projections03:32
DanielBairda specific bioclim?03:32
prologicbioclim var03:32
prologicyeah03:32
DanielBairdi still feel like the url should point out that these are summaries across regions.03:33
DanielBairdso we can use current/var for the raster map or something03:34
prologicyes I agree03:34
prologicso why don't we have03:34
prologic /projections/summaries/...03:34
prologicand03:34
DanielBairdso if you specify a var, you get that single var, but still summarised for every region03:34
prologic /projections/model/scenario/year/var03:35
DanielBairdokay so the projections/summaries/ m / s / y / get the region summaries, and the projections / m / s / y / var gets the raster layer or whatever?03:36
prologicyeah03:36
DanielBairdthat's col03:36
DanielBairdcool, even03:36
prologicand if a var is not given03:36
prologicthen you get a list of all vars03:36
DanielBairdso current data will be at /current/summaries/ and /current/var or something03:37
prologicand /proejctions/m/s/y03:37
prologicis bounded by the bbox of the map on screen03:37
prologicor03:37
prologicby the bbox provided03:37
DanielBairdhmm for proper namespace protection we need longer urls03:37
prologice.g03:37
prologicGET /projections/m/s/y&bbox=[...]03:38
DanielBairdback.03:47
prologichttp://codepad.org/GoQvj86q03:48
DanielBairdoh yeah, so it's bad form to have /projections/summaries alongside /projections/hadgem3/...  the "summaries" bit is contaminating the namespace of the set of models.  probably we should have /projections/summaries and projections/maps/hadgem3 etc03:48
DanielBairdlooks like you sorted that out :)03:49
prologicyeah :)03:49
prologicany corrections?03:49
prologicI tried to be mostly complete03:49
prologicperhaps not03:52
prologicbut for completeness I'll add it in anyway03:52
prologicso you can get the summary of a particular bioclim03:52
prologicremember the layers themselves are the actual bioclims03:52
DanielBairdand you definitely prefer the model / scenario etc in the url?  it'd also be defensible to have them as params if it makes for a simpler implementation03:52
prologicthe model/scenarios/years03:52
prologicare just directories03:53
prologicumm03:53
prologicI don't really think it matters either way03:53
DanielBairdoh also, do we imagine using medians across models.03:53
prologicexcept that with separate URI parts03:53
prologicyou could split the data files up into separate sub-directories03:54
DanielBairdthat could reasonably be a model type itself03:54
prologicand you could then traverse the API03:54
prologicof available models, available scenarios, etc03:54
prologicmedian across models?03:54
DanielBairdyeah03:54
prologicexplain :)03:54
prologicthat sounds like I have to do a lot of work03:54
prologicand heavy computation03:55
DanielBairdit's common to not pick a specific model, but instead run all models, and for each modelled point, pick the median result from all the model outputs.03:55
DanielBairdour guys here do it in R as part of the model-running, just as a final post-process.03:55
DanielBairdobviously not something you'd do in response to a web request..03:56
DanielBairdstand by i'll find something to illustrate03:56
DanielBairdhttp://tdh-tools-2.hpc.jcu.edu.au/bifocal/assets/data/regions/IBRA_Esperance_Plains//absolute.climate.temperature.png03:57
DanielBairdhere's a graph of multiple model outputs for Esperance Plains.03:58
DanielBaird10th and 90th percentiles (of the 18 models) bound the orange section, with the median projection in black line.03:59
DanielBairdso you can see the optimistic, pessimistic, and middle-of-the-road model projections.03:59
DanielBairdanyway without a good reason to pick a specific model, using a median result is generally useful04:00
DanielBairdalso, interestingly, even the most optimistic model, in the "low" emmission scenario, predicts a 1 degree rise by 208504:01
DanielBairdalthough it's strictly incorrect to refer to the most optimistic model as the 10th percentile line isn't necessarily from the same model the whole way along.  but anyway..04:02
DanielBairdi mean, it's technically incorrect to assume there's a single "most optimistic" model, as04:02
DanielBairdblah blah, yay for climate science.  so the upshot is, someone somewhere should work out the median across models, and then treat that like another model.04:03
prologicis this data available already somewhere?04:08
DanielBairdprobably.. all my modelling for the birds site was based on median models04:08
prologickk04:09
prologicso we could probably pull this data in somewehre04:09
prologicin the same structure/format as the projection models04:09
prologicand current04:09
prologicand we could just shove it in /median04:09
prologiclike /current04:09
DanielBairdit'd be projections/median like projections/hadgem304:12
DanielBairdcoz it's separated into scenarios, years etc04:12
DanielBairdso do you have all the model outputs somewhere already?04:13
prologicno!04:14
prologicfar too much data to download04:14
DanielBairdwhere are we supposed to get it?04:18
prologicthat is a very good question04:18
DanielBairdi presumed it was either available already, or some academic would be generating it for us04:18
prologicI've been getting 5km/1km data from wallace site04:18
prologicyes I believe that academic is Jeremy :)04:19
prologichaha04:19
DanielBairdah okay.  yeah that's fine i guess...04:19
DanielBairdJeremy's old RA, the person who normally would *actually* be making the data for us, no longer works here.  I guess that's a problem for Jeremy to resolve..04:20
DanielBairdanyway, as long as it's Jeremy, we can just assume he will supply all the model data including a special model called "median".04:21
DanielBairdthen it's just subsetting or whatever you call getting data within a region's poly, etc etc04:22
prologic*nods*04:23
prologicI assume so as well04:23
DanielBairdhey my fab docs is failing with "no sphinx_bootstrap_theme", do i need to fab build or something?04:23
prologicwhat's an RA person?04:23
prologicoh04:23
prologicResearch Assistant :)04:23
prologicyes I nkow04:23
prologicI fixed and pusehd that04:23
prologicsorry with a dead desktop at home04:23
DanielBairdResearch Assistant, kind of.  Not sure if she was ever actually employed as that, but she did that role04:23
prologicI haven't managed to get the ircnotiier bot online yet :)04:23
prologicyou know how it is with family, baby girl at home04:24
prologicetc etc04:24
prologic:)04:24
DanielBairdalthough i'm only assuming you just have one04:25
DanielBairdyou seem young..04:25
DanielBaird..one kid is most likely given my analysis of the situation04:25
DanielBairdfab build is dying too.  pip isn't getting circuits-dev04:27
prologichrmm04:28
prologicfab --show=debug deploy04:28
prologiclatest version all works04:28
prologicfor local/dev, test and prod deployments04:29
prologicdoing prod deployment now04:29
prologicpaste your terminal output04:29
prologicshowing what you typed and the output04:29
DanielBaird(ui-ccav)C02JKVMWDR53:ui-ccav pvrdwb$ fab --show=debug deploy04:30
DanielBairdUsing fabfile '/Users/pvrdwb/griffith/ui-ccav/fabfile'04:30
DanielBairdCommands to run: deploy04:30
DanielBairdParallel tasks now using pool size of 104:30
DanielBaird[10.0.0.2] Executing task 'deploy'04:30
DanielBairdParallel tasks now using pool size of 004:30
DanielBaird[10.0.0.2] run: /bin/bash -l -c "export PATH=\"\$PATH:\"/usr/local/bin:\" \" && test -e \"\$(echo /webapps/var/env/devccav)\""04:30
DanielBairdTimed out trying to connect to 10.0.0.2 (attempt 1 of 1), giving up)04:30
DanielBairdFatal error: Timed out trying to connect to 10.0.0.2 (tried 1 time)04:30
DanielBairdTraceback (most recent call last):04:30
DanielBaird  File "/Users/pvrdwb/lib/python2.7/site-packages/fabric/tasks.py", line 316, in execute04:30
DanielBaird    multiprocessing04:30
DanielBaird..followed by a traceback04:30
prologicahh yes04:30
DanielBaird10.0.0.2 is your VM ip right..04:30
prologicremember I made the default deployment04:30
prologicgo to 10.0.0.204:30
prologicso if your vagrant vm isn't running on that04:31
prologicit won't work04:31
prologicI hard coded this04:31
prologicyou could override with -H though04:31
prologice.g:04:31
prologic$ fab -H <your_vm_ip> deploy04:31
prologicor just make your vm's ip the same04:31
prologicyour real machine?04:31
prologicas-in not a vm on your macbook?04:32
DanielBairdyes04:32
prologicor your macbook itself?04:32
DanielBairdmacbook04:32
prologicoh04:32
prologichmm04:32
prologicI'm not sure how well that would work of ryou :)04:32
prologicfab deploy tries to do some apache config04:32
prologicand supervisor config as well04:32
prologicit assumes a bunch of stuff04:32
DanielBairdhmm okay04:32
prologicnamely that you provisioned the server with the playbooks :)04:32
prologicyou probably just want04:32
prologicfab build04:32
prologicfab frontend04:32
prologicccav04:32
prologicrather than deploy to your mac :)04:33
prologicwhich wasn't provisioned with the playbooks :)04:33
DanielBairdwell i develop using fab frontend.build on this local machine, that works fine.  i was just trying to build the docs04:33
prologicit'll still run stand alone :)04:33
prologicoh04:33
DanielBairdfab build dies doing pip, it's trying to get to 10.0.0.204:33
prologicyeah for the docs04:33
prologicjust do:04:33
prologic$ fab docs:view=yes04:33
prologicfor local build/viewing of the docs04:33
prologichmm04:34
prologicit shouldn't04:34
DanielBairdfab docs dies coz of the no bootstrap theme.04:34
prologiclemme check04:34
prologicdid you pull/merge?04:34
DanielBairdfab docs is where i started :)04:34
prologicI fixed that a few revs ago04:34
DanielBairdten minutes ago i did, just did another and got one more change04:35
prologic~/ccaih/ccav04:35
prologic$ fab build04:35
prologic[localhost] local: pip install -r requirements.txt04:35
prologic[localhost] local: python2.7 setup.py install04:35
prologicDone.04:35
prologicfab build should not do anything with any remote hosts04:35
prologicit does a local build04:35
prologicI'm confused now ;)04:35
DanielBairdnope it fails for me, on pip requirements.txt04:35
prologicwtf04:35
DanielBairdwhen i run pip install -r requirments.txt, it bitches about getting circuits-dev04:36
prologicshow me again04:36
DanielBairdhere's fab build:04:36
DanielBaird(ui-ccav)C02JKVMWDR53:ui-ccav pvrdwb$ hg fetch && fab build04:36
DanielBairdpulling from ssh://hg@bitbucket.org/ccaih/ccav04:36
DanielBairdsearching for changes04:36
DanielBairdno changes found04:36
DanielBaird[localhost] local: pip install -r requirements.txt04:36
DanielBairdFatal error: local() encountered an error (return code 1) while executing 'pip install -r requirements.txt'04:36
DanielBairdAborting.04:36
DanielBaird(ui-ccav)C02JKVMWDR53:ui-ccav pvrdwb$04:36
DanielBairdhere pip:04:37
DanielBaird(ui-ccav)C02JKVMWDR53:ui-ccav pvrdwb$ pip install -r requirements.txt04:37
DanielBairdRequirement already satisfied (use --upgrade to upgrade): py==1.4.14 in /Users/pvrdwb/.virtualenvs/ui-ccav/lib/python2.7/site-packages (from -r requirements.txt (line 11))04:37
DanielBairdObtaining circuits from hg+https://bitbucket.org/circuits/circuits-dev#egg=circuits (from -r requirements.txt (line 14))04:37
DanielBaird  Cloning hg https://bitbucket.org/circuits/circuits-dev to /Users/pvrdwb/.virtualenvs/ui-ccav/src/circuits04:37
DanielBairdabort: could not find web.cacerts: /Users/pvrdwb/.ssl/dummy.pem04:37
prologicwhat version of pip?04:37
DanielBaird  Complete output from command /usr/local/bin/hg clone --noupdate -q https://bitbucket.org/circuits/circuits-dev /Users/pvrdwb/.virtualenvs/ui-ccav/src/circuits:04:37
DanielBaird  04:37
DanielBaird----------------------------------------04:37
DanielBairdCommand /usr/local/bin/hg clone --noupdate -q https://bitbucket.org/circuits/circuits-dev /Users/pvrdwb/.virtualenvs/ui-ccav/src/circuits failed with error code 255 in None04:37
DanielBairdStoring complete log in /Users/pvrdwb/.pip/pip.log04:37
DanielBaird(ui-ccav)C02JKVMWDR53:ui-ccav pvrdwb$04:37
DanielBairdi skipped some "req satisfied" lines04:38
prologicoh04:38
DanielBaird1.3.104:38
prologicyou're run into that problem04:38
prologicI'm surprised you haven't run into it earlier04:38
prologichttp://mercurial.selenic.com/wiki/CACertificates#Mac_OS_X_10.6_and_higher04:39
prologicyou need to do this04:39
prologicthen mercurial won't complain at you04:39
prologicfor ca cert crap04:39
prologicyou were using http:// urls for bitbucket repos04:40
prologicwhen I introduced circuits-dev del in the requirements.txt04:41
prologicI used a https:// url04:41
prologicI naturally use https:// for most things without thinking04:41
prologicthe .hgrc config I showed you was good04:41
prologicbut I forgot to tell you how to generate the dummy cacert :)04:41
prologicthat the [web] cacerts points to in my config :)04:41
prologicone day I'm going to build an playbook to create my home directory04:42
prologicand a commun development enviornment04:42
prologic:)04:42
prologicyeah04:43
prologicbut I want more than just dot files :)04:43
prologichaha04:43
prologicnew desktop - no worries04:43
prologicrun bootstrap shell script04:43
prologicwhich downloads playbook04:43
prologicand runs it against localhost04:43
prologicfew hours later04:43
prologicdesktop back the way it was :)04:43
prologicone for Mac desktops04:43
prologicone for LInux desktops04:44
prologicand maybe another for remote shells/servers04:44
prologic:)04:44
prologicnothing like consistent environments04:44
prologicprod instance is up04:44
prologicand now we have online docs as well04:45
prologichttp://testccav.terranova.org.au/docs/04:45
prologichttp://ccav.terranova.org.au/docs/04:45
DanielBairdokay now i need sphinxcontrib.bitbucket04:48
DanielBairdsphinx fails when i fab docs04:48
DanielBairdshould sphinx be in the requirements.txt or something?04:50
prologiclocally?04:54
DanielBairdwell fab docs wants it04:55
DanielBairdwell i guess given i have sphinx, i can presume it's in something as a requirement, but I can't fab docs because sphinx wants sphinxcontrib.bitbucket04:56
prologicupdate your repo04:57
prologicfixed that too :)04:57
prologicor rather04:57
prologicfixed it in case the local environment is missing the docs dependencies :)04:57
prologictry now after updating :)04:57
DanielBairdcool, working now04:57
prologicsweet :)04:57
prologicit's better this way anyway04:57
prologicyou can just fab docs04:58
prologicon a new envionrment04:58
prologicand it'll pull in sphinx fo ryou04:58
DanielBairdwe have a meeting now i guess.  lucky coz we're hardly communicated today at all :)04:58
prologicand all required plugins/extnesions04:58
prologiclol04:58
prologickk04:58
prologicbetter head over there :)04:58
DanielBairdhttp://codepad.org/pQ0QzsWE added the region info urls05:01
prologicback05:21
prologichmm05:23
prologicwhat would the Region Data API(s) return?05:23
prologicproperties of the region's vector/polygons?05:23
prologiccause that's sort of covered by the Features API05:23
prologicwhere you can access any collection of vector features in geojson05:24
prologicwith their bbox, polygons, and peoperties05:24
DanielBairdso far i can only think of state and polygon.  maybe area?05:34
DanielBairdurl of the region's managment body..05:34
prologicwell05:34
prologicwe can either call the endpoint regions05:34
prologicor features05:34
prologicI don't care either way05:34
prologicbut afaict they are basically just accessing vector data05:34
prologicwhether it be state boundaries05:34
prologicnrm, lga, ibra regsions05:34
prologicor whatever05:34
prologicright?05:35
DanielBairdthe cool way to do it is to have region urls that get docs that include a feature url for the bounds etc05:35
prologichmm05:36
DanielBairdthat way the freature urls can be pure geo05:36
prologicelaborate05:36
DanielBairdthe feature documents, i mean05:36
DanielBairdbut05:36
prologicand the region data?05:36
DanielBairdbut the Simplest Thing is to have in the region's data in the climate summary, a field called "bounds" or something, which is the URL of the boundary feature.05:37
DanielBairdand once we have proper region data that isn't just a boundary, we start using the region urls for that.05:38
prologichmm05:38
prologicstill confused :)05:38
prologicI see no difference between regions and features05:38
prologicfrom my point of view05:38
prologicthey access the same data05:38
prologicthe vector layers05:38
prologice.g: Shapefiles05:38
prologicwhich contains MultiPolygons05:38
prologicand proeprties05:39
prologicfrom my perspective I see 3 things05:39
prologicsummaries05:39
prologicweb map tiles (ams)05:39
prologicfeatures (vectors)05:39
prologicwhat we're calling a "region" here05:40
prologicis just a collection of features05:40
prologicor a specific feature05:40
prologica vector or collection of vectors05:40
prologice.g: state boundaries05:40
prologicor lgas, etc05:40
prologicam I missing something? :)05:41
DanielBairdI think we shouldn't shape the api from such a data-centric view.. the actual things we're granting information about is IBRA Bioregions, LGA Governmental Areas, etc05:41
prologic*nods*05:42
DanielBairdit's true that all we care about technically is the boundary05:42
prologicand/or it's polygon05:42
prologicwe can just get rid of the Features API05:42
prologicor alias regions/features so they point to the same thing05:43
DanielBairdah okay.. i thought we might also have state and country boundaries, etc as features05:43
prologicaren't they also regions? :)05:43
prologicwhere do we draw the line05:43
prologicI sort of get what you're saying05:43
DanielBairdand small-r region as a geographic feature defined using a set of polygons05:44
prologichmm05:44
prologicok05:44
DanielBairdso, a Region will hav a bounding polygon set, which is a feature05:44
prologicbut currently we have no such data to play with :)05:44
prologicwe only have IBRA revisions right now05:44
prologicat least this is the only set of vectors I have access to05:44
prologicwhich includes properties like the name of the IBRA region05:45
prologicetc05:45
DanielBairdyeah so one way to do it would be to have all geo-polys available from a Features api point05:45
prologicas well as the polygon that makes up the region05:45
prologicok05:45
prologicso GET /api/v1/features/<layer>/<id>05:46
DanielBairdand a Region document points at it's bounding polygon, just using a feature ID (REST says you should have a full URL but obviously it will include the feature id)05:46
prologicwould get the feature from collection layer of id05:46
prologicas a geojson polygon05:46
prologicwhereas GET /api/v1/region/... would get?05:46
DanielBairdthen we can also have addditional features, like the country outline, that aren't Regions, but are just polys etc05:46
DanielBaird(okay reading back)05:46
DanielBaird../regions/<regionid>/ would get you a little JSON doc, e.g.05:48
DanielBaird{   name: "Northern Jungle",05:48
DanielBaird    state: "Queensland",05:48
DanielBaird    boundary: //ccav.org/api/v1/features/122405:48
DanielBaird}05:48
prologicaahh05:49
prologicI see05:49
prologic:)05:49
prologicthank you05:49
DanielBairdthe state boundary URL could also be listed05:49
DanielBairdin fact might as well link the country name and boundary as well.  so we can go global later easily :)05:49
DanielBairdtechnically we should also have state / country etc as separate documents available somehow05:50
prologicwell05:50
prologicthen let's just do that05:50
prologicand rather than the key bondary05:50
prologicfeature05:50
prologicwhich points to a feature URI05:50
DanielBairdin fact it could have region-type: 'ibra', region-parent: URL-to-parent05:50
DanielBairdthen the parent region would have region-type: 'state', region-parent: url-of-australias-doc05:51
DanielBairdno it should be called boundary, and have a feature url05:51
DanielBairdor border, or edge or whatever05:51
prologichmm *shrugs*05:52
prologicok :)05:52
prologicregion boundary -> feature uri05:52
DanielBairdso you can also have 'major rivers" or omething, that would also be feature urls05:52
prologickk05:52
prologichmm05:53
DanielBairdhard to stop making cool apis isn't it05:53
prologicshall we worry about countries/states now05:53
prologicor later?05:53
DanielBairdhmmm05:53
DanielBairdhow about05:54
DanielBaird- implement Regions and Features05:54
DanielBaird- Regions point to their outline Feature05:55
DanielBaird- plan to do state boundaries as features too, and Regions can link to their own state boundaries05:55
DanielBairdthen we can have country & state outlines, but aren't committed to doing another big swathe of api implementation05:56
DanielBairdacutally it occurrs to me that i need to know the right colours for each state..05:56
DanielBairdgah!05:56
prologicand we'd still have a /regions endpoint?05:56
prologicI would have thought we'd have something like05:56
DanielBairdyeah /regions for the doc that names the region, but points to the outline at /features/<someid>05:56
prologic.. /countries/australia/05:57
prologic.. /countries/australia/queensland05:57
prologic.. /countries/australia/queensland/<some_ibra>/05:57
prologicisn't it possible that if this were to be a globally available too05:57
prologicthat two or more countries could list the same ibra region name?05:57
DanielBairdi do like the nested URLs a lot though05:59
prologicwell it's just that if we did go global05:59
prologicthe regions endpoint would no longer work very well05:59
DanielBairdthose REST jerks always recommend /thing/id and no deeper06:00
prologicnamely because it's hard to then pull out regions for a given country06:00
prologicor a state within a country06:00
DanielBairdyeah and we're already only approximating state, there are plenty of ibra regions that overlap state lines06:00
prologicyeah06:00
DanielBairdthere's even NRMs that overlap each other, kind of06:00
prologicI mean the only way we could make /regions work06:00
prologicif going global, etc06:00
prologicis by ensuring each and every region entered into the vector database06:01
prologichas a unique id06:01
prologicregardless of what country/state the region belongs to06:01
prologicI guess it could work the other way around though06:01
prologicwhere (in future)06:01
prologic.. /countries/australia could provide meta data06:02
prologicthat provide a list of regions for australia06:02
DanielBairdyes i was just typing that..06:02
prologicwhich end up being uris to /regions/blah06:02
DanielBairdyes06:02
prologicalright06:02
prologicI'll type this up06:02
prologicbased on your modified paste earlier06:02
prologicthen I'll start trying to implement this on Thur06:02
prologicaway tomorrow06:02
DanielBairdnp06:02
prologichaven you run a full provision/deploy lately?06:03
prologicI wanna know if it all works for you as well06:03
DanielBairdso, i guess we should have a way to get the list of regions first.. reading /regions/ should return a list of all regions06:03
prologicI know it works for me here and against nectar06:03
DanielBairdhaven't tried for a while06:03
prologicyes06:03
prologicGET /regions06:03
DanielBairdi haven't quite gotten ansible / vagrant deploying working yet06:03
prologicthere should be a way to list any resource06:03
prologiceven GET /api/v106:04
prologicshould also list what resources there are06:04
prologicI'll also make the API(s) self documenting06:04
DanielBairdso maybe we can do that, and reading /regions/ gets you a structure divided by country, then region or something.. to avoid having to implement a country endpoint06:04
prologicpulling any doc strings out of the python modules that implement them06:04
prologicdon't get that last statement06:05
prologicin theory you could/should be able to06:05
prologicGET /regions/where=country="Australia"06:05
prologicor something?06:05
DanielBairdyes, but we don't have to do that for now, coz if i get /regions i should get a document that lists all regions, right?06:06
prologiccorrect06:06
DanielBairdand I'll just fetch their boundary features, and draw them all..06:06
prologicok06:07
prologicalthough hang on06:07
DanielBairdif we want, the big list can be grouped by country06:07
prologicGET /regions06:07
prologicwouldn't that give you a document06:07
prologiclisting the types of regions available?06:07
prologice.g: NRM, LGA, IBRA, etc?06:07
prologicthen06:07
prologicGET /regions/nrm06:07
DanielBairdoh yeah, yes if we keep that region type thing.06:07
prologicwould give you a list of all nrm regions06:07
DanielBairdwe could drop it and just have regions/<id>06:07
prologicthat wouldn't work I don't think06:08
prologicthe id won't be unique across region types necssarily06:08
DanielBairdand in the regions doc, it has:06:08
DanielBairdcountry: Australia06:08
DanielBairdregion_groups: {06:08
DanielBaird}06:08
prologicbut /regions/nrm/id and /regions/lga/id would be unique06:09
DanielBairdwe can just use random ids06:09
prologichmm06:09
DanielBairdno need to guess, the regions URLs are listed in the Big List06:09
prologicrandom ids like?06:09
prologica uuid?06:09
prologicand store it in the database?06:10
DanielBairdacutally there's no such thing as country, it should be called "region authority" or something06:10
DanielBairdcould be anything but something like "au_ibra_MySpecialRegion" would be alright06:10
prologichmmm06:11
prologicit means I have to do more work to make these ids06:11
prologicthe only id I have to work with for IBRA (for example)06:11
DanielBairdbut anyway i don't care all that much, we could even go to  regions/au/ibra/regname06:11
prologicis the OBJECTID06:11
prologicwhich is unique per record in the shapefile06:11
prologicotherwise I'd have to "somehow" create unique ids for regions06:11
DanielBairdyeah06:12
DanielBairdthe alternative though is having to make docs at each interim url, e.g. regions/ => list of countries, regions/country => list of region types, regions/country/ibra => list of ibra regions, regions/country/ibra/34 => the region doc.06:13
DanielBairdsry i shouldhave used au in most of those examples, regions/au/etc06:14
DanielBairdi think it's safe to assume that every country has a two letter IANA code06:14
prologicI much prefer this06:17
prologicit means we can already support this out of the box06:18
prologicat least with au and ibra06:18
prologicI have no other region data right now06:18
prologicmeans my directory structure could look like:06:18
prologicregions06:18
prologic|-au06:18
prologic  |-ibra06:18
prologicwith a Shapefile in regions/au/ibra/06:18
prologicI like this approach06:19
DanielBairdyeah i'm okay with a deep url like that, and if your docs are generated from directory names or whatever that's pretty easy06:19
DanielBairddocs = GET request responses06:19
prologic*nods*06:21
prologicwell that's what I was hoping for in the end06:21
prologicI've avoid shoving any of this into a database as yet06:21
prologicas I haven't found the need for any complex relationathips06:21
prologicgonna head home I think06:23
prologicwill see you Thur06:23
DanielBairdnp, see you06:23
*** DanielBaird has quit IRC08:06
*** DanielBaird has joined #cmvt08:36
*** DanielBaird has quit IRC08:45
*** DanielBaird has joined #cmvt09:11
*** DanielBaird has quit IRC09:16
*** DanielBaird has joined #cmvt10:26
*** DanielBaird has quit IRC10:30
*** DanielBaird has joined #cmvt11:26
*** DanielBaird has quit IRC11:31
*** DanielBaird has joined #cmvt13:26
*** DanielBaird has quit IRC13:30
*** DanielBaird has joined #cmvt13:58
*** DanielBaird has quit IRC14:24
*** DanielBaird has joined #cmvt15:06
*** DanielBaird has quit IRC15:14
*** DanielBaird has joined #cmvt16:11
*** DanielBaird has quit IRC16:15
*** DanielBaird has joined #cmvt17:11
*** DanielBaird has quit IRC17:15
*** DanielBaird has joined #cmvt18:11
*** DanielBaird has quit IRC18:16
*** DanielBaird has joined #cmvt19:12
*** DanielBaird has quit IRC19:16
*** DanielBaird has joined #cmvt20:13
*** DanielBaird has quit IRC20:17
*** DanielBaird has joined #cmvt21:14
*** DanielBaird has quit IRC21:18
*** DanielBaird has joined #cmvt22:14
*** DanielBaird has quit IRC22:19
*** DanielBaird has joined #cmvt22:45

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!