IRC Logs for #circuits Monday, 2014-03-17

*** Osso has joined #circuits00:00
prologicOsso:  ping?00:34
Ossohey00:34
prologichey00:34
prologichttps://gist.github.com/therealprologic/9591955 <-- Can you shed any light on this?00:34
prologicone of our production plone sites is just r00ted :/00:35
prologicI mean it's not infected with a r00t kit :)00:35
prologicbut this is really bizare00:35
Ossowhat is the bizarre part ?00:37
prologicthat what it's trying to import is there00:38
prologicit imports part of the module00:38
prologicbut not the BadCommand class00:38
prologicwhich is _clearly_ there as well00:38
Ossoah00:39
Ossoit's commented it shouldn't even import anything?00:41
Ossook yeah nvm00:42
Ossocan you do00:42
Ossoimport supervisor00:42
Ossosupervisor.__file__00:42
Ossoor even better from supervisor import options and options.__file__00:43
prologicyeah I've done all that00:45
prologicthey all points to what I think they should00:45
Ossoit's just not possible00:46
prologicyou're telling me00:47
prologicI know this00:47
Osso:)00:48
Ossoyeah00:48
prologichttps://gist.github.com/therealprologic/9591955#comment-119233300:49
prologichttps://gist.github.com/therealprologic/9591955#comment-119233400:50
Ossosee00:50
Ossothe options.__file__ is different00:50
prologiceh? wait..00:50
prologicomfg00:51
prologicyou're totally right00:51
OssoI still didn't solve your problem but now you are happy00:52
prologicnow why is that happening I wonder00:52
prologicsys.path is being replaced entirely00:52
Ossosomething in the __init__.py ?00:52
prologichttps://gist.github.com/therealprologic/9591955#comment-119233600:54
Ossosupervisor must be playing with paths or something00:55
prologichalf way through it's own imports?00:56
prologicfk'n hell00:56
prologicI dunno what's going on :/00:56
prologicI'm going to remove the system installed (via pip) supervisor00:56
prologicand see if that gets me back to a state where things are working00:56
prologic(sort of)00:56
prologicthis vm has huge/high I/O wait times too00:56
prologicso the site is very very slow :(00:56
prologicSo that seems to have solved that problem00:58
prologicThe I/O issue I can't solve - not my infrastructure :(00:58
prologicand we use a really shitty buildout config to build our plone sites00:58
prologicwith varnish, haproxy, supervisor all built as part of the plone buildout00:59
prologicreally sucks :/00:59
Ossothis http://stackoverflow.com/questions/7785944/what-does-import-pkg-resources-declare-namespace-name-do01:00
Ossoyeah I was wondering why plone needed supervisor01:00
prologicoh it's just because of the shitty way in which we manage our production stuff01:02
prologicit's all self-contained into a single buildout config01:03
prologicso not reusable in any way01:03
prologicand means we can't do proper config management on the box itself01:03
prologicwhy are you showing me this SO question?01:03
Ossosomehow the declare_name is import the wrong files01:04
OssoI am not familiar with the quirks of that function but it's definitely messing up the path01:04
Ossoyou could install the plone in a virtual environment or something that way and still be able to user supervisord01:05
prologichmm01:09
prologicsupervisor doesn't use declare_nsmaepace though?01:10
Ossoit does https://github.com/Supervisor/supervisor/blob/master/supervisor/__init__.py01:10
prologicoh I see01:12
prologicdamnit :)01:12
prologicwell I thank you for helping me figure out this stupid mess01:12
OssoI also have a question for you01:13
OssoI am setting up this dev environment with docker01:13
OssoI am having troubles sharing files outside the vm cause boot2docker does not support nfs01:13
Ossohave you solved this already?01:14
OssoI may just setup a gentoo minimal system and install nfs there and be done with tinycore01:14
OssoI downloaded gentoo and took the 32 bit by mistake01:15
Ossosad daya01:15
Ossohave you had similar problems ?01:16
prologicwait sorry01:24
prologicyou want to share files from one docker container to another?01:24
prologicboot2docker is meant to be a minimal docker environment01:25
prologicyou _could_ just run an docker nfs server as a container01:25
OssoI want to edit the files in my editor01:25
Ossoto do that I need some sharing with virtualbox01:25
prologichttps://index.docker.io/search?q=nfs01:26
prologicahh01:26
prologicso you want to spin up a container01:26
prologicand edit it's files?01:26
Ossoyes01:26
prologicjust shell int oit?01:26
prologicor01:26
prologicrun /bin/bash01:26
prologicthen commit your chnages01:26
Ossoyes you use linux01:26
Ossoit's so easy01:26
prologicwhat's your host?01:27
prologicWindows?01:27
OssoI have a mac01:27
prologicahh01:27
Ossoon macs it runs in the vagrant VM :(01:27
prologicwell01:27
prologicboot2docker is it's own thing now01:27
prologicit doesn't use Vagrant anymore01:27
prologicthank goodness :)01:27
prologic-but-01:27
prologicyou should be able to:01:27
prologicbrew install docker01:27
prologicand01:27
prologicbrew install boot2docker01:27
prologicand do something like:01:27
prologicdocker run -i -t --name="mydev" /bin/bash01:28
prologicfrom your Mac01:28
prologicand you'll get a shell inside that newly named container01:28
prologicwhen you exit form it, commit your changes into an image01:28
Ossoyes well that works01:30
prologicdoes that help?01:31
prologicor am I missing something important you want out of this :)01:31
Ossonoo I still need to access files with my editor01:31
prologicfrom your host?01:31
Ossoyes01:31
prologicusing like Sublime or something?01:31
Ossoyes01:31
prologicyeah so what you would do is this:01:32
prologicdocker run -i -d -v /Users/Osso/myproject:/myproject --name="my project" cents /bin/bash01:32
prologicfor example01:32
prologicwhere /Users/Ossoo/myproject is your project file son your host01:33
prologicwhich are bind mounted (volumes) to /project in the container01:33
Ossoyes but that mounts in the vagrant vm01:33
OssoI really get screwed with this vagrant01:33
prologicno that will mount the host path into the container01:33
Ossoyes docker runs in vagrant01:34
prologicand btw, go get the latest boot2docker too01:34
prologicit doesn't use vagrant anymore01:34
Ossothe host for docker is vagrant01:34
prologic*nods*01:34
prologicoh01:34
prologichang on you're right01:34
prologicno wait01:34
prologicumm01:34
prologicoh geez01:34
Ossoyeah I am right :(01:34
prologiclemme ask Sven01:34
OssoI need this nfs crap01:34
prologicyeah I see what you're saying01:35
prologicyeah01:38
prologicsorry I understand what you'er saying now01:38
prologicsee I don't work in this kind of work flow with Docker01:38
prologicand I don't really think your'e meant to either :)01:38
prologicas such01:38
prologicbut in any case - anything is imsspobie :)01:38
prologicyou do have to use some kind of network file system though01:39
prologicNFS, Samba, SSHFS, etc01:39
prologicand I think the recommended approach is so create tow containers01:39
OssoI tried sshfs it's not playing nice with git01:39
prologica Data container/volume01:39
prologicand your runtime container01:39
prologicand share the data container via some means01:39
prologicand mount it from OS X01:39
OssoI am not sure docker is the right solution either anymore01:41
OssoI gain nothing on top of a standard vm01:41
prologicno well that's the thing01:41
prologicif you treat docker as nothing more than a vm01:41
prologicthen you will gain nothing01:42
prologicyou need to use it for what it was intended for01:42
Ossoand I'll be using vms on digitalocean so running docker on top makes little sense in the end01:42
prologicwell no01:42
prologicit does make sense01:42
prologicfor ons simple reason01:42
prologicif you do it right/properly01:42
prologicthe docker images you create and maintain01:42
prologiccan be run anywhere docker can run01:42
prologicsure vas can be too - sort of01:43
prologicbut not really01:43
Ossoit does make sense in case I switch from digitalocean I suppose01:44
Ossoalso I wanted to work on the prod environment locally then move it online01:44
prologicyeah okay01:45
prologicso you should cahgen some of your development philosophy there01:45
prologicideally you really ought to build your application's code outside of docker01:45
prologicat least I think so01:45
prologiconc you're happy with it01:45
prologicthen write a Dockerfile that packages up your application01:46
prologicthe idea is to have a single image running one part of your application01:46
prologiccomponentizing parts to other images/containers01:46
prologicthis is where volumes and container linking comes into play quite nicely01:46
prologicas a simple trivla demonstration01:47
prologichttps://bitbucket.org/prologic/test_docker_pr_441601:47
Ossowell it does make sense to dev on a vm cause I need to share my environment with other people01:47
Ossodocker makes it easier to export my vms and stuff01:48
prologicyes that's fine of course01:48
prologicbut as you've just found out01:49
prologicgetting into a container is not trivial01:49
prologicespecially on an OS that doesn't natively support Docker itself01:49
prologicbut you can still share your so-called dev with your fellow colleagues01:49
prologicand you should by way of consistent images anyway01:49
OssoI'll write the network filesystem myself if needed01:49
prologic thing is if you're not using "docker build"01:50
prologicto build the image via a Dockerfile01:50
prologicyou're not really guaranteeing consistent image builds01:50
prologicnor utilizing the Docker cache either01:50
Ossoyes and no there's always a place you have to start from01:50
prologicso the workflow really should be thing01:50
prologic(On your Mac OS X):01:50
prologichg clone https://bitbucket.org/osso/myproject01:51
prologiccd myproject01:51
prologichack hack hack01:51
prologictest test test01:51
Ossoit's like saying you can't have consistent images because you are not compiling your kernel in ubuntu01:51
prologicdocker build -rm -t osso/myproject .01:51
prologicdocker push osso/myproject01:51
prologicand then your colleague(s)01:51
prologicbasically the same thing really01:51
prologicoh and throw in a docker run in there too01:51
prologicfor testing the built image/container01:51
prologicwell yeah sort of :)01:52
prologicif you don't use the same build/deploy tools01:52
prologicyou can't get consistency :)01:52
Ossoit don't need it01:52
prologicthink of Docker as a configuration management tool01:53
prologica container management tool01:53
prologicand a social media tool01:53
prologicall rolled into one :)01:53
prologicbuild consistent images01:53
prologicrun containers from prebuilt images01:53
prologicshare images with colleagues or publically01:53
Ossothe thing is why would you build the image each time ?01:54
prologicyou get guarenteed consistency across the board01:54
prologicany docker image you build here01:54
prologicwill run there01:54
prologicyou don't build it each time01:54
prologicyou update it01:54
Ossothe consistentcy is guaranteed by a starting point + a way to go from there01:54
prologicremember I said Docker has a cache?01:54
prologichave a look at:01:54
Ossomy starting point already includes everything01:54
prologichttps://bitbucket.org/prologic/ircnotifier/src/0151077a71714c74167ed1ab07d84a793362476e/Dockerfile?at=default01:55
prologicNote L5: FROM prologic/crux-python01:55
prologicI maintain a base image (stock standard CRUX) called crux01:55
prologicfor the crux linux distro that I use01:55
prologicon top of that image I maintain a crux-python image in my personal image repo01:55
prologicwhich ircnotifier uses as a base for it's application01:56
prologicyeah re starting point01:56
prologicyour starting point _should not_ contain everything01:56
prologicbecause you loose the ability to split up your application into separate components01:56
prologicsay you're building a web application01:56
prologicit might consist of the web app itself01:56
prologica load balance in front of it01:57
prologicand maybe a reverse proxy in front of that01:57
prologicand maybe it might also be linked to a database (e.g: mysql or similar)01:57
prologicif you roll your image/container into a single thing01:57
prologicyou may as well just use Vagrant or a VM01:57
prologicbut then you loose flexibility01:57
Ossouhm yeah I can see that01:57
prologicwhat if someone else has a different database01:58
prologicor doesn't want to run it up behind a load balancer01:58
prologicwhat if someone is using a different reverse proxy01:58
Ossothat's too bad for them01:58
prologiclol01:58
prologicthis is the typical way we've been doing things so far though01:58
prologicwhich is why Docker is great01:58
prologicit gets rid (if you do it properly) of all that01:58
prologicsuddenly you can scale infrastructure and applications horizontally and vertifally01:59
prologicand not have to worry about all the different ways in which things can go wrong01:59
OssoI started with trying to use templates01:59
prologicthis version of that01:59
prologicdoesn't with with this, that or the other01:59
Ossothe problem is say emerge --sync takes way too long01:59
Ossoso I decided to include it in the image02:00
prologicso one of the key things here obviously02:00
prologicis use a base image02:00
prologicif the gentoo base image is not good enough02:00
prologicbuild a new one on top of it02:00
prologictake advantage of the layered file system(s) of Docker02:00
prologicDocker based infrastructure supports both AUFS and BTRFS nowasays02:00
prologicI use CRUX+Docker (on the Desktop) at home on a BTRFS storage backend (1TB SATA)02:01
prologicbut as I've pointed out above02:01
prologicthat's precisely why I have a prologic/crux-python base image02:01
prologicfor all my circuits/python apps :)02:01
prologichttps://index.docker.io/u/prologic/crux-python/02:02
Ossomy question is02:02
prologicSee TurnKeyLInux rolls everything into a single image02:02
Ossowhy would use the docker templates system02:02
prologicbut they have to because that's what they do02:02
Ossoit's easier to maintain an ansible recipe or puppet02:03
prologicbut it means if I spin up TuenKeyLInux based appliances I have no flexibility about how those appliances are run, what databases they use, where they store their data, etc02:03
prologicwell that's the thing02:03
prologicI wouldn't necessarily say it's easier to maintain a set of ansible playbooks02:03
prologicor puppet recipes02:04
prologicsure you can and docker + ansible can even work nicely together02:04
OssoI was planning to do that02:04
prologicbut historically puppet/anisble tend to build entire infrastructure(s)02:04
prologicI kinda tend to like the docker approach02:04
prologictreating everything as an application02:04
prologica first-class process02:05
prologicrather then a "system" on a "server"02:05
prologicat some point the docker devs are going to solve the problem of02:05
prologic"How do we configure and maintain Docker based infrastructure?"02:05
prologicAnd right now you can already do it a number of ways02:05
prologicBash scripts02:05
prologicFabric02:05
prologicAnsible02:05
prologicetc02:05
prologicincluding managing and configuring what Docker "runs"02:06
prologicI've suggested that Docker grow another file format called Appfile02:06
prologicor something02:06
prologicwhere you can (say using YAML) configure your application, the images it uses, other containers it links to, etc02:06
prologicand just do: docker run myapp.yml02:07
prologictaking several "docker run" commands into just 102:07
Ossomaintaining the docker is more like a problem on top02:08
prologic*nods*02:08
prologicthat's what I said too02:08
prologicso I'm not sure what they plan on doing at this stage02:08
prologicbut something will come out of it02:08
prologicthere are already several things02:08
prologiclike fig02:08
prologicand dies02:09
prologicdeis*02:09
prologichttp://deis.io/02:09
prologichttps://github.com/orchardup/fig02:09
OssoI can see the value of docker as vm I can run everywhere so far02:11
Ossothat's better than nothing02:12
Ossoso besides the reproduction of the steps02:13
Ossoif I setup an environment and then I move it prod and it works the same that's great02:14
prologicdiscoverability02:14
prologiclinking02:14
prologicisolation and security02:14
prologiclightweight containers vs. heavyweight vms02:14
Ossoisolution is good but do we really need it in most cases?02:14
prologicremote api(s) and realtime events02:14
Ossothe database will run on a different server anyway02:14
prologicisolation can be argued either rway02:15
prologicbut I think it's important02:15
Ossoit comes for free with docker so it's nice02:15
prologicwell it comes from lxc/containers - so yeah02:16
prologicDocker plans to build more execution engines into Docker02:16
prologichopefully by the 1.0 release02:16
prologicso you'll probably see execution engines such as02:17
prologiclibvirt, ivm, etc be introduced at some point02:17
prologicthis means you'll be able to use the same Docker commands and API(s) to spin up things that can't be Containerized/Dockerized02:17
prologice.g: Windows vms02:17
prologicbut treat them almost the same way as everything else02:17
Ossosure but you end up with bigger problems02:20
Ossonow you have a dozillion instances to maintain02:20
Ossowith their own os02:20
prologicno, not OS02:24
prologic:)02:24
prologicdon't think of them as an OS02:24
prologicit's just convenient to start with a Linux distro of some kind (an OS) as a starting point02:24
prologicI can see in the future we'll get out applications to a point where for example:02:24
prologica base image for running C applications02:24
Ossowell lxc would not be an OS it's really is a lightweight VM02:24
prologicwill be nothing more then glibc and the C application itself02:25
Ossodocker has the whole image installed02:25
prologicjust as an example02:25
Ossoyou have to keep it updated02:25
prologicwho says you have to keep it updated? :)02:25
Ossome02:25
Osso!02:25
prologichopefully the exact same image I created 10 years ago02:25
prologicshould still run today on Docker :)02:25
prologicbut off Docker is only a year old02:25
prologicso let's say 10 years form now02:25
prologicthe same prologic/ircnotifier image should still run perfectly fine02:26
Ossoit'll run with 1000 security problems02:26
Ossobut it'll run02:26
prologicwell not really02:26
prologicit's isolated :)02:26
prologicso who cares02:26
prologicthe only interface to irc notifier is http and irc02:26
prologictbh I dbf if someone managed to hack my application container02:26
prologiccbf*02:27
Ossoyou care because if there's a buffer overflow in some lib, you still get hacked02:27
prologicsure02:27
prologicbut that's jut _oen_ container02:27
prologicbtw your arguments are off sound and just02:27
prologicI'm just countering them :)02:27
Ossoeven if it's just the database container it reveals all your data02:28
Ossoit's not nice02:28
prologicsure02:28
prologicbut that's kind of a different problem02:28
prologicDocker is no magic bullet :)02:28
prologicbut what it does provide is quite nice02:28
prologicit provides a consistent and robust set of tools for development and deployment of applications and infrastructure02:29
Ossoyes right02:29
prologicessentially solving many conflicting deployment and dependency issues that you typically encounter with normal vm-based approahces02:29
prologicI can be assured that giving my sysadmin team a docker image it will run as intended02:30
prologicand they won't be able to "fuck it up" :)02:30
prologicand what's more02:30
prologicthe same production instnaces02:30
prologicare identical to the development ones02:30
prologicand vice versa02:30
prologicthere is no need to do any of this stupid rubbish of:02:30
prologicworks in production02:30
prologicdoesn't work in development02:30
prologicor the other way around02:30
prologicor the other stupid crap like02:30
prologic"works for me"02:31
Ossoyes that02:31
prologicIHMO Docker is a whole new philosophy02:31
prologicand whole new approach to Development and Deployment02:31
prologicthat has to be adopted in full :)02:31
Ossobut see that logic leads me to a signle VM because I want everything consistent within it02:31
prologicor it's just a wrapper around lxc then02:31
prologicand you may as well stick to whatever you've used before02:31
prologiclike I said02:32
prologicthere's nothing wrong with an all-in-one approach02:32
prologicjust that you loose some flexibility that's all02:32
prologicand you probably won't take advantage of "Container Linking" then02:32
prologicwhich is fine - if that's what you want02:32
prologicTurn Key Linux does this02:32
prologicthey now provide Docker images of all their appliances02:33
prologicas all-in-one just like the rest of their images02:33
prologicOopenStack, VMWare, VirtualBOx, OpenVZ and now Docker02:33
Ossothe linking is only useful for production though02:33
prologicwell no it really isn't :)02:33
prologicit's useful for all sorts of things02:33
prologicnot just limited to "production" :)02:33
prologicconsider for example that on your dev docker based enviornment02:34
Ossowhere I am hazy is how I will sync my dev environment and my prod environment together02:34
prologicyou always have a mysql container setup and running02:34
prologicexposed to your host for convenience02:34
prologicand you develop applications that use a mysqldb02:34
prologicwhen you test/develop them you just link against your already running db container02:34
prologicIHMO02:34
prologicyour dev/prod environments _should_ be identical02:34
prologicin every way02:34
prologicthere should be absolutely no difference whatsoever02:35
Ossoright02:35
prologicthe differences come from the sysadmin team02:35
prologicor whoever runs up the prod instnaces02:35
prologice.g: container linking02:35
prologicwhat database to use02:35
prologicwhat to log things02:35
prologicwhat options to run it with02:35
prologicetc02:35
Ossoprod is still different in places because02:35
Ossoyou'll use more caching02:35
prologicbut otherwise you should be able to run what's in prod02:36
prologicbac in dev locally02:36
prologicand vice versa02:36
prologicsure02:36
prologicbut you should/can do that in dev too02:36
Ossoand the database will store data in a non-docker partition02:36
prologicthat's what I say02:36
prologicdev == prod == dev02:36
prologicthey should not be different02:36
Ossoyeah I suppose02:36
Ossobut here is my problem02:36
prologicif they are different in some way02:36
prologicthen that's a problem02:36
OssoI can either give my colleagues a VM02:37
Ossoor I can give them 1002:37
OssoI don't see that working out02:37
prologicuse fig :)02:37
prologicsplit it up02:37
prologicuse fig to help automate this02:37
prologicgive them 1002:37
prologic:)02:37
prologicbut they need not care about it02:37
prologicI think fig helps to manage this in a nice way02:37
prologicI haven't used it myself yet mostly because I haven't exactly done anything too complex like this yet02:38
prologiconly a trivial 2-container setup which works rather nicely and how I expect02:38
prologicif you split it up as well02:39
prologicsome parts of your system (the whole)02:39
prologicwill not change02:39
prologice.g: the database image will probably never change02:39
prologicnor would things like the reverse proxy image, cache, etc02:39
prologichttp://crosbymichael.com/dockerfile-best-practices.html02:41
prologichttp://crosbymichael.com/dockerfile-best-practices-take-2.html02:41
prologicbbs - getting some food/lunch02:42
OssoI am going ot bed02:42
OssoI asked about nfs I end at 4am02:43
Ossohere02:43
*** qwebirc87078 has joined #circuits02:46
prologicoh Osso sorry!02:52
prologichaha02:52
prologicI feel bad :(02:52
Ossoit's ok but I need sleep02:53
prologic*nods*02:53
prologicg'night :)02:53
*** Osso has quit IRC03:54
*** ircnotifier has joined #circuits07:08
*** qwebirc87078 has quit IRC07:20
*** Osso has joined #circuits07:50
*** Osso has quit IRC08:54
*** Osso has joined #circuits11:09
prologicmorn'n Osso :)12:08
Ossohello12:09
prologicsleep well?12:09
Ossoyeah12:11
OssoI took the day off12:11
Ossonow I am getting  RPC: Authentication error; why = Client credential too weak :D12:11
prologichaha12:14
*** Osso has quit IRC16:46
*** Osso has joined #circuits16:49
*** Osso has quit IRC20:06
*** Osso has joined #circuits20:49
*** jgiorgi has joined #circuits21:47
jgiorgii'm getting an attribute error using circuits.web21:47
spaceonepaste it21:47
jgiorgiline 44 of wrapper it's trying to del self.session, i'm not using sessions (just a single controller and the server) and my controller isn't using the session attribute21:48
jgiorgii'll pb the error one sec21:48
jgiorgihttp://codepad.org/DNbyqXSl21:48
jgiorgiand the code http://codepad.org/d2LBJA6E21:49
spaceonehmm, this should be impossible21:51
spaceone160                 if hasattr(self, "session"):21:51
spaceone161                     del self.session21:51
jgiorgii got it21:52
spaceone 43                 if hasattr(self, "session"):21:52
spaceone 44                     del self.session21:52
jgiorgii have a method named session21:52
spaceone:/21:52
jgiorgiman that was dumb21:52
*** qwebirc72436 has joined #circuits22:05
prologichello all22:05
prologichi jgiorgi22:05
prologicjgiorgi:  you had a method called session?22:06
prologichmm22:06
prologic:)22:06
prologicit's times like these that I'm not very happy with _some_ parts of circuits.web22:06
spaceonejuhuu22:07
prologicI want to make it more event-driven like the rest of circuits22:07
spaceonei will do this22:07
prologicoh good :)22:07
prologicthere are two main things I want improved22:07
prologicmore event-driven style in circuits.web22:08
prologicand better more consistent objects that represent various parts of the process22:08
prologichttp objects, request, response objects, etc22:08
prologicI feel/find that they're a bit inconsistent at times22:08
prologicalso spaceone if you have a few mins to chat22:08
prologicI've been thinking...22:08
spaceoneall this is addresses in my current implementation22:08
spaceones22:08
spaceoneyes i have22:08
prologicwhy should circuits.web do things like:22:08
spaceonein query?22:08
prologicparse http headers22:08
prologicparse query string parameters22:09
prologicparse the body (depending on the content-type being posted)22:09
prologicetc etc22:09
prologicwhy should it do this (or any of it) at all22:09
spaceoneall this i already implemented22:09
prologicIHMO it should do it lazily22:09
prologicor22:09
prologicupon demand/request by the developer writing a web application22:09
prologicmy view is this22:09
prologicwhy bother doing CPU bound work22:09
prologicif you don't have to and it never gets used22:09
prologicsure22:09
prologickeep to the spec22:09
prologicprovide such functionality in some way22:09
prologicbut if your web app never uses or cares about any of the incoming http headers22:10
prologicwhy parse them22:10
prologicif you web app never cares about the query string parameers22:10
prologicwhy parse them22:10
prologicit's wasted CPU effort22:10
spaceoneok, query string is parsed when it is accessed in my implementation22:10
prologicsee that's great22:11
spaceonesome headers must be parsed to ensure that valid http behavior is ensured22:11
prologicbecause that means if you don't ever use it, it doesn't get aprsed22:11
prologicand doesn't waste CPU22:11
prologicif we can do this across the board with circuits.web22:11
spaceonebut ofc. not all headers must be splitted and so on22:11
prologicI think you'll get a 2-3x improvement in performance22:11
prologicand better concurrent loads22:11
prologicdepending on what your app does22:11
spaceoneyes22:11
prologicyes eel specially things like22:12
prologicAccept:22:12
prologicdoesn't need to be parsed necessarily22:12
spaceonehmmm22:12
prologicjust stored and lazily evaulated22:12
spaceonedisagree22:12
prologicihmo it's application dependent22:12
prologicor Static (dispatcher) dependent22:12
spaceonethe server should always do content-negotiation22:12
prologicagh yes but22:12
prologicof you just have Server()22:13
prologicand nothing else22:13
prologicwhy would it do content negotiation at all?22:13
prologicthere's nothing to ngeotiatite with22:13
prologicit should only do so in conjunction with Static() dispatcher22:13
spaceonehmm… but when do you have such a server?22:13
prologicwell you would most of the time22:13
prologicServer() + Root()22:13
prologicwith a simple Hello World22:13
prologicor just a restful server with no static content22:13
spaceoneuhm, ok.. but Root() is a resource22:14
prologicwhere the content negotiation is done via the request handlers22:14
prologicyes but it _doesn't_ have to negoitatite content types22:14
spaceoneit should22:14
prologicihmo that should d be handled by application code22:14
spaceoneimho not ^^22:14
prologicok let's assume you have a simple trivla web app22:15
prologicthat does nothing but responds to /22:15
spaceonea resource is meant to have different representations22:15
prologicand returns text/plain22:15
prologicof "Hello World!"22:15
prologichow would it negotiate content?22:15
prologicwhy would it?22:15
prologicYes okay, but Root(Controller) != Resource22:15
prologiccircuits.web _should_ probably grow Resource(es)22:16
prologicin which case I agree22:16
prologicall Controller(s) do really is route requests based on URI(s) to events/channels22:16
prologicI have some thoughts on that too (if we're finished and agree finally on the content negoitation)22:16
prologicin short Server + HTTP should not do it -- Resources should22:17
prologicResources and/or Static should22:17
spaceoneyes22:17
spaceonethats the way i implemented it22:17
spaceoneResource() += ContentNegotiation()22:18
prologicokay good22:18
prologicnow on to Controller(s)22:18
prologicI believe this should grow a new feature enhcnacement22:18
prologicand break the API of circuits.web a little22:18
prologicspecifically22:18
prologicI think it would be nice to be able to do something like this22:19
prologichttp://codepad.org/SQv2sg6c22:21
prologic^^^ This is the first idea I had...22:21
prologicThis structure would result in the following requests and paths:22:21
prologicGET /22:21
prologicGET /foo22:21
prologicWhere the default dispatcher attaches Foo on to Root22:21
prologicbased on it's channel22:22
*** Osso has quit IRC22:22
spaceone^^22:22
prologichttp://codepad.org/nE4PnQIx22:22
prologicAnd this would attach Bar() to Foo22:23
prologicand result in22:23
prologicGET /22:23
prologicGET /foo22:23
prologicGET /foo/bar22:23
prologici.e: use the registration hirachies22:23
spaceonei also implemented this22:23
spaceonebut a little bit different22:23
prologicwell great :)22:23
prologicbut I want to see Dispatcher grow this feature22:23
prologicI don't think Controller needs to change at all22:23
prologicbut I would like to get rid of the @expose wrapper rubbish22:24
spaceonein my implementation you can define absolute and relative paths22:24
prologicthat sets convenience attributes that get deleted after the request22:24
prologicthey might be convenient, but they have edge cases I don't like22:24
prologicI'd rather see things like:22:24
prologicJSONSerializer() component that listens to resposnes22:24
prologicand serializes the body to JSON22:24
prologicrather than wrapping method calls22:25
spaceoneyes22:25
prologici.e: make it all event-driven like it should have been22:25
prologicalso I'd like to see one more thing22:29
prologicthe HTTP protocol moved out into circuits.protocols22:29
prologicand support both client/server operation(s)22:29
prologicand a proper/better/much-improved circuits.web.client module22:29
prologicimplementing Client()22:29
spaceonehttp://cxg.de/_8141b0.htm22:31
spaceonehmm22:31
spaceonei created httoop the HTTP library which can be used for all HTTP purposes22:31
spaceonee.g. server, client, proxy, cache, intermediary, etc.22:32
spaceoneit implements a state machine which can parse any kind of messages giving you appropriate objects you can work with22:32
prologiccool22:34
prologicwe'll have to see about working it into circuits.web soon22:34
prologicmaybe/hopefully for a circuits 3.122:34
prologicwithout breaking too much :)22:34
spaceonei guess 3.122:34
spaceonei need more time!22:35
prologicof course if we break too many API(s)22:35
prologicwe'll have to release circuits 3.022:35
prologicerr22:35
prologic4.022:35
prologic:)22:35
spaceoneyes22:35
spaceonei will definately break one API22:35
prologicdon't worry there'll be plenty of time22:35
spaceonei don't give request and response as parameters22:35
prologicyeah I wonder how strict semantic versioning really is22:35
spaceonei will give a client as param22:35
prologictechnically speaking the circuits API is the same22:35
prologicbut circuits.web is kind of like a builtin package of circuits22:36
prologicthis is why I wanted to split eveyrhing up22:36
prologicinto separate packages22:36
prologicwith namespaces (but they don't really work that wlel)22:36
prologicand have different versions for things22:36
prologicbut it's all "too hard"22:36
spaceoneyes22:36
spaceonemaybe namespace packkages is the wrong approach22:37
spaceone(it's not a python builtin)22:37
prologichmm just had a look at the example you proivde22:38
prologicI like some iof it22:38
prologicwe'll talk about ti more later in more specifics when you have more time22:38
prologicyeah I still think we should go down the plugins route22:39
prologicnamespaces just don't work22:39
spaceonethe example is only the first approach22:39
*** Osso has joined #circuits22:41
prologicfor one thing...22:42
prologicI'm not sure that22:42
prologic@method22:42
prologicdef GET(...)22:42
prologicis entirely very useful22:42
prologicsince all resources should only respond to http verbs anyway22:42
prologici.e: if you are subclassing Resource22:42
prologicclass Foo(Resource):22:42
spaceoneyou must in theory be able to handle a http method with a function which has a different name22:43
spaceonealso decorators hide implementation details which makes it easy to change underlying implementation22:44
prologice.g:22:44
prologic@method22:44
prologicdef bar(...):22:44
prologicthat makes no sense?22:44
spaceone@method('GET')22:44
spaceonedef bar()22:44
prologicI see22:45
prologicbut I kinda don't like this22:45
prologicbecause it means bar is a little tiny sub resource22:45
prologicunder it's larger resource22:45
spaceoneno22:45
spaceoneno22:45
spaceoneno22:45
prologicno?22:45
spaceoneHTTP applications must use HTTP verbs22:45
spaceoneMUST*22:45
prologichow would you request what @method("GET")\ndef bar() provides?22:46
spaceonehm?22:46
spaceoneclass Foo():22:46
spaceone    path = '/'22:46
spaceone@method22:46
spaceonedef GET(): pass22:46
spaceone→ GET / HTTP/1.122:46
prologicand if it had22:47
prologic@method("GET")22:47
prologicdef bar(): pass22:47
spaceoneGET / HTTP/1.122:47
spaceonea i understand22:47
spaceoneah*22:47
prologic:)22:47
spaceonethats the wrong approach in circuits.web22:47
prologicyes it is :)22:47
prologicthat was my initial point22:47
spaceonethe -a-fractal-of-bad-design-routing-in-circuits-web22:48
prologica Resource should be exactly that22:48
spaceonebecause you can do this in circuits web:22:48
spaceoneclass Foo(BaseCircuitsWebComponent)22:48
prologicwell you can do this with Controller/Dispatcher22:48
prologicbut I don't think Resources should22:48
spaceone    def foo(bar, baz): pass22:48
spaceonethen you can do:22:48
spaceoneGET /foo/a/b22:48
spaceoneor22:48
spaceoneGET /foo?bar=a&baz=b22:49
prologicof course22:49
prologicthis is quite standard and normal practice22:49
prologictypically called object dispatching22:49
spaceonebut bad practice22:49
prologicwhich is useful in it's own right22:49
prologicno22:49
prologicit's not bad practice at all :)22:49
prologicit's just not rRESTful22:49
spaceoneit's not HTTPful22:49
prologicit's like I said22:49
prologiccircuits.web should grow Resources22:49
spaceoneREST is very much more22:49
prologicRESTful resources22:49
prologicwell no I disagree22:50
spaceoneREST doesn't have anything to do with this22:50
prologicit's still quite valid HTTP22:50
prologicyou're just abusing the path22:50
prologicand mapping it onto objects22:50
prologicHTTP doesn't disallow this at all22:50
spaceoneit does22:50
prologicshow me! :)22:50
prologicthen explain why every freak'n web framework does this22:50
prologicand has this feature22:50
prologic:)22:50
spaceoneif a resource is available under multiple URL's it must inclide a Content-Location header22:50
prologicPylons, Pyramid, Django, Flask, Bottoe., Zope, CherryPy22:51
spaceonebecause they are all violating the rules22:51
prologicI don't see that as violating the rule of: if a resource is available under multiple URL's it must inclide a Content-Location header22:51
prologicanyway it doesn't matter eally22:52
prologicHTTP is flexible and was designed to be22:52
spaceonealso for the implementation this is very bad22:52
prologicit doesn't matter that some food want to use object dispatching22:52
prologicand some folks want to remain pure22:52
prologicpure to HTTP/REST22:52
prologicwhich is fine22:52
spaceoneit exposes internals, opens mutliple ways to get to 1 resource22:52
spaceoneopens security issues22:52
prologicso we _shoudl_ provide and support both22:52
prologicThe only security issue with the current circuits.ewb implemtnation22:53
prologicis that any event handler can be requested by a http client22:53
prologicwhich we _should_ fix :)22:53
prologiconly request handlers of a Controller should ever be exposed22:53
prologicbut that'll be fixed22:53
prologicI think you lodged an issue about it22:53
prologicjust so we're clear ...22:53
prologic:)22:53
spaceone*invisible security issues*22:54
prologicthe current and supported API(s) of circutis.web22:54
prologicshould not go away22:54
prologicexisting web apps written on top of circuits.web should still work22:54
prologicwith little to no modifications22:54
prologicwhat I would like22:54
prologicis to incorporate your improvements, fixes22:54
prologicand new features22:54
prologicspecifically around proper HTTP22:54
prologicand Resources22:54
prologicand providing a framework for building RESTful web applications and apis22:54
spaceonewait, don't use the term REST22:55
prologicas well as keeping the existing functionality in tact (mostly)22:55
prologicsome of I _want_ to change/fix22:55
prologicwhy not? :)22:55
spaceonebecause it is misunderstood22:55
prologicBy RESTful I mean HTTP + HATEOS22:55
prologicno22:55
prologicnot by me :)22:55
prologicI know precisely what it means22:55
prologic:)22:55
spaceonedid you read the dissertation?22:55
prologichttp://ccav.terranova.org.au22:55
prologichas a set of RESTful API(s)22:55
prologicwhich I built on top of circuits.web22:56
prologicyes I did :)22:56
spaceone"There is also a JSON Schema based API that can be accessed via /api/ "22:56
spaceoneOo22:56
prologicyeap22:56
spaceone/api/ → a single entrypoint22:56
prologic*nods*22:56
spaceoneJSON → not a REST format22:56
prologicerr22:57
prologicsays who :22:57
prologic:P22:57
prologicit's all content negotiated22:57
spaceonecurl -X PUT http://ccav.terranova.org.au/22:57
spaceone→ treats as GET22:57
prologicumm22:58
prologicsorry dude22:58
prologicit _is_ RESTful22:58
prologicbut ti only supports GET22:58
prologicit's all statically generate data22:58
spaceoneit event doesn't label the resource either implicit or explicit as cacheable22:58
prologicthere ewas no need to support the addition, creation or removal of resources22:58
prologicI think I disabled caching because of a bug :/22:59
prologicsorry :)22:59
prologicotherwise all resources would have an ETag22:59
spaceonebut it must return 405 method not allowed then22:59
spaceoneetag is not enough22:59
spaceonecache-control23:00
spaceonemay-age23:00
spaceonemax-age23:00
*** qwebirc72436 has quit IRC23:01
prologic405 method not allowed?23:02
prologichmm23:02
spaceonealso a bug flaw in circuits.web23:02
spaceoneGET /POST23:02
prologichmm?23:02
spaceoneyou can define the handler for POST23:02
spaceoneand access it via GET23:02
spaceonei didn't check the other way round23:03
spaceonebar / HTTP/1.1 would call bar() ?23:03
prologichmm23:03
prologicthat's not a nice bug :)23:03
prologicfile an issue about that one too23:03
prologicsucky :/23:03
spaceonei don't find my writeup about this23:04
prologicyeah that's new to me23:04
prologicthe GET /POST thing23:04
spaceonethe opens security issues23:04
prologiccreate a issue23:04
spaceonebecause you cannot rely on the path23:04
spaceonenot now…23:04
spaceonei have this always in mind23:04
spaceonei will create issues when the it is time for them23:05
prologicokay23:05
prologicI'll t/l :)23:05
prologicttl*23:06
spaceoneprologic: btw. because of these security issues i was able to retrieve your bitcoin credentials23:06
spaceone(proove in query)23:07
spaceonei have to sleep now23:07
spaceoneand the next 2 weeks i am working on my final-project23:08
*** jgiorgi has quit IRC23:31
*** ircnotifier has quit IRC23:54

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!