Oscar Vilaplana: Orchestrating Python projects using CoreOS

See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.

This is about orchestrating stuff. Not specifically Python, but the examples will be with Python.

Reliability. Your app does not care where it is running. Runs locally, runs on the cloud, same thing. Portability, repeatable. Loose coupling, compose micro services, makes it easier to scale them, mix and extend.

Cluster-first mentality. Even development machines can run in clusters. Lot of different containers, servers, ports, all connected, how do you manage this. Do you need to be developer and operations officers in one? Let the system figure it out. Let other smart people figure out and determine where the containers are.

One deployment tool for any service, makes them better and better, new service is then not much extra work.

Demo: deploy db, flask app, scale them. RethinkDB, 3 replicas. Give a name to the service, connect to it from the app by name. With kubectl start the service with the wanted amount of replicas.

CoreOS: Kernel + Docker + etcd. Read-only root, no package manager, self auto-updating, systemd for setting limits to a service and making sure it starts and restarts when it is stopped. Socket activation, starting a service only when someone starts using it.

etcd is a distributed configuration store. Atomic compare and swap, change this setting from 1 to 2, otherwise fail. HTTP API. Configurable at runtime. etcd set /pygrunn/current_year 2015

fleet is a distributed systemd. Start a service somewhere in a cluster. Coordination across the cluster. Rules like do not start A and B on same server because they are cpu hungry.

Service discovery: ambassador pattern, talk to someone who knows where the services are.

flannel does something like that. Per-cluster and per-machine subnet. Every container has an ip on a subnet.

Kubernetes: process manager. A pod is a unit of scheduling, name, container image, resources. Labels for finding things.

Replication controllers: create a pod from a template. Ensure that the exact number of pods are running. Upgrades.

Service discovery for pods. An ip per service.

Demo.

It took me a while to wrap my head around it. Look at the slides in the calm of your home.

Me on Twitter: https://twitter.com/grimborg

And see http://oscarvilaplana.cat