At Skyscanner we use lots of microservices for hotels. Microservices talk to each other with HTTP. The languages that we use all know how to talk HTTP.

We need to know what is happening in our micro services, like: which requests take more than 20 seconds? With aioHTTP middlewares we added code to check this. We make this visible in Kibana.

We wanted to follow the code path taken by a request. aiotask-context stores information within the current asyncio.task instance. The request id is stored on the task by a middleware. That way we can follow a request through the code.

DNS in AWS with aioHTTP. DNS TTL is usually 60 seconds in AWS, so IP addresses can change, and it can be more addresses. But aioHTTP versions below 2 did not support this: they cached the DNS requests. We created code to handle this and cache this for only a short time. And we throttled the DNS events, to avoid querying the DNS a hundred times when you fire a hundred requests very quickly after each other.

We call microservices with a timeout of one second, and catch the error. Timeouts can get triggered when the reactor is saturated. We use asyncio.Future and can cancel such a future when we detect a timeout.

Desired plans:

• trace queued operations when the HTTP pool connection limit is reached
• AWS Xray support
• Back pressure at the HTTP layer. When the reactor is too busy, return a 504 error. Scale horizontally when you get lots of 504 errors.

See the slides.

Python 2 reaches end of life in 2020. Plone does not run on Python 3 yet. We started talking about that in the Bristol conference in 2014, and said it could not be done. Especially hard would be the packages AccessControl and RestrictedPython.

AccessControl is handled meanwhile. I will talk about RestrictedPython. Why should that be a blocker?

Hanno Schlichting, Zope Release Manager, once said: 'Every piece of Zope2 that was not adopted by Plone is literally dead.' So we adopted it. Problem: RestrictedPython had almost no documentation, and low test coverage.

'Where Zope leads, Python follows,' used to be a saying. There are some pieces in core Python that are there specifically for Zope, like the compiler package that is used in RestrictedPython. This RestrictedPython is used in TTW (Through The Web) code, like scripts that some authorized users can write.

In 1994 in the first PyCon at NIST, there was already talk about creating some kind of restricted Python. RestrictedPython is no sandbox: someone is always going to be smarter than you are, and break out of the sandbox. So you get a limited, safe subset of the Python language and grammar. You don't get file access, for example.

John Johnson: 'First solve the problem, then solve the code.' In other words: first understand the problem.

Python 2 had the compiler module and its ast class. But: not fully documented, and no upgrade path for Python 3 described. Compiler knowledge was necessary to port RestrictedPython. I knew something about it from my studies, so I gave it a try. We started looking into it at the Plone Open Garden 2015.

Ken Beck: 'Any fool can write code that a computer can understand. Good programmers write code that humans can understand.' So: I wanted to make the code more readable and documented and tested, so others can work on it too.

Instead of compiler.ast there is ast.AST in Python 3, so that was a start. The builtin compile function since Python 2.6 accepts ast.AST as input and compiles it to Python byte code. With Python 2.6 and 3.4 as minimum, we could get somewhere. I was happy to be able to go to a Plone conference in Japan to work on it.

I started with making it comply with our coding conventions, without changing anything else. And added tests. We started at 18 percent test coverage, and are now at 95 percent.

RestrictedPython depends only on the standard Python library, which made the test setup easier. I switched to tox so we could test multiple Python versions at the same time. Also pytest, which allows parameterisation, which means we could test the old and new implementation next to each other, so we can prove that they work the same.

People from Gocept and other Zope core developers joined in, and together we made it. In May 2017 we released version 4.0a1, and now we have 4.0b2.

Ian Hickson: 'Things that are impossible just take longer.'

My wish is to make RestrictedPython more known, and enable other projects and frameworks to use a 'safe' Python interpreter through the web. Think of projects like Jupyter, where you can write online code in a sandbox; if you are knowledgeable, you could probably kill the server, and RestrictedPython would help here. It could be interesting for Django, Pyramid, guillotina.

Lessons learned:

• impossibly to port code: probably not
• adopt modern tools and frameworks like tox and pytest

Update the best practices for Plone development.

See the slides.

Symbol of generosity for us in the United States is Santa Claus. He distributes gifts for free in the evening before Christmas. Now there is something called SantaCon, where young men go around dressed as Santa and get drunk.

I saw a young Santa who was asked for money by a homeless man in a wheel chair. He gave five dollar. The homeless man did not say thank you. Maybe he was just tired. But then the Santa got enraged, demanding that the man thank him. So here we see power in action. Not a pretty sight. How does that happen.

A new colleague of mine at MongoDB asked me about something. I said it was going to be in the next release, and thought that would finish the conversation. Then he asked me to make more effort: what is the ticket number. For some reason this angered me, and I answered: 'I memorised all ticket numbers, it is 12345.' Then he came back at me: 'Are you sure? I cannot find that ticket.' I regretted my answer.

It is not okay to get mad and lash out at people because they make us angry.

On StackOverflow Python is projected to become the most popular language next year. I think one of the reasons is that we are an inclusive community. We try to be nice to new users. But we are not as welcoming as we could be. I searched for rude answers on StackOverflow and it was not hard to find.

• Question: 'How to I convert GIF to PNG in Python?'
• Answer: 'Just Google it, you are lazy.'

That is not a good answer. It hurts our community, because it drives people out.

Why do we do this? There cam be several reasons:

• We are generous, but get no reward, like the bad Santa.
• Unexpected difficulty: I give an answer, and then the person does not understand me and asks a followup question, and suddenly I am facing more trouble than I want.
• Sense of obligation. We sense we need to give some kind of answer, so we give a rude one.

If I get a promotion, or delete ten emails, it gives temporary satisfaction, but does not really make me more satisfied in the long run. I got a lazy (in my eyes) comment on a Python video that took long to make, and I got angry. I have learned to not make a rude remark. Instead I closed the browser tab. That is the easy way.

When faced with a question, ask yourself: am I angry?

• Yes, I am angry. Walk away. If you can't say anything nice, don't say anything at all.
• Yes, I am angry, but what is my intention? My job may be to answer questions. I may want to take good care of my code, or nurture the next generation of programmers. Thinking about that, am I still angry?
• No. Start your answer with: 'Good question!' Even if it was not really a good question at all. It makes you and the other feel good.
• Is it complete? You are done.
• No, the person asks a followup question. Go back to the first step: am I angry?

Take the 3-breath vow: "I vow to take 3 breaths before I answer a question online."

I am a cloud development advocate at Microsoft.

"The Plone community is a drinking community with a software problem," quoting Alexander Loechel.

"It works on my machine..." You hear that a lot. CICD (Continuous Integration and Continuous Development) help here. I will show you that you too can be a CICD master.

You want to automate your builds, your deployments, and your testing.

You want to have change management. Sure, you use a repository for your source code. But what about database schemas and configuration?

Continuous Integration needs:

• frequest commits
• test runs for each commit
• fixing production is the main priority

Continuous Delivery, why?

• avoid a deployment risk
• real progress: a bug is fixed, and the fix is not just sitting there in a repository, but is actually live
• you get quick feedback from your users

Tasks:

• automate the build
• automate the test
• automate the deployment

For Angular you can use Angular CLI: ng build --prod --build-optimizer. Lots of flags, like --aot for ahead of time compilation, for a faster web site.

On Circle CI you can register with your github account and let it build your project, and then you can use it on for example Docker. It does require access to all your projects currently.

[Demo of Circle CI and Docker.]

There is also an Azure Container registry, where you can have private docker images. You can setup hooks that watch for a new release here, and immediately deploy the new image to production.

• code for demo: https://github.com/simonaco/circleci
• slides on slideshare