Category: Sessions

Docker

Posted on by Tessa (the external)

James

Google input

Even finance companies are considering it

Only a young project

What

Packaging
1 app = 1 container

What’s the diff

Uses less space,  only store what is necessary

Start instantaneously

Use cases :
Development
Test prod architecture
Can combine with Jenkins CI CD

Can use base images
And then keep the docker configuration file clean

Is command line tool,  but can use other tools to make simpler

Fig is a helpful tool for creating an infrastructure

Possible mention to http://deis.io/overview/ and Heroku

Almost no down time,  if upgrading a component

Look at mounting volumes for running local dev code

Can use another app to orchestrate the dockers apps

The Twelve-Factor App by Kristian Glass

Posted on by Tessa (the external)

Notes taken during a talk at PyConUK

#1 one code repo many deploys

#2 what about declaring your dependencies, e.g. gcc compiled items, deply containers

#3 read config from environment

debug flag, db connections etc
avoids secret leakage,
add defaults
make it language independent

populating env – foreman, supervisor (d?), ansible, bash -whatever etc

doismellburning – django app

www.doismellburning. co.uk .com?

NB: can be in another repo, just def outside of your app

#4 Loose coupling

#5 build, release (when config gets blended), run

build tip
…
fpm – builds an os package (ruby, don’t snigger too much – it’s awesome)

run stage needs to be a simple as possibly, do as much work in advance.  e.g. fetch eggs in build, collect static etc

#6 stateless

assume nothing about RAM, persist data to external serices

#7 Port binding:
clean interface, should be able to tell your app a protocol and port and go

#8 Scale out not up, start more processes, don’t daemonise – let the env do it

#9 Stop quick;y, stop nicely

#10 Keep env similar: time, people, tool gap

Deploy more often

people
-> get people together, devops, …

tools
-> it works on my machine

#11 Treat logs as event streams

Run a log agreegator

one event, one line

=> q. how do you handle exceptions – let the agreegotr handle this, some differently code new lines.

#12 Run admin tasks in your app, eg clean up old sessions, as a django management command not a one off script on some dev machine

Language and framework agnostics

Performance

Posted on by Tessa (the external)

Notes taken during a Performance talk given at PyConUK.

function calls

c method instead
cython instead

Use operators

Loops

Avoid

list comprehension is quicker than for loop over an emnumerator

What is op?

xrange vs range
– no wasted memory

NB: In python3 renamed range

itertools

Or use a c extension

mstexttools

Sorting

l.sort vs sorted() – any iterator

Key based:

look up operator -> operator.itemgetter

Can use on object attribute

Strings:

“”.join([a, b, c]) faster but less readable

% method not that fast an ugly – don’t use

Dictionary
Int keys are faster than string keys

interned string keys – no garbage collection

sys.intern – P3

Variable look ups

Mentioned global variable trick

local var faster than global in general

Look up: Old style/new style classes??

Slot attributes ?? look up more just seemed to be attributes???

Don’t use exceptions as an often use case. => much faster say to use getattr than catch a valueexception

Try to rely on c methods

Profiling tools:
c profiler built into Python

cpython vs number – can make pythonrun as fast as C – suggestion

A Pinch of Salt by Scott Walton

Posted on by Tessa (the external)

Scott Walton gave a talk about how they use Salt, at their company Pebble.

Slides can be found here: https://docs.google.com/presentation/d/1q82GUdE-zHUlTSYJxkIdW79CcewO3-KXuBwPe9ir_aA/edit#slide=id.p

Salt is a remote config management tool, similar to: Ansible, Puppet, Chef. It seems from talks with others at PyConUK Ansible and Salt were being described more favourably than Puppet and Chef.  Either due to their complexity or through battle testing and reaching their limitations.

Salt terminology

  • Master
  • Minion
  • Grain – data on a minion – like an attribute, how much memory (-G)
  • Pillar – data about a set of a minion, e.g. stage of server
  • Matching – rules to identify minions
  • Modules – commands
  • State – end result

Pebble use Salt to manage a complex system, finance, web servers, databases, load balancing, managing various environments – test, stage, prod etc.

Their journey was via fabric

Implemented in Python

Zero MQ, encrypted secure connections are used

Used apt-get vs pip to install – as in his opinion, it included a flaky dependency

Can add watches for changes to take actions, e.g. if web server config file changes, restart the web server

Can use Jinga in config files pillar (and grain) are available in these templates

Salt can take fall back actions for common features, e.g. running git clone or git pull depending upon whether the repo exists.

– started using 2 years ago

Comparisons:

– chef & puppet looked more complicated
– salt looked for familiar
– friendly community
– easy to get involved in the code
– can easily contribute back pull requests
– documentation has gotten better
– includes very simple state
– reference is very good
– e.g. ‘return as’ is not as well documented as yet

Easy to configure

can command Less files TBC
If not built in, can just run bash commands e.g. could run a Django migration