Getting the Edge with Network Analysis with Alan Nir

Today I am lucky enough to be attending PyCon UK 2018.  I have chosen to get a brief introduction into network analysis.

Alan Nir is giving a high level intro into how network analysis can benefit your work/research.  He is giving a brief introduction and no mathematics and algorithms will be introduced in this session.

My favourite line, a network in just a collection of points & lines.

Computer science use this terminology:

  • points AKA nodes
  • lines – AKA edges, lines

Alan’s demo and recommendation is to start with this Python library, NetworkX.  A comprehensive Python library with a nice API and good documentation which the logic behind the functions. More information can be found here:

Alan created a demo of creating and visualising a network in only a few lines of Python.

You can input data in various formats including tuples, JSON, and pickles.

NB: Alan will share slides later, spoke rather quickly sadly I was sat too far away to read the projected screen

Examples shown where networks showing Karma cheaters & Eurovision nation voters.

Demo data was from a public payment platform based in the US, Venmo.

Question: Why does this platform exist?  Why would you want to publicise your payments?

NB: The Speaker obfuscated the data, why as it is already in the public domain. I suppose reuse rights may be in question.

 

 

PyCon UK 2017

Yay. I have made it to PyCon UK again this year.

I’m starting on day 2 this year but still glad to be here.

As usual the friendly Python community are making everyone feel welcome.

I shall be posting notes and tips I pick up over the next few days.

Visit the conference site for more information about PyConUK.

Look out for the video uploads of the talks post conference too.

Scrum master training

I really enjoyed refreshing my Scrum practice, when I recently attended Scrum master training with Paul Goddard from Agilify.  Below is a quick overview of Scrum.

What is Scrum?

Scrum is an Agile framework for software development.

The ability to respond to change and deliver useful quality products quickly through incremental development and client participation.

Scrum makes use of…

Cross skilled self-organising groups that work at a sustainable rate.

This leads to my favourite outcome:

Sane happy people make nice code!

Scrum manifesto

Scrum has a manifesto of core principles which I will paraphrase below as:

  1. Continuous customer satisfaction
  2. Advantage through change
  3. Frequent functional deliveries
  4. Constant team collaboration
  5. Trust the team
  6. Get together often
  7. Delivery = Progress
  8. On-going sustainable development
  9. On-going best practice
  10. Keep it simple – Think less waste and minimal viable product.
  11. Collaboration optimises results
  12. Learning from experience

What are the roles within a Scrum team?

Scrum Master

This role is to facilitate and coach the team.  To assist in the application of the Scrum framework.  As teams are self-managing, this role must support the group developing the product not dictate to the team nor assign tasks.

Product owner

Essentially the client.  Scrum relies on strong client participation, co-locating with the team.  Continual prioritisation of tasks and feedback help to deliver a relevant product and high customer satisfaction.

Team members

The doers – makers, fixers, testers, designers, sysadmins, developers, … Scrum teams tend to be made up of between 4 to 9 people.  Smaller teams are more likely to suffer from a lack of a range of skills.

Key Scrum meetings and artefacts

  • Product vision – what are we building and why.
  • Product backlog – a continually reprioritised list of high-level product features or tasks.
  • Daily stand up meetings – 15 minutes long and should include all team members.
  • Sprints – development blocks of a fixed length between 1 to 4 weeks long.
  • Deliverables – A functional and potentially deployable deliverable should be produced at the end of each sprint.
  • Sprint planning meetings – this can be scheduled to last for 1 hour for each week of sprint’s length.  Product backlog items are selected to be implemented in this sprint and more detailed planning of their implementation is done at this point.
  • Sprint review meetings – A wider user engagement meeting where more user and or clients are shown the Sprint deliverable
  • Sprint retrospective meetings – a look back at how the previous sprint went and ways to improve going forward.

Find out more

Starting with a short fun overview of key Scrum concepts – muppets included.

Scrum for Schmucks https://www.youtube.com/watch?v=oJV-kgj4m68&feature=youtu.be

Learn about Scrum https://www.scrumalliance.org/why-scrum

What is Scrum https://www.scrum.org/resources/what-is-scrum

Notes on R

R – an interpreted programming environment for statistical computing and graphics

Coming to R from Python and other programming languages, my notes are comparative versus exhaustive. See the following for a good guide.

There is a mechanism for installing R local packages (akin to the usage of virtualenv for isolated Python package installations)

 install.packages

Lists are like dictionaries

$ is a key (like [‘sfsf’] or dot)

data.frames are like pandas

dot just a naming convention

<- and <= assignment

Speed tips:
Vectorise (map/reduce like functions) vs looping in r

Can easily import/export CSV

Space tip – If you need to store large amounts of data, consider using binary vs ASCII or other plain text files.

Can access DBs directly, using data frames

Note to self, look up NETCDF (commonly used as a climate data format)

Nice built in datasets

Very easily generate charts and save the output to jpeg, pdf, etc. See for more options: