Dev8D 2011 – day 2

I attended Dexy and Molly workshops and Ask the Experts discussions.

Ask the Experts about how to go about Dealing with dirty data

Top tips

  • Google Refine – CSV clean up; output into other formats
  • AntiWord – Word formats to plain text converter
  • FMT (formatting)
  • Beautiful soup (python) – scrapper
  • Scrapperwiki – remember this can be useful – can be used like a remote data store
  • Python unicodedata.normalize – to format data into normal form C – flatten aceented characters
  • Mozilla has auto detect character encoding tools

Dev8D 2011 – day 1

For the second year I have attended Dev8D, the JISC funded developer conference.  An excellent opportunity to network with other developers in Higher Education and to learn about new technologies.

Below are some highlights of the sessions and talks I attended on day 1.

Blackboard

About

  • written in Java
  • had a stable API since 2001
  • many recent API and plugin development improvements
  • direct database querying is now licensed
  • entity relationship diagram is published
  • lots of existing plugins, including text/SMS services
  • plugin exposure can be targeted to specific users
  • supports plugins not written in alternative languages
  • uses SOAP but a Newcastle chap has created a REST API
  • CourseSites.com offers 5 free courses (supports subset of OpenID like providers)
  • granular security policies for web service API – down to the per function  level

General VLE tit bits

More VLEs are moving towards using *IMS common cartridges* providing greater interoperability

Look into further

  • LIS
  • SIS
  • OCELOT community
  • SCORM stands for Sharable Content Object Reference Model
  • IMS LTI (Learning Tools Interoperability) standard uses SOAP

Lightening talks

See: http://data.dev8d.org/2011/programme/session-type.php?type=http://data.dev8d.org/2011/programme/dev8d_programme.rdf%23sessiontype-Lightning

Molly

  • feed aggregator
  • HE produced using a sustainable approach
  • Python/Django
  • Opensource
  • XML
  • JSON
  • HTML5
  • Format X – future formats
  • can still target handsets – smart/feature phone
  • very inclusive
  • long term support, University of Oxford has committed 2.5 FTE to project
See related application: MyMobile Bristol

www.dreamspark.com

  • Free Microsoft professional tools for students.
  • .NET

Naturelocator

The JISC funded “Nature Locator” project will help the researchers by creating mobile applications that provide geo-tagged photographs, and visualisation tools to facilitate crowd-sourced verification of the data submitted during 2011

Related technologies

  • Titanium – translates your hard won web skills into native applications that perform and look just like they were written in Objective-C [iPhone and iPad] or Java [Android].
  • PhoneGap – an opensource mobile phone framework using HTML5, CSS3, JavaScript and Cloud deployment

All your bases belong to us: L10N & L18N @ Dev8D

Malte Ressin discusses his Phd research at Thames Valley University and the issues that affect internationalised and localised projects
  • pluralisation (2 kinds in Russian)
  • sort order (in Spanish ll is one letter not double l)
  • affects many areas: sales, UI design, legal, marketing, publisher, …
  • early research required to discover any cultural content guidelines
Contact Malte Ressin if interested in being part of his case studies

What makes Dexy so Sexy for creating beautiful code documentation?

Dexy is an open source document automation tool that can help you create documents using your favourite programming languages and your favourite software.
  • Output formats include: CSV, latex, PDF, and WordPress posts
  • Can incorporate testing output (cucumber or was that water??)
  • Can automate screen shots – part of implementation of the test
  • Python command line tool

Lucero

Linking University Content for Education and Research Online
http://lucero-project.info/lb/

Look up

Uses for RDFa to promote details for pricing info in Google

Archives Hub Data and APIs

  • union catalogue
  • could be institutional records, …
  • use EAD (Encoded Archival Description – XML)
  • provides searches/indexes of the archives
  • uses CQL (Contextual Query Language)

LOCAH

  • project using Linked Data
  • related to COPAC data

Look up related projects

DBpedia – a community effort to allow you to ask sophisticated queries against Wikipedia

Molly 1

About

  • feed aggregator
  • Opensource
  • Python >= 2.6 < 3
  • Django 1.2
  • PostgreSQL but others DB could be used
  • mobile web vs native
  • could be just used as an aggregation service not for mobile web
  • produced by University of Oxford
  • nearing version 1.0
  • akin to Java based MyMobile Bristol application developed by ILRT, University of Bristol during the same period

Features

  • Geodata including Open Street Map
  • nearest real time bus info (Oxford specific)
  • contacts via LDAP
  • library search
  • maps – well targeted for feature (non smart) phones
  • podcasts
  • feedback/voting
  • url shortener
  • QR codes (3D barcodes)
  • oAuth
  • batch processing
  • easy to override blocks of templates and media
  • easy to plug in different data providers
  • installer will ask most setting questions
University of Oxford runs on Ubutntu 10.04 LTS VMWare ESXi, 2GB RAM
Also learnt: oAuth supports disabling device access from source application.  Useful if a device is lost.

Day 3 at dev8D 2010

Some quick notes about the lightening talks I attended.

Archiving Tweets with Twapper Keeper

Why archive tweets?

  • Twitters search limits results by number (1500) or time, approx 7 days
  • You may want to capture an events output and ensure this data is not lost

Linked data in the web

We should integrate more, it’s not all or nothing. Where do I start:
  • model your stuff
  • identity with cool URIs
  • link your html together
  • link to machine friendly formats
  • use RDF to make your database structure available on the web
Comment from audience: Use RDFa to annotate link relationships easily.

Mendeley

  • an endnote alternative
  • last.fm for researchers papers
  • startup working with several partner universities
Future plans for an OpenAPI with various export formats
  • REST
  • oAuth
  • JSON
  • Export groups:
    • open ‘anonymised’ data due March 2010
    • user data due April 2010

Finding Nero

  • about collections/repositories
  • Culture24 API available, they want feedback about output format
  • CultureGrid – aggregator

Collaborative tools project

A talk about a new project being under taken at the University of York
  • LifeRay – portal builder
  • Network Thinking
  • JISC – People Project – people recommendations

Mistakes I have made building web applications

By Juliette Culver, Open University

  • Not dealing with character encoding from the start
  • Not establishing which web browsers you support
  • Bad choices about 3rd party code (not using them when I should have)
  • Under estimating time required for legal and acquiring domain names
  • Not dotting i’s and cross t’s
  • Admin and stats most often get the extra change requests
  • Asking permission from non techies for time for refactoring, accessibility, web security – add non as these elements are essential, a no  techie will always say no
  • Not colour contrast testing on design work
  • Putting off usability testing
  • Underestimating the problem of spam, see Mollum
  • Not protecting programming time – turn off other messenging for example/block off programming time from meetings, see makers schedule article
  • Stop expecting anyone to appreciate what you do

Comments from audience

  • Under estimating the time it takes to go from prototype to production
Visualising Data with Processing
by Ross McFarlane
  • Processing is a Java related language for graphics
  • 3D – OPENGL less buggy than the built in

Day 2 at dev8D 2010 – Lightening talks and Ruby on Rails coding lab

I attended the Ruby on Rails code lab and the following lightening talks:

Top tips

  • Stay out of core code at all costs, extend using plugins
  • Check out OWASP top 10 tips web security tips
  • Don’t save back up config files in the web root, for instance config.php.bak will be readable
  • Symbian apps are easy to make
  • RepRap rocks!
  • Django handles configuring paths better than Ruby on Rails

Manufacturing for the masses, RepRap notes

A fantastic 3D printer that can create plastic mouldings.

  • can print 50% of own parts (in 2.5 days)
  • most non plastic parts are widely available
  • GPL
  • costs approx €350
  • future aims to be able to shred and recycle your own plastic bottles or use starch based sources

More info:

Some commercial outlets, such as Bits from bites, make t machines that don’t replicate themselves, but can replicate parts for a RepRap machine.

Day 1 at dev8D 2010 – Pipes lab and lightening talks

I attended the first half of the Pipes lab and following up by catching some of the lightening talks.

Yahoo Pipes notes

About

  • will mash up various data formats; e.g. RSS1, RSS2, atom
  • will normalise out from multiple input formats
  • can output as XML, JSON, stub of PHP, image/media version of feed, etc.
  • each pipe has a unique URI
  • uses a GUI interface, with flow chart style objects, to create new feeds
  • requires a Yahoo account
  • pipes can be claimed or cloned, NB: only an owner can only delete a pipe
  • could a useful tool for useful for examining feeds

Sources of data

  • RSS
  • CSV
  • HTML
  • Yahoo searches/queries
  • existing pipes

Downsides

  • Don’t seem to be able to export the structure of the feed mash up
  • A third party hosted service

Alternatives

The open source version, Deri pipes, can be locally hosted.

Notes on uses of modules Can mash up search RSS feeds (e.g. Plone/WordPress search results) and pass in arguments
– see url builder and user input modules

Bear in mind if using search RSS results these feeds may be paginated or limited to n number of output
– example uses uses search term – url builder – fetch feed – output

Can compound your feeds, i.e. call a search feed for each search result of another feed – see loop function

Private input – protects for instance API keys from cloned pipes
– remember there can be many levels of caching – at source, in yahoo pipes, in your own application

Lightening Talks

I attending lightening talks about WordPress, Teaching programming to non programmers and building Android applications.

Android phone application notes

  • Use Eclipse IDE (plugin may be necessary)
  • Install Android SDK, includes
    • emulator
    • command line tool
    • demo code
  • Uses a locked version of Linux
  • API demos is good demo code to look at

Description of 3D modelling in Android