EuroPython 2010
July 24, 2010
Well, I’m back from another great EuroPython – my thanks to John Pinner and his EuroPython crew, the presenters, Josette from O’Reilly, and all my fellow delegates. Regrettably, I might not be able to make the next EuroPython in the wonderful city of Florence, but I might be able to make it to PyCon AU in Sydney next year. Fingers crossed on that one – I might even pluck up the courage to do a talk!
Monday
Monday’s talks kicked off with Raymond Hettinger’s “Idiomatic Python”. He guaranteed everyone would learn something new and he definitely delivered. I walked away with some interesting gems of Pythonic knowledge, from basic stuff like enumerate() through to more advancing knowledge like the surprising behaviour of super().
Ezio Melotti followed after the break with a useful overview of the Python development process, which inspired me to join the Python Core sprint on Friday. I’ve already contributed my first couple of patches to Python 3.2 – the first of which has been committed to trunk already.
“How Import Works” was a run through by Brett Cannon on how, you guessed it, Python imports code. Although you’ll probably never need to modify the standard import process, it’s fascinating to discover how it all works. Best of all, new utilities in Python 3 make it much easier to customise things safely if you ever need to.
“PostgreSQL’s Python Soup” was a little disappointing, with a good chunk of the talk being a general one about relational databases. The interesting stuff about connecting to PostgreSQL came quite near the end when we were introduced to the many different connectors. And I mean many.
Monstrum is a new HTTP functional testing framework for Python 3, and was detailed in “Testing HTTP Apps with Python3″. It looks very cool, and the logo would make a great t-shirt! Continuing the web testing theme, Raymond Hettinger followed up with “Python & Selenium Testing”. Alongside an overview of Selenium, Raymond introduced Sauce Labs, a company providing commercial support for Selenium, the web testing framework, via their cloud service.
To round off the day, I attended two talks by Richard Watts and Tony Ibbs of Kynesim who presented Muddle, their open source build system which looks very cool, and KBUS which is an elegant and lightweight messaging system implemented as a Linux kernel extension.
Tuesday
The day’s talks began with a rather packed presentation by Guido on AppStats, an AppEngine monitoring tool. I must admit that I did partially go to hear him talk, as did probably a lot of other people, but also because I was interested to learn more about AppEngine. I picked up some interesting bits of information about AppEngine internals, and the tool looks fantastic. However, I did feel a bit out of my depth.
Bart Demeulenaere’s “Pyradiso Rapid Software Development” was a call to arms, a request and discussion on how to create a rapid application development framework in Python that supports the multicore world out-of-the-box.
Engaging both my computer science and art historical sides, I then attended Richard Barrett-Small’s “The Trojan Snake”. Richard works for the Victoria & Albert Museum, and has been instrumental in introducing Python and Django as a way to clean-up and replace a mess of bespoke PHP, ASP.NET and Java. An interesting look at how an organisation faced issues with bespoke applications in a variety of technologies, and found Python to be a flexible and effective solution. It’s a shame more people didn’t attend.
I was an avid reader of Michele Simionato’s blog posts, covering his experiences with Scheme, so it was great to hear him speak at EuroPython with his talk entitled “Intro to Functional Programming”. Functional programming is receiving a surge of interest amongst programmers, thanks to the search for better ways to deal with concurrency, so it was good for someone to cover other reasons to start exploring FP.
“Tickery, Pyjamas & FluidDB” by Terry Jones was a run through of Tickery, a FluidDB application for analysing Twitter friend sets. Regular readers of this blog will know that I’ve been interested in FluidDB since attending an introductory talk by Nicholas Tollervey. Definitely worth a look.
Rounding out the end of the day were Andrew Godwin’s “Fun with Databases and Django” and Russel Winder’s “Pythonic Parallelism with CSP”. Andrew’s talk provided a general overview of some of the more advanced features of the Django ORM, including Django’s interaction with non-relational databases such as MongoDB and CouchDB. Meanwhile, Russel Winder is one of the key players in pushing Python support for concurrent and parallel programming. CSP has been around for over 30 years, and now Python programmers can begin to take advantage of the ideas thanks to projects like Python-CSP and PyCSP.
Wednesday
Wednesday kicked off with Nicholas Tollervey’s “Organise a Python Code Dojo!”. Nicholas shared his experiences running the London Code Dojo – the joys, the successes… and the things that didn’t quite go to plan.
“HotPy – A comparison” was Mark Shannon’s comparison between three different VMs for Python: PyPy, Unladen Swallow and his own HotPy project which runs on top of the GVMT.
Henrik Vendelbo gave two short talks in succession. The first was “Real Time Websites with Python” which gave a simple example of the Tornado web server, released as open source by FriendFeed. The second was “Custom Runtimes with PyPy”, a very interesting talk on using PyPy to provide an easy way to bundle up a custom Python application as a self-contained app.
qooxdoo is a full-featured JavaScript framework for developing Rich Internet Applications. It’s quite an impressive piece of work, not only because of the sheer size of the code base. Managing large code bases can be a headache, especially if you also need to process that source to produce more compact, optimised versions for delivery to specific browsers. In “Python for Javascript Apps”, Thomas Herchenröder explained how they use a set of Python tools to make dealing with the code a much more pleasant experience.
“A Python’s DNA” by Erik Groeneveld illustrated an issue I’ve hit myself: how to configure a complex component-based system in a simple, maintainable way. The answer Erik came up with for Meresco is not to use XML or similar to express configurations, but use Python instead to provide a rich, flexible, and elegant way to define dependencies, data flows and configurations.
It’s not often a talk begins by telling you that the subject of the talk is now officially dead, and a new project has taken over. Kay Schlühr’s “The Trails of EasyExtend” quickly changed to “The Trails of LangScape”. LangScape evolved from a toolkit for writing language extensions to Python, but takes a much broader view with the aim of allowing easy ways to extend many different languages.
Thursday
The morning began with “Building a python web app”. Anthony Theocharis and Nathan Wright took us through their journey to find the right Python web framework to implement MediaCore. They discussed Django, TurboGears and Pylons, illustrating their experiences with each and why they settled on Pylons. The different approaches to database access and templating languages were also examined in brief, followed by how they open sourced the project. An entertaining and interesting talk, refreshing by the fact they apparently came from a non-Python background so lacked any preconceptions.
Next up, Denis Bilenko introduced gevent in his talk “gevent network library”. gevent is a network library for handling large numbers of connections efficiently and, more importantly, elegantly. Optional, seamless monkey patching of standard library modules makes it very easy to work with existing code and libraries. I also discovered Gunicorn a Python implementation of Ruby’s Unicorn webserver.
Finally, my selection of conference talks closed with “Arduino and Python” by Michael Sparks. The talk should probably have been renamed “How not to blow up your computer” as Michael crammed in a quick intro to electricity and electronics into a few slides to dispel concerns about inexpensive pieces of electronics destroying your computer. The talk was illustrated by his prototype of a rather interesting TV remote control developed at a BBC R&D workshop, built with an Arduino… and Python of course.
Friday
Big thanks to Brett Cannon and Richard Jones for putting up with me all day while I got to grips with downloading, building, testing and patching Python 3.2. It was a really enjoyable day, and very satisfying to have a patch committed to the Python source code – no matter how small.
FluidDB Objects in 10 Minutes
July 14, 2010
When looking at FluidDB‘s tags (see FluidDB Tags in 10 Minutes), we briefly touched upon FOM‘s Object class (not to be confused with Python’s native object class). I instantiated an object representing the planet Mars:
>>> from fom.mapping import Object >>> mars = Object(about="planet:Mars") >>> mars <Object planet:Mars>
The thing to note is that I supplied the about keyword. By instantiating this way, I have no way of knowing if it existed before or not – the appropriate FluidDB object is created if one doesn’t exist already. In the big scheme of things it probably doesn’t matter, but it’s important to be aware.
The about keyword sets the value of a useful tag that is often assigned to objects when created: fluiddb/about. The about tag is used to label an object with something descriptive. It can be anything, but certain conventions are slowly evolving amongst FluidDB developers and users.
For example, an application I am developing uses the convention “band:<band name>” to label an object as representing a particular musical band. Nicholas J Radcliffe has written more on the subject on his About Tag blog which is a recommended read.
Let’s take a look at that tag:
>>> mars.get("fluiddb/about")
('planet:Mars', 'application/vnd.fluiddb.value+json')
An optional about tag is very useful but not all objects have, or indeed need, one. Fortunately, we can also refer to any object by its unique GUID which is guaranteed to exist:
>>> mars.uid '276da99e-c8d9-42c9-99ae-3db69a5e9ef0'
If I wanted to load an object by its GUID, I simply supply the uid parameter (it’s a good job I know the ID!):
>>> red_planet = Object( uid='276da99e-c8d9-42c9-99ae-3db69a5e9ef0' )
>>> red_planet
<Object planet:Mars>
>>> red_planet.get("fluiddb/about")
('planet:Mars', 'application/vnd.fluiddb.value+json')
Tags form the real power of FluidDB, so it’s important to be able to determine what tags have been added to an object. Let’s take our Mars example first:
>>> red_planet.tag_paths ['metaljoe/foo', 'fluiddb/about'] >>> red_planet.tags [<fom.mapping.Tag object at 0x57edb0>, <fom.mapping.Tag object at 0x57fd10>]
The first attribute stores the tag names, while the second returns these tags instantiated as FOM Tag objects. We can check they are indeed proper Tag instances by peeking at the descriptions:
>>> [ tag.description for tag in red_planet.tags ] ['behold: the foo tag', 'A description of what an object is about.']
I can also test for the presence of a tag by using the has() method:
>>> red_planet.has( "metaljoe/foo" ) True >>> red_planet.has( "the_mekon/likes" ) False
URLs were briefly mentioned in an earlier post about tags – each tag has a corresponding URL. FluidDB’s RESTful API means each object in the system also has a unique URL. For example, on the FluidDB sandbox instance our planet:Mars object can be referenced by the URL http://sandbox.fluidinfo.com/objects/276da99e-c8d9-42c9-99ae-3db69a5e9ef0 – pointing your browser to that URL returns a little bit of JSON:
{"tagPaths": ["metaljoe\/foo", "fluiddb\/about"]}
An object’s tags can also be referenced via URLs. For example, the fluiddb/about tag is referenced by the following URL:
http://sandbox.fluidinfo.com/objects/276da99e-c8d9-42c9-99ae-3db69a5e9ef0/fluiddb/about
Which yields the contents of the tag for that object:
"planet:Mars"
Note that a browser will be making GET requests to these resources. Being RESTful, we can make full use of different HTTP methods (GET, PUT, POST, DELETE) to perform reading, creating, updating and deleting of our FluidDB resources, but that’s a subject for another time.
EuroPython 2010 – Revised Talks of Interest
July 9, 2010
With about a week to go until EuroPython 2010, I realised the talks timetable has shifted around a bit so some clashes have been resolved, others created, and some talks have suddenly grabbed my interest.
I’m currently looking at the following:
Monday 19th July
- Idiomatic Python – Raymond Hettinger
- The Development Process of Python – Ezio Melotti
- How Import Works – Brett Cannon
- PostgreSQL’s Python soup – Hannu Krosing
- Django and the Semantic Web – Zachary Voase
- Python & Selenium Testing – Raymond Hettinger
- Muddle, inventing a build system – Richard Watts
- KBUS: A simple messaging system – Tony Ibbs
Tuesday 20th July
- Appstats – Guido van Rossum
- A case for accessibility – Stuart Bowness
- The Trojan Snake – Richard Barrett-Small
- Intro to Functional Programming – Michele Simionato
- Tickery, Pyjamas & FluidDB – Terry Jones
- Visualizing Software Quality – Mark Fink (clashes with Andrew’s talk)
- Fun with Databases and Django – Andrew Godwin (clashes with Mark’s talk)
- Pythonic Parallelism with CSP – Russel Winder
Wednesday 21st July
- Taming Twisted with Generators – Raymond Hettinger
- Adapting libraries to Twisted – Esteve Fernandez (clash!)
- Deferred Gratification – Terry Jones (clash!)
- Real Time Websites with Python – Henrik Vendelbo
- Custom Runtimes with PyPy – Henrik Vendelbo
- Python for Javascript Apps – Thomas Herchenröder
- A Python’s DNA – Erik Groeneveld
- The Trails of EasyExtend – Kay Schlühr
Thursday 22nd July
- Building a python web app – Stuart Bowness
- gevent network library – Denis Bilenko
- Arduino and Python – Michael Sparks
Look forward to seeing everyone there, new faces and old, and I’m in the market for some new Python challenges.
FluidDB Tags in 10 Minutes
July 7, 2010
So, with your namespaces structured it’s now time to add tags.
I’ll assume you already have a FluidDB session open. To add a tag to your current namespace:
>>> namespace.tag_names
[]
>>> namespace.create_tag( "foo", "behold: the foo tag", False )
<FluidResponse (201, 'application/json', None, {'id': 'cac795ce-777f-4d64-94cc-6df5356eb651', 'URI': 'http://fluiddb.fluidinfo.com/tags/metaljoe/foo'})>
>>> namespace.tag_names
['foo']
Okay, let’s take a look at that in more detail. The create_tag() method has three arguments: the tag name, the tag description and a mysterious boolean value at the end. The last value specifies whether the tag should be indexed – in this case, I have left it unindexed. At present, I’m not sure how the indexing works or if it gives any noticeable advantages at present. I should really make an effort to find out….
When my tag has been created, FOM returns a response from FluidDB. For those not fluent in HTTP, 201 is the Created response status: my tag was successfully created. You can also see that FluidDB has assigned my tag a unique GUID, and that the URI of my tag is http://fluiddb.fluidinfo.com/tags/metaljoe/foo – everything in FluidDB can be referenced by a URL, FluidDB’s RESTful API, and everything has a GUID.
>>> namespace.tag_paths ['metaljoe/foo']
Yup, that looks pretty conclusive.
Having a tag defined is one thing, the power of that tag comes when you apply it to objects in the system. Let’s find a suitable object first:
>>> from fom.mapping import Object >>> mars = Object(about="planet:Mars") >>> mars <Object planet:Mars>
Tagging the object is trivial:
>>> mars.set( "metaljoe/foo", None )
As is retrieving the value of that tag:
>>> mars.get( "metaljoe/foo" ) (None, 'application/vnd.fluiddb.value+json')
Now, setting the value to None is pretty dull. How about we set it to something more interesting?
>>> mars.set( "metaljoe/foo", "The red planet" )
>>> mars.get( "metaljoe/foo" )
('The red planet', 'application/vnd.fluiddb.value+json')
Actually, we might not want the value set in the system as a FluidDB JSON value. Let’s try setting it to the text/plain MIME type:
>>> mars.set( "metaljoe/foo", "The red planet", "text/plain" )
>>> mars.get( "metaljoe/foo" )
('The red planet', 'text/plain')
Python lists can also be stored easily enough:
>>> mars.set( "metaljoe/foo", ["The red planet", "My favourite planet"] ) >>> mars.get( "metaljoe/foo" ) (['My favourite planet', 'The red planet'], 'application/vnd.fluiddb.value+json')
Finally, tags have their own class in FOM:
>>> from fom.mapping import Tag >>> tag = Tag( "metaljoe/foo" ) >>> tag <fom.mapping.Tag object at 0x5b62d0> >>> tag.description 'behold: the foo tag'
FluidDB Namespaces in 10 Minutes
July 4, 2010
I’ve been working on a pet project with FluidDB recently and thought I’d post a few observations.
FluidDB is the core technology being developed by a company called FluidInfo. It’s in alpha test and the general idea is a central (I’m loathe to use the word “cloud”) database populated by objects. That’s objects as in things rather than the object oriented programming concept of object. Objects are tagged by users, and those tags define information and relationships between objects.
Each user can build up and tear down their selection of available tags, arranging these tags in a series of namespaces. For example, I have a namespace defined for an application I’m building:
metaljoe/music
That namespace consists of my root namespace (me, my user name) with a child namespace entitled music. There are no rules on how people structure their namespaces – I could easily call it metaljoe/aural or metaljoe/rhubarb if I wanted. As FluidDB evolves, conventions will naturally arise as people, hopefully, find certain naming styles preferable or popular. For the moment, anything goes.
First, I need to instantiate and bind my proxy to FluidDB. Using The excellent FOM library for Python: (please note that I am using the FluidDB sandbox rather than the live database)
>>> from fom.session import Fluid >>> fluid = Fluid( "http://sandbox.fluidinfo.com" ) >>> fluid.db.client.login( "metaljoe", <password> ) >>> fluid.bind()
I then retrieve my user’s namespace:
>>> from fom.mapping import Namespace >>> namespace = Namespace( u"metaljoe" )
To retrieve the list of child namespaces as either “leaf” names or full paths:
>>> namespace.namespace_names ['music'] >>> namespace.namespace_paths [u'metaljoe/music']
If I want to create a new namespace under the current one, I can do this…
>>> namespace.create_namespace( u"test", u"A test namespace" ) <fom.mapping.Namespace object at 0x743670>
…or if I decide to remove it:
>>> namespace.namespace_names ['music', 'test'] >>> namespace = Namespace( u"metaljoe/test" ) >>> namespace.delete() <FluidResponse (204, 'text/html', None, '')> >>> namespace = Namespace( u"metaljoe" ) >>> namespace.namespace_names ['music']
You’ll notice a little bit of FluidDB’s RESTful API shows through here, because we obtain a 204 (No Content) response code on a successful deletion.
As well as child namespaces, a namespace can have child tags:
>>> namespace = Namespace( u"metaljoe" ) >>> namespace.tag_names []
Hmm, okay, let’s find a better example:
>>> namespace = Namespace( u"metaljoe/music" ) >>> namespace.tag_names ['related_bands', 'source_url', 'band_name']
Or if I want the full path for the tags:
>>> namespace.tag_paths [u'metaljoe/music/related_bands', u'metaljoe/music/source_url', u'metaljoe/music/band_name']
I’ll cover tags another time.
So there we have it, a real quick tour of FluidDB namespaces – add, delete and view namespaces, and find out what child namespaces and tags are present.
