EuroPython 2010
July 24, 2010
Well, I’m back from another great EuroPython – my thanks to John Pinner and his EuroPython crew, the presenters, Josette from O’Reilly, and all my fellow delegates. Regrettably, I might not be able to make the next EuroPython in the wonderful city of Florence, but I might be able to make it to PyCon AU in Sydney next year. Fingers crossed on that one – I might even pluck up the courage to do a talk!
Monday
Monday’s talks kicked off with Raymond Hettinger’s “Idiomatic Python”. He guaranteed everyone would learn something new and he definitely delivered. I walked away with some interesting gems of Pythonic knowledge, from basic stuff like enumerate() through to more advancing knowledge like the surprising behaviour of super().
Ezio Melotti followed after the break with a useful overview of the Python development process, which inspired me to join the Python Core sprint on Friday. I’ve already contributed my first couple of patches to Python 3.2 – the first of which has been committed to trunk already.
“How Import Works” was a run through by Brett Cannon on how, you guessed it, Python imports code. Although you’ll probably never need to modify the standard import process, it’s fascinating to discover how it all works. Best of all, new utilities in Python 3 make it much easier to customise things safely if you ever need to.
“PostgreSQL’s Python Soup” was a little disappointing, with a good chunk of the talk being a general one about relational databases. The interesting stuff about connecting to PostgreSQL came quite near the end when we were introduced to the many different connectors. And I mean many.
Monstrum is a new HTTP functional testing framework for Python 3, and was detailed in “Testing HTTP Apps with Python3″. It looks very cool, and the logo would make a great t-shirt! Continuing the web testing theme, Raymond Hettinger followed up with “Python & Selenium Testing”. Alongside an overview of Selenium, Raymond introduced Sauce Labs, a company providing commercial support for Selenium, the web testing framework, via their cloud service.
To round off the day, I attended two talks by Richard Watts and Tony Ibbs of Kynesim who presented Muddle, their open source build system which looks very cool, and KBUS which is an elegant and lightweight messaging system implemented as a Linux kernel extension.
Tuesday
The day’s talks began with a rather packed presentation by Guido on AppStats, an AppEngine monitoring tool. I must admit that I did partially go to hear him talk, as did probably a lot of other people, but also because I was interested to learn more about AppEngine. I picked up some interesting bits of information about AppEngine internals, and the tool looks fantastic. However, I did feel a bit out of my depth.
Bart Demeulenaere’s “Pyradiso Rapid Software Development” was a call to arms, a request and discussion on how to create a rapid application development framework in Python that supports the multicore world out-of-the-box.
Engaging both my computer science and art historical sides, I then attended Richard Barrett-Small’s “The Trojan Snake”. Richard works for the Victoria & Albert Museum, and has been instrumental in introducing Python and Django as a way to clean-up and replace a mess of bespoke PHP, ASP.NET and Java. An interesting look at how an organisation faced issues with bespoke applications in a variety of technologies, and found Python to be a flexible and effective solution. It’s a shame more people didn’t attend.
I was an avid reader of Michele Simionato’s blog posts, covering his experiences with Scheme, so it was great to hear him speak at EuroPython with his talk entitled “Intro to Functional Programming”. Functional programming is receiving a surge of interest amongst programmers, thanks to the search for better ways to deal with concurrency, so it was good for someone to cover other reasons to start exploring FP.
“Tickery, Pyjamas & FluidDB” by Terry Jones was a run through of Tickery, a FluidDB application for analysing Twitter friend sets. Regular readers of this blog will know that I’ve been interested in FluidDB since attending an introductory talk by Nicholas Tollervey. Definitely worth a look.
Rounding out the end of the day were Andrew Godwin’s “Fun with Databases and Django” and Russel Winder’s “Pythonic Parallelism with CSP”. Andrew’s talk provided a general overview of some of the more advanced features of the Django ORM, including Django’s interaction with non-relational databases such as MongoDB and CouchDB. Meanwhile, Russel Winder is one of the key players in pushing Python support for concurrent and parallel programming. CSP has been around for over 30 years, and now Python programmers can begin to take advantage of the ideas thanks to projects like Python-CSP and PyCSP.
Wednesday
Wednesday kicked off with Nicholas Tollervey’s “Organise a Python Code Dojo!”. Nicholas shared his experiences running the London Code Dojo – the joys, the successes… and the things that didn’t quite go to plan.
“HotPy – A comparison” was Mark Shannon’s comparison between three different VMs for Python: PyPy, Unladen Swallow and his own HotPy project which runs on top of the GVMT.
Henrik Vendelbo gave two short talks in succession. The first was “Real Time Websites with Python” which gave a simple example of the Tornado web server, released as open source by FriendFeed. The second was “Custom Runtimes with PyPy”, a very interesting talk on using PyPy to provide an easy way to bundle up a custom Python application as a self-contained app.
qooxdoo is a full-featured JavaScript framework for developing Rich Internet Applications. It’s quite an impressive piece of work, not only because of the sheer size of the code base. Managing large code bases can be a headache, especially if you also need to process that source to produce more compact, optimised versions for delivery to specific browsers. In “Python for Javascript Apps”, Thomas Herchenröder explained how they use a set of Python tools to make dealing with the code a much more pleasant experience.
“A Python’s DNA” by Erik Groeneveld illustrated an issue I’ve hit myself: how to configure a complex component-based system in a simple, maintainable way. The answer Erik came up with for Meresco is not to use XML or similar to express configurations, but use Python instead to provide a rich, flexible, and elegant way to define dependencies, data flows and configurations.
It’s not often a talk begins by telling you that the subject of the talk is now officially dead, and a new project has taken over. Kay Schlühr’s “The Trails of EasyExtend” quickly changed to “The Trails of LangScape”. LangScape evolved from a toolkit for writing language extensions to Python, but takes a much broader view with the aim of allowing easy ways to extend many different languages.
Thursday
The morning began with “Building a python web app”. Anthony Theocharis and Nathan Wright took us through their journey to find the right Python web framework to implement MediaCore. They discussed Django, TurboGears and Pylons, illustrating their experiences with each and why they settled on Pylons. The different approaches to database access and templating languages were also examined in brief, followed by how they open sourced the project. An entertaining and interesting talk, refreshing by the fact they apparently came from a non-Python background so lacked any preconceptions.
Next up, Denis Bilenko introduced gevent in his talk “gevent network library”. gevent is a network library for handling large numbers of connections efficiently and, more importantly, elegantly. Optional, seamless monkey patching of standard library modules makes it very easy to work with existing code and libraries. I also discovered Gunicorn a Python implementation of Ruby’s Unicorn webserver.
Finally, my selection of conference talks closed with “Arduino and Python” by Michael Sparks. The talk should probably have been renamed “How not to blow up your computer” as Michael crammed in a quick intro to electricity and electronics into a few slides to dispel concerns about inexpensive pieces of electronics destroying your computer. The talk was illustrated by his prototype of a rather interesting TV remote control developed at a BBC R&D workshop, built with an Arduino… and Python of course.
Friday
Big thanks to Brett Cannon and Richard Jones for putting up with me all day while I got to grips with downloading, building, testing and patching Python 3.2. It was a really enjoyable day, and very satisfying to have a patch committed to the Python source code – no matter how small.
FluidDB Namespaces in 10 Minutes
July 4, 2010
I’ve been working on a pet project with FluidDB recently and thought I’d post a few observations.
FluidDB is the core technology being developed by a company called FluidInfo. It’s in alpha test and the general idea is a central (I’m loathe to use the word “cloud”) database populated by objects. That’s objects as in things rather than the object oriented programming concept of object. Objects are tagged by users, and those tags define information and relationships between objects.
Each user can build up and tear down their selection of available tags, arranging these tags in a series of namespaces. For example, I have a namespace defined for an application I’m building:
metaljoe/music
That namespace consists of my root namespace (me, my user name) with a child namespace entitled music. There are no rules on how people structure their namespaces – I could easily call it metaljoe/aural or metaljoe/rhubarb if I wanted. As FluidDB evolves, conventions will naturally arise as people, hopefully, find certain naming styles preferable or popular. For the moment, anything goes.
First, I need to instantiate and bind my proxy to FluidDB. Using The excellent FOM library for Python: (please note that I am using the FluidDB sandbox rather than the live database)
>>> from fom.session import Fluid >>> fluid = Fluid( "http://sandbox.fluidinfo.com" ) >>> fluid.db.client.login( "metaljoe", <password> ) >>> fluid.bind()
I then retrieve my user’s namespace:
>>> from fom.mapping import Namespace >>> namespace = Namespace( u"metaljoe" )
To retrieve the list of child namespaces as either “leaf” names or full paths:
>>> namespace.namespace_names ['music'] >>> namespace.namespace_paths [u'metaljoe/music']
If I want to create a new namespace under the current one, I can do this…
>>> namespace.create_namespace( u"test", u"A test namespace" ) <fom.mapping.Namespace object at 0x743670>
…or if I decide to remove it:
>>> namespace.namespace_names ['music', 'test'] >>> namespace = Namespace( u"metaljoe/test" ) >>> namespace.delete() <FluidResponse (204, 'text/html', None, '')> >>> namespace = Namespace( u"metaljoe" ) >>> namespace.namespace_names ['music']
You’ll notice a little bit of FluidDB’s RESTful API shows through here, because we obtain a 204 (No Content) response code on a successful deletion.
As well as child namespaces, a namespace can have child tags:
>>> namespace = Namespace( u"metaljoe" ) >>> namespace.tag_names []
Hmm, okay, let’s find a better example:
>>> namespace = Namespace( u"metaljoe/music" ) >>> namespace.tag_names ['related_bands', 'source_url', 'band_name']
Or if I want the full path for the tags:
>>> namespace.tag_paths [u'metaljoe/music/related_bands', u'metaljoe/music/source_url', u'metaljoe/music/band_name']
I’ll cover tags another time.
So there we have it, a real quick tour of FluidDB namespaces – add, delete and view namespaces, and find out what child namespaces and tags are present.
Why I Picked Mercurial
February 28, 2010
I was a little dubious about the hype over distributed version control until I bought Jon Loeliger’s excellent “Version Control with Git” at EuroPython last year. I was sold – it made sense. I’d used both Git and Mercurial a little before reading the book, but more out of necessity than design to be able to checkout fresh code from a couple of different open source projects. Like many people, I didn’t “get” DVCS at first – how could it possibly work in practice??? (The answer is: very well, thank you)
Deciding your choice of distributed version control is as much down to personal preference as your choice of editor, programming language, or operating system. Git impressed me by its massive range of tools, good Subversion interaction, pragmatic approach and the fact it’s great to drop the word “Git” into polite conversation. I have a lot of respect for Linus Torvalds and his pragmatic approach to development, and his decision-making process in crafting Git to make kernel development easier. The problem was that with great power comes great responsibility and, more importantly, complexity.
For personal development, I drifted towards Mercurial because it seemed to fit my development process much more comfortably, and had a slightly more elegant feel. The learning transition from Subversion to Mercurial was small compared to the transition from Subversion to Git. However, Subversion repository interaction from Mercurial is not so mature. Go figure. Despite that, Mercurial makes a great replacement for Subversion, eliminating all the niggles and problems of the latter. It feels more lightweight than Git, yet packs all the power and flexibility I need for the kinds of projects I work on.
Mercurial is written in Python, which is useful but not an essential reason, and very extensible. It works well on multiple platforms, while Git seems to be fine on my home OS environments (OS X, Fedora) but less developed for Windows which I have to use at work. Mercurial Queues are a fantastic idea that I really wish I’d had access to on several occasions in the past – they would’ve saved me so much time and pain. History can’t be altered, like in Git, although I do think the feature has its uses – just that it can also have its abuses. Mercurial seems to handle renames in a more natural way to me. HTTP-based sharing in Mercurial is pretty good, and efficient too.
Don’t get me wrong. I think Git is a fantastic piece of version control technology, and GitHub rocks as a supportive environment, but Mercurial has the edge for me. It rocks, and development is improving it at a respectable pace.
If you haven’t already seen Joel Spolsky’s Mercurial tutorial that has been doing the rounds, you can find it at: http://hginit.com/ – it’s the best introduction I’ve seen so far.
If you haven’t switched to distributed version control, take a look at Mercurial or Git (or Bazaar, if that way inclined) because it really is a significant evolution over centralised systems like CVS and SVN.
Wikis
August 28, 2009
If you’ve not seen a Wiki, where have you been??? Thanks to projects like Wikipedia, the wiki has entered mainstream culture and deservedly so. They’re an amazingly simple, but incredibly powerful, collaborative tool and my thanks go to Ward Cunningham, creator of WikiWikiWeb in the mid-90s.
My first experience of a wiki was in something like 2000. I’d like to say it had an impact on me as deep as my first use of the World Wide Web in 1994, but it didn’t. It was more subtle, more natural than that – it felt like the next logical step from web pages, rather than some huge evolution.
I have a confession to make: I actually like writing documentation. I don’t always get the time to write as much documentation as I would like, but I don’t find it the chore some developers do. The main problem is that documentation gets stale very quickly, and when I write I often want to branch off at a tangent or start in the middle. A bit like the Web, really.
Wikis reduce the barriers to collaboration, which is why they appeal to me, and make it easier to keep content fresh. Rather than those horrible documentation systems that lock files and force you to deal with file format issues like whether you’re using a particular version of Word or OpenOffice, wikis are open through the “standard” interface of a web browser. Using a simple and usually intuitive markup language, you can begin adding and updating content quickly. Look how successful Wikipedia has become, because people can contribute as much or as little as they want in order to collaboratively build and refine the pool of knowledge.
I love the fact our wiki at work has similarly built up over time into a repository of all kinds of useful information. It’s so easy for people to contribute tiny changes and fixes, reorganise information and create cross-references. Best of all, a lot of the information is overkill for a formal document and thus would be stored in people’s heads or even lost. I fear the day when someone who doesn’t understand the power of wikis forces us to change to the proprietary, locked-down and inflexible world of something like Sharepoint.
MoinMoin is my preferred wiki software at the moment. It’s open source, written in Python and provides more than enough features to cover my needs at home and at work. Our office wiki is powered by MoinMoin as well, and I’ve installed personal copies on my home machines to act like digital whiteboards. Using a wiki, I can keep project notes, build up documentation and organise my To Do lists easily.
So it may come as a surprise to find that I haven’t used wikis for one thing yet: testing! Fitnesse uses a wiki front-end to build Fit-style acceptance tests. It sounds like it could be ideal for me because I want to start researching better ways to undertake acceptance testing. It’s on my, wiki-based, To Do list.
Perhaps a subject for a future post…
Useful resources:
