Django Templates and Emails
October 22, 2009
A Django application I’ve been working on sends a confirmation email near the end of a workflow. The content of that email closely matches the final web page of the workflow. Constructing an email in Django is pretty straightforward, and using templates makes the process a lot easier and more flexible.
My naive initial approach was to construct my HTML in a string, using Python’s string formatting, and then use the string as the body of the email. That was a reasonable starting point, but it sucked because it’s messy, it duplicates functionality and frankly it’s a waste of the power Django has to offer.
The next step was to harness templates to generate my content for the email. I actually need an email with both HTML and plain text content, and I already mentioned that the content is reasonably close to that used elsewhere, so there’s plenty of potential for code reuse.
The django.template.loader module contains the handy render_to_string function which, surprise, allows us to render our template to a string. It parallels the more common render_to_response function found in your view code, so I can supply a template name and a context in the same way I would for my view code: (contrived example follows)
from django.template.loader import render_to_response html_content = render_to_string( "my_template.html", my_context )
I’ll go into that HTML content shortly, but suffice to say the template can reuse snippets of template code from the rest of my application. I can therefore create reusable content that is embeddable in both my web page and email body. Likewise, I can generate a text version of the body, though reusing HTML snippets is obviously not so easy or even appropriate:
text_content = render_to_string( "my_template.txt", my_context )
I can then take these strings and use them to construct my email:
from django.core.email import EmailMultiAlternatives
my_mail = EmailMultiAlternatives( subject="Foo Confirmed",
body=text_content,
from_email="test@foo.bar",
to=[ "recipient@foo.bar" ]
)
my_mail.attach_alternative( html_content, "text/html" )
Now, back to the HTML content. There are a few things to be aware of with HTML in email:
- Keep the <head></head> section empty as the content is likely to be ignored and could even be stripped out
- Use a <style></style> section after your <body> tag to inline your CSS code
- Don’t use JavaScript (do you really need to be told about this?)
- Inline references to external resources like images are likely to be blocked
Think of the HTML in your email body as needing to be a self-contained environment. I’ve included some online resources at the end of this post which will give you more in-depth information.
That’s fine for CSS, but how do you add inline images to your emails? The answer is to create MIMEImage instances of any images and attach to your email with appropriate Content-ID and Content-Disposition headers. There’s a very simple example at http://www.djangosnippets.org/snippets/1507/ that worked nicely for me.
Hope the above is a useful starting point!
Further reading:
- http://docs.djangoproject.com/en/dev/ref/templates/api/
- http://docs.djangoproject.com/en/dev/topics/email/
- http://www.djangosnippets.org/snippets/1507/
- http://articles.sitepoint.com/article/code-html-email-newsletters
- http://www.anandgraves.com/html-email-guide
- http://www.campaignmonitor.com/templates/
Testing Django Emails
October 18, 2009
In the past, my experience testing the sending of emails from web frameworks had not been a particularly smooth or easy one. There was often a bit of hackery involved, and attempting to do so from a test harness of some sort was often tricky. Fortunately, testing emails from Django (1.0+) test cases is pretty straightforward and very pleasant to use.
When you run the Django testrunner, the runner will temporarily override the SMTPConnection class. Any emails sent from code called by the testrunner will not be sent out, but captured and stored for analysis within your tests.
Let’s say I have some code that will send out a notification email, and I want to test that it works. I could have a test case along the lines of:
class NotificationEmailTest( TestCase ): def test_send_notification_email( self ): send_notification_email( "Foo!" ) ## test goes here
But how do I check the email was sent? Well, the overridden SMTPConnection stores emails in the django.core.mail.outbox list. I can therefore check to see if I have one email message in the outbox. This outbox is only available within tests, it does not exist in normal Django operation.
First, I need an import so I can access the outbox:
from django.core import mail
Then I have to revise the above test to include my assertion:
class NotificationEmailTest( TestCase ): def test_send_notification_email( self ): """ Test send_notification_email sends out one email """ send_notification_email( "Foo!" ) self.assertEqual( len(mail.outbox), 1 )
The outbox is guaranteed to be empty at the start of each test, so you don’t need to do anything in terms of setup or teardown.
Testing presence of an email is fine, but we normally want to test the content of a message (otherwise it could be junk!). Each item in the outbox is an instance of EmailMessage, with all the expected attributes. If we wanted to add a second assertion to the above test, say to check the subject, we could do so:
class NotificationEmailTest( TestCase ): def test_send_notification_email( self ): """ Test send_notification_email sends out one email """ send_notification_email( "Foo!" ) self.assertEqual( len(mail.outbox), 1 ) self.assertEqual( mail.outbox[0].subject, "Foo!" )
In this case, the single parameter for send_notification_email is actually the subject of the email, so my test checks to see if this matches.
When testing the contents of to and bcc, there is apparently no cc, be aware these are tuples of addresses when writing your tests.
class NotificationEmailTest( TestCase ): def test_send_notification_email( self ): """ Test send_notification_email sends out one email """ send_notification_email( "Foo!" ) self.assertEqual( len(mail.outbox), 1 ) self.assertEqual( mail.outbox[0].to, [ "test@foo.bar" ] )
This is fine if we have one recipient, but if our recipient list is going to have multiple addresses we need to allow for this in our tests. If we knew the exact ordering we could compare to a list but, unless the ordering matters, we end up making the test fragile. Instead, we can test for the presence of email addresses individually:
self.assertTrue( "test@foo.bar" in mail.outbox[0].to ) self.assertTrue( "wibble@amiga.foo" in mail.outbox[0].to )
Django makes testing of emails easy, so there’s no excuse for not incorporating email tests into your Django test suites.
Hope that gives you a starting point. Have fun!
Further reading:
Nose-Driven Development
September 15, 2009
One of the best phrases I’ve encountered in my professional programming career is “smelly code”. It’s great: it sounds childish enough to make me smile, yet also highlights an important indicator in code quality.
Like many, I first encountered the concept of code smells in Martin Fowler’s essential “Refactoring” book, although the term was coined by Kent Beck and predates the book. We’ve all seen examples, and even contributed our own aroma to code. Code duplication; excessively large classes, methods, functions, modules; unreadable code; tightly-coupled code; fragile code; overly complex code; code that circumvents encapsulation.
Over the years, I’ve picked up a nose for code smells. I get a feel for code that is starting to smell, and usually get that prick of guilt when my own code begins to emit an unpleasant fragrance, even if I don’t always know immediately how to deal with it. Deodorant comments like “# REFACTOR: this feels wrong” or “// REFACTOR: this code is ugly!” start to crop up as a warning sign that things need to be looked at, if immediate action can’t be taken.
The nose kicked in this afternoon while working on a Django view. Many smells start off with the best of intentions, and this was no exception. In order to make my tests pass, I started off with a fairly harmless bit of code that interacted with a form object. Over a period of a few tests, the code began to expand and some duplication kicked in. List comprehensions entered into the equation. The nose began to kick in. I can’t show the code, unfortunately, and I’m too lazy to concoct some example code that illustrates my point, so bear with me…
Once all the functionality I needed had been implemented and the code checked-in, I cast an eye over the code and realised it looked ugly. The warning signs were there:
- two almost-identical list comprehensions
- temporary variables used to make the list comprehensions more readable
- several lines to do something which should be simple
- the need to re-read the code more than once to “see” what it does
*facepalm* What was I thinking when I wrote that??
First step was to extract the code into a utility function, in the hope of making the view code more readable and make it easier to eliminate the duplication. I find extracting code can be an excellent first step in dealing with ugly code, but it has to be used responsibly: otherwise you end up moving the smells into another layer of abstraction.
In the process of creating the function, another smell became apparent:
- borderline intimate knowledge of the form class
This smell instantly pointed out where and how to refactor the code. The extraction was still necessary, but instead of creating a utility function in the view it was apparent that the code could be moved into two methods of the form. A quick change and a re-run of the tests confirmed that the extraction worked and the view instantly looked simpler and more readable. However, the new methods still didn’t quite feel right.
Next step was to take the list comprehensions and break them out into multi-line for loops. It might sound counter-intuitive to take code that occupies one line (albeit with preparatory code) and turn it into four or five lines of code, but the result was for the best. The intent of the code became clearer and the need for those temporary variables was removed.
All that remains is a minor whiff, a single bit of hardcoding used to identify particular fields in the form. My nose is telling me that introspection will lead me down the path of perfumed code…
Further reading:
- http://c2.com/xp/CodeSmell.html
- http://en.wikipedia.org/wiki/Code_smell
- http://wiki.java.net/bin/view/People/SmellsToRefactorings
Also of note:
(I’ve not tried this yet – just found it!)
Erlang In My Head (Part 2) – String Reversal, The Educational Way
September 12, 2009
This is the second in an irregular series of musings on Erlang, with references to Python. The main aim is to encourage me to use spare moments in the evening to look at aspects of Erlang. Hopefully my murmurings will be of interest to fellow Pythonistas (and maybe others!) as well as generate feedback from the Erlang community.
Spare time is a scarce commodity at the moment, so I decided the best way to get back into learning and using Erlang is to pick small, often trivial, tasks that can be solved in a few minutes, but which would hopefully yield enlightenment on a facet or two of the language.
One example is that contrived task of reversing a string. Both Python and Erlang have efficient in-built or library methods/functions for reversing lists and strings, but calling something like my_string.reverse() or lists:reverse() isn’t very educational.
In Python, one quick’n'dirty solution would be:
def reverse_string(my_string):
new_string = ""
for character in my_string:
new_string = character + new_strin
return new_string
I could also dust off my computer science skills and approach the problem recursively, without the need for new_string:
def reverse_string(my_string):
if my_string:
return reverse_string(my_string[1:]) + my_string[0]
return ""
It works, but it doesn’t look as tidy as the first function. There’s also a limit on the depth of recursion, the CPython default of which (I believe) would restrict string size to about 1000 characters. Not a scalable solution.
Approaching the problem in Erlang using the first, iterative, approach is hampered by the fact I can only assign a variable to a value once. It might be possible to do it iteratively, but it would probably look as awkward as the recursive approach felt in Python.
Erlang treats strings as lists of numbers. While this means string handling is not as natural as it is in Python, it does offer us some help when approaching the reversal problem. My string is a list, and I can adopt the same approach as I did for the recursive Python function: append the head of the list to a reversed copy of the tail.
reverse_string([Head|Tail]) ->
reverse_string(Tail) ++ [Head];
reverse_string([]) ->
[].
My reverse_string function has two clauses. The first clause takes a list and splits it into a Head (the first element) and a Tail (everything else). It then concatenates the result of reverse_string(Tail) to the Head. Simple!
Or not. My first attempt suffered a gotcha: I was concatenating a list to an integer, which errored. Turning Head into a single item list, [Head], allowed the two to be concatenated.
The second clause deals with the case of an empty string. When you reach the end of the string your Tail will be empty, which matches this clause and ends our recursion. If you supplied an empty string initially, this would also match. Using an empty string “” instead of [] works equally as well, but I decided to keep things consistent with the list notation used in the first clause.
With this exercise complete, I’ll stick to lists:reverse in future.
Update: See http://metaljoe.wordpress.com/2009/09/13/erlang-in-my-head-part-2-1-feedback/
Erlang In My Head (Part 1)
September 8, 2009
Coming from an imperative programming background, one of the earliest things to note about Erlang is that variables don’t actually vary. Erlang allows you to assign a value to a variable once, and that’s it: the assignment is permanent. I like to think of it as going from the unknown to the known, or unbound to bound if you want to be precise.
In Erlang:
1> X. * 1: variable 'X' is unbound
Which isn’t any different to Python:
>>> X Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'X' is not defined
In both cases X has an unknown value, it isn’t bound to anything, so Erlang/Python doesn’t know what to do with it.
Thus we need to assign a value to X:
2> X = 42. 42 3> X. 42
Now Erlang is happy, and so is Python:
>>> X = 42 >>> X 42
However, that = sign has a different meaning in the two languages. In Erlang, it’s not an assignment operator but a pattern matching operator. Erlang takes a look at both sides of the “equation” and attempts to make sense of it. Since X is unknown up to that point, the only way it can reconcile the statement is to assign X the value of 42. 42 equals 42, and the world is at one with itself again.
Python takes the value 42 and applies the label X to it. I could easily move the X label to something else:
>>> X = "foo" >>> X 'foo'
What I can’t do is the same with Erlang:
4> X = "foo". ** exception error: no match of right hand side value "foo"
Erlang, quite rightly, complains because it knows that X has the value 42 (since we bound it to that earlier) and this definitely doesn’t match the string “foo”. If I try it against the value 42, I get the value 42 i.e. a match.
5> X = 42. 42
In terms of naming, variables in Erlang must begin with a capital letter, not lowercase:
6> x = 7. ** exception error: no match of right hand side value 7
The reason this failed is that Erlang has something called an “atom”, and atoms start with lowercase letters. An atom is effectively a constant that has a value of itself: it’s pre-bound, so you can’t assign it to anything.
7> x. x
The atom x is equal to x, which is why Erlang raised an exception when x was compared to 7.
Why would you want variables that can’t change? Well, for those of you who have done any concurrent programming you will know how tricky it is dealing with shared data. Functional programming languages like Erlang remove a whole class of concurrency-related problems by making data immutable. If the data is immutable, you can’t have one thread or process changing the contents of a variable while another is looking at it. Nor can a process overwrite the changes another process has just made. That means no need for locks and semaphores to protect data integrity, all of which have performance implications.
Another thing it makes easier is debugging. Once a variable is set, it stays set – defective code can’t update a variable it shouldn’t. You can also more easily prove an algorithm, and its implementation, is correct when you know that code is free of side-effects.
If you’re a Pythonista looking at this and haven’t given Erlang a go, I can recommend it as a way to try something different and expand the mind. You might even gain a fresh perspective of the way you write Python. If you’re already an Erlang programmer, I hope my understanding of the above is correct and any feedback is always welcome!
10 Minutes With Django’s TestCase
September 1, 2009
I’ve been playing with Django for a while now, but today was the first day writing proper production code which will end up on a public-facing website. It will be the first Django application in use by the company, so there’s a lot riding on getting it right.
Django is great for writing web applications quickly and easily – whenever I hit upon a problem, I find something in Django that already does what I need, or I can implement something small in Python that fills in the gap.
One really nice feature is that it’s easy to test Django using both of Python’s in-built testing frameworks: unittest and doctest. I’m not a fan of doctests, though I can see why some people find them useful. I do love unittest though, and so it’s great that Django has provided enhancements to unittest to support things like basic browser client tests, loading test data fixtures and using a test database (sqlite is great for this).
The client doesn’t aim to replace dedicated web test systems such as Selenium, Windmill or Twill. It just offers a lightweight way to test your Django application’s functionality out-of-the-box. You can check response codes, test which template is being rendered, fire test data to forms, or check for text in returned pages. This was very useful today when fleshing out the first part of the application: a form with various fields, buttons, drop-downs and validation logic. I could write my tests first, a bit at a time, then implement the functionality to pass the tests. Supported by the confidence given by the tests, I could check the specified logic was correct, keep refactoring the code, and keep the design supple at the same time.
Of course, I will eventually need to invest in dedicated web testing, probably Selenium, but for the moment the Django TestCase gives me a very quick and flexible way to check functionality while things are changing frequently. Django rocks!
Further reading:
Tech Podcasts I Love
August 30, 2009
The first tech podcast I started listening to was LUG Radio, sadly demised. The second one was This Week In Django, which has been dormant since the beginning of the year. I worry that I jinx podcasts.
Podcasts give me a great way to keep up with what’s happening in areas I’m interested in. I sometimes listen to them at work, but usually in the evenings when I can listen while doing other things. My current list of tech-related podcasts is as follows (in order of appearance in my freedom-hating iTunes podcast list):
Distrocast – http://distrocast.org/
JD and Jeremy cast their highly critical gaze over various Linux distributions, and other things too. I love the fact nothing is sacred and they’re not afraid to rip things to shreds and hurl abuse. Not one for the easily offended or the “Linux and Open Source is Always Perfect” brigade. A much needed contribution to the Linux community. They make me chuckle, they make me think, and they introduced me to ArchLinux, which is a pretty cool distro.
FLOSS Weekly – http://www.twit.tv/FLOSS
Randal Schwartz, Leo Laporte, and guest appearances from the mighty Jono Bacon. FLOSS weekly takes a look at different free, libre open source projects. Always entertaining and informative, it’s great to hear more about projects I use or have heard of, as well as offering an excellent way to discover new projects.
The MDN Show – http://www.mac-developer-network.com/category/shows/
Formerly Late Night Cocoa, which ended up becoming a paid-for service. Scotty and John discuss various aspects of Mac development in an easy to listen format, albeit with some very cheesy music. The supporting tips from “The World According to Gemmell” are always small, thought-provoking chunks of advice that are great to try out, whether you’re a Mac developer or not.
Linux Outlaws – http://www.linuxoutlaws.com/
Fab and Dan’s show has proved a more than capable replacement to LUG Radio, covering Linux and free open source software… with the odd beer review, creative commons music, and Fail. Linux Outlaws is laid-back, light hearted and always interesting, packing a lot into each show.
MacCast – http://www.maccast.com/
Adam Christianson’s enthusiasm for all things Macintosh really shines through. MacCast is a great way to keep up-to-date with the latest news from Apple and the Mac community.
.NET Rocks – http://www.dotnetrocks.com/
I had a bit of trouble finding a decent .NET podcast when I realised I needed to start delving into the murky world of Microsoft and .NET development. Carl and Richard were my salvation. A slick podcast which covers the breadth and depth of .NET, it’s the perfect way for me to keep an eye on the .NET ecosystem. If you’re used to the world of open source and UNIX development, I’d recommend giving this show a go to see things from the other side of the fence. You’ll learn some new things and maybe appreciate what you’ve got a bit more.
Pragmatic Podcasts – http://www.pragprog.com/podcasts
Irregular, short podcasts from the excellent Pragmatic Programmers publishing house. The podcasts involve interviews with authors and are usually linked to new releases – it’s good PR because I’ve ended up checking out books based on the interviews.
Python411 – http://www.awaretek.com/python/
I’m not sure how to categorise Python411, produced by Ron Stephens. Since the dormancy of This Week in Django, it’s the only active Python-related podcast, albeit irregularly. It has a pleasantly cosy and informal feel to it, and there are some useful gems of information to be found in the episodes.
Sod This – http://www.sodthis.com/podcast/
Billed as “Brain Burps for the Tech Savvy”, Gary and Oliver’s podcast is another one that’s difficult to categorise as it’s relatively new and quite an eclectic experience. I can’t write much more than that, but it’s worth a listen for things such as an interview with IronPython guru Michael Foord or the Women In Technology episode.
The Software Freedom Law Show – http://www.softwarefreedom.org/podcast/
Bradley and Karen from the Software Freedom Law Center cover the legal side of the open source community. If you’re an open source software developer, or a developer interested in the issues surrounding patents, copyright and intellectual property, I can highly recommend this podcast. It’s very accessible so you don’t need to be a lawyer to understand it.
TuxRadar Linux Podcast – http://www.tuxradar.com/podcast
Not as good as Linux Outlaws or LUG Radio, sorry guys, but still a worthy addition to anyone’s Linux podcast collection. News and chat on Linux and open source software from the Linux Format crew.
A VerySpatial Podcast – http://veryspatial.com/avsp/
I was umming and ahhing about adding this, but it is tech-related. A Very Spatial Podcast is the long-running weekly show covering geography and geospatial technology. Being quite new to GIS, and then only currently dealing with very basic geocoding of data, parts of the show can be a little indecipherable to my newbie mind. It’s well presented and covers a good range of topics – I might not understand everything, but it’s helping me to learn fast. If you have any interest in the geospatial technology, professional or amateur, check it out.
Five Languages That Rocked My World
August 23, 2009
I’ve been fortunate enough to have used a variety of different programming languages over the years. Some for home projects, some for use at work, some just to tinker with to see what the fuss is about. Five languages in particular have had a big effect on me:
BASIC
Pretty much everyone who cut their programming teeth on 8-bit micros started off on BASIC. You switched the computer on and there you were: an inviting command prompt waiting for you to type in some BASIC code. No wait, no fuss. We’d laboriously type out listings from computer mags, manuals supplied with the computer or the old Usborne books. A great experience for learning about computers and programming which I worry is lacking these days. It encouraged you to play around and experiment, and the environment BASIC was running on was fairly limited so it was easy to understand the system as a whole. Maybe I’m intertwining BASIC and 8-bit computers?
BASIC was my introduction to programming. I’d sit there hunched over my CBM Plus/4 hooked up to a blurry TV and bash out programs that were crude and contrived, but were fun to write and gave a real sense of achievement. Even when I moved to an Amiga A500, I continued to use BASIC (in the form of the much maligned AmigaBASIC) for a long time because it was quick and easy to write software that met my needs. I remember writing more serious tools on the Amiga, including code for things like GCSE maths coursework.
BASIC might be frowned upon by many these days, but it remains a decent language for introducing people to the general idea of programming – which was its original purpose. The chapter on BASIC in the excellent O’Reilly “Masterminds of Programming” book is a terrific read, if you like that sort of thing.
Modula-2
From BASIC, I went towards assembly language (6502 and then 68000) and C but was never particularly serious about either at the time. When I started university, the course taught Modula-2 as the language used to express the concepts of programming properly, and the serious art of computer science (sic). C might’ve been more powerful, more lower-level and more widely supported, but it had sharp edges that could catch the unwary, including me.
Modula-2 was a revelation and it helped move me completely away from BASIC. It retained a clear, readable syntax and I found it great for writing structured, robust code thanks to features such as its use of modules for encapsulating data and procedures. There was even support for coroutines, something that at the time was a relatively specialised feature for general microcomputer programming.
Prolog
I’d heard of Prolog back in the 80s and knew that it stood for Programming in Logic (“programmation en logique” if you want to be correct), but that was the extent of my knowledge. I didn’t encounter it properly until late in my university course when I began to study logic programming and artificial intelligence. It was the first time I’d encountered a language that was very different from the imperative programming I had undertaken before.
Prolog is a declarative language. Instead of writing code as a sequence of steps and flow control statements, i.e the “how” of solving a problem, you declare the problem space itself and the rules defining that space. It’s a very different approach to tackling the writing of code in languages like C and was an excellent mental exercise for myself. It had a big effect on the way I looked at programming and its influence can be seen in my fascination with things like domain specific languages or my recent look at Erlang.
Perl
Perl was in some ways a return to my roots and in others a radical departure. I learnt Perl in a weekend when I started at my first job in 1997. There were some CGI scripts that needed maintaining and they were written in Perl, a language no one at the company had any in-depth knowledge of.
For all the jokes about being a write-only language, which admittedly can be very true, and it being the Swiss army chainsaw of text utilities, it’s a pretty powerful language. Perl, to me, was always about getting the job done. The lack of compilation gave the ability to make quick changes and experiments easily, just like BASIC. It excelled with text manipulation and the support for regular expressions turned me into a serious regex fanatic. I find regular expressions incredibly useful and have felt comfortable writing and debugging some insanely complex pattern matching in other languages as a result.
Perl gave me a great stepping stone into developing dynamic web sites (as opposed to static HTML) as well as dealing with the processing of large amounts of text. It didn’t occur to me at the time, but it was also the origins of my love for dynamic programming languages. I might not have an interest in working in Perl again, but I’m glad I spent the first few years of my professional programming career using it.
Python
Ah, Python. Truly the best all-round language I have used to date – dynamically typed, supports quick experiments as well as large-scale software development, clean syntax, simple to learn, object-oriented, with a few nods to functional programming in places. It also taught me that life without braces or begin/end statements is not only possible, but actually quite liberating. Python is fun.
Once I’d got past the whole indentation weirdness, Python proved to be an excellent replacement for Perl. I could do everything I did in Perl easily, and more. It also proved to be the first object-oriented language that made me appreciate objects. I never really saw the point of object-oriented code until I started working with Python and Zope. Suddenly it all made sense.
The great thing about Python was that the indentation gave rise to a rather interesting side-effect: readable code. No more wars over bracing style, no more complaints about differng layouts of code amongst team members. You could step into someone else’s code and figure out what it did far more easily than other languages. It’s still possible to write obfuscated Python code, but you have to make a special effort.
Python’s standard library also covers quite a wide range of functionality, which has been a great help when writing code – when people say that Python has batteries included, they aren’t kidding! Perl might have CPAN, which is very large, but I find myself having to hunt out third party libraries less often than I did with Perl, and even less than I did with languages like Modula-2 or C/C++.
I don’t know if or when something will take over from Python, but for now it’s my main choice of language.
