Why I’m Still Learning Clojure

Last year was C# and Erlang, this year was Clojure thanks in part to the excellent London Clojure Code Dojo.

(Bear with me for the next paragraph, and the subjective nature of later bits)

Clojure has been an interesting experience because I have neither a Lisp nor JVM background. I’ve found myself having to deal with prefix notation and a parenthesis overdose, as well as the unfamiliarity of the Java ecosystem. The official documentation is patchy, the language is still new and evolving, dedicated development tools are relatively immature, and there are moments when the JVM leaks through. It’s the first time I’ve felt a little out-of-my-depth when learning a new programming language, which is not necessarily a bad thing.

Yet I keep going to the dojos, I still flick through the book when I get the odd spare moment, I keep improving my development environment and I’m still umming and ahhing between Clojure and Erlang for a possible home project I want to work on next year, once my FluidDB and game ideas have been unleashed upon the world properly.

So why am I still learning Clojure?

I asked myself that question recently. I still disagree with Neal Stephenson’s comment that Lisp “is the only computer language that is beautiful”, but Clojure has helped me to understand why people like Lisp as well as clean up some of the ugliness of the language. I’m sorry guys, but Lisp has always looked pretty ugly to my eyes, despite the lack of boilerplate found in languages like, say, Java or C#. When Clojurians say that it’s Lisp reloaded, they mean it – which has upset some Lisp programmers but pleased others.

There’s a greater sense that people are using Clojure to solve real problems, evolving the language to deal with real issues rather than thought exercises from the ivory towers of academia. The community itself is generally a friendly and positive one with a good mix of people from different backgrounds, not just the Lisp and Java worlds. Being there as the language and community evolves has been fascinating and quite exciting.

Getting back to Clojure’s Lisp origins, I still believe it should be the starting point, not the direction. Clojure needs to find its own way, its own metaphors and patterns – retaining Lisp’s strengths while confidently discarding the legacy baggage.

Take macros, for example: much as I can see the power of them, I think they’re a distraction rather than a tool in this day and age. A controversial view for sure, but I guess I’ve never seen a conclusive argument for their use – indeed, the wisdom seems to be that you shouldn’t really need to use them. It feels like there should be a more contemporary or Clojure-like way of providing the power of macros, without the complexity or obfuscation. I don’t know what that way is yet – but I’m not sure others do either.

Clojure encourages immutable data structures, but acknowledges that the real world doesn’t work that way so a pragmatic language needs to handle mutable state. Clojure implements this in more controlled ways to other languages. My favourite option is software transactional memory. STM is a natural fit to someone who has used relational databases for the last ten years. It feels right for many applications in a concurrent environment, and it’s right there in the language from the start – not hacked in later or provided as a third-party module.

It might not quite fit in my brain like Python, I don’t quite grok the intricacies of the language yet, and the sooner more of Clojure is implemented in Clojure the better. Yet it still has me interested – I want to learn more about the language and write more Clojure code because deep down there’s something kinda cool about it.

Bit Counts in Python, Erlang and Clojure

The subject came up yesterday about counting bits. I remembered the subject from university in the context of error checking, but I couldn’t remember the specifics during the conversation. It was only later that I recalled the subject of Hamming values or weights, and from there I tracked down an algorithm by Brian Kernighan (or Peter Wegner, or Derrick Lehmer). For fun, I translated the C source to Python to yield the following:

def bit_count( v ):
    c = 0
    while v > 0:
        v &= v - 1
        c += 1
    return c

For a bit of practice, I then opted to convert into Erlang:

bitcount(V) ->
  bitcount(V, 0).
bitcount(0, C) ->
  C;
bitcount(V, C) ->
  bitcount( (V band (V - 1)), (C + 1)).

I could probably roll the two arity 2 clauses into one, but this is a minor habit from my old Prolog days and I always feel it looks a bit cleaner than opting for a case statement. Feel free to disagree.

Since Clojure is my “language for 2010″, and I’ve been rather slack on keeping up with it lately:

(defn bit-count
  ([v]
    (bit-count v 0) )
  ([v c]
    (if (zero? v)
      c
      (recur (bit-and v (dec v)) (inc c)))))

This feels like it could be expressed more functionally, dropping the two argument version for a compact one-argument version. This might be an interesting exercise for when I have a spare moment, unless someone out there already has a solution?

Just thought I’d share….

Clojure Kata Two: Karate Chop (Pt. 5)

Lying ill in bed probably isn’t the most conducive environment for programming. I’ll spare you the full medical details, but in a strange way it put me in an evil mood… the kind of mood that seemed fitting to give macros a go.

I’ve never really seen the point of macros, but then again my experience of them has been the watered-down variety featured in languages like C. Reading up on macros in Scheme and Clojure more recently, macros come across as predominantly a way to do evil things. I say predominantly, because I accept that there are very rare occasions when a macro makes profound sense. Please note that the fifth solution I am about to present is definitely not an example of a good reason.

Last week, I was re-reading Jon Bentley’s excellent Programming Pearls book. I had been trying to avoid reading anything about binary searches while working on the code katas, so I picked a completely inappropriate book because binary searches are featured quite a bit. With my brain now sullied by a classic binary search algorithm, I went and undertook the fourth solution and tried to be a bit radical. It failed.

For the fifth one, I decided not to come up with my own solution but try a couple of things. The first was to implement Bentley’s algorithm in Clojure, and the second was to try and make it more compact through the use of macros – as a cheeky excuse to give them a try. The algorithm is already pretty elegant anyway, so I wasn’t sure what I could improve.

As ever, the code is on BitBucket at: http://bitbucket.org/metaljoe/metaljoe_codekata/

First thing was to implement the original algorithm in Clojure: (edited from changeset 9b3ef06116fd for clarity)

(defn chop [ item list ]
  (loop [ lower 0
          upper (- (count list) 1) ]
    (let [ mid (quot (+ lower upper) 2) ]
      (if (> lower upper)
        -1
        (if (< (get list mid) item)
          (recur (+ mid 1) upper )
          (if (= (get list mid) item)
            mid
            (recur lower (- mid 1))))))))

Okay, the first thing that stands out is the let, which holds the value of mid. Wouldn’t it be great to get rid of that? However, copying the code for mid back into the references would bloat the code out. I could use a function, but I’d be passing three parameters and it wouldn’t aid readability much. I would probably leave it for production code, but since I have a macro hammer in my hand, this looks like a macro nail waiting to be banged.

(defmacro mid []
  (list 'quot '(+ lower upper) 2) )

This will replace mid with the appropriate code in the macro. I create a list, all Clojure code is a list, and turn off parsing for quot and the lower + upper addition. If I don’t do that, they will both be evaluated in that context and will fail because neither are available in the namespace. I feel bad about these references, but this is just an exercise. I also realised at this point that list already exists in Clojure, and I should’ve used a different name for the argument to chop!

Once I’ve created the macro, I need to adjust the references to mid in the code. The references need to be wrapped in parentheses to avoid an error.

The next observation is that “(get list mid)” is repeated twice. It hasn’t quite reached the three-strikes rule, but it is duplication. I could set a variable to hold the value of the list at mid, or I could write a function to do it. The former would add an extra line to the code, and I would like to reduce the amount of variables if possible, especially as I’ve just gotten rid of one. The latter option wouldn’t offer much advantage because I’d still be passing list and mid to a function, and probably one with a less concise name than “get”. Basically, it’s another contrived situation for me to try a macro – though I would not do this in production code.

(defmacro get-mid []
  (list 'get 'list '(mid)))

This follows the same pattern as the mid macro – and even calls said macro.

(defn chop [ item list ]
  (loop [ lower 0
          upper (- (count list) 1) ]
    (if (> lower upper)
      -1
      (if (< (get-mid) item)
        (recur (+ (mid) 1) upper )
        (if (= (get-mid) item)
          (mid)
          (recur lower (- (mid) 1)))))))

Not a massive improvement to the code readability, and I’m not entirely happy with get-mid as a name. The nicest thing is that the let statement has been removed.

How about dealing with the statement to initialise the upper variable? Shunning a more obvious function…

(defmacro get-upper []
  (list '- '(count list) 1 ))

Okay, time for one more thing… well, two actually. The recur statements are fine, but perhaps we could make them more declarative?

(defmacro look-left []
  (list 'recur 'lower '(- (mid) 1)))

(defmacro look-right []
  (list 'recur '(+ (mid) 1) 'upper ))

This changes the loop code to read:

(loop [ lower 0
        upper (get-upper) ]
  (if (> lower upper)
    -1
    (if (< (get-mid) item)
      (look-right)
      (if (= (get-mid) item)
        (mid)
        (look-left))))))

In fact, while typing this up, it looks odd now because it’s defining two variables that are apparently never updated.

First off, let’s remove the last visible references to lower and upper, using (surprise!) a suitably named macro…

(defmacro search-exhausted []
  (list '> 'lower 'upper))

Now time to deal with that loop statement. Well, my plan was to create a macro like this:

(defmacro do-search [ & body ]
  `(loop [ ~'lower 0
           ~'upper (get-upper) ] ~@body
     {:lower ~'lower :upper ~'upper }))

This macro is based on Stuart Halloway’s evil-bench example from the Programming Clojure book code, which is very evil. But since this kata solution is all about evilness, I only feel slightly guilty.

The aim was to replace the loop with code that looks like this:

(do-search
  (if (search-exhausted)
    -1
    (if (< (get-mid) item)
      (look-right)
      (if (= (get-mid) item)
        (mid)
        (look-left))))))

Unfortunately, this raises the following exception:

Exception in thread "main" java.lang.UnsupportedOperationException: Can only recur from tail position (code_kata_2e.clj:64)

The macroexpand and macroexpand-1 functions weren’t giving me anything useful, probably because they were overwhelmed by nastiness. Oh well, not to matter as I don’t exactly grok macros just yet and I’m happy overall with what I’ve learnt this time. The code is on BitBucket, so if anyone with better macro development skills can point out the problem, please let me know.

Clojure Kata Two: Karate Chop (Pt. 4)

“The programmer builds from pure thought-stuff: concepts and very flexible representations thereof. Because the medium is tractable, we expect few difficulties in implementation; hence our pervasive optimism. Because our ideas are faulty, we have bugs; hence our optimism is unjustified.”

- Frederick Brooks, The Mythical Man-Month

I was on an evening train home when, re-reading The Mythical Man Month for the first time in almost ten years, I came across the above words. It summed up my fourth attempt at the Karate Chop code kata: because our ideas are faulty, we have bugs.

My fourth algorithm idea was faulty, I just didn’t quite want to admit it. My ideas were faulty because of lack of mental preparation. I was tired (I’ve not been sleeping well for a while now), distracted, and in a hurry to get past the Karate Chop kata and move onto another challenge. I didn’t think things through when I started, I just fired up Aquamacs and began to type.

The idea was simple enough – strip the algorithm down of a lot of state and just “bounce” from position to position within the list. Starting from the 0th element, we determine an offset which is equal to half the length of the list. The offset is added to the starting point to determine the position to check.

If we don’t match, we halve the offset and either add or subtract the offset from the current position to yield our new position, depending on whether the item we checked was higher or lower in value than the one we’re searching for.

It sounds simple enough, and plausible enough, so I really should’ve run a quick check in my head or on a sticky note to verify it actually worked. The fact I don’t recall seeing this process before should’ve tipped me off to needing to double-check it.

Thing were fine to begin with, except for one failing test. It was the first off-by-one error I had encountered so far during the Karate Chop implementations, I’m rather proud to say, so it seemed to be quite a simple thing to fix. Oh was I wrong: a simple change resulted in multiple failed tests. Hasty debug statements (I’m guessing I’ll need to learn a Java debugging tool for Clojure development) began to highlight much weirdness with each loop through the search. Values didn’t seem to tally to the expected values, and the search position was often wildly off.

A hasty bit of refactoring to eliminate duplication and side-effects as possible causes didn’t offer much improvement, though I learnt to apply some interesting bits of Clojure in the process. In the end, I had a pretty good idea that the algorithm was flawed, but I kept convincing myself that things like padding the list or adding special cases would massage it into place.

Some days you just have to down tools, stand back, learn from your experiences and move on. The idea started off simple but started to grow… and smell.

In the current form, the algorithm fails on three tests:

(assert (=  0 (chop 1 [1 3 5]) ))
(assert (=  2 (chop 5 [1 3 5]) ))
(assert (=  0 (chop 1 [1 3 5 7]) ))

The bit that stood out straight away was that they were all tests for a value on the boundary of the list. Analysis of the code determined that the search algorithm never reaches the first and last elements in the list. And padding the list doesn’t help either.

In the first test case, our initial offset is 1 since our list consists of three elements, and integer division of three by two yields 1.

Position 1 of the list (3) is not what we’re looking for, so we halve the offset (integer halving of 1 yields 0). 3 is greater than the value we’re looking for, so we subtract 0 from the current position i.e. we stay where we are. The algorithm detects the zero offset, and evaluates to not-found. See the problem?

Searching for 5 in the same list has the same problem, except we try to add the zero offset to our current position with identical result.

What they both should do is try an offset of 1 again, but that wouldn’t involving halving the offset each time.

The four element list has an initial offset of 2, which yields the value 5 at list index 2. We know to check the lower half of the list, so we halve the offset to yield 1 and then subtract that from the current position. Our new position yields the value 3, which means we have to search left again. Alas, our offset of 1 is halved to 0 and the search terminates. We never get to check that first element in the list.

Going through the problem on a sticky note diagnosed the problem really easily, and working through for 5, 6 and 7 element lists made me realise that the maths would need a reworking to cover those edge cases. Typing up my experiences just now, I’ve even had a thought about one way to solve the problem without getting too complex… but I will revisit another time.

All-in-all, this was actually the most interesting experience so far during the kata. We often learn a great deal from our mistakes than our successes, failure can be good for us sometimes – it takes us out of our comfort zone, which is where true learning happens.

As ever, source for this and the previous solutions is available from http://bitbucket.org/metaljoe/metaljoe_codekata/ under the MIT license. If you want to know more about the Karate Chop code kata, head over to http://codekata.pragprog.com/2007/01/kata_two_karate.html and take a look.

Clojure Kata Two: Karate Chop (Pt. 3)

This is my third Clojure-based solution to the Karate Chop kata detailed at http://codekata.pragprog.com/2007/01/kata_two_karate.html

The write up is going to be a little shorter than before, because I’m pretty tired as I type this. As always, source code to the solution is available from BitBucket at http://bitbucket.org/metaljoe/metaljoe_codekata/ under the MIT license and feedback is definitely always welcome.

For this solution, the aim was to produce a concurrent version of the code. Surprisingly, this proved to be quite tricky to write compared to implementing an equivalent concurrent solution in a language like Python or Erlang. The main problem was that the “Programming Clojure” book apparently missed a few important details about Agents (Agents can’t call Agents, shutdown-agents needs to be called for cleaning up). I got rather frustrated with the documentation and examples available, with the code raising assorted semi-cryptic Agent-related exceptions that weren’t adequately explained anywhere.

After a considerable amount of time on Google, and a drastic scaling back of my intended implementation, I finally created a working solution this evening.

The original aim was to keep dividing the list in two, spawning a new Agent/thread to investigate each new list by repeating the splits. This would continue until a match was found or an empty list reached. Not very efficient or elegant, but a way to incorporate an element of concurrency into the kata.

Since Agents can’t call Agents, the first major hurdle I had, I ended up running just two Agents: one to check the left half of the initial list, the other to check the right. The spawning of further Agents was eliminated in favour of an inefficient recursive check of each half of these lists, and so on. I opted to keep this style of checking, to “simulate” what my original implementation would have operated like.

This worked well, but the script would not quit until killed with Control-C. After another search through the book and docs, I found a reference to calling shutdown-agents. This only worked when called at the end of the script, not within the function calling the agents for various reasons.

I also took the opportunity to try and make a little more legible approach, as well as try out the split-at function.

Now I need sleep.

Clojure Kata Two: Karate Chop (Pt. 2)

This is the second attempt at a Clojure-based solution to the Karate Chop kata detailed at http://codekata.pragprog.com/2007/01/kata_two_karate.html

Source code is available from BitBucket at http://bitbucket.org/metaljoe/metaljoe_codekata/ and is offered under the liberal MIT License. Since the aim of a kata is to learn and become a better programmer, feedback is always welcome.

Since the previous solution became a recursive one, my aim for this attempt was to work on an iterative implementation. Concerned about the rather ugly feel to the recursive code, I was also looking to produce something a bit more compact, a bit more elegant.

The code actually came to me pretty quickly compared to the first attempt. One advantage is that the problem domain is clearer on the second attempt, my brain has had time to consider the pitfalls and map out a more concise understanding of the problem. The other factor is that writing my own Clojure code, as opposed to quick REPL exploration or fill-in-the-blank exercises, has become more natural. I have begun to start thinking in Clojure. Note that I said “have begun to” – I still have more work to do on that front.

As far as implementation went, it was pretty smooth until I hit one snag: an infinite loop. It took a while to figure out what was going on, until I spotted a glaring issue. If the slice/partition of the list I’m looking at reaches one element, I check it for a match. If it didn’t match, I would loop around and search it again. An embarrassing newbie error, but a useful mistake to make. I added the appropriate check and all the tests passed.

After completing the implementation I spotted the second mistake when I reviewed the code later on. I was testing for an empty list on each iteration. While it has no impact on the output, it’s an unnecessary check for the implementation – if the list does suddenly become empty, we have bigger problems!

One thing that did intrigue me was that with almost no effort, the solution was much smaller and more readable than the first approach. I usually find the opposite with recursive versus iterative code, but in the case it’s probably because the problem seems more natural expressed iteratively. Or maybe my recursive approach just sucked

Here’s the completed algorithm, with tests removed and the code tidied for clarity. See the Mercurial repository at BitBucket for the original source.

(defn chop [item list]
  (if (empty? list)
    -1
    (loop [ min 0 mid (quot (count list) 2) max (count list)]
      (if (= item (get list mid))
        mid
        (if (or (= mid min) (= mid max))
          -1
          (if (< item (get list mid))
            (recur min (quot (+ mid min) 2) mid)       ; look left
            (recur mid (quot (+ mid max) 2) max))))))) ; look right

If I broke my rule to revisit the code, I’d change a few minor things for readability. For example: why did I put those two comments there, and not elsewhere? The loop initialisation looks a bit of a blur at first glance, the or statement might need to be wrapped in an intention-revealing function name. However, the code is good enough for the task at hand and that’s all that really matters.

Clojure Kata Two: Karate Chop (Pt. 1)

I thought it might be interesting to work through the code katas created by Pragmatic Programmer Dave Thomas to assist with my Clojure learning. If you’re looking for Kata One, don’t worry about searching – I started with Kata Two as I needed an actual coding exercise.

The instructions for the Kata can be found at: http://codekata.pragprog.com/2007/01/kata_two_karate.html

The task is to implement five different binary search algorithms, one per day. I don’t actually have five consecutive days to do this, so don’t expect the five parts to be written about each day. There will be gaps.

It’s been about 15 years since I wrote a binary search algorithm, using the language Modula-2. I was toying with the idea of going off and reading up on binary searches, but I decided the best way to approach it was from first principles – only allowing myself access to a Clojure reference manual. After all, I go on about the importance of learning basic algorithms even in this day and age… so about time I put my money where my mouth is!

The trickiest part of the kata wasn’t so much coming up with the algorithm, an iterative binary search is easy, but expressing the algorithm using Clojure’s syntax. That’s mainly the inexperience with the language showing, but the resulting code didn’t feel too elegant. In particular, the parentheses became more of a hindrance than a help – they start to look plain ugly, get in the way and I don’t think I’ve ever relied on my IDE’s parenthesis matching so much. I still don’t agree with Neal Stephenson that LISP is the only language which can truly be described as beautiful… despite all the power and flexibility it can yield in the right hands.

I may not have reached Clojure enlightenment yet, but gradually I built up a rhythm with the coding. Once the tests passed, I decided to stop rather than try to rework the code into something that looked and felt better. Take the lessons learnt and apply to the next algorithm – the work for this one has already been done.

First step with the coding was to create the tests. Although not a fan of Python’s doc tests, I opted to use Clojure’s equivalent for quickness – and I’ve decided I’m not a massive fan of them either. Moan, moan, moan.

(defn
 #^{:test (fn []
 (assert (= -1 (chop 3 []) ))
 (assert (= -1 (chop 3 [1]) ))
 (assert (=  0 (chop 1 [1]) ))

 (assert (=  0 (chop 1 [1 3 5]) ))
 (assert (=  1 (chop 3 [1 3 5]) ))
 (assert (=  2 (chop 5 [1 3 5]) ))
 (assert (= -1 (chop 0 [1 3 5]) ))
 (assert (= -1 (chop 2 [1 3 5]) ))
 (assert (= -1 (chop 4 [1 3 5]) ))
 (assert (= -1 (chop 6 [1 3 5]) ))

 (assert (=  0 (chop 1 [1 3 5 7]) ))
 (assert (=  1 (chop 3 [1 3 5 7]) ))
 (assert (=  2 (chop 5 [1 3 5 7]) ))
 (assert (=  3 (chop 7 [1 3 5 7]) ))
 (assert (= -1 (chop 0 [1 3 5 7]) ))
 (assert (= -1 (chop 2 [1 3 5 7]) ))
 (assert (= -1 (chop 4 [1 3 5 7]) ))
 (assert (= -1 (chop 6 [1 3 5 7]) ))
 (assert (= -1 (chop 8 [1 3 5 7]) )) ) }
 chop
 [item list]
 ...

I made the chop function a wrapper around the real action, meaning I could reuse it for different algorithms easily as well as keep the implementation separate. I opted to call a function called bsearch, passing the list of values, the value to search for, and a start and end index to define the “partition” under examination. The initial start and end indexes set the partition to be the whole of the list – since we have no idea where the value is to begin with.

(bsearch list item 0 (count list))

The initial check in bsearch is for an empty list. If we have nothing to search, we know for sure that the value doesn’t exist in it.

Next, we take our partition, or slice as I called it in the code, for examination. If the slice contains one item, we have a simple case to check: either that item is the one we’re looking for, or it isn’t.

If we have more than one item, our proper binary search comes into play. We find the mid-point and check the value of that item. As with the single item list, it’s either the value we want, or it isn’t. If it isn’t, we split the slice on that point and search the “left half” (values lower than the mid value) or the “right half” (values higher than the mid value) accordingly.

Although my intention was to do an iterative algorithm, Clojure’s syntax nudged me into a more recursive approach. For the tests supplied, blowing the stack is unlikely but for larger lists the algorithm won’t be the most efficient approach.

A couple of helper functions first:

(defn is-single-item? [ list ]
 (= 1 (count list)))

(defn mid-index [ list ]
 (quot (count list) 2))

These were just to make a couple of spots in the code slightly more readable at a glance.

And the bsearch function in all its nested glory:

(defn bsearch [ list value slice-start slice-end ]
 (if (empty? list)
  -1
  (let [ slice (subvec list slice-start slice-end)]
   (if (is-single-item? slice)
    (if (= value (first slice) )
     slice-start
     -1)
    (let [ mid (mid-index slice) ]
     (let [ mid-value (get slice mid) ]
      (if (= value mid-value)
       (+ slice-start mid)
       (if (< value mid-value)
        (bsearch list value slice-start (- slice-end mid))
        (bsearch list value (+ slice-start mid) slice-end)))))))))

Check out the multiple closing parentheses. That just looks wrong, but I see lots of example Clojure and LISP code like that. When writing the code, I ended up writing the parentheses in alignment for readability, then when I finish coding I reposition the closing parentheses to match the style above. I can’t help but think it’s a code smell indicator.

Certainly an interesting exercise and my first go at writing Clojure code from scratch, not following along with the Programming Clojure book. Definitely has room for improvement.

Now, how am I going to write my second binary search algorithm…?

Clojure First Impressions

I’m only a few pages into Stuart Halloway’s “Programming Clojure” book but I’m already starting to like Clojure, and by extension LISP. As with every language, it has its good points and its bad points, but so far it’s been a positive experience and is bringing the same smile to my face that Erlang brought me last year (mental note to self: need to carry on exploring Erlang).

I’ve yet to be fully convinced about the “everything is data” approach, though I am aware of the merits, and worry that being able to redefine aspects of the language through macros can be both a curse and a blessing for maintenance of code. My mind is open though, and I’m looking forward to covering Clojure’s macro capabilities, something that has been of interest to me since reading Michele Simionato’s excellent “The Adventures of a Pythonista in Schemeland“.

Clojure has taken some useful steps in cleaning up LISP’s minimal syntax: less parentheses, clearer function parameters, commas as whitespace (trust me, it actually makes sense), more readable ames for built-in functions (is that the correct terminology in the LISP/Clojure world?), things which had traditionally been mildly offputting aspects of the LISP family.

One thing that can still confuse some people is Clojure’s use of prefix notation, since I’m not aware of any other general purpose language that uses it. Most kids these days have only ever seen Algol-descended languages… even if they have no idea what the heck Algol is. It’s actually not that difficult to adjust to, but seeing something like (+ 1 2) can look odd to many… until you realise it’s not that different to saying +(1,2) or add(1,2), rather than the more usual syntax of 1 + 2.

In some ways, it’s probably more natural to many of today’s programmers than Erlang’s Prolog-derived syntax would be (I’m conveniently ignoring the more imperative aspects of Erlang). Years ago someone told me that LISP was an expression of syntax trees, and that mental image has stuck with me – after all, the syntax of LISP is about lists of data/code, or more correctly symbolic expressions. Erlang is more like Prolog’s expression of Horn Clauses.

The prefix syntax does remove the problem of precedence, which is touted by some as an advantage over other languages. While true, it does feel a little bit like cheating, because writing something like (* (+ 1 2 ) 3 ) is no different than requiring explicit use of precedence-defining parentheses in other languages: ( ( 1 + 2 ) * 3 ). However, it does improve extensibility… sort of. Infix notation is a little limited: 1 + 2 is stuck rigidly to two parameters, and the singular case of + 1 isn’t quite as much fun. How do you express 1 + 2 + 3, but using only a single + character? Clojure’s syntax allows us to use + as a summing function as well as for simple addition: (+ 1 2 3 4 5). Okay, something like sum(1,2) and sum(1,2,3,4,5) in some other languages. The thing is, Clojure defines a consistent approach, whereas most languages have special-cased some of the basic mathematical operations.

Speaking of which…

user=> (/ 1 2)
1/2

Huh? Ah, I see! Integer division results in a fraction or ratio. I can see the benefits of this evn though it does feel a little odd, but that’s probably because I’ve been conditioned by other languages that would return 1, 0.5, 0 or other interesting floating point variations. To get the expected answer, you need to use (quot 1 2) for an integer value or (/ 1 2.0) to return a floating point value. The presence of the “keyword” (symbol?) quot shows that Clojure hasn’t entirely dispensed with the abbreviations that could make older versions of LISP a little cryptic and newbie-unfriendly, but it’s a heck of an improvement.

Clojure’s use of transactions for modifying mutable data is interesting. At first glance, the simple introductory example in the book looks a little unwieldy with the calls to dosync and alter just to update a single variable. But I can see how it might shine in more complex examples waiting in later chapters.

Now back to the book…

Things to look at in 2010

Time permitting, here are some technologies I want to learn / evaluate / play with:

• Clojure
• CouchDB
• Natural Language Toolkit (NLTK)
• PyGame
• Twisted
• XMPP

I’ve already started learning Clojure as it’s my new language for the year. And by “started”, I mean I have a copy of the Programming Clojure book, have installed Clojure and written a Hello World program. I realise I have some way to go, but from little acorns…

CouchDB is a late entrant. I’ve yet to be convinced by non-relational databases, with the possible exception of document storage. For my current work, they would be extremely inappropriate except maybe for logging. However, I really want to get experience of a non-relational database under my belt – so I can properly argue where they make sense and where they don’t. CouchDB sparked my interest for three reasons: it was the first of the new generation of non-relational databases I’d heard of, it uses Erlang and it has a RESTful interface.

NLTK would be an interesting additional to my technical tool belt and I can see potential applications in my current job, where we’re finding ourselves increasingly sifting through textual data to determine various facts.

I rarely play computer games, but one game that has had me hooked for the last five years is Hearts of Iron II. I’m an armchair military historian, particular WW2 and other 20th Century warfare, so HOI2 has great appeal to me. I’ve long wanted to develop wargames in a similar style (though perhaps a little more towards the board game format), and even went so far as to writing story cards for various aspects of one game idea I’ve had. I’ve decided this year that I will make a start at coding, eventually releasing the code under an open source licence. I’m contemplating writing the code in Python, and have decided that PyGame would offer me a rich library for support (based on an excellent presentation at a recent London Python Code Dojo).

Twisted has been an interest of mine since I attended a talk at one of the PyConUK events. I even bought the, outdated, O’Reilly book on Twisted, but have never actually had need to take a serious look at it. However, things are picking up and I can see some possible areas which might be appropriate and I want to take this motivational opportunity to investigate Twisted more.

After reading the O’Reilly XMPP book a while back, XMPP is something else I want to investigate further. I actually had a play with XMPP from Python a couple of years back, but the Python library I used (I can’t remember the name offhand) but it was buggy, lacked certain functionality I needed, and wasn’t being actively maintained. Thanks to the book, I’ve discovered there are other libraries out there for Python and they look a whole lot better, as well as still actively developed.

The XMPP and Twisted investigations are sort-of related, since I’m keen to work on web service and messaging systems to support the infrastructure at work. We’ve actually started introducing SQL Server Service Broker, for better or worse, and I’m keen to find ways to interface with Service Broker in more architecture and language neutral ways i.e. without locking in to .NET, SQL Server and Windows. Optimistic?

Why I’m Learning Clojure

Recently, I posed the question of which language I should learn in 2010. The options were Squeak, Common Lisp, Clojure, Groovy… or to make a second attempt with Ruby. I must admit I was veering towards Lisp when I wrote it, but all options were still possibilities. Out of the two Lisp dialects, Common Lisp was probably my preference over Clojure but in the end I decided I would give the latter a shot.

So why Clojure?

Well, the first reason is down to it being a new language. There’s something quite interesting about trying a language in the early days. Since I’m not going to be writing production code with it for the foreseeable future, having a moving target isn’t really an issue. The community surrounding the language is still forming and there’s a chance to watch and interact with a forming and evolving Clojure ecosystem. London also has a Clojure code dojo running, aimed at introducing people to Clojure, which offers an excellent opportunity to help with my learning – and best of all, there’s an overlap with some members of the London Python community. Very handy.

Secondly, Clojure is a Lisp dialect. Lisp has been on my list of languages to learn for a long time, but I’ve always ended up looking elsewhere and my only real experiences with the Lisp world have been trivial at best, and never with “the real thing”. In some ways I guess I’m still following that path, though Clojure feels more like a proper Lisp because at heart it is.

Clojure’s primary environment is of the JVM, an area I’ve only limited experience of. There was a time when Java was appealing to me, but every time I take a second/third/umpteenth look at the language it seems to have accumulated more baggage and bloat. It’s a huge and complex language and ecosystem to get into these days, though very mature and powerful. Clojure would give me an opportunity to delve into the JVM world a bit more.

The language is also making progress in supporting the .NET world. Much as I’m definitely not a Microsoft fan, .NET has grown on me and there’s some interesting work happening with it and the open source Mono implementation. I actually rather like C#, though wish someone would take the core language and move it out of the .NET world – probably a weird idea to some. Anyway, Clojure on the CLR is an interesting idea, so adds a little to the interest because I have to mix things up with .NET developers at work.

So that’s all the fluffy stuff out of the way – what are the other reasons?

Well, Clojure is a functional programming language, which is a current interest of mine. Spurred on by my adventures with Erlang, functional programming is growing on me. It’s also dynamically typed, which has been one of the big revelations during my career, perhaps even more than object-oriented development. It compiles to JVM bytecode; being Lisp, it has read/eval/print which makes language exploration and prototyping a joy; and I can delve into a language with a proper, powerful macro capability.

Clojure is also geared up for the big issue of the now: concurrency. Erlang interested me because of the way it tackled concurrency, something that all programmers are going to have to face up to sooner or later as we live in a multicore world. I like Erlang’s approach a lot, but there are other options out there and I feel it is my duty to examine a few of the options. In particular, the use of Software Transactional Memory (STM) holds an appeal to me. Having used databases the last ten years, transactions feel quite a natural approach for many problems, but definitely not all, and incorporating them into other software domains can only be a good idea. I was considering Haskell at one point to investigate STM, but I must admit to not being a massive fan of parts of Haskell’s syntax so never pursued the idea. With Clojure, I can at last take the chance to delve into STM.

I’m not going to stop learning and exploring Erlang, far from it, but I’m looking forward to adding Clojure to my programming toolkit.