Writing a Simple Clojure Library

I’ve been learning/using Clojure on and off for about 2 years. The lispy syntax isn’t a deterrent for me at all, in fact, I’m quite fond of it and consider it very elegant. However, it does take some time to get used to. I don’t use Clojure or anything remotely close in my day job, but I love to find something useful to implement using Clojure. In the past few days I found such niche.

Terminal colours

Ever wonder how some console applications can output coloured text? Most terminals and terminal emulators (think iTerm or konsole) support colour through a system of escape sequences.

An Escape Sequence starts with the ASCII code for the escape character (0x1b or 33). There a list of control characters you can specify after the escape character that controls the colour and style of the text after it. For example, sending “[31mfoo” to a terminal means “output red coloured text from now on”. Everything the terminal output will be coloured red, until the terminal reads another escape sequence, including “[ESC]0m”, which tells the terminal to reset the styles to the default.

So I had this idea to implement this in Clojure as a library so that console application authors can use it to output stylized text from their application using idiomatic Clojure, and here comes Lumiere.

Implementing it in Clojure

Design the interface

What’s our end goal here? We would like to output colour sequence wrapped text with Clojure function calls such as

(red "foo") ; output foo in red colour
(bg-green "bar") ;"bar" with green background colour
(bold (magenta "nyan cat")) ; bold and magenta
(-> "nyan cat" red bg-white bold) ; use Clojure's "thread macro" to combine these functions, resulting in red foreground, white background and bold "nyan cat"

Toolchain

I wrote a blog post a year ago about how I liked Cake the Clojure build system alternative to the defacto Leiningen. The recent news on this is that Cake and Leiningen are merging. This time, I decided to use Leiningen from the start. Even though Leiningen hasn’t ported some of the Cake goodies, but I’m hoping they will get ported soon.

First working version:

With lein new lumiere, Leiningen generates the default project layout. By default, Leiningen generates src/lumiere/core.clj and test/lumiere/core.clj. Because Lumiere is such a small script, we don’t need the ‘lumiere.core namespace, rather I’d like the functions to be in the ‘lumiere namespace. The easiest way is to delete src/lumiere/ folder and create lumiere.clj under src. Same goes for the test file.

Following the spirit of TDD, off I went to write my first test:

(ns lumiere.test
  (:use [lumiere])
  (:use [clojure.test]))

(deftest test-only-foreground
  (is (= "\033[30msome black text\033[0m" (black "some black test"))))

This tests that the correct sequences of characters are generated by the call to (black "..."). 33 is the ASCII code for .

To make this test pass, I added this in src/lumiere.clj,

(ns lumiere)
(defn black [text]
  (format "\033[0m" 30 text)) ; 30 is the code for black foreground.

This passes the tests, but obviously there’s room for abstraction:
* the red foreground code is 31, green 32, etc…
* the black background code is 40, red 41, green 42, etc…

So here we go:

(defn- colour ; defn- makes this function private to the current namespace.
  ( (format "\033[%dm" (+ (if is-bg? 40 30) code)))
  ( (colour code false)))

(defn black [text] (colour 0))
(defn red [text] (colour 1))
(defn bg-black [text] (colour 0 true))
(defn bg-red [text] (colour 1 true))

Clojure supports "default" arguments through method overloading. Here we adjust the offset based on whether or not that colour is a foreground or background.

Second take: use macro to define declaratively create colour functions

One advantage of lispy syntax is the convergence of programming with meta-programming. If you read the core Clojure library code, you'll find that Clojure defines a few special forms and most control structures are written in macros. In my case, however, I'm using macros to define colour/style functions in a declarative way. Some may argue it doesn't justify using macro for this purpose, but I just want to practice writing macros, and it does make the public interface of the library a bit prettier.

First off, again, we need to define what the end result should look like. I would still like the functions to remain the same, e.g., red, bg-red, black, bg-black, etc. However, defining these functions takes a lot of boilerplate code. I'd like to simply call (defcolour BLACK 0) or the like to generate the `black` and `bg-black` functions for me.

Disclaimer: I'm a novice macro writer. Advanced readers, please hold your nose and tell me what I did wrong or any improvements could be made 🙂

(def RESET "\033[0m")
(defmacro defcolour [colour-func-name bg-colour-func-name colour-code]
  `(do
    (defn ~colour-func-name [text#]
      (format "%s%s%s" (colour ~colour-code) text# RESET))
    (defn ~bg-colour-func-name [text#]
      (format "%s%s%s" (colour ~bg-colour-code true) text# RESET))))

(defcolour black bg-black 0)
(defcolour red bg-red 1)
(defcolour green bg-green 2)
; etc...

A few special reader macros you need to know about when writing a macro:
- tick (`) indicates the following code should be quoted and treated as a template.
- tilda (~) indicates that the symbol should not be quoted (unquote), and should be replaced with the value in the current context.
- hash (#) indicates that the macro system should generate a unique name for this symbol so it doesn't conflict. Otherwise, it will be expanded to its fully qualified name.

Run tests and because we didn't change our public interface, everything all tests should still pass.

Take 3: combining styles

Alright, now that we have a fully functional style system, we can cascade the styles, e.g., (red (bg-green "foo")). It work as expected when trying it in a REPL, but the character sequence it generates is "33[31m33[42mfoo33[0m33[0m" and surely it isn't optimal. If we want to add styles such as bold, it's going to get even worse.

So, we need some abstraction here. When you call (red "some text") it shouldn't generate the character sequence right away. Instead, the caller should decide when the sequence should be generated. We need some data structure to represent a "luminated" text. In Clojure we can define a "record".

(defrecord Lumiere [text opts])

"opts" is a map with keys :fg, :bg, :styles. We also want to override the toString() method so when the user calls (str lumiered-text), he will get the character sequence ready to be printed to the console. The downside of this is that we modified the interface, so we need to go back and change the tests so they call (str (red "foo")):

  (is (= "\033[30msome black text\033[0m" (str (black "some black test")))))

To override toString, we need to extend our Lumiere type to conform to the IObject protocol:

(defn- ansi-escape-seq [& codes]
  (format "\033[%sm" (join ";" (filter #(not= % nil) codes))))

(defrecord Lumiere [text fg bg styles]
  Object
  (toString [this]
    (let [prefix (ansi-escape-seq (:fg this) (:bg this) (:styles this))]
      (format "%s%s%s" prefix (:text this) RESET))))

Next we need to let the colour/style functions return Lumiere object, rather than plain character sequence. There are two situations we need to adapt:
1. When we first start decorating a text, text input is going to be a plain string. In this case, we need to create a Lumiere object with the text and options.
2. When the return of a colour/style function is chained into another colour/style function, we need to modify the options of the Lumiere object.

(defn- adapt-lum [text option value]
  (let [local-option-map (merge {:fg nil :bg nil :styles nil} {option value})]
    (cond
      (instance? String text) (Lumiere. text (:fg local-option-map) (:bg local-option-map) (:styles local-option-map))
      (instance? Lumiere text) (assoc text option value)
      :else (throw (java.lang.IllegalArgumentException.)))))

and we need to modify the macros so "adapt-lum" helper is used:

(defmacro defcolour [colour-func-name bg-colour-func-name ^Integer colour-code]
  `(do
     (defn ~colour-func-name [text#]
       (adapt-lum text# :fg ~colour-name))
     (defn ~bg-colour-func-name [text#]
       (adapt-lum text# :bg ~bg-colour-name))))

Publish to Clojar.org

Now that the library is in a relatively stable state. I'd like to publish this snapshot version to a repository. Clojars.org is the most popular clojure library repository. Register on clojars.org, add let them know your public key. Then do lein pom && lein deploy, voila!

Advertisements

Spectrum.vim – My first Vim plugin

Over the past few months, I’ve been using Vim as my primary development tool at work and at home, and I have to say, I’m addicted to it! I’m thinking about writing a blog post of why I get hooked on “walking without crutches”, but for this post, I’m just going to introduce you to my first plugin in Vim – Spectrum.

Introduction

Spectrum is a vim colorscheme roulette. Ever getting tired of staring at the same colorscheme every day? Having hundreds of colorschemes in your repo but too lazy to deterministically pick one? Spectrum helps you by randomly pick colorschemes from your vim runtime path or from the web. From there, you have the chance of voting up a colorscheme so Spectrum will have a higher probability to pick it or voting down a colorscheme so you wouldn’t see it again.

Development

To be honest, I’m not a fan of Vim script – the language is not very expressive and doesn’t have a lot of object oriented features. Semi-fortunately, since Vim 7, they have added support for Python, Ruby and Perl scripts. I said ‘semi-fortunately’ because the support isn’t too comprehensive. For most core vim features, you still have to resort to calling Vim commands to achieve them, but at least I don’t have to use Vim script for the most part.

Spectrum is written in Python, and use the “vim“ module to interact with the hosting Vim instance. There is a bit of bootstrapping to do if you want to separate most of the Python code out of the entry point vim script (see https://github.com/kevinjqiu/spectrum.vim/blob/master/plugin/spectrum.vim). Many vim plugins written in Python require you to install the python code into your Python runtime before you can use them, but for a simple module like Spectrum, I opted for monkey patching syspath to include modules in the plugin folder.

Anyhow, give it a try and hope you like it.

https://github.com/kevinjqiu/spectrum.vim

New Year’s Resolution

2011 here we come! In the spirit of continual learning, I’m going to write down the technology I’d love to learn this year.

  • Haskell
  • Now that I’m more interested in functional languages, I’d love to look into this “pure” functional language that inspired countless other ones of its kind.

  • Lift
  • Last year I scratched the surface of Scala, a hybrid JVM language. I’m very fond of it, and think it has tremendous potential. Twitter and FourSquare are already using Scala, so it has been put to the test of some pretty high-profile usages. Lift is the most popular web framework built on top of Scala. It claims to have the rapid application development benefits from Rails and the benefits from statically typed language.

  • More advanced features of Clojure
  • In the past two years, I explored Clojure and on off. I love the Lisp idea of homoiconicity which unifies programming and meta-programming. That said, I haven’t been using macro in Clojure too much, and there are other cool ideas of Clojure I haven’t been able to explore deeply, such as protocols and software transactional memory (STM)

  • Ruby and Rails 3
  • For the past few years, I’ve been refraining myself from learning Ruby, because I’m already quite adept at Python, and I feel I should be learning languages that are different from what I already know. However, the more I heard about Ruby and its ideas, the more interesting it appears to me. On top of that, Rails 3 has come out, and it appears to have improved significantly. Moreover, there are a lot of advancement in the Ruby VM world such as JRuby, which means Ruby applications don’t have to run on the old and dreadful (hear-say) MRI. I wouldn’t mind taking Ruby and Rails out for a spin this year.

  • Android
  • Last year I got an HTC Legend Android phone, with the intention of developing for Android at some point of time. It didn’t work out that way, though, but Android continues to be a very interesting and fast growing platform. Mobile *is* the future, and I’d like to poke into the Android world this year, primarily because it’s open source. iOS is equally interesting technically, but I don’t own a Mac and I don’t like the idea of paying Apple $99 for SDK even though I don’t intend to publish on AppStore.

I think that’s enough for a year…or is it? There’s a few more technologies I wish to learn or keep up with:

  • GWT and Google App Engine
  • 2 years of professional GWT development made me a firm fan of this Google technology. I got out of GWT for different reasons, but I love the engineering effort they put into GWT. It may not take over the world but it’s definitely a solid player in the front-end web development arena. Especially now they integrated with the Spring framework and made deploying to App Engine easy, it may pick up more traction this year. I think the Java language is both the pros and cons of GWT. I’d love to see an alternative language (Scala) being implemented for GWT, but it may not happen any time soon.

  • A “NoSQL” database
  • Let’s face it, “NoSQL” is a terrible name, but it grabs people’s attention. I flirted with CouchDB briefly last year, and would love to continue this journey this year. Also, MongoDB seems interesting too.

  • Node.js
  • It’s the least I think I’d learn this year. It’s hot in the geekdom right now, and it has its value, for example, having both the server and client side written in Javascript eliminates the need to implement the validation logic in two different languages. However, I’m just not a big fan of Javascript. I think it’s a language that’s by a chain of serendipitous events became the world’s most widely used language. It carries a huge historical burden, and although it has cool features, some other modern languages have them too and do better. Regardless, given the stardom status of Node.js, it deserves some looking into 🙂

Genesis

Well, I’ve been thinking about going back into blogging for some time now…I haven’t been able to make the determination.  However, now I think the era I’m going through right now is too precious to go wasted and without any trace.  So, I’m going to try my best to write blog more often to record this period, although not everything is wonderful and pleasant, the memories will be there forever and will remain part of myself.  Looking back several years later, even the bitterest moment will be just as precious and every bereavement I go through will be an asset to my life.