Use Python decorator to curry functions

It’s been a while since the last time I wrote about Python. This morning, I was listening to a podcast on my way to work. They were discussing functional programming and dynamic languages…I learned Python before I went into Computer Science, and then I learned about functional programming and through learning of Scala and Clojure, my functional programming concepts have been enriched. As I was listening, it suddenly appeared to me that there isn’t a way in Python to curry a function. Not that it’s critical to everyday development, but wouldn’t it be neat if I can curry a function in Python?

Then the hosts of the podcast discussed how dynamic languages are so flexible that you can pretty much do anything to it. “You can take a function as parameter, return a function from a function, and so on.” Hey, isn’t that what Python’s decorator can do? I learned decorators before, but I haven’t used it beyond the scope of creating properties and certainly haven’t written any decorators. I thought this would be a good exercise for learning decorators.

Here’s a simple example of what function currying: suppose you have a method

def add(x,y):
  return x+y

Then calling add(1,2) should be the same as add(1)(2). add(1) is what they call a partially applied function. It’s a function that takes one parameter.

Our goal here is to write a decorator “curried” that takes a function with n parameters and transform it in a way that can be applied n times and get the final result.

We’ll start with unit tests first:

import unittest

class CurryTest(unittest.TestCase):

	def test_with_no_args(self):
		@curried
		def do_nothing():
			return ""
		self.assertEquals("", do_nothing())

	def test_with_int_args(self):
		@curried
		def add_int(x,y):
			return x+y
		self.assertEquals(3, add_int(1)(2))
	def test_with_str_args(self):
		@curried
		def add_str(x,y):
			return "%s%s"%(x,y)
		self.assertEquals("ab", add_str("a")("b"))

So we make sure that a currying on a function takes no parameter is valid but should be a pass through, and also the “curried” decorator can be applied to any function with arguments (excluding positional arguments and keyword arguments)

A decorator is simply a function that takes a function as parameter:

def curried(fn):
  pass

and @curried is simply a syntactic sugar for:

def fn(...): ...
fn=curried(fn)

So, now we can write “curried” decorator.
To make the test for function with no argument pass, in curried() function, we can test to see if fn has arguments. Python’s standard library provides inspect.getargspec method:

def curried(fn):
  argspec = inspect.getargspec(fn)
  if len(argspec.args)==0:
    return fn
  else:
    # later

Now the first test passes.

For the other two cases, here’s the strategy. In Python, when a class defines __call__ method, the instance of that class is said to be “callable”. For instance:

class A(object):
  def __call__(self, arg):
    return arg

f=A()
f("echo")  # this gives you "echo"

This is very similar to Scala’s apply() function. Now that we have this in our inventory, we can define a `PartialFunction` class, take all the required parameters of the original function, and allow them to be applied one at a time. So the __call__ method of PartialFunc will look like this:

def __call__(self, value):
  # Xxx

If all the required parameters are passed in, PartialFunc should evaluate the original function with the complete argument list. Otherwise, PartialFunc stores the parameter in an instance variable, and returns itself.

Here’s the complete code:

class PartialFunc(object):
	def __init__(self, fn, argspec):
		self.fn = fn
		self.argspec = argspec 
		self.args = []

	def __call__(self, value):
		self.args.append(value) 
		if len(self.args) == len(self.argspec.args):
			arglist = ",".join(["self.args[%d]"%i for i in range(0, len(self.args))])
			return eval("self.fn(" + arglist + ")")
		else:
			return self

and the curried decorator:

def curried(fn):
	argspec = inspect.getargspec(fn)
	if len(argspec.args) == 0:
		return fn
	else:
		return PartialFunc(fn, argspec)

It’s pretty straightforward. When the parameters are complete, I construct a python statement that calls the original function with the complete argument list, and then pass the statement into an eval statement. I know evals are evil, but I can’t find a way in Python to dynamically change the signature of the original method and make it accept a variable length argument (varargs).

So this is it. It’s quite simple. Python methods can have varargs and keyword args, the situation gets a little more complicated. The thing is, both varargs and keyword args are not mandatory, so it’s hard for the curried function to know whether the argument list has been completed…Also, if you take default values into account, it could get even more complicated.

Advertisements

My impression on Scala so far

I’ve been exploring Scala on and off for some time now. Here’s my highly subjective and very limited impression of Scala.

What I like about Scala:

0) It’s statically-typed language.

That’s right! I don’t care what you ninjas say. As much as I love dynamic languages, I just prefer statically typed language for big projects. The benefit of having type information is enormous for a project with a large code base. I know you have to write unit tests for it anyways, but when 80% of your unit test code is checking types, it’s just counter-productive.

1) The ability to combine functional programming style with OOP.

This is what Scala is known for – a hybrid language that boasts the better in both OOP world and FP world. I’m not a functional programming purist, but I think Scala did a good job blending the functional programming elements into OOP. Writing Scala programs, I find myself much more empowered to be able to choose from both styles where I see fit. The ability to pass functions around and apply high order functions can significantly reduce boilerplate code and visual clutter.

2) The ability to create your own control flow (well, sort of).

I had a blog post about how to use function curry to create a customized control flow. This is very empowering and yet, it’s not an one-off syntactic sugar that’s sprinkled randomly into the language like some other language would do. Customization of control flow is a result of combining generic language features (such as currying, by-name parameter, implicit conversion, etc).

3) Object instead of static

Scala does away with the Java static keyword. Instead, it provides “object” keyword to define a class that only has one instance. It has the same power as singleton design pattern, without the extra boilerplate code. (if you think implementing singleton in Java is simple, think again. Especially in the multi-threaded context). Also, you can use the same construct to define “companion” objects, which can be used to implement the factory pattern, and put in miscellaneous methods, implicit conversion methods and so on. Extremely powerful yet elegant.

4) Operator overloading (well, sort of).

OK, it’s not exactly operator overloading – Scala allows many non-alpha-numeric characters in the method name, so operators are essentially methods. duh! if you think about it…why should + be treated any different from “add()”? Being able to use special characters in method names makes code easy to understand. However, I’m a bit disappointed that question mark (?) cannot be used in method names…One of the things I liked about LISP is you can define a function (odd? (x) (…)) and readers immediately know this function returns a boolean. I don’t have to ponder whether I should name it isOdd() or hasChildren(). Anyway…

5) Traits

In most cases, multiple inheritance is manageable. Seems Java threw the baby out with the bath water – rejecting multiple inheritance completely. It’s ironic that the AOP guys cracks open Java classes in the byte code level to do the “mixins”…With Scala, this seems to be natural.

6) Built-in parser-combinator library

It’s easy to do some non-mission critical parsing using the Scala standard library. The library itself is implemented as an internal DSL such that writing parser rules feels like writing EBNF directly.

Here’s what’s not so hot for me so far:

1) The generics still is obtrusive sometimes.

That said, however, it’s vastly superior than Java’s generics system with far better type inferencing. But sometimes, you still have to write down a lot of types especially on a parameterized method. What’s worse is because Scala is more strict than Java wrt types, you cannot ignore generic types the way you can in Java code. Moreover, because Scala runs on JVM, and because of type-erasure, you still can’t write code such as val t = new T()…although we can’t blame this on Scala.

2) The collection library.

For each collection data structure, Scala has the mutable implementation, immutable implementation, and the native Java Collection Library. If you’re writing code that calls Java library, very often you have to wrap JCL lists/maps/sets into the Scala ones, because you want to use the nice functional features Scala provides. I’m not smart enough to know a better solution to this, I’m just merely pointing out a sore spot. Clojure has a very clever and elegant way of integrating JCL data structures into Clojure code, but since Clojure and Scala are vastly different, I don’t know how relevant this is.

3) Start up time

One of Scala’s goal is to be able to both scale up and scale down. Scala programs can be run as scripts by the interpreter, but it can’t compete with Python or Ruby in terms of development turn-around time in terms of scripting. Again, this is because of the dreadful JVM startup process.

4) Lack of good tooling

The best Scala development environment I found so far is the Scala plugin for IntelliJ. It’s still not great for debugging, but for coding and integration with Maven and such, it’s great. The Eclipse plugin lags significantly, despite having built a brand new website and seemingly having more resource pour into it…I know I shouldn’t be overly critical of a community effort, but it’s just frustrating that it can’t even do auto import of classes…

Overall, I think Scala really has the potential to become a big name in the programming language landscape. It can do everything Java can and does better. It has so much features that Java doesn’t dare to add to keep backward compatibility or keep the language dead simple or politics or whatever reason…

Scala is not a hype, but in order to be adopted more rapidly, I think it needs to

1) Stablize the language and core library.

2) Tooling, tooling, tooling! Seriously.