Use Python decorator to curry functions

It’s been a while since the last time I wrote about Python. This morning, I was listening to a podcast on my way to work. They were discussing functional programming and dynamic languages…I learned Python before I went into Computer Science, and then I learned about functional programming and through learning of Scala and Clojure, my functional programming concepts have been enriched. As I was listening, it suddenly appeared to me that there isn’t a way in Python to curry a function. Not that it’s critical to everyday development, but wouldn’t it be neat if I can curry a function in Python?

Then the hosts of the podcast discussed how dynamic languages are so flexible that you can pretty much do anything to it. “You can take a function as parameter, return a function from a function, and so on.” Hey, isn’t that what Python’s decorator can do? I learned decorators before, but I haven’t used it beyond the scope of creating properties and certainly haven’t written any decorators. I thought this would be a good exercise for learning decorators.

Here’s a simple example of what function currying: suppose you have a method

def add(x,y):
  return x+y

Then calling add(1,2) should be the same as add(1)(2). add(1) is what they call a partially applied function. It’s a function that takes one parameter.

Our goal here is to write a decorator “curried” that takes a function with n parameters and transform it in a way that can be applied n times and get the final result.

We’ll start with unit tests first:

import unittest

class CurryTest(unittest.TestCase):

	def test_with_no_args(self):
		@curried
		def do_nothing():
			return ""
		self.assertEquals("", do_nothing())

	def test_with_int_args(self):
		@curried
		def add_int(x,y):
			return x+y
		self.assertEquals(3, add_int(1)(2))
	def test_with_str_args(self):
		@curried
		def add_str(x,y):
			return "%s%s"%(x,y)
		self.assertEquals("ab", add_str("a")("b"))

So we make sure that a currying on a function takes no parameter is valid but should be a pass through, and also the “curried” decorator can be applied to any function with arguments (excluding positional arguments and keyword arguments)

A decorator is simply a function that takes a function as parameter:

def curried(fn):
  pass

and @curried is simply a syntactic sugar for:

def fn(...): ...
fn=curried(fn)

So, now we can write “curried” decorator.
To make the test for function with no argument pass, in curried() function, we can test to see if fn has arguments. Python’s standard library provides inspect.getargspec method:

def curried(fn):
  argspec = inspect.getargspec(fn)
  if len(argspec.args)==0:
    return fn
  else:
    # later

Now the first test passes.

For the other two cases, here’s the strategy. In Python, when a class defines __call__ method, the instance of that class is said to be “callable”. For instance:

class A(object):
  def __call__(self, arg):
    return arg

f=A()
f("echo")  # this gives you "echo"

This is very similar to Scala’s apply() function. Now that we have this in our inventory, we can define a `PartialFunction` class, take all the required parameters of the original function, and allow them to be applied one at a time. So the __call__ method of PartialFunc will look like this:

def __call__(self, value):
  # Xxx

If all the required parameters are passed in, PartialFunc should evaluate the original function with the complete argument list. Otherwise, PartialFunc stores the parameter in an instance variable, and returns itself.

Here’s the complete code:

class PartialFunc(object):
	def __init__(self, fn, argspec):
		self.fn = fn
		self.argspec = argspec 
		self.args = []

	def __call__(self, value):
		self.args.append(value) 
		if len(self.args) == len(self.argspec.args):
			arglist = ",".join(["self.args[%d]"%i for i in range(0, len(self.args))])
			return eval("self.fn(" + arglist + ")")
		else:
			return self

and the curried decorator:

def curried(fn):
	argspec = inspect.getargspec(fn)
	if len(argspec.args) == 0:
		return fn
	else:
		return PartialFunc(fn, argspec)

It’s pretty straightforward. When the parameters are complete, I construct a python statement that calls the original function with the complete argument list, and then pass the statement into an eval statement. I know evals are evil, but I can’t find a way in Python to dynamically change the signature of the original method and make it accept a variable length argument (varargs).

So this is it. It’s quite simple. Python methods can have varargs and keyword args, the situation gets a little more complicated. The thing is, both varargs and keyword args are not mandatory, so it’s hard for the curried function to know whether the argument list has been completed…Also, if you take default values into account, it could get even more complicated.

Advertisements

My impression on Scala so far

I’ve been exploring Scala on and off for some time now. Here’s my highly subjective and very limited impression of Scala.

What I like about Scala:

0) It’s statically-typed language.

That’s right! I don’t care what you ninjas say. As much as I love dynamic languages, I just prefer statically typed language for big projects. The benefit of having type information is enormous for a project with a large code base. I know you have to write unit tests for it anyways, but when 80% of your unit test code is checking types, it’s just counter-productive.

1) The ability to combine functional programming style with OOP.

This is what Scala is known for – a hybrid language that boasts the better in both OOP world and FP world. I’m not a functional programming purist, but I think Scala did a good job blending the functional programming elements into OOP. Writing Scala programs, I find myself much more empowered to be able to choose from both styles where I see fit. The ability to pass functions around and apply high order functions can significantly reduce boilerplate code and visual clutter.

2) The ability to create your own control flow (well, sort of).

I had a blog post about how to use function curry to create a customized control flow. This is very empowering and yet, it’s not an one-off syntactic sugar that’s sprinkled randomly into the language like some other language would do. Customization of control flow is a result of combining generic language features (such as currying, by-name parameter, implicit conversion, etc).

3) Object instead of static

Scala does away with the Java static keyword. Instead, it provides “object” keyword to define a class that only has one instance. It has the same power as singleton design pattern, without the extra boilerplate code. (if you think implementing singleton in Java is simple, think again. Especially in the multi-threaded context). Also, you can use the same construct to define “companion” objects, which can be used to implement the factory pattern, and put in miscellaneous methods, implicit conversion methods and so on. Extremely powerful yet elegant.

4) Operator overloading (well, sort of).

OK, it’s not exactly operator overloading – Scala allows many non-alpha-numeric characters in the method name, so operators are essentially methods. duh! if you think about it…why should + be treated any different from “add()”? Being able to use special characters in method names makes code easy to understand. However, I’m a bit disappointed that question mark (?) cannot be used in method names…One of the things I liked about LISP is you can define a function (odd? (x) (…)) and readers immediately know this function returns a boolean. I don’t have to ponder whether I should name it isOdd() or hasChildren(). Anyway…

5) Traits

In most cases, multiple inheritance is manageable. Seems Java threw the baby out with the bath water – rejecting multiple inheritance completely. It’s ironic that the AOP guys cracks open Java classes in the byte code level to do the “mixins”…With Scala, this seems to be natural.

6) Built-in parser-combinator library

It’s easy to do some non-mission critical parsing using the Scala standard library. The library itself is implemented as an internal DSL such that writing parser rules feels like writing EBNF directly.

Here’s what’s not so hot for me so far:

1) The generics still is obtrusive sometimes.

That said, however, it’s vastly superior than Java’s generics system with far better type inferencing. But sometimes, you still have to write down a lot of types especially on a parameterized method. What’s worse is because Scala is more strict than Java wrt types, you cannot ignore generic types the way you can in Java code. Moreover, because Scala runs on JVM, and because of type-erasure, you still can’t write code such as val t = new T()…although we can’t blame this on Scala.

2) The collection library.

For each collection data structure, Scala has the mutable implementation, immutable implementation, and the native Java Collection Library. If you’re writing code that calls Java library, very often you have to wrap JCL lists/maps/sets into the Scala ones, because you want to use the nice functional features Scala provides. I’m not smart enough to know a better solution to this, I’m just merely pointing out a sore spot. Clojure has a very clever and elegant way of integrating JCL data structures into Clojure code, but since Clojure and Scala are vastly different, I don’t know how relevant this is.

3) Start up time

One of Scala’s goal is to be able to both scale up and scale down. Scala programs can be run as scripts by the interpreter, but it can’t compete with Python or Ruby in terms of development turn-around time in terms of scripting. Again, this is because of the dreadful JVM startup process.

4) Lack of good tooling

The best Scala development environment I found so far is the Scala plugin for IntelliJ. It’s still not great for debugging, but for coding and integration with Maven and such, it’s great. The Eclipse plugin lags significantly, despite having built a brand new website and seemingly having more resource pour into it…I know I shouldn’t be overly critical of a community effort, but it’s just frustrating that it can’t even do auto import of classes…

Overall, I think Scala really has the potential to become a big name in the programming language landscape. It can do everything Java can and does better. It has so much features that Java doesn’t dare to add to keep backward compatibility or keep the language dead simple or politics or whatever reason…

Scala is not a hype, but in order to be adopted more rapidly, I think it needs to

1) Stablize the language and core library.

2) Tooling, tooling, tooling! Seriously.

Use function currying to reduce repetition and make code clean

Lately, I’ve been writing a parser using the Scala parser-combinator framework to parse some saves from a game. As a responsible programmer (:P), I write unit tests for each rule. However, I found myself having to write the following code over and over again:

@Test
def testRule1() {
  parserRule.apply(new CharSequence("someInput")) match {
    case Success(result, _) => {
      assertEquals("expected", result)
      /* other asserts if the result is a collection of something else */
      }
    case NoSuccess(msg, _) => fail(msg)
  }
}

It’s worth noting that you cannot pass in a string value to a Parser. A parser expects a CharSequence CharSequenceReader object. Luckily, with Scala, you can define implicit conversion to turn a string into a CharSequence CharSequenceReader:

implicit def str2charSeqReader(v:String) = new CharSequenceReader(v)

Another goody Scala offers is that obj.apply(arg) method can be invoked by obj(arg), so the above code becomes:

/* ... */
parserRule("someInput") match {
/* ... */
}

It’s much cleaner.

However, the matching part is not clean still. It’s not that bad for a single test case, but it’s hardly a good practice to copy/paste it all over the test code base. We need to refactor this. The first thing comes to mind is to extract the matching part into a separate method, and let the caller provide the assertion part. Because Scala is a functional language, we can declare the method takes a function as a parameter, instead of using the delegate pattern, creating another class to encapsulate the method to be called, as you would do in Java:

def tryMatch[T](pr:ParseResult[T], f:(T)=>Unit) {
  pr match {
    case Success(r, _) => f(r)
    case NoSuccess(msg, _) => fail(msg)
  }
}

Now the consumer (the test code) looks like this:

@Test
def test1() {
  tryMatch(rule1("input"), (result)=>{assertEquals("expected",result)})
}

It’s better, except that when you have multiple assert* methods need to be called, it quickly becomes ugly:

@Test
def test2() {
  tryMatch(listRule("1,2,3,4"), (result)=>{assertEquals(1,result(0));assertEquals(2,result(1));assertEquals(3,result(2));assertEquals(4,result(3))})
}

To do better, we can use function currying. I read about function currying before, but not until now did it dawn on me that I can use this technique to make my code look cleaner. Basically, currying means that a function with n parameters is the same as the function being applied one parameter at a time in succession. e.g., function add(x1:Int,x2:Int,…,xn:Int) adds all its parameters. add(1,2)=[add(1)](2), where add(1) becomes a partially applied function that takes an integer and returns an integer. Then the partial function is applied to the second parameter which gives the eventual result.

This is all very theoretical, but the practical implication is that we can transform tryMatch() method shown above from a method with 2 parameters into a curried function. Why? Because with Scala, when you have a method that only takes a single parameter (as is the case with the partially applied function), you can replace parentheses with curly brackets! Let’s see how it works:

def tryMatch[T](pr:ParseResult[T])(f:(T)=>Unit) {
  pr match {
    case Success(r, _) => f(r)
    case NoSuccess(msg, _) => fail(msg)
  }
}

Basically, we break the two parameter definitions. This creates a PartialFunction in Scala. No other code needs to be changed at all. Now we can call tryMatch like this:

@Test
def testList() {
  tryMatch(listRule("1 2 3")) {
    result => {
      assertEquals(1, result(0))
      assertEquals(2, result(1))
      assertEquals(3, result(2))
    }
  }
}

This is like creating your own control flow (like try-catch-finally). Ain’t that neat?

Finding Happy Numbers using Scala

The problem was posted on Programming Praxis. The algorithm itself is pretty straightforward, anyone can do it with a few if/else/fors, but to coerce myself to think functionally, I decide to practice writing it in Scala.

A number is a happy number if the sum of square of its digits eventually arrive at 1. For example, 7=>72=49=>42+92=97=>92+72=>130=12+32+02=10=>12+02=1, so 7 is a happy number. 17 is not a happy number because by applying the above process, it goes into a loop.

Step1: calculating the sum of squares of a number

To get a list of numbers from a given number, we can first convert the number into a string, and then map every character of the string to its corresponding integer value. A more mathematical way is to divide the number continually by 10 until the original number becomes 0, adding the remainder to a list each time…The first method is easier to visualize, so here it goes:
n.toString.toCharArray.map{digit=>Integer.valueOf(""+digit)}

It’s quite wordy but I’ve yet to find a better way. I’m pretty sure there is so if you know, do inform me.
Now I need to square each item in the list. So modify the above statement:
n.toString.toCharArray.map{digit=>Math.pow(Integer.valueOf(""+digit).doubleValue,2.0)}
One thing bothers me is that I have to explicitly call someInt.doubleValue…I thought Scala does the implicit conversion for me? Then I realized Integer.valueOf(…) gives me java.lang.Integer, not scala.lang.Int. So I have to write a implicit conversion function myself:

implicit def integer2double(i:Integer):Double = i.doubleValue

Now I can get rid of the hideous someInt.doubleValue.
Since we’re doing implicit conversion already, why not just implicitly convert a numeric character to a double so that it can be accepted by Math.pow?

implicit def char2double(ch:Char):Double = Integer.valueOf("" + ch).doubleValue

Now the code can be shortened to

n.toString.toCharArray.map { digit => Math.pow(digit, 2) }

Isn’t that sweet? Implicit conversion is cool but it’s easy to get carried away and do everything implicit, which makes the code hard to maintain, so there gotta be a balance somewhere. In the scope of this small exercise, I guess it’s OK to use it.

Now that we have the squares of individual digits in a list, we can calculate the sum by reducing or folding it:

((n.toString.toCharArray.map { digit => Math.pow(digit, 2) }).foldLeft(0.0) { _ + _ }).toInt

{_ + _} seems a lot like line noise. The underscore converts the statement into a closure. It’s a shortcut for (a,b)=>a+b. It’s succinct yet should be used sparingly.

Step2:return

lol, yeah, no extra fluff is needed. We can capture the constraints in one return statement, but before that, I need to decide on what states I need to carry on from each recursive step. (You know I’m going to use recursion, don’t you? :P)

There are a couple of states involved:
1) the limit. It’s the number of steps we allow before the final “1” is reached. It took number seven 5 steps to reach 1. This variable is the cut-off: if the given number can’t reach 1 before the limit, it’s considered unhappy (return false)
2) the current number of steps
3) the set of numbers already appeared. In the case of seven, the set is {49 97 130 10}. This set is used to determine if the calculations fall into a loop. If the current calculated number appears in this set, the original number is unhappy.

So here’s our method:

def isHappyNumber(n:Int, limit:Int, numOfTries:Int, alreadySeen:Set[Int]):Boolean = {
val sos = ((n.toString.toCharArray.map { digit => Math.pow(digit, 2) }).foldLeft(0.0) { _ + _ }).toInt
return sos == 1 ? true : (!alreadySeen.contains(sos) && numOfTries+1 <= limit && isHappyNumber(sos, limit, numOfTries+1, alreadySeen+sos))
}

The second line basically says: when sos (sum of squares) is 1, return true, otherwise, is the number already in the set of numbers seen during the calculation?, if not, does the number of calculation exceed the limit? if not, repeat the calculation, with sos being the “original” number, increase the counter and put sos in the alreadySeen set.

Note that this is a tail recursive function, we can add @tailrec annotation to the method to let the compiler optimize it – turn the recursion into a loop so that it won’t grow in stack.

Now that we’ve had the body of the function, we can write an overload method that provides initial values:

def isHappyNumber(n:Int):Boolean = isHappyNumber(n, 10, 0, Set[Int]())

To find all happy numbers between 1 and 100:

println (1 to 100 filter { isHappyNumber(_) })

Launch Windows Explorer and goto the current working directory in shell

I like working in the command prompt – it’s fast and you can do batch processing, etc, but sometimes, I need a GUI (Windows explorer) to explore files in the current directory. I know I can type “explorer.exe” on the command line to launch Windows explorer, but can it automatically go to the directory I’m working in? The answer is yes, although hardly ever advertised:

Now, we need to find a way to find out the current working directory. On the command prompt, it’s stored in the environment variable %CD%, so combine the two:

explorer /root,%CD%

and it works beautifully.

Put this in a .bat file and put it on your path, you can launch Windows explorer anywhere and have it go directly to the folder you’re in on the command line!

Google IO 2010 is here!

This year’s Google IO conference is finally here! I don’t want to sound like a fanboy but I benefited a lot from last year’s IO sessions and looking at this year’s session list, I’m sure I won’t be disappointed.

This year’s keynote speech is an overview of Google’s strategy to bring web to the next level. It seems HTML5 is rapidly gaining grounds. Although the specification is not finalized yet, many of the new features are already implemented in most modern browsers, such as AppCache, WebSQL, LocalStorage, Web Worker, and WebSocket. GWT is still in a very strategic place in Google’s plan, and this year, they partnered with Spring to make it dead easy to setup a GWT-Spring application from ground up. I tried it myself shortly after I saw the keynote and it brought a lot of the agile element into Java development – scaffolding, ORM, test driven, fast reloading time in development and project lifecycle management with Maven. It all looked very good. I certainly will look more into this development setup.

GWT code split is awesome

For the past few days, I’ve been working on code splitting our existing GWT application. The application is fairly big with about 20k loc (excluding javadoc, comments and tests). The download size (with obfuscated mode) is about 1.5M, and with pretty mode, is a whooping 10+M. Obviously this is not acceptable.

GWT2 provides developer guided code splitting feature. It sounds complicated and under the hood, it may very well be (involves a lot of graph theories to figure out dependencies and so on) but from the user’s point of view, it’s very easy. You just have to wrap your potentially big operations in a GWT.runAsync() call.

However, since our application is using GWT best practices (dependency injection, MVP pattern), it’s not as straightforward as GWT doc describes. A bit of digging on the internet leads me to this page http://code.google.com/p/google-gin/issues/detail?id=61

Gin patch

Someone contributed a patch to gin, which made split points transparent to the user of gin. The presenters that aren’t needed initially can be wrapped in an AsyncProvider<T> instance – which by the blessing of deferred binding, translates into a GWT.runAsync call in the generated code. The patch hasn’t been accepted into gin’s trunk yet, but it’s fairly easy to apply the patch and rebuild. A huge thanks to fazal.asim who hacked and contributed this patch.

Results

The result of code splitting is encouraging – with very little structural change, we’re able to reduce the initial download size to 29% of the total size:

Changes

I mentioned we needed very little structural change, but we did have to change something around. This is because with code splitting, the presenters that are split out from the initial download are not instantiated until they’re used/downloaded. This means you cannot put logic in their constructors, and responding to place change has to be initiated by the container presenters.

Improvements

Code splitting is awesome. However, if I’m allowed to voice a complaint, the report compiling time is just excruciating! For our application, it usually takes about 10 minutes to compile SOYC report – maybe a few minutes too long given the specs of my machine isn’t too bad (Quad Core, 3G memory). Also, the compiled SOYC report takes up 600M of hard disk space! Ouch! Maybe instead of emitting HTML pages, they can make SOYC report a JS application, with data being encoded in JSON format?

Anyway, this doesn’t take anything away from the awesome job GWT team has done for developers.

Follow-up

Thanks to AsyncProxy, which provides a blocking (synchronous) interface while utilizing GWT.runAsync. This way, I’m able to build a view proxy that implements the same interface while keeping the real view components out of the initial call graph. The result of this, is a further reduction of the initial download size.

The initial download size is 13.77% of the total code size! Sweet!

Building a GAE+GWT application using the best practices, Part 5

This is the fifth installment of the series “Building a GAE+GWT application using the best practices“.

In the last post of the series, I’ve shown how to setup a client side GIN+MVP project. This post will discuss the actual building of the UI and GWT-RPC web service calls using GWT-Dispatch on the client side.

Use cases

Before we dive into the code, let’s go over again the use cases our little app has.

  • When the UI loads, it should automatically fetch the latest saved rate
  • Also, the table below the latest rate should display the 10 most recent rates stored in the data store
  • The “Refresh” button does the above two steps
  • The “Fetch latest” button fetches the rate from the bank website, store it in the data store, and insert the latest rate in the recent rate table

MVP pattern

For traditional desktop application, there’s the MVC (Model-View-Control) pattern that’s been in existence for over 20 years, since the Smalltalk days. However, the responsibility separation between the view and controller hasn’t been well defined and over the years, people have been arguing about what whether the business logic should be entirely in controller.

With the MVP (Model-View-Presenter) pattern, the view itself doesn’t contain any logic. The presenter manipulates the view according to business logic. Therefore, the pattern is also called “supervising controller” or “passive view”. This brings a huge benefit because now unit tests only have to deal with presenters, and mock out the view through the display interface. With this setup, the presenter unit tests can be run entirely in JVM. Otherwise, the tests need to involve GWT widgets, which can only be tested with GWTTestCase, which takes a lot longer to run.

MainView

Here I’m using the new GWT2 UiBinder feature. UiBinder makes developing UI less boring.

MainView.ui.xml

<!DOCTYPE ui:UiBinder SYSTEM "http://dl.google.com/gwt/DTD/xhtml.ent">
<ui:UiBinder xmlns:ui="urn:ui:com.google.gwt.uibinder"
	xmlns:g="urn:import:com.google.gwt.user.client.ui">
	<ui:style>
		.rateDisplay {
			font-size: x-large;
			font-weight: bolder;
		}
		
		.mainPanel {
			padding: 10px;
		}
	</ui:style>

	<ui:with type="ratechecker.client.bundles.RateCheckerClientBundle"
		field="bundle"></ui:with>
	<g:DecoratorPanel height="200px">
		<g:VerticalPanel styleName="{style.mainPanel}" verticalAlignment="ALIGN_TOP">
			<g:Label>Latest Selling Rate</g:Label>
			<g:Image resource="{bundle.loading}" visible="false"
				ui:field="loading"></g:Image>
			<g:Label ui:field="rateDisplay" styleName="{style.rateDisplay}"></g:Label>
			<g:HorizontalPanel>
				<g:Button ui:field="fetchLatest">Fetch Latest</g:Button>
				<g:Button ui:field="refresh">Refresh</g:Button>
			</g:HorizontalPanel>

			<g:Label>Recent rates</g:Label>
			<g:FlexTable ui:field="rateTable">
			</g:FlexTable>
		</g:VerticalPanel>
	</g:DecoratorPanel>
</ui:UiBinder> 

UiBinder can inter-operate with GWT2 ClientBundle. If you have used GWT1.x’s ImageBundle, ClientBundle is similar to that, except now with ClientBundle, other client resources are able to be bundled such as CSS stylesheet and external text resource.

MainPresenter.Display

The display interface is the only thing presenter knows about the UI, and the presenter operates/manipulates UI only through the display interface.

The display interface can be standalone, but I find it’s much more convenient to have it as an inner interface inside the presenter class.

	public interface Display extends WidgetDisplay {
		HasText getRateDisplayLabel();
		HasClickHandlers getFetchLatest();
		HasClickHandlers getRefresh();

		void setEnabledFetchLatestButton(boolean isEnabled);
		void setShowLoadingCurrentRate(boolean isLoading);
		/**
		 * Add the rate to the recent rate table.
		 * @param rate
		 * 		The {@link Rate} object
		 * @param toHead
		 * 		<code>true</code> - rate is inserted to the beginning of the table
		 * 		<code>false</code> - rate is appended to the end of the table
		 */
		void addToRecentRates(Rate rate, boolean toHead);
		/**
		 * Clear the recent rates table.
		 */
		void clearRecentRates();
	}

Here we use the “characteristic interface” of the UI elements as return type as they can be mocked. For things that cannot be returned as characteristic interfaces (like FlexTable), we provide methods for the presenter to manipulate the state of the UI objects (such as clearRecentRates()).

MainView.java

Now we have the display interface, we can map these interface methods onto our UI.

package ratechecker.client.mvp;

public class MainView extends Composite implements MainPresenter.Display {

	private static MainViewUiBinder uiBinder = GWT
	.create(MainViewUiBinder.class);

	interface MainViewUiBinder extends UiBinder<Widget, MainView> {
	}

	@UiField
	Button refresh;

	@UiField
	Button fetchLatest;

	@UiField
	Label rateDisplay;

	@UiField
	FlexTable rateTable;

	@UiField
	Image loading;

	private final DateTimeFormat _dateTimeFormat;

	@Inject
	public MainView(final DateTimeFormat dateTimeFormat) {
		_dateTimeFormat = dateTimeFormat;
		initWidget(uiBinder.createAndBindUi(this));
	}

	@Override
	public HasClickHandlers getFetchLatest() {
		return fetchLatest;
	}

	@Override
	public Widget asWidget() {
		return this;
	}

	@Override
	public HasText getRateDisplayLabel() {
		return rateDisplay;
	}

	@Override
	public void setEnabledFetchLatestButton(final boolean isEnabled) {
		fetchLatest.setEnabled(isEnabled);
	}

	@Override
	public void addToRecentRates(final Rate rate, final boolean toHead) {
		final int newRowIdx = toHead ? 0 : rateTable.getRowCount();
		rateTable.insertRow(newRowIdx);
		rateTable.setText(newRowIdx, 0, _dateTimeFormat.format(rate.getTimeFetched()));
		rateTable.setText(newRowIdx, 1, String.valueOf(rate.getRate()));
	}

	@Override
	public void clearRecentRates() {
		rateTable.removeAllRows();
	}

	@Override
	public HasClickHandlers getRefresh() {
		return refresh;
	}

	@Override
	public void setShowLoadingCurrentRate(final boolean isLoading) {
		loading.setVisible(isLoading);
		rateDisplay.setVisible(!isLoading);
	}

	@Override public void startProcessing() { }

	@Override public void stopProcessing() { }
}

A lot of this is boilerplate code to satisfy both UiBinder and GWT-presenter.Display interface. Ideally, the VIew shouldn’t do too much, if any at all. Realistically, this is harder to achieve.

MainPresenter

Finally, we can show you the presenter code.


public class MainPresenter extends WidgetPresenter<MainPresenter.Display> {

	private final DispatchAsync _dispatch;

	private final ILog _logger;

	@Inject
	public MainPresenter(final Display display, final EventBus eventBus, final DispatchAsync dispatch, final ILog logger) {
		super(display, eventBus);
		_dispatch = dispatch;
		_logger = logger;
	}

	@Override
	protected void onBind() {
		registerHandler(display.getFetchLatest().addClickHandler(new ClickHandler() {

			@Override
			public void onClick(final ClickEvent event) {
				fetchSellingRate();
			}

		}));

		registerHandler(eventBus.addHandler(RateFetchedEvent.TYPE, new RateFetchedHandler() {

			@Override
			public void onRateFetched(final Rate rate) {
				saveRate(rate);
			}

		}));

		registerHandler(eventBus.addHandler(RateSavedEvent.TYPE, new RateSavedHandler() {

			@Override
			public void onRateSaved(final Rate rate) {
				display.addToRecentRates(rate, true);
			}

		}));

		registerHandler(display.getRefresh().addClickHandler(new ClickHandler() {

			@Override
			public void onClick(final ClickEvent event) {
				getLatestSavedRates();
			}

		}));

		getLatestSavedRates();
	}

	void getLatestSavedRates() {
		display.setShowLoadingCurrentRate(true);

		final GetRates getRates = new GetRates();

		_dispatch.execute(getRates, new AsyncCallback<GetRatesResult>() {

			@Override
			public void onFailure(final Throwable caught) {
				display.setShowLoadingCurrentRate(false);
				_logger.error("Unable to get saved rates: " + caught.getMessage());
			}

			@Override
			public void onSuccess(final GetRatesResult result) {
				display.setShowLoadingCurrentRate(false);
				display.clearRecentRates();

				for (final Rate rate : result.getRates()) {
					display.addToRecentRates(rate, true);
				}

				// Put the latest rate in the box
				if (result.getRates().size() > 0) {
					final Rate latestRate = result.getRates().get(0);
					display.getRateDisplayLabel().setText(String.valueOf(latestRate.getRate()));
				}
			}
		});

	}

	void fetchSellingRate() {
		display.setShowLoadingCurrentRate(true);
		final CheckRate checkRate = new CheckRate(RateType.Selling);
		_dispatch.execute(checkRate, new AsyncCallback<CheckRateResult>() {

			@Override
			public void onFailure(final Throwable caught) {
				display.setShowLoadingCurrentRate(false);
				_logger.error("Unable to fetch rate: " + caught.getMessage());
			}

			@Override
			public void onSuccess(final CheckRateResult result) {
				display.setShowLoadingCurrentRate(false);
				// enable the fetch button
				display.setEnabledFetchLatestButton(true);
				display.getRateDisplayLabel().setText(String.valueOf(result.getRate().getRate()));
				eventBus.fireEvent(new RateFetchedEvent(result.getRate()));
			}

		});

		// disable the fetch button until RPC succeeds
		display.setEnabledFetchLatestButton(false);
	}

	void saveRate(final Rate rate) {
		final SaveRate saveRate = new SaveRate(rate);

		_dispatch.execute(saveRate, new AsyncCallback<SaveRateResult>() {

			@Override
			public void onFailure(final Throwable caught) {
				_logger.error("Unable to save rate: " + caught.getMessage());
			}

			@Override
			public void onSuccess(final SaveRateResult result) {
				eventBus.fireEvent(new RateSavedEvent(rate));
			}

		});

	}

	@Override protected void onPlaceRequest(final PlaceRequest request) { }

	@Override protected void onUnbind() {}

	@Override public void refreshDisplay() {}

	@Override public void revealDisplay() {}

	@Override public Place getPlace() { return null; }
}

In the binding process, the event handlers are attached to the view components. MainPresenter.bind() was explicitly called by AppPresenter.go(). This is a simple application with one presenter. If there are more presenters, AppPresenter needs to manage the state of these sub-presenters: if they’re active, the bind() method is called. If the presenter is no-longer active, the presenter’s unbind() method should be called to un-attach the handlers, so they don’t interfere with the event handlers that are currently in the active presenter.

The presenter is also responsible for making web service calls and deal with the returns. To call GWT-RPC web service using GWT-dispatch, we inject a DispatchAsync, which is an asynchronous counter part of the DispatchServlet introduced a few posts ago.

To call a web service, we simply construct an action object with required parameters and pass it in DispatchAsync.execute() and expect an AsyncCallback of type result that’s coupled with the action. (remember each action has a coupled result type). Also, in this application, every action has a related event to indicate whether the action is successful. The event is thrown onto the event bus, so any interested party can handle that. The main benefit of using event bus is that my web service calls don’t have to be coupled with the subsequent actions. For example, saveRate() method is responsible for making the web service calls, but the subsequent action (adding the saved rate to the recent rate table) isn’t part of saveRate() method, and it shouldn’t be. If in the future, some other actions need to be carried out when a rate is saved, we just have to add the action in the RateSavedHandler, and indeed, if another part of the UI (not visible by main presenter) need to do something after the rate is saved, that presenter only needs to handle that event in there without affecting saveRate() method at all.

For view the full source code, take a look at the project I created on GitHub.

EDIT:For any Google App Engine experts out there happened to be reading this post, I’m having trouble with the performance of this simple app. Seems like the data store is taking way too much time executing my query. Initially I thought it was because URL fetch is slow, but I recently added a property in Rate entity to track the time spent on fetching the URL and every request takes less than 1 second. However, the GetRates action takes a long time to return (usually ~3 to 5 seconds, sometimes even over 10 seconds). It’s a simple query ordering on a single property so no complex index is needed. So I’m wondering what’s wrong here.

Building a GAE+GWT application using the best practices, Part 4

This is the fourth installment of the series “Building a GAE+GWT application using the best practices“.

In the last blog post, we went over how to write GWT-RPC handlers using GWT-dispatch and dependency injection (Guice). This section, we’re going to see how the client side is set up.

Dependencies

We need the following dependencies

They need to be on the classpath when you compile your GWT code, but not under the war directory like the server dependencies need to be.

Module definition

The first step is to declare the inherited GWT modules in the module XML file:
RateChecker.gwt.xml

<?xml version="1.0" encoding="UTF-8"?>
<module rename-to="ratechecker">
    <inherits name="com.google.gwt.user.User" />
    <inherits name="com.google.gwt.inject.Inject" />
    <inherits name="net.customware.gwt.dispatch.Dispatch" />
    <inherits name="net.customware.gwt.presenter.Presenter" />
    <inherits name="com.allen_sauer.gwt.log.gwt-log-DEBUG" />
    <inherits name="com.google.gwt.user.theme.chrome.Chrome"/>

    <entry-point class="ratechecker.client.RateChecker" />

    <source path="client" />
    <source path="shared" />

</module>

Here we specify the explicitly the packages need to be included for compiling – “client” and “shared”. If not specified, GWT will by default compile every source file under the client package.

Create a Gin Module

Gin and Guice implements the same API (or rather, Gin has the same interface as Guice), but Gin uses GWT deferred binding “magic”. Similar to what we have on the server side, on the client side, we start by defining our module:
RateCheckerClientModule:

public class RateCheckerClientModule extends AbstractPresenterModule {

	public RateCheckerClientModule() {
	}

	@Override
	protected void configure() {

		bind(EventBus.class).to(DefaultEventBus.class).in(Singleton.class);
		bind(PlaceManager.class).in(Singleton.class);

	}
}

To start up, we bind EventBus and PlaceManager in the singleton scope. They’re both provided by GWT-mvp library.

AppPresenter

There are different ways to facilitate the MVP pattern but the way I find the most convenient is to have an AppPresenter manage all subsequent presenters. The view the AppPresenter represents is the RootPanel of GWT.

public class AppPresenter {

	private HasWidgets _container;

	private final MainPresenter _mainPresenter;


	@Inject
	public AppPresenter(final MainPresenter mainPresenter) {
		_mainPresenter = mainPresenter;
		_mainPresenter.bind();
	}


	public void go(final HasWidgets container) {
		_container = container;
		_container.clear();
		_container.add(_mainPresenter.getDisplay().asWidget());
	}
}

Here, MainPresenter is the actual UI. The go() method of AppPresenter is for the module entry point to call when the module first initializes. We need to add the bindings to the client module:

        ...
	@Override
	protected void configure() {

		bind(EventBus.class).to(DefaultEventBus.class).in(Singleton.class);
		bind(PlaceManager.class).in(Singleton.class);
		bind(ILog.class).to(GwtLogAdapter.class).in(Singleton.class);
		bind(AppPresenter.class);

		bindPresenter(MainPresenter.class, MainPresenter.Display.class, MainView.class);
	}
        ...

Dependencies

We need the following dependencies

They need to be on the classpath when you compile your GWT code, but not under the war directory like the server dependencies need to be.

Module definition

The first step is to declare the inherited GWT modules in the module XML file:
RateChecker.gwt.xml

<?xml version="1.0" encoding="UTF-8"?>
<module rename-to="ratechecker">
    <inherits name="com.google.gwt.user.User" />
    <inherits name="com.google.gwt.inject.Inject" />
    <inherits name="net.customware.gwt.dispatch.Dispatch" />
    <inherits name="net.customware.gwt.presenter.Presenter" />
    <inherits name="com.allen_sauer.gwt.log.gwt-log-DEBUG" />
    <inherits name="com.google.gwt.user.theme.chrome.Chrome"/>

    <entry-point class="ratechecker.client.RateChecker" />

    <source path="client" />
    <source path="shared" />

</module>

Here we specify the explicitly the packages need to be included for compiling – “client” and “shared”. If not specified, GWT will by default compile every source file under the client package.

Create a Gin Module

Gin and Guice implements the same API (or rather, Gin has the same interface as Guice), but Gin uses GWT deferred binding “magic”. Similar to what we have on the server side, on the client side, we start by defining our module:
RateCheckerClientModule:

public class RateCheckerClientModule extends AbstractPresenterModule {

	public RateCheckerClientModule() {
	}

	@Override
	protected void configure() {

		bind(EventBus.class).to(DefaultEventBus.class).in(Singleton.class);
		bind(PlaceManager.class).in(Singleton.class);

		bind(AppPresenter.class);
		bindPresenter(MainPresenter.class, MainPresenter.Display.class, MainView.class);
	}

We’ll defer the introduction of MainPresenter, MainView and other UI related components to the next post.

Ginjector

Similar to “Injector” interface on the server side, the client side needs to define a Ginjector that act as a gateway for Gin managed object instances.

RateCheckerGinjector.java


@GinModules({RateCheckerClientModule.class, ClientDispatchModule.class})
public interface RateCheckerGinjector extends Ginjector {

	AppPresenter getAppPresenter();

}

Here the annotation @GinModules({…}) makes the instances managed by RateCheckerClientModule and ClientDispatchModule available for the ginjector. ClientDispatchModule binds DispatchAsync interface, which is what we will use to interface with the web service methods.

Entry Point

Finally, here’s the module entry point:

/**
 * Entry point classes define <code>onModuleLoad()</code>.
 */
public class RateChecker implements EntryPoint {

	RateCheckerGinjector _injector = GWT.create(RateCheckerGinjector.class);

	@Override
	public void onModuleLoad() {

		final AppPresenter appPresenter = _injector.getAppPresenter();
		appPresenter.go(RootPanel.get("root"));
	}

}

GWT.create(…) statement here creates the ginjector at runtime. Behind the scene, it generates a class (by the name of something like RateCheckerGinjector_Impl) that contains the code to instantiate the bound classes, and when Gin sees a @Inject annotation on a class’s constructor, it provides the instances with the correct scope from the dependency injection container (Ginjector) to the constructor so that the said class can be instantiated.

The onModuleLoad() method doesn’t do much. It simple binds the appPresenter with the RootPanel where the app’s UI is going to be displayed.

I know a lot of the concrete UI creation has been left out of this post, but hopefully it will become clearer once the next post is in.

Building a GAE+GWT application using the best practices, Part 3

This is the third installment of the series “Building a GAE+GWT application using the best practices“.

In this part of the series, we’re going to explore the designing of the web services for RateChecker and coding them using the command pattern from GWT-dispatch based on Ray Ryan’s presentation.

The big picture


To correctly implement web services using the command pattern, we first have to get the big picture. There are three “actors” involved in this: an action, a result and a handler.

Action

An action is used to store the parameters of the web service call (if any). For instance, a CheckRate action needs to know what type of rate the user is checking: buying rate? selling rate? currency?

Result

A result object stores the result (duh…) of the web service call. In the case of CheckRate method, the result is the rate object containing the details of the rate.

Handler

A handler is the actual “worker” that actually does the work of checking the rate. In this case, the check rate method fetches the posted rate page , parses the text (if needed) to get the rate information.

Dispatch and DispatchAsync

A Handler is executed on the server side (by the DispatchServlet we saw in the last post). On the client side, there’s a counterpart “DispatchAsync”, which is the asynchronous interface that the client code calls.

Implementing web service methods

Now that we have the big picture in place, we’re going to look into how to actually implement them.
The first step is to define a domain model. In this case, it’s our Rate class:

@PersistenceCapable(identityType=IdentityType.APPLICATION)
public class Rate implements Serializable {

	private static final long serialVersionUID = -4415279469780082174L;

	@PrimaryKey
	@Persistent(valueStrategy=IdGeneratorStrategy.IDENTITY)
	private Long id;

	@Persistent
	private RateType type;

	@Persistent
	private Date timeFetched;

	@Persistent
	private Double rate;

	public Rate() {
	}
        // ... getters and setters omitted
}
public enum RateType {
	Selling,
	Buying,
}

In our example application, we are going to define three simple web methods:

  • Check rate: use Url fetch to get the posted rate page and return a rate object from that.
  • Save rate: persist the rate object into the data store.
  • Get rate: get the rates from the data store.

Check rate

As we have shown in the big picture, every action needs three pieces: action, result (both in shared package, as they will be used by both the client and the server) and the handler (lives in the server package).

For CheckRate action, we need to specify the type of rate it needs to check. For simplicity, I’m always dealing with USD/CAD rate. The parameter here is only for whether to check for buying rate or selling rate.

public class CheckRate implements Action<CheckRateResult> {

	private static final long serialVersionUID = -1716760883016361503L;

	private RateType _type;

	public CheckRate() {
	}

	public CheckRate(final RateType type) {
		_type = type;
	}

	public void setType(final RateType type) {
		_type = type;
	}

	public RateType getType() {
		return _type;
	}

}

The result is designed to hold the returned Rate object.

public class CheckRateResult implements Result {

	private static final long serialVersionUID = -9099789297842581458L;

	private Rate _rate;

	public CheckRateResult() {
	}

	public CheckRateResult(final Rate rate) {
		_rate = rate;
	}

	public void setRate(final Rate rate) {
		_rate = rate;
	}

	public Rate getRate() {
		return _rate;
	}
}

Word of caution: because both action and result are serialized and sent over the wire as part of GWT-RPC call, they are required to have a default public constructor.

Now, on to the handler:

public class CheckRateHandler implements ActionHandler<CheckRate, CheckRateResult> {

	public static final String URL_BUY = "http://www.ingdirect.ca/en/datafiles/rates/usbuying.html";

	public static final String URL_SELL = "http://www.ingdirect.ca/en/datafiles/rates/usselling.html";

	public CheckRateHandler() {
	}

	@Override
	public CheckRateResult execute(final CheckRate action, final ExecutionContext ctx) throws ActionException {
		final CheckRateResult retval = new CheckRateResult();

		String strUrl = null;
		switch (action.getType()) {
		case Buying:
			strUrl = URL_BUY;
			break;
		case Selling:
			strUrl = URL_SELL;
			break;
		}

		try {
			final URL url = new URL(strUrl);

			BufferedReader br = null;
			try {
				br = new BufferedReader(new InputStreamReader(url.openStream()));

				final double dRate = Double.parseDouble(br.readLine());

				final Rate rate = new Rate();
				rate.setRate(dRate);
				rate.setType(action.getType());
				rate.setTimeFetched(new Date());

				retval.setRate(rate);

			} finally {
				if (br != null)
					br.close();
			}
		} catch (final MalformedURLException e) {
			e.printStackTrace();
			throw new ActionException(e);
		} catch (final IOException e) {
			e.printStackTrace();
			throw new ActionException(e);
		} catch (final NumberFormatException e) {
			e.printStackTrace();
			throw new ActionException(e);
		}

		return retval;
	}
        // ... other methods omitted
}

As you can see, the handler does the actual work of fetching the rate using URL Fetch service offered by Google App Engine.

The other two web method implementations are similar. You can follow the project on Github here. In the next section, I’m going to go over the building of the UI in GWT, as well as making AJAX calls from GWT to the server.