Javatech

I recently blogged about Freemarker performing 10-20x slower than pure Java analogues. I got to thinking about our webapp and the fact that I wasn’t quite testing the correct scenario. In a web environment there might be numerous threads all running Freemarker templates simultaneously. So, I had to put the pure Java analogues up against the simple Freemarker template in a multithreaded test. I ran tests for each with 10,20,40,60 and 80 threads. The results are more impetus towards moving away from Freemarker for the most used and most basic cases ( input tags, labels, script includes ).

So how bad is it…drum roll…

Class[10] indicates the pure Java method running a 10 Thread test. FTL[20] indicates the Freemarker test running a 20 Thread test. And so on… Results are listed from best to worst with the avg and max results in nanoseconds.

Class[10]: 1000100 trials, 18046 avg, 67973245 max 1.0x
Class[80]: 1000800 trials, 28960 avg, 104784259 max 1.6x
Class[20]: 1000200 trials, 30387 avg, 106770756 max 1.7x
Class[60]: 1000200 trials, 35917 avg, 96320112 max 2.0x
Class[40]: 1000400 trials, 37220 avg, 162898182 max 2.1x
FTL[10]: 1000100 trials, 293093 avg, 104114975 max 16.2x
FTL[20]: 1000200 trials, 587117 avg, 158432568 max 32.5x
FTL[40]: 1000400 trials, 1204641 avg, 190966244 max 66.8x
FTL[60]: 1000200 trials, 1843190 avg, 284686324 max 102.1x
FTL[80]: 1000800 trials, 2498693 avg, 358187778 max 138.5x

The last piece of info on each line is how much slower that particular test was ( on average ) than the fastest. Notice how the pure Java tests don’t jump right into the crapper when under load. But the FTL tests get progressively worse as the thread counts are increased. Under an 80 thread load, not only is the pure Java way 138 times faster on average, but the FTL way has max time of 358 milliseconds vs a 104 millisecond max for pure Java. This is a perceptible 250 millisecond difference when rendering a single button tag!

The case for easing up on the Freemarker templates is solidifying