Understanding JSF 2 and Wicket: Performance Comparison

See the updated comparison for 2013 at JSFCentral:

Article about web framework comparison at JSFCentral Part 3

Suggestions, comments, critics or any discussion about it can be posted below.

----------------------------------------------------------------------------------------------

Performance is important in a web framework, even though it is certainly not the only consideration. But anyway, it is a topic that deserves an in-deep analysis. This is not a topic that can be reduced to just some few numbers, instead, web frameworks try to make a balance performance and other aspects like usability, extensibility, and much more.

In the last year (2011-2012), a lot of performance improvements has been done inside Apache MyFaces JSF Implementation (aka MyFaces Core). To see how good are the improvements, a comparison was done against a similar Java Web Framework like Wicket. In order to save some work, the reference application provided in this link was used:

http://ptrthomas.wordpress.com/2009/01/14/seam-jsf-vs-wicket-performance-comparison/

http://perfbench.googlecode.com/svn/trunk/perfbench/

Fortunately, the wicket guys have been so kind to updated the original application running with 1.4.x to version 1.5.x, so we'll compare JSF against two Wicket versions (1.4.20 and 1.5.5). A comparison against Tapestry might happen later. In the end, the intention is to look at how JSF works from a performance perspective, so Wicket will be taken only as a reference point.

Checking the code provided in that blog and the related data, some rather serious problems were discovered. Anyway, the code example is good enough to reuse it, while obviously fixing the detected problems. The updated code, configuration used, detailed documentation and experimental data can be found in the internet (github).

https://github.com/lu4242/performance-comparison-java-web-frameworks

Detailed Document (21 pages, PDF Format)

http://www.scribd.com/doc/94007042/Performance-Comparison-Leonardo-Uribe-FINAL-Detailed

Keep in mind that this blog entry is just a resume of the detailed document.

A good benchmark should consider different aspects. So, for this comparison, the following aspects has been checked:

• Speed benchmark: Check the speed of the code under different situations.
• Memory Consumption : Involves checking how much memory does the application need to allocate for a specific task, and how the application behaves under low memory conditions.
• Session Size : Check how much memory is used to store the session.

It is quite obvious why these three aspects should be considered:

• If the framework is fast it will process more requests with less CPU consumption.
• If the framework requires less memory, less time will be used in garbage collection and the framework will perform better under stress conditions.
• If it uses less memory as session storage, it will be able to serve more users with the same memory size, and if the session is stored somewhere (database, file, memory cache... ), less time will be used in processing that information (usually serialization/deserialization and dealing with I/O operations).

For the guys who are impatient these are the results: NOTE: This information is just related to the test application used and its results cannot be generalized. If you want to see the underlying discussion please read the detailed document.

Comparing MyFaces and Mojarra in the graph, you can see MyFaces 2.1.7 has become the fastest by a wide margin (more than 40%!). Comparing Wicket 1.4.20 and 1.5.5 you can see 1.4.20 is faster. In this case, a better programming model in 1.5.5 has cost some speed. In Wicket, disk storage (default) and http session storage has different trade-offs.

In this case, MyFaces uses a lot less memory to do the same task and the memory used between Mojarra and Wicket 1.4.20 is more or less the same. Wicket disk storage requires more memory than session storage, because the pages need to be saved and restored from disk. Most memory is allocated by Wicket 1.5.5, but it is interesting that there is no significant difference between disk storage and session storage.

From a memory perspective, MyFaces is the winner but note that allocating more or less memory doesn't say much because part of this allocated memory could have a short life, but it is relevant if the web server has not a lot of memory. In a few words, the less bytes and objects allocated the better.

JSF average view size is very small compared to the user session size, imposing a reasonable overhead. In this case, MyFaces do the best job, keeping the state size very small, and the important part here is note that the state overhead for JSF grows very slowly. In Wicket case, since it stores the whole page into the state, it easily grow more quickly.

CONCLUSION

Definitively JSF 2.0 was a big move in the right direction. JSF session size overhead is significantly lower than Wicket. MyFaces Core 2.1.7 looks very well from a performance perspective against a web framework like Wicket. In the end, MyFaces provides the better trade-off between CPU usage and session size. The bet of Wicket on disk storage is good, but the JSF bet on partial state is even better. MyFaces also provides the lowest memory usage, which allows it to perform better in environments with limited CPU and memory.

Comparing Wicket 1.4 vs Wicket 1.5, it seems that the improvement in its programming model is taking a toll. In my opinion it is something completely reasonable. A good web framework needs to balance different aspects in order to keep evolving over the time. Including more assisting code usually means lower response times, but it could improve developer productivity.

This is not the end of the story. Web frameworks will keep improving and the hope is this information can be useful to find new ways to enhance them (community over code is the Apache way). Remember: performance is just one aspect that you have to consider when choosing a web framework; usually it is necessary to strike a balance between this and several other aspects.