viernes, 18 de mayo de 2012

Understanding JSF 2 and Wicket: Performance Comparison

See the updated comparison for 2013 at JSFCentral:


Suggestions, comments, critics or any discussion about it can be posted below.

----------------------------------------------------------------------------------------------

Performance is important in a web framework, even though it is certainly not the only consideration. But anyway, it is a topic that deserves an in-deep analysis. This is not a topic that can be reduced to just some few numbers, instead, web frameworks try to make a balance performance and other aspects like usability, extensibility, and much more.

In the last year (2011-2012), a lot of performance improvements has been done inside Apache MyFaces JSF Implementation (aka MyFaces Core). To see how good are the improvements, a comparison was done against a similar Java Web Framework like Wicket. In order to save some work, the reference application provided in this link was used:


Fortunately, the wicket guys have been so kind to updated the original application running with 1.4.x to version 1.5.x, so we'll compare JSF against two Wicket versions (1.4.20 and 1.5.5). A comparison against Tapestry might happen later. In the end, the intention is to look at how JSF works from a performance perspective, so Wicket will be taken only as a reference point.

Checking the code provided in that blog and the related data, some rather serious problems were discovered. Anyway, the code example is good enough to reuse it, while obviously fixing the detected problems. The updated code, configuration used, detailed documentation and experimental data can be found in the internet (github).


Detailed Document (21 pages, PDF Format)


Keep in mind that this blog entry is just a resume of the detailed document.

A good benchmark should consider different aspects. So, for this comparison, the following aspects has been checked:
  • Speed benchmark: Check the speed of the code under different situations.
  • Memory Consumption : Involves checking how much memory does the application need to allocate for a specific task, and how the application behaves under low memory conditions.
  • Session Size : Check how much memory is used to store the session.
It is quite obvious why these three aspects should be considered:
  • If the framework is fast it will process more requests with less CPU consumption.
  • If the framework requires less memory, less time will be used in garbage collection and the framework will perform better under stress conditions.
  • If it uses less memory as session storage, it will be able to serve more users with the same memory size, and if the session is stored somewhere (database, file, memory cache... ), less time will be used in processing that information (usually serialization/deserialization and dealing with I/O operations).
For the guys who are impatient these are the results: NOTE: This information is just related to the test application used and its results cannot be generalized. If you want to see the underlying discussion please read the detailed document.


Comparing MyFaces and Mojarra in the graph, you can see MyFaces 2.1.7 has become the fastest by a wide margin (more than 40%!). Comparing Wicket 1.4.20 and 1.5.5 you can see 1.4.20 is faster. In this case, a better programming model in 1.5.5 has cost some speed. In Wicket, disk storage (default) and http session storage has different trade-offs.

In this case, MyFaces uses a lot less memory to do the same task and the memory used between Mojarra and Wicket 1.4.20 is more or less the same. Wicket disk storage requires more memory than session storage, because the pages need to be saved and restored from disk. Most memory is allocated by Wicket 1.5.5, but it is interesting that there is no significant difference between disk storage and session storage.

From a memory perspective, MyFaces is the winner but note that allocating more or less memory doesn't say much because part of this allocated memory could have a short life, but it is relevant if the web server has not a lot of memory. In a few words, the less bytes and objects allocated the better.


JSF average view size is very small compared to the user session size, imposing a reasonable overhead. In this case, MyFaces do the best job, keeping the state size very small, and the important part here is note that the state overhead for JSF grows very slowly. In Wicket case, since it stores the whole page into the state, it easily grow more quickly.

CONCLUSION


Definitively JSF 2.0 was a big move in the right direction. JSF session size overhead is significantly lower than Wicket. MyFaces Core 2.1.7 looks very well from a performance perspective against a web framework like Wicket. In the end, MyFaces provides the better trade-off between CPU usage and session size. The bet of Wicket on disk storage is good, but the JSF bet on partial state is even better. MyFaces also provides the lowest memory usage, which allows it to perform better in environments with limited CPU and memory.

Comparing Wicket 1.4 vs Wicket 1.5, it seems that the improvement in its programming model is taking a toll. In my opinion it is something completely reasonable. A good web framework needs to balance different aspects in order to keep evolving over the time. Including more assisting code usually means lower response times, but it could improve developer productivity.

This is not the end of the story. Web frameworks will keep improving and the hope is this information can be useful to find new ways to enhance them (community over code is the Apache way). Remember: performance is just one aspect that you have to consider when choosing a web framework; usually it is necessary to strike a balance between this and several other aspects.

11 comentarios:

  1. Hi,

    I'm Martin Grigorov from the Wicket team.
    I was the one who helped you with the migration of the app to 1.5.5. I was also the one who asked you to do this perf benchmarking together, to make it in the Apache way - open. By 'open' I meant by creating an public repository and communicating via open channels - mailing list, forum, etc what and how should be done to make this perf benchmark useful for both of our communities. But you just ignored me once after receiving the upgraded app...

    Well, all I can say is: Congratulations for the good product you have!

    ResponderEliminar
  2. Well, that's the idea to upload all doc in github, which is a public repo where anyone can create branches, make updates and do it in a more colaborative way. I did not ignore you, just I had too much work, and stuff like that needs to be done right from the start. Thanks for the app.

    ResponderEliminar
  3. Hi Leonardo,

    I'm very surprised with the difference between Mojarra and Myfaces, it's very interesting. I always thought that Mojarra impl. was far better than Myfaces one. Well, Myfaces team deserves congratulations.

    I don't know if you've already read this blog post about stateless jsf, but probably it's possible to improve jsf performance when using this stateless approach.
    http://industrieit.com/blog/2011/11/stateless-jsf-high-performance-zero-per-request-memory-overhead/

    ResponderEliminar
  4. Hi Rafael

    In fact, Mojarra was faster than MyFaces in versions before 2.1.5, because the priority in MyFaces side has been always get the code into an stable state, which means resolve "the most difficult and craziest problems" found by JSF users when they start to mix code.

    I have read the blog related to stateless jsf carefully and this one too:

    http://www.znetdevelopment.com/blogs/2009/05/01/jsf-suggestion-for-performance-improvement/

    because I think both are very useful feedback from the community. It is too hard to explain the details behind the improvements, but the interesting part is it is possible to have a good performance in JSF without get into "stateless" mode.

    I think it is possible to get a better performance with a "stateless JSF", but with the changes done in MyFaces it will be a minimal gain. Note it is necessary to investigate it in deep, I still have some ideas left in my hand, but maybe it will be done for JSF 2.2. Help and feedback is always welcome in MyFaces Community.

    If you compare with previous versions of MyFaces, the improvement was huge (very, very huge) in all aspects (speed, memory, session size, bug fixing, ...). MyFaces code has been always high quality stuff, it just needed some polishing. Is like a good wine, you need to taste it to enjoy it.

    ResponderEliminar
  5. Indeed this post it very nice, and i am surprised to see MYFace so much better than Mojarra. Nice work and as ponte said, stateless jsf can be a good choice for the future.

    ResponderEliminar
    Respuestas
    1. I have seen this answer in stackoverflow some weeks ago:

      http://stackoverflow.com/questions/9923343/memory-usage-of-myfaces-and-richfaces/9936964#9936964

      "... the memory usage of our application dropped enormously. From a staggering 50MB+ per session to almost none. We used to consume at least 1GB for every 20 users and that dropped to less then 200MB for any amount of users (<50 tested). ..."

      I think the improvements in Partial State Saving are the best ones, more than the improvements in speed/memory. In few words, MyFaces puts a thin layer to get a minimal state and reuse all "stateless" part of the component tree, even when using dynamic facelets tags that changes the component tree structure (c:if, ui:include src="#{}", ...).

      Eliminar
  6. There are reasons to believe that HybridJava will beat both Wicket and Facelets in performance.

    ResponderEliminar
    Respuestas
    1. It could be, a hard coded servlet app could be faster, but note the tricky part in that case could be the "balance" between what the framework do for you and what you have to do when you build your custom web application. It is easy to fall into wrong conclusions looking only one aspect, but at the end it is a matter of "taste". I think in MyFaces there is a good balance and the code performs very well, compared with the functionality provided.

      Eliminar
  7. Hello,
    I was supprised thah average session size in myfaces is only 5KB
    I have app on myfaces 2.1.7 with @ViewScope beans and my sesion size is 1,5 Mb I have the samo config of myfaces in web.xml

    ResponderEliminar
    Respuestas
    1. If you are using @ViewScope, it means you store the bean in the view, and if you are using server side state saving it means the bean is stored into session, so basically your session will be bigger or smaller according to what you put there. But the interesting part is note how tiny is the overhead of MyFaces into the state, and in that way, into session. In this case you should take a look at your code, and keep into view scope only what you really need, and instead use request scope to fetch in each request what can be retrieved someway.

      Eliminar
  8. @ViewScope is very usefull for the ajax purpose I using primefaces and all by datatable providers have to be in @viewscope

    ResponderEliminar