Web Applications and Memory Abuse

As a performance engineer, part of my job is to make our software run really really fast. Another part is to figure out how to make sure that it still runs as fast with 1000 users as it does with 10. That's usually the harder part. You can usually squeeze gains out of each transaction enough to make the single-user cases perform fine. Jeff Atwood describes it very well in this blog post. Says Atwood:

"Everything is fast for small n. Don't fall into this trap. It's an easy enough mistake to make. Modern apps are incredibly complex, with dozens of dependencies...The only way to truly know if you've accidentally slipped an algorithmic big O bottleneck into your app somewhere is to test it with a reasonably large volume of data."

As an example:

A big reason for poor scalability is the abuse of memory. Everything is faster when loaded from memory, right? That's always good, right? Matter of fact, why not put everything the user sees in their session? That way all they have to do is go straight to their own little section of memory so that everything they do is fast fast fast. Hmm. Good when the n is small. Bad when the n is big. Let's say, for kicks that every user session grows to 2MB. That's not unrealistic when considering some of the data-driven applications that you use on a day-to-day basis. Not so bad, right? After all, memory is cheap! Now, let's say your application is running on 32-bit Windows. For reasons explained here, you can only fire up a 1.5 GB JVM. So let's say you have 500 active users on your system. Do the math:

1500MB total heap size - 250MB server bootstrap - ( 500 users * 2MB session ) = 250MB for processing. Now things aren't looking so peachy, eh. However! We can just cluster, right, so we have multiple JVMs running, to take advantage of all that memory! Ha! In the words of Lee Corso, "Not so fast, my friend!". The default method of session propagation is simply to multicast the session changes across cluster nodes. Now you have 1GB of user-specific crap clogging up each of your JVMs. That leaves little heap for actual system processing needs. Now, factoring in the fact that most users don't proactively log out of systems, rather letting their session lapse to timeout, and the fact that many (non-financial) systems have rather long timeout intervals, you could have ALL THAT MEMORY just sitting there for users who may not even be in the same room as the computer that was associated with that session. Bad. Smokey says that "Only you can prevent forest fires". Well I am saying "Only you can prevent OutOfMemoryExceptions!" You could also put it like this: "Friends don't let friends bloat their sessions."

However you say it, you need to understand the trade off between saving processing time on the database and ruining the overall ability of your application server to respond to increasing numbers of sessions and requests.

Terracotta Clustering

A product I have been reading about is Terracotta. Currently at my company we use Tomcat's native clustering, which does session replication across all nodes - this is practical for many of our subsystems, but obviously it has to change your perspective on the way you use (abuse) the session for storage. A solution like Terracotta is great because you aren't replicating session across every JVM, and you don't have to deal with annoying things like making sure any object that is replicated implements java.io.Serializable. I hope to test it as part of some R&D for my company as a potential configuration. The architecture is explained by this diagram:

The architecture diagram would make it appear that in a clustered environment, you have introduced a single point of failure, which would be pretty counterintuitive, but of course, you can cluster the Terracotta servers as well. Effectively, Terracotta with its granular session update writes, configured in a clustered setup, should theoretically scale in a near-linear fashion. I am curious to see if this is the case. Furthermore, I am interested to see if we can integrate it with our application without having to completely hack the installer!

Some interesting links for anyone who is interested:

Interview with Terracotta
Terracotta Confluence Site
Terracotta from theServerSide
Terracotta's Scalability Story

I will post an update with my experiences.

Performance Engineering Reading

While performance engineering isn't necessarily entrenched at many companies (outside of a few people recording and playing back LoadRunner or SilkPerformer scripts), there are some really smart people writing about it in the 'interweboblogosphere'. A couple of things I am reading about today:

Coding Horror on "little n"
jsLex - A JavaScript Profiler - saw Mr. Buffone present at JavaOne - it's a neat tool and a great idea.