Coldfusion Optimization and the Windows Legacy
Coldfusion has been navigating the Internet waters in the good ship Java for some time now (6 or 7 years I think). If you are old enough in Internet years to remember Coldfusion 5 you probably know that Coldfusion was originally written for windows in C++ and ported over to Linux, Solaris and UNIX. These ports were not reputed to be particularly good and Coldfusion was largely considered to be a windows server application (and probably justifiably so). In 2002 with "CFMX" Coldfusion offered the Web world an application that was not just a windows application with a port to other platforms. Instead it was a truly cross-platform effort. Since that time Coldfusion has found it's way onto Linux in increasing numbers. In fact we are seeing more Coldfusion on Linux than ever before - particularly users who need Coldfusion Enterprise.
And why not? Coldfusion runs splendidly on Linux. We use CentOS at CF Webtools and it sings along happily with very few issues. In fact, it is possible to allocate about 50 percent more memory to the heap on a Linux machine (using the 32 bit JVM). That is a significant advantage that CF on Linux has over CF on windows. Here's a blog post by CF Webtools own Linux guru Ryan Stille on that topic. Meanwhile, here's a take on the server optimization from the good old days to today:
Coldfusion 4.x and 5.x Black Box Resources
One of the semi-icky things about CF 5 was that there were very few options for tuning it. Indeed it was programmed to latch onto as many resources as it needed to get the job done. In many cases this caused it to be a bit leaky and a good deal of the server problems that we had in the old days were related to CF's "black box" approach to handling system resources. Coldfusion was pretty much in charge of memory, threading and the like. The only way to influence it was to code carefully around the known nuances of the platform. Who can forget the constant locking that we had to implement for session and application variables - not to prevent race conditions (a challenge in any language), but to prevent server memory corruption. Lack of locking could actually crash the server (yikes!).
Starting with Coldfusion 6.1 and continuing with versions 7 and now 8 Coldfusion comes with a lot more internal options for fine tuning how it handles system resources. On a busy system these options can add dizzying complexity to the process of optimizing a server. One problem that people still have, however, is continuing to treat Coldfusion as if it were a "black box" application. They install it in the default configuration, they don't touch any of the settings, and they blame Coldfusion for being "un-scalable" when it starts to fail.
The Default Configuration
Here's Mark's best tip for optimizing your server. Never never never install it in the default configuration and expect good results. I'm not just talking about Coldfusion. This is true of any server. If you install Windows Server 2003 without hand picking the services you need - shame on you! A server is a special machine with specific purposes. It should not be tasked with anything that it doesn't explicitly need.
With regard to a Coldfusion server (versions 6-8) the default configuration is practically useless for anything more than a developer's workstation. For example, the "simultaneous threads" setting is set to 8 by default. If you have a dual core dual proc machine you will want it to be more like 20 or even 28 (see my previous post on a Case for Higher Simultaneous Requests). And the heap settings are not just inadequate for a server, they are woefully inadequate. The default heap has no minimum amount and a maximum amount of 512 megabytes. Let's think about that for a minute.
A maximum of 512 megs means that if you install a beefy 4 gig machine that is intended to be a web server and nothing else - it will still never use more than 600 or 700 megabytes of memory for Coldfusion (512 plus some overhead like "permGenSize" etc). A typical server with the single task of web server should be configured to use the maximum amount of memory possible without impacting the rest of the server. And what about the minimum? No minimum means that the JVM is going to be tasked with "cleaning up" the heap to make it smaller all the time. In a typical optimized configuration the heap should be configured with little distance between maximum and minimum amounts.
Of course there are many other settings that need to be considered - Apache or IIS settings, specific changes to the jrun.xml file, specific switches for garbage collection etc. The main point is that you should take the time to optimize your server after you have installed it. Do not rely on the default configuration.
Adobe if You are Listening...
I realize that it is important to test settings. I know that every server is tasked with a different set of responsibilities and that one application that runs well with one set of options might suffer with another equally valid set of options. These are the reasons given for not providing some automated optimization assistance during the initial installation of Coldfusion. Let me just say that, while I agree with all of these points, I do think there is room for some helpful strategies during installation. For example, Adobe could add a "server tuning wizard" with all the standard caveats mentioned above. The wizard could ask about the servers resources, tasking and expected usage and then propose some "recommended" settings with a warning that experience may vary depending on usage. It could even leave it up to the sys admin to implement these settings. What do you think Muse readers? Is there room for some rudimentary help from Adobe in this regard? I'd be interested in hearing from some of my more informed readers on the subject.
11 Comments
-
Pretty bizarre you went to the effort for this post and yet still spell it "Coldfusion" - it has been "ColdFusion" for years. -
Some good points there about the default Java settings. A server tuning wizard sounds like an obvious suggestion, so why don't we have one already? The wizard can see how much RAM the server has so it only needs to ask the user if the server is dedicated to running CF only. If yes it can change the heap settings to make the most of the available RAM and to minimise the cleaning up. If no then it can ask questions about what else will run on the server (e.g. a db) or cut to the chase and ask what % of RAM to dedicate to CF.
Anything would be better than the current situation of users not knowing how CF utilises memory or where to look to change settings and what it all means.
The beauty of CF is its simple language syntax and RAD, so thinking that users know how to optimise JVM heaps or whatever flies in the face of CF's main selling point and its target audience.
Don't worry Mark, I won't criticise you for not holding down the shift key when pressing F. Tsk, Angus, really. ;-) -
Regarding simultaneous requests, where I used to work we had major trauma with a CF instance and it was because we had 10 or 15 threads set in the instance on a single proc. machine. Because it was a production stoppage issue, we opened a ticket with Macromedia, and the CF engineer informed me that in most cases (not all) you should set that to no more than 5 threads per processor. It even says so in the install docs (I wish our admin had read that doc).
So it may be safe on a dual proc box to set it to 20, but keep in mind that your mileage may vary. I am in the habit now, as time and resources permit, to fiddle with that setting during load testing to make sure I can set it at a reasonable level without sending the instance, or the entire server, out to lunch. -
@Angus
I have always spelled it Coldfusion and not ColdFusion. I hardly think it is my fault that the product keeps getting sold and rebranded. I don't spend my days reading marketing material or hanging around with Adobe executives.
FYI - useless comments like this are not welcome on my blog. You could have said it nicely as in "hey... I think it's ColdFusion and not Coldfusion.... "
@Andy,
I'm not so sure. CMFX was out when I left my full time job and started my company. That would be at least 2001 - possibly 2000. 1999 is probably wrong though... I'll edit that.
@Gary,
thanks for the excellent contribution to my blog :)
@Lincoln,
Don't believe everything that Adobe or MM support tells you. In this specific case there are 2 procs and 2 cores so 20 is a reasonable number (5 x 2 x 2). I have done this quite a bit now on a wide array of production installations and I can tell you that there are indeed cases where you need to boost this number and it can be effective.
As you say, it is important to test and know what you are up against. -
I know it's not important in the grand scheme of things, but ...
http://en.wikipedia.org/wiki/Adobe_ColdFusion#Rele... -
Mark,
I work with CF since version 3.0 (Allaire days) and server tuning was always an issue. I've read every single document that went out about it but there remains some questions not answered.
I'm pretty sure Adobe can help us by:
1. Implementing a wizard, as you suggested and Gary added;
2. Fully exposing the configuration "inside the hook" in Administrator, with complementary comments on how they affect the overall performance and, if possible, direct links for liveDocs documents about the configuration (liveDocs allow comments and that helped me too many times for me to count);
3. Promote a place/wiki/forum specifically for CF tuning with Adobe experts and employees participation.
Hope it adds as I'm not sure if I'm one of the "more informed readers" here ;) -
Mark, I forgot to mention that back in later 2002 my ex-company was trying to patch a large system on-line, but CFMX refused to work the way it was supposed to till CFMX 6.1 was out.
So I can confirm that CFMX was out about later 2001 or early 2002. -
I've been having a *lot* of problems with this. I run CF in a multi-server config where each website is its own CF instance (which unduly upsets CF8 sometimes and makes it not start up properly after a reboot, complaining about licensing).
The problem is that the server had fairly limited resources and each site was eating approximately equal chunks of them, even though one site gets 20 hits a week and another gets millions. The busy site on one of the two servers on the load balancer kept hanging after complaining about running out of threads... quickly determined it was because that server had one extra coldfusion instance on it, a sort of staging site.
We ended up having to install extra memory on the box, which on its own didn't fix the problem... CF wasn't *using* the memory... so I tweaked the JVM.config to give the busy site enough headroom (no clue what I was doing really, some of the posts here helped a good bit).. but the tweak effects all the sites, so again, they're going to run into issues.
I looked around for a way to tweak *just* the busy site to give it more memory, but there just wasn't much at all out there about the problem. -
@JC,
You should install them with separate JVM arguments. For example, the lightly used site should get no minimum heap and a max of 256 (or whatever the "minimum max" is). See this article
http://www.coldfusionmuse.com/index.cfm/2006/4/17/...
Sites to Watch
Aggregators
RSS
