Ok... you've made config changes and tested your connections and performance and you know with the certainty of Rush Limbaugh making a right turn that the server is fully tuned and as lean and mean as possible, but you still have a problem. Now what? Well, it's time to look at the code. Start by enabling slow page logging in the CF admin during a high traffic low performance period. Take the list of templates that are running slow and start digging into them to determine why they are dragging their little cyber feet.
Here are a couple of code related items that you might be looking for.
Read More
In step 1 we discussed gathering information. Without getting good information you are shooting in the dark. Make sure you take the time to know the system well enough to make educated guesses about what to try. That's our next step - trying stuff. I started out making a priority list... as in first try A, then B, but it soon became obvious that it wouldn't do to dictate the order in which you would attempt any of these changes.
In fact, some of these changes fall more under the auspices of "best practice" for configuration and you should probably do them regardless of whether they fix your problem or not. Still, if you are tracking a particular issue then you might need to try something, test, and wait for a result. So here are my “things to try” in no particular order. Your experience will have to help you figure out where to go first.
I get a lot of calls from folks with dedicated Coldfusion servers who have suddenly developed problems. Usually the server has suddenly stopped responding in a seemingly random fashion. Often the caller indicates that Jrun is "pegged" at 99 percent. If you search through this blog (and many others) you will find a great number of tips and hints on how to attack this problem. The next three posts summarize my own process and give you a quick resource to the posts regarding troubleshooting a sick server.
Keep in mind that there is no substitute for experience. If you are novice at this you will need to get comfortable with the idea that you will spend several hours (sometimes days) working through possibilities. If the server is mission critical and there is money at stake you should consider calling in the cavalry. A troubleshooter is also a unique animal - a "technologist" of sorts. The best troubleshooters have gathered a good deal of knowledge and experience in several areas (app server specifics, database, hardware, networking etc). The best one s are also a special breed that think in a certain way. They make lists, figure out test patterns, know when to make a leap and when to keep digging etc. So with that in mind, let's talk about how to start.
Step 1 is to gather information.
Read More
If you are like the muse you keep your eye on the log files in the /runtime/logs/ directory. There's some good information in there if you care to poke around. You might notice one item that appears in the *.events.log from time to time - usually in a long list of similar errors. It looks something like:
This annoying error that occurs when using JDBC and MS SQL has been around for a while. The JDBC system in Coldfusion caches information about the tables you access. If you change a table (add a column for example) the column ids get out of whack and they no longer match the correct column. For Example, the first time you run a query you might get back the following:
So you have a template that runs fine when you are loading it into a browser but runs like a water buffalo on Percodan when it's run as a Coldfusion scheduled task? Here's a tip I saw on CF-Talk that serves as a good reminder. Take a look at the debug settings. By default, debugging includes the address 127.0.0.1 (often called the "loopback" or "local" address). Coincidentally this is the very IP address that the Coldfusion Scheduler will report as the calling address (cgi.remote_addr) - meaning the template in question will generate all of that lovely debug information. This can result in an otherwise acceptable level of performance turning painfully slow. Muse rule of thumb - disable debugging in production environments. Ok... so sometimes I enable it briefly to get some information when I'm troubleshooting a specific process, but it should always be off during normal operations.
Let me add as an anti-thesis, turn on debugging in a development environment. It's surprising how many developers work without it. Personally, I spend more time looking at the debug info than the actual output. One of the first things you should do when writing a new query for example, is check it out in the debug. How many records returned? How long did it take? Can I make it better? One of Coldfusion's big advantages (in my view) is the copious amount of debug information provided for you with each request. Use it!
The phrase "memory leak" can cause shivers to run down the spine of the most seasoned developer. Having some process on your server that is gloaming onto memory and failing to release it is a guaranteed all nighter lurking somewhere in your future. Recently we were debugging a new, soon to be released application. We discovered what looked like a memory leak. The JVM memory used would climb steadily toward the maximum heap size. When the runtime garbage collection kicked in it would reduce the memory only by about a third of the increase. So, for example, memory use would climb from 300 megs to 600 megs and then GC would reduce usage back to 500 megs and so on. This situation would inevitably lock up the server with out of memory errors. What follows is a recap of our troubleshooting journey.
Read More
I'm not a huge fan of upgrading software. I have found, in the words of Sam Gamgee's gaffer, that "Short cuts make long delays." In my view you will usually save time troubleshooting by doing a full install rather than an upgrade. Recently we needed to upgrade a customer from Coldfusion 7 to Coldfusion 8. It was a single server installation and we needed a minimum of downtime so, after writing a rollback plan and prepping the server we chose the "upgrade". All seemed to go well. All the data sources, services and settings ported over to the new Coldfusion 8 installation and the server fired up just fine. We thought we were home free until....
Read More