ColdFusion Muse

IIS 7 Max Worker Processes and ColdFusion Updated

In my previous post on this topic I indicated that IIS 7 seemed to be a constraining factor. That post lead to conversations with a couple of CF gurus (Charlie Arehart and Russ Michaels) who clued me in to a number of additional settings. If you are truly interested take the time to read the previous post and (especially) the comments before you read this post. What bothered me was that the issue I discovered (a cap on requests) can be affected by both IIS settings or JRUN settings (or both).

My conclusion is that the behavior I was trying to affect is actually the bug that Charlie pointed out to me on Adobe's site (found here). Charlie rightly indicates that this issue is under-recognized (I certainly had not run into yet). The behavior of this bug can be affected (fixed or mitigated) by adjusting IIS as described in my previous post as well as by using the Adobe-provided instructions. This lead to a bit of Muse head-scratching. How do these various processes really work together? This post hopes to clear that up (or at least add to our compendium of knowledge).

Read More

IIS 7 Constraining Simultaneous Requests Limit?

I have been doing some performance testing for a company with a large server farm over the last couple of weeks. Although the farm had 20 or more servers, we started with just one sever to try and get some numbers we could use to extrapolate the overall tolerance of the larger system. The servers were all Windows 2008r2 64bit, IIS 7, running CF 9 enterprise with plenty of RAM. We were also running Fusion Reactor to help introspect ColdFusion.

As I slowly poured on more load I noticed something strange that I had never seen before. Although my "simultaneous requests" setting was set to 48. I could not get ColdFusion to handle more than 25 active connections. Under ordinary circumstances I could pound away at a server using my test framework and get enough requests active to overrun that simultaneous request setting and see the queue kick in. I was trying to max out the server but it was not behaving as expected. Active requests would "cap out" at 25 - as if my simultaneous request setting was set to 25 - but there were never any requests in the request queue. It was a head scratcher - but I kinda love those! Here's the skinny....

(NOTE: The comments on this post are important as well. And this Follow up post clears up some of the confusion.)

Read More

ColdFusion Builder 2 Hotfix 1 Fails

If you have an issue where you are trying to install "hotfix 1" for CF Builder 2 on a Windows 64bit platform and it fails after decompressing the file, you might try the following. Look for the uncompressed files (usually in the temp directory), drill down and find the *.lax file. This is the file that install anywhere (a Java installer) uses to launch its own install procedure. In the file is a path to the JVM - look for the line that says something like lax.nl.current.vm=\\... Then set the path to a known good 64bit JRE SDK. You should have one if you are running eclipse. Once you make the change launch the install exe in the decompressed files folder (instead of the original file). This is a version of the fix found in my previous post on CF 8 64bit on Windows 2003 64bit Web edition. The previous post has all the details and then some. I'm told by CF aficionado Christian N. Abad that the fix works for CF Builder Hotfix 1 installs as well. I suspect as a general rule that it would work for any "install anywhere" process with this specific problem.

Jmeter Part 3: Script Recording and Crawling

This is my last post on the topic of Jmeter testing. I wanted to finish up by showing you 2 things. First, CF Guru Larry Lyons pointed me to a different test tool called "bad boy". This test tool has a good deal of capability in it's own right but the main reason I bring it to your attention is that you can record a test and export it to Jmeter in a few simple steps. Second, as promised in my previous posts (Jmeter testing and Jmeter Distributed testing) I'm going to share a Jmeter script that is able to crawl your site using the magic of regular expressions. First, let's explore Bad Boy.

Read More

Jmeter Part 2: Distributed Testing

In my last post I detailed using Jmeter to set up a simple Web Application test. Jmeter can bring a lot of users to the table with just a single workstation and you can affect some real pressure on your system. But there are times when the resources of the server simply exceed the amount of traffic you send its way. For example:

• The site is heavily cached or pure HTML and can handle bucket loads of traffic.
• The server is a huge horse with tons of RAM, fast disks and teamed NICs.
• You are testing a cluster or load balancer - multiple servers working together for high capacity.

The idea behind distributed testing is to use multiple workstations with a single test plan. Jmeter makes this extremely easy (surprisingly easy considering how difficult it is with other testing products). Here are the details.

Read More

Site Testing With Jmeter

Recently I did a user group presentation on testing an application using Jmeter. Notice that I did not say "load" or "stress" or "performance" testing. There are many different types of testing you can do to your application. All applications can benefit from testing for performance or load - although there is a cost factor involved that can prevent some folks from taking the time to test smaller apps or apps with an expected predictable or lite load (like an internal application for example). Still, it is something that every developer should consider. Generally speaking there are 3 increasing levels of testing that rise "up the scale" in complexity and cost.

• Performance Testing - To do a performance test you usually you put a reasonable or expected load on an application and examine the internals. By internals I mean memory usage, processor usage, database performance and specific pages that you suspect need tuning. There might be many other things you want to look at here - networking, third party services etc. The goal is to find bottlenecks and improve performance under the expected load. Almost everyone does some form of performance testing - even if it's just loading up a page and looking at the debug info to see what sort of time it takes to run a query or execute a routine. When finished you have a "tuned baseline" - a set of expected benchmarks you can reasonably expect your application to meet (until the database crashes or some data center guy trips over the power cord anyway).
• Load Testing - Load testing is sort of generically thrown around to mean both performance and stress testing. In reality a true "load test" (at least by the book) means maximum load for a sustained period. In other words, pushing your application to the top of the curve and leaving it there for anywhere from an hour to a couple days to see if it degrades over time. So really it is more like "marathon" testing or "endurance" testing. Load testing teases out things you might not expect - bugs that occur over time in large data sets, crashes that have a threshold or duration attached to them, memory leaks, and timed problems that occur when events conflict (like backups or scans). Most folks do not do any load testing. Load testing is very expensive and time consuming and simply understanding the data set will require experience and patience. If you have invested in a high level application where perforamance and capacity are important long term factors then you should probably be load testing.
• Stress Testing - The goal of a stress test is to see where your application will finally break, and how it recovers after crashing. In other words you add an increasing amount of requests to your app until it stops functioning, then you see if it recovers. Does it require a hard kill? Is any permanent damage done? Are there corrupt files? Orphaned DB records? Finding out where your system crashes gives you a baseline for capacity planning and allows you to fix issues related to potential crashes.

The following post details just the "minimum" or "basics" of setting up a test with a product called Jmeter (part of the awesome treasure trove of products found in the Apache Foundation vault). You can extrapolate from these instructions to do performance testing right away. With a little extra effort you can set up a stress test as well.

Read More

Preparing for When You Need Me Most

In the new order of things practically no one has a server "in-house" anymore. Unless you are a large company with a heavy IT presence, it is simply easier to rent space at a data center. This works very well for the most part, but one of the nuances of this system is remote access. Most data centers have a number of different methods for providing you with remote access. In some cases they use a VPN like Cisco AnyConnect (which I think is excellent). Once you are connected to the VPN you access the server or servers via an "internal" NAT address - an unroutable address usually beginning with 192 or 10. Then you use something like RDP (remote desktop protocol) to log into the server desktop and configure the server, deploy code or whatever. In other cases the NOC might simply provide open ports to your own static IP for the services you need (like RDP, MSSQL, MySQL etc). This approach opens the NOC up to the your security, but it's certainly better than open ports.

In addition, some hosts may allow these ports to be open and trust the next layer of security to keep the bad guys out. In fact, virtually all hosts offering shared hosting or web panels etc have to offer these open ports of necessity. The Muse hates this idea. Allowing open ports for RDP (3389), SQL (1433), and especially FTP is a bit like Goliath strutting up and down daring the Israelites to challenge him. Meanwhile, all rock throwing David could think is, "how could I possibly miss that guy?" But of course there's more to the story...

Read More

Accessing Both 64bit and 32bit Assemblies

Many readers will doubtless recall the war waged by the Muse against .NET on 64bit ColdFusion 8. If not, you can read all about it in this series of posts on Muse Vs. .NET Integration. It was clear from the outset that using 64bit CF against a 32bit assembly was not working for us. Naturally we went down the path of recompiling everything into 64bit and we had multiple obstacles to overcome. But the fact that we could not communicate with 32bit assemblies always puzzled me. The communication (like most things on a web server) happens through a socket to a listener provided by the integration service (just like Verity and Sequelink). I could not reasonably explain to my own satisfaction why it should be that a 32 bit assembly was inaccessible. After all, our tests using the assembly directly worked fine. I assumed in a vague sort of way that differences in how variables were stored and passed back and forth must be to blame.

A few weeks ago I got a tip from the always insightful Rick Root on CF-Talk - who figured this out with the help of the folks at Just CF (SupportObjective). His problem was different than mine. He had a mixed environment with both 32bit assemblies and 64bit assemblies. In his experimentation he discovered something that solves both our problems. When you call and assembly (any assembly) using .NET integration ColdFusion creates some jar files under the hood. You will find them in the /cfclasses/dotnetproxy folder. Here's a sample folder where the server is using 2 .NET assemblies:



Of course, one of the Jar files that ColdFusion creates is the actual assembly interface which "mirrors" the methods and properties of the .NET interface. It's the one with the cryptic name that looks like "-23480983_3023903.jar". The other file however is special. It's the one that is always named dotNetCoreProxy.jar. It is compiled only once (unless you delete it) on the first time you instantiate a .NET assembly. It provides (presumably) the interface to the proxy listener.

The Fix

Now here's the kicker. If the first assembly you call is a 32bit assembly - in other words, if you call the 32bit framework first - then this little jar file is compiled to access the 32 bit framework and cannot access the 64bit framework. In fact, there's some good evidence that it can't reliably access 32bit DLLs either (it seems error prone). However, if you access a 64bit assembly first then the jar file is compiled in such a way as to be able to access both the 64bit and 32bit frameworks.

Conclusion

To avoid having to run this gauntlet I would suggest accessing a .NET Assembly on the 64 bit framework first and backing up the subsequent dotNetCoreProxy.jar file to avoid future confusion. I'm sure you can find a nice "Hello World" assembly to compile to 64bit. If you have legacy .NET integration code and you are moving from 32bit to 64bit then it will likely not work out of the box until you have done this (or it may work once or twice and then give you inscrutable errors). You will need to get the correct dotNetCoreProxy.jar file compiled (the 64bit version) before you can access your 32 bit assembly dlls successfully and reliably. The good news is that you don't necessarily need to recompile all your assemblies to use the 64bit CLR - although there may be a performance or compatibility reason to do so.