Fixing Locked Jrun Log Files Without Restarting Coldfusion
In a follow up to my previous post on Jrun Processor Pegging Issues and Solutions I was intrigued by an idea given to me by Steven Erat. The problem is that the "-err" and "-out" logs do not roll over like the event logs. This is a problem that is slated to be fixed in the next version. To "fix" this problem Steven thought it might be possible to move or delete the file using CFFILE without restarting CF. He reasoned that while windows could not defeat the lock, perhaps CF could defeat it because it owned the lock to begin with. Here's what I found.
Deleting, Moving and Renaming the File will all Fail
If you try to delete the file:
file="*path*\logs\coldfusion-out.log"
/>
...OR...
<Cffile action="DELETE"
file="*path*\logs\*instance*-out.log"
/>
Move
source="*path*\logs\coldfusion-out.log"
destination="*path*\oldlogs\"
/>
...OR...
<Cffile action="MOVE"
source="*path*\logs\*instance*-out.log"
destination="*path*\oldlogs\"
/>
Rename
source="*path*\logs\coldfusion-out.log"
destination="*path\logs\oldOut.log"
/>
...OR...
<Cffile action="RENAME"
source="*path*\logs\*instance*-out.log"
destination="*path\logs\oldOut.log"
/>
Overwriting the File
When I tried the write attribute like this:
action="WRITE"
file="*path*\logs\coldfusion-out.log"
output="## Created by Bob, May 3, 2006"
addnewline="yes"
/>
The Catch
The thing to note here is that the data is now gone. To quote Monty Python's Dead Parrot sketch, "It is deceased, it is no more, it has passed away, crossed over, pushing up daisies, gone toward the light, kicked the bucket, given up the ghost, bought the farm..." So before you try the code above, copy the file using windows so you have a copy of the data if you need it.
Why copy it using windows? Why not just do both operations using CFFILE? Ah... well, if you read my post from yesterday (5/2/2006) you will note that the problem we are most often trying to solve is that the "out" and "err" log files grow quite large (hundreds of megabytes perhaps even gigabytes) and do not "roll over" like the other "event" style log files. This creates a problem and I suspect it is the culprit in at least one case of JRUN pegging at 100%. Have you ever tried to copy a very very large file using Coldfusion? Quite simply, Coldfusion is not the best tool for this sort of thing. I suspect that Coldfusion reads the entire file into memory before writing it out to disk. If it's a very large file, it may even exhaust your server resources. Instead, use an alternative method for backing up the file, then overwrite it using the code above. This will give you the best of both worlds.
18 Comments
-
Thanks for the tip. My server has CF on C, and C is a bit small, so I run into this problem quite often. -
> Instead, use an alternative method for backing up the file, then overwrite it using the code above.
I like the idea Mark! Thank you!
I'm going to do it and suffer the consequences of the "yeah buts" later I know it ... as long as we're overwriting the file in ColdFusion, couldn't we grab a copy of the data and output it to a incrimented log file based on a schedule you choose based on needs? For example, archive the file daily / weekly / annually - whatever works for your situation? -
Jason,
Yes - that would be possible. A scheduled task to run periodically and simply grab the contents and offload them - then overwrite the existing file contents. The big issue there would be file size. The task would need to run often enough so that it doesn't have to manage very large files. Still, that could be a nice automated solution. I like it - thanks for the input. -
>> "I suspect that Coldfusion reads the entire file into memory before writing it out to disk."
Ok, another test using the same set I commented about in your related post.
I created a 200MB coldfusion-out.log file (not 2GB this time just so the motherboard wouldn't explode), then wrote a script to copy the log to the temp directory.
The script ran for about 30 seconds, and the memory for ColdFusion did not substantially increase. I piped top to a file to capture the system and process memory usage (using pstree -p | grep cfmx7 to find the parent process of the ColdFusion server, the second pid in that tree). Here's the file:
http://www.talkingtree.com/blog/downloads.cfm?item...
So file copies (on Linux) do not impact ColdFusion server memory usage. -
Could you not build a Custom Gateway that montiors the log files and if a file goes over xmb then back it up and nam eit using say server-err_YYMMDD-HHMMSS.log
Just an idea? -
Kev,
that's a great idea - a bit overkill perhaps, but still - I like it.
-mk -
A windows scheduled task which calls a btach file, which makes a copy of the file would be the solution. Better than doing it manually.
If you set the scheduled task to run with the same permissions as CF, it should also be able to write the new blank file as well. -
I had thought of doing that but couldn't find a way to zero length the files while still locked?
-
Don't use ColdFusion for this at all. Write a program in CMD, perl, whatever your system programming language of choice is, etc, and then wrap it as a service with srvany.exe. Cleaning up old files is a problem that isn't specific to ColdFusion.
Even better would be to have this app run on some internal server with domain privileges, and have it clean up multiple servers. -
If you are running Windows 2000, it is not a good idea to allow the Windows Scheduler to run. It is a pretty large security vulnerability, easy to run escalation of privilege attacks. Not so much of a problem in 2003 though. -
I tried this and an interesting thing happened. The code worked properly and replaced the coldfusion-out file however after a few moments the file swelled back to 383 MB. Any ideas on why that may happen? -
I had a similar experience as Patrick McDonough.
I overwrote the file 'coldfusion-out.log' (which changed the filesize from 1.1Gb to 1Kb) but it came back the next time coldfusion wrote to the file (right back up to 1.1Gb). It's almost as if coldfusion remembers where it was saving to the last time it wrote to the file and writes to that same location again - instantly restoring the original filesize. -
My CF Version:
Server Details
Server Product ColdFusion MX
Version 7,0,0,91690
Edition Enterprise (DevNet) -
Run the Latest Hotfix for Coldfusion to fix the problem:
http://www.adobe.com/cfusion/knowledgebase/index.c...
Sites to Watch
Aggregators
RSS
