Fun with SMTP Relay
This is a post about solving a particular problem with SMTP relay that involves mass emails. Whenever I write a post on this topic there are 2 things that my savvy readers always feel compelled to tell me:
- "Hey Muse, make sure you are not sending SPAM" - Thanks for the advice. The Muse takes a dim view of SPAM. Like everyone else I'm tired of being told I've won the lottery, have friends in Nigeria, and need to be more concerned about my size. The emails in this case are not spam - but if you are tempted to make that comment I appreciate that you have the Muse' back.
- "Hey Muse, you are crazy to do this yourself - Thanks for that as well. And please don't hesitate to tell me (again) about the various services that are out there - all of which are better equipped technically, mentally, physically, and ecumenically to handle my email so that I need not be an expert on the topic. I always appreciate that input. The only thing I like better than leaving money on the table is transferring it to another vendor. In fact it's an axiom of business to never do anything for money that you can pay someone else to do for you. Next week I'm writing about server troubleshooting and that will give you an additional opportunity to tell me (again) about how that no one needs to worry about that anymore either because the cloud fairies do it all magically.
Dev Tip 101 - the HOSTS File
Fair warning - this is a pretty "101" post so it might be a tad ho-hum for some of you. It's surprising to me how many developers I meet who stare at me blankly when I suggest that the use their HOSTS file for one purpose or another. The HOSTS file has been around since the first networks - although it's gone by a few names over the years. In its current version HOSTS has been around largely unchanged since ARPANET and is in fact the predecessor to DNS (which has reached a venerable age in its own right). As a web developer, learning some easy fundamentals about the use of this file is a practical and fundamental skill - so let's explore it a bit shall we?
Read MoreArcane Networking Tip Number 702 - Non-Static Mapped IPs
This falls along the lines of one of those tips that matters only to sys admins, firewall managers or network engineers. So if you aren't a networking geek (or don't aspire to become one) you can skip this tip. Here's the skinny.
When setting up a windows server I like to use an "internal only" IP address - one that is not statically NAT'ted to anything - as the source IP address. In most cases this means the IP address presented when making outgoing requests is the external address of the firewall instead of the "real static" ip. If you don't know what I mean by source IP, remote into your server and use a brower to go to what is my ip. Whatever it gives you back is your "source" IP address - the IP presented by outgoing requests. In fact if you check your source IP from your desktop in a typical corporate office and then go to a neighbor's computer and check it there it there you will likely see the same IP address. This is because most desktops sit behind a firewall and the firewall has an assigned IP address that it presents as the "source" IP for most traffic. And that "external firewall address" is also the one I often choose to use for outgoing traffic from a server.
Ok, so why is that a problem? Well a server is a little different. In most cases it will have one or more IPs that are "statically mapped" to its own live internet ips. For example, let's say the DNS record for www.example.com is pointed to 72.10.20.10. "Inside" the network the server actually has an IP of 192.168.10.10. When web traffic hits the firewall for the "external" address (72.10.20.10) it looks at its translation table and knows that the "inside" address that "equals" 72.10.20.10 is actually 192.168.10.10 - and then it checks to see if the traffic is allowed (that's the "firewall" function of a firewall) and forwards the traffic to port 80 on 192.168.10.10. That's "network address translation" to a "statically mapped ip address" (whew!!). Ok, take a drink of water - the dizziness will pass momentarily.
Now for a variety of reasons I often don't want the IP address that the server presents when making outgoing connections to be "statically mapped". Instead I often prefer it to present the outside IP address of the firewall (as a sort of a generic proxy for my whole network). That used to be pretty easy. In Windows server 2000 and 2003 that was easy. I would just make sure that the first IP address I added (the one that you actually "see" in the little network IP properties window before you click on "advanced" - was a non-statically mapped IP. All outgoing traffic would "choose" this first IP by default and Voila! I have the results I was looking for. Then I could just add my other "statically mapped" IPs in the advanced tab and move on.
With Windows 2008r2 however this source IP address can switch to one of the other IPs in the pool. So even though I added my non-static IP first eventually my server might switch to using the statically mapped IP. This is probably only an annoyance for me. But if you are one of the tiny minority of people who geek out over such things here is the solution.
The Fix
In order to get the behavior you want you start out the same. Add your non-static IP as the first IP address per usual. Then instead of adding additional IPs using the "advanced" tab, open a command line and use netsh to add them with the netsh command and the "skipassource=true" flag. It's that "skipassource" flag that does the magic. Here's the syntax for you.
netsh int ipv4 add address network_1 192.168.10.10 255.255.255.0 skipassource=trueOne note - the label network_1 in the syntax above is the "name" of the adapter or "network" you are adding to. You can find this in network properties. By default it is "Local Area Connection" but I always rename it to something without spaces so I don't have to do too much head scratching (with quotes? without quotes? single quotes?). If you add your subsequent IPs like this from the command line using the skipassourceflag then your "non-static" IP will always be the default preferred IP for outgoing traffic. Hope this is of use to someone. Happy coding.
Host Files and the Resolver Cache
This is something I assume everyone knows sometimes - but it's actually a networking thing. Surprisingly, quite a few very bright developers get that deer-in-the-headlights look when dealing with networking issues. The question is, "How do I test a domain without making it live." This is an important question. For example you might have code that is domain specific. That happens on those sites with a "shared codebase" or with Ajax or whatever. In addition, you might be developing on your local machine (many - perhaps most folks do) and want to set it up so that the domain www.xyz.com points to your local machine. You might have a site about to "go live" and you want to thoroughly test it prior to changing DNS. So there are many reasons you may need to do this. Here's a quick tutorial on how to use the hosts file to make this happen. Note - the examples are for Windows, but MAC an Linux also have hosts files so the principle still applies.
Read MoreThe Muse Goes "Deeper" on the Mac
I will probably get hammered for this, but it really annoys me that the "finder" does not expose everything on the machine to my prying eyes. I don't need an operating system that obscures anything from me. I want to poke into every single little nook and cranny of the file system, device drivers, logs, config files, run levels, permissions.... I'm not interested in a "friendly" interface that pats me on my hand and shows me just what I need to know. I want to dig into the whole banana and find out everything I can.
So I spent an hour looking for something like window's "Folder Options" or 08r2's "god mode" console - where I could enable file extensions and hidden and system folders. Finally My good friend and colleague Wil Genovese told me to download "Deeper" - an apple utility whose sole purpose seems to be to enable various things that are "off" by default on a Mac. I can now see the whole file system (thanks Wil!). On to the next challenge!
The Muse and the Mac
I bought a new mac yesterday and I'm diving in trying to figure things out. The Muse knows his way around every flavor of Windows going all the way back to Windows 95 and up to every version of the server product. I cut my teeth in IT as an MS network engineer. But I've seen and fiddled with Macs before. I have added hardware (SCSI drives, controllers, RAM etc.), configured print drivers, and connected to shares on the network - all in the way of support for some of my design buddies like Erin Osterberg (a beautiful and wonderful video editor working for my good friend Rob Helling at Sonburst Communications).
But I've never actually been a Mac user. Mostly I try to stick to what works and having a modest aptitude for PC's and servers I found my niche there. It's also hard to stomach the price. It seems with the Mac you are paying twice as much for the same hardware that is in a PC... except you get that thar fancy brushed aluminum casing and a shiny mouse and brushed aluminum keyboard that looks to be made for a child.
Still, I have a need to work with some I-phone apps, so I need a Mac to run Xcode. I bought an Imac with a giant screen, set that bad boy up, rubbed my hands together and started in.
From this point forward I suspect that some of my readers will likely treat me with a rueful chuckle and some ribbing. It may be painfully obvious in the next few paragraphs how clueless I am. Anyway - here goes. Mac's reputation for being easy to use is well earned. I did not have any trouble getting my network configured and figuring out all of the personal preferences. I managed to install Firefox (and firebug), chrome and eclipse. I found the "software updates" and ran them, nicely updating a good many things on my machine. I managed to register the machine in active directory and add my domain permissions to my keychain. I even figured out how to remove all the foofy stuff I'll never use from the dock (Iphoto, Itunes, Imovie, Ichat, Ical, IstartEverythingWithI). So far so good.
Install How Exactly?
So my first "issue" is with something weird happening with installs. I installed FireFox then dragged it to the applications folder and (I think) to the dock. But on my desktop there is an item that says FireFox with an icon like a drive. There's another one that says "chrome" after that install. As far as I can see I have Chrome and FireFox in the dock. When I try to delete them Mac asks me if I wish to "eject" them. Guru Toby Tremane tells me that Mac files are downloaded as ".dmg" files - disk image files. How they got on the desktop I'll never know.
Network Follies
The muse is all about work at the office, so my next task was network resources. I was feeling good about it too. I managed to get connected to my network printer ok and I've mapped server shares before on a Mac. I have about 10 or 12 shares to mount representing various projects, shared doc storage and servers that I keep track of and visit from time to time. This turned out to be an exercise in frustration. As I see it at this point (and this may change when I find out the myriad of things I don't yet know) Mac's don't really like to play nice as windows network clients. For one thing there's no drive letter.
I knew this and expected it of course, but what I did NOT expect was the complete inability to go to the file explorer, enter a UNC path and see the share content. Surely there is something on a Mac that allows me to simply browse UNC paths ad hoc without the necessity of going through the whole "connect to a server" dance. And please, if you are going to clue me in, don't forget to tell me the shortcut keys. I go hours without touching the mouse on a PC, but the Mac seems to want me to "drag" things around to make use of them.
Once I did get a drive "mapped" (sort of) using the "connect to a network server" widget, I could only see it in the "finder" under "Devices" (not "drives" or "shares" or "network resources" but "devices"?). Furthermore I could not seem to rename the "device" once I had established the linkage. This was a problem to say the least because I had shares of the same name. For example, I mapped to ServerA with a UNC of "\\serverA\webs" and ServerB with a UNC of "\\serverb\webs". In my "devices" I now saw 2 devices both of which were named "webs". There was no way to simply rename them so I could tell them apart either. I did find I could make a sym link (an alias) to these drive mappings and rename that link. I did that on the desktop and that got me a little further.
Now, I went to open some projects (in eclipse) at these locations and I had a terrible time finding them from the browse application. Some command line searching (thank god the Linux command line is still operable) and it turns out these mapped drives are actually linked to the "volumes" folder. Well of course they are – Linux under the hood remember Mark!!
Remember my "webs" example? Closer examination showed that I had a /volumes/webs folder (mapped to serverA) and a /volumes/webs-1 folder (mapped to serverB). But in the chooser I had 2 "devices" both of which simply said "webs". And here's the kicker - clicking on either one of the devices opened only /volumes/webs-1. In other words, the short cuts in chooser were crossed up and pointed to the same share. If I navigated to the volumes folder on my own and clicked on one or the other I could get the content I was looking for, but neither the chooser nor the aliases on the desktop seemed capable of getting me to both locations.
First Take
I'm impressed with the speed and the aesthetics of the system. I suspect it will take me a week or two to really feel comfortable. Next I have to install the IOS SDK for Iphone development, Skype, Photoshop, Parallels and a few other widgets to make it more usable. The screen is also impressive. Chrome and FF both work quite well. Eclipse seems to load and run adequately. I am also seeing why some people (like super genius and senior CF Webtools ColdFusion and Java developer Guy Rish) prefer a really giant screen to several smaller ones. I have 3 21 inch monitors and I thought that was pretty grand - but that 27 inch monitor really makes a difference with Eclipse. I can get code, debugging, log tailing and file explorer on the screen without sacrificing a clear view of any of them.
Final Caution
As you can tell from this post, I'm not afraid to put myself out there. I'm trying something new and I want to share, both personally and professionally, my take from the experience. I welcome comments to my blog as all my readers know. Indeed, some of the best coldfusionmuse.com content is often found in the comments. So with that caveat, I want to say that this is not the time for the old Mac vs. Windows debate. If you wish to flame and draw out that argument I can assure you that your comments won't last here. Please keep the discourse civil. If you have tips about how to help an old windows hand get the hang of a Mac, that would be splendid. If you want to address any of my specific points in this post - have at it. But if you only wish to jump in and start a holy war, please refrain. I'm sure there is plenty for us to learn without resorting to useless and trivial arguments. Ok... now that wasn't so bad was it Muse readers? Don't worry - the Novocain wears off in about 90 minutes :)
IIS 7 and the Web.config File
If you are new to IIS 7 you may not know about the Web.config file. This file acquires its initial properties from the global settings that you set at the server level (as opposed to the site level). If you make certain changes to the global settings (like adding a default doc for example) then a new web.config file is automatically created and put in the root of each site you add. Or possibly it's created when you fiddle with the site specific settings and deviate from the global settings. I'm not clear on when it is and is not created. But you can of course create one for yourself. The format looks like this:
<configuration>
<system.webServer>
<defaultDocument>
<files>
<clear />
<add value="index.cfm" />
<add value="index.html" />
<add value="Default.htm" />
<add value="Default.asp" />
<add value="index.htm" />
<add value="iisstart.htm" />
<add value="default.aspx" />
</files>
</defaultDocument>
<directoryBrowse enabled="false" />
</system.webServer>
</configuration>
An important note (and something I just ran into today) is that this file is pretty specific to the server you are on. It's not a great idea to put this file in your source code repository and have it deployed for example. In our case we deployed the production file to staging. The production file had a specific line in it from the production web server that implemented DotNetDefender (a nice URL scoping filter that helps weed out DOS, buffer overruns and other pernicious attacks). Our dev site (which is all internal to our network) doesn't have this filter installed. When the web.config file was deployed it resulted in our requests all returning 404 errors. It took about 20 minutes of head scratching before I figured out what was going on. Imagine how panicked we would be if we had deployed a dev web.config file to production with the same result (yikes!).
Anyway, like many site specific files (ini files, sometimes Application.cfm or .cfc files etc) you should carefully consider whether you want this particular file to become a part of the "official" code base.
Finally, there are many things you can do with the web.config file - much like the venerable httpd.conf file. You can add rules for mod_rewrite, add specific redirects, control permissions for folders etc. It's a very versatile new tool in the IIS arsenal. And yes Wil, you can manipulate this and all other IIS properties from the command line. Indeed, with Win08r2 Core you don't even need a desktop to be running at all. How does that grab you?
Dynamic Compression on ColdFusion 9 and IIS7
Maybe your already know that web servers can compress outgoing content. Compressed content arrives at the browser which decompresses it and is able to render it. This is all generally seamless to the user and results in a more effective use of bandwidth. Now, compressing static files (like .html files) is a no brainer for web servers. They simply pre-compress the files and store them in a file cache somewhere. When the original file is called for the web server serves up the compressed file instead.
Dynamic files are more problematic. There's no correlation between the file name and the buffered output of a ColdFusion page for example. Consider search results. One user might receive 10 results and another user might receive 10 completely different results. Still another user might receive 100 results. How is the web server supposed to compress that data? Like your app server it does it "on the fly". It waits for ColdFusion to return the response buffer, compresses the file in memory (as I understand it) and then outputs the buffer to the browser. At least that's the way it works in theory. In practice you might find that ColdFusion 9 and IIS 7 don't quite have this figured out yet.
Before I give you the blow-by-blow (and thankfully a solution) I want to make it clear that this problem and solution come to me by way of my good friend and colleague Vlad Friedman of Edgeweb Hosting. EdgeWeb consistently receives the highest possible reviews from its customers and Vlad is one of the brightest folks I know in our corner of the IT world. Now let's talk about our little problem shall we?
Read MoreSites to Watch
Aggregators
RSS
