ColdFusion Muse

Using CDN for Entire Website and Country Blocking - Part 3

This is Part 3 in a short series of articles about blocking entire countries from a website. Parts one and two cover CloudFlare and CloudFront.

CF Webtools has been asked numerous times to block an entire country or countries by many clients. The issue is that there's a lot of hacker activity from certain identified countries and the client(s) does not do any business with those countries. Typically it's entire server hacking attempts, but more recently it's to use the client's shopping cart to "test" stolen credit cards. This is a very serious problem and as such clients are asking us to help them prevent this from happening. One potential solution is to block the IP addresses that these attacks are coming from. I refer to this as the Whack-A-Mole method because it's just like that arcade game. As soon as you block one IP they switch to another IP address.

We need a better solution. I looked into what we could do and how reasonable and feasible the various options are in terms of technology and cost. In my previous two articles I wrote about using CloudFlare and AWS CloudFront. In this article I'm writing about using a slightly better hammer in the Whack-A-Mole method to block entire countries. This is one of the simplest but also least effective methods.

The option many of us have traditionally done is blocking problematic IP's on a case by case basis and in extreme cases blocking entire IP ranges. I've often referred to this as the Whack-A-Mole method. It's reactive and not proactive. A real hacker would not use their own personal IP and there is no guarantee that the IP will always remain with an unscrupulous user. Normally I do not block an IP because bad stuff happened from that IP once. However, I have noticed the same IP or IP ranges launching attacks on multiple unrelated, hosted at different locations, and different client's servers. That's when I start pounding the IP with the ol' Ban Hammer! Also, blocking and entire country with this method would mean being able to know all the possible IP addresses or address blocks assigned to a particular country. This is knowable!

I did some research on this and found a few very helpful resources. Resources like this http://ipdeny.com/ipblocks/ and this https://www.sitepoint.com/how-to-block-entire-countries-from-accessing-website/. These sites keep an updated list of IP addresses assigned to every country in the world. These are made available in the form of individual text files per country. And in the case of the SitePoint page, you can download a pre-scripted config file for many versions of web servers and firewalls. Hammer Time!

In the case of the country our client wants to block there are over 130 IP entries. These are in the form of CIDR IP ranges. This is the good news. The harder part here is that means there would have to be 130 plus entries manually added into IIS or a firewall. And this is for a smaller country. Larger countries, including countries that are known for hacking, have many thousands of CIDR IP ranges. But at least I can download the scripts for Apache and IIS from the SitePoint page and paste them into the respective config files.

What are the downsides to this method? First off I do not know if there would be any performance hit to IIS or Apache if we were to start entering thousands of IP restrictions. I do know that AWS restricts Network ACL's to an absolute max of 40 rules in their VPC's due to "performance issues" if more were added. We're still whacking at moles. IP assignments for countries can change thus you would need to update your static list of IP bans in your web server.

This is an example of how Apache 2.4 is configured.

This is an example of how the IIS XML web.config is configured. The CIRD notation needs to be converted to IP and network mask format.

In conclusion each option; CloudFlare, CloudFront, and IP Banning, each have their benefits and costs. CloudFront was the easiest of the three to setup and if the downsides of the IP address masking isn't an issue then it is likely the most viable solution. The AWS CloudFront solution may be best if you are already on AWS and you have an understanding of AWS Solutions Architecting. Both CDN options have country restrictions (and rate limiting) that will help in preventing potential credit card scammers from misusing your shopping carts. IP Banning is simplistic, it has no additional dollar costs. But it may be a performance hit to your web server if you have a very large number of IP restrictions. You may also have to update the IP lists if IP assignments to a country change. It's also worth noting that all methods can be bypassed via proxies.

CF Webtools is an Amazon Web Services Partner. Our Operations Group can build, manage, and maintain your AWS services. We also handle migration of physical servers into AWS Cloud services. If you are looking for professional AWS management our operations group is standing by 24/7 - give us a call at 402-408-3733, or send a note to operations at cfwebtools.com.

Using CDN for Entire Website and Country Blocking - Part 2

This is Part 2 in a short series of articles about blocking entire countries from a website. See Part 1.

CF Webtools has been asked numerous times to block an entire country or countries by many clients. The issue is that there's a lot of hacker activity from certain identified countries and the client(s) does not do any business with those countries. Typically it's entire server hacking attempts, but more recently it's to use the client's shopping cart to "test" stolen credit cards. This is a very serious problem and as such clients are asking us to help them prevent this from happening. One potential solution is to block the IP addresses that these attacks are coming from. I refer to this as the Whack-A-Mole method because it's just like that arcade game. As soon as you block one IP they switch to another IP address.

We need a better solution. I looked into what we could do and how reasonable and feasible the various options are in terms of technology and cost. In this article I'm writing about using Amazon Web Services CloudFront to block entire countries.

Amazon AWS CloudFront
AWS CloudFront does offer country blocking. I thought this would be an easy setup, but it isn't. When I tried to setup AWS CloudFront to 'front' an entire website I found there are many pieces that needed to be in place in order for CloudFront to handle the entire website.

Route 53 is needed or any other DNS that allows an ALIAS record for the Zone Apex record. This is because the Zone Apex record (root record) will be set to the URL provided by CloudFront and not an IP address.

Elastic Load Balancing is needed. The CloudFront origin (EC2 server) needs to be behind an TCP Elastic Load balancer. If there is only one site then the ELB target can be the instance itself. If the EC2 instance hosts multiple different sites, then we need to add multiple internal IP addresses to the instance and configure the origin site to be on it's own IP. Then the ELB should be configured to that internal IP address and not instance. If you are passing host headers in the CloudFront 'Behavior' section then you can have a single IP on the web server with multiple sites per usual for virtual name hosting. You have to setup the TCP ELB as TCP port 80 passthrough in order to pass the original IP addresses to the web server.

AWS Certificate Manager is needed to create a new free SSL for the domain name being setup in CloudFront. (I say it's needed because all sites should be using TLS protocols these days.) I found a wild card certificate works well.

Then lastly AWS CloudFront itself can be setup. The settings are a bit tricky. The Origin will be the ELB which will then pass requests to the EC2 instance. If you want or need forms to be posted to the website then you need to select "GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE" option for Allowed HTTP Methods. If you need to allow logins then you have to choose "All" for Forward Cookies.

There are costs to each part. Route 53 charges by zone and number of requests. Elastic Load Balancing charges by the hour and by data transfer amounts. Then Cloud Front charges by data transfer amount.

There are downsides to this method as well. In addition to the AWS method being harder and more complex to setup there are more costs involved. I can pass the original requesting IP address through to the web server, it still comes through in the X-Forwarded-For custom header. In Apache it's easy to globally capture and place this value into log files or the CGI scope. IIS does not allow this to be done at a global level meaning each IIS site must be configured for the custom headers. Additionally, you may need to custom code the web application to read X-Forwarded-For no matter which web server you are using.

After you have all of that setup, configured, and working you can now start blocking countries. This is done in the AWS CloudFront Restrictions section. You can add a Geo-Restriction blacklist or whitelist by country.

Part 3 will cover using IIS and Apache and a slightly better hammer in the Whack-A-Mole method.

CF Webtools is an Amazon Web Services Partner. Our Operations Group can build, manage, and maintain your AWS services. We also handle migration of physical servers into AWS Cloud services. If you are looking for professional AWS management our operations group is standing by 24/7 - give us a call at 402-408-3733, or send a note to operations at cfwebtools.com.

Using CDN for Entire Website and Country Blocking - Part 1

CF Webtools has been asked numerous times to block an entire country or countries by many clients. The issue is that there's a lot of hacker activity from certain identified countries and the client(s) does not do any business with those countries. Typically it's entire server hacking attempts, but more recently it's to use the client's shopping cart to "test" stolen credit cards. This is a very serious problem and as such clients are asking us to help them prevent this from happening. One potential solution is to block the IP addresses that these attacks are coming from. I refer to this as the Whack-A-Mole method because it's just like that arcade game. As soon as you block one IP they switch to another IP address.

We need a better solution. I looked into what we could do and how reasonable and feasible the various options are in terms of technology and cost. In this article I'm writing about using CloudFlare CDN to block entire countries.

CloudFlare
I was not familiar with CloudFlare other than it's a CDN. They do offer advanced services for a price. There is a free tier that has CDN capability and limited Firewall features. The firewall features include the ability to setup 5 firewall rules.

To test the features and capabilities of CloudFlare I created a free account for myself and setup my blog to use CloudFlare. My blogs uptime is not critical like the client's business is and it gets real traffic thus it can be used to test various features.

Using the free firewall features I can block multiple countries in a single firewall rule. The rules allow for chaining filters with AND OR statements. See the example below.

I don't know yet if there is a limit to the number of conditions I can add to a single rule. However, at the moment it seems to be sufficient.

The negative side effect that I can see so far is that all the IP addresses that get logged on the origin web server are from CloudFlare. This defeats many clients needs/desires to have a valid IP address of their valid customers. Cloudflare does offer the option to pass through the original HTTP headers, but that is under their top Enterprise plan. They do not provide a cost for this. You need to request an estimate.

CloudFlare does pass through custom headers that has the original IP and other custom headers. However, these are not standard and web servers need to be configured to first read the custom header fields and then the application code needs to be updated to use the custom headers fields. It's far easier to do this in Apache than it is in IIS. IIS does not allow this to be done at a global level meaning each IIS site must be configured for the custom headers. Additionally, you may need to custom code the web application to read X-Forwarded-For no matter which web server you are using.

Another issue is that CloudFlare requires you move your DNS to them. Depending on the client, gaining access to their DNS and registrar can be challenging.

Part 2 will cover using AWS CloudFront to achieve the same results.

CF Webtools is here to fill your needs and solve your problems. If you have a perplexing issue with ColdFusion servers, code, connections, or if you need help upgrading your VM or patching your server (or anything else) our operations group is standing by 24/7 - give us a call at 402-408-3733, or send a note to operations @ cfwebtools.com.

It's "Retired" Jim!

In another chapter of "The Cloud Never Crashes", I woke up Sunday to one of my AWS instances that was 'crashed' with a notice of "Amazon EC2 Instance scheduled for retirement". Retirement? What does that mean? I went to check my email and realized that the "retired" instance was the email server. Doh! It took me a little while to figure out what they meant. It means this "An instance is scheduled to be retired when AWS detects irreparable failure of the underlying hardware hosting the instance." This serves as a good reminder that the cloud is really someone else's server.

In theory this is an easy fix. The instructions at Amazon claims that stopping and restarting the instance will launch it on new hardware. In practice I could not get the instance to stop. This is where having physical hardware and a power cord to pull would have been nice. Failing to get the instance to stop I could not detach the EBS root volume. Even force detaching the EBS root volume didn't work. This is where daily snapshots of EBS volumes comes in handy. I was able to launch a new EC2 instance and then convert the last snapshot to an EBS volume and attach that to the new EC2 instance. Then I moved the elastic IP from the "Retired" instance to the new instance and hit "start'. Full recovery!

Now I'm left with a hanging EC2 instance that is still "Stopping" and an EBS volume that I cannot use, detach, delete etc. I tried reissuing stop commands a couple times. Eventually I noticed a "Force Stop" option. I do not remember seeing this on earlier attempts. I do not know if this shows up after the first failed stopped attempt or after several. I'm not sure, but I think that sends a trained monkey into the datacenter to pull the power cord. In any case it worked. This let me detach my EBS volume. From there was was able to stop the new instance, detach the EBS volume and attach my original EBS root volume. Now I have full recovery and I was able to clean up the loose ends.

Amazon Web Service has given us a new euphemism. Retired means It's Dead Jim!

CF Webtools is an Amazon Web Services Partner. Our Operations Group can build, manage, and maintain your AWS services. We also handle migration of physical servers into AWS Cloud services. If you are looking for professional AWS management our operations group is standing by 24/7 - give us a call at 402-408-3733, or send a note to operations at cfwebtools.com.

Authorize.NET Temporarily Ending TLS 1.1 and TLS 1.0 Support

At CF Webtools we have been preparing for this inevitable day for the past few years. We've been upgrading our clients servers and services to handle TLS 1.2 calls to Authorize.Net and other third party processors for a while now. Recently Authorize.Net announced a "Temporary Disablement of TLS 1.0/1.1" for "a few hours on January 30, 2018 and then again on February 8, 2018." This is in preparation for the final disablement of TLS1.0/1.1 on February 28, 2018.

As you may be aware, new PCI DSS requirements state that all payment systems must disable earlier versions of TLS protocols. These older protocols, TLS 1.0 and TLS 1.1, are highly vulnerable to security breaches and will be disabled by Authorize.Net on February 28, 2018.

To help you identify if you're using one of the older TLS protocols, Authorize.Net will temporarily disable those connections for a few hours on January 30, 2018 and then again on February 8, 2018.

Based on the API connection you are using, on either one of these two days you will not be able to process transactions for a short period of time. If you don't know which API you're using, your solution provider or development partner might be a good resource to help identify it. This disablement will occur on one of the following dates and time:

• Akamai-enabled API connections will occur on January 30, 2018 between 9:00 AM and 1:00 PM Pacific time.
• All other API connections will occur on February 8, 2018 between 11:00 AM and 1:00 PM Pacific time.

Merchants using TLS 1.2 by these dates will not be affected by the temporary disablement. We strongly recommend that connections still using TLS 1.0 or TLS 1.1 be updated as soon as possible to the stronger TLS 1.2 protocol.

This means that if you are using older methods to make calls to Authorize.Net that are not capable of making TLS 1.2 connections then you will NOT be able to process credit card transactions.

This affects ALL ColdFusion versions 9.0.2 and older! This also affects ColdFusion 10 Update 17 and older. If your server is running any of these older versions of ColdFusion and your server is processing credit cards with Authorize.Net then this advisory applies to your server.

CF Webtools has been successfully mitigating this issue for clients servers for the past couple years and we are very experienced in resolving these security related issues. In a previous blog post I tested which SSL/TLS levels were supported by various ColdFusion versions on various Java versions and produced an easy to read chart.

If your ColdFusion server is affected by this or if you do not know if your ColdFusion server is affected by this then please contact us (much) sooner than later. Our operations group is standing by 24/7 - give us a call at 402-408-3733, or send a note to operations at cfwebtools.com.

Can't Save applicationHost.config?

In an email group I am in there was a problem brought to the floor regarding removing the wildcard mapping from the applicationHost.config file in IIS 7.5. This file resides in the system32/inetsrv/config directory of your windows server and it contains the defaults for all sites. The defaults can be overridden using a web.config file in the root directory of an individual site. The user in question was trying to manually uninstall the connectors for ColdFusion 9. After going through all the steps there was one thing he could not seem to get rid of - a wildcard mapping for the jrun_iis6.dll in the handler mappings of IIS. He went down the list of things he had done and we all agreed that each file and step was complete - including removing the "global" mapping for this handler from applicationHost.config. While we were puzzling super guru and CFCelebrity Charlie Arehart (he of CF 411 fame) reminded us of a "gotcha" that occurs with files in the system32 directory and subdirectories.

The scoop is, if you open the file using a 32bit editor (say Notepad++) from system32/inetsrv/Config Windows does the old switcheroo and opens the identical (but unused) file in the SysWoW64/inetsrv/Config directory. You are editing the wrong file and you don't even know it (wow! ... or maybe I should say Wow64!). As an aside, this poorly named directory, SysWoW64, stands for "Windows on Windows 64 bit" - meaning files in this directory are "old 32 bit" windows files running on this "64 bit" version of windows. Not only is that unclear, it makes the continued use of system32 confusing. Why not just use system32 and system64? Ah well, I digress.

The Fix

The long and short of it is, to make sure you are editing the correct file, use notepad (the 64bit version ships with the OS). To avoid mistakes, open notepad first as Administrator, then navigate to and open the file in the /system32 directory. If you don't you will pull out your hair trying to figure out why your changes won't take. For more information check out this blog post on the issue by Mike Ratcliffe.

As always Muse readers thanks for you patronage. Especially thanks to those of you who have begun to pass referrals to folks who may need us - we are riding high because of you.

Fun and Games With Googlebot

When planning for scalability one of the things that is sometimes left out is the impact of indexing bots on your site. If you have a news or ecommerce site that is constantly changing, you definitely want bots to be indexing your site. How else are the latest and greatest products or stories going to show up in organic searches after all? But you also want bots to be well behaved. It would great if you could greet the bots at the door and say "Hey... it's 2:00am, not much going on so index to your heart's content." Or, "Whoa there fella - do you have a reservation? This is Cyber Monday and I'm afraid all our seats are full for paying customers. Can you come back in 12 hours?" But that sort of smart interaction is sadly not in the cards. Some bots have defined rules, some do not. Some honor things you put in the robots.txt file others do not. So here are some tips that might save you some time.

Read More

FileZilla Vs. FileZilla - Battle of the Century

Here's a problem that comes under the heading of the right hand being unaware of what the left hand is doing. Seriously the right hand is like sitting back, relaxing, maybe pushing the remote and the left hand playing the drums, waving at passers-by and tap-dancing wearing a little tuxedo. The right hand says, "Hey.. what in the ham sandwich is going on over there?" And the left hand says... Eh... I'd say that is as far as I want to go with the whole left-hand-right-hand thing.

The Problem

From time to time I end up troubleshooting an FTP connection. Like most system admins I hate FTP with a capital ick. Insecure, clunky, fault intolerant... It's like arriving at the Oscars in a VW Bus. It gets you there but there has to be a better transport than this! Where we need to support FTP (and always through a VPN please - do the rest of us a favor!), we use a product called FileZilla FTP Server. It's a great product and you can be up and running in about 5 minutes on just about any windows platform. Simply add users, folders, IP restrictions etc. and you are off and running.

The only problem is that occasionally people will call and simply can't get connected. The server is able to recognize them and verify their credentials but when it issues its first server side command (list I think) it times out and drops the connection. After a while I realized that the folks who were experiencing this problem were people running the FileZilla client. That's right - the FileZilla client has trouble connecting to the FileZilla server. If I asked them to use a different client the problem went away.

The Solution

A few weeks ago I finally figured out the solution based on a comment by super-genius-guru Wil Genovese of Trunkful.com who (I'm thrilled to say) works for the Muse and does miraculous things almost every day for our company and staff. He mentioned the problem with FileZilla and UTF-8 encoding. Some experimentation helped me determine an actual fix. Apparently the FileZilla client is passing the string UTF-8 for encoding (which looks ok to me) and the FileZilla server is expecting "UTF8". Fortunately the site manager has a way of specifying a "custom encoding" string. Click on a site then click on the "charset" properties tab. One of the choices is "custom character set". Choose it and enter UTF8 (without the dash). You should be able to connect fine after that.



Note: this problem doesn't exist with every version of client and server combination.