Multiple A DNS records and browser retries failover / redundancy

I wanted to know how firefox / chrome etc behave when there are two A DNS records for a website, and one of those IPs does not respond.  The use-case for this is to have the same website hosted in two locations, and when one server/network becomes offline that the other one can take over.

I couldn’t find any up to date info about this so I thought I’d just do some tests and compile my own info. Test case used only two A records for the same hostname.

NOTE: If the connection returns an error sooner, presumably these browsers will failover faster. The test here was a complete drop of traffic with no TCP reset/icmp error coming back from the first attempted IP.

Using multiple DNS A records is a plausible failover mechanism in modern browsers!

PROBLEM: Browsers seem to switch between IPs after a period of inactivity (even just a few minutes).  If your site requires state, you will need to make state be stored inside the browser (preferably) or some kind of shared state system between servers.

Pros:
Multi-server failover is relatively easy, and fast in modern browsers
Traffic is balanced between both locations

Cons:
If one A record server stops responding, the user may experience a delay when loading the site
If one A record server stops responding while the user was already on the site, they may see an error until they reload the page.
If state required, need to store in browser or shared state system

End of Life – PHP version 4 webhosting

PHP version 4 was officially retired 8 years ago and no longer receives any kind of official patches or support. To preserve functionality for our resellers who have customer services requiring PHP v4 we have maintained our own patch subset for PHP v4 since 2008.

Our volume of PHPv4 websites has now reached a low enough level that it no longer makes business sense to maintain our PHPv4 Webhosting services. Effective 1 August 2017, we will no longer offer / support PHPv4 websites.

We recommend to all customers to either upgrade their websites to PHPv5, PHPv6, or PHPv7. You can select the different version from our webhosting panel to temporarily change your php version to see how it may affect your website. We would recommend trying PHP 5.2.

If you need help with migration we recommend placing a RackCorp support ticket. Any sites still operating on PHPv4 will be automatically upgraded to PHPv5.2 on August 1st. Please note there is a VERY HIGH chance that this will break sites that were built for PHPv4.

Virtual Server Image Backups – Performance Improvement

Tonight we’ve deployed the latest version of our Virtual Server backup manager. This software is responsible for:
1) Creating LVM snapshots of Virtual Servers
2) Reading entire snapshots, 1MB at a time and generating data hashes
3) Comparing those data hashes to our backup repositories (usually in off-site datacenters)
4) Copying any changed blocks to our backup repositories

Traditionally, most of our customers run on small VMs (20GB-30GB), so the above process has worked quite well albeit not optimal. This year we’ve seen a significant uptake of large 200GB+ VM’s which has made us re-evaluate our backup manager. Tonight’s update is a significant one in terms of performance:

• Change 20150516-001: Backup Repository no longer re-calculates hashes during a backup. They’re calculated once when the backup is first written, used for comparison purposes, and updated as new blocks come in.
• Change 20150601-001: snapshot blkio limiting. Previously, backup processes were allowed to utilise full sequential read speeds due to oversight in the blkio settings. This has now been remedied, with sequential reads now limited to 60% of the virtual server’s allocated device speed.
• Change 20150601-002: posix_fadvise(POSIX_FADV_SEQUENTIAL) utilisation. Added libc binding to make a call to posix_fadvise to set the POSIX_FADV_SEQUENTIAL flag. This should mean the Linux Disk Cache doesn’t get blown away during backups.

Coming up in the future will be the much requested feature addition to allow file-level mounts of backups!

Cisco ASA 5512-X Allow outside/external ssh access

To enable external SSH access to a brand new Cisco ASA 5512-X, follow these example instructions:
X.X.X.X is the IP address you wish to allow access from.
yourusername is the username you want to use
XXXXXXXXXXXXXXXXX is your password

ciscoasa# conf t
ciscoasa(config)# ssh X.X.X.X 255.255.255.255 outside
ciscoasa(config)# crypto key generate rsa general-keys modulus 2048
ciscoasa(config)# username yourusername password XXXXXXXXXXXXXXXXX
ciscoasa(config)# aaa authentication ssh console LOCAL
ciscoasa(config)# ssh version 2

You may also want to set an enable password:
ciscoasa(config)# enable password XXXXXXXXXXXXXXXXX
(You should probably make this different to your user password)

CVE-2015-0235 “Ghost” Linux glibc vulnerability

For those that missed it, CVE-2015-0235 (aka Linux Ghost) was announced today which details a glibc library bug that is still on many Linux distributions. glibc is used by many applications including webservers, mail servers, php applications etc.

The specific bug was in the gethostbyname() and gethostbyname2() functions (hence “ghost” name!), so only applications that call these are potentially vulnerable. Even then, there is limited scope for exploitation, but there already has been a PoC for the Exim mail server developed so it certainly is possible (given the right conditions). Luckily, these two functions lack IPv6 support, so many newer applications and services have chosen to stop using these functions, and instead use IPv6-enabled functions instead. As has been seen however, some popular ones such as Exim do still use the older IPv4-only functions.

The bug itself has been around since 2000, and was inadvertently patched in August 2014 without realising the implications. Unfortunately since the security issues were not detected at the time, many Linux distributions didn’t back-port the patch into Linux distributions. This is what has occurred today.

Accordingly, we have now taken the following actions:

We will be taking the following actions that may result in a few minutes downtime for some sites tonight:

In addition:

In terms of customer patch cycles, we are treating this as a critical bug for some customers, and moderate (normal patch cycle) for others depending upon the attack vector surface area. All affected customers will have received an email accordingly. If you are unsure of the impact for your specific services, please raise a support ticket accordingly.

Additional useful resources:
Ars Technica Writeup on Linux Ghost
gethostbyname() vs getaddrinfo() by Erratasec