Apache 2 + FastCgi + PHP 5 compile / build

Building the three of these is fairly easy, but a bit tricky if you’re used to how it worked in previous versions of Apache or PHP.  Please note that we install everything into prefix trees and use a tool called graft to manage the versions.  If you simply install your versions into default locations, you can probably leave out anything to do with prefix=, and top_dir.  We also download all install archives into /usr/local/PKG_BUILD.

Anyway to get started, the first trick is to build Apache 2 first by itself.  Here is an example:

cd /usr/local/PKG_BUILD
tar -xvzf httpd-2.2.8.tar.gz
cd httpd-2.2.8
./configure –prefix=/usr/local/PACKAGES/httpd-2.2.8 –enable-file-cache –enable-cache –enable-deflate –enable-mime-magic –enable-expires –enable-headers –enable-version –enable-ssl –enable-http –enable-cgi –enable-rewrite –enable-so –with-ssl=/usr/local/PACKAGES/openssl-0.9.8
make
make install

This will install Apache into /usr/local/PACKAGES/httpd-2.2.8.  If you want to make some modules shared (such as how we’re going to build mod_fastcgi shortly) then you can do so by adding things like: –enable-rewrite=shared.

The first part of installing fastcgi is to download fcgi from http://www.fastcgi.com.  To install this:

cd /usr/local/PKG_BUILD
tar -xvzf fcgi-2.4.0.tar.gz
cd fcgi-2.4.0
./configure –prefix=/usr/local/PACKAGES/fcgi-2.4.0
make
make install

The next step is to download mod_fastcgi (this is a DIFFERENT package to the fcgi one).  You can also download this from http://www.fastcgi.com/

cd /usr/local/PKG_BUILD
tar -xvzf mod_fastcgi-2.4.6.tar.gz
cd mod_fastcgi-2.4.6
cp Makefile.AP2 Makefile
make top_dir=/usr/local/PACKAGES/httpd-2.2.8/
make install top_dir=/usr/local/PACKAGES/httpd-2.2.8/

This will actually find your httpd source and compile mod_fastcgi for you, as well as install it with the Apache 2 modules! (Easy huh?).

The final step is to download and install PHP 5.  You can obtain PHP from http://www.php.net.  After downloading:

cd /usr/local/PKG_BUILD
tar -xvzf php-5.2.4.tar.gz
cd php-5.2.4

Now the next part will no doubt differ depending upon what features you want to compile PHP with.  That’s left up to you and google to work out how to compile the other modules.  I’ll show what we compile some of our systems with, but the only ones that are really applicable to this blog are the –enable-force-cgi-redirect –enable-fastcgi ones.

./configure  –prefix=/usr/local/PACKAGES/php-5.2.4-norm –with-openssl=/usr/local/PACKAGES/openssl-0.9.8d –with-zlib –enable-force-cgi-redirect –enable-fastcgi –with-mysql=/usr/local/PACKAGES/mysql-5.0.41 –enable-static –with-config-file-path=/etc/php.ini –with-imap –with-mcrypt –with-curl=/usr/local/PACKAGES/curl-7.16.1/ –with-kerberos –with-imap-ssl
make
make install

And that’s it for the compiling stage.  I’ll put up a blog at a later date on how to configure Apache 2 to work with FastCgi.  Stay tuned.

Spam Statistics #2

Some of you who had read our old NSCorp blog would recall that we love periodically looking at spam statistics in order to best tweak our spam systems.

A question was proposed on a hosting forum board a few days ago of just why do spammers target secondary/tertiary mail servers, and we ofcourse suggested it was due to two things:

1) The secondary mail server is more likely to accept the email due to various configurations out there of relaying whole domains rather than individual accounts. Thereby increasing the chance of at least getting a non-delivery bounce being sent to somebody (a bounce is better than no bounce).
2) When the spam software on the primary gets the email, anti spam software is less likely to look at past hops rather than the immediate one, and due to the trust relationship with the secondary/tertiary mail servers, it is less likely to be detected as spam.

That’s all good, reasonable logic behind why spammers would do such things. But do they? So we decided to analyse our mail servers and check what volume of spam really do come in via the primary vs. secondary vs. tertiary mail servers over the month of April:

So there we have some raw numbers…..suggesting that of the email the primary server receives, between 48% and 66% of the email is spam, while the figures are between 80% and 94% for the secondary and tertiary mail servers. These figures aren’t actually too much of a surprise as we ran up similar numbers internally late last year.

While we’ve got the numbers open, let’s do a day-by-day analysis for April too:

So it looks like Tuesday/Wedneday are the biggest days for receiving spam by around 20%, and the weekends the lowest.

There are some limitations on the above stats that could affect the quality of the stats:

1) Some RackCorp customers have greylisting turned on on their accounts/aliases. The above stats are only based upon what has happend AFTER greylisting – so it’s probable that the spam % stats are actually higher.
2) The per-day stats are based upon total emails – more real emails are sent on certain days, which would impact upon the stats.
3) For heavily spammed domains, RackCorp employs a tertiary MX record that accepts connections but never accepts email. This means that some spamming software would send, not get the error, and would never try again. Once again this means it is probable that the spam % stats are higher.

So that wraps up this stats session. There’s nothing much we could use there to improve services, except perhaps increasing the score applied to all emails received via secondary/tertiary MX servers. For now….we’re fairly happy with our anti-spam services. If things get worse, we might have to fall back to this information.

– RackCorp

Power outage on uk-rs-vhost*, uk-rs-cs*

As reported, on Wednesday, 30/04/08 at 21:36 AEST, the primary datacentre used by RackCorp for all UK equipment suffered a power issue resulting in power being cut to all equipment for a few minutes. The following is the accepted publication of cause by the datacentre:

The nature of the UPS failure was unique according to the manufacturer. There are over 3000 systems of the same model installed worldwide, and not one of these has experienced a similar failure. The failure was extremely unusual and unexpected, and the UPS manufacturer is having an engineer from Switzerland over to examine the failed system in more detail. The manufacturer has not yet determined the exact cause of the fault and as such we are not in a position to update you on what action we will be taking.
When the UPS failed a number of capacitors and other electrical components were overloaded and burnt out, this in turn triggered our fire alarm and all employees were evacuated from the building as part of our standard procedure to ensure their safety. [....]

Their publication goes on to acknowledge that the fire alarm combined with a number of software issues caused further problems with other services operated by them. We’ve had some queries from clients, and can confirm that the ONLY item above that affected any RackCorp client was the power outage itself. Servers were offline for a few minutes, and soon powered back on. Due to the sub-5 minute outage interval, offsite switchover did NOT occur on any RackCorp UK hosted services.

On the up side, we can proudly say that EVERY UK-RS server managed by us booted up in a flawless manner on the day, without incident. All equipment was checked over the following 26hrs by our staff and it is understood that no detectable damage is understood to have occurred to any equipment in use by us or our customers. This result can be attributed to our procedural approach to server management, and many of you have already complimented us on that outcome – so on behalf of the staff who were working on that night we certainly thankyou for your kind feedback.

– NetOps

php_network_getaddresses failed

We came across a problem today which we have seen a few times before. They were attempting to install wordpress akismet and when attempting to verify their API key they would always receive the following error:

There was a problem connecting to the Akismet server

This rather trivial message in this instance, and in other instances has been due to a badly set up chroot environment. The actual PHP error being generated is this one:

Warning: fsockopen(): php_network_getaddresses: getaddrinfo failed: Name or service not known

Which simply means that it could not resolve the hostname into an IP address in this instance. There is talk over at PHP bugs about it in which it has been repeated many times that this is NOT a PHP bug! Unfortunately people tend not to read before they post….

Anyway, the important files that you are going to need to make sure are in your chroot environment (with appropriate permissions) to resolve this problem are:

/etc/resolv.conf
/etc/hosts
/etc/nsswitch.conf
/etc/protocols
/etc/services
/etc/host.conf
/lib/libresolv.so.*
/lib/libnss_files.*
/lin/libnss_dns.*

And if you have any wierd and wonderful domain configuration going on, you may need other libnss files too.

RackCorp Industry Blog

Hey Everyone,

Welcome to our new RackCorp branding, and just as importantly, our new RackCorp blog!

For anybody who is new to the history of RackCorp, I might just give you a quick history as to how RackCorp came to be. RackCorp is both owned and operated by Network Synergy Corporation Pty. Ltd, an Australian company that has been around since 2003 in it’s present form, and has a history dating back to 1996 in terms of acquired staff, customers, services, equipment, contracts, and most importantly – experience!

Network Synergy Corporation (NSCorp) has never been a one size fits all company, nor have we ever been a follower in terms of technology. After acquiring 2 web services companies at the end of 2007, we found ourselves maintaining 9 websites, and 18 service portals either on behalf of our own services or services that we had acquired over the years. The maintennance for this many platforms (many of which were acquired in states of high-maintennance) was so high that innovation had been at an all time low throughout 2007. We sat down with management in January this year and made a big decision to invest heavily into combining all of our service portals into one – and if you haven’t already worked it out, this is where RackCorp.com came from!

In later blogs I hope to show off some of the great technology that we’ve built in-house over the years, and hopefully I’ll finally be able to stop leaving customers bemused by always saying “Yeah we can do that, in fact we could have set you up on that years ago”. In doing so I really do hope that our customers can finally get that little something more out of RackCorp that wasn’t available before.

So that’s it for our first blog. A bit of a tease I know, but we’ll keep new blogs coming quick and fast over the coming weeks. There really is months of tech stuff to show off and talk about here at RackCorp, so do check back with our blog regularly as I’m sure there will be plenty of things to interest everybody!

– RackCorp