Spam Statistics #2

Some of you who had read our old NSCorp blog would recall that we love periodically looking at spam statistics in order to best tweak our spam systems.

A question was proposed on a hosting forum board a few days ago of just why do spammers target secondary/tertiary mail servers, and we ofcourse suggested it was due to two things:

1) The secondary mail server is more likely to accept the email due to various configurations out there of relaying whole domains rather than individual accounts. Thereby increasing the chance of at least getting a non-delivery bounce being sent to somebody (a bounce is better than no bounce).
2) When the spam software on the primary gets the email, anti spam software is less likely to look at past hops rather than the immediate one, and due to the trust relationship with the secondary/tertiary mail servers, it is less likely to be detected as spam.

That’s all good, reasonable logic behind why spammers would do such things. But do they? So we decided to analyse our mail servers and check what volume of spam really do come in via the primary vs. secondary vs. tertiary mail servers over the month of April:

So there we have some raw numbers…..suggesting that of the email the primary server receives, between 48% and 66% of the email is spam, while the figures are between 80% and 94% for the secondary and tertiary mail servers. These figures aren’t actually too much of a surprise as we ran up similar numbers internally late last year.

While we’ve got the numbers open, let’s do a day-by-day analysis for April too:

So it looks like Tuesday/Wedneday are the biggest days for receiving spam by around 20%, and the weekends the lowest.

There are some limitations on the above stats that could affect the quality of the stats:

1) Some RackCorp customers have greylisting turned on on their accounts/aliases. The above stats are only based upon what has happend AFTER greylisting – so it’s probable that the spam % stats are actually higher.
2) The per-day stats are based upon total emails – more real emails are sent on certain days, which would impact upon the stats.
3) For heavily spammed domains, RackCorp employs a tertiary MX record that accepts connections but never accepts email. This means that some spamming software would send, not get the error, and would never try again. Once again this means it is probable that the spam % stats are higher.

So that wraps up this stats session. There’s nothing much we could use there to improve services, except perhaps increasing the score applied to all emails received via secondary/tertiary MX servers. For now….we’re fairly happy with our anti-spam services. If things get worse, we might have to fall back to this information.

– RackCorp

Power outage on uk-rs-vhost*, uk-rs-cs*

As reported, on Wednesday, 30/04/08 at 21:36 AEST, the primary datacentre used by RackCorp for all UK equipment suffered a power issue resulting in power being cut to all equipment for a few minutes. The following is the accepted publication of cause by the datacentre:

The nature of the UPS failure was unique according to the manufacturer. There are over 3000 systems of the same model installed worldwide, and not one of these has experienced a similar failure. The failure was extremely unusual and unexpected, and the UPS manufacturer is having an engineer from Switzerland over to examine the failed system in more detail. The manufacturer has not yet determined the exact cause of the fault and as such we are not in a position to update you on what action we will be taking.
When the UPS failed a number of capacitors and other electrical components were overloaded and burnt out, this in turn triggered our fire alarm and all employees were evacuated from the building as part of our standard procedure to ensure their safety. [....]

Their publication goes on to acknowledge that the fire alarm combined with a number of software issues caused further problems with other services operated by them. We’ve had some queries from clients, and can confirm that the ONLY item above that affected any RackCorp client was the power outage itself. Servers were offline for a few minutes, and soon powered back on. Due to the sub-5 minute outage interval, offsite switchover did NOT occur on any RackCorp UK hosted services.

On the up side, we can proudly say that EVERY UK-RS server managed by us booted up in a flawless manner on the day, without incident. All equipment was checked over the following 26hrs by our staff and it is understood that no detectable damage is understood to have occurred to any equipment in use by us or our customers. This result can be attributed to our procedural approach to server management, and many of you have already complimented us on that outcome – so on behalf of the staff who were working on that night we certainly thankyou for your kind feedback.

– NetOps

php_network_getaddresses failed

We came across a problem today which we have seen a few times before. They were attempting to install wordpress akismet and when attempting to verify their API key they would always receive the following error:

There was a problem connecting to the Akismet server

This rather trivial message in this instance, and in other instances has been due to a badly set up chroot environment. The actual PHP error being generated is this one:

Warning: fsockopen(): php_network_getaddresses: getaddrinfo failed: Name or service not known

Which simply means that it could not resolve the hostname into an IP address in this instance. There is talk over at PHP bugs about it in which it has been repeated many times that this is NOT a PHP bug! Unfortunately people tend not to read before they post….

Anyway, the important files that you are going to need to make sure are in your chroot environment (with appropriate permissions) to resolve this problem are:

/etc/resolv.conf
/etc/hosts
/etc/nsswitch.conf
/etc/protocols
/etc/services
/etc/host.conf
/lib/libresolv.so.*
/lib/libnss_files.*
/lin/libnss_dns.*

And if you have any wierd and wonderful domain configuration going on, you may need other libnss files too.