Wednesday, February 23, 2011

Spamware Hiccups

Once again I find my mail server logs filled with crap from a broken spam spewer, illustrating the inner workings of the spam software. In this case it's a snowshoe spammer residing in - - - - -

For well over 24 hours, they have been attempting to send email from "<info@##domain_rotation##>" and each time they have been had a 4xx message returned stating that the domain is unrouteable as per the default Exim config.

This shows that the spam software they are using would normally be configured with a range of domains, one of which is used for every message sent out. These computers are obviously misconfigured - either without a list of domains to use, or some kind of stuff up with the token replacement.

These particular spammers have reached my limit of allowable behaviour and are now firewalled from contacting any servers I manage. The state these IPs will be tested once a month for 6 months, then every 6 months thereafter, and only delisted once they no longer host a snowshoe spammer.

Whois info
network:Organization;I:org-139-8830208-0 c/o Network Operations Center, Inc.
network:Street-Address:PO Box 591

Looking at the hostname, I decided to look around the other IP addresses in the area and was quite shocked to find EVERY IP address that wasn't assigned to a seemingly legitimate customer was assigned to a snowshoe spammer. Surely, no legitimate host would do this? If I was sent a request to provide rDNS with this kind of smell to it, I'd be telling the customer go take a long walk off a short pier.

The complete list of rDNS lookups for - none - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - none

Monday, December 6, 2010

Thunderbird, Outlook and SSL Certificates for SMTP AUTH

This doesn't have much to do with spam directly, but more to do with running SSL protected SMTP AUTH and webmail servers.

I've been paying $70 to $240 a year for SSL certificates for a long time. Each year, going through the annoying task of validating it with tests that seemed to defy the amount of money I was throwing at them. How on earth can it cost $140 to send an automated email to a role account with a link to click on? Admittedly, sometimes is 2 or 3 emails! How could a blurry set of faxed identification papers possibly be considered enough to prove the non-domain email address was authoritative?

So how does that compare to the free Certificate Authorities? I can sign up for free server and client certificates with CACert or StartSSL and get the exact same level of guarantee that the certificate is legitimate. I get an email to a role account, click on a link and as a result, I can create a base level certificate that prevents 3rd parties from eaves dropping and clients can be relatively secure in thinking that they're connected to the correct host.

Actually, most users wouldn't even think about it that much, but more on that in a second.

Unfortunately CACert still haven't got their Root Certificate included in (m)any operating systems or browsers which means that I would still have to get people to manually installed the Root CA before they can get rid of any warnings. StartSSL however, has been included in Mozilla products and MacOS for a few years (2006?) and Microsoft finally came on board around 22nd September 2009 with a non-security update that included it. (You keep your client computers up to date right?) Opera is the final stickler. Finally, a free CA with some penetration that has a chance to rid us of warnings.

After all, warnings are the only things that users notice! Most end users wouldn't even notice the difference between an insecure connection, normal certificate secured connection and an extended validation certificate connection. People don't notice the little padlock at all, but the thing they do notice is if they get a warning they have to click on every time they visit a site. Removing the warning with a proper rooted certificate is the only thing the end user cares about and you'll be beating your head against a wall until it bleeds to get any more than that out of them.

By reducing the need to pay for every certificate, we can increase the amount of things being encrypted across the internet and hopefully start making things a bit more secure. As demonstrated by the FireSheep plugin for Firefox, everything is in the open on WiFi unless you are using some kind of encryption! Even StartSSL has more highly validated certificates, but at a hugely cheaper rate than a normal provider.

At another provider, every single certificate is validated and charged at a set rate. If you have 20 servers, that's 20 separate charges! When you start heading up the levels of validation at StartSSL, you are charged once for the validation after which you can create multiple certificates for multiple domains under your control. The exception being for Extended Validation certificates since they have different requirements. Overall, certainly a better way to run things than the price gouging of the more mainstream Certificate Authorities.

Given all that, I recently swapped SSL cert providers[1] to StartSSL who use an intermediate signing certificate to separate out the classes or levels of validation. Having never used one configured like this before, I simply installed the certificate like I had for all the previous ones by placing just the certificate content they gave me into a file and restarting everything. Alas, the following errors popped up in Thunderbird and Outlook which had me completely baffled. At least in Thunderbird I could accept the exception and that would be that, but Outlook insisted on popping up a message every time it was opened leading to quite a few complaints.

Thunderbird Error
Certificate Status
This site attempts to identify itself with invalid information

Unknown Identity
Certificate is not trusted, because it hasn't been verified by a recognized authority.
Outlook Error
The SSL Certificate failed one or more certificate validation checks
After much searching around the internet, I found vague mentions of chained certificates and needing to send not only the server certificate to the connecting client, but all Intermediate Certificates as well. I had presumed that because the Root CA was already installed on the client computer, that any Intermediate Certificates would also be installed. After a quick look on a Window 7, Windows XP and Ubuntu 10.x computers, not one of them had the Intermediate, but at least they all had the Root Certificate. Checking with some of the other CA's certificates, this turns out to be rather normal.

Without a local copy of the Intermediates and the hint that I could send more than one in the connection, I simply used cat (or copy in the Windows command line) to create a file that listed each of them in the chain, one after the other. Downloading and including the Root CA isn't necessary here unless you know you will have clients that do not already have it. It is much easier to have a message pop up in a mail client than stepping through downloading and installing a Certificate from scratch.

The StartCom Root CA:
The StartCom Class 1 Intermediate Server CA:

Once downloaded, combine them into the one bundle, ready to be served out to connecting clients.
cat server.pem ca.pem > server-combined.pem

On windows, the copy command the command line is just as fast.
copy server.pem + ca.pem + server-combined.pem

After adjusting the configuration files of all the services to use the new combined certificate file and restarting the services, all the errors when away. I hope this helps someone else having the same problem I did without the hours of trawling the internet looking for tidbits of information to stitch together.

Saturday, September 19, 2009

Webmasters! Web Developers! Heed this call!

Stop making your stupid contact forms use whatever random email address the user puts in as the FROM address of the email!

It's been a common spammer tactic for months now to send mail FROM an account TO an account. Most servers have been rejecting these emails for years anyway but it seems that Contact Form developers haven't heard the word yet.

When they try to use a filtering mail service that knocks out these blatant forgeries, as dictated by their carefully thought out SPF records, or because what the spammer entered into their web form was 100% spam, they of course BOUNCE! This bounce then become collateral spam to the innocent 3rd party who had their email address entered into the form, and now all a spammer has to do is use an automated script from his thousands of compromised machines to bounce storm his spam out to the world and ruin your web servers' reputation.

If you have a contact form on your website, please follow these simple guidelines and help make the world a saner place.

  • DO NOT TRUST USER INPUT - EVER. Take whatever they entered and sanitise it by only allowing known characters. A person's name should not need to have <script> tags in it. Don't forget about "Bobby'); DROP TABLE students;". For PHP, check out filter_var() which will even validate an email address in one call.

  • Use a WORKING email address to send the email FROM and do not allow the user to influence this. Bounces to this address should go to a human so problems can be spotted quickly, or at least some processing script that alerts a human if something is fishy.

  • If you can, do not allow the user to influence the TO address ..

  • Or if that's the whole point of the form, do not allow users to enter both content AND an email address that the form will send to. This is just asking for abuse.

  • DO NOT TRUST USER INPUT - EVER! I'm hoping that has sunk in by now. When using the PHP mail() function, look out for users including the characters "\r\n.\r\n" followed by their own email headers and content. It is a common mailform exploit which should have been weeded out 10 years ago. Keep in mid that the \r\n is not literally, but a single Carriage Return character followed by a single New Line character.

Now, if only there was some sort of qualification or test you needed to pass before you were allowed to program on the internet.

Monday, July 20, 2009

Silly spammers

Some of the spam that turns up in my traps stands out from the rest. Not only are they completely useless because they are spam, they are completely useless to the spammer because they screwed up the settings in their spam software. These screw ups give some insight into the working of their spam software.

Over the past few months the abuse of .cn (China) domains has been at pandemic levels[1]. There are several different styles of spam containing these domains differentiated by their template or structure. Some have simply the raw domain name, others include a random page, yet others have a random hostname and a page. While the text content of the email changes dramatically (randomly) between spam runs, the domain structure remains the same.

Spam that enters my traps is usually processed auto-magically by a parsing and checking program. Anything that doesn't fit a previously seen pattern is flagged so that I can have a look and see what has changed. Just the other day I got one that included the following curious looking URI.


As you can see, this is not a valid URI but instead the raw template text from the spam software. From this we can see why all their URI look the same. The {symbol[4-6]} would normally be substituted for a 4 to 6 letter word (in all the emails I have seen from this group/bot/template) even though it says "symbol". The two instances are evaluated at different times and the word is usually different. {_2cndomains} implies they have a list of .cn domains from which they choose one randomly. Since this URI string occurs anywhere from once to 10+ times in the a single email, we get to see multiple domains in every email.

The end result of all this - If they're using repeating pattern to send out millions (billions?) of spams every day, that pattern can be recreated and used to fight it. Even random characters are a pattern.


Sunday, July 5, 2009

And the answer is .......

No. Yes, it's no. Not yes, but no. Yes, indeed, email addresses just sitting around in blogspot pages (well at least this one) do not get picked up by the email harvesting spiders. The email addresses on the very poorly linked website were picked up inside a few months but none of the ones posted in here.

And now, a small update on hunting the best way to get spam. Warning - non-sensical semi rant inbound.

Signing up for all the scams from googling well known anti-malware, and clicking on the ads on facebook has given me a huge stream of "opt-out" & "CAN-SPAM compliant" spammers. They all dutifully provide unsubscribe links but I only have 3 working traps from the hundred or so signups I did so I'm loathe to see if any of them work lest I lose all this wonderful spam, although there is always the possibility it would promote further spams. There are several distinct groups that have somehow ended up with the addresses, most likely through the affiliate agreements that I was never given the opportunity to choose opt-into let alone opt-out.

Each group cycles their postal and main domain every couple days to several months, providing a seemingly endless supply of places I am required to opt-out. If I did start opting out, I'd have a lot of clicking to do. There are ethical qualms with getting spam this way - Is it really spam? Should I be trying to unsubscribe since I do not want this stuff (yet, I really do)?

Quite frankly, the crap I'm getting from those signups is completely and utterly useless. No one in their right mind would ever use any of the "products". Most seem to either be companies that made a poor decision on who to pay to market their product for them, some poor shmucks affiliate program being abused, or further scams. If they were arriving on my normal email account, I'd be pissed off. Someone got my email address in good faith for something I was interested in, and now it's being flooded with things that are not. To that end, I declare it is spam. Not a legal definition, but that's where the line has to be drawn.

If I gather all the information I can from these email addresses and use that information to block emails going to real users, am I doing something wrong? Some user has been silly enough to enter their email address into part 1 of a 2 part web form (part 2 is where you find out that this is going to cost money and that you've just been signed up for the affiliate program with your previous click because it's in the terms of service which you can read here) and they are now going to get flooded with all the same crap as my seeded addresses. Should that user have to try to unsubscribe from all the groups that are now going to be sending him crap? In a perfect world they should be able to go back to that original website and say that they no longer wish to participate and that would be that. Don't laugh - that's how it should be. Alas, that user will now come to me and want it all to stop thank you.

I don't unsubscribe because I want a current list of all the domains and servers in use by those spammers (I really do seem to have gone off trying to call these people anything but spammers regardless of their actual status) so that the user above never has to come to me. When they signed up for whatever it was, they will never get the first email, nor any of the others. If someone really wants it, I can allow that group through to their email address only and still keep everyone else safe and sane.

Is it spam? By definition probably not. Is it wanted? Hell no. Should I unsubscribe? No. I'm just collecting it all without rejecting it and collating information from it. Am I doing something wrong by blocking users from getting this crap in the first place? Hell no! Accessing my mail servers is not a right given to everyone on the planet. It is a privilege extended to those people trusted not to abuse it.

Thursday, September 18, 2008

Are spammers data mining the spamtraps and troublesome addresses?

A while back blogspot became a major headache for spam filtering. Spammers were using the free accounts to redirect hapless victims to their scam/pills/phising sites, bypassing the URIBL by using this whitelisted domain.

Thankfully, the URIBL added blogspot to the Grey list and allowed submitters to include the sub-domains in the Black list. The problem subsided, but the spammers had discovered a new source of really annoying URI and they were not going to give up that easily.

The spammers have since spread out to a number of free hosters, the most annoying for me is A unique sub-domain turns up in one of my spamtraps at the rate of around 1 per 5 minutes. Most of the domains or emails that turn up in my traps have already been submitted to the lists so I don't have to do much, however these livefilestore domains are always unique and no one else seems to submit the same ones as me.

This brings me back to a previous thought I had regarding these unilimited, unique sub-domains. Back when blogspot spams where at their peak, I noticed that within a day of submiting a sub-domain to the URIBL, the amount of spam received by that trap was reduced to a crawl.

I conducted an experiment at the time of only submitting blodpost sub-domains on a subset of the total spamtraps and was gobsmacked at the fall in the spam load on only the addresses from which I had reported. Just to make sure I wasn't seeing things, I submitted all the domains from all the spamtraps and observed the same dramatic fall on them all.

As a result of this, I stopped submitting blogspot domains to the URIBL for a while and wrote a script that clicked the "flag" button in the navigation bar instead. Spammers being what they are, the spam load on all the spamtraps (and quite a few new ones that still have me scratching my head) increased to the previous load over the next few months. The blogspot staff were relatively swift at removing the offending sites, and I still got my spam to train filters with ;)

Warp forward to the current day and the livefilestore sub-domains (the main domain of which has been listed in Black for over a month now). I seem to have an unlimited supply of spamtraps over quite a few domains since some moron decided to pick up the message ids in my posts to mailing lists and newsgroups as email addresses! For every email I have ever sent to a public forum, I get at least 5 distinct traps. I ended up configuring my mail server to just trap anything sent to an address that looks like a message id. They aren't even close to any legitimate email addresses on the domain so there is no possibility of a typo finding its way into a spamtrap.

The end result of which is that I have a lot of unique traps. Over the past few weeks, those traps have been getting mostly unique livefilestore sub-domains delivered to each of them. There have been a few repeats, but they mostly happen when an email is dilvered to more than one address in a single connection. Upon submission of these sub-domains, there was an increase in the spam load over all, but to different traps. By now, most of the addresses I had seen already, had a reported URI on them. It almost seemed like a systematic delivery to every single address to feel them out.

Following the weekend lull, the livefilestore domains started comming thick and fast across all the traps again. They still seem to end up in traps that have never seen spam before, but there are a larger quantity heading to previously well known addresses too. Tentatively probing further addresses while dumping crap on known "working"#1 addresses.

If only they knew that I only get a daily summary and everything else is handled by computer, never to be read by a human and hopefully helping countless humans from seeing their crap :P

So are spammers using unique sub-domains to data mine the location of the email addresses that report them? Are they using this to help increase the effectiveness of their spam runs by avoiding those who would report them? I probably haven't put my story forward well enough to tell, especially since I don't provide raw figures to play with (I don't have the metrics in place to gague things properly). Hopefully, I've put enough of an idea forward for someone to have a think about this and look at their data to see if the same idea fits there too.

... of course, that only works if someone reads this ;) As if!

#1 I permanently reject all messages sent to spamtraps

Monday, July 28, 2008

The crawlies have found my hooks

The seeded address that I lost back in March (Lost in the ether) turned out to be hidden in the most straightforward place - on my home page for the first domain I ever registered.

It has turned out to be one the most successful spamtrap seeds, considerably out-doing the dynamically generated email address present on the same page.

The dynamic address includes a static component as well as the day and IP address of the computer that accessed the page encrypted with a simple Caesar cipher to make it look like a long english address. Enough information to find the all related accesses in the web server logs. Every different host getting a different email address must make it look too fishy for the spammers as I've only recieved one spam on that type of address, while the static address has been getting progressively more emails as the months go on.

Since starting this adventure, only 4 of 50 seeded addresses have given any results at all and only one has received multiple messages. The email addresses I have actually used in the past to subscribe to published mailing lists, bug trackers, and forums have been a consistently strong source of spamtraps. The best ones have always been on well indexed on a number of sites, or included in the source code.

There are a number of other locations I have considered planting seed addresses that I haven't had the opportunity to explore yet, but I expect to play with them in the coming months. They should provide a significantly better success rate.