If you stare deep enough, you can find the answer to all your problems

[andrewducker]

I am old enough to remember an internet without spam. I was online a good two years before the Green Card spam first hit the newsgroups in 1994.

This means that I remember when you didn't get any email, except from people who had something to say to you. They might be complete wierdos wanting to pick an arguemtn, , but they were an individual, sending you something personal. This was before companies really discovered the internet, so I wasn't even getting emails from Amazon, as they didn't come into existence until 1995.

Anyway, nowadays a vast amount of my email is spam. I don't both to keep track of it, and thankfully I'm sitting behind a series of filters - spamassassin throws away anything it scores over a 10. Anything scored 5-10 gets passed on to me with ****SPAM**** in the subject line. And then Thunderbird's adaptive filter has a go at what's left.

This kills off 95%, leaving about 20 a day to delete by hand. The vast majority of which are part of a fascinating development in spam - one which has stymied the majority of spamfilters - putting the text into an image.

Embedded images along with random text make it almost impossible for the content to be scanned. Some spam filters have started putting OCR in,to catch this - but then the spammers are altering the text enough to make it extremely hard to OCR it. It's entirely possible that this escalating warfare between spammers and spamfilterers will end up finding new wrinkles in OCR technology and visual perception, in their attempts to produce text that can be read only by people/produce code that can read said text.

Eventually I expect spam to consist of magic-eye pictures, where you haveto stare really hard at swirls of pseudo-random text until the name of the stock they want you to buy appears. Only three percent of the population will be able to actually see these messages, but these will also be the three percent of people who both have loads of money, and are entirely gullible about the stock market.

Comments

[nameandnature]

I've never thought content filtering (spamassassin, Bayesian filters and so on) would do the job: it's an arms race where the spammers can always find something which is legible to humans but not to computers. There's a diminishing return for the spammers, I guess, since they mostly still need to get you to visit a website and most of their buyers probably need something to click on rather than type into the address bar of their browser, but if URLBody style filtering becomes popular enough, they'll switch to hosting their sites on compromised PCs which aren't yet blacklisted anywhere. And these latest pump'n'dump image spams don't need a link to click, of course.

I'm waiting on Spamhaus Zen incorporating their PBL. Then their users (including me) are going to stop accepting mail from the huge numbers of compromised Windows boxes that the spammers are currently using, without having to wait for the XBL to catch up. The image spammers have compromised so many machines that it's easy for them to pick those which aren't XBL listed and use them to get their spam through. That's the cause of the majority of spam filter escapes I'm seeing.

It's a shame that the PBL will just include vast swathes of consumer ISP space. I won't be able to run my own mail server off my ADSL line, but it's a small price to pay. There are already "dynamic IP" BLs out there, but I don't trust them as much as Spamhaus, so I'm not using them. I bet a lot of Spamhaus's big users are the same. If someone can do this properly, I think it'll be very effective.

I imagine the next stage in the arms race after that will be spammers using the outbound servers of the ISP whose network those compromised PCs are on. That and looking for networks the PBL isn't covering, or trying to take down Spamhaus itself.

[andrewducker]

The PBL is an interesting idea. I do think that ordinary users should be presumed to _not_ be using SMTP directly. They should be able to register to do so, of course.

Everyone using SPF would have a somewhat similar effect. Tricky to do though.

[nameandnature]

Everyone using SPF includes the spammers using it to say that anyone may send mail for their domains, of course :-)

The other problem is that it involves everyone doing something, as you've pointed out. That's one of Vernon Schryver's FUSSP points.

[andrewducker]

SPF doesn't _solve_ spam - but it can add to the weightings when working out whether a particular is spam. If a domain has SPF records, and the mail is valid according to them, and the domain is not on any blacklists, then that indicates a lack of spamminess. On the other hand, if a domain has SPF records, and the mail is not valid according to them, or has been blacklisted, then that indicates that it is spam. This can be done while ignoring SPF for all sites that don't have SPF.

If SPF ever gets massive traction then you can start assuming that sites without SPF are spammers - but that's a way off.