If you stare deep enough, you can find the answer to all your problems

[andrewducker]

I am old enough to remember an internet without spam. I was online a good two years before the Green Card spam first hit the newsgroups in 1994.

This means that I remember when you didn't get any email, except from people who had something to say to you. They might be complete wierdos wanting to pick an arguemtn, , but they were an individual, sending you something personal. This was before companies really discovered the internet, so I wasn't even getting emails from Amazon, as they didn't come into existence until 1995.

Anyway, nowadays a vast amount of my email is spam. I don't both to keep track of it, and thankfully I'm sitting behind a series of filters - spamassassin throws away anything it scores over a 10. Anything scored 5-10 gets passed on to me with ****SPAM**** in the subject line. And then Thunderbird's adaptive filter has a go at what's left.

This kills off 95%, leaving about 20 a day to delete by hand. The vast majority of which are part of a fascinating development in spam - one which has stymied the majority of spamfilters - putting the text into an image.

Embedded images along with random text make it almost impossible for the content to be scanned. Some spam filters have started putting OCR in,to catch this - but then the spammers are altering the text enough to make it extremely hard to OCR it. It's entirely possible that this escalating warfare between spammers and spamfilterers will end up finding new wrinkles in OCR technology and visual perception, in their attempts to produce text that can be read only by people/produce code that can read said text.

Eventually I expect spam to consist of magic-eye pictures, where you haveto stare really hard at swirls of pseudo-random text until the name of the stock they want you to buy appears. Only three percent of the population will be able to actually see these messages, but these will also be the three percent of people who both have loads of money, and are entirely gullible about the stock market.

Comments

[drdoug.livejournal.com]

Blimey, yes, I remember those days. I did get a lot more chain emails back then, though - up to three or four a day IIRC - mostly from friends but sometimes from more distant acquaintances. I get maybe two a month tops these days. Either my friends and family are smarter than they used to be or my response to them works ...

[shannon_a]

Yup, I too remember the Internet without spam, though in ye truly olden days I used to get the equivalent: talk spam, which consisted of guys from all over the world randomly sending me talk messages that were essentially wanna-fucks because of my name.

I was just talking with some friends about spam the other week, and my general assessment is that email is on the verge of uselessness because of spamming. The amount of spam keeps going up by astronomical amounts. A year or two everyone gasped when it hit 50% of all email, and now I hear numbers like 80%. We have a hardware spam filter at our site, and it can barely keep up with all of the email that hits it, most of which is spam. I recently had to shut down my original site (erzo) because the spammers had killed its usefulness as an email hub.

Either someone is going to come up with a technological magic bullet (unlikely), politicians are finally going to get off their asses and do something in the countries that produce the most spam which includes the US (unlikely), or email is going to become absolutely useless by the end of the decade (likely)--and maybe that will spur someone to action, too little too late.

[andrewducker]

Sadly, the technological things that would help (email signing, SPF) mostly require technical knowledge than the average user can cope with (although I might be nicely surprised).

I still find email useful - but mostly because I filter it in various ways, and then pick over what's left.

Thunderbird, thankfully, has a "person is in my address book" filter, which I find terribly useful for whitelisting people.

[makyo.livejournal.com]

I've got my spam filters trained pretty well now, so that the only spam I actually see consists of absolute gibberish and/or random collections of words. Having said that, the university IT Services department are just about to force us all to use some clunky Exchange system (which, we're assured, will be stabler and unclunkier than the existing Groupwise system which I opted not to use when they originally brought it in). One of the side-effects of this is the breakage of one layer of my spam-filtering regime - Exchange won't interface with procmail, apparently.

I'm kind of hoping that eventually we'll get to the point where the only way spammers can slip stuff past the filters is by removing any coherent information from the messages they send, and will shortly afterwards give up.

[nameandnature]

I've never thought content filtering (spamassassin, Bayesian filters and so on) would do the job: it's an arms race where the spammers can always find something which is legible to humans but not to computers. There's a diminishing return for the spammers, I guess, since they mostly still need to get you to visit a website and most of their buyers probably need something to click on rather than type into the address bar of their browser, but if URLBody style filtering becomes popular enough, they'll switch to hosting their sites on compromised PCs which aren't yet blacklisted anywhere. And these latest pump'n'dump image spams don't need a link to click, of course.

I'm waiting on Spamhaus Zen incorporating their PBL. Then their users (including me) are going to stop accepting mail from the huge numbers of compromised Windows boxes that the spammers are currently using, without having to wait for the XBL to catch up. The image spammers have compromised so many machines that it's easy for them to pick those which aren't XBL listed and use them to get their spam through. That's the cause of the majority of spam filter escapes I'm seeing.

It's a shame that the PBL will just include vast swathes of consumer ISP space. I won't be able to run my own mail server off my ADSL line, but it's a small price to pay. There are already "dynamic IP" BLs out there, but I don't trust them as much as Spamhaus, so I'm not using them. I bet a lot of Spamhaus's big users are the same. If someone can do this properly, I think it'll be very effective.

I imagine the next stage in the arms race after that will be spammers using the outbound servers of the ISP whose network those compromised PCs are on. That and looking for networks the PBL isn't covering, or trying to take down Spamhaus itself.

[andrewducker]

The PBL is an interesting idea. I do think that ordinary users should be presumed to _not_ be using SMTP directly. They should be able to register to do so, of course.

Everyone using SPF would have a somewhat similar effect. Tricky to do though.

[nameandnature]

Everyone using SPF includes the spammers using it to say that anyone may send mail for their domains, of course :-)

The other problem is that it involves everyone doing something, as you've pointed out. That's one of Vernon Schryver's FUSSP points.

[andrewducker]

SPF doesn't _solve_ spam - but it can add to the weightings when working out whether a particular is spam. If a domain has SPF records, and the mail is valid according to them, and the domain is not on any blacklists, then that indicates a lack of spamminess. On the other hand, if a domain has SPF records, and the mail is not valid according to them, or has been blacklisted, then that indicates that it is spam. This can be done while ignoring SPF for all sites that don't have SPF.

If SPF ever gets massive traction then you can start assuming that sites without SPF are spammers - but that's a way off.

[vereybowring.livejournal.com]

No spam !
Yes, those were the days.

Luckily, my hotmail filter is all trained up and my private mail server has a reasonable filter and I use thunderbird to filter the rest. Poor little me only gets maybe one spam a month through all the filters - but a year ago I didn't get any.

I find just being careful what you look at and click on vastly dictates what spam gets to you. I'm not signed up to masses of news groups and the like which I believe contributes greatly to my lack of spam.

At the end of the day though, messenger software can leave messages for offline contacts these days and that might have to take over a bit - at least you can refuse stuff from users you don't know or choose who to allow stuff from.

Spammers are getting trickier - but at least they end up spamming each other which is a comfort at least. Ideally, they realise what an arse up they make of the internet and stop - but since many are paid for this activity we will never see the back of them until some get locked up or fined so heavily it will only leave the hardcore spammers to track down.

These days we would probably find it quite eerie if we didn't get any spam at all. Most of us might think our accounts had failed for some reason (one reason I noticed a glitch in one of mine recently).

Never mind folks, life wouldn't be life if we had nothing to gripe about. Living with spam is like living with reality TV. Mostly crap, but sometimes makes you laugh.

[aberbotimue.livejournal.com]

You've heard how I deal with it havn't you.. create new addresses for each site I sign up for. since its only one or two that sell them off, then you firstly know who sold your address, and secondly, you can turn it off.

Where most customers that don't use the sytem winge about 1000's of messages a week, the few that do use it at the moment, winge about 1 or two a week.

[andrewducker]

I've had my email address for over 6 years now. Changing to a new one would be a huge pain in the neck, and if I want to get rid of spam I'd have to do that.