Spam Assassin rules updated

robmueller · 5th February 2005, 01:04 PM

Since we upgraded to SA3, some of the spam blocking has been a bit inconsistent. We're working on getting per user bayes db's setup, but we're doing a bit of work to ensure that there won't be any performance issues for the system as a whole, it's quite DB intensive.

Until then, we've just added in a bunch of new SA rules that should hopefully help improve spam detection. These are well used rules from the spam assassin rules emporium.

Feedback from people on whether this helps would be appreciated.

Rob

XB77 · 6th February 2005, 03:19 AM

1) What is the date of the SA3 implementation?
2) What is the date of the tweak (additional rules) to SA3 that you mentioned?
Reason for questions is don't want to send anything in that pre-dates above dates.
3) You mention feedback about how well SA3 w/ new rules is working. Where/how do you want that feedback (see recent related posts essentially asking same)?

robmueller · 6th February 2005, 06:18 AM

We're using SA 3.0.1, though there haven't been many changes between 3.0.1 and 3.0.2, the latest version

The tweaking was done just before I posted the above to the forum, so check the date on the post in your timezone.

Feedback here is fine, just want to get an idea of if it's helping or not to improve spam detection rates. I'm not looking for specific emails, just general feeling.

Rob

PJS · 6th February 2005, 07:28 AM

Will any SPF checks be enabled? Mails could at least be tagged with zero rated scores then I could take my own action with sieve.

kander · 7th February 2005, 03:47 PM

Rob,

I am definitely noticing the changes. A bunch of those 'Quality software for low prices' spams managed to get through all of the sudden, while previously they would be blocked just fine.

--K

Starion · 7th February 2005, 06:09 PM

Does spam assasin filter mail for guest accounts? I know that guests have basic spam filtering, but I don't know if spam assasin is part of the basic spam filtering system.

I haven't recieved any spam since I signed up for my Fastmail.fm account.

Later

**Sherry** · 8th February 2005, 08:48 AM

Quote:

Originally posted by Starion
Does spam assasin filter mail for guest accounts? I know that guests have basic spam filtering, but I don't know if spam assasin is part of the basic spam filtering system.

Perhaps if you read the following it will explain it for you.

(Guest and Member accounts are in the "Basic" part)

Spam Protection

Sherry

robmueller · 8th February 2005, 09:10 AM

Ah, that sounds bad, it's supposed to be the other way round! I haven't actually removed any rules, I've only added some, so if anything is getting through, it's because they're somehow dodging older rules and the newer rules.

*sigh*

Better keep working on getting per-user bayes in place...

Rob

Quote:

Originally posted by kander
Rob,

I am definitely noticing the changes. A bunch of those 'Quality software for low prices' spams managed to get through all of the sudden, while previously they would be blocked just fine.

--K

Starion · 8th February 2005, 09:37 AM

Quote:

Originally posted by Sherry
Perhaps if you read the following it will explain it for you.
(Guest and Member accounts are in the "Basic" part)

Sherry

Thanks for the link! I'll go read the page now. Sorry for the interruption.

david chaney · 8th February 2005, 11:01 PM

Quote:

Originally posted by robmueller
Since we upgraded to SA3, some of the spam blocking has been a bit inconsistent. We're working on getting per user bayes db's setup, but we're doing a bit of work to ensure that there won't be any performance issues for the system as a whole, it's quite DB intensive.

Until then, we've just added in a bunch of new SA rules that should hopefully help improve spam detection. These are well used rules from the spam assassin rules emporium.

Feedback from people on whether this helps would be appreciated.

Rob

This may explain the rash of false positives I've had the last couple of days.

Do you all want feedback on those messages?

David

XB77 · 9th February 2005, 01:20 AM

In the absence of individual DBs or the harvesting of JunkMail folders (was that killed due to privacy concerns?), it would be nice if there were one or two addresses where carefully selected spam filter failures could be forwarded (with full headers) so that, as time and resources permitted, whatever the current implementation of SA could be tweaked. I would suggest one forwarding address named falsepositives@whatever.tld and one named falsenegatives. This shouldn't have any relation to the spam not-spam "buttons" in FM. Rather it could just be something known by some of the more sophisticated members here or given out to FM clientele by FM support at their discretion.

Misha · 30th August 2005, 02:26 PM

Quote:

Since we upgraded to SA3, some of the spam blocking has been a bit inconsistent. We're working on getting per user bayes db's setup, but we're doing a bit of work to ensure that there won't be any performance issues for the system as a whole, it's quite DB intensive.

Has there been any news/progress on this?

I love everything about fast-mail, except the lack of user-specific bayes filtering. I use an email address that's been active for a long time, and so gets several hundred spam messages a day. In my own experience, user-specific bayesian filters help tremendously, and I'd love to see them on Fastmail.

robmueller · 31st August 2005, 07:20 PM

Most of the code is in place, it just hasn't been tested and we haven't checked the performance implications on our database. Like most things, the final 10% takes almost as long to tune and get right as the original 90%. I think once Bron has the new file storage stuff all done, this might be a good project to work on and get done...

Rob

Misha · 1st October 2005, 01:42 PM

Quote:

Originally posted by robmueller
Most of the code is in place, it just hasn't been tested and we haven't checked the performance implications on our database. Like most things, the final 10% takes almost as long to tune and get right as the original 90%. I think once Bron has the new file storage stuff all done, this might be a good project to work on and get done...

Rob

How far into that final 10% are things? It'd be so great to see this.

I had a per-user bayes spamassassin setup before I came to FastMail, and it was really really good - tremendously accurate.

Any progress on this? I really enjoy the Fastmail service, and would love to see this added...

robmueller · 3rd October 2005, 01:51 PM

Haven't done anything recently, it's all been file storage stuff so far. Will definitely announce it when done

Rob

5th February 2005, 01:04 PM	#1
robmueller Intergalactic Postmaster Join Date: Oct 2001 Location: Melbourne, Australia Posts: 6,098 Representative of: Fastmail.FM	Spam Assassin rules updated Since we upgraded to SA3, some of the spam blocking has been a bit inconsistent. We're working on getting per user bayes db's setup, but we're doing a bit of work to ensure that there won't be any performance issues for the system as a whole, it's quite DB intensive. Until then, we've just added in a bunch of new SA rules that should hopefully help improve spam detection. These are well used rules from the spam assassin rules emporium. Feedback from people on whether this helps would be appreciated. Rob

9th February 2005, 01:20 AM	#11
XB77 Essential Contributor Join Date: Dec 2002 Location: Los Angeles Posts: 365	We need someplace to send example filter failures. In the absence of individual DBs or the harvesting of JunkMail folders (was that killed due to privacy concerns?), it would be nice if there were one or two addresses where carefully selected spam filter failures could be forwarded (with full headers) so that, as time and resources permitted, whatever the current implementation of SA could be tweaked. I would suggest one forwarding address named falsepositives@whatever.tld and one named falsenegatives. This shouldn't have any relation to the spam not-spam "buttons" in FM. Rather it could just be something known by some of the more sophisticated members here or given out to FM clientele by FM support at their discretion.

6th February 2005, 03:19 AM	#2
XB77 Essential Contributor Join Date: Dec 2002 Location: Los Angeles Posts: 365	1) What is the date of the SA3 implementation? 2) What is the date of the tweak (additional rules) to SA3 that you mentioned? Reason for questions is don't want to send anything in that pre-dates above dates. 3) You mention feedback about how well SA3 w/ new rules is working. Where/how do you want that feedback (see recent related posts essentially asking same)?

6th February 2005, 06:18 AM	#3
robmueller Intergalactic Postmaster Join Date: Oct 2001 Location: Melbourne, Australia Posts: 6,098 Representative of: Fastmail.FM	We're using SA 3.0.1, though there haven't been many changes between 3.0.1 and 3.0.2, the latest version The tweaking was done just before I posted the above to the forum, so check the date on the post in your timezone. Feedback here is fine, just want to get an idea of if it's helping or not to improve spam detection rates. I'm not looking for specific emails, just general feeling. Rob

6th February 2005, 07:28 AM	#4
PJS Member Join Date: Jul 2003 Location: London Posts: 71	Will any SPF checks be enabled? Mails could at least be tagged with zero rated scores then I could take my own action with sieve.

7th February 2005, 03:47 PM	#5
kander The "e" in e-mail Join Date: Jan 2002 Location: The Netherlands Posts: 4,110	Rob, I am definitely noticing the changes. A bunch of those 'Quality software for low prices' spams managed to get through all of the sudden, while previously they would be blocked just fine. --K

7th February 2005, 06:09 PM	#6
Starion Essential Contributor Join Date: Aug 2004 Location: Washington D.C. Posts: 238	Does spam assasin filter mail for guest accounts? I know that guests have basic spam filtering, but I don't know if spam assasin is part of the basic spam filtering system. I haven't recieved any spam since I signed up for my Fastmail.fm account. Later

31st August 2005, 07:20 PM	#13
robmueller Intergalactic Postmaster Join Date: Oct 2001 Location: Melbourne, Australia Posts: 6,098 Representative of: Fastmail.FM	Most of the code is in place, it just hasn't been tested and we haven't checked the performance implications on our database. Like most things, the final 10% takes almost as long to tune and get right as the original 90%. I think once Bron has the new file storage stuff all done, this might be a good project to work on and get done... Rob

3rd October 2005, 01:51 PM	#15
robmueller Intergalactic Postmaster Join Date: Oct 2001 Location: Melbourne, Australia Posts: 6,098 Representative of: Fastmail.FM	Haven't done anything recently, it's all been file storage stuff so far. Will definitely announce it when done Rob