What does SPAM button do?

[fiepx250]

Like others replying to this thread, I have a custom spam handling system with which FastMail's "Report spam" and "Report not spam" don't work well at all. I'd love to improve FastMail's spam filtering by identifying misfiled spam or ham, but can't with the system as it works now.

Specifically, the mailbox in which I keep spam until I can review it is not called "Junk Mail", but using FastMail's "Report spam" ignores that mailbox, and instead causes a new "Junk Mail" mailbox to be created, and the reported spam messages to be put there. I then have to move the messages to my real spam mailbox and delete the unnecessarily created mailbox.

Could FastMail not simply use the mailbox selected in the "Probable Spam" item of the Custom settings of the Spam/Virus Protection options to move reported spam to?

On a somewhat unrelated note: I moved to fastmail.fm partly because the spam filtering tool I was using before (Mac OS X's Mail.app) had too much assumptions built into its spam handling implementation to be useful, and I would love it if FastMail's spam handling stayed away from assumptions about user behavior and instead provided simple commands that did simple things. For example, I would much rather rely on simple "Report spam" and "Report non-spam" to train your filters, rather than having various actions I might take on my message have some magic (and possibly tragic :-) filter training implications.

Thanks for listening.

[davidbstanley]

Welcome to the forum fiepx250 and than you for your contribution. I think it is clear that the various options for the spam handling need to be configurable by each user.

I use Outlook for my email. Outlook places spam in a folder called 'Junk E-mail', which cannot be changed on the IMAP server.To work well for people like me, there needs to be a way to report messages in this folder as spam to the FM system. The only way to do this would be to automatically scan the 'Junk E-mail' folder periodically, which is not ideal or report everything when the folder is emptied from the mail client if that is possible. Firstly, we need to be able to either change the name of the spam folder on the FM system or mark more than one folder as a spam folder for the system to process.

[XB77]

Note: This query was originally posted to the Help/Problems section, but there were no responses there.

I recently received a mail item in my Inbox that, when opened, had the Not Spam button above (not the Spam button, as I would have expected for items in the Inbox). Aren't items in the Inbox (or other non-Junk Mail folders) supposed to have a Spam button and items in the Junk Mail folder supposed to have a Not Spam button? FWIW, the item was actually spam and I only noticed that it didn't have the correct button when I went to click the Spam button (that wasn't there). I moved it to the Junk Mail folder (where it retained its Not Spam button, as appropriate in that folder), so that it would eventually be picked up by FM accumulation of spam examples, as per the discussion above.

However. in the meantime, am I missing something in how this is supposed to be setup, or is this a bug? Should I forward the e-mail item in question to someone in ops? (This was not on the Beta server.)

[fiepx250]

Hi davidbstanley,

You wrote:

Originally posted by davidbstanley
I use Outlook for my email. Outlook places spam in a folder called 'Junk E-mail', which cannot be changed on the IMAP server.To work well for people like me, there needs to be a way to report messages in this folder as spam to the FM system. The only way to do this would be to automatically scan the 'Junk E-mail' folder periodically, which is not ideal or report everything when the folder is emptied from the mail client if that is possible.

Wouldn't FM be able to know which messages your email client marks as spam, or which ones you manually mark as spam or non-spam (ham), at the time the message is moved to or from the spam mailbox? This moving must include a call to FM's IMAP server, which could flag/unflag the messages as they're moved. For flagging purposes, FM could allow you to indicate which of you mailboxes contain only spam, which ones contain only ham, and which ones may contain a bit of both (moving messages to these last mailboxes wouldn't change their spam/ham status). This could be done by adding a pop-up column in the Folders page ("this mailbox contains: no spam/only spam/some spam"). The default values of this setting could be "only spam" for any mailbox whose name started with "junk" or "spam" and "no spam" for any other mailbox, except for the mailbox set to receive "Probable Spam" in the Advanced Spam settings, which would have that setting hard set to "only spam". Help about that Folders setting could simply state it's used to assist FM in training its spam filters for your account.

This is assuming marking messages as spam at the time of their addition to spam mailboxes is acceptable (that is, assuming any harm done by having a ham messages marked as spam will be sufficiently undone by later marking that message as ham, and vice versa). If that's not a good assumption, a delay (for example one week) could be introduced between the time when a message is first received and when its spam/ham status is collected to train the spam filters, or a shorter delay (for example one hour) between when it is first read or marked as read by a human (assuming FM's IMAP servers can determine that) and when its spam/ham status is used. This would allow for manual client-side correction of spam/ham status without requiring all users to delete all their spam (some users may need to keep some of their spam for various useful purposes).

In any case, I'd like to see a "disable spam training" check box in the Advanced Spam Protection Settings for those for whom assumptions like those above don't work, with those users being left to rely on the training provided by other users.

[PhiloVance]

I didn't even know the spam button was there, see this post.

It was right next to the delete button and I hit spam instead. I was told to mark the same message as not-spam (to correct the error), how do I do that. I've since moved it from my junk mail back to it's original folder...afraid to delete it now.

FTR I have my own spam filtering with Mailwasher which operates independently of Fastmail. I thought between the two we had this spam thing licked.

I don need no stinkin SPAM button.

[mlevin]

Originally posted by Jeremy Howard
Kind of - there's actually two kinds of reports "explicit" and "implicit". The 'empty junk' creates an "implicit report". We will deal this these differently, and probably add an option to allow you to not train from implicit reports at all.
No. The system is not currently doing any learning from these reports - they are not yet being used at all, except for our own internal statistics, so we can get some idea of how we might be able to use them in the future. Before we use this information, we will provide appropriate docs and options, and make an announcement. We will clear the reports at that time and start afresh.
No.

Any further developments on these items above? Any further thoughts on Bayesian filters?

I'm referring specifically to posts http://www.emaildiscussions.com/...765#post257765 and http://www.emaildiscussions.com/...06#post259406.

Thanks

[hadaso]

A few days ago I saw something that made me think that a "this is spam" button letting all users define spam might not be a good idea (depending on what the button does. Just for personal bayesean filters it's fine).

My wife has signed up for several newletters berfore our trip to the US on february. Now we're back, and I saw that she was still getting those newletters, so I mentioned that she might want to unsubscribe from these now that she doesn't need them. Her immediate response was that I'm right, and her immediate action was to click the "this is spam" button in her Hotmail account for that newsletter (that she subscribed to!).

So my conclusion is that "ordinary people", i.e., people that just use email and don't bother learning about it, might view the "this is spam" button as a tool to get rid of mail theydon't want, regardless of its solicited/unsolicited status. If the input is then used to create spam reports, or even if it is just used to for a shared ham/spam datrabase for Bayesean filtering, it would be bad!

[elvey]

I agree, somwhat. If there's a dumber user base than AOL, I haven't heard of it. They manage to get it to work OK, but not ideally.

But the FM version of the button by default shouldn't do much; it should be customizable via the options screen, as detailed on the wiki...

[mlevin]

So can I look forward to a day when I have my own private bayesian spam/ham database on fastmail?

I've been getting a lot of false positives lately and it is a pain to weed them out of the junk folder. I'd really like to be able to train the thing to filter my mail more effectively.

[elvey]

From what I've heard, per-user Bayesian filtering is too resource-intensive to be affordable for fastmail. I doubt we'll see it.

BTW, one of my clients has an associate with an AOL account.
He has us forward all his email (to him@client.dom) to his AOL account.
This includes all the spam he gets at that address.
He reports it with AOL's 'Spam' button, and AOL thinks we're the spammers. AOL is unwilling to help with this problem. We've told the associate to stop doing this, which he did for several months, but he just did it again.

[spruce]

Two more possible folder buttons:

1) Pre-Sieve Reject: When I *KNOW* everything sent to a folder is spam: immediately reject before accepting the message body. Possible reason options: "I am an unknown user" or "you are a known spammer" (the former could be the default in case AOLuser uses this to cancel mailing list subscriptions)

2) Extreme Pre-Sieve Reject: (only applicable to MX domain users or some kind of group spam opt-in filter) Not only do everything for the former button, but also immediately and automatically add the current SMTP sender to my local SMTP server deny list for my entire domain.

oh yes, and immediate deletion of the message is assumed since it is never even accepted.

Those two buttons would reduce bandwidth and may foil some dictionary attacks on virtual domains. Any thoughts?

[hadaso]

I don't know what you mean by the word "button" here. I don't think it is a good idea to automatically add SMTP servers to a "deny list". But I do wish there was a way to reject messages by matching the recipient address to a pattern (for subdomain addresses). If this can be done while allowing the user a custom string to be included in the SMTP reject message it would be even better (there are several uses I can think of for this capability for using the mail protocol without letting spam in).

Another thing that can be cool is being able to apply greylisting selectively to addresses that match a pattern (or to selected aliases).

The setup I would really want to see is to be able to have for each alias a list of patterns, and for each pattern a list of actions to do on matching addresses: accept, reject (with custom message), delay (greylist). Greylisting is more difficult to achieve beause it means sender info should be saved so the message is eventually accepted. There are more complicated actions I can think of (such as doing countdowns like spamgourmet, only unlike spamgourmet do real SMTP rejects when the countdown is complete) but these are much more difficult to implement. Just accept/reject with custom message would be enough to let users build their own schemes on (the reject can include instructions on what to do so a message is accepted, that a human can follow but a bot cannot).

[mikev99]

I'd like to smtp reject based on domain. I.e., anything from China I do not want - spamcop mail had that feature. One could select several known spam domains for automatic smtp rejection. China, Korea, Brazil, etc. come to mind.

[spruce]

Originally posted by elvey
...
But the FM version of the button by default shouldn't do much; it should be customizable via the options screen, as detailed on the wiki...

OK, detailed on the wiki where?
If you'r referring to #IsSpam/Bounce at http://wiki.fastmail.fm/wiki/index.p..._for_two.21.29
it doesn't seem immediately clear in what options screen this would be customized in.

I am imagining a possible buttons (or fields or whatever) to configure a folder to do #True Reject as stated in the above wiki automatically for email directed at a folder (via plus addressing/subdomain addressing)

It may be confusing to actually put this in the Options/Folders screen as manually moving already received messages into these faux folders is questionable for the same reason that greylisting is questionable. That is, once the SMTP session is over, bouncing traffic can no longer go to the sending SMTP server if the header is forged, as is most often the case with spam, if I understand things correctly here. Maybe an Options/"Spam/Virus Protection" or Options/Define Rules field would be more appropriate.

The FAQ http://www.fastmail.fm/docs/faqparts....htm#MbxBounce
is confusing on terminology here and says

under "reject"
Since FastMail.FM pays for the bandwidth to receive the body of rejected email, users are charged for it. Little bandwidth is used to bounce messages, so users are not charged for that.

I believe users *ARE* charged for use of the bounce button, as they have indeed already received the spam in order to read it and manually determine that it is spam.

A folder should however be able to have all its messages deleted and then converted into a Pre-Sieve Reject or Extreme Pre-Sieve Reject folder so that you can change your mind about the trustworthyness of a contact after having used the folder for a little while.

[elvey]

spruce:
I changed the wiki in line with your sugggestion.

If you're charged bandwith for mail on the way in that you later bounce, that does NOT suggest that you're charged for it on the way out as well.