Spam

[JamieKitson]

Hi,

I've been a Fastmail user since August, and generally the spam filtering has been good, especially since I turned on spam learning. However over the last few weeks I've been getting more and more spam through to my inbox, four yesterday and two today already (it's 10:45am here in the UK), with only two filtered spam emails both days. Is there anything I can do about this? The only thing I wonder is if having Fastmail discard obvious spam messages could have harmed the spam learning?

Cheers, Jamie

[paleolith]

I think something changed. See the end of the thread "After very little... lots of spam - How about you? system ok?".

Edward

[JamieKitson]

Just to update, my current spam situation is amazing, I'm only getting one a day in my spam box (I have highly rated spam deleted) and none in my inbox.

[paleolith]

Unfortunately mine's gone in the other direction recently. There's a pattern of spam with lots of hyphens in the subject line and not hitting the block lists. Haven't had time to investigate.

Edward

[wombat88]

Until a few weeks ago, my Fastmail account has been blissfully free of unwanted spam. Now I seem to be getting tons of it daily, all from .eu domains. I've been a Fastmail user for well over ten years, so this is unusual compared to what I've seen previously.

While most of this spam is being marked as spam, about 10% is not. I've set up an additional filter to ensure these go into my spam training folder rather than my inbox, but the volume is becoming annoying (10-20 or more per day). I did discover that up until now I didn't have enough reported spam messages to trigger the Bayes filter, but previously it hadn't been a concern as I so rarely got spam through Fastmail. That is being fixed quickly with the volume of spam I'm getting.

Two questions:

1. I mark items as spam when the Bayes score is 7 or higher. Is that too low? Previously, I'd had it set to 5 or 6, but valid email was being marked as spam, so I moved it higher.

2. I've been logging in to Fastmail once or twice daily and reporting the spam in the training folder. However, once I've reported the spam, I've been deleting it. I've got the spam training folder set to purge every 7 days, but once I've manually reported the emails, is it okay to delete them or should I leave it in the spam folder in the web interface? I know the IMAP spam training folder is scanned daily so I don't really need to manually report it, but I also don't want the spam taking up my storage space.

Is anyone else having similar problems lately?

Wendy

PS: I am careful about emails. I don't click on links, don't "unsubscribe" obvious spam email, never give out my "real" email address, and use dot-addressing where I can. These emails are being sent to one of my aliases, but it is unfortunately the primary one I use for personal email, so it isn't an alias I'd like to throw away. I use IMAP with Thunderbird as my email client.

[n5bb]

Hi, Wendy.

Originally Posted by wombat88

Until a few weeks ago, my Fastmail account has been blissfully free of unwanted spam. Now I seem to be getting tons of it daily, all from .eu domains. I've been a Fastmail user for well over ten years, so this is unusual compared to what I've seen previously.

If spammers send you hundreds of spam messages a day from apparently good servers and domains they purchased a few days ago, you will get quite a few delivered to your account. The messages can appear very similar to non-spam messages, especially to non-human analysis techniques. Why were you targeted? Only the spammer knows. See:
Where Did That Scammer Get Your Email Address?

Originally Posted by wombat88

1. I mark items as spam when the Bayes score is 7 or higher. Is that too low? Previously, I'd had it set to 5 or 6, but valid email was being marked as spam, so I moved it higher.

Some details in your comment concern me:

• You describe the "Bayes score", but I think you must really mean the X-Spam-score header value, which is optionally shown in the message subject inside curly brackets (braces), such as {SPAM 6.5}.
  • The X-Spam-hits header contains the details on how the spam score was assigned, and some of these spam tags are BAYES_99 and BAYES_999.
  • All of the BAYES_ tags are created by comparing the message with others which have been reported as spam or ham (non-spam), and then creating a likelihood that the message is spam based on these earlier reports using Bayesian statistics.
  • At this time, a very spammy message receiving both 99% and 99.9% spam tags with get a spam score of 3.7 (3.5 for BAYES_99 and 0.2 for BAYES_999). So the Bayes test alone can't cause the very high false positive scores you report (over 6), and other spam indicators must be active.
  • So first be absolutely sure that the messages you think are ham (non-spam) receiving high spam scores are good messages. Some scammers are very good at simulating messages from banks and other organizations. I suggest examining the full headers of those apparently non-spam messages with high spam scores, especially the X-Spam-source header (which shows the source IP, sending server, and other information about the sending email system). See which X-Spam-hits tags are getting activated and creating those high scores.
  • You can whitelist (always treat as non-spam) messages from known good senders by placing those addresses in your online Fastmail address book. It's easiest to do this online, since there isn't an easy direct way to do this in your IMAP email client. When reading such a message in the current (not classic) Fastmail web interface, click the sender's name, then in the pop-up click Add to Contacts. In the Advanced>Spam/Virus Protection screen, be sure that in the custom spam protection section (which it appears you are using) the Address Book selection is "Always accept" or "Weigh by subtracting" a significant score (such as 4.0 or 5.0). Once you do this, the only false positives (non-spam placed in the Spam folder) should be for new contacts or addresses you forgot to add to your online Address Book.
• If you use Address Book whitelisting and the user Bayes filter, you can set the spam threshold much lower than 5 and still not get any false positives. I currently have my Probable Spam threshold set to >= 1.0 and Definite Spam set to >= 9.5 and it works great, but I only recommend these settings if you use whitelisting and user Bayes filter.

Originally Posted by wombat88

...2. I've been logging in to Fastmail once or twice daily and reporting the spam in the training folder. However, once I've reported the spam, I've been deleting it. I've got the spam training folder set to purge every 7 days, but once I've manually reported the emails, is it okay to delete them or should I leave it in the spam folder in the web interface? I know the IMAP spam training folder is scanned daily so I don't really need to manually report it, but I also don't want the spam taking up my storage space.

You need to leave the spam (or non-spam) in the appropriate folder at least 24 hours for it to be scanned. During the daily scan (which is at a different time for each user), your account is automatically scanned to discover which folders contain contain new messages (which weren't in that specific folder during the previous scan). After the scan completes, you can permanently delete the message, since it will only be used once to train the user Bayes spam filter as long as it remains in the folder. You can see the current account scan time by examining the Spam Learning status for the Junk Mail folder in the Advanced>Folders screen. You will see the date and time (in your timezone) when that folder was last scanned. As described earlier, it will only be scanned if a new message was placed in that folder since the previous scan. The new message can be due to a newly arrived email automatically filed into the folder, or an action in the online web interface or IMAP client to move a message between folders.

If you have Inbox set to learn as non-spam (default) and Junk Mail (Spam in the web interface) set to learn as spam, then you can move messages at any time between those folders in your IMAP client to correct mistakes. So if a spam message appears in your Inbox and you don't notice it until after your account was scanned, just mark the message as spam in the web interface or move it to the Junk Mail folder using an IMAP client and the previous scan will be forgotten and the new scan spam or non-spam results will be applied to your user Bayes database.

So you can see that this all works automatically in an easy to use manner. You just need to keep messages in folders at least one day for the daily scan to apply them to the user Bayes filter.

Bill

[wombat88]

Thanks for the detailed reply, Bill.

Unfortunately I've got a few friends and relatives who aren't that computer-savvy so I'm sure their contacts lists have been harvested by someone, or else they didn't bcc when they forwarded something they thought I'd just love to see (not!). Given the volume of emails I'm getting, I'd guess I'm on a list being sold somewhere. "Lucky" me.

Yes, I meant X-Spam Score. I do use advanced filtering, and have nearly 700 emails classified as non-spam for training purposes, but I've not done that type of training for long time so I'll set up a "ham" folder to do more. I am rapidly reaching the "spam" training threshold of 200 though so hopefully it can do a better job of filtering out the junk. As for knowing genuine from non-genuine email, I always check the source if I'm not sure but generally most spam is pretty darn obvious! It always surprises me how many people are fooled by those things. Usually there are dead giveaways, such the content, grammar, or links that don't point to where they say they do.

I do use address book whitelisting, but I've only got a handful of email addresses in Fastmail because I always use IMAP with an email client. Most often, the "ham" emails are from professional organizations that occasionally send out newsletters via a list. I'll do what you suggested and set the x-spam score lower, move "ham" messages into another training folder to help with the false positives, and also add senders to my contacts list so the whitelisting can work. So far, the last filter I set up on the problem domain is working as the messages are not getting into my inbox at least.

Thanks again for your help. If all else fails, I can always dump the alias, but hopefully I'll be able to avoid that.

Wendy

[lane]

Originally Posted by wombat88

I do use address book whitelisting, but I've only got a handful of email addresses in Fastmail because I always use IMAP with an email client.

Most email clients will allow you to export your address book (you need only names and email addresses), and then you could import that into Fastmail. A quick way to get your current addresses up to the web.

[lane]

Originally Posted by wombat88

If all else fails, I can always dump the alias, but hopefully I'll be able to avoid that.

I also have a problem with one particular alias that receives about 98%+ of my spam. What I do is set a much lower reject point (in the spam score) for that alias than for anything else. It works like a charm. I have been gradually removing correspondents from that alias, and whitelisting the remaining ones.

[mballas]

Originally Posted by lane

I also have a problem with one particular alias that receives about 98%+ of my spam. What I do is set a much lower reject point (in the spam score) for that alias than for anything else. It works like a charm. I have been gradually removing correspondents from that alias, and whitelisting the remaining ones.

How do you set that score for one alias and not for others?

[lane]

Originally Posted by mballas

How do you set that score for one alias and not for others?

Go to Advanced Rules: (1) in the new interface, click the down arrow by "Mail", and choose Advanced, then click Rules; (2) in the classic interface, click on Settings, then Define Rules. At that point the interfaces are almost identical.

Choose the Discard tab. Create a new rule:

1. Message with: Advanced
2. That: just skip, it will change later to N/A
3. The Text: allof(header :contains "X-Delivered-to" "problemalias@mydomain.com", header :value "ge" :comparator "i;ascii-numeric" ["X-Spam-score"] ["6"], not header :contains ["X-Spam-known-sender"] "yes" )

In conjunction with this, I currently move to the spam folder and change the subject to reflect the spam score, whenever the spam score is 5 or more. I don't discard any other spam than the problem alias at the moment, but during a spam-storm I am happy to discard above about 8 or 10.

Save your changes.

[mballas]

Originally Posted by lane

[The Text: allof(header :contains "X-Delivered-to" "problemalias@mydomain.com", header :value "ge" :comparator "i;ascii-numeric" ["X-Spam-score"] ["6"], not header :contains ["X-Spam-known-sender"] "yes" )[/list]In conjunction with this, I currently move to the spam folder and change the subject to reflect the spam score, whenever the spam score is 5 or more. I don't discard any other spam than the problem alias at the moment, but during a spam-storm I am happy to discard above about 8 or 10.

Save your changes.

Thank you. I didn't know that was possible.

[randian]

Originally Posted by n5bb

After the scan completes, you can permanently delete the message, since it will only be used once to train the user Bayes spam filter as long as it remains in the folder.

So I can set the SPAM folder to Auto-Purge and the received SPAM will still get added to the Bayes database?

[n5bb]

Originally Posted by randian

So I can set the SPAM folder to Auto-Purge and the received SPAM will still get added to the Bayes database?

Yes, auto-purging works fine. As long as the message is in the Spam folder at least 24 hours it will be scanned and added to the user Bayes database.

Bill

[Bahb]

Spam HAS increased since the weekend. Periodically this happens, usually following a weekend, which is when changes to FM always happen too. After a few days, everything calms down and I'm back to getting almost no spam. Maybe coincidence, but I've been around long enough that I expect the unexpected here, immediately following a weekend.