|
FastMail Forum All posts relating to FastMail.FM should go here: suggestions, comments, requests for help, complaints, technical issues etc. |
|
Thread Tools |
9 Sep 2019, 04:47 AM | #16 | |
Cornerstone of the Community
Join Date: Jan 2003
Location: Oxfordshire, UK
Posts: 603
|
Quote:
So I have tried this, but everything is now ending up in my inbox. I have traced the error to this line: HTML Code:
if header :matches "X-Spam-Known-Sender" ["yes*", "no*in-addressbook*"] HTML Code:
"no*in-addressbook*" Is there something wrong or out of date with "no*in-addressbook*"? cheers, James. |
|
9 Sep 2019, 06:11 AM | #17 |
Essential Contributor
Join Date: May 2018
Posts: 474
|
If you look in section 3 you will see FM's sieve's own code test' X-Spam-Known-Sender for stuff being in the the contacts list as:
Code:
if not header :matches "X-Spam-Known-Sender" "yes*" { My second test for "no*in-addressbook*" covers cases like the following: X-Spam-known-sender: no ("From == To and no DKIM or SPF for from domain, likely forged"); in-addressbook X-Spam-known-sender: no ("Email failed DMARC policy for domain"); in-addressbook because I want names in my contacts list to always get through to my inbox no matter what. Let's not get into a philosophical debate about this. It's just my preference. The additional match ("no*in-addressbook*") is for "no" followed by anything followed by "in-addressbook" as illustrated in those examples. Probably don't need the trailing star but it shouldn't make a difference. I wrote that code based only on my what I have seen. I only have my own email to use as source examples. If there's other situations where something in the contacts list can cause X-Spam-Known-Sender to say "no" and also not say "in-addressbook" for a contact that is in the contacts list I don't know about them. So in all your matches that went into the inbox do you have any where X-Spam-Known-Sender was "no" and didn't contain "in-addressbook"? Are you sure there isn't some pattern matching mistake in the address :matches "From" section? |
9 Sep 2019, 06:43 AM | #18 |
The "e" in e-mail
Join Date: May 2003
Location: mostly in Thailand
Posts: 3,090
|
I cannot see any particular error with your code, but I always try to simplify tests as far as possible. Tests with several wildcards make me nervous because they can cause tests to timeout. I would be inclined to write those tests as
Code:
if header :contains "X-Spam-Known-Sender" ["yes", "in-addressbook"] |
9 Sep 2019, 07:03 AM | #19 | |
Cornerstone of the Community
Join Date: Jan 2003
Location: Oxfordshire, UK
Posts: 603
|
Quote:
HTML Code:
X-Spam-known-sender: no ("From == To and no DKIM or SPF for from domain, likely forged"); in-addressbook thanks! |
|
9 Sep 2019, 12:56 PM | #20 |
Essential Contributor
Join Date: May 2018
Posts: 474
|
The setting of the known_sender to true and the way it's tested is mainly to get it past the FM generated spam test (but I also use it in other places as well). So once beyond there you're own your own to filter any way you want.
I also have an organize rule for emails sent to myself. I filter them not as spam but place them in a folder named "Unexpected". No one knows my actual FM email address. Only my forwarding service's address is known. So any email sent with an explicit To of my FM address I put in a folder of their own called "unexpected". The only legitimate time the To should be my actual FM address is when I am doing it for testing purposes or FM is sending me something directly. So part of the test is to let those emails reach my inbox (with one exception mentioned later). Below is the sieve code (albeit generated with an actual organize rule - sieve code "reads" so much better and is more concise than the organize rule UI page which defined them). Code:
if allof( address :is "To" "xxxxx@fastmail.com", not address :matches "From" "*@fastmail.com" ) { fileinto "INBOX.Unexpected@fastmail^com"; } But there is as case I do want to pick off and I can't do it with an organize rule (or if I can it's not worth the effort since it's easier in sieve). That's the case fastmail.com directly sends me email. Obviously they do know my actual FM email address too! I have it grouped in a special block of code following section 8 where do some other similar operations (alias address filtering but that's off topic). Without going into the actual context of what I do there since that complicates the description I include a test that basically looks like: Code:
if allof(address :is "From" "support@fastmail.com", header :matches "Subject" "*.* * Mail fetch retrieval error") { fileinto "${1}^${2}"; } The reason for this test is I have (had) a mail fetch account where about 30% of the time resulted in "Mail fetch retrieval error" being reported from fastmail.com. In the "old days" when mail fetch was using POP it was a timeout error and I could test for this and just ignore it since eventually a future fetch would succeed. Now with IMAP it's just a general "Mail fetch retrieval error" so I sort them into the email address 's folder. If I didn't they would end up in my inbox or if I didn't exclude fastmail.com from my "unexpected" test they would go into "unexpected". I want these error reports to go into the mail fetch identity's folder not elsewhere. I have 4 other mail fetch identities set up. This problem only happens with one of the (the yahoo servers). While the above test is a general filtering rule to handle these errors, I've since changed one offending email account to forward from there and disabled my mail fetch. Screw 'em! Last edited by xyzzy : 9 Sep 2019 at 01:09 PM. |
9 Sep 2019, 04:16 PM | #21 |
The "e" in e-mail
Join Date: Jul 2002
Location: VK4
Posts: 3,013
|
Would this work
elsif allof( not header :contains "X-Spam-Known-Sender" "in-addressbook", address :is "To" "your email address" ) { fileinto "INBOX"; } |
9 Sep 2019, 04:58 PM | #22 |
Essential Contributor
Join Date: May 2018
Posts: 474
|
It's valid code but what are you trying to accomplish and where is this code being placed? As a organize rule it would act only on non-spam.
On face value is says "send any message from anyone not in your contacts" to the inbox. The To, in most cases, will be always be you. Is this in reference to something I said or some of the code I included in one of the previous posts? I'm a bit confused about the context here. Last edited by xyzzy : 9 Sep 2019 at 05:30 PM. |
9 Sep 2019, 05:54 PM | #23 |
The "e" in e-mail
Join Date: Jul 2002
Location: VK4
Posts: 3,013
|
I just thought that it may work for the OP.
I would have thought the sent email would be put in his inbox instead of the spam folder. |
9 Sep 2019, 06:43 PM | #24 |
Essential Contributor
Join Date: May 2018
Posts: 474
|
Oh, all the way back there. Not sure why that email is being classed as spam if the sender is in the contacts list. Maybe the same kind of X-Spam-Known-Sender examples I show in post 17 so the spam bypass test at the start of section 3 fails and the stuff really has a high enough spam scores.
The code I showed is a more general case for overriding the spam check when the sender is in the contacts list. Senders will always get past the spam check no matter what. Your code isn't going to work since as a organize rule it's too late since the OP's message would still be picked off by the spam test. And if you place your code before the spam check almost all email that's not in the contacts will end up in the inbox. So to summarize the object of the "sport" was to only bypass the spam check if the sender is in the contacts list leaving everything else the same. That's all. Once past the spam check organize rules can do their thing to filter what they want and what's left will end up in the inbox by default. Last edited by xyzzy : 9 Sep 2019 at 07:08 PM. |
9 Sep 2019, 06:51 PM | #25 | |
Cornerstone of the Community
Join Date: Jan 2003
Location: Oxfordshire, UK
Posts: 603
|
Quote:
from: me to: me The spam rules caught it correctly, but your script was (of course) bypassing the spam rules because the sender (me) was in my contacts list. This will be a problem for all spam that uses your own contacts as their from: address (I guess because you also are also in their contacts list when their email was compromised). I get those from time to time. |
|
14 Apr 2020, 08:39 PM | #26 |
Cornerstone of the Community
Join Date: Jan 2003
Location: Oxfordshire, UK
Posts: 603
|
Why doesn't this work
So, I have been thinking about this some more - anything to stay busy during the lockdown
I wrote these lines above section 1: Code:
### 0. Whitelist test if address :matches "from" "name@domain.tld" { deleteheader "X-Spam-Known-Sender"; addheader "X-Spam-Known-Sender" "yes from sieve Whitelist"; } The code "worked" in that I got this header: Code:
X-Spam-Known-Sender: yes from sieve Whitelist Agh! Where has my logic failed? The spam section starts with Code:
if not header :matches "X-Spam-Known-Sender" "yes*" { Here are the headers in question: Code:
X-Spam-Known-Sender: yes from sieve Whitelist Return-Path: <name@domain.tld> Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by sloti5d1t04 (Cyrus 3.1.7-1130-gd0f8b30-fmstable-20200414v1) with LMTPA; Tue, 14 Apr 2020 06:29:16 -0400 X-Cyrus-Session-Id: sloti5d1t04-1586860156-318776-2-9389288406526221069 X-Sieve: CMU Sieve 3.0 X-Spam-sender-reputation: 0 (email) X-Spam-score: 5.8 X-Spam-hits: BAYES_50 0.8, DCC_CHECK 1.1, FREEMAIL_FROM 0.001, HTML_MESSAGE 0.001, ME_SENDERREP_DENY 4, RCVD_IN_DNSWL_NONE -0.0001, RCVD_IN_MSPIKE_H2 -0.001, SPF_HELO_PASS -0.001, SPF_PASS -0.001, LANGUAGES unknown, BAYES_USED user, SA_VERSION 3.4.2 X-Spam-source: IP='40.92.19.13', Host='mail-dm6nam11olkn2013.outbound.protection.domain.tld', Country='US', FromHeader='com', MailFrom='com' X-Spam-charsets: plain='iso-8859-1', html='iso-8859-1' X-Resolved-to: me@fastmail.com X-Delivered-to: me@mydomain.tld X-Mail-from: name@domain.tld |
15 Apr 2020, 12:35 AM | #27 |
Essential Contributor
Join Date: Jan 2017
Posts: 278
|
Reproducing the problem exactly was a bit of a hassle for me, but I tried the following at the top of my custom sieve
Code:
if header :contains "subject" "wibble" { deleteheader "X-Spam-Known-Sender"; addheader "X-Spam-Known-Sender" "yes from sieve Whitelist"; if header :matches "X-Spam-Known-Sender" "yes*"{ fileinto "INBOX.test"; } stop; } Are you sure the email isn't being sent to spam later? In a rule or by an external spam filter in a client. This would go some way to explaining the next bit. This looks like a problem: Code:
X-Spam-sender-reputation: 0 (email) ... X-Spam-hits: BAYES_50 0.8, ... ME_SENDERREP_DENY 4, |
15 Apr 2020, 02:24 AM | #28 |
Cornerstone of the Community
Join Date: Jan 2003
Location: Oxfordshire, UK
Posts: 603
|
Hi SideshowBob,
Thanks for verifying my snippet - appreciated. I used an address that I knew would "normally" end up in junk otherwise I couldn't test the snippet for being able to whitelist. I knew it generated the header correctly, but couldn't understand why it then didn't stop Fastmail's sieve section 3 from sending to junk. I take your point about something else sending it to junk instead, but I have no other rules that would do that and I only use the web interface or Fastmail's own app (iPhone/iPad). My intention is to create a whitelist without addresses being in my contacts (for addresses that I will never send to). J. edit: I should add that if I add the email address to my contacts, it does not go to junk - it goes to my inbox. Last edited by JamesHenderson : 15 Apr 2020 at 02:30 AM. Reason: added last line |
15 Apr 2020, 07:07 AM | #29 |
Essential Contributor
Join Date: May 2018
Posts: 474
|
It did? Who were you sending that to? Yourself? If so then the test of X-Spam-Known-Sender for "yes*" was always going to test true.
Here's what I tried: Code:
if allof(true, header :is "Subject" "test") { deleteheader "X-Spam-Known-Sender"; addheader "X-Spam-Known-Sender" "yes!"; if header :matches "X-Spam-Known-Sender" "*" { addheader "Debug" "X-Spam-Known-Sender=${1}"; } else { addheader "Debug" "Huh?"; } stop; } Code:
X-Spam-Known-Sender: yes! Debug: X-Spam-Known-Sender=yes ("Self sent message"); in-addressbook, self-send I've submitted a ticket on this. Update: Sieve Tester has the same behavior. I wonder if there's something in an RFC about this. Last edited by xyzzy : 15 Apr 2020 at 10:10 AM. |
15 Apr 2020, 05:41 PM | #30 | |
Cornerstone of the Community
Join Date: Jan 2003
Location: Oxfordshire, UK
Posts: 603
|
Quote:
I couldn't understand why my script *only* worked when the email address was still in my contacts (and didn't work when it wasn't), but that would explain it. ...albeit I don't understand how it is possible to test a header that has been deleted. I appreciate you sending the bug report. I did ask my original question of Fastmail support but haven't received an answer yet, If you want to associate your bug report with my query, the ticket ID is: PTN636591 I wonder if it is possible to to explicitly test the for of the last occurrence of a header (in the same way that *deleteheader* can have an index number to specify which occorance is deleted (not specifying an index deletes all occurrences). |
|