Identities & Fetch Fetch settings vs. it's Sieve code

[xyzzy]

I was going to submit the following as a ticket. But I thought I would post it here for opinions of what I believe is a bug.
-------

I have three identities enabled and created; a.com, b.com, and c.com. Also three corresponding folders to filter into; a, b, c. Sieve generates the following code (fake X-LinkId for this example of course):

### 8. Sieve generated for fetch mail filing
elsif header :is "X-LinkId" "12345" {
  fileinto "INBOX.a;
}
elsif header :is "X-LinkId" "67890" {
  fileinto "INBOX.b";
}
elsif header :is "X-LinkId" "54321" {
  fileinto "INBOX.c";
}

Fine, that's what you would expect. But now I disable b's Fetch by unchecking "Fetch mail from this account into b and keep a copy". Or I keep it enabled and change "keep a copy" to "delete the copy", none of this has any effect on the sieve code. The condition and fileinto filter remains unchanged in the sieve code.

I can understand the possibility of the "keep a copy" or "delete the copy" being done at the server level thus not affecting the sieve code. But why does disabling the fetch by unchecking "Fetch mail from..." not cause the corresponding fileinto sieve clause to be deleted? With it deleted the b.com mail would end up in the Inbox unless an explicit organize rule is created. And that is exactly what happens if I have the "Fetch mail from..." set to Inbox instead of b. The sieve clause is deleted.

So in summary, enabling or disabling a Fetch identity's "Fetch mail from this account this account into..." has no effect on the sieve code unless the destination folder is the Inbox. I would expect the sieve clause to be deleted exactly as if Inbox were specified as the destination (unless an explicit filtering rule is also defined by the user). As it stands now mail will always get sorted into it's specified folder whether it's Fetch is enabled or disabled unless the destination folder is the Inbox.

What's the reasoning behind this or is this a bug (as opposed to a "feature"), or what am I missing in my reasoning?

[BritTim]

There is no reason for any change to the sieve code when you simply disable a POP link. The relevant code will simply never have any effect. Also, whether you keep or delete emails on the original server from which you fetch using a POP link is not controlled by the sieve code.

Are you seeing any unexpected results from use of these identities, or are you just trying to understand the sieve code to gain a better appreciation of how FastMail works?

[xyzzy]

Originally Posted by BritTim

There is no reason for any change to the sieve code when you simply disable a POP link. The relevant code will simply never have any effect.

I don't know how you can say that. As I discussed in my post if the destination folder is not Inbox it will get sorted into that specified folder even if the fetch is "disabled". If the code was removed when disabling the Fetch, as I was originally expecting, it would sort into the Inbox by default unless there was an explicit organize rule sorting it to some other folder.

Also, whether you keep or delete emails on the original server from which you fetch using a POP link is not controlled by the sieve code.

I mentioned (speculated) that too.

Are you seeing any unexpected results from use of these identities, or are you just trying to understand the sieve code to gain a better appreciation of how FastMail works?

Mostly to understand the reasoning behind this particular chunk of code and why it is what it is.

I first stumbled on this when I forgot the Fetch (pop link) was created. Without thinking I created an organize rule for the incoming email to the same folder the Fetch would sort it to. Since organize rules don't have a stop statement I could see it would end up sorting the same email to the destination folder twice; first from the organize and second from the Fetch (pop link). So I disabled the Fetch and that's when I noticed it didn't have any effect. Code was still there. I've gotten into the habit of checking the sieve code when I use the UI to see how using the UI affects the sieve code and that does help in understanding the language.

Of course all I have to do is remove my organize and I did. So all this now is just to confirm whether I am seeing a bug or this is deliberate and if so the reasoning behind it

[BritTim]

I don't know how you can say that. As I discussed in my post if the destination folder is not Inbox it will get sorted into that specified folder even if the fetch is "disabled". If the code was removed when disabling the Fetch, as I was originally expecting, it would sort into the Inbox by default unless there was an explicit organize rule sorting it to some other folder.

X-LinkId code refers only to messages that are retrieved using POP links. Each POP link uses a separate X-LinkId code. If the POP link is not used, even if you are receiving messages in a different way, the sieve snippet that indicates what to do during retrieval using that POP link will never be executed.

[xyzzy]

So are you saying that if the Fetch is disabled the incoming mail from that linked email service will never be delivered by the FM servers? In other words the FM servers also look at that enable/disable flag just like they look at the "keep a copy" and "delete the copy" flags? That is the one case I hadn't even considered.

If that's true then that's the missing piece of the puzzle in my understanding of that code that I've been looking for! And that's why you previously said the pop-link code would never be executed (i.e., email with the associated X-LinkId will never make it to the sieve code).

[BritTim]

Originally Posted by xyzzy

So are you saying that if the Fetch is disabled the incoming mail from that linked email service will never be delivered by the FM servers? In other words the FM servers also look at that enable/disable flag just like they look at the "keep a copy" and "delete the copy" flags? That is the one case I hadn't even considered.

If that's true then that's the missing piece of the puzzle in my understanding of that code that I've been looking for! And that's why you previously said the pop-link code would never be executed (i.e., email with the associated X-LinkId will never make it to the sieve code).

No, I am saying that sieve code created to process messages fetched using a POP link is only used at the time messages are received using that link. If you look at the headers of a message delivered normally to your mailbox (i.e. not fetched using a POP link) you will note that the message does not have an X-LinkId header. If you want to have filing rules for messages received normally at your account, you will need to create the appropriate rules yourself.

FastMail will not try to figure out whether the messages arriving via normal mail delivery are similar in some way to those that were previously fetched using a POP link. That would be a highly complex and error prone process.

[xyzzy]

Originally Posted by BritTim

No, I am saying that sieve code created to process messages fetched using a POP link is only used at the time messages are received using that link.

If that's the case I will go back to my original premise and we are going around in a circle. If I fetch mail from one of the identities it will of course have its requisite X-LinkId. If the server delivers that email to the sieve script, be that fetch disabled or not, the sieve code checking for that X-LinkId will be executed. I cannot see how interpreting that script has some kind of implicit condition to not execute that piece of sieve script if I flag that fetch as disabled and the server delivers that email.

In other words if the fetch is done on that pop link** and the server delivers that email to the sieve script there is no way I can see that chunk of code not executing when it matches the X-LinkId if the code remains in the script even though the fetch is disabled. Back to square one.

I think we may be talking past one another and missing something in the translation. I'll repeat it again, disabled or not, if I fetch that email, if that email has a X-LinkId (which for that identity it will) that code will be executed. I may not be a sieve expert but I've worked on compilers and interpreters long enough to say I don't know how that X-LinkId condition could fail to match its X-LinkId even if the fetch is disabled so long as that X-LinkId in the headers and the script.

------------

** Maybe that's some of the confusion here, the term "pop link" is not defined by Fastmail in any of it's documents (I searched) and I believe they removed the term from the UI after I questioned them on it. It's why I usually am referring to it as the "fetch identity".

[BritTim]

If that's the case I will go back to my original premise and we are going around in a circle. If I fetch mail from one of the identities it will of course have its requisite X-LinkId. If the server delivers that email to the sieve script, be that fetch disabled or not, the sieve code checking for that X-LinkId will be executed. I cannot see how interpreting that script has some kind of implicit condition to not execute that piece of sieve script if I flag that fetch as disabled and the server delivers that email.

In other words if the fetch is done on that pop link** and the server delivers that email to the sieve script there is no way I can see that chunk of code not executing when it matches the X-LinkId if the code remains in the script even though the fetch is disabled. Back to square one.

I think we may be talking past one another and missing something in the translation. I'll repeat it again, disabled or not, if I fetch that email, if that email has a X-LinkId (which for that identity it will) that code will be executed. I may not be a sieve expert but I've worked on compilers and interpreters long enough to say I don't know how that X-LinkId condition could fail to match its X-LinkId even if the fetch is disabled so long as that X-LinkId in the headers and the script.

I am clearly doing a terrible job of explaining, for which I apologise. Let me backtrack and start by explaining the two main ways messages can arrive at your account.

1. Most common, anyone can send a message to one of the email addresses associated with your account. I refer to this as "normal email delivery".
2. You can set up an Identity that includes manually fetching messages from another account. This is done by specifying the POP3 server, user and password associated with that other account. When such a fetch Identity is created, FastMail will periodically poll the Inbox of that other account, downloading any messages it finds there into your account. A long digression on why you might want to have messages arrive in your account this way is possible, but let's leave that for another time. If the fetch Identity (POP link) is disabled, the other account will not be polled. That does not prevent normal email delivery from occurring. However, nothing you put in a fetch Identity is relevant to normal email delivery.

For normal email delivery to your account, you can create Rules based on things like From header, To header, Subject line or message size to control filing of messages into a different folder than the default of Inbox. This has nothing to do with the X-LinkId header.

Messages that are manually fetched using a fetch Identity (from another account's POP server) are automatically given an extra header (the X-LinkId header you see referred to in your sieve script). At the same time you create the fetch Identity, you can also indicate the folder to be used when messages are fetched. When you do this, FastMail automatically adds a snippet to the sieve script to control this delivery to a specific folder. If no messages with a specific X-LinkId header are ever received (because the fetch Identity is disabled) the sieve snippet associated with that X-LinkId header will have no effect. Code of the form

if header Xx-LinkId :is "12345" {
  fileinto "Inbox.MyFolder;
}

will never be true and will be ignored.

If this is still unclear, perhaps someone else will help me out. A guy called Bill is often here and is good at explaining clearly.

[xyzzy]

You were doing great, I fully agree 100% with what you said, up to the following part:

Originally Posted by BritTim

Messages that are manually fetched using a fetch Identity (from another account's POP server) are automatically given an extra header (the X-LinkId header you see referred to in your sieve script). At the same time you create the fetch Identity, you can also indicate the folder to be used when messages are fetched. When you do this, FastMail automatically adds a snippet to the sieve script to control this delivery to a specific folder.

When you say "manually" I assume you mean when you explicitly click the identity's mailbox to spin the double arrows in the webmail. That as opposed to only other choice, i.e., the automatic 5-minute polling.

You are saying that the X-LinkId is added to that identity's mail "manually fetched using a fetch Identity". I submit that the X-LinkId will also be (and is) added if the auto 5-minute poll option is chosen instead of doing a manual refresh.

The confusion/disagreement, and the reason I started this thread, is over that "snippet" as you referred to it. That "snippet" is always created for a fetch identity when the destination folder to sort into is not the Inbox (the "specific folder" you referred to). And it doesn't matter whether the identity's "Fetch mail" is checked or unchecked (enabled or disabled). The snippet is created.

If no messages with a specific X-LinkId header are ever received (because the fetch Identity is disabled) the sieve snippet associated with that X-LinkId header will have no effect.

And this is where the confusion lies. Of course the snippet will not sort into the destination folder if the X-LinkId is missing or doesn't match the X-LinkId if one is there. My argument/confusion, and what started all this, is how disabling a fetch identity prevents messages for an identity being delivered in the first place, be it when auto polled or manually? Or alternatively, how is the snippet prevented from sending the message to its destination folder if the X-LinkId is present but the fetch identity is disabled?

It hadn't occurred to me when I started this thread but now seems the only reasonable possibility (which is what I thought you meant in a earlier post) is if the server doesn't access the identity's POP server if the fetch identity is disabled. So it is (must be) accessing the enable/disable state just like it accesses the "keep a copy"/"delete the copy" setting.

In other words the server has to know the enable/disable status of that identity to ensure, one way or another, the snippet condition will never be true when the fetch identity is disabled. Because if it doesn't check and inserts the X-LinkId for that identity, that snippet will always sort into the destination folder whether the fetch identity is enabled or not.

This is why I started this thread. What I hadn't thought of before I apparently, according to you, misunderstood what you were trying to say (and now I don't think so) is how could the fetch identity be disabled without the server knowing about it because the snippet is always there?

I don't see any other alternatives. Either the server accesses the identity's POP server or it doesn't. Not accessing the identity's POP server is the most reasonable interpretation of what disabling the fetch means. Auto or manual refresh doesn't affect this.

So the server has to know the enable/disable status (or some other way to keep it from fetching from the identity's POP server) because if it doesn't, and the message gets delivered to the sieve script including its X-LinkId that snippet is going to sort it to the destination folder whether enabled or not. As I said, not thinking about the server knowing the enable/disable status is what led to my confusion.

Are you still saying this can't be right, server doesn't know the enable/disable state, and somehow that snippet magically never sees or matches on the X-LinkId? If that was true the fetch would would end up placing the message in the Inbox if the fetch is disabled. And that doesn't happen.

[JeremyNicoll]

Originally Posted by xyzzy

... how disabling a fetch identity prevents messages for an identity being delivered in the first place, be it when auto polled or manually? Or alternatively, how is the snippet prevented from sending the message to its destination folder if the X-LinkId is present but the fetch identity is disabled?

If you turn off the feature that asks Fastmail to collect mail from somewhere else (ie 'disable the fetch identity'), Fastmail's system will not go and look for them. It's not Sieve filtering of incoming messages that controls that, but a separate process happening before Sieve takes effect.