Suggestion: synchronized forwarding as a way to redice backscatter

[hadaso]

This idea is something I thought about as a result of discussions on the "refuse" action in Sieve on the FastMail forums.

I see a lot of misdirected bounces of email with my domain forged in the "from" that result from users at ISPs forwarding their email (mainly to Gmail that bounces Sober worm email). The final recipient does an SMTP bounce (550 refusal to accept the message) but the forwarding server then sends an email message to the forged address.

One way to avoid this would be "synchronized forwarding": when an email is received for an alias that is configured to forward all email elsewhere, before the email is accepted for delivery, the reciving server would contact the next server in the chain, and start the delivery process. If then it receives an SMTP error message, it can relay that message to the original server since that SMTP session is still open.

Of course there might be complications that might force the process to revert to the "old way" of first accepting the message, and then later forward it, e.g., when the final recipient's server is not available, but even then the amount of backscatter would be greatly reduced if most forwarded email would be relayed synchronously, and it seems that most email services nowadays are reliable enough to make such a scheme work for almost 100% of legitimate forwarded mail.

Does currently available server software allow such a scheme to work? Would it be possible to make such a scheme work? Any issues with this? It seems something that big forwarding services such as POBox should consider (it could certainly solve the incompatibility between SPF and forwarding).

[miley]

A few issues to overcome:
- what to do if the next guy also forwards. You can't know definitively if they also sync forwards (well, without out of band knowledge). Even if they do, do you want to hold the connection while they try (what about the next guy after that...)
- what to do if the forwarding address is a mailing list. I suppose its a special case of the above... Probably doesn't happen all that often, except for role accounts.
- what to do if the connecting server to you times out after your sync'd connection accepts? The original connecting server will retry, leading to dup messages (unless the sync forwarded can figure it out).

Anyways, this doesn't help solve the SPF forwarding problem. The final receiving system will *still* think the IP is the forwarder not the original sender.

So far, the SPF folks answer to forwarding is to get every forwarder on the planet to change to rewrite the bounce address to one they control. Issues like re-writing verp/batv'ed addresses and multi-forwarded mail are lft to the reader, as are incentive and timing issues.

All that aside, the problem you state can be solved with BATV. A sender just tags their bounce addresses in a predictable manner -- could be cryptographic; could be verp-style etc. When that sender gets a bounce notice (ie, the bounce address is '<>'; also suggested to look at mailer-daemon), they see if they generated the RCPT TO as a bounce address. If not, they didn't send the original mail, and they can drop the bounce notice.

The nice thing about this is that it is entire self contained -- no need for anyone else to adopt, and the adopter gains a nice benefit.

[hadaso]

Originally posted by miley
- what to do if the next guy also forwards. You can't know definitively if they also sync forwards ... (what about the next guy after that...)

Yeah, I thought about this problem. One way is to revert to the "old way" if the "next guy" doesn't respond fast enough. Actually after the "next guy" has indicated it is willing to accept the message the connection can be dropped and the message sent the "old way".

Originally posted by miley
- what to do if the connecting server to you times out after your sync'd connection accepts? The original connecting server will retry, leading to dup messages (unless the sync forwarded can figure it out).

Here I think you do not complete the "next step" until you have indicated to the connecting server that the email was received. It would avoid this problem (though it would miss bounces made at "end of data" that might include bounces made using Sieve "refuse" action).

I don't think of this as a perfect solution. There are many ways it can fail but in all cases I can see the solution is to revert to "store and forward", so it would not avoid all backscatter, but perhaps can significantly reduce the amount.

[beq]

Seems to me similar backscatter concerns exist for some store-and-forward backup MX servers?

Anyways, I think I've seen cases that probe the destination server before accepting from the source.

For example, TM's MX relay service has the following description (which is a couple of years old though):

The MX relay servers probe the destination server to see if it will accept mail addressed to the recipient before the MX relay servers will accept the message. If the destination server rejects the recipient the relay server will reject it also. This is done so as not to contribute to the backscatter problem. A database is kept of good and bad recipients so each message does not generate a probe to the destination server.
...
If the relay server knows that the recipient is valid from previous probes the message is accepted and queued. If the destination server can not be reached then the message is rejected with a temporary error and it is queued on the sending server.

Obviously MX relay services (ala Postini) are different than forwarding, since the message envelope recipient is preserved. And the above is only probing the destination server to check on that envelope recipient, not checking whether the destination accepts the whole message including body. With the probe, I assume the actual message is sent to the destination in a later SMTP session.

Whereas in your forwarding scenario, I assume the forwarder transfers everything including the message body (after DATA) to the destination in a synchronous session while keeping the sending server's SMTP session on hold? As has been discussed previously, I'm not sure how feasible that is for large messages...


P.S. [OT] What I also don't like with forwarding is the destination server's inherent inability to check (and reject) on the HELO/EHLO or IP address of the original sending server, among other things (not to mention extra work involved with SRS for envelope sender address).

[hadaso]

Originally posted by beq
...Whereas in your forwarding scenario, I assume the forwarder transfers everything including the message body (after DATA) to the destination in a synchronous session while keeping the sending server's SMTP session on hold? As has been discussed previously, I'm not sure how feasible that is for large messages...

What TM does seems good enough to stop most backscatter. It is sort of synchronized. In a more complicated setting a server might sometimes relay the whole message synchronously and sometimes decide to abort and revert to store and forward later, but if it happens after the final recipient has accepted the email (after positive response to RCPT) then there's not nuch difference.