EmailDiscussions.com

EmailDiscussions.com (http://www.emaildiscussions.com/index.php)
-   The Technical Zone... (http://www.emaildiscussions.com/forumdisplay.php?f=15)
-   -   Mail forwarding: good or bad? (http://www.emaildiscussions.com/showthread.php?t=72802)

hans2010 29 Jun 2017 05:18 PM

Mail forwarding: good or bad?
 
This is a continuation of a discussion from another thread. Some interesting technical issues were discussed there, with some points that I agree with. But I disagree with the opinion that mail forwarding is "wrong" (or "broken" as stated in the cited Richweb article).

Quote:

Originally Posted by jarland (Post 601981)
People want email to just work and they want it to just work like it did 10 years ago, and frankly that isn't possible. It doesn't anymore. Everything changed in the months following Yahoo's adoption of DMARC in late 2013.

...

Email forwarding only works for people who don't care. If your email is vital for your business, you're missing very important things. There is no other way that it can happen. Read this, and keep in mind that SRS doesn't totally fix all of this, it just provides a temporary bandaid for one single aspect of the problem:

http://archive.richweb.com/why_email...is_broken.html

There's nothing "wrong" with email forwarding. First, it conforms to published internet standards (RFCs 821, 822, and their successors) and is supported by those standards as a valid use case. Second, people in the real world have actual reasons for using this feature. They are using a legitimate feature in the way it was designed to be used, and get a benefit from it.

Let's look at the above-cited "Why Email Forwarding Is Broken" article from Richweb, and see why their article is broken:

1. The article characterizes legitimate use cases of forwarding as being "impersonation," which it isn't. As I mentioned, that characterization is supported neither by the RFCs nor by the users legitimately using the feature. Labeling it as "impersonation" seems to be an attempt by the authors at spin-doctoring (i.e., resorting to pejorative wording, instead of just relying on the merits of their technical argument).

(The above is the main problem with the Richweb article, the rest are perhaps less significant.)

2. They state that SPF stands for "SenderPermitted From". It actually stands for "Sender Policy Framework."

3. In the "Rosalie" example (test.com forwards to hotmail.com), they said "test.com is now seen as a SPAMSOURCE" in the scenario described. That is inaccurate. In that scenario, while the IP address (and possibly hostname) of any server transmitting the message might be identified as a spam source (including the server that hosts the "test.com" domain, whose domain name is presumably that of the hosting provider), the name of the hosted domain ("test.com") would not be identified as such, because that domain name appears in the "To" address of the message, not the "From" address (it is the "From" addresses that get scrutinized in this kind of analysis).

4. The article is poorly formatted (has many words run together due to missing spaces).

Although Richweb claims to have deep technical experience, the above 4 points detract from my impression of them as a source of expertise on this issue. (It's also a bit humorous that, at the bottom of the article, they claim over 17 years experience, which is apparently true... but they can claim a somewhat higher number if they've really been around since 1995!).

Here's my own use case. I have my own domain (website and email) hosted on Unix-based web hosting provider. I have various configured address of the form alias@mydomain which all arrive at my webhost. There, the messages get stored into flat Unix text files, and also get forwarded to my account at FastMail (with different aliases directed to different folders in my FastMail account). There are various reasons why I do this.

A. The webhost is good for Unix-level file actions (like grep, emacs, and any other ways of manipulating/archiving/organizing files that Unix allows, which is maximal flexibility), while FastMail is good as a mail UI.

B. I like having my mail stored in two places automatically, in case I need to access something while one service is down or not accessible, or in case of accidental data loss (could be me pushing the wrong button) on either service.

C. While FastMail is my day-to-day UI, the webhost is the best "first arrival" location because they do an SMTP-level reject of unknown aliases. I can't configure that on FastMail with folder-addressing and all the various folders I have there... even if I delete the folder, the mail still gets put into the Inbox.

D. Hosting an email domain on FastMail would cost me extra, whereas email hosting for my configured domain is included in the price of my service on the webhost.

The good news is: Forwarding works well for me. And with the minimum possible spam filtering, I rarely ever see any spam (probably because I'm careful to use throw-away aliases when signing up for online subscriptions, etc.). I'm not saying everyone should do exactly what I do, I'm just saying that my use case is legitimate, and it works. Saying that there's something "wrong" with my use case has no technical merit.

Perhaps someone might say that it may not be working, that I may be missing emails and never know about them. Perhaps, but if my webhost got blacklisted for forwarding spam (such as in Spamhaus, Invaluement, etc.), then other customers who use email services on that webhost would be affected, and the webhosting company would have an incentive to get off the blacklist quickly. And perhaps that's what motivates some service providers to do what the Richweb article mentioned, "Many web hosts are now banning email forwarding to third party emailaccounts."

But the solution isn't to outlaw forwarding. That's technically weak, because it incurs what's called "functional degradation," i.e., depriving users of legitimate functionality that used to work. Email providers (like Google and Microsoft) are now some of the richest companies in the world, so they ought to spend some money and brainpower coming up with solutions that don't incur functional degradation, not just punt on the problem and say "you can't forward anymore." I should point out that mailing lists are also forwarders. You can't just say, "no more mail groups, folks, sorry."

I'm not on the payroll of those companies (not anymore, anyway), but I'm willing to help think about this problem for free! Let's see....

Well, one thing FastMail does that helps is that they actually have an account setting where I can specify one or more hosts that I receive forwarded mail from. The form says, "Use only if forwarding email to FastMail from another system. The Received headers of these hosts are parsed to find the true email source." It seems that they're actually thinking about this problem with the customer in mind!

I think the ultimate solution has to do with reputation management. I think we're part of the way there, with clearinghouses such as Spamhaus and Invaluement. When an MTA receives a message, it should be able to assess a reputation depending first on who the immediate sender is, in combination with whatever upstream reputation assessment that sender passes along (ideally involving the reputation of the individual who sent the message, e.g., if I've been a paying customer for years with a service provider, they should attach a higher rating to my message than if I just signed up that day). Messages with low or unknown reputation at any link in the chain get rated lower, and then their messages might get throttled, filed separately, or rejected, depending on the rating, along with service provider policy and end-user preferences. Importantly, we should give the end user maximum choice over how they want dubious messages handled (e.g., FastMail has settings for what to do with messages at higher spam scores).

At the technical level, I'm not sure if this should involve Authenticated SMTP, SMTP over SSL, more reputation clearinghouses, or something else (they're not paying me enough to come up with the complete solution!). But I think it's a combination of things that could evolve from this point (perhaps along the lines I suggested above).

Quote:

Originally Posted by jarland (Post 602016)
it's companies like Yahoo, Microsoft, and Google making these changes to the industry. The rest can either play ball or ignore customer complaints. All because people who should probably just be using aliases insist on forwarding email.

I'd change that last statement to, "All because some companies that carry weight in the email market aren't willing to do the due diligence and come up with a sound technical solution that keeps the user's legitimate needs at a priority."

jarland 30 Jun 2017 02:28 AM

Quote:

Originally Posted by hans2010 (Post 602625)
This is a continuation of a discussion from another thread. Some interesting technical issues were discussed there, with some points that I agree with. But I disagree with the opinion that mail forwarding is "wrong" (or "broken" as stated in the cited Richweb article).

It's not an opinion. I don't mean to be that guy that's like "I'm always right" but there are a few things I know to be true in life without question. Up is up, down is down, IMAP was not designed for sending email, and forwarding is a feature that is on life support at best. People who desperately want forwarding to be a perfect thing, as well as providers who want to market to the large number of potential customers that demand this feature, will try to disagree for selfish reasons. I speak what I know, not what I wish, and it isn't intended to convince you to buy something. In fact, I sell something that I would actively say "Do not buy" if you insist on using email forwarding. I'll let you use it, I'll try to keep it up, but I'll point back at the customer when they say "I didn't receive an email that I really needed" and say "Then you should be treating your email as though it is important and not subjecting it to such a thing."

Quote:

There's nothing "wrong" with email forwarding.
Incorrect. You WANT there to be nothing wrong with email forwarding. So do I, but it doesn't matter what I want. I'm not the CEO of Google, Microsoft, Yahoo, Verizon, AT&T, or any other number of major providers that have rendered it to a state ranging from intermittently functional to completely broken.

Quote:

First, it conforms to published internet standards (RFCs 821, 822, and their successors) and is supported by those standards as a valid use case. Second, people in the real world have actual reasons for using this feature. They are using a legitimate feature in the way it was designed to be used, and get a benefit from it.
Here's an RFC for the use of carrier pigeons: https://tools.ietf.org/html/rfc1149

It doesn't matter what an RFC says. You can't shout at your monitor to Google or Microsoft "You're violating RFC" and expect change. RFC doesn't make reality, and anyone is free to violate them at any time. Microsoft very frequently accepts emails with a positive return code and then never delivers them, no error or bounce. This violates RFC standards. They don't care.

Quote:

1. The article characterizes legitimate use cases of forwarding as being "impersonation," which it isn't. As I mentioned, that characterization is supported neither by the RFCs nor by the users legitimately using the feature. Labeling it as "impersonation" seems to be an attempt by the authors at spin-doctoring (i.e., resorting to pejorative wording, instead of just relying on the merits of their technical argument).
No offense intended here, but I don't think you understand how SPF is used. Violating someone's SPF policy is absolutely impersonation. If yahoo.com says "Emails from yahoo.com ONLY come from these IP ranges" and you send an email from another IP range stating it to be from a @yahoo.com address, you have spoofed/impersonated their domain. It's their domain, they define what is impersonation, and most email providers respect this. How you feel about it is irrelevant, SPF defines where email can be sent from and providers respect it, therefore it is fact.

Quote:

2. They state that SPF stands for "SenderPermitted From". It actually stands for "Sender Policy Framework."
That's very minor, and doesn't change that SPF is relevant, or what it does.

Quote:

3. In the "Rosalie" example (test.com forwards to hotmail.com), they said "test.com is now seen as a SPAMSOURCE" in the scenario described. That is inaccurate. In that scenario, while the IP address (and possibly hostname) of any server transmitting the message might be identified as a spam source (including the server that hosts the "test.com" domain, whose domain name is presumably that of the hosting provider), the name of the hosted domain ("test.com") would not be identified as such, because that domain name appears in the "To" address of the message, not the "From" address (it is the "From" addresses that get scrutinized in this kind of analysis).
You are incorrect, but you may not know why yet. I'll explain two reasons this might be:

1. Forwarding MTA might add the domain elsewhere in a header that recipient server doesn't recognize shouldn't be included in content filtering/learning.
2. SRS, which is vital for forwarding, will have the email sent from the recipient domain to pass SPF. Recognizing this at the end point is not a promise not to filter based on it, and some do. Whether intentional or not, it happens.

Quote:

4. The article is poorly formatted (has many words run together due to missing spaces).
It didn't at one point, but something happened and that was the result. Regardless, I don't think accidental spacing issues when performing some kind of change to the blog later indicates any lack of experience or intelligence.

Quote:

Although Richweb claims to have deep technical experience, the above 4 points detract from my impression of them as a source of expertise on this issue. (It's also a bit humorous that, at the bottom of the article, they claim over 17 years experience, which is apparently true... but they can claim a somewhat higher number if they've really been around since 1995!).
I have quite a bit of experience and I chose that article for a reason. I didn't learn from it, I used it to express things that I already knew from managing mail servers at scale at HostGator, and continuing to do so at MXroute. I know what I say to be true because I deal with it. I also deal with people trying to project their feelings onto it regularly, telling me that I'm wrong because they so desperately want me to be. Obviously my patience for that grows a bit thin over time. I'm the one dealing with the problems and trying to prevent them from impacting other customers, where they usually do and providers just shrug their shoulders.

Quote:

The good news is: Forwarding works well for me. And with the minimum possible spam filtering, I rarely ever see any spam (probably because I'm careful to use throw-away aliases when signing up for online subscriptions, etc.). I'm not saying everyone should do exactly what I do, I'm just saying that my use case is legitimate, and it works. Saying that there's something "wrong" with my use case has no technical merit.
What if that one marketing email you get is the last straw for one particular IP and the next customer gets their email rejected at random when trying to send an email to their family member? You wouldn't know. I would. That's the reality I deal with. Being on the receiving end of consistently good dice rolls does not mean that you have found me, the one actually dealing with this stuff as a server admin, to be wrong.

Quote:

Perhaps someone might say that it may not be working, that I may be missing emails and never know about them. Perhaps, but if my webhost got blacklisted for forwarding spam (such as in Spamhaus, Invaluement, etc.), then other customers who use email services on that webhost would be affected, and the webhosting company would have an incentive to get off the blacklist quickly. And perhaps that's what motivates some service providers to do what the Richweb article mentioned, "Many web hosts are now banning email forwarding to third party emailaccounts."
Precisely.

Quote:

But the solution isn't to outlaw forwarding. That's technically weak, because it incurs what's called "functional degradation," i.e., depriving users of legitimate functionality that used to work. Email providers (like Google and Microsoft) are now some of the richest companies in the world, so they ought to spend some money and brainpower coming up with solutions that don't incur functional degradation, not just punt on the problem and say "you can't forward anymore." I should point out that mailing lists are also forwarders. You can't just say, "no more mail groups, folks, sorry."
That's cool, but how do you intend to make sure that they do that? How do you advise that anyone does that? If you assume that there's some super secret meeting that one can attend, I assure you there is not.

My reply was too long for this forum, you can read the rest here:
https://paste.jarcloud.pw/ejolaqupot.sql

hans2010 30 Jun 2017 08:12 PM

Quote:

Originally Posted by jarland (Post 602642)
It's not an opinion. I don't mean to be that guy that's like "I'm always right"

OK, then let's say I disagree with the indicated statement (not "opinion").

Quote:

Incorrect. You WANT there to be nothing wrong with email forwarding. So do I, but it doesn't matter what I want. I'm not the CEO of Google, Microsoft, Yahoo, Verizon, AT&T, or any other number of major providers that have rendered it to a state ranging from intermittently functional to completely broken.
First, if they "rendered" something broken that used to work, it doesn't mean that the thing in question is flawed technically. It means that some companies created flawed implementations that didn't used to be flawed. Following your logic, one would say "chocolate is bad for you" if two or three of the major suppliers started putting toxins in it. In that case, the statement may be valid in some sense, but I'd still disagree because it inaccurately characterizes the problem.

Second, the "range" is actually wider than you describe (that is, the range goes to far better than "intermittently functional"... even if it didn't where you worked). I do comparisons of messages received on the host that initially receives my mail vs. the host I forward to. I can tell if something failed to forward. I estimate that I've lost less than one message per 10,000 on a forward. The one I can think of was because the sender's domain was on the Invaluement blacklist. My webhost doesn't check that blacklist, so it accepted the message, but FastMail does, so they rejected the forwarded message (not because it was forwarded). Now, if I had done what you advocate and not do any forwarding, I'd never know that the sender was on the blacklist. Good thing I knew, because I was able to help them get off it (it was an arts school with nobody technical working there).

Not trying to argue with you, just adding a data point that you might not have known existed (someone using forwarding successfully in the present age). Revising your sentence for accuracy, it would say, "ranging from nearly flawless to completely broken."

Quote:

Here's an RFC for the use of carrier pigeons: https://tools.ietf.org/html/rfc1149
Non-sequitur. I said "Internet standards." That RFC isn't on the standards track.

Quote:

It doesn't matter what an RFC says. You can't shout at your monitor to Google or Microsoft "You're violating RFC" and expect change. RFC doesn't make reality, and anyone is free to violate them at any time. Microsoft very frequently accepts emails with a positive return code and then never delivers them, no error or bounce. This violates RFC standards. They don't care.
You may be interested to know that the very companies you mentioned actually collaborate on these specifications and publish them as RFCs. Strange, since, according to you, they don't care about them.

What matters to me more is if the service providers I pay money to care about the standards. I have found that they do, and that's backed up in action by implementing systems that do, in fact, implement those standards competently (or so has been my experience with them). I'm not on any of the major providers you mentioned, so I don't care what they do other than allow me to get mail to/from their users via my service providers. I haven't had any trouble getting mail back and forth to people who are on those major services. I don't interact with the general public, mainly just people who I see in person or who I might talk with on the phone (and various businesses like my bank, etc.). So if they ever had trouble getting mail to me, I'd have heard from them about it by now. Apparently, the major providers implement the standards well enough that my service providers have a high success rate getting mail back and forth to them... even with the forwarding I'm doing.

Again, not arguing, just adding a data point that you apparently thought could not exist.

Quote:

No offense intended here, but I don't think you understand how SPF is used.
Actually, I do. No offense, but I don't think you understand how to tell that someone understands!

Quote:

Violating someone's SPF policy is absolutely impersonation. If yahoo.com says "Emails from yahoo.com ONLY come from these IP ranges" and you send an email from another IP range stating it to be from a @yahoo.com address, you have spoofed/impersonated their domain. It's their domain, they define what is impersonation, and most email providers respect this. How you feel about it is irrelevant, SPF defines where email can be sent from and providers respect it, therefore it is fact.
I know you don't care about RFCs, but those documents don't label people who do forwarding with pejorative labels like spoofers and impersonators. I guess if you and Richweb are higher authorities than the RFC authors then that's that. But I disagree with it.

Also, if you're not relying on the RFCs, then I don't know what you're referring to when you say "SPF defines". And if you are relying on the RFCs, then your statement is inaccurate, in that they recognize forwarding as a legitimate action and recommend strategies for getting SPF to work in cases where forwarding is happening. And they do not refer to it as spoofing/impersonation.

Maybe you should cite what definition of SPF you are referring to, because it's certainly not the one I found.

I agree that how I feel about it is irrelevant, except when I make choices what service providers I give my money to. And I'm satisfied with the providers I have.

Quote:

Being on the receiving end of consistently good dice rolls
It's not the dice rolls. It's me being picky about what service providers I give my money to, and how competently they deliver their services. They've done a good job, and they continue to do so.

Quote:

That's very minor, and doesn't change that SPF is relevant, or what it does.
I already said that points 2, 3, and 4 were relatively minor. Still, it was my experience when I was in the industry that the people who knew what they were talking about didn't make up their own definitions for acronyms (like the Richweb article did). Plus, they knew how to represent themselves professionally, meaning that their writings (either online or on paper) weren't littered with errors that a simple copyedit would fix (as seen in the Richweb article). These things definitely correlated to technical competence.

That was just my experience. Yours may be different.

Quote:

You are incorrect, but you may not know why yet. I'll explain two reasons this might be:
Those two reasons weren't mentioned in the Richweb article, they just presented a more general forwarding case. If they meant to be discussing a special situation where the breakage occurs (i.e., that doesn't occur in the general case), then a competent technical article would've explained that.

In any case, I can add another data point for you. I've been using my own domain in combination with the forwarding arrangement I described previously (which is very similar to the 'Rosalie' example) for 15 years. In all that time, even though I was a recipient of some amount of spam over that time, I never had the experience where my domain name got blacklisted as a result of appearing in the "To" header.

Again, not disagreeing with you. Just adding a data point that shows that the system does work, at least on competently designed servers.

Quote:

That's cool, but how do you intend to make sure that they do that? How do you advise that anyone does that? If you assume that there's some super secret meeting that one can attend, I assure you there is not.
The secret meeting is called IETF. If you want to advocate that forwarding be prohibited, you should go there to make your case. You'll get more mileage by advocating your position there than by writing about it on this forum.

I'm no longer on the payroll of any company that's involved in email, so I probably won't show up there myself. I'm just an end user, hoping that the people who are paid to work on this stuff work out a solution that's in our best interests, not just punt on the problem and impose functional degradations on us all. Yes, it may be hard work, but that's what the 'E' in 'IETF' is for. It means to think and work constructively, not just give up. In my previous message, I pointed out how FastMail is looking at the problem constructively, and alluded to how further progress can be made. Being optimistic, my prediction is that the ultimate solution will be something along those lines.

I stand by my earlier statements.

jarland 3 Jul 2017 07:45 AM

Content filters are imperfect. Content filters are used to block sending servers. Forwarders cause you to be a sender of emails that trigger content filters. Different people will have varying degrees of impact on this due to matters outside of their control. Namely, who targets them for spam. That is something a mail provider has no control over, how many spam lists a customer's email address is on.

We can wax poetic about it all day, but those are hard facts and it doesn't matter how anyone feels about them. We can argue about what/why, but forwarders get you and other people blocked by major mail providers far more commonly than a lack of forwarders. Period. Cold, hard fact. It's unacceptable to allow one customer to harm another's quality of service. Individual anecdotal experiences do not compare to my experience. Not trying to be condescending, but it's like being an accountant and someone on the street walks up to you and tells you that you're not an accountant. That's about as annoyingly condescending as it gets for me. I'm either lying or you're wrong, period. I'm not lying. You do the math.

Now I've given you every opportunity to understand what it's like, you refuse to accept it, that's your problem and no longer mine. I have 3,000-10,000 emails per hour to check up on every day, I don't have much time to explain why I'm not hallucinating or making up stories. If you're not interested in my insights, I'm just as happy to not give them.

SideshowBob 4 Jul 2017 12:52 AM

Quote:

Originally Posted by jarland (Post 602642)
Incorrect. You WANT there to be nothing wrong with email forwarding. So do I, but it doesn't matter what I want. I'm not the CEO of Google, Microsoft, Yahoo, Verizon, AT&T, or any other number of major providers that have rendered it to a state ranging from intermittently functional to completely broken.


That's not really fair. The large free email providers have implemented, or are about to implement, ARC (Authenticated Received Chain) which aims to fix the the forwarding problem.

I think you are overstating the current problem, the reality is that experiences range from unreliable to just fine.

jarland 4 Jul 2017 05:22 AM

Quote:

Originally Posted by SideshowBob (Post 602817)
That's not really fair. The large free email providers have implemented, or are about to implement, ARC (Authenticated Received Chain) which aims to fix the the forwarding problem.

I think you are overstating the current problem, the reality is that experiences range from unreliable to just fine.

Hopefully providers who struggle with this will let their customers know, the next time one of their emails to a family member gets rejected because someone's forwarder caused an IP to be rate limited, that you said it was working fine. I'm certain that will alleviate their concern and the mail provider won't lose a customer over it. I'll certainly send the next customer of mine who complains about the filtering I have to do, for forwarding to work with any consistency, to you and let you tell them that they're imagining problems.

In this thread: People who think "because it works for me" is any relevant indication of what happens at scale.

SideshowBob 4 Jul 2017 10:48 PM

Quote:

Originally Posted by jarland (Post 602819)
Hopefully providers who struggle with this will let their customers know, the next time one of their emails to a family member gets rejected because someone's forwarder caused an IP to be rate limited, that you said it was working fine. I'm certain that will alleviate their concern and the mail provider won't lose a customer over it. I'll certainly send the next customer of mine who complains about the filtering I have to do, for forwarding to work with any consistency, to you and let you tell them that they're imagining problems.

In this thread: People who think "because it works for me" is any relevant indication of what happens at scale.

Behind the sarcasm and general obnoxiousness it sounds like you are actually agreeing with me that it works for some people and is unreliable for others. That's a big change from "intermittently functional to completely broken".

hans2010 9 Jul 2017 04:26 AM

Quote:

Originally Posted by jarland (Post 602786)
Now I've given you every opportunity to understand what it's like, you refuse to accept it, that's your problem and no longer mine.

I understand the problem exactly. I'm not the one constantly complaining and refusing to take a constructive stance. So you should reconsider who it is that has a problem.

Quote:

If you're not interested in my insights, I'm just as happy to not give them.
I laid out a way to look at the problem constructively but you've shown no openness to it. You should consider that fact before criticizing others for lack of openness.

Quote:

Originally Posted by jarland (Post 602819)
(snip) ... I'll certainly send the next customer of mine who complains about the filtering I have to do, for forwarding to work with any consistency, to you and let you tell them that they're imagining problems....
(snip)

Quote:

Originally Posted by SideshowBob (Post 602846)
Behind the sarcasm and general obnoxiousness it sounds like you are actually agreeing with me that it works for some people and is unreliable for others. That's a big change from "intermittently functional to completely broken".

I agree with SideshowBob in all aspects of his post.

I'm glad my service providers have attitudes that lead them to solving problems rather than getting mired in them.

Good day.

TenFour 25 Jul 2017 08:47 AM

I'm not experiencing problems based on my own checking of various email addresses being forwarded to Gmail. Not losing any emails.


All times are GMT +9. The time now is 01:36 PM.


Copyright EmailDiscussions.com 1998-2022. All Rights Reserved. Privacy Policy