fastmail reliability

[arw4f]

What can anyone (Jeremy) say about the reliability of fastmail.fm? One of my big concerns about an email provider is that "the mail will get through"! When people send messages to me that bounce back undeliverable, or delayed or yadda-yadda-yadaa... it isn't good.

When the service is 'down' - how is the mail handled? Take for example, when it's hit by spammers, as happened last week.

-Adam

[Jeremy Howard]

That's a really good question. Last week was the first time in two years that we went down for more than half an hour. With the diagnostics that we generated from that attack we've instituted a lot of new controls. They include:

• Every sent and received mail is tracked per user. If the number or size in a period of time is too great, the user is warned at their FastMail and alternative email address. If the unacceptable use continues, the account is locked for one hour and an email sent to the alternative address, and Rob and I are paged. If it still continues the user is locked out for good, and all email to the user is blocked at the SMTP server. This all happens automatically
• Every 10 minutes a script runs which tries to log in to the FastMail web interface, send a message, and receive the message. If this fails the web and mail server software is restarted (resulting in about 2 minutes of downtime). If it fails again in 10 minutes all non-core services are shut-down and Rob and I are paged
• Every 10 minutes the disk free, mail queue, and system load are tested, and we are paged if they are outside of limits
• Any user near to their quota is alerted that FastMail will stop accepting mail to them if they go over the quota. When they are over quota, messages are not bounced immediately, so that the user has a chance to purge old messages to allow new messages in.

In order to ensure that our network provider does not lose network access, we have selected a premises in the US that has multiple redudent diesel backup generators, backup air conditioners, and is located in a geologically stable area. See http://www.rackspace.com/infrastruct...tid=0104-99999 for details .

If FastMail is down, messages are not bounced to the sender, but instead are automatically redirected to Telstra's backup server. Telstra is the largest telecommunications provider in Australia. It is very unlikely that both Australian and US networks will be down simultaneously--we figure that in that case there's probably a big enough catastrophe going on that you don't mind losing email If FastMail is down for 3 days then Telstra will send a bounce notification to the sender. After 1 day they will send a warning that the message has not yet been delivered.

Obviously no-one can guarantee perfect uptime, but I can tell you that we are trying hard to provide a reliable service. We try hard to keep things secure too, blocking at a kernel level all non-essential ports, and using software known for its security. We support SSL for web, POP, and IMAP access, which means that your email and passwords are encrypted between fastmail.fm and your PC. For situations where security is mission-critical we suggest using encryption (such as PGP) to ensure that no-one but the desired recipiant reads your message.

[Lvsheng]

In my opinion, no one can really guarantee that his service is 100% reliable. Error sometimes did occur and even the most subtle testing can't reveal all possible errors. And suddenly some bad guy use a lot of zombie machine to ddos attack your site... uh oh. The site down, server crash, all incoming mail get bounced. So I think it is pretty hard to really be totally reliable. All admin of course strive to make their service as reliable as possible, but I think no one can really guarantee about it. It is quite beyond our control, we can secure all elements, but if someone really deliberately want your site down, I think they still can do it, with the right skill and time and efford.

Basically I judge a service reliability by its normal uptime, and pray that everytime I wanna log in, the site is still up. About how the mail is handle, I heard this method before. You still remember last time when MRB upgrade their Novell system? When they 'shut down or disconnect' their server for upgrade, incoming mail won't bounce. Instead, the mailer server wait and place the message in queue for remail after a predefined interval. Eg, you mail from yahoo to MRB, MRB is upgrading, you cannot log in MRB, site cannot access. But yahoo still want to send mail to it. So MRB 'tell' yahoo to wait a while (by sending a special transcript) and resend it later when MRB is 'free'. Then yahoo knows that MRB is 'busy', it places your message in a queue along with other people message (that previously placed in queue). Yahoo will mail all these messages in queue after 30 min, and if your messages still cannot be deliver after 3 days, yahoo will purge it from its queue. You will get a notification about your mail placed in queue or deleted if non-delivered.

This is how MRB handle those mail when it is down last time. Whether this method can widely use or not I dunno.

[Jeremy Howard]

Originally posted by abcd
In my opinion, no one can really guarantee that his service is 100% reliable. Error sometimes did occur and even the most subtle testing can't reveal all possible errors. And suddenly some bad guy use a lot of zombie machine to ddos attack your site... uh oh.

This is really good advice. If somebody tells you that they can handle a full distributed denial of service attack, they're lying. See http://grc.com/dos/grcdos.htm to learn how difficult DDOS is to fight. Also synchronize your email with IMAP or download it with POP regularly, and include your mail file in your regular backup routine.

About how the mail is handle, I heard this method before. <...> the mailer server wait and place the message in queue for remail after a predefined interval. Eg, you mail from yahoo to MRB, MRB is upgrading, you cannot log in MRB, site cannot access. But yahoo still want to send mail to it. So MRB 'tell' yahoo to wait a while (by sending a special transcript) and resend it later when MRB is 'free'. Then yahoo knows that MRB is 'busy', it places your message in a queue along with other people message (that previously placed in queue). <...> Whether this method can widely use or not I dunno. [/b]

Yep, this is completely standard. We do it--it takes all of one line in our DNS files to set it up Oh, and we have to pay the backup provider, of course. Having a backup provider in a different country is a pretty good idea (our servers are in the US and Australia).

[Lvsheng]

Oh, I am typing this thread offline, so I dunno that Jeremy has answer the question before me.

[arw4f]

Thanks for the explanation Jeremy. That helps me to feel a bit better.

Part of me would like to have a for-pay service only, where the free accounts wouldn't be available for the spammers utilize. The hope there being that a for-pay service might offer a barrier to entry for those spammers obtaining accounts.

That said, your steps seem to cover users abusing the service...so that's nice.

I guess my point here is...
My current setup is a mail relay address sending to Runbox currently. I had every intention of using them even while they start charging for their service (per my comments above). However, there are some things that I don't like about RB.

If FM is a reliable - that's the real key for me if we can't tell already - free or not, it may be up there on my list. However, I guess I'm just worried about signing on with a service that seems to have just surfaced. Not to tear down what seems to be a very powerful and well thought out service!! I just tend to be cautious in these things.

Granted, I've switched my relaying address between RB and FM no less than 5 times in the past 2-3 days...really depending on what my thoughts on each service are at the moment.

Ok, I will stop babbling now. But thanks again for your comments Jeremy!

Adam

[Jeremy Howard]

Originally posted by arw4f
If FM is a reliable - that's the real key for me if we can't tell already - free or not, it may be up there on my list. However, I guess I'm just worried about signing on with a service that seems to have just surfaced. Not to tear down what seems to be a very powerful and well thought out service!! I just tend to be cautious in these things.

Frankly, I think that that's a very reasonable level of caution. Although FastMail has been running for a couple of years, it's only in the last few weeks that we've started taking new users (it was just our 200 beta testers previously). But it's most important to us that fastmail.fm provides a good level of service, particularly because it's what we use for all of our own email, and the whole reason we created this in the first place is because we wanted to build a system that we liked using!

Regarding freebie users impacting paid users, I'll reserve judgement on whether we can deal with this or not. Rob and I see this as a challenge--where most other services have either gone paid only, or else have gone the low-performance ad-supported route, we would like to win the battle against the spammers and habitual mailing list subscribers rather than just give in... If we can't win this battle, then we'll remove the free account option rather than give up on providing a high level of service.

But hopefully it won't come to that. One thing I've been thinking about if the existing limits aren't enough (and I hope that they are!) is requiring a one-time signup fee of $5 for 'free' accounts. That would avoid people signing up who are just going to subscribe to mailing lists and then ignore them, and it would also force people to provide a valid credit-card and therefore a valid identity, which would make it less attractive to spammers. What do you think--would that be very off-putting to new users? What if we provided 2 months free, then a one-off fee of $5 to keep the account permanently?

[arw4f]

One thing I've been thinking about if the existing limits aren't enough (and I hope that they are!) is requiring a one-time signup fee of $5 for 'free' accounts. That would avoid people signing up who are just going to subscribe to mailing lists and then ignore them, and it would also force people to provide a valid credit-card and therefore a valid identity, which would make it less attractive to spammers. What do you think--would that be very off-putting to new users? What if we provided 2 months free, then a one-off fee of $5 to keep the account permanently?

That's an interesting idea. You might think about accepting paypal payments too...or might that bypass your goal of confirming identity? I can't remember the credentials required by paypal.

Anyway, I'd even pay $5 annually and consider it free!

I will try to keep thinking of ideas!
Adam

[princessmaeve]

.well, as for the fee or free issue.. well, I wouldn't mind paying 5 dollars as a fee in theory, but my love of privacy kind of conflicts with that...

I definitely understand the need to keep spammers off of the server, but I think that if any service required a 5 dollar setup fee I probably would just move on to a different server... I'm definitely a try-before-I buy person....

Perhaps you could set up a limited account for people until they pay (like 10 mails/day, or something that if abused, wouldn't tax the servers too much) or only so many of these limied accounts allowed per day. I'm not experienced in this type of thing, so I don't know how feasable this is...

I'll try to think of ideas, too. I'm definitely liking my fastmail.fm account.

[Jeremy Howard]

Originally posted by princessmaeve
.well, as for the fee or free issue.. well, I wouldn't mind paying 5 dollars as a fee in theory, but my love of privacy kind of conflicts with that...

Understood. I've been thinking about that too...

Perhaps you could set up a limited account for people until they pay (like 10 mails/day, or something that if abused, wouldn't tax the servers too much) or only so many of these limied accounts allowed per day. I'm not experienced in this type of thing, so I don't know how feasable this is...

I really like that idea--thanks Princess Another thing that we'd probably do is to make the payment completely optional. We'd just ask those that don't pay if they could drop us an email letting us know how they want to use the account and why they don't want to pay. That should have the same effect of putting off people who are signing up for 100 accounts to abuse the space or rate limits.

[princessmaeve]

yeah... sending an explanatory email is easy enough for normal users like me but still complicated enough that it should stop spammers, IMO...

[robu]

I thought you'd be reading here...so I wanted to let you know that the referring links on the Front Page of: http://emailaddresses.com is incorrect. You may want to let the author know.

Bob

Originally posted by Jeremy Howard
That's a really good question. Last week was the first time in two years that we went down for more than half an hour. With the diagnostics that we generated from that attack we've instituted a lot of new controls. They include:

• Every sent and received mail is tracked per user. If the number or size in a period of time is too great, the user is warned at their FastMail and alternative email address. If the unacceptable use continues, the account is locked for one hour and an email sent to the alternative address, and Rob and I are paged. If it still continues the user is locked out for good, and all email to the user is blocked at the SMTP server. This all happens automatically
• Every 10 minutes a script runs which tries to log in to the FastMail web interface, send a message, and receive the message. If this fails the web and mail server software is restarted (resulting in about 2 minutes of downtime). If it fails again in 10 minutes all non-core services are shut-down and Rob and I are paged
• Every 10 minutes the disk free, mail queue, and system load are tested, and we are paged if they are outside of limits
• Any user near to their quota is alerted that FastMail will stop accepting mail to them if they go over the quota. When they are over quota, messages are not bounced immediately, so that the user has a chance to purge old messages to allow new messages in.

In order to ensure that our network provider does not lose network access, we have selected a premises in the US that has multiple redudent diesel backup generators, backup air conditioners, and is located in a geologically stable area. See http://www.rackspace.com/infrastruct...tid=0104-99999 for details .

If FastMail is down, messages are not bounced to the sender, but instead are automatically redirected to Telstra's backup server. Telstra is the largest telecommunications provider in Australia. It is very unlikely that both Australian and US networks will be down simultaneously--we figure that in that case there's probably a big enough catastrophe going on that you don't mind losing email If FastMail is down for 3 days then Telstra will send a bounce notification to the sender. After 1 day they will send a warning that the message has not yet been delivered.

Obviously no-one can guarantee perfect uptime, but I can tell you that we are trying hard to provide a reliable service. We try hard to keep things secure too, blocking at a kernel level all non-essential ports, and using software known for its security. We support SSL for web, POP, and IMAP access, which means that your email and passwords are encrypted between fastmail.fm and your PC. For situations where security is mission-critical we suggest using encryption (such as PGP) to ensure that no-one but the desired recipiant reads your message.

[Jeremy Howard]

Originally posted by robu
I thought you'd be reading here...so I wanted to let you know that the referring links on the Front Page of: http://emailaddresses.com is incorrect. You may want to let the author know.

Oh, Edwin's done a review--cool!

I'll send me an email to let him know that the link is wrong.

[Lvsheng]

Yeah i got a new reliability issue in Fm.

TCP/IP: connection was refused while logging in to server.
Account: Fastmail

Time: 5.43am (GMT+8)

I am using IMAP. Before this error message, I am able to login, perform client side filtering etc. Suddenly this error message come out. I try again to login but fail too.

[Lvsheng]

Huh, how interesting, 5 minutes later, the problem gone. FM always like this, this minute it got problem, few minute later no more problem.