EmailDiscussions.com  

Go Back   EmailDiscussions.com > Email Service Provider-specific Forums > FastMail Forum
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
Stay in touch wirelessly

FastMail Forum All posts relating to FastMail.FM should go here: suggestions, comments, requests for help, complaints, technical issues etc.

Reply
 
Thread Tools
Old 1 Jul 2012, 01:37 AM   #1
brong
The "e" in e-mail
 
Join Date: Jul 2004
Location: Melbourne, Australia
Posts: 2,687

Representative of:
Fastmail.fm
Crashing servers

We were hit by the leap second bugs in the Linux Kernel. I've written up my experience on our technical blog.

http://blog.fastmail.fm/2012/07/03/a...aping-seconds/

Original post here below:

We're having a really bad day of crashing spam scanning servers. Mostly users won't see any issues, apart from slight delays (5-10 minutes) on incoming emails when one of the servers crashes.

Interestingly, it seems to not just be us - other departments in Opera are having failures too. At first we suspected the specific hardware, but we've had crashes on other equipment too. It might be a rough day.

On the plus side, our robust failover systems are working fine The process for dealing with a failed server is now so easy I can do it from my phone in ~30sec.

Last edited by brong : 4 Jul 2012 at 07:04 AM. Reason: Add blog entry
brong is offline   Reply With Quote

Old 1 Jul 2012, 06:38 AM   #2
brong
The "e" in e-mail
 
Join Date: Jul 2004
Location: Melbourne, Australia
Posts: 2,687

Representative of:
Fastmail.fm
Getting some press!

http://serverfault.com/q/403732/8453
http://news.ycombinator.com/item?id=4182642

And I think we're all fixed now
brong is offline   Reply With Quote
Old 1 Jul 2012, 11:48 AM   #3
n5bb
Intergalactic Postmaster
 
Join Date: May 2004
Location: Irving, Texas
Posts: 8,706
Bron, Amazon.com has been getting bad press today due to the combination of record high temperatures and high winds taking out utilities in the eastern US. Maybe their attempts at redirecting traffic were thwarted by server bugs.

Did this indeed turn out to be a time rollover bug?

Bill
n5bb is offline   Reply With Quote
Old 4 Jul 2012, 07:02 AM   #4
brong
The "e" in e-mail
 
Join Date: Jul 2004
Location: Melbourne, Australia
Posts: 2,687

Representative of:
Fastmail.fm
Yes. it did.

I've updated the original post of this thread to link to the technical blog, and pointed the technical blog here. Nice and cirular!

Last edited by brong : 4 Jul 2012 at 07:08 AM.
brong is offline   Reply With Quote
Old 6 Jul 2012, 06:04 PM   #5
rabarberski
Master of the @
 
Join Date: Nov 2006
Location: Ghent, Belgium
Posts: 1,027
I liked the long blog post you wrote about. Very entertaining...
(and thanks for handling this issue so well)
rabarberski is offline   Reply With Quote
Old 6 Jul 2012, 06:40 PM   #6
janusz
The "e" in e-mail
 
Join Date: Feb 2006
Location: EU
Posts: 4,820
Quote:
Originally Posted by brong View Post
Nice and cirular!
Nice and what??
janusz is online now   Reply With Quote
Old 6 Jul 2012, 06:54 PM   #7
rabarberski
Master of the @
 
Join Date: Nov 2006
Location: Ghent, Belgium
Posts: 1,027
Quote:
Originally Posted by janusz View Post
Nice and what??
Circular. The missing c was probably typed during a leap second and got lost...
rabarberski is offline   Reply With Quote
Old 6 Jul 2012, 09:28 PM   #8
brong
The "e" in e-mail
 
Join Date: Jul 2004
Location: Melbourne, Australia
Posts: 2,687

Representative of:
Fastmail.fm
I'm relaxing by the C right now...
brong is offline   Reply With Quote
Old 6 Jul 2012, 09:49 PM   #9
n5bb
Intergalactic Postmaster
 
Join Date: May 2004
Location: Irving, Texas
Posts: 8,706
Bron must have been listening to the crashing servers:
Broken Cirular Logic

Bill
n5bb is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT +9. The time now is 02:56 AM.

 

Copyright EmailDiscussions.com 1998-2013. All Rights Reserved. Privacy Policy