EmailDiscussions.com  

Go Back   EmailDiscussions.com > Email Service Provider-specific Forums > FastMail Forum
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
Stay in touch wirelessly

FastMail Forum All posts relating to FastMail.FM should go here: suggestions, comments, requests for help, complaints, technical issues etc.

Reply
 
Thread Tools
Old 28 Jul 2019, 06:56 AM   #1
jd2066
Junior Member
 
Join Date: Jul 2016
Posts: 3
Does FastMail have a better way then IMAP Idle to know when you get new emails?

Hey,
I use the FastMail service in an unusual way.
I have IMAP/STMP services on my personal home server that I can access remotely and such that I prefer over using remotely hosted services for various reasons.
However my home server is not reliable enough to have messages directly delivered to it (partly because I may do something that temporarily makes my server unavailable or my cable internet connection goes down and whatnot).

I use getmail to fetch any new messages from my FastMail account with the following configuration file:
Code:
[retriever]
type = SimpleIMAPSSLRetriever
server = imap.fastmail.com
username = [email protected]
password = mypassword
mailboxes = ("INBOX",)
move_on_delete = "Retrieved using getmail"

[options]
received = false
delivered_to = false
read_all = false
delete = true
verbose = 1
message_log = /var/log/getmailrc-justin/[email protected]
message_log_verbose = false

[destination]
type = MDA_external
path = /usr/libexec/dovecot/dovecot-lda
arguments = ("-e", "-f", "%(sender)", "-d", "justin")
I run getmail using the command
Code:
getmail -g /home/justin/.config/getmail -r [email protected] -i Inbox
.

It creates an IMAP Idle connection to FastMail so getmail knows instantly if there is new mail available and retrieves the email and then puts the email on FastMail in my "Retrieved using getmail" folder which I have set on FastMail to auto-delete the emails in every 30 days or so.

If getmail cannot connect to FastMail for whatever reason, I can login to FastMail.com and every email in the Inbox folder will be one that wasn't retrieved so I can see emails that I did not get due to some error or something and when getmail is running again, those messages will be retrieved.

I prefer IMAP idle over POP3 due to the instant knowing of a new email being available, with POP3 I would need to check every 5 minutes or so.

The MDA_external commend hands the email over to the Dovecot LDA service to run sieve filters on the email and then store the email in the file system so they can be be accessed via the Dovecot IMAP service.

However over the past few months I frequently get this error in the getmail log file "getmailOperationError error (IMAP error (command: UID => socket error: EOF))" which apparently means the connection to FastMail's IMAP service using IMAP Idle has been disconnected.
I'm not sure why but getmail's behavior when the IMAP Idle connection is disconnected is to just to close so until I manually restart getmail again I don't get email retreival.

I'm not sure if FastMail changed something recently, something happened with my internet provider or whatnot but I keep needing to restart getmail at least once a day and usually more now.

In the past, only my internet connection going away could cause the IMAP Idle connection to stop working so I rarely had to restart getmail again.

If there was some FastMail API I could use that was a more reliable way to know about new emails then I could just use that and tell it to run getmail whenever that API says new email is available.

Before FastMail I was using Google's email for domains service but their IMAP server worked in non-standard ways along with other things that I did not like.
jd2066 is offline   Reply With Quote

Old 28 Jul 2019, 08:17 AM   #2
BritTim
The "e" in e-mail
 
Join Date: May 2003
Location: mostly in Thailand
Posts: 2,791
As far as I know, there is no publicly announced API from FastMail that does what you want. I believe FastMail now uses JMAP on the backend, and that ought to provide a better interface for notification of changes than IMAP IDLE. FastMail also has push email when dealing with Apple Mail on both iOS and MacOS. No documentation I have seen on either.
BritTim is offline   Reply With Quote
Old 28 Jul 2019, 10:21 AM   #3
n5bb
Intergalactic Postmaster
 
Join Date: May 2004
Location: Irving, Texas
Posts: 8,524
Arrow JMAP new message push

Here is how Fastmail does this with JMAP:Bill
n5bb is offline   Reply With Quote
Old 28 Jul 2019, 01:03 PM   #4
EricG
Essential Contributor
 
Join Date: Aug 2009
Location: Canada
Posts: 230
Does the server have a static IP? If not, NAT will forget about idle connections.

All email clients deal with connection reset, so getmail has to also. The IMAP IDLE spec says the server can hang up after 29 minutes.
EricG is offline   Reply With Quote
Old 28 Jul 2019, 07:46 PM   #5
TenFour
Cornerstone of the Community
 
Join Date: Feb 2017
Posts: 530
Why not just forward the email from Fastmail or wherever to your local server-based email? I have found forwarding to be faster than IMAP in most cases with other services.
TenFour is online now   Reply With Quote
Old 29 Jul 2019, 12:15 AM   #6
SideshowBob
Member
 
Join Date: Jan 2017
Posts: 64
Quote:
Originally Posted by jd2066 View Post
However over the past few months I frequently get this error in the getmail log file "getmailOperationError error (IMAP error (command: UID => socket error: EOF))" which apparently means the connection to FastMail's IMAP service using IMAP Idle has been disconnected.
I'm not sure why but getmail's behavior when the IMAP Idle connection is disconnected is to just to close so until I manually restart getmail again I don't get email retreival.
Just wrap getmail in a loop.
SideshowBob is offline   Reply With Quote
Old 29 Jul 2019, 10:42 AM   #7
jd2066
Junior Member
 
Join Date: Jul 2016
Posts: 3
Quote:
Originally Posted by TenFour View Post
Why not just forward the email from Fastmail or wherever to your local server-based email? I have found forwarding to be faster than IMAP in most cases with other services.
I tried that once but as my server is on a network with a dynamic IP address and sometimes it can be down for a couple days, forwarding is just not that reliable.
The email server doing the forwarding will usually give up after a certain amount of tries and then I don't get the email at all.
Using IMAP fetch is much better as it's my server asking the email server for email instead of the email server assumed that my server is always available.


Quote:
Originally Posted by EricG View Post
Does the server have a static IP? If not, NAT will forget about idle connections.
The server has a static internal IP address, the external IP address is shared between all devices including the server via an ASUS router that uses NAT routing.

Quote:
Originally Posted by EricG View Post
All email clients deal with connection reset, so getmail has to also. The IMAP IDLE spec says the server can hang up after 29 minutes.
True, I'll have to check but I think the main getmail developers didn't want to bother handling that because they don't care to have proper IMAP Idle support due to the limitations of IMAP Idle.
As getmail is an open source project, someone can and did add IMAP Idle support but I don't think the main developers of getmail really care for the feature at all so they don't intend on improving the feature.
jd2066 is offline   Reply With Quote
Old 29 Jul 2019, 11:40 PM   #8
SideshowBob
Member
 
Join Date: Jan 2017
Posts: 64
Quote:
Originally Posted by jd2066 View Post
I'll have to check but I think the main getmail developers didn't want to bother handling that because they don't care to have proper IMAP Idle support due to the limitations of IMAP Idle.

My understanding is that getmail breaks out of IDLE after 5 minutes if no new mail arrives.

I don't see that getmail is doing anything wrong here. In some cases action may be needed after a networking problem, for example restarting a tunnel. getmail's exiting so a calling script can handle the problem before restarting getmail is sensible IMO.

Your problem is trivial to fix by calling getmail from a loop.
SideshowBob is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT +9. The time now is 07:43 AM.

 

Copyright EmailDiscussions.com 1998-2013. All Rights Reserved. Privacy Policy