We Keep Coding

Sometimes login sometimes not to Exchange by JavaMail

I have problem with randomly login failed. By one hour is happening 2-4 times, and similar time, for example: about 14:26, 14:48, 15:26, 15:49, etc.
I use JavaMail(ver. 1.5.6) to email processing. Mailbox is Microsoft Exchange.
Application run every 5 minute to connect to mailbox.
Pseudocode for email processing:
estabilishConncetion()
prepareMailbox()
downloadEmails()
moveEmailsToSOmething()
closeConnection()
Code connection is like this:
public void establishConnection() throws ConnectionException {
logger.info("Establish connection... ");
props = new Properties();
props.setProperty("mail.imaps.ssl.enable", "true");
props.setProperty("mail.imaps.port", 993);
props.setProperty("mail.imaps.timeout", 60000);
props.setProperty("mail.imaps.connectiontimeout",60000 );
props.put("mail.imaps.auth.plain.disable", "true");
props.put("mail.imaps.auth.ntlm.disable", "true");
props.put("mail.imaps.auth.gssapi.disable", "true");
props.setProperty("mail.debug", "true"); // only for test
Session session = Session.getInstance(props, new Authenticator(){
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(somename, somepassword);
}
});
try {
store = session.getStore("imaps");
store.connect(EmailServerAddress, EmailAddress,Password);
} catch (MessagingException e) {
throw new ConnectionException(" Problem occured when try connect to host: " + ServerAddress + " on address: " +EmailAddress);
}
logger.info("Connected to mailbox.");
}
Logging when connected:
DEBUG IMAPS: trying to connect to host "somehost", port 993, isSSL true]]
* OK The Microsoft Exchange IMAP4 service is ready.]]
A0 CAPABILITY]]
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN
STARTTLS UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+]]
A0 OK CAPABILITY completed.]]
DEBUG IMAPS: protocolConnect login, host=somehost,
user=someuser, password=]]
DEBUG IMAPS: mechanism LOGIN not supported by server]] DEBUG IMAPS:
LOGIN command trace suppressed]] DEBUG IMAPS: LOGIN command result: A1
OK LOGIN completed.]]
A2 CAPABILITY]] * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI
AUTH=PLAIN STARTTLS UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+]]
A2 OK CAPABILITY completed.]]
DEBUG IMAPS: AUTH: NTLM]] Connected to mailbox.]]
Logging when not connected:
DEBUG IMAPS: trying to connect to host "somehost", port 993, isSSL
true]]
* OK The Microsoft Exchange IMAP4 service is ready.]]
A0 CAPABILITY]]
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN
STARTTLS UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+]]
A0 OK CAPABILITY completed.]]
DEBUG IMAPS: protocolConnect login, host=somehost, user=someuser,
password=]]
DEBUG IMAPS: mechanism LOGIN not supported by server]] DEBUG
IMAPS: LOGIN command trace suppressed]] DEBUG IMAPS: LOGIN command
result: A1 NO LOGIN failed.]] //trying to login one more time with
failure
I checked this and its ok in my code:
1. Using JavaMail to connect to IMAP getting "A1 NO LOGIN failed" exception
2. https://javaee.github.io/javamail/FAQ#Exchange-login
3. I have unit test realted with connection and they always passed
Any sollution, clues, how to better debug this? :)

How to receive email from hotmail via IMAP and OAuth?

I want to receive Email from Hotmail by IMAP, but I have to authenticate by OAuth. From the Microsoft documentation, it supports that, http://msdn.microsoft.com/en-us/library/dn440163.aspx;
But I have to idea how to use it in java (by Javamail). I tried to use google code sample (correct email and access_toket), but failed.
String email = "stony#hotmail.com";
String oauthToken = "XXXXXXXX";
initialize();
IMAPStore imapStore = connectToImap("imap-mail.outlook.com",
993,
email,
oauthToken,
true);
System.out.println("Successfully authenticated to IMAP.\n");
Folder folder = imapStore.getDefaultFolder();
folder.open(Folder.READ_ONLY); //****The linke throw exception****
for (Message msg : folder.getMessages()) {
String subject = msg.getSubject();
System.out.println(":" + subject);
}
The log printed on system console is,
DEBUG: setDebug: JavaMail version 1.4
DEBUG: mail.imap.fetchsize: 16384
DEBUG: enable SASL
DEBUG: SASL mechanisms allowed: XOAUTH2
* OK Outlook.com IMAP4rev1 server version 17.4.0.0 ready (DUB451-IMAP183)
A0 CAPABILITY
* CAPABILITY IMAP4rev1 CHILDREN ID NAMESPACE UIDPLUS UNSELECT AUTH=PLAIN AUTH=XOAUTH2 SASL-IR
A0 OK CAPABILITY completed
IMAP DEBUG: AUTH: PLAIN
IMAP DEBUG: AUTH: XOAUTH2
DEBUG: protocolConnect login, host=imap-mail.outlook.com, user=mebeautyful#hotmail.com, password=<non-null>
IMAP SASL DEBUG: Mechanisms: XOAUTH2
IMAP SASL DEBUG: SASL client XOAUTH2
A1 AUTHENTICATE XOAUTH2
+
IMAP SASL DEBUG: challenge: :
IMAP SASL DEBUG: callback length: 1
IMAP SASL DEBUG: callback 0: javax.security.auth.callback.NameCallback#61ead9ba
IMAP SASL DEBUG: response: user=mebeautyful#hotmail.comauth=Bearer
DEBUG: connection available -- size: 1
A2 EXAMINE ""
A1 NO [AUTHENTICATIONFAILED] OAuth authentication failed.
A2 BAD Examine Command is not permitted in current state (NotAuthenticated)
A3 LOGOUT
* BYE Logout requested
Exception in thread "main" javax.mail.MessagingException: A2 BAD Examine Command is not permitted in current state (NotAuthenticated);
nested exception is:
com.sun.mail.iap.BadCommandException: A2 BAD Examine Command is not permitted in current state (NotAuthenticated)
at com.sun.mail.imap.IMAPFolder.open(IMAPFolder.java:829)
at com.google.code.samples.oauth2.OAuth2Authenticator.main(OAuth2Authenticator.java:155)
Caused by: com.sun.mail.iap.BadCommandException: A2 BAD Examine Command is not permitted in current state (NotAuthenticated)
at com.sun.mail.iap.Protocol.handleResult(Protocol.java:296)
at com.sun.mail.imap.protocol.IMAPProtocol.examine(IMAPProtocol.java:636)
at com.sun.mail.imap.IMAPFolder.open(IMAPFolder.java:811)
... 1 more

I tested successfully! I verified 3 things,
We can use google code sample to collection to outlook's IMAP. The sample code link is https://code.google.com/p/google-mail-oauth2-tools/wiki/JavaSampleCode
Must use the scope wl.imap and wl.offline_access
In my hotmail acccount doesn't work, I can't even get a access_token; but after I use my another live account, it works.

You're using an older version of JavaMail. If you use the latest version, OAuth2 support is built in.

JavaMail - getMessageCount changes after opening imap folder?

I'm writing a program that connects to gmail and checks the number of messages in the inbox. If I call getMessageCount() on the unopened Folder, it returns 554, but after opening it, it returns 541. Why the difference?
By the way, I'm ultimately trying to check for deleted messages (that have not been expunged) and recover them if possible.
Edit: some example code:
Session session = Session.getDefaultInstance(System.getProperties());
Store store = session.getStore("imaps");
store.connect("imap.googlemail.com", 993, email, password);
Folder inbox = store.getFolder("INBOX");
System.out.println(inbox.getMessageCount());
inbox.open(Folder.READ_ONLY);
System.out.println(inbox.getMessageCount());
Edit: protocol trace:
(In the meantime I got some more emails so the numbers are different from above)
DEBUG: setDebug: JavaMail version 1.5.2
DEBUG: getProvider() returning javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle]
DEBUG IMAPS: mail.imap.fetchsize: 16384
DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
DEBUG IMAPS: mail.imap.appendbuffersize: -1
DEBUG IMAPS: mail.imap.minidletime: 10
DEBUG IMAPS: trying to connect to host "imap.googlemail.com", port 993, isSSL true
* OK Gimap ready for requests from <ip> vt17mb17567619iec
A0 CAPABILITY
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN
A0 OK Thats all she wrote! vt17mb17567619iec
DEBUG IMAPS: AUTH: XOAUTH
DEBUG IMAPS: AUTH: XOAUTH2
DEBUG IMAPS: AUTH: PLAIN
DEBUG IMAPS: AUTH: PLAIN-CLIENTTOKEN
DEBUG IMAPS: protocolConnect login, host=imap.googlemail.com, user=<email>, password=<non-null>
DEBUG IMAPS: AUTHENTICATE PLAIN command trace suppressed
DEBUG IMAPS: AUTHENTICATE PLAIN command result: A1 OK <email> authenticated (Success)
A2 CAPABILITY
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT
A2 OK Success
A3 LIST "" INBOX
* LIST (\HasChildren) "/" "INBOX"
A3 OK Success
A4 STATUS INBOX (MESSAGES RECENT UNSEEN UIDNEXT UIDVALIDITY)
* STATUS "INBOX" (MESSAGES 585 RECENT 0 UIDNEXT 3389 UIDVALIDITY 2 UNSEEN 0)
A4 OK Success
585
DEBUG IMAPS: connection available -- size: 1
A5 EXAMINE INBOX
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $Phishing $Forwarded NonJunk $NotPhishing Junk)
* OK [PERMANENTFLAGS ()] Flags permitted.
* OK [UIDVALIDITY 2] UIDs valid.
* 548 EXISTS
* 0 RECENT
* OK [UIDNEXT 3389] Predicted next UID.
* OK [HIGHESTMODSEQ 687472]
A5 OK [READ-ONLY] INBOX selected. (Success)
548

JavaMail IMAP message content SSL/NON SSL

I am experiencing weird problems over IMAP in JavaMail.
I already know alot in terms of JavaMail and I also know where to look for Questions/Answers, yet I cannot find a solution for this issue.
I have IMAP channel, on which I can set SSL enabled or disabled.
When I create new email message (let's say, Subject: "TEST", Body: "HELLO") and send it from Thunderbird client, to be able to download it with my IMAP channel, things are getting weird.
On SSL ENABLED, everything works fine. (in my IMAP channel) I get message from email folder, after successfuly connecting to the store, then i getContent() from this message (it is ALWAYS String type, not MimeMultipart or Multipart), and output is "HELLO".
However, on SSL DISABLED, everything also works fine, but I receive different body. It looks like whole message parsed (look below)
message.getContent().toString(); using SSL
HELLO
message.getContent().toString(); NOT using SSL
Return-Path: and#127.0.0.1
Received: from [127.0.0.1] (localhost [127.0.0.1]) by PC ; Wed, 22 Jan 2014 13:23:17 +0100
Message-ID: <52DFB835.9020300#127.0.0.1>
Date: Wed, 22 Jan 2014 13:23:17 +0100
From: "and#127.0.0.1" <and#127.0.0.1>
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130509 Thunderbird/17.0.6
MIME-Version: 1.0
To: and#127.0.0.1
Subject: TEST
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Does anyone have any idea what's going on? Below are fragments of my code (to show you most important parts:
// properties used to create session
String protocol = useSSL ? "imaps" : imap";
properties.setProperty("mail.store.protocol", protocol);
properties.setProperty("mail." + protocol + ".host", ...);
properties.setProperty("mail." + protocol + ".user", ...);
properties.setProperty("mail." + protocol + ".port", ...);
properties.setProperty("mail." + protocol + ".password", ...);
// only when using ssl
properties.setProperty("mail.imaps.auth", "true");
properties.setProperty("mail.imaps.starttls.enable", "true");
properties.setProperty("mail.imaps.ssl.checkserveridentity", "true");
properties.setProperty("javax.net.ssl.trustStore", ...);
properties.setProperty("javax.net.ssl.trustStorePassword", ...);
// get store and other important things
Store store = this.session.getStore(protocol);
Folder folder = store.getFolder("INBOX");
folder.open(Folder.READ_WRITE);
Message[] newMessages = folder.getMessages();
for (Message newMessage : newMessages) {
Object o = newMessage.getContent();
System.out(o.toString()); // should write "HELLO", but instead it parses whole message (only on NON SSL MODE!)
}
// edit : added debug info
// edit 2 : debug info contains now full details with fetching folder, using { CONTENT_INFO, FLAGS, ENVELOPE }
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc]
DEBUG: mail.imap.fetchsize: 16384
DEBUG: protocolConnect returning false, host=127.0.0.1, user=and#127.0.0.1, password=<null>
* OK IMAPrev1
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
A0 OK CAPABILITY completed
DEBUG: protocolConnect login, host=127.0.0.1, user=and#127.0.0.1, password=<non-null>
A1 LOGIN and#127.0.0.1 password
A1 OK LOGIN completed
DEBUG: connection available -- size: 1
A2 SELECT INBOX
* 1 EXISTS
* 0 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1378802425] current uidvalidity
* OK [UNSEEN 50167] unseen messages
* OK [UIDNEXT 50168] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
A2 OK [READ-WRITE] SELECT completed
A3 SEARCH UNSEEN ALL
* SEARCH 1
A3 OK Search completed
A4 FETCH 1 (BODYSTRUCTURE)
* 1 FETCH (UID 50167 BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "UTF-8") NIL "test" "7bit" 7 2))
A4 OK FETCH completed
A5 FETCH 1 (BODY[TEXT]<0.7>)
* 1 FETCH (UID 50167 ENVELOPE ("Tue, 28 Jan 2014 15:08:59 +0100" "test" (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) NIL NIL NIL "<52E7B9FB.2030409#127.0.0.1>") BODY[HEADER] {479}
Return-Path: and#127.0.0.1
Received: from [127.0.0.1] (localhost [127.0.0.1]) by A-PC ; Tue, 28 Jan 2014 15:08:59 +0100
Message-ID: <52E7B9FB.2030409#127.0.0.1>
Date: Tue, 28 Jan 2014 15:08:59 +0100
From: "and#127.0.0.1" <and#127.0.0.1>
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130509 Thunderbird/17.0.6
MIME-Version: 1.0
To: and#127.0.0.1
Subject: test
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
BODY[TEXT]<0> {7}
HELLO
)
A5 OK FETCH completed
A6 FETCH 1 (FLAGS)
* 1 FETCH (UID 50167 FLAGS (\Seen))
A6 OK FETCH completed
A7 FETCH 1 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS BODYSTRUCTURE)
* 1 FETCH (UID 50167 RFC822.SIZE 462 FLAGS (\Seen) INTERNALDATE "28-Jan-2014 15:08:59 +0100" ENVELOPE ("Tue, 28 Jan 2014 15:08:59 +0100" "test" (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) NIL NIL NIL "<52E7B9FB.2030409#127.0.0.1>") BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "UTF-8") NIL "test" "7bit" 7 2))
A7 OK FETCH completed
A8 STORE 1 +FLAGS (\Seen)
* 1 FETCH (FLAGS (\Seen) UID 50167)
A8 OK STORE completed
A9 CLOSE
A9 OK CLOSE completed
DEBUG: added an Authenticated connection -- size: 1
IMAP DEBUG: IMAPProtocol noop
A10 NOOP
A10 OK NOOP completed
DEBUG: connection available -- size: 1
A11 SELECT INBOX
* 1 EXISTS
* 0 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1378802425] current uidvalidity
* OK [UIDNEXT 50168] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
A11 OK [READ-WRITE] SELECT completed
A12 SEARCH UNSEEN ALL
* SEARCH
A12 OK Search completed
A13 CLOSE
A13 OK CLOSE completed
DEBUG: added an Authenticated connection -- size: 1
IMAP DEBUG: IMAPProtocol noop
A14 NOOP
A14 OK NOOP completed
A15 LOGOUT
* BYE Have a nice day
A15 OK Logout completed
DEBUG: IMAPStore connection dead
DEBUG: IMAPStore cleanup, force false
DEBUG: IMAPStore cleanup done
server is hMailServer, running on 127.0.0.1 with SSL and NONSSL ports enabled.
It appears, that I get only headers from mail, but not the body..
Thanks for any suggestions, have a great day!

I fixed the problem. The bug I have encountered was linked to the "Unable to load BODYSTRUCTURE" error (links below - read about IMAP complexity).
https://java.net/projects/javamail/pages/Exchange
http://www.oracle.com/technetwork/java/javamail/faq/index.html#imapserverbug
All that had to be done, to get proper body message, was creating copy of the message, using MimeMessage constructor.
// newMessages - it is array of messages from the mail inbox
// create copy of the message using MimeMessage constructor
MimeMessage message = new MimeMessage(newMessages[0]); // without it, the content was as in my first post
// get content from the mssage
Object messageContent = message.getContent();
// output it as a string
System.out.println(messageContent.toString());
The problem is solved, now I receive what I want and what I should. The only thing left is, why there was a difference between SSL and NONSSL message content? Maybe antivirus problem?
Thanks for help and interest.
Have a great day.

JavaMail intermittently throws FolderClosedExcpetion on IMAPMessage.loadEnvelope()

I have a java service that consumes emails from a Postfix mail server. It's handling about 1 email per second and processes the content of the emails.
However, very intermittently I get this exception:
javax.mail.FolderClosedException : * BYE [ALERT] Fatal error: Invalid argument - javax.mail.FolderClosedException: * BYE [ALERT] Fatal error: Invalid argument
at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1256)
at com.sun.mail.imap.IMAPMessage.getSubject(IMAPMessage.java:335)
at com.mailparser.data.MessageParser.parse(MessageParser.java:33)
If I exit the process and restart it again it will always throw the same exception on the same messages and thus they get stuck in the inbox.
here's a code snippet of where it's failing:
if (!message.getFolder().isOpen()) {
message.getFolder().open(Folder.READ_WRITE);
Logger.log("Tried to reopen folder"); //This is never hit
}
ReceivedEmail incomingMessage = new ReceivedEmail();
try {
incomingMessage.setSubject(message.getSubject()); // Exception happens here
[...]
} catch {
catch (FolderClosedException ex) {
message.getFolder().open(Folder.READ_WRITE);
Logger.log("Reopened", LogType.debug);
return null;
}
}
As you can see I'm checking to see if the folder is open before I try to get the subject, so I suspect that something else is causing this failure. Once it has thrown the expections for the stuck messages it carries on successfully using exactly the same folder and connection.
Does anyone have any idea on this?
Thanks!
Edit:
Debug info as requested by Bill:
DEBUG IMAP: LOGIN command trace suppressed
DEBUG IMAP: LOGIN command result: A0 OK LOGIN Ok.
A1 CAPABILITY
* CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION
A1 OK CAPABILITY completed
A2 LIST INBOX ""
* LIST (\Marked \HasChildren) "." ""
A2 OK LIST completed
A3 LIST "" INBOX.TEST
* LIST (\HasChildren) "." "INBOX.TEST"
A3 OK LIST completed
DEBUG: connection available -- size: 1
A4 SELECT INBOX.TEST
* FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent)
* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1351176418] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
A4 OK [READ-WRITE] Ok
DEBUG IMAP: IMAPProtocol noop
A7 NOOP
A7 OK NOOP completed
A8 FETCH 1 (FLAGS)
* 1 FETCH (FLAGS (\Answered \Seen))
A8 OK FETCH completed.
A9 FETCH 1 (ENVELOPE INTERNALDATE RFC822.SIZE)
* BYE [ALERT] Fatal error: Invalid argument