I am experiencing weird problems over IMAP in JavaMail.
I already know alot in terms of JavaMail and I also know where to look for Questions/Answers, yet I cannot find a solution for this issue.
I have IMAP channel, on which I can set SSL enabled or disabled.
When I create new email message (let's say, Subject: "TEST", Body: "HELLO") and send it from Thunderbird client, to be able to download it with my IMAP channel, things are getting weird.
On SSL ENABLED, everything works fine. (in my IMAP channel) I get message from email folder, after successfuly connecting to the store, then i getContent() from this message (it is ALWAYS String type, not MimeMultipart or Multipart), and output is "HELLO".
However, on SSL DISABLED, everything also works fine, but I receive different body. It looks like whole message parsed (look below)
message.getContent().toString(); using SSL
HELLO
message.getContent().toString(); NOT using SSL
Return-Path: and#127.0.0.1
Received: from [127.0.0.1] (localhost [127.0.0.1]) by PC ; Wed, 22 Jan 2014 13:23:17 +0100
Message-ID: <52DFB835.9020300#127.0.0.1>
Date: Wed, 22 Jan 2014 13:23:17 +0100
From: "and#127.0.0.1" <and#127.0.0.1>
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130509 Thunderbird/17.0.6
MIME-Version: 1.0
To: and#127.0.0.1
Subject: TEST
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Does anyone have any idea what's going on? Below are fragments of my code (to show you most important parts:
// properties used to create session
String protocol = useSSL ? "imaps" : imap";
properties.setProperty("mail.store.protocol", protocol);
properties.setProperty("mail." + protocol + ".host", ...);
properties.setProperty("mail." + protocol + ".user", ...);
properties.setProperty("mail." + protocol + ".port", ...);
properties.setProperty("mail." + protocol + ".password", ...);
// only when using ssl
properties.setProperty("mail.imaps.auth", "true");
properties.setProperty("mail.imaps.starttls.enable", "true");
properties.setProperty("mail.imaps.ssl.checkserveridentity", "true");
properties.setProperty("javax.net.ssl.trustStore", ...);
properties.setProperty("javax.net.ssl.trustStorePassword", ...);
// get store and other important things
Store store = this.session.getStore(protocol);
Folder folder = store.getFolder("INBOX");
folder.open(Folder.READ_WRITE);
Message[] newMessages = folder.getMessages();
for (Message newMessage : newMessages) {
Object o = newMessage.getContent();
System.out(o.toString()); // should write "HELLO", but instead it parses whole message (only on NON SSL MODE!)
}
// edit : added debug info
// edit 2 : debug info contains now full details with fetching folder, using { CONTENT_INFO, FLAGS, ENVELOPE }
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc]
DEBUG: mail.imap.fetchsize: 16384
DEBUG: protocolConnect returning false, host=127.0.0.1, user=and#127.0.0.1, password=<null>
* OK IMAPrev1
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk
A0 OK CAPABILITY completed
DEBUG: protocolConnect login, host=127.0.0.1, user=and#127.0.0.1, password=<non-null>
A1 LOGIN and#127.0.0.1 password
A1 OK LOGIN completed
DEBUG: connection available -- size: 1
A2 SELECT INBOX
* 1 EXISTS
* 0 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1378802425] current uidvalidity
* OK [UNSEEN 50167] unseen messages
* OK [UIDNEXT 50168] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
A2 OK [READ-WRITE] SELECT completed
A3 SEARCH UNSEEN ALL
* SEARCH 1
A3 OK Search completed
A4 FETCH 1 (BODYSTRUCTURE)
* 1 FETCH (UID 50167 BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "UTF-8") NIL "test" "7bit" 7 2))
A4 OK FETCH completed
A5 FETCH 1 (BODY[TEXT]<0.7>)
* 1 FETCH (UID 50167 ENVELOPE ("Tue, 28 Jan 2014 15:08:59 +0100" "test" (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) NIL NIL NIL "<52E7B9FB.2030409#127.0.0.1>") BODY[HEADER] {479}
Return-Path: and#127.0.0.1
Received: from [127.0.0.1] (localhost [127.0.0.1]) by A-PC ; Tue, 28 Jan 2014 15:08:59 +0100
Message-ID: <52E7B9FB.2030409#127.0.0.1>
Date: Tue, 28 Jan 2014 15:08:59 +0100
From: "and#127.0.0.1" <and#127.0.0.1>
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130509 Thunderbird/17.0.6
MIME-Version: 1.0
To: and#127.0.0.1
Subject: test
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
BODY[TEXT]<0> {7}
HELLO
)
A5 OK FETCH completed
A6 FETCH 1 (FLAGS)
* 1 FETCH (UID 50167 FLAGS (\Seen))
A6 OK FETCH completed
A7 FETCH 1 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS BODYSTRUCTURE)
* 1 FETCH (UID 50167 RFC822.SIZE 462 FLAGS (\Seen) INTERNALDATE "28-Jan-2014 15:08:59 +0100" ENVELOPE ("Tue, 28 Jan 2014 15:08:59 +0100" "test" (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) (("and#127.0.0.1" NIL "and" "127.0.0.1")) NIL NIL NIL "<52E7B9FB.2030409#127.0.0.1>") BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "UTF-8") NIL "test" "7bit" 7 2))
A7 OK FETCH completed
A8 STORE 1 +FLAGS (\Seen)
* 1 FETCH (FLAGS (\Seen) UID 50167)
A8 OK STORE completed
A9 CLOSE
A9 OK CLOSE completed
DEBUG: added an Authenticated connection -- size: 1
IMAP DEBUG: IMAPProtocol noop
A10 NOOP
A10 OK NOOP completed
DEBUG: connection available -- size: 1
A11 SELECT INBOX
* 1 EXISTS
* 0 RECENT
* FLAGS (\Deleted \Seen \Draft \Answered \Flagged)
* OK [UIDVALIDITY 1378802425] current uidvalidity
* OK [UIDNEXT 50168] next uid
* OK [PERMANENTFLAGS (\Deleted \Seen \Draft \Answered \Flagged)] limited
A11 OK [READ-WRITE] SELECT completed
A12 SEARCH UNSEEN ALL
* SEARCH
A12 OK Search completed
A13 CLOSE
A13 OK CLOSE completed
DEBUG: added an Authenticated connection -- size: 1
IMAP DEBUG: IMAPProtocol noop
A14 NOOP
A14 OK NOOP completed
A15 LOGOUT
* BYE Have a nice day
A15 OK Logout completed
DEBUG: IMAPStore connection dead
DEBUG: IMAPStore cleanup, force false
DEBUG: IMAPStore cleanup done
server is hMailServer, running on 127.0.0.1 with SSL and NONSSL ports enabled.
It appears, that I get only headers from mail, but not the body..
Thanks for any suggestions, have a great day!
I fixed the problem. The bug I have encountered was linked to the "Unable to load BODYSTRUCTURE" error (links below - read about IMAP complexity).
https://java.net/projects/javamail/pages/Exchange
http://www.oracle.com/technetwork/java/javamail/faq/index.html#imapserverbug
All that had to be done, to get proper body message, was creating copy of the message, using MimeMessage constructor.
// newMessages - it is array of messages from the mail inbox
// create copy of the message using MimeMessage constructor
MimeMessage message = new MimeMessage(newMessages[0]); // without it, the content was as in my first post
// get content from the mssage
Object messageContent = message.getContent();
// output it as a string
System.out.println(messageContent.toString());
The problem is solved, now I receive what I want and what I should. The only thing left is, why there was a difference between SSL and NONSSL message content? Maybe antivirus problem?
Thanks for help and interest.
Have a great day.
Related
I'm using below code in a Springboot application:
#Bean
public IntegrationFlow mailListener() {
return IntegrationFlows.from(Mail.imapInboundAdapter(receiver()), e -> e.poller(Pollers.fixedRate(60000).maxMessagesPerPoll(-1)))
.<Message>handle(message -> logMail(message)).get();
}
private org.springframework.messaging.Message<?> logMail(org.springframework.messaging.Message<?> message) {
System.out.println("received a mail********** !");
// System.out.println(message.getPayload());
// process message
return message;
}
#Bean
public ImapMailReceiver receiver() {
ImapMailReceiver receiver = new ImapMailReceiver(
"imaps://username:pwd#mail.company.com/INBOX");
receiver.setShouldMarkMessagesAsRead(true);
receiver.setJavaMailProperties(javaMailProperties());
return receiver;
}
private Properties javaMailProperties() {
Properties javaMailProperties = new Properties();
/*
* javaMailProperties.setProperty("mail.imap.socketFactory.class",
* "javax.net.ssl.SSLSocketFactory");
* javaMailProperties.setProperty("mail.imap.socketFactory.fallback","false");
* javaMailProperties.setProperty("mail.store.protocol","imaps");
*/
// javaMailProperties.setProperty("mail.debug","true");
return javaMailProperties;
}
This springboot applications has been deployed in 2 different servers (dev and stage). Whenever this email ID receives a new mail it will be received by either application that's running on dev server or stage and not both. How do I retrieve new mails in all instances of my applications running on different servers ?
UPDATE : debug logs
DEBUG IMAPS: IMAPProtocol noop
A20 NOOP
A20 OK NOOP completed.
A21 LIST "" INBOX
* LIST (\Marked \HasNoChildren) "/" INBOX
A21 OK LIST completed.
2020-06-05 03:06:39.003 DEBUG 15568 --- [ scheduling-1] o.s.integration.mail.ImapMailReceiver : opening folder [imaps://user.name%40company.com#mail.ad.company.com/INBOX]
DEBUG IMAPS: connection available -- size: 1
A22 SELECT INBOX
* 1438 EXISTS
* 1 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)] Permanent flags
* OK [UNSEEN 228] Is the first unseen message
* OK [UIDVALIDITY 2278797] UIDVALIDITY value
* OK [UIDNEXT 2855] The next unique identifier value
A22 OK [READ-WRITE] SELECT completed.
2020-06-05 03:06:39.752 INFO 15568 --- [ scheduling-1] o.s.integration.mail.ImapMailReceiver : attempting to receive mail from folder [INBOX]
2020-06-05 03:06:39.752 DEBUG 15568 --- [ scheduling-1] o.s.integration.mail.ImapMailReceiver : This email server does not support RECENT or USER flags. System flag 'Flag.FLAGGED' will be used to prevent duplicates during email fetch.
A23 SEARCH NOT (ANSWERED) NOT (DELETED) NOT (SEEN) NOT (FLAGGED) ALL
* SEARCH 1438
A23 OK SEARCH completed.
2020-06-05 03:06:40.488 DEBUG 15568 --- [ scheduling-1] o.s.integration.mail.ImapMailReceiver : found 1 new messages
A24 FETCH 1438 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS BODYSTRUCTURE)
* 1438 FETCH (ENVELOPE ("Thu, 4 Jun 2020 17:36:21 -0400" "test message" (("sender,name (Company)" NIL "Sender.name" "company.com")) NIL NIL (("~receiver name" NIL "user.name" "company.com")) NIL NIL "<KL1P101MB0152E33B47someid#KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>" "<KL1P101MB0152E33B4someid#KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>") INTERNALDATE "04-Jun-2020 17:36:28 -0400" RFC822.SIZE 2638 FLAGS (\Recent) BODYSTRUCTURE ("text" "html" ("charset" "us-ascii") NIL NIL "7BIT" 0 0 NIL NIL "en-US" NIL))
A24 OK FETCH completed.
2020-06-05 03:06:41.536 DEBUG 15568 --- [ scheduling-1] o.s.integration.mail.ImapMailReceiver : Received 1 messages
2020-06-05 03:06:41.536 DEBUG 15568 --- [ scheduling-1] o.s.integration.mail.ImapMailReceiver : USER flags are not supported by this mail server. Flagging message with system flag
A25 STORE 1438 +FLAGS (\Flagged)
* 1438 FETCH (FLAGS (\Flagged \Recent))
A25 OK STORE completed.
A26 FETCH 1438 (BODY.PEEK[]<0.16384>)
* 1438 FETCH (BODY[]<0> {3198}
Received: from ALPMBHT04.e2k.ad.company.com (3.159.19.197) by
ALPURAPA30.e2k.ad.company.com (3.159.16.198) with Microsoft SMTP Server (TLS) id
14.3.487.0; Thu, 4 Jun 2020 17:36:28 -0400
Received: from APC01-HK2-obe.outbound.protection.outlook.com (10.38.143.172)
by mail.o365.company.com (3.159.19.197) with Microsoft SMTP Server (TLS) id
14.3.487.0; Thu, 4 Jun 2020 17:36:28 -0400
Received: from KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM (2603:1096:808:3::27) by
KL1P101MB0151.NAMP101.PROD.OUTLOOK.COM (2603:1096:808:3::26) with Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.3045.25; Thu, 4 Jun 2020 21:36:21 +0000
Received: from KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM ([129.75.125.219]) by
KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM ([129.75.125.219]) with mapi id
15.20.3045.029; Thu, 4 Jun 2020 21:36:21 +0000
From: "sender,name (Company)" <Sender.name#company.com>
To: ~receiver name <user.name#company.com>
Subject: test message
Thread-Topic: test message
Thread-Index: AdY6s3z4MSq0ceMSRqCEtXhoAxXllAABK3Aw
Date: Thu, 4 Jun 2020 17:36:21 -0400
Message-ID:
<KL1P101MB0152E33B47someid#KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>
References:
<KL1P101MB0152E33B47someid#KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>
In-Reply-To:
<KL1P101MB0152E33B47someid#KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>
Accept-Language: en-IN, en-US
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 04
X-MS-Exchange-Organization-AuthSource: KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM
X-MS-Has-Attach:
X-MS-Exchange-Organization-SCL: -1
X-Message-Flag: Follow up
X-MS-TNEF-Correlator:
Content-Type: text/html; charset="us-ascii"
MIME-Version: 1.0
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
#font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
#font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
#page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-IN" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Test content<o:p></o:p></p>
</div>
</body>
</html>
)
A26 OK FETCH completed.
A27 EXAMINE INBOX
* 1438 EXISTS
* 0 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS ()] Permanent flags
* OK [UNSEEN 228] Is the first unseen message
* OK [UIDVALIDITY 2278797] UIDVALIDITY value
* OK [UIDNEXT 2856] The next unique identifier value
A27 OK [READ-ONLY] EXAMINE completed.
A28 CLOSE
A28 OK CLOSE completed.
DEBUG IMAPS: added an Authenticated connection -- size: 1
2020-06-05 03:06:44.788 DEBUG 15568 --- [ scheduling-1] o.s.i.mail.MailReceivingMessageSource : received mail message [org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage#3904c3]
2020-06-05 03:06:44.796 DEBUG 15568 --- [ scheduling-1] o.s.i.e.SourcePollingChannelAdapter : Poll resulted in Message: GenericMessage [payload=org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage#3904c3, headers={id=7c50d96e-36da-b55f-9862-2215117ef304, timestamp=1591306604796}]
2020-06-05 03:06:44.798 DEBUG 15568 --- [ scheduling-1] o.s.integration.channel.DirectChannel : preSend on channel 'bean 'mailListener.channel#0'; defined in: 'class path resource [com//mail/MailPoller.class]'; from source: 'bean method mailListener'', message: GenericMessage [payload=org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage#3904c3, headers={id=7c50d96e-36da-b55f-9862-2215117ef304, timestamp=1591306604796}]
got a new mail
2020-06-05 03:06:44.799 DEBUG 15568 --- [ scheduling-1] o.s.integration.channel.DirectChannel : postSend (sent=true) on channel 'bean 'mailListener.channel#0'; defined in: 'class path resource [com/mail/MailPoller.class]'; from source: 'bean method mailListener'', message: GenericMessage [payload=org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage#3904c3, headers={id=7c50d96e-36da-b55f-9862-2215117ef304, timestamp=1591306604796}]
DEBUG IMAPS: IMAPProtocol noop
It's because of receiver.setShouldMarkMessagesAsRead(true);.
If you change it to false, I think both will get it; the problem is then you will have to manually mark it as SEEN somehow to prevent re-fetching it next time the apps start.
You should also be able to use a custom SearchTermStrategy. The default strategy includes
if (supportedFlags.contains(Flags.Flag.SEEN)) {
NotTerm notSeen = new NotTerm(new FlagTerm(new Flags(Flags.Flag.SEEN), true));
if (searchTerm == null) {
searchTerm = notSeen;
}
else {
searchTerm = new AndTerm(searchTerm, notSeen);
}
}
With a custom search term, you should also be able to use a different userFlag in each instance; enabling each to mark the message as read that way (as long as your IMAP server supports Flag.USER.
See ImapMailReceiver.DefaultSearchTermStrategy for how the default one works.
I have a email processor java application which connects outlooks and lists folder messages and then if it decide it downloads email. I have the following problem with some emails.
Listing with following code:
Session session = Session.getInstance(props, new myAuthenticator());
store = session.getStore(PROTOCOL);
store.connect(HOST, USERNAME, PASSWORD);
inbox_unknown = store.getDefaultFolder().getFolder("INBOX_UNKNOWN");
inbox_unknown.open(Folder.READ_ONLY);
messages = inbox_unknown.getMessages();
for(Message m : messages){
Enumeration headers = m.getAllHeaders();
while(headers.hasMoreElements()){
Header element = (Header)headers.nextElement();
if ("Message-ID".equals(element.getName()))
System.err.println(element.getValue());
}
}
and the result is :
<CAFSBYLHvG9d=MkE2Gf+q_EgoaAN8yRVHHsg5emO0w92GV6-7LA#mail.gmail.com>
<CAFSBYLGs-w4=JrgrYh_6kpiEc9NhjCsc76gXVvLWyW-=QN+kMw#mail.gmail.com>
However When I search with these message id with following code :
messages = inbox_unknown.search(new MessageIDTerm("<CAFSBYLGs-w4=JrgrYh_6kpiEc9NhjCsc76gXVvLWyW-=QN+kMw#mail.gmail.com>"));//not found
messages = inbox_unknown.search(new MessageIDTerm("<CAFSBYLHvG9d=MkE2Gf+q_EgoaAN8yRVHHsg5emO0w92GV6-7LA#mail.gmail.com>"));//found
Although message with id <CAFSBYLHvG9d=MkE2Gf+q_EgoaAN8yRVHHsg5emO0w92GV6-7LA#mail.gmail.com> is listed, when i search this single item i get empty return array. For other message id it returns the message object as expected. Do you have any idea what can be the problem ?
PS:This mails from same person, and this is just for example, there are many similar mails which are not found although they can be listed.
Note: using java7, java.mail-1.4 api, ms outlook server,IMAP4 as protocol.
Note2: Added java debug output
DEBUG: setDebug: JavaMail version 1.5.5
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle]
DEBUG IMAP: mail.imap.fetchsize: 16384
DEBUG IMAP: mail.imap.ignorebodystructuresize: false
DEBUG IMAP: mail.imap.statuscachetimeout: 1000
DEBUG IMAP: mail.imap.appendbuffersize: -1
DEBUG IMAP: mail.imap.minidletime: 10
DEBUG IMAP: closeFoldersOnStoreFailure
DEBUG IMAP: trying to connect to host "outlook.abck.com.tr", port 143, isSSL false
* OK The Microsoft Exchange IMAP4 service is ready.
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAP: AUTH: NTLM
DEBUG IMAP: AUTH: GSSAPI
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: protocolConnect login, host=outlook.abck.com.tr, user=usera, password=<non-null>
DEBUG IMAP: AUTHENTICATE PLAIN command trace suppressed
DEBUG IMAP: AUTHENTICATE PLAIN command result: A1 OK AUTHENTICATE completed.
A2 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A2 OK CAPABILITY completed.
DEBUG IMAP: AUTH: NTLM
DEBUG IMAP: AUTH: GSSAPI
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: connection available -- size: 1
A3 EXAMINE Test
* 2 EXISTS
* 2 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS ()] Permanent flags
* OK [UIDVALIDITY 268114] UIDVALIDITY value
* OK [UIDNEXT 4] The next unique identifier value
A3 OK [READ-ONLY] EXAMINE completed.
A4 SEARCH HEADER Message-ID <CAFSBYLGs-w4=JrgrYh_6kpiEc9NhjCsc76gXVvLWyW-=QN+kMw#mail.gmail.com> ALL
* SEARCH
A4 OK SEARCH completed.
A5 SEARCH HEADER Message-ID <CAFSBYLHvG9d=MkE2Gf+q_EgoaAN8yRVHHsg5emO0w92GV6-7LA#mail.gmail.com> ALL
* SEARCH 2
A5 OK SEARCH completed.
A6 FETCH 1 (BODY.PEEK[HEADER])
* 1 FETCH (BODY[HEADER] {1403}
MIME-Version: 1.0
Received: from mail01.abck.com.tr (62.108.64.49) by
EXCPRDHCS02.abck.abc (10.81.42.19) with Microsoft SMTP Server
(TLS) id 14.3.195.1; Tue, 17 May 2016 14:31:39 +0300
Received: from mail-yw0-f170.google.com ([209.85.161.170]) by
mail01.abck.com.tr with ESMTP/TLS/AES128-GCM-SHA256; 17 May 2016
14:31:31 +0300
Received: by mail-yw0-f170.google.com with SMTP id x189so11988152ywe.3;
Tue, 17 May 2016 04:31:31 -0700 (PDT)
Received: by 10.37.14.212 with HTTP; Tue, 17 May 2016 04:31:30 -0700 (PDT)
Subject: =?utf-8?B?VEFMxLBNQVQ=?=
Thread-Topic: =?utf-8?B?VEFMxLBNQVQ=?=
Thread-Index: AQHRsC+uNbCd970/KU2ZmbAy2JPC2w==
Date: Tue, 17 May 2016 14:31:30 +0300
Message-ID:
<CAFSBYLGs-w4=JrgrYh_6kpiEc9NhjCsc76gXVvLWyW-=QN+kMw#mail.gmail.com>
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-AuthSource: EXCPRDHCS02.abck.abc
X-MS-Has-Attach: yes
X-MS-Exchange-Organization-SCL: 0
X-MS-TNEF-Correlator:
Content-Type: multipart/mixed;
boundary="_003_CAFSBYLGsw4JrgrYh6kpiEc9NhjCsc76gXVvLWyWQNkMwmailgmailc_"
)
A6 OK FETCH completed.
A7 FETCH 2 (BODY.PEEK[HEADER])
* 2 FETCH (BODY[HEADER] {4900}
MIME-Version: 1.0
Received: from mail01.abck.com.tr (62.108.64.49) by
EXCPRDHCS01.abck.abc (10.81.42.18) with Microsoft SMTP Server
(TLS) id 14.3.279.2; Wed, 25 May 2016 10:30:08 +0300
IronPort-PHdr: =?us-ascii?q?9a23=3AVhJs6B+zz3F7Iv9uRHKM819IXTAuvvDOBiVQ1KB7?=
=?us-ascii?q?0eIcTK2v8tzYMVDF4r011RmSDdSdt6gP0rCN+4nbGkU+or+5+EgYd5JNUxJXwe?=
=?us-ascii?q?43pCcHRPC/NEvgMfTxZDY7FskRHHVs/nW8LFQHUJ2mPw6anHS+4HYoFwnlMkIt?=
=?us-ascii?q?f6KuSt+U0578jrrps7ToICx2xxOFKYtoKxu3qQiD/uI3uqBFbpgL9x3Sv3FTcP?=
=?us-ascii?q?5Xz247bXianhL7+9vitMU7q3cYk7sb+sVBSaT3ebgjBfwdVWx+cjN92Mq+/zTH?=
=?us-ascii?q?TAXH3nYcVmQImxwAQy3I4xiydZr3ribg/tFg0iSBPIjVSro1ER6r6aRiQx7siS?=
=?us-ascii?q?gGdnRxpGzTi8I2lK9bpBO/rhpXzY/SaYeOPvN4feXWetZcWXAXGo4beiBISqqg?=
=?us-ascii?q?ZpBHT9EcN+sQl4DvrkEKpB63TSmtFubi1ndti2X10L87kq5pRQzK1UopA9sVmH?=
=?us-ascii?q?rVqtzzL60bV+Pzx67NiynZObcexT756Y3BbjgvoPeFW6l6eMzSj0IoEkXZlAa+?=
=?us-ascii?q?s4vgag248sVFnEW9QKI0Tv+jhmAqsQRsqyKHycIli42PjYUQnAOXvR5lyZo4cI?=
=?us-ascii?q?XrAHVwZsSpRd4J73mX?=
X-Cloudmark-SP-Filtered: true
X-Cloudmark-SP-Result: =?us-ascii?q?v=3D2=2E1_cv=3DF5f3DvVN_c=3D1_sm=3D1_tr?=
=?us-ascii?q?=3D0_a=3Dyrkiwgmsf1kA=3A10_a=3DxWK-KtZqUfIA=3A10?=
=?us-ascii?q?_a=3De186Yzns-9f1=5FKZu-cIA=3A9_a=3DQEXdDO2ut3YA=3A10_a=3DOKp-?=
=?us-ascii?q?Gz0vqPG3m8aTRrIA=3A9?=
=?us-ascii?q?_a=3DKQqxNPgzF0kA=3A10?=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0HiAwB/U0VXf7OhVdFbhQoGpn6HY4NNg?=
=?us-ascii?q?xSCJYQTBCRxB4YkBzwQAQEBAQEBAQECDwEBCQsLCSEbFIIuCjkQVQIrTA0RHQE?=
=?us-ascii?q?bHgMSAwUBAgUSAQELGAIEFQEKAREBBQEREQEYHIdyAQMXlVGPQoExPjGKBoE1g?=
=?us-ascii?q?WqCWAWIKwoZJw1Sg1cMFwYCBhCSI4JZBY5VhwWCXYMrgWmJDIFTAY1Ijg0SHoE?=
=?us-ascii?q?PN4QbOTKKBwEBAQ?=
X-IPAS-Result: =?us-ascii?q?A0HiAwB/U0VXf7OhVdFbhQoGpn6HY4NNgxSCJYQTBCRxB4Y?=
=?us-ascii?q?kBzwQAQEBAQEBAQECDwEBCQsLCSEbFIIuCjkQVQIrTA0RHQEbHgMSAwUBAgUSA?=
=?us-ascii?q?QELGAIEFQEKAREBBQEREQEYHIdyAQMXlVGPQoExPjGKBoE1gWqCWAWIKwoZJw1?=
=?us-ascii?q?Sg1cMFwYCBhCSI4JZBY5VhwWCXYMrgWmJDIFTAY1Ijg0SHoEPN4QbOTKKBwEBA?=
=?us-ascii?q?Q?=
Received: from mail-yw0-f179.google.com ([209.85.161.179]) by
mail01.abck.com.tr with ESMTP/TLS/AES128-GCM-SHA256; 25 May 2016
10:29:57 +0300
Received: by mail-yw0-f179.google.com with SMTP id o16so39498708ywd.2;
Wed, 25 May 2016 00:29:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:date:message-id:subject:from:to;
bh=w8xSZyobRcFMKx+IrtarhnHXAm9n7xJEYfaDMmptcKI=;
b=WN8u7uLAyNhcAoB+oQ5xbsKdpV4z4BGWD6CrryDyJqq7tosj82QH3d3MQWHaqf2mBa
AqlmCBq5w3ntokCoKrQlzdQNGecJ7AvIrc/CPFkYLeT3MK1kcY8gzDANJC/Pm5NO75td
S/mBW9goEL1gKFwpd/DahOzIwKbT5UEz9hthcTU5JIPwJR305/boxMIP3zcu1M+WPVPY
RhnojBbe41iCfqemB11IIvFeyRMqck6x7dDnWnejriEP2Gp8x4s2FYLz5rysEloNe0wU
qKmmNXtsIStwJhdk8jjByMW/UsK+94t3+jr+rzbTyYndh/Hs3SbZsmDQipBG8+SHxRQu
e/TQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:mime-version:date:message-id:subject:from:to;
bh=w8xSZyobRcFMKx+IrtarhnHXAm9n7xJEYfaDMmptcKI=;
b=iXyugXgd1ER8sfK9PJXcq88MO8z2/VKl31XNf/Bp2E1BvC42ZxLsAGjeHKL6KqqzFb
CNdHhBLnyfAnGoaa9pIK+qoC2HT69e323Rmym3w6HxQr1ot396xThXXxL22ZManrpICu
RAPegTlsmXQ1VBqhYQjgX8NwX6FtoqtzBW9tSOnl+HTfdyv3CNNf0mAysA7jyCHwQy6j
Nc+JK/uUZjccYkic672MiDSUa4FfwJSx0L3nyWI/CZMVDaNVW3mtp04cpgvtshLkloLn
61bz3dlGNL1g3gIVwNFiY8SLGxFdPzRc7xJ9VJEgXoA1h2OO2QVqyEUUyivhzehbgvL5
Bhtw==
X-Gm-Message-State: ALyK8tLPd9mwwszKuBwk9ndltkf3tyZ91virPLkoGP8PigQLffqE7vB7NVGiauqhPC9vcXsZnsU/mzB4zSTZbg==
X-Received: by 10.13.217.151 with SMTP id b145mr1363469ywe.53.1464161395972;
Wed, 25 May 2016 00:29:55 -0700 (PDT)
Received: by 10.37.14.212 with HTTP; Wed, 25 May 2016 00:29:55 -0700 (PDT)
Date: Wed, 25 May 2016 10:29:55 +0300
Message-ID: <CAFSBYLHvG9d=MkE2Gf+q_EgoaAN8yRVHHsg5emO0w92GV6-7LA#mail.gmail.com>
Subject: =?UTF-8?B?QsOcxZ5SQSBFVMSwIE1VU1RBRkEgRVTEsCBLSURFTSBUQVpNxLBOQVRJIMOWREVNRSBUQQ==?=
=?UTF-8?B?TMSwTUFUSQ==?=
Content-Type: multipart/mixed; boundary="001a114fa43ee6ce4a0533a5a3c5"
Return-Path: abc#gmail.com
X-MS-Exchange-Organization-AuthSource: EXCPRDHCS01.abck.abc
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-AVStamp-Mailbox: SMEXw]nP;1254500;0;This mail has
been scanned by Trend Micro ScanMail for Microsoft Exchange;
X-MS-Exchange-Organization-SCL: 0
X-PP-Proceessed: 162a0e36-1ac8-484f-b83f-d5c2dc8a3c0b
)
A7 OK FETCH completed.
A8 CLOSE
A8 OK CLOSE completed.
DEBUG IMAP: added an Authenticated connection -- size: 1
A9 LOGOUT
* BYE Microsoft Exchange Server 2010 IMAP4 server signing off.
A9 OK LOGOUT completed.
DEBUG IMAP: IMAPStore connection dead
DEBUG IMAP: IMAPStore cleanup, force false
DEBUG IMAP: IMAPStore cleanup done
The http file example under test:
https://github.com/netty/netty/tree/4.1/example/src/main/java/io/netty/example/http/file
I compiled the above example with netty-4.1.0.beta8.
My test result:
$ ab -k -n 2 -c 1 -v 6 http://127.0.0.1:8080/test.sh
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)...INFO: POST header ==
---
GET /test.sh HTTP/1.0
Connection: Keep-Alive
Host: 127.0.0.1:8080
User-Agent: ApacheBench/2.3
Accept: */*
---
LOG: header received:
HTTP/1.1 200 OK
content-length: 462
content-type: application/octet-stream
date: Fri, 26 Feb 2016 06:34:52 GMT
expires: Fri, 26 Feb 2016 06:35:52 GMT
cache-control: private, max-age=60
last-modified: Fri, 19 Feb 2016 02:35:40 GMT
connection: keep-alive
LOG: Response code = 200
LOG: header received:
**MY TEST.SH CONTENT**
WARNING: Response code not 2xx (500)
apr_poll: The timeout specified has expired (70007)
Total of 1 requests completed
It seemed that my test.sh content is considered to be the second request's headers.
My further test:
$ ab -k -n 1 -c 1 -v 6 http://127.0.0.1:8080/test.sh
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)...INFO: POST header ==
---
GET /test.sh HTTP/1.0
Connection: Keep-Alive
Host: 127.0.0.1:8080
User-Agent: ApacheBench/2.3
Accept: */*
---
LOG: header received:
HTTP/1.1 200 OK
content-length: 462
content-type: application/octet-stream
date: Fri, 26 Feb 2016 06:39:02 GMT
expires: Fri, 26 Feb 2016 06:40:02 GMT
cache-control: private, max-age=60
last-modified: Fri, 19 Feb 2016 02:35:40 GMT
connection: keep-alive
LOG: Response code = 200
..done
Server Software:
Server Hostname: 127.0.0.1
Server Port: 8080
Document Path: /test.sh
Document Length: 0 bytes
Concurrency Level: 1
Time taken for tests: 0.005 seconds
Complete requests: 1
Failed requests: 0
Write errors: 0
Keep-Alive requests: 1
Total transferred: 263 bytes
HTML transferred: 0 bytes
Requests per second: 209.25 [#/sec] (mean)
Time per request: 4.779 [ms] (mean)
Time per request: 4.779 [ms] (mean, across all concurrent requests)
Transfer rate: 53.74 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 5 5 0.0 5 5
Waiting: 5 5 0.0 5 5
Total: 5 5 0.0 5 5
You can see the document length is 0 byte.
I run similar commands against www.google.com and it works fine.
Could you please help on this? Thanks in advance.
This is a bug in ApacheBench.
ApacheBench does not follow the http specifications fully, and assumes headers have a certain capitalization. Because the headers returned by the netty application doesn't have the character case ApacheBench expects, it assumes a default of 0 bytes for the content length.
Http specification 4.2:
HTTP header fields, which include general-header (section 4.5),
request-header (section 5.3), response-header (section 6.2), and
entity-header (section 7.1) fields, follow the same generic format as
that given in Section 3.1 of RFC 822 [9]. Each header field consists
of a name followed by a colon (":") and the field value. Field names
are case-insensitive. The field value MAY be preceded by any amount of
LWS, though a single SP is preferred. Header fields can be extended
over multiple lines by preceding each extra line with at least one SP
or HT. Applications ought to follow "common form", where one is known
or indicated, when generating HTTP constructs, since there might exist
some implementations that fail to accept anything beyond the common forms.
Because the keep-alive switch of ApacheBench forces it to assume the server supports keep alive, it doesn't detect a "missing" header for the keep alive packet.
This problem if however something that should be solved in ApacheBench, as it is a bug in that application that causes it to miss the proper content-length.
I'm writing a program that connects to gmail and checks the number of messages in the inbox. If I call getMessageCount() on the unopened Folder, it returns 554, but after opening it, it returns 541. Why the difference?
By the way, I'm ultimately trying to check for deleted messages (that have not been expunged) and recover them if possible.
Edit: some example code:
Session session = Session.getDefaultInstance(System.getProperties());
Store store = session.getStore("imaps");
store.connect("imap.googlemail.com", 993, email, password);
Folder inbox = store.getFolder("INBOX");
System.out.println(inbox.getMessageCount());
inbox.open(Folder.READ_ONLY);
System.out.println(inbox.getMessageCount());
Edit: protocol trace:
(In the meantime I got some more emails so the numbers are different from above)
DEBUG: setDebug: JavaMail version 1.5.2
DEBUG: getProvider() returning javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle]
DEBUG IMAPS: mail.imap.fetchsize: 16384
DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
DEBUG IMAPS: mail.imap.appendbuffersize: -1
DEBUG IMAPS: mail.imap.minidletime: 10
DEBUG IMAPS: trying to connect to host "imap.googlemail.com", port 993, isSSL true
* OK Gimap ready for requests from <ip> vt17mb17567619iec
A0 CAPABILITY
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN
A0 OK Thats all she wrote! vt17mb17567619iec
DEBUG IMAPS: AUTH: XOAUTH
DEBUG IMAPS: AUTH: XOAUTH2
DEBUG IMAPS: AUTH: PLAIN
DEBUG IMAPS: AUTH: PLAIN-CLIENTTOKEN
DEBUG IMAPS: protocolConnect login, host=imap.googlemail.com, user=<email>, password=<non-null>
DEBUG IMAPS: AUTHENTICATE PLAIN command trace suppressed
DEBUG IMAPS: AUTHENTICATE PLAIN command result: A1 OK <email> authenticated (Success)
A2 CAPABILITY
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT
A2 OK Success
A3 LIST "" INBOX
* LIST (\HasChildren) "/" "INBOX"
A3 OK Success
A4 STATUS INBOX (MESSAGES RECENT UNSEEN UIDNEXT UIDVALIDITY)
* STATUS "INBOX" (MESSAGES 585 RECENT 0 UIDNEXT 3389 UIDVALIDITY 2 UNSEEN 0)
A4 OK Success
585
DEBUG IMAPS: connection available -- size: 1
A5 EXAMINE INBOX
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $Phishing $Forwarded NonJunk $NotPhishing Junk)
* OK [PERMANENTFLAGS ()] Flags permitted.
* OK [UIDVALIDITY 2] UIDs valid.
* 548 EXISTS
* 0 RECENT
* OK [UIDNEXT 3389] Predicted next UID.
* OK [HIGHESTMODSEQ 687472]
A5 OK [READ-ONLY] INBOX selected. (Success)
548
I have a java service that consumes emails from a Postfix mail server. It's handling about 1 email per second and processes the content of the emails.
However, very intermittently I get this exception:
javax.mail.FolderClosedException : * BYE [ALERT] Fatal error: Invalid argument - javax.mail.FolderClosedException: * BYE [ALERT] Fatal error: Invalid argument
at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1256)
at com.sun.mail.imap.IMAPMessage.getSubject(IMAPMessage.java:335)
at com.mailparser.data.MessageParser.parse(MessageParser.java:33)
If I exit the process and restart it again it will always throw the same exception on the same messages and thus they get stuck in the inbox.
here's a code snippet of where it's failing:
if (!message.getFolder().isOpen()) {
message.getFolder().open(Folder.READ_WRITE);
Logger.log("Tried to reopen folder"); //This is never hit
}
ReceivedEmail incomingMessage = new ReceivedEmail();
try {
incomingMessage.setSubject(message.getSubject()); // Exception happens here
[...]
} catch {
catch (FolderClosedException ex) {
message.getFolder().open(Folder.READ_WRITE);
Logger.log("Reopened", LogType.debug);
return null;
}
}
As you can see I'm checking to see if the folder is open before I try to get the subject, so I suspect that something else is causing this failure. Once it has thrown the expections for the stuck messages it carries on successfully using exactly the same folder and connection.
Does anyone have any idea on this?
Thanks!
Edit:
Debug info as requested by Bill:
DEBUG IMAP: LOGIN command trace suppressed
DEBUG IMAP: LOGIN command result: A0 OK LOGIN Ok.
A1 CAPABILITY
* CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION
A1 OK CAPABILITY completed
A2 LIST INBOX ""
* LIST (\Marked \HasChildren) "." ""
A2 OK LIST completed
A3 LIST "" INBOX.TEST
* LIST (\HasChildren) "." "INBOX.TEST"
A3 OK LIST completed
DEBUG: connection available -- size: 1
A4 SELECT INBOX.TEST
* FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent)
* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1351176418] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
A4 OK [READ-WRITE] Ok
DEBUG IMAP: IMAPProtocol noop
A7 NOOP
A7 OK NOOP completed
A8 FETCH 1 (FLAGS)
* 1 FETCH (FLAGS (\Answered \Seen))
A8 OK FETCH completed.
A9 FETCH 1 (ENVELOPE INTERNALDATE RFC822.SIZE)
* BYE [ALERT] Fatal error: Invalid argument