We Keep Coding

Unable to get a handle on the body of a multipart/form-data POST request in XPage ServiceBean

I have an xpage based on a bootstrap framework which uses jQuery $post to submit forms contained within the page. These get posted using multipart/form-data content-type so I have no control over that. I have set up a REST.xsp containing a RESTService which calls a java ServiceBean. The latest method I have tried is as follows:
private void doPost(HttpServletRequest request, HttpServletResponse response) throws IOException {
// TODO Auto-generated method stub
try{
ServletFileUpload upload = new ServletFileUpload();
FileItemIterator iterator = upload.getItemIterator(request);
System.out.println("test");
while (iterator.hasNext()) {
System.out.println("test2");
FileItemStream item = iterator.next();
InputStream stream = item.openStream();
if (item.isFormField()) {
byte[] str = new byte[stream.available()];
stream.read(str);
String pFieldValue = new String(str,"UTF8");
System.out.println("text Value : "+pFieldValue);
if(item.getFieldName().equals("textBoxFormElement")){
System.out.println("text Value : "+pFieldValue);
}
}
}
}catch (Exception e) {
e.printStackTrace();
}
}
but I just can't seem to get the server to pick up the POSTed form data. I can see the "test" on the console but it's not hitting "test2".
I have tried every suggestion I could find including:
try {
InputStream inputStream = request.getInputStream();
Reader inputStreamReader = new InputStreamReader(inputStream);
String allChars = "";
System.out.println("test");
int data = inputStreamReader.read();
while(data != -1){
System.out.println("test2");
char theChar = (char) data;
System.out.println(theChar);
allChars = allChars + theChar;
data = inputStreamReader.read();
}
inputStreamReader.close();
}
catch (IOException e) {
System.out.println("No body data readable: " + e.getMessage() );
}
(still no "test2")...and:
String requestBody = extractPostRequestBody(request);
System.out.println(requestBody);
private static String extractPostRequestBody(HttpServletRequest request) {
Scanner s = null;
try {
s = new Scanner(request.getInputStream(), "UTF-8").useDelimiter("\\A");
} catch (IOException e) {
e.printStackTrace();
}
return s.hasNext() ? s.next() : "";
}
(nothing printed to console).
I have also tried using a REST client to post some multipart form data values to the URL but again nothing is returned. However if I try POSTing some json using application/json content-type and use the last method shown above I can see the json getting printed to the console.
Is there maybe some issue with using the extLib REST control and a java servicebean with multipart POSTs? I am aware that you can only read the request stream once so that previous use of eg request.properties will render the stream empty thereafter but this is not the case and presumably the json wouldn't display either if it were.
Many thanks

If it's multipart/formdata that's send with the POST request to the ExtLib Rest control, you can get a handle to the submitted data like this:
for (Object e : req.getParameterMap().entrySet()) {
Map.Entry entry = (Map.Entry) e;
System.out.println("got " + entry.getKey() + "/" + entry.getValue().getClass().getCanonicalName());
}
The entries in the map can either be the form fields or a file that's uploaded.

multipart/alternative getContent() throws IOException: No content

i have a problem with a code of mine. I try to get the text of an email from my Mailserver. After I retrieve the mail, I analyse the message and try to get the bodypart with the type "text/plain" and/or "text/html".
I use following method to split up a MimeMultipart:
private String getTextFromMimeMultipart(MimeMultipart mimeMultipart) throws MessagingException, IOException{
String result = "";
int count = mimeMultipart.getCount();
BodyPart bodypart;
String html;
System.out.println("Number of parts: "+count);
for (int i = 0; i < count; i++) {
bodypart=mimeMultipart.getBodyPart(i);
System.out.println("Type of part "+(i+1)+" "+bodypart.getContentType());
System.out.println("Class: "+bodypart.getClass());
if(bodypart.isMimeType("text/plain")){
try{
//if(bodypart==null){
// System.out.println("Null in bodypart");
//}else{
System.out.println((String)bodypart.getContent());//cant get Content!
//}
}catch (IOException e){
System.out.println("IO Exception on attachment"+(i+1)+": "+e+"\n");
}
//result=(String) bodypart.getContent();
}else if (bodypart.isMimeType("text/html")){
try{
html = (String) bodypart.getContent();//cant get Content!
System.out.println(org.jsoup.Jsoup.parse(html).text());
//result = org.jsoup.Jsoup.parse(html).text();
}catch (IOException e){
System.out.println("IOException on part"+(i+1)+": "+e+"\n");
}
}else if (bodypart.isMimeType("multipart/*")){
System.out.println("Openning Multipart bodypart...!\n");
getTextFromMimeMultipart((MimeMultipart) bodypart.getContent());
}else{
System.out.println("No none part?!: "+bodypart.getContentType());
}
}
return result;
}
My Problems lies in .getContent(). it always throws IOException: no Content, eventhough i know there is a text in my mail.
Following output was generated:
Trying:***mail Subject***...
-------------------------------------------------------------------------------
Number of parts: 1
Type of part 1 multipart/alternative;
boundary=_000_************************************************_(snippedout)
Class: class com.sun.mail.imap.IMAPBodyPart
Openning Multipart bodypart...!
Number of parts: 2
Type of part 1 text/plain; charset=Windows-1252
Class: class com.sun.mail.imap.IMAPBodyPart
IO Exception on attachment1: java.io.IOException: No content
Type of part 2 text/html; charset=Windows-1252
Class: class com.sun.mail.imap.IMAPBodyPart
IOException on part2: java.io.IOException: No content
Do I miss something crucial? Am i handling the bodypart correct?
Additional note:
Sorry for not providing the calling function. I have indeed opened the folder
Funktion it is called from:
public void findJunkonBody (String pfolder, String pdestfolder)throws MessagingException, IOException{
Folder folder ;
Store store = null;
Folder destfolder;
MailUtil mutil = new MailUtil(smtphost,user,pass,"imap");
String [] junktermsbody=new String[1];
junktermsbody[0]="Empfangsbestätigung";
try{
store=buildIMAPStore();
store.connect(smtphost,user,pass);
folder = store.getFolder(pfolder);
destfolder = store.getFolder(pdestfolder);
folder.open(Folder.READ_WRITE);
Message[] msgs = folder.getMessages();
int i;
int j;
MimeMultipart mmpart;
for (i=0; i<=(msgs.length-1);i++){
for(j=0;j<=(junktermsbody.length-1);j++){
if (msgs[i].isMimeType("text/plain")){
if(msgs[i].getContent().toString().contains(junktermsbody[j])){
System.out.println("Palin text: "+msgs[i].getSubject());
//move logik
}
}else if (msgs[i].isMimeType("multipart/*")){
mmpart =(MimeMultipart) msgs[i].getContent();
System.out.println("Trying:"+msgs[i].getSubject()+"...\n-------------------------------------------------------------------------------");
if (getTextFromMimeMultipart(mmpart).contains(junktermsbody[j])){
System.out.println("Time to move it!");
//move logik
}
}
}
}
folder.close(true);
}finally{
if (store != null) {store.close();}
//System.out.println("Junk on Body moved successfully");
System.out.println();
}
}

When it comes to plain/text messages you need to open first the java server mail folder.
You need to open the folder before reading the content.
Folder folder = message.getFolder();
// Open folder in read-only mode
if (folder.isOpen()) {
if ((folder.getMode() & Folder.READ_WRITE) != 0) {
folder.close(false);
folder.open(Folder.READ_ONLY);
}
} else {
folder.open(Folder.READ_ONLY);
}
now you can read the message body content:
Object content = message.getContent();

Using Java how do I strip the html from a pop3 email when reading it using javamail?

I need to let users submit an email to an address which will be used to populate entities in my database. My program will periodically check the inbox for new mail and when it finds a new mail item, I need to use the subject, from, sent date, attachments and body to populate DB entities. I have been able to get all of the fields, but I am having trouble with the body when it contains html. I just need to store the text of the email. I would like to strip out all tags, signatures, etc. from the body. Is there a better way to do this other than regex?
Here is the function I am using to get the body text. My problem lies when the mimetype hits the "multipart/*" case in the last part of the function. The function returns the html message. What can I do to strip the tags in that section other than regex?
/**
* Return the primary text content of the message.
*/
private String getText(Part p) throws MessagingException, IOException {
if (p.isMimeType("text/*")) {
String s = (String)p.getContent();
textIsHtml = p.isMimeType("text/html");
return s;
}
if (p.isMimeType("multipart/alternative")) {
// prefer html text over plain text
Multipart mp = (Multipart)p.getContent();
String text = null;
for (int i = 0; i < mp.getCount(); i++) {
Part bp = mp.getBodyPart(i);
if (bp.isMimeType("text/plain")) {
if (text == null){
text = getText(bp);
}
continue;
}
else if (bp.isMimeType("text/html")) {
String s = getText(bp);
if (s != null){
return s;
}
}
else {
return getText(bp);
}
}
return text;
}
else if (p.isMimeType("multipart/*")) {
Multipart mp = (Multipart)p.getContent();
for (int i = 0; i < mp.getCount(); i++) {
String s = getText(mp.getBodyPart(i));
if (s != null)
return s;
}
}
return null;
}
Any and all help is much appreciated.
I've been trying the following, but it is resulting in the spanish á problem I have commented about below.
else if (p.isMimeType("multipart/*")) {
Multipart mp = (Multipart)p.getContent();
for (int i = 0; i < mp.getCount(); i++) {
String s = getText(mp.getBodyPart(i));
Document doc = Jsoup.parse(s);
String retText = doc.text();
retText.replaceAll("[0%d0%a]", "<br />");
if (retText != null)
return retText;
}
}
I have also tried [\r\n] and [\n] as my regex.

You can use some HTML parser like jsoup to traverse the HTML code and extract just the text that you want.
Take a look:
http://jsoup.org/cookbook/introduction/parsing-a-document
http://jsoup.org/apidocs
http://jsoup.org/apidocs/org/jsoup/Jsoup.html

JavaMail - Parsing email content, can't seem to get it to work! (Message.getContent())

For a few weeks I have been developing a email client for android, I have been ignoring parsing email content for a while as I have never been able to get it to work. Thus, the time has come to ask for help!
I have been looking around and I have come across a few methods I have tried but never had much success with! Currently my closest attempt would have to be:
private String parseContent(Message m) throws Exception
{
//Multipart mp = (Multipart)c;
//int j = mp.getCount();
/*for (int i = 0; i < mp.getCount(); i++)
{
Part part = mp.getBodyPart(i);
System.out.println(((MimeMessage)m).getContent());
content = content + part.toString();
//System.out.println((String)part.getContent());
}*/
Object content = m.getContent();
String contentReturn = null;
if (content instanceof String)
{
contentReturn = (String) content;
}
else if (content instanceof Multipart)
{
Multipart multipart = (Multipart) content;
BodyPart part = multipart.getBodyPart(0);
part.toString();
contentReturn = part.getContent().toString();
}
return contentReturn;
}
But it does not work and I get gibberish such as "javax.mail.internet.MimeMultipart#44f12450".
Can anyone see where I am going wrong?
Thanks,
Rhys

None of the above suggestions is valid. You don't need to do anything complex here. Mimemessage has got message.writeTo(outputStream);
All you need to print the message is:
message.writeTo(System.out);
The above code will print the actual mime message to the console (or you can use any logger).
Save the content to .eml and you can open it in outlook. Simple as that!

Multipart multipart = (Multipart) content;
BodyPart part = multipart.getBodyPart(0);
part.toString();
contentReturn = part.getContent().toString();
When you have BodyPart part, you should keep testing
if(part.getContent() instanceof String){
...
}

I had the same issues while parsing Message of javax mail. In my workaround i found a weird thing. Listing mail from POP3 was not giving me Mail body content. So used IMAP which worked for me. Now i'm able to parse Text/plain as well as Text/Html and read the body. To parse the same i used following method.
public String printMessage(Message message) {
String myMail = "";
try {
// Get the header information
String from = ((InternetAddress) message.getFrom()[0])
.getPersonal();
if (from == null)
from = ((InternetAddress) message.getFrom()[0]).getAddress();
System.out.println("FROM: " + from);
String subject = message.getSubject();
System.out.println("SUBJECT: " + subject);
// -- Get the message part (i.e. the message itself) --
Part messagePart = message;
Object content = messagePart.getContent();
// -- or its first body part if it is a multipart message --
if (content instanceof Multipart) {
messagePart = ((Multipart) content).getBodyPart(0);
System.out.println("[ Multipart Message ]");
}
// -- Get the content type --
String contentType = messagePart.getContentType();
// -- If the content is plain text, we can print it --
System.out.println("CONTENT:" + contentType);
if (contentType.startsWith("TEXT/PLAIN")
|| contentType.startsWith("TEXT/HTML")) {
InputStream is = messagePart.getInputStream();
BufferedReader reader = new BufferedReader(
new InputStreamReader(is));
String thisLine = reader.readLine();
while (thisLine != null) {
System.out.println(thisLine);
myMail = myMail + thisLine;
thisLine = reader.readLine();
}
}
System.out.println("-----------------------------");
} catch (Exception ex) {
ex.printStackTrace();
}
return myMail;
}
Hope it helps someone.

I also got the same error any tried almost every thing, but the solution worked for me is
private String getFinalContent(Part p) throws MessagingException,
IOException {
String finalContents = "";
if (p.getContent() instanceof String) {
finalContents = (String) p.getContent();
} else {
Multipart mp = (Multipart) p.getContent();
if (mp.getCount() > 0) {
Part bp = mp.getBodyPart(0);
try {
finalContents = dumpPart(bp);
} catch (Exception e) {
e.printStackTrace();
}
}
}
return finalContents.trim();
}
private String dumpPart(Part p) throws Exception {
InputStream is = p.getInputStream();
// If "is" is not already buffered, wrap a BufferedInputStream
// around it.
if (!(is instanceof BufferedInputStream)) {
is = new BufferedInputStream(is);
}
return getStringFromInputStream(is);
}
private String getStringFromInputStream(InputStream is) {
BufferedReader br = null;
StringBuilder sb = new StringBuilder();
String line;
try {
br = new BufferedReader(new InputStreamReader(is));
while ((line = br.readLine()) != null) {
sb.append(line);
sb.append("\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return sb.toString();
}
Hope this will help someone.

Not entirely sure, but what it sounds like is you want to convert a MimeMessage into a String. Strictly speaking, there is no "standard" translation for MimeMessage to String, but here is a chunk of code that I wrote a couple of years back that attempts to generate one such translation. Only tested on English messages, so i18n will have to be something you think about yourself.
Unfortunately SO's stupid filter seems to think that my code sample is an image and won't let me post it: take a look at the class at http://pastebin.com/pi9u5Egq : the code is doing a bunch of otherwise unnecessary things (it was spitting it out into HTML that was rendered using flying saucer), and also requires Jakarta Commons Lang, javamail and activation libraries, but it works for me. You would invoke it like this:
fetchText(message, null, false, false);
To get a text string. Try that out for size.

Download attachments using Java Mail

Now that I`ve downloaded all the messages, and store them to
Message[] temp;
How do I get the list of attachments for each of those messages to
List<File> attachments;
Note: no thirdparty libs, please, just JavaMail.

Without exception handling, but here goes:
List<File> attachments = new ArrayList<File>();
for (Message message : temp) {
Multipart multipart = (Multipart) message.getContent();
for (int i = 0; i < multipart.getCount(); i++) {
BodyPart bodyPart = multipart.getBodyPart(i);
if(!Part.ATTACHMENT.equalsIgnoreCase(bodyPart.getDisposition()) &&
StringUtils.isBlank(bodyPart.getFileName())) {
continue; // dealing with attachments only
}
InputStream is = bodyPart.getInputStream();
// -- EDIT -- SECURITY ISSUE --
// do not do this in production code -- a malicious email can easily contain this filename: "../etc/passwd", or any other path: They can overwrite _ANY_ file on the system that this code has write access to!
// File f = new File("/tmp/" + bodyPart.getFileName());
FileOutputStream fos = new FileOutputStream(f);
byte[] buf = new byte[4096];
int bytesRead;
while((bytesRead = is.read(buf))!=-1) {
fos.write(buf, 0, bytesRead);
}
fos.close();
attachments.add(f);
}
}

Question is very old, but maybe it will help someone. I would like to expand David Rabinowitz`s answer.
if(!Part.ATTACHMENT.equalsIgnoreCase(bodyPart.getDisposition()))
should not return all atachments as you expect, because you can have mail where mixed part is without defined disposition.
----boundary_328630_1e15ac03-e817-4763-af99-d4b23cfdb600
Content-Type: application/octet-stream;
name="00000000009661222736_236225959_20130731-7.txt"
Content-Transfer-Encoding: base64
so in this case, you can also check for filename. Like this:
if (!Part.ATTACHMENT.equalsIgnoreCase(part.getDisposition()) && StringUtils.isBlank(part.getFileName())) {...}
EDIT
there is whole working code using condition descibed above.. Because each part can encapsulate another parts and attachment should be nested in, recursion is used to traverse through all parts
public List<InputStream> getAttachments(Message message) throws Exception {
Object content = message.getContent();
if (content instanceof String)
return null;
if (content instanceof Multipart) {
Multipart multipart = (Multipart) content;
List<InputStream> result = new ArrayList<InputStream>();
for (int i = 0; i < multipart.getCount(); i++) {
result.addAll(getAttachments(multipart.getBodyPart(i)));
}
return result;
}
return null;
}
private List<InputStream> getAttachments(BodyPart part) throws Exception {
List<InputStream> result = new ArrayList<InputStream>();
Object content = part.getContent();
if (content instanceof InputStream || content instanceof String) {
if (Part.ATTACHMENT.equalsIgnoreCase(part.getDisposition()) || StringUtils.isNotBlank(part.getFileName())) {
result.add(part.getInputStream());
return result;
} else {
return new ArrayList<InputStream>();
}
}
if (content instanceof Multipart) {
Multipart multipart = (Multipart) content;
for (int i = 0; i < multipart.getCount(); i++) {
BodyPart bodyPart = multipart.getBodyPart(i);
result.addAll(getAttachments(bodyPart));
}
}
return result;
}

Some time saver for the code where you save the attachment file :
with javax mail version 1.4 and after , you can say
// SECURITY LEAK - do not do this! Do not trust the 'getFileName' input. Imagine it is: "../etc/passwd", for example.
// bodyPart.saveFile("/tmp/" + bodyPart.getFileName());
instead of
InputStream is = bodyPart.getInputStream();
File f = new File("/tmp/" + bodyPart.getFileName());
FileOutputStream fos = new FileOutputStream(f);
byte[] buf = new byte[4096];
int bytesRead;
while((bytesRead = is.read(buf))!=-1) {
fos.write(buf, 0, bytesRead);
}
fos.close();

You can simply use Apache Commons Mail API MimeMessageParser - getAttachmentList() along Commons IO and Commons Lang.
MimeMessageParser parser = ....
parser.parse();
for(DataSource dataSource : parser.getAttachmentList()) {
if (StringUtils.isNotBlank(dataSource.getName())) {}
//use apache commons IOUtils to save attachments
IOUtils.copy(dataSource.getInputStream(), ..dataSource.getName()...)
} else {
//handle how you would want attachments without file names
//ex. mails within emails have no file name
}
}

Returns list of body parts with attachments.
#Throws(Exception::class)
fun getAttachments(message: Message): List<BodyPart>{
val content = message.content
if (content is String) return ArrayList<BodyPart>()
if (content is Multipart) {
val result: MutableList<BodyPart> = ArrayList<BodyPart>()
for (i in 0 until content.count) {
result.addAll(getAttachments(content.getBodyPart(i)))
}
return result
}
return ArrayList<BodyPart>()
}
#Throws(Exception::class)
private fun getAttachments(part: BodyPart): List<BodyPart> {
val result: MutableList<BodyPart> = ArrayList<BodyPart>()
if (Part.ATTACHMENT == part.disposition && !part.fileName.isNullOrBlank()){
result.add(part)
}
val content = part.content
if (content is Multipart) {
for (i in 0 until (content ).count) {
val bodyPart = content.getBodyPart(i)
result.addAll(getAttachments(bodyPart)!!)
}
}
return result
}