Just got a request from my boss for an application I'm working on. Basically we're getting an email address setup for an external client to submit excel files to.
What I need is a way to automatically pick up any email sent to this address, so I can take the attachment, process it and save it to a folder.
Any information of even where to start would be helpful.\
Note: We're using a lotus notes server to do this, but a generic way would be more helpful (If possible).
Email -> mailserver ->[something] -> file-on-disk.
File on disk is pretty easy to parse, use JavaMail.
The [something] could be:
listener for smtp connections (overkill)!
Pop3/imap client
Maildir/Mailbox
Edit: since I first wrote this answer, Wiser has moved and now claims to only be a unit testing tool, so take the answer below with a pinch of salt...
Svrist's answer is good, but if you want to avoid his middle step (the mailserver that writes the mail to disk for later pickup by the Java system) you can use Wiser.
Wiser lets you start an in-Java mailserver:
Wiser wiser = new Wiser();
wiser.setPort(2500);
wiser.start();
Then you can just poll it periodically for mail:
for (WiserMessage message : wiser.getMessages())
{
String envelopeSender = message.getEnvelopeSender();
String envelopeReceiver = message.getEnvelopeReceiver();
MimeMessage mess = message.getMimeMessage();
// mail processing goes here
}
Use a mail in database (your Domino administrator can set that up for you but it's in the help file as well).
In that database, you can create an agent that runs periodically to process all new documents. That agent will use the EmbeddedObjects property of the NotesRichTextItem class and the ExtractFile method of the NotesEmbeddedObject class to get a handle on the file attachment and extract it to the location you specify.
For example, this script goes through all the file attachments, object links, and embedded objects in the Body item of a document. Each time it finds a file attachment, it detaches the file to the SAMPLES directory on the C drive and removes the attachment from the document
Dim doc As NotesDocument
Dim rtitem As Variant
'...set value of doc...
Set rtitem = doc.GetFirstItem( "Body" )
If ( rtitem.Type = RICHTEXT ) Then
Forall o In rtitem.EmbeddedObjects
If ( o.Type = EMBED_ATTACHMENT ) Then
Call o.ExtractFile( "c:\samples\" & o.Source )
Call o.Remove
Call doc.Save( False, True )
End If
End Forall
End If
I've done quite a bit lately with Java agents on Domino servers. The Domino 8.5 server supports Java 6 and its embedded so it won't take someone with a bit of Domino development experience long to put together an agent that runs when new mail arrives. In LotusScript its even easier but that needs more specialised skills which you'd probably need to get a contractor in to provide.
The limitation your likely to encounter concerns the extracted file, you can easily place it on the Domino server's file structure but you may be limited by the OS security from placing it on a different server.
Lotus Notes/Domino stores mail in a Notes database. There are APIs available for getting documents (emails), reading field values (From, Subject), and detaching files.
APIs include
-LotusScript (VB variant, available within the Notes database)
-Java (from within or external to the database)
-C API (external)
-Same API available through COM server
You can create a "scheduled agent" within the database (using LotusScript or Java) that can locate documents created since it last ran, locate the attachments, and extract them. The agent will need to be signed with an ID that has the appropriate permissions on the server, including those required to write to the file system and initiate any other processes.
External to the database, you can use any API except LotusScript to log-in to the server/mail database, and follow a similar process, e.g. extracting the files locally on a client or separate server. C API and COM require a notes client install, but Java applications can be set up to run via CORBA/DIIOP without a full install.
Consult the Domino Designer help (or IBM's website for C API) for more information.
As to a "generic way" to do this, if you are accessing data in Notes and needing to extract attachments, I believe these APIs are your best option. If you envision porting the application to another mail system, consider decoupling the API routines via an "interface" so you only need to add a new implementation of that interface to support a new mail system.
You can access Notes Documents relatively easily using DIIOP, would be a lot easier than going down the C Api road...
Try POP3Client in the Net Commons package; it'll let your Java program check for new mail for a particular account at whatever interval you want (every few minutes? hourly?), and get/delete messages as desired.
SMTP/POP3 can be enabled on the Domino server. Worked with this before and gotten Squirrel Mail running with it. SMTP is a bit resource intensive, but well worth the effort because then you don't have to descend into LotusLand to get things working. Just write a small Java CLI program that will check a specific email box (POP3 or SMTP), and parse through the messages, pulling the attachments and placing them where needed.
Plenty of documentation and examples here:
http://java.sun.com/products/javamail/
The techniques that you develop taking this approach will be more widely applicable in your future career than anything Lotus/Domino specific.
No matter what you do, you'll need an understanding of the Lotus Notes data structures. The good news is that a fully automated solution can be built in Notes very easily.
Your best bet is to have it built within Notes, and it can be set up to run automatically whenever new mail is received. Gary's answer is dead on, but without any experience, it would probably be hard to figure out how to implement it yourself. On the other hand, it really shouldn't take any competent Notes programmer more than an hour or two to set it up.
Related
I need to get contact list from Outlook Exchange. The problem is that I have to use Java and I totally don't know where to start. Can anyone tell me what I have to do firstly?
How can I programmaticaly connect to Outlook?
If you are running on Windows you can probably use JaWin. It is an open source library that wraps COM object and provides you a Java API to access them. As far as I remember its distribution contains example of how to connect to MS Exchange server.
Other similar packages I know are
Jintegra (costs some money)
Jinterop (open source too)
Both libraries implement DCOM protocol in Java, so you can run application that uses them on any platform and connect to exchange server.
Other way is to use POP3 or SMTP protocol also supported by Exchange. There are a lot of packages that support them, e.g. JavaMail.
And the last way: if your application is running on client side, i.e. on the client's computer it can parse files created by outlook itself. I do not remember where these files are stored but I remember that many years ago I have discovered the issue and saw that all emails are stored in file system in clear text format.
EDIT: Recently I found out JACOB: other library that uses JNI (like JaWin).
I am developing a Java app that reads all the emails from GMAIL and save them run time on objects of self made Class that holds some of the attributes of the email. Now I want to delete an email object and when I do that I want it to be removed from the internet as well. In order to do so I will need some information to query the GMAIL account to delete that particular email I am referring to. Is there a possible way to do that? Maybe by saving the header and then searching for that? I couldn't find a good way to do it.
I assume you are accessing GMAIL with IMAP. Then you could store the Message, call Message.setFlag(Flag.DELETED) then expunge the Folder, or search the Folder yourself. Or maybe you can search the Google Gmail api and find a better way. For example you can use the unique message ID
UPDATE this is a google project that can help you: java-gmail-imap. There is also a companion project for OAuth authentication (needed for run the sample)
I'd recommend you to take a look on James.
The Apache James Project delivers a rich set of open source modules and libraries, written in Java, related to Internet mail communication which build into an advanced enterprise mail server.
It depends on the time span between when you access the message and when you want to delete it. If it's short, and the folder is still open, you can just set the DELETED flag and close the folder to expunge the message.
But I'm assuming that you need to locate the message some time later. In that case, you should learn about IMAP UIDs. See the UIDFolder interface in JavaMail and the IMAP spec for details. In essence, you need to keep track of the folder's UIDVALIDITY value and the message's UID value. When you come back to the folder, you can make sure it's the same folder by checking the UIDVALIDITY value and then look up the message based on its UID.
I have an Enterprise Java web application that all of the business is migrated from client-server application. So I need to adapt all requirement including printing on every client (Using Dot Matrix / Plain text).
Printing requirement for client on separated client is they must get the report right away after the process is finished, let say print some system information. On Client-server is easy just for printing, but when I migrate to web apps, we used Server/applet service that embedded to the apps. So the applet will be initialized when the client open the browser.
It works, but too many issues. Sometime the applet is closed suddenly without action from client, so automatically the applet will be failed for checking the queued report from database. Is there any suggestion for this case?
I use Spring Framework as a business layer.
I had the same problem years ago.
This is what I did.
Since my processing was taking place on the server, I knew when the transaction was finished. So what I did was to send the print request directly from the server. The client printer was configured in the server and since it was used in the LAN it could be easily reached. That way I finished with all the problems I had with the applet, the tradeoff was they can only print in the office, but in my situation that was fine, not necessarily applies for you, give it a try.
UPDATE
In my case, the print was needed by different departments along the country.
Each department had an specific, unique printer for that task. So what I did was have that printer mounted in the server filesystem ( in Unix something like /Volumes/printers/EPSON-12345 , in Windows as x:\printers\EPSON-12345 ) When the user finished the transaction in the webapp I start in the server a print request. Java does not have any problem at all to print to a local printer and since a mapped printer is taken as local ( even though it is a remote one ) you don't need further authorizations or anything special.
Since I knew what the specific printers were mapped, and I knew what kind of transaction that was, I knew where to print.
Finally I use a property file to allow the printer to change from time to time and/or add more printers.
I don't quite remember the details but it was something like this in the server:
ResourceBundle bundle = ResourceBundle.getBundle("printers");
String serviceName = bundle.getString("MEXICO.CITY.PRINTER");
PrintService service = PrintService.find( serviceName );
service.print( // document and etc );
But that was almost 4 yrs. ago so I don't remember exactly the details, but my program is still running :)
You can read more about the printing service in java here. It is way much simpler than what it look like.
Other ways to achieve a good result:
generate a PDF (using iText, or better JasperReports) on the server, and send it for download to the client, where he can print it.
have a print.css, and generate a web-page that is suitable for printing.
If you do not want the client browser to orchestrate the printing (i.e. by sending a binary blob with a custom MIME-type which the user must set up to be sent to the printer), you must let your server do it.
This usually means that the user must have a default printer attached to their profile, and that the server knows how to send prints to that printer. Lots of fun :)
We have a php setup for our web pages that is secure with HTTPS. The web app talks to a DB but we also want it to talk to a java server we have.
The java server is a standalone java application (not web). We simply want a callback action after the PHP page finished writing to the DB done in the java server. What is a good way for this php page to talk to the java program to get something done?
I usually recommend against quick and dirty but here :
You can dump data in a file if it can be asynchronous. Then a cron job from java, checking for that kind of file at a regular interval, do the specified command.
For example, you can dump the word ExecuteCmd1 in a file. The java thread reads it, interprets it and choose that he must execute the method or class with the same name.
You can do the same thing over to go back to php.
Probably via a TCP/IP connection. If your Java application runs a server, then the PHP script can connect and send a message informing the Java app that the DB has been written to.
Do a quick and dirty JSON RPC from PHP to Java. You could probably get it up and running in one cup of coffee.
Use CURL on php (http://php.net/curl) and json_encode() to POST a json string to your Java server. (scroll down and find the curl wrapper class that someone wrote in the comments. It's easy.)
Use JSON (http://www.json.org/java/) in Java to decode it and use it immediately. Send your response back in JSON too.
I had a similar XML RPC system running in production for years. PHP -> IP -> Java works great.
Google Protocol Buffers Not so much dirty, but works, and works well, regardless of which launguage you use.
You can try the PHP/Java bridge. I used it a while ago to use Java logic inside Typo3, a PHP CMS.
My advice, whether you use the bridge or not: make sure you know where the errors come from if something doesn't work. Check both PHP and Java logs. Be verbose if an exception occurs.
How much data do you need to transfer?
How many requests per second?
Does the Java application have to handle the request immediately, or is it enough to handle the request in a few minutes?
Does the Java application need to return data to the user's browser?
If the answers to questions 3 and 4 are no and no, you could just create a database table for the jobs, have the PHP app insert a new job, and have the Java app poll the job table every minute or so.
I have written a nice program in Java that connects to a gmail account and download atachments sent to it. Once an attachment has been downloaded, it is marked as read and is not downloaded ever again. This program will have to run in multiple instances with each program downloading unique attachments so that a single attachment is never downloaded twice. The problem is that at the moment if the attachment is of a decent size, one program is still downloading it, when another instance connects and also starts to download the attachment before it has been marked as read.
I have tried checking and setting various flags and checking whether the folder is open, nothing seems to work. Any solutions?
Update: Thank you for the quick answers, sadly IMAP is not an option due to other reasons.
Consider using IMAP instead - it is designed for client-server interaction.
From RFC1939 (Post Office Protocol - Version 3):
POP3 is not intended to provide
extensive manipulation operations of
mail on the server; normally, mail is
downloaded and then deleted. A more advanced (and complex) protocol, IMAP4, is discussed in RFC1730.
I don't think POP3 is made for multiple simultaneous access.
Ask yourself this: do i really need multiple processes accessing the same mailbox?
If you do, you'll have to find a way to have these processes communicate to each other.
Use a common database or server process to coordinate actions.
IMAP does have more options, but i'm not sure if you can "lock" a single mail to mark it as being processed.
As the others have mentioned, POP3 isn't really intended for this kind of scenario.
If you absolutely have to use POP3, I'd suggest downloading all the e-mail to an intermediate server which sorts the messages and makes them available for each of the other clients.
It sounds like you're just trying to distribute the processing of the e-mails. If that's the case, you can just have each client connect to your intermediate server to retrieve the next available message.
I'm not sure what your constraints are, but you may even want to consider receiving the attachments some other way besides e-mail. If people are uploading files, you could set up a web form that automatically sends each file to the next available instance of your application for processing.
If you need to stay with a POP3 connection, you could keep a local database of previously downloaded message ids. Then new instances could check against that before downloading again. The best solution is just to use IMAP, though, as IMAP is able to set the read/unread flags before downloading.
You could mark the mail as read before starting the download, and then start downloading it.