We Keep Coding

Apache Storm integration with Spring framework

I'm new to Apache Storm. Currently I'm working on legacy project that involves some streaming processing using Apache Storm. I want to integrate current project with Spring. I found couple comments (Storm and Spring 4 integration, http://mail-archives.apache.org/mod_mbox/storm-user/201605.mbox/%3CCAMwbCdz7myeBs+Z2mZDxWgqBPfjcq-tynOz_+pmPrmY6umfUxA#mail.gmail.com%3E) saying that there are concerns doing that. Can someone explain me how to do such an integration or why it is impossible?

Fair warning, I haven't used Spring in Storm, so this is based solely on my knowledge of Storm, and having used Spring on non-Storm projects, i.e. this is really just guesswork.
I think you can use Spring with Storm, but there are some caveats you should be aware of. Whether Spring is still worth using given these caveats is up to you.
Unlike e.g. a Spring MVC application, Spring will not be responsible for object instantiation or application flow. Storm doesn't know about Spring, and when you run your topology it will be Storm that calls your bolt/spout methods. This means you have to be aware that some parts of your application will be called outside the Spring context.
Here's my guess at where you could use Spring during different phases of a topology deployment.
When you set up your topology and submit it (all your code up to and including StormSubmitter.submitTopology), you can most likely use Spring just like you would in any standalone Java application. e.g. you could start your application like in this example, and put all your submission and wiring code in Main.start. All bolt/spout constructors will run in this phase, so you can use autowiring here if you like. You have to ensure that your spouts and bolts are serializable though.
After topology submission, Storm will serialize your spouts and bolts (and any non-transient fields in these objects), and send them to the supervisor machines, where they will be deserialized. At this point if you need a context available in the worker, you could create one in a worker hook (added to the topology via TopologyBuilder.addWorkerHook), and expose it through a static method on the hook (which is a little ugly, but I don't see any other way to make it available to other parts of the code).
Just to reiterate, if you decide to run a Spring context inside your workers, you have to be aware that the spout/bolt methods will be invoked by Storm outside the Spring context.

Implement Java Web Server with database, GCM and background threads

I am trying to create a web server that provides web service functions, communicates with android devices using GCM, uses a database and probably also has some background work to do.
I am currently unsure on the architecture of such a server.
I know how to create a simple web service, but have not found tutorials or descriptions that go beyond simple "Hello World" examples. As far as i know i can create a class with the #WebService annotation, and once deployed to e.G. Tomcat the server will create at least one instance of this class and provide the annoteted functions in this class as web services.
Now i wonder how to best implement database connection. From what i know the server would create an instance of this class for every request, wich will be garbage collected once the connection is closed. Since the web server needs the database for nearly every function it provides i think it would not be a good idea to create a database connection for every instance, but rather use a global connection to query the database.
From what i found out so far this could be achieved by injecting a class that handles the database connection as #Resource or #Singleton into the web service class.
But is this the proper way to do this ? Or am i worring too much and just creating a new database connection for each request is fine ?
Then i want to send GCM messages (the simple POST ones using the HTML google server), so i would probably create a controller or manager class to handle these requests. Would this also be injected as a #Resource or #Singleton into the web service class ?
And last but not least the server probably has to do some work periodicaly, wich would be some kind of background thread, that is independant of the requests the web service is recieving. Here i am at a loss on how to do this. A web service does not have a main() method i am aware of, so i am unsure on how to create this.
Can anyone give me a guick overview on how to design such a web server or can point me to documentation that describe on how to achieve this ?
All i found so far were simple examples that don't cover advanced stuff like this.

Ok, so you have 3 questions/problems.
How to create a Webservice
How to manage database connections
Execute batch process
All of this, in the same App. First of all, I should advice you to split into 2 app, one for consume (Web Services) and another for batch processing, including push notifications (IMHO). But lets go one step at a time.
1. Webservice: It's depends on the framework you choose. I usually made a choice between Spring-MVC or Jersey
2. How to manage database connections When you are querying a database, you usually don't want to open and close connections crazily. You want to use a connection pool. In a connection pool you'll ask for an open connection, will use it, and will free it when you are done. Normally, a connection pool is managed por the application server. If you want to manage the connections manually, you have to use a singleton to centralise acquiring and releasing.
3. Execute batch process You probably should use a singleton to manage batch process. This job manager will launch the job executions on other threads
Don't know if that answers your question/concern.. please let me know.

Ryu,
I found myself in a similar scenario. After going through the webs for over 2 days, I stumbled upon this solution of running a background thread which is triggered during the initialization of a servlet(init method).
Perhaps you can give this a shot and let me know if it works for you.
Here's the link which has an example to try - http://docstore.mik.ua/orelly/java-ent/servlet/ch03_05.htm
Cheers!

Execute Command from Web Archive

I included a scheduled job in my WAR file through Quartz and Spring. In case that the scheduled job misses, I have to execute a method Class_A.Method_A() explicitly.
In order to execute the method, I plan to create a static method main() under Class_A so that I can execute jar -cp $CLASSPATH Class_A. However, the class is inside the WAR file, how can I do it?
In addition, the WAR file has its data source and log4j configuration and the method Method_A does database access and logging through them, if I call it on command prompt, is there any conflict?
If calling it through a comamnd prompt is not a good practice, what is a better way? Please help.

Why are you trying to execute 'jar -cp'. This will be a separate jvm execution and hence you will not be able to directly access the resources in the jvm running the web application (this means objects spawned in the jvm's memory space by the web application). [This is answer to your question about conflict/
Please mention what application server on which your web application is running.
Seeing your comment about System Administrator (though I would have mentioned this regardless of this as well). Have you ever heard of ServiceMBeans, you can try them.
Your scenario is a very generalized scenario, where people need to access a particular class (better say instance of the class) running inside a JVM. You certainly need something which loads up along with the application.
You can write a Service MBean to run along (inside) your web application. This would mean you are exposing action. Then you can write a java client to interact with the MBean and make call to its exposed methods.
In case your application server provides authentication for accessing MBeans.
Other option is JMS implementation. Setup a JMSQueue, whose listener will execute the action interacting with the classes of web application. Obviously the listener would load along side web application. EJB implementation would allow you to load the listener via simple ejb xml or through annotations.
Then you write up a separate java code which can send message commands to the JMSQueue.
All application server provides the option of authentication.

Implementing a Hl7 parser listener to java

Background
I'm writing a medical record app for a friend who is a Doctor. I was told to write a listener in the app that awaits HL7 messages. That way a hospital can send out HL7 messages and my listener will catch them. So I came to the HAPI site and viewed this example. What I understand from it is that it's creating a server to listen for a message.
I'm developing this in Eclipse using JSF 2.0 on Tomcat 7.0. Where does one normally put this kind of listener in a project with JSF? I've tried searching online for this answer and found nothing!
My question
I know this code goes inside a class. When the class gets called the socket will be "turned on" and it's going to wait for a response. So I want to call this class as soon as the project is deployed. How is that done? How do I call that class only once (when the app is starting) in order to turn on the listener?
Any and all help is greatly appreciated! And if I'm not being clear on something let me know!

You don't normally get port listeners running inside an application hosted in Tomcat. You're usually best to keep the two things separated. In the main, web servers aren't meant to run separate threads of execution outside of their control.
You could consider using something like Spring Integration, JBossESB or Apache Camel to receive the messages and process them into a database, file folder (or whatever) that your Tomcat hosted web application then allowed you to manipulate. The ESB container could be hosted in the same JVM process as Tomcat but I wouldn't take that approach myself - I'd have a separate one doing the message processing and another running the webapp.
If you really wanted a "single application" you could consider creating a Java application that kicked off a listener as per the example you have, then started up an embedded version of Tomcat.
If you really really wanted to run it inside Tomcat, as part of web application itself, you could create a class which did the listening and get it loaded into the Application context of the web application. You can do this by adding an instance of it into the appl context within an autoloaded servlet - use <load-on-startup>1</load-on-startup> within the servlet definition. You'd code the servlet to check if there was already an instance in context before adding a new one (on the off-chance it was ever manually invoked), or go down the Spring container route to manage this object as a singleton.
* EDIT: 20120114T004300Z *
Apache Camel is an example of a routing engine that might be used by an Enterprise Service Bus (ESB) such as Apache ServiceMix which allows multiple applications to interoperate by exchanging messages. You'd only use a fraction of the functionality availability for this app by the sounds of it. For what you're doing you might just be able to use Camel capability embedded in Spring, for example.
In essence, the ESB runs "adapters" (or endpoints) - one types of which would by the socket "listener" you talk about here, or might be watching a folder for files to arrive, or polling a database table for rows to appear, or waiting on a JMS queue, etc. The transport (the means by which the "message" (in your case the HL7 file) arrives becomes abstracted away from the functionality of the application itself. The adapter puts the message onto a channel which can be configure to transform the message en-route. Camel actually ships with a HL7 component which can understand the HL7 file format and unmarshal it into a HL7 model. (It also gives you the listener/adapter you need). You'd then set up routing in the ESB to pass that model into a "consumer" Java class that does whatever you need to do with it.
If you're dealing with "standard" transports, protocols and message types most of the file receipt, parsing, and routing is just handled by declarative configuration of the ESB rather than coding.
Your Tomcat webapp can run completely autonomously to this message handling. As mentioned, there are various deployment options as to how exactly you'd do this - including loading Camel inside a Spring container hosted in Tomcat by your webapp if you want to.
Apologies if this is a bit woffley. Take some time to read around the subject on the web, given that HL7 is a standard you'll probably find a lot of code/components already out there that might save you a lot of time in re-implementing the basic file handling so you can concentrate on the value-add webapp for your friend.

A "Listener" is just a class which listens on an open port. In Java, this is mostly done through the Socket API, although you may find a library that better suits your purpose.
The Java Tutorial has some examples here:
http://docs.oracle.com/javase/tutorial/networking/sockets/index.html
In this case, you'd be writing a server (the listening half of a client-server arrangement), whereas the Hospital system sending the message would play the role of client.
Once you're listening on the port, then HL7 messages arrive as plain text onto that socket's inputstream. You can either hand-parse the message (viable if you're only interested in one or two details from a message) or if you're planning on handling dozens of types of messages you can look into one of the HL7 parsing libraries out there.
Keep in mind though, that different implementors of HL7 messages can sometimes send data in subtly different arrangements. (Many users treat HL7 as a 'recommendation' rather than a 'standard', unfortunately!) If you're planning on supporting lots of different feeds from lots of different providers, you'd be much better off using a middleware layer like MirthConnect to handle the parsing and translation of messages into something your application is designed to understand.

Over a year old so you probably figured it all out, but an Enterprise Service Bus (ESB) is a type of middleware (when you think of software, there is back-end i.e. Database/Analytics/Admin Tools and front-end i.e. App/WebApp/GUI displayed to and interacted with end-user), middleware sits in between and helps perform integration or separation/coordination of tasks. Apache ServiceMix (an ESB which contains Apache Camel routing engine) is probably what you want and can be used to implement a number of different Enterprise Integration Patterns such as "Message Routing" (the one you want).
Apache Camel has a built-in HL7 v2 Message parser (uses HAPI) which is the Tab-Separated variant of HL7: http://camel.apache.org/hl7.html
For HL7 v3 messages which are in XML format you can use the toolkits available here under v3 utilities:
http://www.hl7.org/participate/toolsandresources.cfm?ref=nav
There are both server (message listening and reading) and client (message creation and sending) examples.

"Listener" is usually an event listener in Java.
In the example you posted a link to, you have a server class, which handles the business of opening a network socket and waiting for messages to arrive.
The Application objects are the event listeners. These are added to an internal collection of the server class (in this case, with additional parameters that tell the server which listeners to route which classes of HL7 message to).
Each Application class must implement a particular interface - this constitutes the event listener. The SimpleServer class will call the methods of this interface ; processMessage() ; in order to perform actions based on message content, you write a class that implements this interface, and pass instances of it to the server class. In the processMessage() method, you perform all the required actions.
Since you can register multiple listeners, you can implement a number of actions, e.g. you could have two listeners for ADT A01 messages (admit patient) ; one that booked them in, and one that assigned them a bed.

I would suggest looking at Mirth Connect http://www.mirthcorp.com/community/mirth-connect as your HL7 message integration engine. Internally it makes use of HAPI.

Communication between two Web Services

I have a problem with web services. They are programed in Java and are running on a WASCE server ( both are on the same server).
My problem that i want to solve:
We have two Web services: App1 and App2
In App1 i want to call a function that is in App2. How can i do this? Is this even possible?
I tried creating a soapClient inside the App1 so i can connect to the App2 but that doesn't work.
exp:
I have a client that calls app1 gets data from app1 and send it to app2 then get back the response data from app2 and send it to an other function into the app1.
What i want to do is to skip the client part and do it directly so that app1 can send directly the data to the app2 and then receive an answer do whatever it needs to do.
For the note: Both of the web services use the connection to the database.
Thank you in advance.
(it has been edited with additional data)

What does "doesn't work" mean? Exactly what happens?
Start by generating some client code for App2. Can you use that from some simple Java environment, or say a Servlet. If that works, what happens when you try to call it from inside your App 1 Service implementation code?
However: if these are related services running in the same JVM can you not set up some simpler relationship using java libraries. My preferred way of developing a service is first to develop some useful Java code, and make sure that works, then "wrap" it as a Web Service. In which case I have a callable routine that can just be invoked as Java.

It's definitely possible, with differing levels of complexity and feasibility depending on exactly what it is you want, and the restrictions you place on it.
Probably one of the simplest ways to go about this, if you don't have a problem with the method in App2 being public, is to simply create a web service exposing that method and call if from App1.
If you want App2's method to be essentially "protected", so that it can be called by App1 but not by public clients, then there are several alternative options. Firstly, you could use firewalls or equivalent to prevent external requests to the service URL. Alternatively, you could expose the method through some form of interprocess communication; RMI would be the obvious native one for Java (set up an RMI method in App2 and export this through a manager, then obtain the reference in App1 and invoke the method remotely). Depending on exactly what it is you want to do, you may be better off with a framework that does all this under the covers; e.g. distributed objects through something like Terracotta.
You should give more detail in your question, though - currently the only thing you've really specified is that you want to call "a function" in App2 from App1. There are dozens (if not hundreds) of ways to go about this and the best one(s) will depend on the details of what you're trying to do.
EDIT (in light of comments): It's not the details of what you want to do that are lacking - I understand fine that you want to call some method in App2 from within App1. It's more the architectural details - what languages are both clients coded in, what libraries are you using to do the web services, are both clients on the same machine or separate ones (and if same machine, same JVM or not), are there any firewall issues that could inhibit certain kinds of connectivity, are there any office-political restrictions that could inhibit your options, are there any security restrictions that could do the same (such as whether you can expose the functionality of App2's method publically or not). All of these will shape what is possible and what is optimal - because at the end of the day, all networking is basically I want to use resources on that remote computer from here. Without more architectural specifics, there are literally dozens of ways that you could achieve this.
Regarding exposition: You would create a web service to expose App2's function in the same way you would create any other web service (with the details being dependent on the tool/framework you're using). As an example if you're using a tool that supports the JSR-181 annotations, you'd write a method in App2 that performs this function, and annotate it with #WebMethod. Then you'd ensure that if this method is not part of an existing webservice class you'd annotate its class with #WebService. I was presuming that since you already have a couple of web services, you'd know how to write/define them.
As for accessing the web service from App1, this can be done quite simply by a Java SOAP client. A tool such as WSDL2Java can create a stub class modelling the remote service that you can call; alternatively you can get a richer interface with something like CXF.
What WS library are you using currently, and what errors have you encountered when trying to use it to perform this interaction?