We Keep Coding

How To Build An Integration Flow For Different Types Of Integrations

I know that the spring DSL can be a very powerful tool but, how do you know how to build one from scratch depending on your different needs? When I see an integration flow online it's easy to follow and I can generally make sense of the flow. However, when building something a little bit more custom how do you know what goes into the integration flow? Im new to spring integration and the Spring DSL in general so it would be useful to know where is a good place to start and what is the best way of thinking about things when it comes to spring integration.

I treat this question as a fundamental matter familiarity. Consider to start from Enterprise Integration Patterns, learn what is Message, Endpoint and Channel. Then you re-think the logic of your application as some service calls, which could be initiated by messages sent to endpoints for those service activator. When it is all good for you, you start combining the logic into a flow, when an output channel of one endpoint becomes as an input for the next one. Essentially this is exactly what is going on underneath with the mentioned IntegtionFlow abstraction. Then you go further and determine for your self that several flow could be connected via the same not one time already mentioned channels. It is not quick one stop shop, but it is fun to rethink the logic of your application as messaging exchange.
See more info in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#messaging-endpoints-chapter.
The Java DSL with its IntegationFlow definition is just a higher level of abstraction on top of endpoint implementations for specific EI patterns.

Using Camel as a Fast Back-end Bus with REST services

This is the use case I am trying to address:
I have:
a client C, which is a web app,
a frontend module A, exposing a REST API,
a back-end module B, also exposing a REST API. (I can't change this module which is not mine and which only has a REST API)
For one of the services, C calls A which calls B. Everything happens in REST. It is fast as the REST calls are synchronous (i.e. blocking).
Now I would like to decouple A and B, for A to not know about B directly.
So I am wondering about putting Camel as a message bus in the middle. It would give:
C ---(REST call)----> A ----> Camel ---(REST call)---> B
My question
Is it a good idea and if yes would it still be fast (without anything between A and B, I have a response in less than 10 ms on a local network)?
TieBreaker
Would it work the same with ServiceMix?
Thanks!

Are you asking whether or not a message bus is a good architecture or are you asking, given that you've chosen to use a message bus, if Camel / ServiceMix is the right solution?
I'll take a stab at both - IMHO message-based architectures (specifically event-driven architectures) are great. Do a search on Event Driven Architectures on infoq.com for a wealth of info. Provides fantastic decoupling, scalability, separation of concerns, etc.
Will it be fast? Too many factors there to answer. It will certainly be a bit more overhead than a peer-to-peer rest call, but maybe not enough to matter. Also depends on what messaging broker you choose, if you need selective queues, etc etc etc. I use Camel for a solution like this and it's 'fast enough' for what I need. You'll have to try it.
Not sure how to answer 'will it work the same'.. ServiceMix uses Camel for its routing, so some of it will work the same. But ServiceMix also has built in ActiveMQ, so if you decide to use something other the ActiveMQ (eg RabbitMQ) then it will work differently :)

Java EE alternative to publish subscribe using JMS

I'm new to Java EE and have been looking for a common way to implement a publish/subscribe type model without having to use JMS. I'm interested in creating a simple app that displays strings as they are push from the server to the client. I want to investigate was of doing this without polling to reduce unnecessary requests. The events would be varied quite a bit so I don't think polling over a set amount of time would be the best solution but the client should display the event immediately.
I've read about different ways of doing this outside of Java EE such as HtML5 with sockets api. But I want to know how to do this in Java EE, I'm assuming there is something very common that does this but I have not come across it yet. Really I'm just looking for the technology name so that I can do further research on its implementation.

Maybe Hazelcast is worth for you to have a look at. It offers an easy to use Distributed Topic feature for publish/subscribe messaging.
A simple example from the docs:
import com.hazelcast.core.Topic;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.MessageListener;
public class Sample implements MessageListener {
public static void main(String[] args) {
Sample sample = new Sample();
Topic topic = Hazelcast.getTopic ("default");
topic.addMessageListener(sample);
topic.publish ("my-message-object");
}
public void onMessage(Object msg) {
System.out.println("Message received = " + msg);
}
}

I'll answer only the Java EE part, as I'm not familiar with the Apple technology you quoted.
It seems to me enterprise Java is not appropriate for this kind of task. The main use case for a Java EE application is: "lots of users do lots of small, mostly independent tasks on a centralized application". Java EE provides means for the central application to scale to any number of users. And it's mostly users who initiate and steer the dialogue.
Your use case, however, requires the server to be the active part. You can, of course, run almost any kind of logic on a Java EE application server, asynchronous tasks, too, but it doesn't mean you should.

I am not an expert on this topic but since no one has answered I will try to explain what I know.
First of all J2EE uses JMS spec as its fundamental publish/subcribe mechanism. There are various JMS brokers out there. The important point here is that some of these brokers are not specifically tied to any J2EE application server and can work stand alone. Check out Apache's ActiveMQ and it works well as a standalone JMS broker. It has bindings to many languages a well. So you can freely compose a non J2EE architecture using a JMS broker.
Second, there are other message queue brokers that comply to other standards that can be used within a J2EE architecture as well. DDS (Data Distribution Service) is an example. It is an OMG standard and has bindings for Java and can be used in a J2EE architecture if desired.
Third, Web Services standards define a WS-Notification Broker standard. Again this is not part of J2EE as far as I know but is supported by many SOA providers.
So you have many alternatives that can be freely mixed in J2EE architecture.
I hope this helps.

Redis has a Publish/Subscribe mechanism that may be of interest: http://redis.io/topics/pubsub

What remoting approach for Java application would you recommend?

I wonder how is the best way to integrate Java modules developed as separate J(2)EE applications. Each of those modules exposes Java interfaces. The POJO entities (Hibernate) are being used along with those Java interfaces, there is no DTO objects. What would be the best way to integrate those modules i.e. one module calling the other module interface remotely?
I was thinking about: EJB3, Hessian, SOAP, JMS. there are pros and cons of each of the approaches.
Folks, what is your opinion or your experiences?

Having dabbled with a few of the remoting technologies and found them universally unfun I would now use Spring remoting as an abstraction from the implementation.
It allows you to concentrate on writing your functionality and let Spring handle the remote part with some config. you have the choice of several implementations (RMI, Spring's HTTP invoker, Hessian, Burlap and JMS). The abstraction means you can pick one implementation and simply swap it if your needs change.
See the SpringSource docs for more information.

The standard approach would be to use plain RMI between the various service components but this brings issues of sharing your Java interfaces and versioning changes to your domain model especially if you have lots of components using the same classes.
Are you really running each service in a separate VM? If these EJBs are always talking to each other then you're best off putting them into the same VM and avoiding any remote procedure calls as these services can use their LocalInterfaces.
The other thing that may bite you is using Hibernate POJOs. You may think that these are simple POJOs but behind the scenes Hibernate has been busy with CGLib trying to do things like allow lazy initialization. If these beans are serialzed and passed over remote boundaries then you may end up with odd Hibernate Exception getting thown. Personally I'd prefer to create simple DTOs or write the POJOs out as XML to pass between components. My colleagues would go one step further and write custom wire protocols for transferring the data for performance reasons.
Recently I have been using the MULE ESB to integrate various service components. It's quite nice as you can have a mix of RMI, sockets, web services etc without having to write most of the boiler plate code.
http://www.mulesource.org/display/COMMUNITY/Home

Why would you go with anything other than the simplest thing that works?
In your case that sounds like EJB3 or maybe JMS, depending on whether the communication needs to be synchronous or asynchronous.
EJB3 is by far these easiest being built on top of RMI with the container providing all the additional features you might need - security, transactions, etc. Presumably your POJOs are in a shared jar and therefore can simply be passed between your EJBs, although I tend towards passing value objects myself. The other benefit of EJB is, when done right, that it's the most performant (that's just my opinion btw ;-).
JMS is a little more involved, but not much and a system based on asynchronous communication affords certain niceties in terms of parallelizing tasks, etc.
The performance overhead of web-services, the inevitable extra config and additional points of failure make them, IMHO, not worth the hassle unless you've a requirement that mandates their use - I'm thinking interop with non-Java clients or providing data to external parties here.

If you need network communication between Java-only applications, Java RMI is the way to go. It has the best integration, most transparency and the least overhead.
If, however, some of your clients aren't Java-based, you should probably consider other options (Java RMI actually have an IIOP-dialect, which allows it to interact with CORBA, however - I wouldn't recommend doing this, unless it's for some legacy-code integration). Depending on your needs, webservices are probably your friend. If you are conserned with the networkload, you could go webservices over Hessian.

You literally mean remotely? As in running in a different environment with therefore different availability characteristics? With network overheads?
Assuming "yes" my first step would be to take a service approach, set aside the invocation technology for a moment. Just consider the design and meaning of your services. You know they are comparativley expensive to invoke, hence small busy interfaces tend to be a bad thing. You know that the service system might fail between invocations, so you may favour stateless services. You may need to retry requests after failure, so you may favour idempotent service designs.
Then consider availability relationships. Can your client work without the remote system. In some cases you simply can't progress if the remote system isn't available (eg. can't enable the employee if you can't get to the HR system) in other cases you can adopt a "fire-and-tell-me-later" philosophy; queue up the requests and process responses later.
Where there is an availability depdency, then simply exposing a synchronous interface seems to fit. You can do that with SLSB EJBs, if everything is Java EE, that works. I tend to generalise expecting that if my services are useful then non Java EE clients may want them too. So SOAP (or REST) tends to be useful. These days adding a web service interface to your SLSB is pretty trivial.
But my pet theory is that any sufficiently large IT system ends up needing aynch communications: you need to decouple the availability contraints. So I would tend to look for a JMS-style relationship. An MDB facade in front of your services, or SOAP/JMS is not too hard to do. Such an approach tends to highlight the failure-case design issues that were probably lurking anyway, JMS tends to make you think: "suppose I don't get an answer? suppose my answer comes late?"

I would go for SOAP.
JMS would be more efficient but you would need to code up an message driven bean for each interface.
SOAP on the other hand comes with lots of useful toolkits that will generate your message definition (WSDL) and all the neccesary handlers (client and server) when given an EJB.
With soap you can (but dont have to) deal with certificate security and secure connections over public networks. As the default protocol is HTTP over port 80 you will have minimal pain with firewalls etc. SOAP is also great for hetrogenious clients (in your case anything that isn't J2EE) with good support for most common languages on most common platforms.

Message Oriented Middleware (MoM) Vs. Enterprise Service Bus (ESB)

I come from a background of MoM. I think I understand ESB conceptually. However, I'm not too sure about the practical differences between the two when it comes to making a choice architecturally.
Here is what I want to know
1) Any good links online which can help me in this regard.
2) Can someone tell me where it makes sense to use one over the other.
Any help would be useful.

Messaging tends to concentrate on the reliable exchange of messages around a network; using queues as a reliable load balancer and topics to implement publish and subscribe.
An ESB typically tends to add different features above and beyond messaging such as orchestration, routing, transformation and mediation.
I'd recommend reading about the Enterprise Integration Patterns which gives an overview of common patterns you'll tend to use in integration problems which are all based above a message bus (though can be used with other networking technologies too).
For example using open source; Apache ActiveMQ provides a loosely coupled reliable exchange of messages. Then you can use Apache Camel to implement the Enterprise Integration Patterns for smart routing, transformation, orchestration, working with other technologies and so forth.

I put MOM solutions and ESB solutions on two distinct planes.
I consider MOM a building block for ESB solutions. In fact, ESB solutions reach their own loose coupling and asynchronous communication capabilities, just using the paradigm offered by the specific MOM implementation.
Therefore, MOMs represent solutions for data/events distribution at customized level of QoSs (according to the specific vendor implementation), instead ESBs represent solutions providing capabilities to realize complex orchestrations in a SOA scenario (where we have multiple providers offering their services, and multiple consumers interested in consuming the services offered by the first ones).
Complex orchestrations imply communication between legacy systems, everyone of these with its own data domain representation (rules and services on specific data) and its own communication paradigm (one consumer interact with the ESB using CORBA, another one using WS, and so on).
It is clear that ESB represents a more complex architectural solution aimed to provide the abstraction of data-bus (such as the electronic buses that everyone have in his own pc), able to connect a plethora of service providers to a not well specified plethora of service consumers, hiding heterogeneity in (i) data representation and (ii) communication.
Sorry for the long post, but the concepts are complex and it is very difficult to be effective and efficient in a short statement.

An ESB is typically a layer that routes, logs, transforms, and performs other 'technical' (i.e. non-business) functions on messages. It could process messages from a messaging system (such as something JMS-based), or it could work with other types of message (such as SOAP-based web services). In that respect, it's more general than MoM.
Disclaimer: I am an IBM WebSphere consultant - although I am not contributing here in an official capacity.

ESB with web services in its true form provides Application loose coupling by sending the data through one the elements of the message.
MOM provides not only Application Loose coupling but process loose coupling along.
ESB comes with additional features supporting Governance centric approach.
Both can be used independently or together depending upon the scenario.

IBM and Oracle have SOA certifications. Since they're the leaders in the marketplace (Gartner Magic Quadrant), I would read about how they define SOA and ESBs (along with methodology and the components needed to support SOA like Governance, Registry, etc etc)

EBS is just yet another buzzword, as is SOA 2.0.
You can have a ESB system easily implemented with normal Web Services with a queue behind them.
You can have message routing and or orchestration with SOA 1.0 (Tibco, BizzTalk), one things does not stop the other really. More importantly, it is the semantics given to the messages exchanged in the system that play an important role, in this case events. Messages as events, are triggers about something that happened in your system, so the context is different.