I am building a social networking kind of site. I am looking for a
highly scalable free and open source framework for event processing.
e.g when a user does some action on website, it would trigger an
event of a particular type for backend. A number of listeners will be waiting
for this type of event and as soon as those listen to event, they would do
some application logic for that event e.g. sending emails/sms, or data mining or start a bulb
.. literally anything.
Does anyone know any such framework? Let me know if I am not clear enough.
Thanks,
Nilesh
Look at the Axon framework.
Axon Framework helps build scalable, extensible and maintainable
applications by supporting developers apply the Command Query
Responsibility Segregation (CQRS) architectural pattern. It does so by
providing implementations of the most important building blocks, such
as aggregates, repositories and event buses (the dispatching mechanism
for events). Furthermore, Axon provides annotation support, which
allows you to build aggregates and event listeners withouth tying your
code to Axon specific logic. This allows you to focus on your business
logic, instead of the plumbing, and helps you to make your code easier
to test in isolation.
JMS provides this. Send a message to a topic, and all the listeners on that topic will receive the message.
There are several free implementations available (ActiveMQ, JBoss Messaging, etc.)
Maybe Hazelcast is interesting for you, e.g. it offers distributed listeners and events among other interesting features for distributed applications like distributed maps, locks, distributed topic for publish/subscribe messaging etc.
Hazelcast allows you to register for entry events to get notified when entries added, updated or removed. Listeners are cluster-wide. When a member adds a listener, it is actually registering for events originated in any member in the cluster. When a new member joins, events originated at the new member will also be delivered.
Related
So I'm a bit new with CQRS (not totally a beginner though). I'm trying to understand the best practices when it comes to aggregates interaction. I read a bit about using Integration Events (instead of Domain Events) in these situation, also a bit about Domain Services (that would supposedly link the 2 aggregates) but couldn't find any good definitive answer anywhere (especially not on the axonIQ Getting Started guide
Also another not too related question is that in layered architecture usually we have the controller directly linked to a service and this service can interact with other services (or repos) while with CQRS the controller is usually sending a command to the aggregate. So if my api call needs to interact with 2 aggregates do I have to build a middle-man service that would send commands (or listen to events) from the 2 services?
The interaction between components in a CQRS system can happen on a couple of levels.
On way to think about it is as Maxime suggest, with Microservices, very clearly showing the messaging focus of it all.
Regardless though, this can just as simply happen within one Application/Monolith which has several Aggregate types that together need to trigger some operation.
I feel that Maxime is providing you the answer you need. The Aggregate instances which you send commands to, act on their own and do not tie in to one another directly, at all. You'd thus react on the events as the driving force the start an interaction between both.
You can either do this by having a Event Handling Component which listen to both the events and performs the business transaction you're dealing with.
If the business transaction is a little more complex, looking at Saga's might be a good start.
Lastly, you state the 'Getting Started' part of the Axon Reference Guide is not clear about this topic. I think that's a valid conclusion, as from Axon's perspective this is not part of the Getting Started. Take a look at the Saga portion of the guide to get an idea of the interaction between Aggregates and/or Bounded Contexts.
If you think of this in term of microservices (which is a philosophy that fits CQRS very well) you should have one aggregate for one microservice. So you can't communicate between aggregates in memory because they're not part of the same process. A good way to do it is by using events that you can publish in a event bus. So the client send a command to "aggregate A" using the API of this microservice (i.e. "microservice A") (or maybe an API gateway). Then "aggregate A" is saved and the events generated by "aggregate A" are published to the event bus so that some process (aka. event handler) in "microservice B" can catch the event(s) and send the appropriate commands to "aggregate B".
It's just one way to do it there is many more and it can be very more complex than that, but I hope it's helping getting the big picture.
I would like to test a distributed algorithm that supposed to run over multiple servers (each server running the same code and logic). The end-points will communicate by broadcasting messages to each other.
For the purpose of pre-testing the algorithm I thought of developing a single process application where each end-point is simulated by a single thread.
Is there any framework that provides something similar in terms of just defining how many threads, implementing the messages and the code that will be executed by each thread?
Thanks
You might find that Akka does what you need. Akka uses actors as the implementation of business logic - these actors react to events produced by other actors.
Akka provides this API and deals with the coordination of the actors - while Threads must be used underneath, the developer doesn't have to deal with them.
The final benefit in the context of your question is that Akka can be distributed over multiple machines - I don't believe this change from single-machine to multiple-machine involves much if any modification of the program. I assume you need to ensure your events implement Serializable.
I work on a data processing application in which concurrency is achieved by putting several units of work on a message queue that multiple instances of a message driven bean (MDB) listen to. Other than achieving concurrency in this manner, we do not have any specific reason to use the messaging infrastructure and MDBs.
This led me to think why the same could not have been achieved using multiple threads.
So my question is, in what situations can asynchronous messaging (e.g. JMS) be used as an alternative to mutithreading as a means to achieve concurrency ? What are some advantages/disadvantages of using one approach over another.
It can't be used as an alternative to multithreading, it is a way of of implementing multithreading. There are three basic kinds of solutions here:
You are responsible for both ends of the queue;
You are responsible for sending data; or
You are responsible for receiving data.
Receiving data is the kicker here because there's really no way of doing that without some form of multithreading/multiprocessing otherwise you'll only be processing one request at a time. Sending data without multithreading is much more viable but there you're only really pushing the responsibility for dealing with those messages to an external system. So it's not an alternative to multithreading.
In your case with message driven beans, the container is creating and managing threads for you so it's not an alternative to multithreading, you're simply using someone else's implementation.
There are two additional bonuses that I don't think has been mentioned: Transactions and durability.
While it isn't required and quite often isn't the default configuration, JMS providers can be configured to persist the messages and also to participate in a XA transaction with little or no code changes.
In an EJB container, actually, there is no alternative, since you're not allowed to create your own threads in an EJB container. JMS is doing all of that work for you, at a cost of running it through the queue processor. You could also create a Java Connector, which has a more intimate relationship with the container (and thus, can have threads), but it's a lot more work.
If the overhead of using the JMS queue isn't having a performance impact, then it's the easiest solution.
Performance-wise multi-threading should be faster than any messaging, because you add an additional network layer with messaging.
Application-wise messaging helps you to avoid locking and data sharing issues as there is no common object.
From a scaling perspective messaging is a lot better as you can configure just more nodes on several server by configuring the message service instead of changing the application.
Messaging can reduce number of errors in multithreaded applications greatly, since it reduces risk of data races. It also simplifies adding new threads without changing the rest of app.
Although I think JMS is slightly misused here. java.util.concurrent's thread-safe queues and libraries like jetlang may provide you better performance.
Using multi-threading you can achieve concurrency by sharing core of CPU. But if you use JMS instead you can balance the load and can delegate the task to other system.
e.g. Suppose your application demands to send email on completion of certain task. And you want to send email concurrently. Either you can pull a thread and process it asynchronously. Or you can delegate this task of mail sending to other system using JMS. No of receiver threads can be configurable in jms. Also multiple nodes can listen to same JMS queue which balance the loads. And you can use further applications like persistent queue, transaction managed queue as per application.
In simple words, JMS can be better alternative to multi-threading depends on application architecture
Building a client-side swing application what should be notified on a bus (application-wide message system, similar in concept to JMS but much simpler) and what should be notified using direct listeners?
When using a bus, I always have an unescapable feeling of "I have no idea who uses that and where". Also, no set order, hard to veto events, hard to know exactly what's going on at a set time.
On the other hand, using listeners means either directly referencing the source object (coupling) or passing the reference through myriad conversions (A--b_listener-->B, B--c_listener-->C only because a needs to know something only C can to tell, but B has no interest in).
So, are there any rule of the thumb regarding this? Any suggestion how to balance?
In my experience, trying to make Swing do something it wasn't designed for, or doesn't want you to do, is extremely painful.
I would go with the simplest thing that would work; keep your code clean, do it the "Swing Way" until you start seeing problems.
Event buses are very, very useful tools for providing decoupling in certain architectures. Listeners are easy to implement, but they have significant limitations when your object and dependency graph gets large. Listeners tend to run into problems with cyclic dependencies (events can 'bounce' in odd ways, and you wind up having to play games to ensure that you don't get stuck. Most binding frameworks do this for you, but there's something distasteful about knowing that listener events are shooting off into a million places).
I make this kind of decision based on project size and scalability. If it's a big app, or there are aspects of the app that can by dynamic (like plugin modules, etc...) then a bus is a good way to keep the architecture clean (OSGI-like module containers are another approach, but heavier weight).
If you are considering a bus architecture, I recommend that you take a look at the Event Bus project - it works very well with Swing and provides a robust, out of the box solution for what you are asking about.
The convention in Java Swing is to use listeners heavily. Sticking with the convention improves maintainability but stifles innovation.
I've not encountered the bus approach in Swing, but I find it interesting.
Well, I can imagine the approach where models are updated using BUS like system and events from models are delegated using listeners. Simple scenario: I got server side which represents producer of data. Then on client side a got consumer interface which consumes all incoming messages and transform them into my internal messages/DTOs and push them into bus which distributes them into application model(s). These model process incoming messages and decide to notify interested components using listeners.
I am looking for lightweight messaging framework in Java. My task is to process events in a SEDA’s manner: I know that some stages of the processing could be completed quickly, and others not, and would like to decouple these stages of processing.
Let’s say I have components A and B and processing engine (be this container or whatever else) invokes component A, which in turn invokes component B. I do not care if execution time of component B will be 2s, but I do care if execution time of component A is below 50ms, for example. Therefore, it seems most reasonable for component A to submit a message to B, which B will process at the desired time.
I am aware of different JMS implementations and Apache ActiveMQ: they are too heavyweight for this. I searched for some lightweight messaging (with really basic features like messages serialization and simplest routing) to no avail.
Do you have anything to recommend in this issue?
Do you need any kind of persistence (e.g. if your JVM dies in between processing thousands of messages) and do you need messages to traverse to any other JVMs?
If its all in a single JVM and you don't need to worry about transactions, recovery or message loss if a JVM dies - then as Chris says above, Executors are fine.
ActiveMQ is pretty lightweight; you can use it in a single JVM only with no persistence if you want to; you can then enable transactions / persistence / recovery / remoting (working with multiple JVMs) as and when you need it. But if you need none of these things then its overkill - just use Executors.
Incidentally another option if you are not sure which steps might need persistence/reliability or load balancing to multiple JVMs would be to hide the use of middleware completely so you can switch between in memory SEDA queues with executors to JMS/ActiveMQ as and when you need to.
e.g. it might be that some steps need to be reliable & recoverable (so needing some kind of persistence) and other times you don't.
Really lightweight? Executors. :-) So you set up an executor (B, in your description), and A simply submits tasks to the executor.
I think Apache Camel covers all your needs. It's works within the JVM and supports SEDA style (http://camel.apache.org/seda.html) and simpe routing. Can be used on it's own, or with spring, with a JMS provider or other adaptors.
Sorry for resurrecting an old thread, but maybe it helps somebody else reading it... I think FFMQ is a good candidate for a lightweight messaging framework.
UPDATE: however I'm not sure if it supports redelivery delays (the dead-letter-queue problem). I would find this usable even for lightweight providers. But I guess it could be possible with a combination of MessageSelector query and message properties.
For help to somebody else read this thread:
One of the lightest messaging framework is Mbasseder.
MBassador is a very light-weight message (event) bus implementation following the publish subscribe pattern. It is designed for ease of use and aims to be feature rich and extensible while preserving resource efficiency and performance.
The core of MBassador's high performance is a specialized data structure that minimizes lock contention such that performance degradation of concurrent access is minimal.
Features: Declarative listener definition via annotations, sync and/or async event delivery, weak-references, message filtering