I've been reading something about, found some libraries which really messed with my thoughts, like Akka, Quasar, Reactor and Disruptor, Akka and Quasar implements the Actor Pattern and the Disruptor is a Inter-Thread Messaging library, and Reactor is based on . So what are the advantages, use cases for using a message driven architecture over simple method calls?
Given a RabbitMQ queue listener, I receive a message from the method, decide which Type the RabbitMQ message is (NewOrder,Payment,...).
With a Message Driven library I could do.
Pseudo code:
actor.tell('decider-mailbox',message)
Which basically says "I'm putting this message here, when you guys can handle it, do it") and so on until it gets saved.
And the actor is ready again to receive another message
But with directly calling the method like messageHandler.handle(message), wouldn't be better and less abstracted ?
The Actors Model looks a lot like people working together; it is based on message-passing but there is much more to it and I'd say that not all message-passing models are the same, for example Quasar actually supports not only Erlang-like actors but also Go-like channels which are simpler but don't provide a fault-tolerance model (and fibers BTW, that are just like threads but much more lightweight, which you can use even without any message-passing at all).
Methods/functions follow a strict, nestable call-return (so request-response) discipline and usually don't involve any concurrency (at least in imperative and non-pure functional languages).
Message passing instead, very broadly speaking, allows looser coupling because doesn't enforce a request-response discipline and allows the communicating parties to execute concurrently, which also helps in isolating failures and in hot-upgrades and generally maintenance (for example, the Actors Model offers these features). Often message passing will also allow looser data contracts by using a more dynamic typing for messages (this is especially true for the Actors Model where each party, or actor, has a single incoming channel, that is his mailbox).
Other than that the details depends a lot on the messaging model/solution you're considering, for example the communication channels can synchronize the interacting parts or have limited/unlimited buffering, allow multiple source and/or multiple producers and consumers etc.
Note that RPC is really message passing but with a strict request-response communication discipline.
This means that, depending on the situation, one or the other may suit you better: methods/functions are better when you're in a call-return discipline and/or you're simply making your sequential code more modular. Message-passing is better when you need a network of potentially concurrent, autonomous "agents" that communicate but not necessarily in a request-response discipline.
As for the Actors Model I think you can build more insight about it for example by reading the first part of this blog post (notice: I'm the main author of the post and I'm part of the Parallel Universe - and Quasar - development team):
The actor model is a design pattern for fault-tolerant and highly scalable systems. Actors are independent worker-modules that communicate with other actors only through message-passing, can fail in isolation from other actors but can monitor other actors for failure and take some recovery measures when that happens. Actors are simple, isolated yet coordinated, concurrent workers.
Actor-based design brings many benefits:
Adaptive behaviour: interacting only through a message-queue makes actors loosely coupled and allows them to:
Isolate faults: mailboxes are decoupling message queues that allow actor restart without service disruption.
Manage evolution: they enable actor replacement without service disruption.
Regulate concurrency: receiving messages very often and discarding overflow or, alternatively, increasing mailbox size can maximize concurrency at the expense of reliability or memory usage respectively.
Regulate load: reducing the frequency of receive calls and using small mailboxes reduces concurrency and increases latencies, applying back-pressure through the boundaries of the actor system.
Maximum concurrency capacity:
Actors are extremely lightweight both in memory consumption and
management overhead, so it’s possible to spawn even millions in a
single box.
Because actors do not share state, they can safely run in parallel.
Low complexity:
Each actor can implements stateful behaviour by mutating its private state without worrying about concurrent modification.
Actors can simplify their state transition logic by selectively receiving messages from the mailbox in logical, rather than arrival order.
The difference is that the processing takes place in a different thread so the current one is ready to receive and forwards the next message. When you call the handler from the current thread it is blocked until processing is finished.
In a way it is just a matter of defining abstractions. Some say that originally object oriented programming was actually supposed to be based on message passing, and calling a method on an object would have the semantics of sending it a message (with similar async non-blocking behavior as in actors).
The way we implemented OO in most popular languages is such that it became what it is today - a "synchronous blocking order" to an object, controlled and ran from the same execution context (thread/process) as the caller. This is nice because it is easy to understand, but it has its limitations when designing concurrent systems.
In theory, you could create a language with similar syntax as Java, but give it different semantics - making object.method(arg) actually internally be something similar to actor.tell(msg). There are a lot of idioms that try to hide asynchronous calling and message passing behind simple method invocations, but as always it depends on the use case.
Akka provides a nice new syntax which makes it clear that what we are doing is something completely different than invoking methods on an object, in part to cause less confusion and make the message passing more explicit. In the end, you are stating the same thing - you are sending a message to an actor in the system, but you are doing it with less constraints than if you were calling one of its methods directly.
Related
This question already has answers here:
How does Actors work compared to threads?
(2 answers)
Closed 5 years ago.
I'm trying to understand the difference in definition of Actors and threads. Some articles say that actors are lightweight threads, while other articles states that actors are not threads. I also understand that a thread can run multiple actors. I'm confused as to how to exactly differentiate actors from threads. Please help me understand thanks!
Actors are at a higher level of abstraction when compared to Threads.
Thread is a JVM concept, whereas an Actor is a normal java class that runs in the JVM and hence the question is not so much about Actor vs Thread, its more about how Actor uses Threads.
What is an Actor?
At a very simple level, an Actor is an entity that receives messages, one at a time, and reacts to those messages.
How does Actor use Threads?
When Actor receives a message, it performs some action in response. How does the action code run in the JVM? Again, if you simplify the situation, you could imagine the Action executing the action task on the current thread. Also, it is possible that the Actor decides to perform the action task on a thread pool. It does not really matter as long as the Actor makes sure that only one message is processed at a time.
I'm trying to understand the difference in definition of Actors and threads.
Honestly, they really don't have much to do with each other, so it is kinda hard to talk about their difference. It's like talking about the difference between a Toyota Camry and the color blue.
An Actor is a self-contained, encapsulated entity that communicates with other self-contained, encapsulated entities purely by sending messages.
Now, if that sounds exactly like the definition of Object to you, you are right! There is a deep connection between Actors and Objects: the Actor Model of Computation was conceived by Carl Hewitt, partially based on the Message-Directed Execution of early Smalltalks (Smalltalk-71, Smalltalk-72). Alan Kay, in turn, cites PLANNER' Goal-Directed Execution as a heavy influence on Smalltalk's Message-Directed Execution … PLANNER in turn had been designed by Carl Hewitt. (PLANNER is also the precursor to Prolog, which in turn was the precursor to Erlang; Erlang is based on Prolog, started out as a library in Prolog, and the first implementations were written in Prolog.) Also, both Carl Hewitt and Alan Kay cite the early work of Vint Cerf and Bob Kahn on what would later become the Internet as inspiration, and both of them were heavily inspired by nature, Carl Hewitt by physics and Alan Kay by microbiology. So, the deep connections and similarities are very much not surprising.
Actors and Objects are pretty much the same thing. We usually expect Message sends in Object-Oriented Systems to be synchronous, instantaneous, reliable, and ordered, while no such guarantees exist for Actors. That is the main difference between Actors and Objects: Actors are Objects where Messages may get lost, re-ordered, take a long time, and are asynchronous. (The only guarantee is that a Message may get delivered At Most Once.) Also, the Receiver of a Message typically has no implicit reference to the Sender, so if the Receiver wants to reply to the Message, the Sender needs to pass its own reference along with the Message (unlike typical Object-Oriented Systems, where I can always return something to the Sender).
An Actor can do three things, and exactly those three things and nothing more:
Create new Actors
Send Messages to Addresses of Actors it already knows (i.e. Addresses of Actors it created, and Addresses it got sent in a Message)
Designate how to handle the next Message (this is essentially how to model mutable state)
Note that in order to send a Message to an Actor, you need to have an Address for the Actor. You can only send Messages to Addresses. An Actor may have multiple Addresses, and multiple Actors may be hidden behind a single Address. Addresses are unforgeable, you cannot construct one yourself, you have to have it handed to you. (This is an important difference to pointers in C, and more like object references in Java.)
There is a very nice video of a whiteboarding session (mostly) between Carl Hewitt himself and Erik Meijer about Actors on Microsoft's Channel9 community website: Hewitt, Meijer and Szyperski: The Actor Model (everything you wanted to know, but were afraid to ask). It explains among other things the fundamental ideas behind Actors as well as the fundamental Axioms of Actors in an accessible way.
Carl Hewitt was also invited to give a lecture at Stanford's EE380 Computer Systems Colloquium, where he talked about How to Program the Many Cores For Inconsistency Robustness, including an introduction to Actors.
Threads, OTOH are just strands of execution. They are essentially nothing but an instruction pointer and a call stack. In particular, Threads don't have memory (of their own). All Threads of a Process share the same memory. Which means among other things, that a single misbehaving Thread can crash all Threads of the Process, and that all access to this shared memory must be carefully controlled.
Some articles say that actors are lightweight threads, while other articles states that actors are not threads. I also understand that a thread can run multiple actors.
You are confusing the concept of an Actor with a possible implementation.
A very simple implementation of Actors in an existing environment would be to make every Actor a Process. That's how BEAM (the dominant implementation of Erlang) works, for example. This works, because on BEAM, Processes are extremely cheap and lightweight; a process only weighs about 300 Byte, and there can be tens of millions of Processes on a cheap 32 Bit single-core laptop from 10 years ago. This would not work with Windows Processes, for example, which are much more expensive.
There is also an implementation of Erlang on the JVM (called Erjang) which uses Kilim, an implementation of extremely lightweight Actors on the JVM using bytecode re-writing. (Note that both Erjang and Kilim appear to be dead.)
If you want to implement Actors on Threads, you have the problem that Actors are completely isolated, whereas Threads are completely shared. So, you need to provide this isolation somehow.
A common way to implement Actors on systems such as the JVM, is to implement Actors as Objects, and then schedule them to run on a Thread Pool to get the asynchronous properties. That's how Akka works.
I highly recommend to read through official documentation first:
https://doc.akka.io/docs/akka/current/scala/general/actors.html
Behind the scenes Akka will run sets of actors on sets of real threads, where typically many actors share one thread, and subsequent invocations of one actor may end up being processed on different threads. Akka ensures that this implementation detail does not affect the single-threadedness of handling the actor’s state.
A thread is a "sequence of activity" - independent of specific objects. It can execute code that belongs to arbitrary objects.
Whereas an actor is about a specific object that "owns" its own "sequence of activity". But please note that this actor sequence isn't necessarily a thread. To the contrary!
Looking at this from the implementation side, one could use one thread to drive many actor objects.
I need to use memcached Java API in my Scala/Akka code. This API gives you both synchronous and asynchronous methods. The asynchronous ones return java.util.concurrent.Future. There was a question here about dealing with Java Futures in Scala here How do I wrap a java.util.concurrent.Future in an Akka Future?. However in my case I have two options:
Using synchronous API and wrapping blocking code in future and mark blocking:
Future {
blocking {
cache.get(key) //synchronous blocking call
}
}
Using asynchronous Java API and do polling every n ms on Java Future to check if the future completed (like described in one of the answers above in the linked question above).
Which one is better? I am leaning towards the first option because polling can dramatically impact response times. Shouldn't blocking { } block prevent from blocking the whole pool?
I always go with the first option. But i am doing it in a slightly different way. I don't use the blocking feature. (Actually i have not thought about it yet.) Instead i am providing a custom execution context to the Future that wraps the synchronous blocking call. So it looks basically like this:
val ecForBlockingMemcachedStuff = ExecutionContext.fromExecutorService(Executors.newFixedThreadPool(100)) // whatever number you think is appropriate
// i create a separate ec for each blocking client/resource/api i use
Future {
cache.get(key) //synchronous blocking call
}(ecForBlockingMemcachedStuff) // or mark the execution context implicit. I like to mention it explicitly.
So all the blocking calls will use a dedicated execution context (= Threadpool). So it is separated from your main execution context responsible for non blocking stuff.
This approach is also explained in a online training video for Play/Akka provided by Typesafe. There is a video in lesson 4 about how to handle blocking calls. It is explained by Nilanjan Raychaudhuri (hope i spelled it correctly), who is a well known author for Scala books.
Update: I had a discussion with Nilanjan on twitter. He explained what the difference between the approach with blocking and a custom ExecutionContext is. The blocking feature just creates a special ExecutionContext. It provides a naive approach to the question how many threads you will need. It spawns a new thread every time, when all the other existing threads in the pool are busy. So it is actually an uncontrolled ExecutionContext. It could create lots of threads and lead to problems like an out of memory error. So the solution with the custom execution context is actually better, because it makes this problem obvious. Nilanjan also added that you need to consider circuit breaking for the case this pool gets overloaded with requests.
TLDR: Yeah, blocking calls suck. Use a custom/dedicated ExecutionContext for blocking calls. Also consider circuit breaking.
The Akka documentation provides a few suggestions on how to deal with blocking calls:
In some cases it is unavoidable to do blocking operations, i.e. to put
a thread to sleep for an indeterminate time, waiting for an external
event to occur. Examples are legacy RDBMS drivers or messaging APIs,
and the underlying reason is typically that (network) I/O occurs under
the covers. When facing this, you may be tempted to just wrap the
blocking call inside a Future and work with that instead, but this
strategy is too simple: you are quite likely to find bottlenecks or
run out of memory or threads when the application runs under increased
load.
The non-exhaustive list of adequate solutions to the “blocking
problem” includes the following suggestions:
Do the blocking call within an actor (or a set of actors managed by a router), making sure to configure a thread pool which is either
dedicated for this purpose or sufficiently sized.
Do the blocking call within a Future, ensuring an upper bound on the number of such calls at any point in time (submitting an unbounded
number of tasks of this nature will exhaust your memory or thread
limits).
Do the blocking call within a Future, providing a thread pool with an upper limit on the number of threads which is appropriate for the
hardware on which the application runs.
Dedicate a single thread to manage a set of blocking resources (e.g. a NIO selector driving multiple channels) and dispatch events as they
occur as actor messages.
The first possibility is especially well-suited for resources which
are single-threaded in nature, like database handles which
traditionally can only execute one outstanding query at a time and use
internal synchronization to ensure this. A common pattern is to create
a router for N actors, each of which wraps a single DB connection and
handles queries as sent to the router. The number N must then be tuned
for maximum throughput, which will vary depending on which DBMS is
deployed on what hardware.
I've searched the site a bit for help understanding this, but haven't found anything super clear, so I thought I'd post my use case and see if anybody could shed some light.
I have a question about the scaling of jvm threads vs os threads when used in akka for io operations. From the akka site:
Akka supports dispatchers for both event-driven lightweight threads, allowing creation of millions threads on a single workstation, and thread-based Actors, where each dispatcher is bound to a dedicated OS thread.
The event-based Actors currently consume ~600 bytes per Actor which means that you can create more than 6.5 million Actors on 4 G RAM.
In this context, can you all help me understand how that matters on a workstation with only 1 processor (for simplicity). So, for my example use case, I want to take a list of say 1000 'Users' and then go query a database (or several) for various information about each user. So if I were to dispatch each of these 'get' tasks to an actor, and that actor is going to do IO, wouldn't that actor block based on the os thread limit for the workstation?
How does the akka actor model give me lift in a scenario like this? I know that I am probably missing something as I am not wildly knowledgeable on the interworkings of vm threads vs os threads, so if one of the smart folks here could spell it out for me, that would be great.
If I use Futures, don't I need to use await() or get() to block and wait for the reply?
In my use case, regardless of actors, would it end up just 'feeling' like I'm making 1000 sequential database requests?
If code snips are useful in helping me understand this, Java would be preferred as I am still coming up to speed on scala syntax - but a nice clear textual explanation of how these millions of threads can interoperate on a single processor machine while doing database IO would be fine too.
It is really hard to figure out what you are actually asking here, but here are some pointers:
If you are running on a modern JVM, there is typically a one-to-one relationship between Java threads and OS threads. (IIRC, Solaris allows you to do this differently ... but that's the exception.)
The amount of real parallelism you will get using threads, or anything built on top of threads is limited by the number of processors / cores that are available to the application. Beyond that, you will find that not all threads are actually executing at any given instant.
If you have 1000 Actors all trying to access the database "at the same time", then most of them will actually be waiting on the database itself, or on the thread scheduler. Whether this amounts to making 1000 sequential requests (i.e. strict serialization) will depend on the database and the queries / updates that the actors are doing.
The bottom line is that a computer system has hard limits on the resources available for doing stuff; e.g. number of processors, speed of processors, memory bandwidth, disc access times, network bandwidth, etc. You can design an application to be smart about the way it uses available resources, but you can't get it to use more resources than there actually are.
On reading the text that you quoted, it seems to me that it is talking about two different kinds of actors:
Thread-based actors have a 1 to 1 relationship with threads. There's no way you could have millions of this kind of actor in 4Gb memory.
Event-based actors work differently. Instead of having a thread at all times, they would mostly be sitting in a queue waiting for an event to happen. When that happened, an event processing thread would grab the actor from the queue and execute the "action" associated with the event. When the action finished, the thread moves onto another actor / event pair.
The quoted text is saying that the memory overhead of an event-based actor is ~600 bytes. They don't include the event thread ... because the event thread is shared by multiple actors.
Now I'm not an expert on Scala / Actors, but it is pretty obvious that there are certain things that you should avoid when using event-based actors. For instance, you should probably avoid talking directly to an external database because that is liable to block the event processing thread.
I think there may be a typo there. I think they meant to say:
Akka supports dispatchers for both event-driven lightweight actors,
allowing creation of millions actors on a single workstation, and thread-based Actors, where each actor is bound to a dedicated OS thread.
The event-driven actors use a thread pool - all of the (potentially millions of) actors share the same pool of threads. I'm not that familiar with Akka actors but generally you would not want to do blocking I/O with event-driven actors, otherwise you could cause starvation.
I am kicking off my final year project right now. I am going to be investigating the concurrency approaches from java and scala perspectives. Having come out of a java concurrency module, I can see why people say that the shared state threading approach is difficult to reason about. You have critical sections to worry about, run the risk of race conditions and deadlocks etc due to the non deterministic way in which java threads operate. With 1.5 this reasoning was given some clarity ,but still, far from crystal clear.
At first view, scala appears to remove this complex reasoning through the actors class. This has given the programmer the ability to develop concurrent systems from a more sequential viewpoint and easier to conceptualize. But, for this positive, am I right in saying that there are some drawbacks? For instance, say we want to sort a large list in both scenarios - with java you create two threads split the list in two, worry about the critical sections, atomic actions etc and go code. With scala, because it is "share nothing" you actually have to pass the list/2 to two actors to peform the sort operation, right?
I guess my question is that the price you pay for simpler reasoning is performance overhead of having to pass the collection to your actors, in scala?
I was thinking of doing some benchmark tests to this effect (selection sort, quick sort etc;) but because one is functional and one is imperative - I will not be comparing apples with apples from an algorithm viewpoint.
I would really appreciate any views you guys have on the above to give me some ideas to get me started.
Many thanks.
The nice thing about Scala is that you can do concurrency the Java way if you want. All the Java classes are available.
So it really boils down to the difference between a model where you have threads with concurrent access to mutable variables, and a model where you have stateful actors which send messages to each other but do not peek into each others' internals. And you're absolutely right that in some scenarios you have to trade off performance against ease of getting the code correct.
I generally find as a rough rule of thumb that if you're going to have a pile of threads spending a significant amount of time waiting for a lock to open up, using a Java model, and there is no clean way to separate the work to avoid having everyone waiting for that resource, and if the execution switches between threads quickly, then the Java model is far superior to an actor model where the actor sends an "I'm done" message back to a supervisor, which then sends out a "Here's new work!" message to an existing non-busy actor. Sorting algorithms, depending on how you envision them, can very much fall into this category.
For most everything else, the performance penalty associated with actors doesn't amount to much as far as I've seen. If you can conceive of your problem as lots and lots of reactive elements (i.e. they only need time when they've received a message), then actors can scale particularly well (millions available, though only a handful are working at any given instant); with threads, you'd need to have some sort of extra internal state to keep track of who should be doing what work, since you couldn't handle that many active threads.
I'm just going to point out here that Scala does not copy arguments passed to actors, so actors can share whatever it is passed to them.
As opposed to Erlang, it is the programmer's responsibility to avoid sharing mutable stuff. However, there is no penalty in sharing immutable stuff, since there's no need to lock it, as all accesses to it are read-only. And Scala has strong support for immutable data structures.
I would like to design a simple application (without j2ee and jms) that can process massive amount of messages (like in trading systems)
I have created a service that can receive messages and place them in a queue to so that the system won't stuck when overloaded.
Then I created a service (QueueService) that wraps the queue and has a pop method that pops out a message from the queue and if there is no messages returns null, this method is marked as "synchronized" for the next step.
I have created a class that knows how process the message (MessageHandler) and another class that can "listen" for messages in a new thread (MessageListener). The thread has a "while(true)" and all the time tries to pop a message.
If a message was returned, the thread calls the MessageHandler class and when it's done, he will ask for another message.
Now, I have configured the application to open 10 MessageListener to allow multi message processing.
I have now 10 threads that all time are in a loop.
Is that a good design??
Can anyone reference me to some books or sites how to handle such scenario??
Thanks,
Ronny
Seems from your description that you are on the right path, with one little exception. You implemented a busy wait on the retrieval of messages from the Queue.
A better way is to block your threads in the synchronised popMessage() method, doing a wait() on the queue resource when no more messages can be pop-ed. When adding (a) message(s) to the queue, the waiting threads are woken up via a notifyAll(), one or more threads will get a message and the rest re-enter the wait() state.
This way the distribution of CPU resources will be smoother.
I understand that queuing providers like Websphere and Sonic cost money, but there's always JBoss Messaging, FUSE with ApacheMQ, and others. Don't try and make a better JMS than JMS. Most JMS providers have persistence capabilities that for provide fault tolerance if the Queue or App server dies. Don't reinvent the wheel.
Reading between the lines a little it sounds like your not using a JMS provider such as MQ. Your solution sounds in the most parts to be ok however I would question your reasons for not using JMS.
You mention something about trading, I can confirm a lot of trading systems use JMS with and without j2ee. If you really want high performance, reliability and piece of mind don't reinvent the wheel by writing your own queuing system take a look at some of the JMS providers and their client API's.
karl
Event loop
How about using a event loop/message pump instead? I actually learned this technique from watching the excellent node.js video presentation from Ryan which I think you should really watch if not already.
You push at most 10 messages from Thread a, to Thread b(blocking if full). Thread a has an unbounded [LinkedBlockingQueue][3](). Thread b has a bounded [ArrayBlocking][4] of size 10 (new ArrayBlockingQueue(10)). Both thread a and thread b have an endless "while loop". Thread b will process messages available from the ArrayBlockingQueue. This way you will only have 2 endless "while loops". As a side note it might even be better to use 2 arrayBlockingQueues when reading the specification because of the following sentence:
Linked queues typically have higher
throughput than array-based queues but
less predictable performance in most
concurrent applications.
Off course the array backed queue has a disadvantage that it will use more memory because you will have to set the size prior(too small is bad, as it will block when full, too big could also be a problem if low on memory) use.
Accepted solution:
In my opinion you should prefer my solution above the accepted solution. The reason is that if it all posible you should only use the java.util.concurrent package. Writing proper threaded code is hard. When you make a mistake you will end up with deadlocks, starvations, etc.
Redis:
Like others already mentioned you should use a JMS for this. My suggestion is something along the line of this, but in my opinion simpler to use/install. First of all I assume your server is running Linux. I would advise you to install Redis. Redis is really awesome/fast and you should also use it as your datastore. It has blocking list operations which you can use. Redis will store your results to disc, but in a very efficient manner.
Good luck!
While it is now showing it's age, Practical .NET for Financial Markets demonstrates some of the universal concepts you should consider when developing a financial trading system. Athough it is geared toward .Net, you should be able to translate the general concepts to Java.
The separation of listening for the message and it's processing seems sensible to me. Having a scalable number of processing threads also is good, you can tune the number as you find out how much parallel processing works on your platform.
The bit I'm less happy about is the way that the threads poll for message arrival - here you're doing busy work, and if you add sleeps to reduce that then you don't react immediately to message arrival. The JMS APIs and MDBs take a more event driven approach. I would take a look at how that's implemented in an open source JMS so that you can see alternatives. [I also endorse the opinion that re-inventing JMS for yourself is probably a bad idea.] The thing to bear in mind is that as your systems get more complex, you add more queues and more processing busy work has greater impact.
The other concern taht I have is that you will hit limitiations of using a single machine, first you may allow greater scalability my allowing listeners to be on many machines. Second, you have a single point of failure. Clearly solving this sort of stuff is where the Messaging vendors make their money. This is another reason why Buy rather than Build tends to be a win for complex middleware.
You need very light, super fast, scalable queuing system. Try Hazelcast distributed queue!
It is a distributed implementation of java.util.concurrent.BlockingQueue. Check out the documentation for detail.
Hazelcast is actually a little more than a distributed queue; it is transactional, distributed implementation of queue, topic, map, multimap, lock, executor service for Java.
It is released under Apache license.