We Keep Coding

2 programs that send messages to each other in Java [duplicate]

I have the following situation:
I have 2 JVM processes (really 2 java processes running separately, not 2 threads) running on a local machine. Let's call them ProcessA an ProcessB.
I want them to communicate (exchange data) with one another (e.g. ProcessA sends a message to ProcessB to do something).
Now, I work around this issue by writing a temporary file and these process periodically scan this file to get message. I think this solution is not so good.
What would be a better alternative to achieve what I want?

Multiple options for IPC:
Socket-Based (Bare-Bones) Networking
not necessarily hard, but:
might be verbose for not much,
might offer more surface for bugs, as you write more code.
you could rely on existing frameworks, like Netty
RMI
Technically, that's also network communication, but that's transparent for you.
Fully-fledged Message Passing Architectures
usually built on either RMI or network communications as well, but with support for complicated conversations and workflows
might be too heavy-weight for something simple
frameworks like ActiveMQ or JBoss Messaging
Java Management Extensions (JMX)
more meant for JVM management and monitoring, but could help to implement what you want if you mostly want to have one process query another for data, or send it some request for an action, if they aren't too complex
also works over RMI (amongst other possible protocols)
not so simple to wrap your head around at first, but actually rather simple to use
File-sharing / File-locking
that's what you're doing right now
it's doable, but comes with a lot of problems to handle
Signals
You can simply send signals to your other project
However, it's fairly limited and requires you to implement a translation layer (it is doable, though, but a rather crazy idea to toy with than anything serious.
Without more details, a bare-bone network-based IPC approach seems the best, as it's the:
most extensible (in terms of adding new features and workflows to your
most lightweight (in terms of memory footprint for your app)
most simple (in terms of design)
most educative (in terms of learning how to implement IPC). (as you mentioned "socket is hard" in a comment, and it really is not and should be something you work on)
That being said, based on your example (simply requesting the other process to do an action), JMX could also be good enough for you.

I've added a library on github called Mappedbus (http://github.com/caplogic/mappedbus) which enable two (or many more) Java processes/JVMs to communicate by exchanging messages. The library uses a memory mapped file and makes use of fetch-and-add and volatile read/writes to synchronize the different readers and writers. I've measured the throughput between two processes using this library to 40 million messages/s with an average latency of 25 ns for reading/writing a single message.

What you are looking for is inter-process communication. Java provides a simple IPC framework in the form of Java RMI API. There are several other mechanisms for inter-process communication such as pipes, sockets, message queues (these are all concepts, obviously, so there are frameworks that implement these).
I think in your case Java RMI or a simple custom socket implementation should suffice.

Sockets with DataInput(Output)Stream, to send java objects back and forth. This is easier than using disk file, and much easier than Netty.

I tend to use jGroup to form local clusters between processes. It works for nodes (aka processes) on the same machine, within the same JVM or even across different servers.
Once you understand the basics it is easy working with it and having the options to actually run two or more processes in the same JVM makes it easy to test those processes easily.
The overhead and latency is minimal if both are on the same machine (usually only a TCP rountrip of about >100ns per action).

socket may be a better choice, I think.

Back in 2004 I implement code which do the job with sockets. Until then, many times I search for a better solution, because socket approach triggers firewall and my clients worry. There is no better solution until now. Client must serialize your data, send and server must receive and unserialize.
It is easy.

Parallel job execution with split-and-aggregate in Java

We are working on rewrite of an existing application, and need support for high number of read/write to database. For this, we are proceeding with sharding on MySQL. Since we are allowing bulk APIs for read/write, this would mean parallel execution of queries on different shards.
Can you suggest frameworks which would support the same in Java, mainly focussing on split-and-aggregate jobs. Basically I will define two interfaces ReadTask and WriteTask, and implementation of these tasks will be jobs and they would be submitted as a list for parallel execution.
I might not have termed this question in the right way, but I hope you got the context from the description. Let me know if there is any info needed for answer.

BLUF: This sounds like a common processing pattern in Akka.
This sounds like a Scatter-Gather patterned API.
If you have 1 job, you should first answer if that job will touch only one shard or more? If it will touch many shards you may choose to reject it (allowing only single-shard actions) or you may choose to break it up (scatter) it across other workers.
Akka gives you APIs, especially the Streaming API, that talk about this style of work. Akka is best expressed in Scala, but it has a Java API that gives you all the functionality of the Scala one. That you are talking about "mapping" and "reducing" (or "folding") data, these are functional operations and Scala gives you the functional idioms.
If you scatter it across other workers, you'll need to communicate the manifest of jobs to the gather side of the system.
Hope that's helpful.

You can use the ThreadPoolExecutor & Executors(factory) in Java to create Thread pools to which you can submit your read & write tasks. It allows for Runnable & Callable based on your situation.

How to delay processing reasonably in Java EE context?

Within a Java EE 5 environment I have the problem to ensure the existence of some data written by another part before continue processing my own data.
Historically (J2EE time), it was done by putting the data object to be processed into an internal JMS queue after waiting for e.g. 500ms via Thread.sleep.
But this does not feel like the best way to handle that problem, so I have 2 questions:
Is there any problem with using the sleep method within an Java EE context?
What is a reasonable solution to delaying some processing within an Java EE 5 application?
Edit:
I should have mentioned, that my processing takes place while handling objects from a JMS queue via an MDB.
And it may be the case, that the data for which I'm waiting never shows up, so there must be some sort of timeout, after which I can do some special processing with my data.

You can use EJB TimerService feature. Using threads in a managed environment should be avoided.

I agree with #dkaustubh about timers and avoiding threads manipulation in JavaEE.
Another possibility is to use JMS queue with delayed delivery. Although it is not a part of JavaEE API, most of messaging systems vendors supports it. check here.

I think, its possible with some advanced Threading approach. More than thinking on manual synchronizations and thread management, you can always use the Java Concurrent package.
Future can be one of the ways to do this. Please refer to Java Concurrent package.

Use notifications and Object#wait() / Object#notifyAll()
i.e. Multithreaded, the producer notifies the consumer.

java API or framework for queue processing

i need an open-source java API or framework for processing items in a queue. i can develop something myself, but do not want to re-invent the wheel (and i don't have much experience in multi-threading). is there such a thing?
the closest solution that i can think of is a business process management (BPM) solution.
right now, i am using multiple Quartz jobs to process the items in my queue. it is not really working out because of scalability and concurrency issues.

Sounds like you'd want to use an Executor

A queue of what sort? How many items? Is Quartz not working out because it's too big or too small?
I'd give some serious thought to using message queues in something like OpenMQ.

You can use JMS with ActiveMQ and can create optimized queue system as well as ESB. And want to manage workflow based system then tpdi is right. Use JBoss jbpm.
You can process JMS messages with ThreadPool also. In this case, you can use Executors.

Would the actor model fit your process? It's based around the idea of asynchronously passing messages between other actors. So you can set up a simple state machine to model your process and have all the transitions handled concurrently.

You need to determine if the problem in is the framework you are using or your code. I suggest you measure how fast your application is running and how fast your framework will go if its not doing anything at all. (just passing trivial tasks around) You should be able to perform between 100K to 1 million tasks per second using your in process framework. Even using JMS you should be able to achieve 10K messages per second. If you need to do closer to 10 million tasks per second, I suggest you try grouping your tasks together so each task does more work.
I would be very surprised if your framework was the bottleneck in which case I would suggest using an Executor.
If the framework isn't the cause of your scalability and concurrency issues (which is more likely) you need to restructure your code so it can run for longer periods of time without inter dependencies. i.e. you have to fix your code, a framework won't do that for you.

I know it is 5 years late, but this might help someone else that has been driven into this question.
Nowadays, there is http://queues.io and it contains a whole lot of queuing (and messaging) frameworks...

Java: What object is more appropriate?

I am writing an application similar to a chat server-client pair.
I am planning on having a central Object which will hold new messages received from clients until they are dealt with by the main thread.
My application is multi-threaded. Each client will be on its own thread, so multiple threads will be adding messages to this central Object.
The main thread will check this Object for messages, remove the "oldest" and deal with it appropriately. Preferably I would like the messages to be handled in the same order they were added (FIFO).
What type of Object is most appropriate to hold the new messages? I looked into Vectors and ArrayLists, but I am confused about the synchronization aspect. I never worked with synchronization or threads before.
Thank you

ConcurrentLinkedQueue sounds likely.
More info here: How to use ConcurrentLinkedQueue?
Javadoc here: http://download-llnw.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html

If this is to become more than a little toy project, you should look into JMS which solves all of the little problems that you're not aware of, yet.
A good implementation of JMS is Apache ActiveMQ (not related to IBM's MQSeries).

You may also consider jgroups for this project.
JGroups is an open source reliable group communication toolkit. Its reliable and simple to use. Here is a basic chat example in the tutorial of same.

While not a direct answer (As #Alison provided a decent enough of one provided you don't need to persist messages) always have a look at the java.util.concurrent package whenever you need data structure classes or utility classes to help with multi-threaded coded: http://download-llnw.oracle.com/javase/6/docs/api/java/util/concurrent/package-summary.html