This question already has answers here:
java.lang.IllegalMonitorStateException: object not locked by thread before wait()?
(3 answers)
Closed 3 years ago.
I've writing a program using HiveMQ Client (an MQTT Open source implementation in Java) that involves using two multithreaded clients. One client is designated as the publisher and the other as the subscriber (I'm aware I could the same client can both publish and subscribe). I'm trying to design a test where the publisher sends 100 messages to the client. The goal is to time how long it takes to send and receive all the messages. I realized if I wanted to time how long it would take for the messages to be received, I would need to have the Subscribing thread wait until the publishing thread was ready to send the message. I decided to use wait() and notify() but I can't seem to implement it correctly. I'm aware that you need to use the same object which I tried to do but I can't get the design correct. I added snipers on code for both of the run methods of the two clients. CommonThread.java isn't actually a thread and I'm not running it but I tried to use it an in between class to be able to wait() and notify() but I'm missing something.
HiveMQ:
https://github.com/hivemq/hivemq-community-edition
https://github.com/hivemq/hivemq-mqtt-client
SubMainThread.java:
public void run() {
// Creates the client object using Blocking API
Mqtt5BlockingClient subscriber = Mqtt5Client.builder()
.identifier(UUID.randomUUID().toString()) // the unique identifier of the MQTT client. The ID is randomly generated between
.serverHost("localhost") // the host name or IP address of the MQTT server. Kept it localhost for testing. localhost is default if not specified.
.serverPort(1883) // specifies the port of the server
.addConnectedListener(context -> ClientConnectionRetreiver.printConnected("Subscriber1")) // prints a string that the client is connected
.addDisconnectedListener(context -> ClientConnectionRetreiver.printDisconnected("Subscriber1")) // prints a string that the client is disconnected
.buildBlocking(); // creates the client builder
subscriber.connect(); // connects the client
ClientConnectionRetreiver.getConnectionInfo(subscriber); // gets connection info
try {
Mqtt5Publishes receivingClient1 = subscriber.publishes(MqttGlobalPublishFilter.ALL); // creates a "publishes" instance thats used to queue incoming messages // .ALL - filters all incoming Publish messages
subscriber.subscribeWith()
.topicFilter(subscriberTopic)
.qos(MqttQos.EXACTLY_ONCE)
.send();
PubSubUtility.printSubscribing("Subscriber1");
System.out.println("Publisher ready to send: " + PubMainThread.readyToSend);
x.threadCondWait(); // <<<<< HOW TO MAKE THIS WORK
System.out.println("Back to the normal execution flow :P");
startTime = System.currentTimeMillis();
System.out.println("Timer started");
for (int i = 1; i <= messageNum; i++) {
Mqtt5Publish receivedMessage = receivingClient1.receive(MESSAGEWAITTIME,TimeUnit.SECONDS).get(); // receives the message using the "publishes" instance waiting up to 5 minutes // .get() returns the object if available or throws a NoSuchElementException
PubSubUtility.convertMessage(receivedMessage); // Converts a Mqtt5Publish instance to string and prints
}
endTime = System.currentTimeMillis();
finalTime = endTime - startTime;
System.out.println( finalTime + PubMainThread.finalTime + " milliseconds");
finalSecTime = TimeUnit.MILLISECONDS.toSeconds(finalTime);
System.out.println(finalSecTime + PubMainThread.finalSecTime);
}
catch (InterruptedException e) { // Catches interruptions in the thread
LOGGER.log(Level.SEVERE, "The thread was interrupted while waiting for a message to be received", e);
}
catch (NoSuchElementException e){
System.out.println("There are no received messages"); // Handles when a publish instance has no messages
}
subscriber.disconnect();
}
PubMainThread.java:
public void run() {
// Creates the client object using Blocking API
Mqtt5BlockingClient publisher = Mqtt5Client.builder()
.identifier(UUID.randomUUID().toString()) // the unique identifier of the MQTT client. The ID is randomly generated between
.serverHost("localhost") // the host name or IP address of the MQTT server. Kept it localhost for testing. localhost is default if not specified.
.serverPort(1883) // specifies the port of the server
.addConnectedListener(context -> ClientConnectionRetreiver.printConnected("Publisher1")) // prints a string that the client is connected
.addDisconnectedListener(context -> ClientConnectionRetreiver.printDisconnected("Publisher1")) // prints a string that the client is disconnected
.buildBlocking(); // creates the client builder
publisher.connect(); // connects the client
ClientConnectionRetreiver.getConnectionInfo(publisher); // gets connection info
PubSubUtility.printPublising("Publisher1");
readyToSend = true;
x.threadCondNotify(); <<<<< HOW TO MAKE THIS WORK
// Think about making the PubClient Thread sleep for a short while so its not too ahead of the client
startTime = System.currentTimeMillis();
for (int i = 1; i <= messageNum; i++) {
publisher.publishWith()
.topic(publisherTopic) // publishes to the specified topic
.qos(MqttQos.EXACTLY_ONCE)
.payload(convertedMessage) // the contents of the message
.send();
}
endTime = System.currentTimeMillis();
finalTime = endTime - startTime;
finalSecTime = TimeUnit.MILLISECONDS.toSeconds(finalTime);
PubSubUtility.printNumOfPublished("Publisher1", messageNum);
publisher.disconnect();
}
public class CommonThread {
private static final Logger LOGGER = Logger.getLogger(SubMainThread.class.getName()); // Creates a logger instance
public synchronized void threadCondNotify() {
notify();
System.out.println("Notified other thread");
}
public synchronized void threadCondWait() {
try {
while (PubMainThread.readyToSend != true) {
System.out.println("Waiting for another thread....");
wait();
}
}
catch (InterruptedException e) {
LOGGER.log(Level.SEVERE, "The thread was interrupted while waiting for another thread", e);
}
}
}
In Sender (rough Java code with some details omitted):
//package statement and imports here
class Sender extends Thread {
public static final Boolean x= new Boolean(true);
public void run() {
//initialize here
synchronized(x) {
x.notify();
}
//send messages here
}
}
In Receiver (start before Sender):
//package statement and imports here
class Receiver extends Thread {
public void run() {
//initialize here
synchronized(Sender.x) {
Sender.x.wait(); //blocks till Sender.x.notify()
}
Date start= new Date();
//receive messages here
Date end= new Date();
int duration_milliseconds= end.getTime()-start.getTime();
}
}
maybe you have to add
try{ /* code here */ } catch (InterruptedException e) {}
Feel free to discuss sense and nonsense of direct use of notify() and wait() especially in Java versions with extended concurrency libraries...
Related
I'm trying to implement a Pulsar client with multiple producers that distributes the load among the threads, but regardless the value passed on ioThreads() and on listenerThreads(), it is always overloading the first thread (> 65% cpu while the other threads are completely idle)
I have tried a few things including this "dynamic rebalancing" every hour(last method) but closing it in the middle of the process certainly is not the best approach
This is the relevant code
...
// pulsar client
pulsarClient = PulsarClient.builder() //
.operationTimeout(config.getAppPulsarTimeout(), TimeUnit.SECONDS) //
.ioThreads(config.getAppPulsarClientThreads()) //
.listenerThreads(config.getAppPulsarClientThreads()) //
.serviceUrl(config.getPulsarServiceUrl()).build();
...
private createProducers() {
String strConsumerTopic = this.config.getPulsarTopicInput();
List<Integer> protCasesList = this.config.getEventProtoCaseList();
for (Integer e : protCasesList) {
String topicName = config.getPulsarTopicOutput().concat(String.valueOf(e));
LOG.info("Creating producer for topic: {}", topicName);
Producer<byte[]> protobufProducer = pulsarClient.newProducer().topic(topicName).enableBatching(false)
.blockIfQueueFull(true).compressionType(CompressionType.NONE)
.sendTimeout(config.getPulsarSendTimeout(), TimeUnit.SECONDS)
.maxPendingMessages(config.getPulsarMaxPendingMessages()).create();
this.mapLink.put(strConsumerTopic.concat(String.valueOf(e)), protobufProducer);
}
}
public void closeProducers() {
String strConsumerTopic = this.config.getPulsarTopicInput();
List<Integer> protCasesList = this.config.getEventProtoCaseList();
for (Integer e : protCasesList) {
try {
this.mapLink.get(strConsumerTopic.concat(String.valueOf(e))).close();
LOG.info("{} producer correctly closed...",
this.mapLink.get(strConsumerTopic.concat(String.valueOf(e))).getProducerName());
} catch (PulsarClientException e1) {
LOG.error("Producer: {} not closed cause: {}",
this.mapLink.get(strConsumerTopic.concat(String.valueOf(e))).getProducerName(),
e1.getMessage());
}
}
}
public void rebalancePulsarThreads(boolean firstRun) {
ThreadMXBean threadHandler = ManagementFactory.getThreadMXBean();
ThreadInfo[] threadsInfo = threadHandler.getThreadInfo(threadHandler.getAllThreadIds());
for (ThreadInfo threadInfo : threadsInfo) {
if (threadInfo.getThreadName().contains("pulsar-client-io")) {
// enable cpu time for all threads
threadHandler.setThreadCpuTimeEnabled(true);
// get cpu time for this specific thread
long threadCPUTime = threadHandler.getThreadCpuTime(threadInfo.getThreadId());
int thresholdCPUTime = 65;
if (threadCPUTime > thresholdCPUTime) {
LOG.warn("Pulsar client thread with CPU time greater than {}% - REBALANCING now", thresholdCPUTime);
try {
closeProducers();
} catch (Exception e) {
if (!firstRun) {
// producers will not be available in the first run
// therefore, the logging only happens when it is not the first run
LOG.warn("Unable to close Pulsar client threads on rebalancing: {}", e.getMessage());
}
}
try {
createPulsarProducers();
} catch (Exception e) {
LOG.warn("Unable to create Pulsar client threads on rebalancing: {}", e.getMessage());
}
}
}
}
}
From what you describe, the most likely scenario is that all the topics you're using are served by one single broker.
If that's indeed the case, and avoiding topic load balancing across brokers, it's normal that it's using a single thread because all these producers will be sharing a single, pooled, TCP connection and each connection is assigned to 1 IO thread (listener threads are used for consumer listeners).
If you want to force more threads, you can increase the "Max TCP connection per each broker" setting, in order to use all the configured IO threads.
eg:
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://localhost:6650")
.ioThreads(16)
.connectionsPerBroker(16)
.create();
I am trying to use the following code which is an implementation of web sockets in Netty Nio. I have implment a JavaFx Gui and from the Gui I want to read the messages that are received from the Server or from other clients. The NettyClient code is like the following:
public static ChannelFuture callBack () throws Exception{
String host = "localhost";
int port = 8080;
try {
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.handler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new RequestDataEncoder(), new ResponseDataDecoder(),
new ClientHandler(i -> {
synchronized (lock) {
connectedClients = i;
lock.notifyAll();
}
}));
}
});
ChannelFuture f = b.connect(host, port).sync();
//f.channel().closeFuture().sync();
return f;
}
finally {
//workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
ChannelFuture ret;
ClientHandler obj = new ClientHandler(i -> {
synchronized (lock) {
connectedClients = i;
lock.notifyAll();
}
});
ret = callBack();
int connected = connectedClients;
if (connected != 2) {
System.out.println("The number if the connected clients is not two before locking");
synchronized (lock) {
while (true) {
connected = connectedClients;
if (connected == 2)
break;
System.out.println("The number if the connected clients is not two");
lock.wait();
}
}
}
System.out.println("The number if the connected clients is two: " + connected );
ret.channel().read(); // can I use that from other parts of the code in order to read the incoming messages?
}
How can I use the returned channelFuture from the callBack from other parts of my code in order to read the incoming messages? Do I need to call again callBack, or how can I received the updated message of the channel? Could I possible use from my code (inside a button event) something like ret.channel().read() (so as to take the last message)?
By reading that code,the NettyClient is used to create connection(ClientHandler ),once connect done,ClientHandler.channelActive is called by Netty,if you want send data to server,you should put some code here. if this connection get message form server, ClientHandler.channelRead is called by Netty, put your code to handle message.
You also need to read doc to know how netty encoder/decoder works.
How can I use the returned channelFuture from the callBack from other parts of my code in order to read the incoming messages?
share those ClientHandler created by NettyClient(NettyClient.java line 29)
Do I need to call again callBack, or how can I received the updated message of the channel?
if server message come,ClientHandler.channelRead is called.
Could I possible use from my code (inside a button event) something like ret.channel().read() (so as to take the last message)?
yes you could,but not a netty way,to play with netty,you write callbacks(when message come,when message sent ...),wait netty call your code,that is : the driver is netty,not you.
last,do you really need such a heavy library to do network?if not ,try This code,it simple,easy to understanding
So I currently have a lot of code, it will be difficult to break it all down into an SSCCE but maybe I will attempt to do so later if necessary.
Anyways, here is the gist: I have two processes communicating via RMI. It works. However I want to be able continue if the communcation if the host process (JobViewer) exits and then returns all in the life of the client process (Job).
Currently I have the binded name saved to a file everytime a Job starts up, and the JobViewer opens this file on startup. It works great, the correct binded name works. However, I get a NotBoundException every time I try to resume communication with a Job that I know for fact is still running when the JobViewer restarts.
My JobViewer implements an interface that extends Remote with the following methods:
public void registerClient(String bindedName, JobStateSummary jobSummary) throws RemoteException, NotBoundException;
public void giveJobStateSummary(JobStateSummary jobSummary) throws RemoteException;
public void signalEndOfClient(JobStateSummary jobSummary) throws RemoteException;
And my Job also implements a different interface that extends Remote with the following methods:
public JobStateSummary getJobStateSummary() throws RemoteException;
public void killRemoteJob() throws RemoteException;
public void stopRemoteJob() throws RemoteException;
public void resumeRemoteJob() throws RemoteException;
How do I achieve this? Here is some of my current code that inits the RMI if it helps...
JobViewer side:
private Registry _registry;
// Set up RMI
_registry = LocateRegistry.createRegistry(2002);
_registry.rebind("JOBVIEWER_SERVER", this);
Job side:
private NiceRemoteJobMonitor _server;
Registry registry = LocateRegistry.getRegistry(hostName, port);
registry.rebind(_bindedClientName, this);
Remote remoteServer = registry.lookup(masterName);
_server = (NiceRemoteJobMonitor)remoteServer;
_server.registerClient(_bindedClientName, _jobStateSummary);
I get a NotBoundException every time I try to resume communication with a Job that I know for fact is still running when the JobViewer restarts.
That can only happen if the JobViewer didn't rebind itself when it started up. More usually you get a NoSuchObjectException when you use a stale stub, i.e. a stub whose remote object has exited. In this case you should reaquire the stub, i.e. redo the lookup().
Why is the client binding itself to a Registry? If you want to register a callback, just pass this to the registerClient() method instead of the bind-name, and adjust its signature accordingly (using the client's remote interface as the parameter type). No need to have the server doing a lookup to the client Registry. No need for a client Registry at all.
My solution was to have the Job ping the JobViewer every so often:
while (true) {
try {
_server.ping();
// If control reaches here we were able to successfully ping the job monitor.
} catch (Exception e) {
System.out.println("Job lost contact with the job monitor at " + new Date().toString() + " ...");
// If control reaches we were unable to ping the job monitor. Now we will loop until it presumably comes back to life.
boolean foundServer = false;
while (!foundServer) {
try {
// Attempt to register again.
Registry registry = LocateRegistry.getRegistry(_hostName, _port);
registry.rebind(_bindedClientName, NiceSupervisor.this);
Remote remoteServer = registry.lookup(_masterName);
_server = (NiceRemoteJobMonitor)remoteServer;
_server.registerClient(_bindedClientName, _jobStateSummary);
// Ping the server for good measure.
_server.ping();
System.out.println("Job reconnected with the job monitor at " + new Date().toString() + " ...");
// If control reaches here we were able to reconnect to the job monitor and ping it again.
foundServer = true;
} catch (Exception x) {
System.out.println("Job still cannot contact the job monitor at " + new Date().toString() + " ...");
}
// Sleep for 1 minute before we try to locate the registry again.
try {
Thread.currentThread().sleep(PING_WAIT_TIME);
} catch (InterruptedException x) {
}
} // End of endless loop until we find the server again.
}
// Sleep for 1 minute after we ping the server before we try again.
try {
Thread.currentThread().sleep(PING_WAIT_TIME);
} catch (InterruptedException e) {
}
} // End of endless loop that we never exit.
I have a background service that runs in its own separate process using
android:process=":deamon"
In the manifest entry for the service. I want to communicate with the the service (remote process) from my activity and receive data from it.
I'm doing that by sending messages to and from the remote process as described in http://developer.android.com/guide/components/bound-services.html#Messenger and as they suggested I followed
If you want the service to respond, then you need to also create a Messenger in the client. >Then when the client receives the onServiceConnected() callback, it sends a Message to the >service that includes the client's Messenger in the replyTo parameter of the send() method.
The thing is, I need to provide a blocking/synchronous API to get data from my remote service, how can my "get" function block the caller and then return the data received in my incoming Handler ?
What would be the best approach to do that ?
This is code for messaging part of Client
SparseArray<CountDownLatch> lockArray = new SparseArray<>();
SparseArray<Bundle> msgDataArray = new SparseArray<>();
public Bundle sendAndWaitResponse(Message msg) throws
RemoteException, InterruptedException {
int msgId = msg.arg2;
Log.d("PlatformConnector", "Sending message to service, Type: "
+ msg.what + ", msgId: " + msg.arg2);
CountDownLatch latch = new CountDownLatch(1);
lockArray.put(msgId, latch);
platformMessenger.send(msg);
latch.await();
Bundle response = msgDataArray.get(msgId);
lockArray.delete(msgId);
msgDataArray.delete(msgId);
return response;
}
void storeResponseAndNotify(Message msg) {
int msgId = msg.arg2;
// Because the message itself is recycled after Handler returns,
// we should store only the data of message
msgDataArray.put(msgId, msg.getData());
lockArray.get(msgId).countDown();
}
private class ClientMessageHandler extends Handler {
#Override
public void handleMessage(Message msg) {
storeResponseAndNotify(msg);
}
}
This is example of utilizing above code.
RandomInt.getNextInt() is my custom static method, which generates random integer with Random.nextInt().
public JSONObject doSomething(JSONObject object) {
Message msg = Message.obtain(null, Constants.MESSAGE_SOMETHING, 0, RandomInt.getNextInt());
Bundle bundle = new Bundle();
bundle.putString(Constants.MESSAGE_DATA_SOMETHING, object.toString());
msg.setData(bundle);
try {
Bundle responseData = sendAndWaitResponse(msg);
return new JSONObject(responseData.getString(Constants.MESSAGE_DATA_RETURN));
} catch (RemoteException e) {
Log.e(TAG, "Failed to send message to platform");
e.printStackTrace();
} catch (InterruptedException e) {
Log.e(TAG, "Interrupted while waiting message from platform");
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return null;
}
Sequence is as follows,
The Client prepares Message and set its arg2 as some random integer
(this integer will be the message id for synchronization).
The Client prepares new CountDownLatch and put it to LockArray.
the Client sends message with sendAndWaitResponse(). It sends message to service via Messenger and invokes latch.await().
Service processes receives message and prepare reply message. The arg2 of this reply message should be same as received message.
Service sends reply message to client via Messenger in replyTo.
Client message handler handles the message with storeResponseAndNotify.
When the blocking of Client thread is finished, the response data would be already prepared in msgDataArray.
CountDownLatch is simple switch to block and unblock the thread.
(http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CountDownLatch.html)
SparseArray is similar to HashMap, but more memory-efficient for smaller sets.
(http://developer.android.com/reference/android/util/SparseArray.html)
Be careful not to block the thread of Messenger. Messenger runs in single thread and if you block from the handleMessage(), it will block all other messages and cause deaklock problem.
im using ZeroMQ to client / server application ,
now in my MT server i try to set timeout, i tried to set on the server :
socket.setReceiveTimeOut(2000);
socket.setSendTimeOut(2000);
with no luck , how do i set timeout on response .
this is my multi thread server code this is taken from zeromq examples for mt server :
/*
* Multithreaded Hello World server in Java
*
* #author Vadim Shalts
* #email vshalts#gmail.com
*
*/
import org.zeromq.ZMQ;
import org.zeromq.ZMQQueue;
class mtserver {
static void main(String[] args) {
final ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket clients = context.socket(ZMQ.ROUTER);
clients.bind ("tcp://*:5555");
ZMQ.Socket workers = context.socket(ZMQ.DEALER);
workers.bind ("inproc://workers");
for(int thread_nbr = 0; thread_nbr < 5; thread_nbr++) {
Thread worker_routine = new Thread() {
#Override
public void run() {
ZMQ.Socket socket = context.socket(ZMQ.REP);
socket.connect ("inproc://workers");
while (true) {
// Wait for next request from client (C string)
byte[] request = socket.recv (0);
System.out.println ("Received request: ["+new String(request,0,request.length-1)+"]");
// Do some 'work'
try {
Thread.sleep (1000);
} catch(InterruptedException e) {
e.printStackTrace();
}
// Send reply back to client (C string)
byte[] reply = "World ".getBytes();
reply[reply.length-1] = 0; //Sets the last byte of the reply to 0
socket.send(reply, 0);
}
}
};
worker_routine.start();
}
// Connect work threads to client threads via a queue
ZMQQueue zMQQueue = new ZMQQueue(context,clients, workers);
zMQQueue.run();
// We never get here but clean up anyhow
clients.close();
workers.close();
context.term();
}
}
Why do you need a timeout? The original examples don't use it.
Why do you need a timeout on the server?
A natural place to put a timeout in the client.
socket.setReceiveTimeOut(int milliseconds); works on a client side.