We have a streaming application taking data from MQTT and load into other resource. And this application have multiple threads to handle some tasks.
Here we have two tasks(threads):
First one is a READER
Second one is a WRITER
So READER will read data from MQTT broker and write on a java queue and WRITER will take this data from that queue and write it over one database. This application itself monitoring these threads for finding any failure. If any one of the threads failed then we will stop remaining threads gracefully. In case of paho MqttClient class (READER Class) wont create a thread even its a threaded class. But it will creating multiple threads in the background.
Because of this we could not check whether these threads is failed or running by java isAlive() function. So we just checking this class have connection by MqttClient isConnected() method. Once isConnected method return false (5 times) , then we will stop Writer thread gracefully. But Reader class threads which spawned in the background are not able to stop. I have tried disconnect() and close()
methods. But its not stopping any of the background threads. Its throws error disconnected threads could not stop.
So please anybody help.
What you suggest sounds like an awkward design.
Why not just use the Paho callbacks, in particular the connectionLost as below?
private final MqttCallbackExtended mCallback = new MqttCallbackExtended() {
#Override
public void connectComplete(boolean reconnect, String brokerAddress) {
mqttClient.subscribe("topic", 1, null, mSubscribeCallback);
}
#Override
public void connectionLost(Throwable ex) {
}
#Override
public void deliveryComplete(IMqttDeliveryToken deliveryToken) {
}
#Override
public void messageArrived(String topic, MqttMessage mqttMessage) throws Exception {
}
};
private final IMqttActionListener mConnectionCallback = new IMqttActionListener() {
#Override
public void onSuccess(IMqttToken asyncActionToken) {
// do nothing, this case is handled in mCallback.connectComplete()
}
#Override
public void onFailure(IMqttToken asyncActionToken, Throwable exception) {
}
};
private final IMqttActionListener mSubscribeCallback = new IMqttActionListener() {
#Override
public void onSuccess(IMqttToken subscribeToken) {
}
#Override
public void onFailure(IMqttToken subscribeToken, Throwable ex) {
}
};
MqttConnectOptions connectOptions = new MqttConnectOptions();
connectOptions.setCleanSession(true);
connectOptions.setAutomaticReconnect(true);
connectOptions.setUserName("username");
connectOptions.setPassword("password".toCharArray());
MqttAsyncClient mqttClient = new MqttAsyncClient("tcp:// test.mosquitto.org");
mqttClient.setCallback(mCallback);
try {
mqttClient.connect(connectOptions, null, mConnectionCallback);
} catch (Exception ex) {
System.err.println(ex.toString());
}
Related
I was struggling an issue or maybe it's because of my small background in programming, the issue was about subscribing to multiple topics and showing the subscribed topics in multiple textviews in android
I used to subscribe it like that :
private void setSub()
{
try{
client.subscribe(topic,0);
}
catch (MqttException e){
e.printStackTrace();
}
}
then I've called the setsub() function after the success of the connection to the MQTT client
then I've implemented the setCallBack method and under the messageArrived I've added the line to change the textview value with the message payload I've received from the subscription
#Override
public void messageArrived(String topic, MqttMessage message) throws Exception {
temp.setText(new String(message.getPayload()));
}
and ofcourse when I define another textview it just took the same result as the first one
so my question is how to configure the MessageArrived function to show each single topic in a single textview?
Thank you in advance.
You can call client.subscribe() as many times as needed with different topics.
private void setSub()
{
try{
client.subscribe(topic1,0);
client.subscribe(topic2,0);
client.subscribe(topic3,0);
}
catch (MqttException e){
e.printStackTrace();
}
}
The messageArrived() callback is passed the topic for each message so you just set up an if statement to decide which textView to update depending on the topic.
#Override
public void messageArrived(String topic, MqttMessage message) throws Exception {
if (topic.equals(topic1) {
temp.setText(new String(message.getPayload()));
} else if (topic.equals(topic2) {
foo.setText(new String(message.getPayload()));
}
}
But you should not be calling the setText() method in the callback as it happens on the client thread. You need to look at using runOnUiThread() to do updates.
I've just figured it out,
if (topic.contains("button") ) {
temp.setText(new String(message.getPayload()));
}
else if (topic.contains("test"))
{
volt.setText(new String(message.getPayload()));
}
I've just put in the name of each topic i wanted to display.
Try runOnUiThread method because onMessageArived() is a client thread.
public void messageArrived(String topic, MqttMessage message)
throws Exception {
String msg = new String(message.getPayload());
Log.i(TAG, "Message Arrived: " + msg);
runOnUiThread(new Runnable(){
public void run() {
textView.setText(msg);
}
});
}
everybody.
Hope you can help me with this one:
I have two threads, which are tasked with handling connections from a client.
This is my code
ServerSocket loginSocket = new ServerSocket(8000);
ServerSocket fileSocket = new ServerSocket(7000);
while (running) {
new LoginThread(loginSocket.accept(),loginInormation).start();
new LoaderThread(fileSocket.accept()).start();
}
When I try to connect to the loginSocket two times, the server will block and stop working, blocking the client, but this doesn't happen if I delete this:
new LoginThread(loginSocket.accept(),loginInormation).start();
I'm not getting any error messages, so why is this happening and how can I fix this?
The accept() method is a blocking method, which means that your program won't continue until a connection is made with loginSocket().
When you're creating your LoginThread, the program waits a connection to set the first parameter of your object, and it will not continue the execution until a connection is made.
The line new LoginThread(loginSocket.accept(),loginInormation).start(); contains the method call loginSocket.accept(), which will be called before this thread is created. This method call will block until a client logs in. (In addition, the second thread will be blocked by fileSocket.accept()).
As for a solution, I would move the accept() calls to inside each of the Threads. You will need to pass the sockets to the threads for them to do this.
Start fileSocket and login socket in different threads
package com.ca.training.task.app;
import java.io.IOException;
import java.net.ServerSocket;
public class App {
public void execute() {
LoginRunnable loginRunnable = new LoginRunnable();
loginRunnable.setLoginInformation(new Object());//Login information
FileRunnable fileRunnable = new FileRunnable();//Data for loaded runnable.
fileRunnable.setParams(new Object());
startLoginThread(loginRunnable);
startFileThread(fileRunnable);
}
private static void startLoginThread(LoginRunnable loginRunnable) {
Thread loginThread = new Thread(loginRunnable);
loginThread.start();
}
private static void startFileThread(FileRunnable fileRunnable) {
Thread loadedThread = new Thread(fileRunnable);
loadedThread.start();
}
class LoginRunnable implements Runnable {
private Object loginInformation;
#Override
public void run() {
try {
ServerSocket loginSocket = new ServerSocket(8000);
loginSocket.accept();
} catch (IOException e) {
e.printStackTrace();
}
}
public Object getLoginInformation() {
return loginInformation;
}
public void setLoginInformation(Object loginInformation) {
this.loginInformation = loginInformation;
}
}
class FileRunnable implements Runnable {
private Object params;
#Override
public void run() {
try {
ServerSocket fileSocket = new ServerSocket(7000);
} catch (IOException e) {
e.printStackTrace();
}
}
public Object getParams() {
return params;
}
public void setParams(Object params) {
this.params = params;
}
}
}
Java Service Stops Running when Rejected Execution triggers
I have a running mqtt-client that acts as a bridge/server for Android apps and database pooling for MySQL and when rejected execution triggers my service stops, I'm really wondering why.
This is my threadpool
public static ExecutorService threadPool = new ThreadPoolExecutor(10, // core size
30, // max size
10*60, // idle timeout
TimeUnit.SECONDS,
new ArrayBlockingQueue<Runnable>(20),
factory,
new DiscardPolicy()); // queue with a size
Ad this is how my message arrived performs
#Override
public void messageArrived(final String topic, final MqttMessage message) throws Exception {
threadPool.execute(new Runnable(){
void run(){
if(topic.equalsIgnoreCase("something")){} // and code goes on like these
}
// code
});
try{
if(Constants.pendingRunnables.size() > 0){
for(java.util.Map.Entry<ThreadPoolExecutor, Runnable> runner : Constants.pendingRunnables.entrySet()){
// runner.getKey().execute(runner.getValue());
try{
if (!runner.getKey().isShutdown()) {
runner.getValue().run();
Constants.pendingRunnables.remove(runner.getKey());
}
}catch(Exception ex){
}
}
}
}catch(Exception e){}
}
public class DiscardPolicy extends ThreadPoolExecutor.DiscardPolicy{
private static final Logger LOGGER = Logger.getLogger(DiscardPolicy.class);
#Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
// TODO Auto-generated method stub
super.rejectedExecution(r, e);
Constants.pendingRunnables.put(e, r);
LOGGER.info("Rejected Runnable ");
}
}
Some runnables have mqtt client in them so that they could publish a response.
What these service do is run runnables and when exception triggers I append them to pending runnables so that I could run them later.
Do I have something missing here?
I am using Apache HttpClient 4 to communicate with a REST API and most of the time I do lengthy PUT operations. Since these may happen over an unstable Internet connection I need to detect if the connection is interrupted and possibly need to retry (with a resume request).
To try my routines in the real world I started a PUT operation and then I flipped the Wi-Fi switch of my laptop, causing an immediate total interruption of any data flow. However it takes a looong time (maybe 5 minutes or so) until eventually a SocketException is thrown.
How can I speed up to process? I'd like to set a timeout of maybe something around 30 seconds.
Update:
To clarify, my request is a PUT operation. So for a very long time (possibly hours) the only operation is a write() operation and there are no read operations. There is a timeout setting for read() operations, but I could not find one for write operations.
I am using my own Entity implementation and thus I write directly to an OutputStream which will pretty much immediately block once the Internet connection is interrupted. If OutputStreams had a timeout parameter so I could write out.write(nextChunk, 30000); I could detect such a problem myself. Actually I tried that:
public class TimeoutHttpEntity extends HttpEntityWrapper {
public TimeoutHttpEntity(HttpEntity wrappedEntity) {
super(wrappedEntity);
}
#Override
public void writeTo(OutputStream outstream) throws IOException {
try(TimeoutOutputStreamWrapper wrapper = new TimeoutOutputStreamWrapper(outstream, 30000)) {
super.writeTo(wrapper);
}
}
}
public class TimeoutOutputStreamWrapper extends OutputStream {
private final OutputStream delegate;
private final long timeout;
private final ExecutorService executorService = Executors.newSingleThreadExecutor();
public TimeoutOutputStreamWrapper(OutputStream delegate, long timeout) {
this.delegate = delegate;
this.timeout = timeout;
}
#Override
public void write(int b) throws IOException {
executeWithTimeout(() -> {
delegate.write(b);
return null;
});
}
#Override
public void write(byte[] b) throws IOException {
executeWithTimeout(() -> {
delegate.write(b);
return null;
});
}
#Override
public void write(byte[] b, int off, int len) throws IOException {
executeWithTimeout(() -> {
delegate.write(b, off, len);
return null;
});
}
#Override
public void close() throws IOException {
try {
executeWithTimeout(() -> {
delegate.close();
return null;
});
} finally {
executorService.shutdown();
}
}
private void executeWithTimeout(final Callable<?> task) throws IOException {
try {
executorService.submit(task).get(timeout, TimeUnit.MILLISECONDS);
} catch (TimeoutException e) {
throw new IOException(e);
} catch (ExecutionException e) {
final Throwable cause = e.getCause();
if (cause instanceof IOException) {
throw (IOException)cause;
}
throw new Error(cause);
} catch (InterruptedException e) {
throw new Error(e);
}
}
}
public class TimeoutOutputStreamWrapperTest {
private static final byte[] DEMO_ARRAY = new byte[]{1,2,3};
private TimeoutOutputStreamWrapper streamWrapper;
private OutputStream delegateOutput;
public void setUp(long timeout) {
delegateOutput = mock(OutputStream.class);
streamWrapper = new TimeoutOutputStreamWrapper(delegateOutput, timeout);
}
#AfterMethod
public void teardown() throws Exception {
streamWrapper.close();
}
#Test
public void write_writesByte() throws Exception {
// Setup
setUp(Long.MAX_VALUE);
// Execution
streamWrapper.write(DEMO_ARRAY);
// Evaluation
verify(delegateOutput).write(DEMO_ARRAY);
}
#Test(expectedExceptions = DemoIOException.class)
public void write_passesThruException() throws Exception {
// Setup
setUp(Long.MAX_VALUE);
doThrow(DemoIOException.class).when(delegateOutput).write(DEMO_ARRAY);
// Execution
streamWrapper.write(DEMO_ARRAY);
// Evaluation performed by expected exception
}
#Test(expectedExceptions = IOException.class)
public void write_throwsIOException_onTimeout() throws Exception {
// Setup
final CountDownLatch executionDone = new CountDownLatch(1);
setUp(100);
doAnswer(new Answer<Void>() {
#Override
public Void answer(InvocationOnMock invocation) throws Throwable {
executionDone.await();
return null;
}
}).when(delegateOutput).write(DEMO_ARRAY);
// Execution
try {
streamWrapper.write(DEMO_ARRAY);
} finally {
executionDone.countDown();
}
// Evaluation performed by expected exception
}
public static class DemoIOException extends IOException {
}
}
This is somewhat complicated, but it works quite well in my unit tests. And it works in real life as well, except that the HttpRequestExecutor catches the exception in line 127 and tries to close the connection. However when trying to close the connection it first tries to flush the connection which again blocks.
I might be able to dig deeper in HttpClient and figure out how to prevent this flush operation, but it is already a not too pretty solution, and it is just about to get even worse.
UPDATE:
It looks like this can't be done on the Java level. Can I do it on another level? (I am using Linux).
Java blocking I/O does not support socket timeout for write operations. You are entirely at the mercy of the OS / JRE to unblock the thread blocked by the write operation. Moreover, this behavior tends to be OS / JRE specific.
This might be a legitimate case to consider using a HTTP client based on non-blocking I/O (NIO) such as Apache HttpAsyncClient.
You can configure the socket timeout using RequestConfig:
RequestConfig myRequestConfig = RequestConfig.custom()
.setSocketTimeout(5000) // 5 seconds
.build();
When, when you do the call, just assign your new configuration. For instance,
HttpPut httpPut = new HttpPut("...");
httpPut.setConfig(requestConfig);
...
HttpClientContext context = HttpClientContext.create();
....
httpclient.execute(httpPut, context);
For more information regarthing timeout configurations, here there is a good explanation.
Her is one of the link i came across which talks connection eviction policy : here
public static class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean shutdown;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
super();
this.connMgr = connMgr;
}
#Override
public void run() {
try {
while (!shutdown) {
synchronized (this) {
wait(5000);
// Close expired connections
connMgr.closeExpiredConnections();
// Optionally, close connections
// that have been idle longer than 30 sec
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
} catch (InterruptedException ex) {
// terminate
}
}
public void shutdown() {
shutdown = true;
synchronized (this) {
notifyAll();
}
}}
I think you might want to look at this.
I am a Java newbie trying to learn network programming and concurrency, and I thought I'd try out writing a simple chat server where input from a client is echoed to all the clients. That's not happening. I added a couple print statements so that the program will announce that it is waiting for connections and each time it receives a connection. I am using Telnet locally to connect to the port on my machine.
The program announces success for the first and second concurrent connections but then does not announce success for subsequent connections until I close all connections. So, for example, I'll connect from five separate terminals, and the program will announce "Connection 1" and "Connection 2" but will not announce "Connection 3", 4, and 5 until I close all the terminals.
I'm looking for help figuring out where my errors lie as well as general advice for how to approach debugging a situation like this.
In a nutshell, my program has
A Main class, which starts the other three threads
A ClientListener class, which uses a SocketReader to listen for connections and stores the Sockets inputstreams and outputstreams in two Sets.
A MessageReader, which iterates over the inputstreams. If it finds a message, it puts it in a SynchronousQueue and waits for the
MessageWriter to remove it. The MessageWriter sends the message to all the outputstreams.
The code is below. Thanks for any help!
public class Main {
public static void main(String[] args) {
ClientListener clientListener = new ClientListener();
Thread clientListenerThread = new Thread(clientListener);
clientListenerThread.setPriority(Thread.MAX_PRIORITY);
clientListenerThread.start();
MessageReader messageReader = new MessageReader(clientListener);
Thread messageReaderThread = new Thread(messageReader);
messageReaderThread.setPriority(Thread.MIN_PRIORITY);
messageReaderThread.start();
MessageWriter messageWriter = new MessageWriter(messageReader, clientListener);
Thread messageWriterThread = new Thread(messageWriter);
messageWriterThread.setPriority(Thread.NORM_PRIORITY);
messageWriterThread.start();
}
}
public class ClientListener implements Runnable {
private static final int DEFAULT_PORT = 5000;
private Set<Scanner> clientIn = Collections.synchronizedSet(
new LinkedHashSet<Scanner>());
private Set<PrintWriter> clientOut = Collections.synchronizedSet(
new LinkedHashSet<PrintWriter>());
public Set<Scanner> getClientIn() {
return clientIn;
}
public Set<PrintWriter> getClientOut() {
return clientOut;
}
#Override
public void run() {
try {
ServerSocket server = new ServerSocket(DEFAULT_PORT);
System.out.println("Listening for connections...");
int connectionNum = 0;
while(true) {
Socket socket = server.accept();
connectionNum++;
System.out.format("Connection %s%n", connectionNum);
Scanner in = new Scanner(socket.getInputStream());
PrintWriter out = new PrintWriter(socket.getOutputStream());
clientIn.add(in);
clientOut.add(out);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
public class MessageReader implements Runnable {
private ClientListener clientListener;
private BlockingQueue<String> messages = new SynchronousQueue<String>();
public MessageReader(ClientListener clientListener) {
this.clientListener = clientListener;
}
#Override
public void run() {
while(true) {
Set<Scanner> clients = clientListener.getClientIn();
synchronized (clients) {
for(Scanner client: clients) {
if(client.hasNext()) {
try {
messages.put(client.next());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
}
public String getMessage() throws InterruptedException {
return messages.take();
}
}
public class MessageWriter implements Runnable {
private ClientListener clientListener;
private MessageReader messageReader;
public MessageWriter(
MessageReader messageReader,
ClientListener clientListener) {
this.messageReader = messageReader;
this.clientListener = clientListener;
}
#Override
public void run() {
try {
while(true) {
String message = messageReader.getMessage();
Set<PrintWriter> clients = clientListener.getClientOut();
synchronized (clients) {
for(PrintWriter client: clients) {
client.println(message);
}
}
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
I'm not a threading expert, but in class MessageReader there is this line
if(client.hasNext())
Javadoc for Scanner.hasNext() say's "This method may block while waiting for input to scan. The scanner does not advance past any input."
If the scanner is still in wait the synchronized method never proceeds and block all other inputs. And as said in my earlier comment the line which says clientIn.add(in); in class ClientListener probably gets blocked given that its a synchronized Set, but since the print statment is written before it, it might give the impression that Connection 2 was succesfully established.