We Keep Coding

How to handle lots of messages being sent from android service?

I am developing an app that communicates using the MAVLINK protocol. I am using dronefleet for this purpose. My app has a service which runs a ReadThread that checks the incoming MAVLINK messages for their type. ReadThread then sends messages to the UI for updating some TextViews with the drone's information like battery status, etc. Here is my code.
ReadThread in DService.java
import io.dronefleet.mavlink.MavlinkMessage;
import io.dronefleet.mavlink.common.Attitude;
import io.dronefleet.mavlink.common.SysStatus;
public static final int ATTITUDE = 1;
public static final int SYS_STATUS = 2;
private class ReadThread extends Thread {
private AtomicBoolean keep = new AtomicBoolean(true);
#Override
public void run() {
while(keep.get()){
if(inputStream == null)
return;
MavlinkMessage message;
try {
---------------------get MAVLINK message from stream here---------------
message = mavlinkConnection.next();
-------------------check MAVLINK message type and then send message to UI for updating related fields--------------------
if(message.getPayload() instanceof Attitude) {
MavlinkMessage<Attitude> attitudeMessage = (MavlinkMessage<Attitude>)message;
myHandler.obtainMessage(ATTITUDE, attitudeMessage).sendToTarget();
}
----------------removing comments causes app to crash---------------------
/*if(message.getPayload() instanceof SysStatus) {
MavlinkMessage<SysStatus> sysStatusMessage = (MavlinkMessage<SysStatus>)message;
int battery = sysStatusMessage.getPayload().batteryRemaining();
myHandler.obtainMessage(SYS_STATUS, Integer.toString(battery)).sendToTarget();*/
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
public void setKeep(boolean keep) {
this.keep.set(keep);
}
}
handleMessage() in MainActivity.java
switch (msg.what) {
case DService.SYS_STATUS:
String battery = (String) msg.obj + "%";
myActivity.get().batteryView.setText(battery);
case DService.ATTITUDE:
MavlinkMessage<Attitude> message = (MavlinkMessage<Attitude>) msg.obj;
String pitch = Float.toString(message.getPayload().pitch());
String roll = Float.toString(message.getPayload().roll());
String yaw = Float.toString(message.getPayload().yaw());
myActivity.get().pitchView.setText(pitch);
myActivity.get().rollView.setText(roll);
myActivity.get().yawView.setText(yaw);
break;
}
}
My problem is my app is crashing if I am checking for more than one type of MAVLINK message in my ReadThread. If I check for (say) either SYS_STATUS or ATTITUDE, then the corresponding TextViewss in the UI are getting updated seamlessly every second (which is the rate of messages sent by MAVLINK). But not for 2 message classes. If I remove comments from one if block, my app crashes.
Whats could be the reason? Is my handleMessage() wrong? Do I need to use MessageQueue or some other android mechanism? Should I run separate threads like ReadThread for each MAVLINK message type?
I am developing on Ubuntu 18, and using Android Studio.

Does logcat have any exceptions being thrown by your program before it crashes?
I would not use multiple threads for each MAVLink message type's handlemessage because I'm guessing the dronefleet parser for the MAVLink connection is stateful (MAVLink uses a magic prefix and payload length to delimit packets) and it would probably either break or be nearly synchronous if called from multiple threads.
If you're worried that you might be sending updates too frequently you could have the read thread store the values from the different MAVLink messages and a separate thread send updates at a fixed interval based on either the latest value for each message or on the set of messages received since the last update.

Event driven and asynchronous serial port communication simultaneously

I'm completely new to serial port communication and need some help grasping it.
I need to communicate with a control board. This board can sometimes send events that I need to react to, and I need to send events to the board and await a response.
We have established a protocol where each event is always 12 bytes and the first 2 bytes determine the event type.
I know that when I send a specific message, I need to await a message with specific signifying bytes. At the same time I want it to be possible to react to events that are sent from the board. For instance the board might say that it is overheating, and at the same time I'm asking it to perform some command and reply.
My question is, if I write to the port and block for a second while awaiting the expected response, how I do ensure I don't "steal" the data my listener expects? E.g. do a serial ports work like a stream, where once I've read I've advanced past the point where it can be re-read.
I've done some implementation of this using jSerialComm, hopefully this can shed some light on my question.
First a listener that is registered using the addDataListener method. I want this to trigger when an event is present on the port that starts with "T".
private static LockerSerialPort getLockerSerialPort(final DeviceClient client) {
return MySerialPort.create(COM_PORT)
.addListener(EventListener.newBuilder()
.addEventHandler(createLocalEventHandler())
.build());
}
private static EventHandler createLocalEventHandler() {
return new EventHandler() {
#Override
public void execute(final byte[] event) {
System.out.println(new String(event));
}
#Override
public byte[] getEventIdentifier() {
// I want this listener to be executed when events that start with T are sent to the port
return "T".getBytes();
}
#Override
public String getName() {
return "T handler";
}
};
}
Next, I want to be able to write to the port and immediately get the response because it is needed to know if the command was successful or not.
private byte[] waitForResponse(final byte[] bytes) throws LockerException {
write(bytes);
return blockingRead();
}
private void write(final byte[] bytes) throws LockerException {
try (var out = serialPort.getOutputStream()) {
out.write(bytes);
} catch (final IOException e) {
throw Exception.from(e, "Failed to write to serial port %s", getComPort());
}
}
public byte[] blockingRead() {
return blockingRead(DEFAULT_READ_TIMEOUT);
}
private byte[] blockingRead(final int readTimeout) {
serialPort.setComPortTimeouts(SerialPort.TIMEOUT_READ_SEMI_BLOCKING, readTimeout, 0);
try {
byte[] readBuffer = new byte[PACKET_SIZE];
final int bytesRead = serialPort.readBytes(readBuffer, readBuffer.length);
if (bytesRead != PACKET_SIZE) {
throw RuntimeException.from(null, "Expected %d bytes in packet, got %d", PACKET_SIZE, bytesRead);
}
return readBuffer;
} catch (final Exception e) {
throw RuntimeException.from(e, "Failed to read packet within specified time (%d ms)", readTimeout);
}
}
When I call waitForResponse("command"), how do I know my blocking read doesn't steal data from my listener?
Are these two patterns incompatible? How would one usually handle a scenario like this?

App Engine URL fetch callback

I am relatively new to App Engine. I do not understand how to asynchronously make an HTTP request with Java. I would have thought that this was a very simple matter using Thread and Runnable. But it seems that App Engine does not permit their use.
public Hashtable someApiMethod(..) {
SomeEntity entity = new SomeEntity(..);
ObjectifyService.ofy().save().entity(entity).now();
makeSomeHttpRequest(entity);
return launchResponse;
}
My question is this: how do I implement the method makeSomeHttpRequest(..) such that it returns without waiting for the URLFetchService.fetchAsync to return. I have tried the following without success:
protected void makeSomeHttpRequest(SomeEntity entity) {
URLFetchService fetcher = URLFetchServiceFactory.getURLFetchService();
try {
URL url = new URL("https://www.example.com");
Future future = fetcher.fetchAsync(url);
HTTPResponse response = (HTTPResponse) future.get();
byte[] content = response.getContent();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bos.write(content);
String responseString = new String(bos.toByteArray());
int responseCode = response.getResponseCode();
// Here I will do something with the responseCode and responseString
if (responseCode == 200) entity.someValue = responseString;
} catch (IOException e) {
// handle this
} catch (InterruptedException e) {
// handle this
} catch (ExecutionException e) {
// handle this
}
}
Really what I am trying to do is perform this HTTP request without forcing the method someApiMethod to wait for the response.

Few things:
First. Future doesn't work this way. Method .get waits for result of Feature execution, so basically you're stopping current thread until other thread finishes its execution. You're making it synchronous, it doesn't make any sense. Usually you call .get much later, when all other work in current thread is finished
Second. Threads in Appengine are limited to current request, you have to complete all async processing during current request. So updating an entity in such way doesn't make much sense, it's still bounded to current request. I mean in your case makeSomeHttpRequest(entity); should work faster than return launchResponse;
What you really need is to send this data to TaskQueue and make processing of SomeEntity entity from there (but don't send entity itself, just send ID and load by id from queue task). Basically it's going to be a new request handler (servlet/controller/etc) that should load entity by id, execute makeSomeHttpRequest (synchronously) and return http status 200.
See TaskQueue docs: https://cloud.google.com/appengine/docs/java/taskqueue/
You need Push Queue most likely: https://cloud.google.com/appengine/docs/java/taskqueue/overview-push

The write() method of a SocketChannel do not throw an exception when it should do

I'm writing a server to exchange messages among clients. One issue left to be solved is how to release a channel when a client happens to be closed. What I do is to start a monitor thread in which the all-Clients map is monitored, and I attempt to remove() a channel if an exception been detected when trying write() to it. However, after closing a client, the write() method in monitor thread don't throw an exception so the useless channel will never be released. Anybody know why?
public class ServerMonitor extends Thread{
private Map<String, SocketChannel> allClients;
private Set set;
private Iterator it;
private Entry entry;
private SocketChannel channel;
private ByteBuffer buf;
public ServerMonitor(Map<String, SocketChannel> allClients) {
this.allClients = allClients;
buf = ByteBuffer.allocateDirect(10);
byte b = 0;
buf.put(b);
buf.flip();
}
public void run(){
while(true) {
if(!allClients.isEmpty()) {
set = allClients.entrySet();
it = set.iterator();
while(it.hasNext()) {
entry = (Entry) it.next();
channel = (SocketChannel) entry.getValue();
try{
channel.write(buf);
} catch(Exception e) {
allClients.remove(entry.getKey());
//set.remove(entry);
}
}
}
try {
Thread.sleep(1000 * 5);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}

Writing to a TCP socket is buffered locally and put on the wire asynchronously. So you can't rely on the first write after the peer closes to fail. You can rely on a subsequent write failing, but it could take a number of writes to get there.

I've run into this issue when writing applications that send data over TCP. You've discovered that the only real way to know if a client has closed the connection is by the IOException on a call to write(...). This is pretty much the way it works.
There is a cleaner solution. First of all, you must always handle the case that a client disconnects without you knowing, and properly remove them when you get the IOException on write(...). However, if the client sends a message telling the server it is disconnecting, you can use that to close the connection when you see it.

How to override a loop fetching messages in a method

I have a Bluetooth server receiving data from a client, a mobile phone. The code I'm using looks the following
#Override
public void run() {
try {
this.localDevice = LocalDevice.getLocalDevice();
this.localDevice.setDiscoverable(DiscoveryAgent.GIAC);
this.server = (StreamConnectionNotifier) Connector.open(URL);
while(true) {
if(this.connection == null) {
this.connection = this.server.acceptAndOpen();
System.out.println("INFO: Bluetooth client connected");
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.openInputStream()));
this.writer = new BufferedWriter(new OutputStreamWriter(connection.openOutputStream()));
String line;
while((line = reader.readLine()) != null) {
if(line.equals("--#do:disconnect")) {
break;
}
System.out.println("INFO: Received from Bluetooth: " + line);
}
System.out.println("INFO: Client disconnected");
}
}
} catch(BluetoothStateException ex) {
ex.printStackTrace();
} catch(IOException ex) {
ex.printStackTrace();
}
}
As you can see, I have an infinitive loop receiving messages until it is told to stop. At the moment the loop receives all the messages. There is a problem with that. The class where the code is used is a model class in MVC. In the class I also have a method called getContacts(). It is used to receive contacts from the phone over Bluetooth. The phone is told to send the contacts when the server sends --#do:getcontacts.
What I need to do is to get the contacts in an ArrayList in the getContacts() method and return it as the return value of the method so that the controller can handle the contacts.
public ArrayList<Contact> getContacts() {
ArrayList<Contact> contacts = new ArrayList<>();
// How do I get the contacts in the ArrayList?
return contacts;
}

I'll have you some suggestions. My examples are no working code, just a working base for you.
First, I strongly suggest you to use threads in a server. Everytime a clients connects to the server, you create a new thread, with parameters containing all data needed to start it:
boolean running = true; //this class variable will allow you to shut down the server correctly
public void stopServer(){ //this method will shut down the server
this.running = false;
}
public void run() {
...
while(running) {
// if(this.connection == null) { // I removed this line since it's unnecessary, or even harmful!
StreamConnection connection = this.server.acceptAndOpen(); //This line will block until a connection is made...
System.out.println("INFO: Bluetooth client connected");
Thread thread = new ServerThread(connection);
thread.start() //don't forget exception handling...
}
}
And in the class ServerThread you implement these lines handling the clients (non-compiled code, without exception handling!):
Class ServerThread extends Thread {
StreamConnection connection;
public ServerThread(StreamConnection connection){
this.connection = connection;
}
public void run() {
...
connection.close(); //closing the connection...don't forget exception handling!
System.out.println("INFO: Client disconnected");
}
}
What is the advantage of this code? Your server is now able to handle a thousand clients at the same moment. You've got parallelisation, and that's how server usually work! A server without threads is like socks without shoes...
Second, if you have a Java client and a Java server, you can use a much easier way to send your objects to the server: ObjectOutputStream/ObjectInputStream. You just send the array (I'll use an ArraList as it would be usual) containing the contacts to the server, and then you read the array. Here is the code for the server (again uncompiled and without any exception handling):
Class ServerThread extends Thread {
StreamConnection connection;
public ServerThread(StreamConnection connection){
this.connection = connection;
}
public void run() {
BufferedInputStream bis = new BufferedInputStream(this.connection.openInputStream());
ObjectInputStream ois = new ObjectInputStream(bis);
ArrayList contacts = (ArrayList) ois.readObject(); //this is a cast: don't forget exception handling!
//You could also try the method ois.readUTF(); especially if you wanna use other non-Java clients
System.out.println("INFO: Received from Bluetooth: " + contacts);
this.connection.close(); //closing the connection...don't forget exception handling!
//ois.close(); //do this instead of "this.connection.close()" if you want the connection to be open...i.e. to receive more data
System.out.println("INFO: Client disconnected");
//here you do whatever you wanna do with the contacts array, maybe add to your other contacts?
}
}
In Java, every class is an object, including ArrayList. And since the end of the object will be regarded as the disconnection, you won't need to do anything else.
Third: You use above server not only for bluetooth connections, but also for WLAN connections, aso. Then you could easily start different threads, like in pseudo code if(connection.isBluetooth()){//create a thread from BluetoothThread} else if(connection.isWLAN()){//create a thread from WLANsThread}. I don't know what your app is about, but maybe one day you would like to expand it to desktop PC, so using WLAN would be the right thing. Also because you anyway need to build in a verification in the client ("which contacts are going to be sent to which server?"), no matter if it's bluetooth or WLAN, because the low range of buetooth can't give you any security. ;)
Fourth, finally about your question: To get something, you need to have a data source and/or a class variable. Here a short example with a file that stores the contacts (but it could also be a database...local or somewhere else!):
public class MyApp(){
ArrayList contacts;
...
public void run(){ //this happens when we start our app
this.contacts = new ArrayList();
FileReader fr = new FileReader ("C:\WhereverYourFileIs\Contacts.file");
BufferedReader br = new BufferedReader(fr);
//here you use a loop to read the contacts via "br" from the file and fill them into your array...I can't provide you more code, since the exact data structure is up to you.
}
//now we want to send our contacts array to the already connected server:
public sendArrayToServer() {
BufferedOutputStream bos = new BufferedOutputStream (this.connection.openOutputStream());
ObjectOutputStream oos = new ObjectOutputStream(bos);
oos.writeObject(this.contacts);
//If you use readUTF() in the server, you need to call here something like oos.writeUTF(this.contacts.toString()); or even need to use another parser method which exactly creates the string you want.
this.connection.close(); //closing the connection...don't forget exception handling!
//oos.close(); //do this instead of "this.connection.close()" if you want the connection to stay open...
}
}
Now in the server you just read out the contacts array as already I described above. What you do with those contacts, remains up to you.
Hope this helps you to understand your problems and find a solution. Programming is all about trial & error..and improving your code.
EDIT:
After our discussion I finally found out what you need: You need a one-thread server called BluetoothManager that interacts with another thread called GUIController. Now since I anyway did the implementation in my head, I can post it for you, together with some explanations. Just note that in this case you don't need to initialize another thread in the server, since the BluetoothManager is already a thread, and you anyway need only one connection at the same moment (the question remains, if that is a "server", I would rather call it a "receiver"):
Public class BluetoothManager extends Thread{
boolean running = true; //this class variable will allow you to shut down the server correctly
GUIController controller;
public BluetoothManager(GUIController controller){
this.controller = controller; //this registers the GUIController in the BluetoothManager
}
public void stop(){ //this method will shut down the "server"
this.running = false;
}
public void run() {
this.localDevice = LocalDevice.getLocalDevice();
this.localDevice.setDiscoverable(DiscoveryAgent.GIAC);
this.server = (StreamConnectionNotifier) Connector.open(URL);
while(running){
StreamConnection connection = this.server.acceptAndOpen(); //This line will block until a connection is made...or running==false!
System.out.println("INFO: Bluetooth client connected");
BufferedInputStream bis = new BufferedInputStream(this.connection.openInputStream());
ObjectInputStream ois = new ObjectInputStream(bis);
ArrayList contacts = (ArrayList) ois.readObject(); //this is a cast: don't forget exception handling!
System.out.println("INFO: Received from Bluetooth: " + contacts);
this.connection.close(); //closing the connection...don't forget exception handling!
System.out.println("INFO: Client disconnected");
this.controller.refreshContacts(contacts);
}
}
}
public class GUIController extends Thread implements Runnable {
ArrayList contacts; //also a HashMap may be appropriate
BluetoothManager manager;
public void run(){
this.contacts = new ArrayList();
FileReader fr = new FileReader ("C:\WhereverYourFileIs\Contacts.file");
BufferedReader br = new BufferedReader(fr);
//here you use a loop to read the contacts via "br" from the file and fill them into your array...I can't provide you more code, since the exact data structure is up to you.
}
public void startBluetoothManager(){ //starting the BluetoothManager
this.manager = new BluetoothManager(this);
this.manager.start();
}
public void abortBluetoothManager(){ //call this when clicking on the "Abort" button
this.manager.stop();
//now the next 2 lines you normally don't need...still may use it if you've problems shutting down the thread:
// try{ this.manager.interrupt(); } //we want to be 100% sure to shut down our thread!
// catch(Exception e){}
this.manager = null; //now the garbage collector can clean everything...byebye
}
public void refreshContacts(ArrayList contacts) {
// synchronize(this.contactArray){ //no synchronisation needed if you have a GUI pop-up with an "Abort"-button!
Iterator i = this.contacts.iterator();
while(i.hasNext()){
this.contacts.add(i.next());
}
//At the end you need remove the "Receiving message" pop-up together with the "Abort Receiving"-button, these are all class variables!
// important note: If you have unique entries, you may need to replace them! In this case I suggest storing all contact objects better in a HashMap contacts, and use the unique ID as a key to find the element. And then you may prompt the user, if there are identical entries, to overwrite each entry or not. These things remain all up to you.
}
}
//As always: This is no compiled code!!
The GUIController first runs the BluetoothManager with startBluetoothManager() and does nothing else, except showing a notification "Receiving contacts" and an "Abort Reveiving"-button. And when the BluetoothManager is finished, he just adds the new contacts into the existing contacts-array inside the GUIController by calling refreshContacts(...). If you push the "Abort Reveiving"-button, you immediately call the abortBluetoothManager() method, which sets running=false in the BluetoothManager to end the server and finish the thread.
The main problem this solution solves: It's not possible for two threads to directly communicate with each other! Once you call thread.start(), every thread is on its own. That's why there is no possibility for the BluetoothManager-thread to tell the GUIController-thread "I've finished!". The only thing those threads can do, is share the same ressource(s), and communicate via this ressource(s). In our case, it's the contacts-ArrayList in the GUIController, which first I thought needs to be synchronized and can be updated by both threads (but not on the same time). And - kind of funnyness - there is a second shared ressource, it's actually the running flag in the class BluetoothManager which can shut it down (but there is never any synchronisation of running needed, this variable is only changed by the GUIController).
Now about the synchronisation: I thought about this problem more and understood, that you can solve your problem also without any "synchronized(...)" call. So, if you don't want to synchronize the ArrayList, you must do it like this: While the server is running, you only show the "Receiving Contacts" pop-up and the "Abort Reveiving"-button. While this happens, you just never access the contact-ArrayList inside the GUIController. This is somehow an "intrinsic synchronisation" which needs no real Java synchronisation. Still you may implement synchronisation, just to be 100% sure that nothing happens if you expand the app in the future.

First of all there are few things in you code need to be reviewed/fixed
1- the ArrayList<Contact> contacts should be defined in your class, so the thread can have access to it and populate it not as a local variable within getContacts() method
public ArrayList<Contact> getContacts() {
//ArrayList<Contact> contacts = new ArrayList<>();
return contacts;
}
2- You should avoid using infinite loop within the run method to be able to stop the thread when you want to.
//while(true)
while(isRunning) { // a flag that is set to true by default
}
3- Checking that the connection equal without setting it to null after disconnecting means that the connection will be accepted only from the first client (assuming that connection is originally set to null) and afterwards your will just have an infinite loop but the code this.connection = this.server.acceptAndOpen(); will not be reachable anymore
if(this.connection == null) {
while((line = reader.readLine()) != null) {
if(line.equals("--#do:disconnect")) {
// You have to set it to null if you want to continue listening after disconnecting
this.connection = null
break;
}
}
}
Or simply remove this check altogether, I see it is useless.
Now back to your question:
You can define your contact list as a class member to be accessible by both run() and getContacts() methods. You can make it final if needed. Then populate this list within the run() method; that's all.
e.g.
class MyServerThread implements Runnable {
private boolean isRunning = true;
ArrayList<Contact> contacts = new ArrayList<>();
public ArrayList<Contact> getContacts(){
// Make sure that your not currently updating the contacts when this method is called
// you can define a boolean flag and/or use synchronization
return contacts;
}
public void run() {
...
while(isRunning ) {
this.connection = this.server.acceptAndOpen();
System.out.println("INFO: Bluetooth client connected");
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.openInputStream()));
this.writer = new BufferedWriter(new OutputStreamWriter(connection.openOutputStream()));
// You need to remove previously received contacts before issuing a new --#do:getcontacts command
contacts.clear();
String line;
while((line = reader.readLine()) != null) {
if(line.equals("--#do:disconnect")) {
break;
}
// Here you can parse the contact information
String contactName = ...
String contactPhone = ...
contacts.add(new Contact(contactName,contactPhone));
}
System.out.println("INFO: Client disconnected");
}
} catch(BluetoothStateException ex) {
ex.printStackTrace();
} catch(IOException ex) {
ex.printStackTrace();
}
}
}
You do not have to use Object serialization, you can construct a simple protocol to send contacts from the phone to the PC, something similar to the commands you send e.g. --#C:name$phone