We Keep Coding

How can I make some class, running in the Main Thread, await concurrently for other class to reach some specific state?

I am having serious difficulties to understand how can I make some AsyncTask children, declared and instantiated in the Main Thread, to await for a Service child instance to reach some specific state.
As code examples here is the relevant part for Service; this code does what expected: receives the JSON response and holds it.
public class MyService extends Service {
private boolean received = false;
private string url = "http://someserver.mine/get-data-in-json-format";
// [...]
#Override
public void onCreate() {
doHttpJsonQuery();
}
public boolean responseReceived() {
return this.received;
}
public List<MyModel> getResponseAsObject() {
if (!this.received) return new ArrayList<MyModel>;
// Many code lines that convert the data into a list.
// [...]
return the_list;
}
// [...]
private void doHttpJsonQuery() {
OkHttpClient client = new OkHttpClient.Builder()
.build();
Request request = new Request.Builder()
.url(url)
.build();
client.newCall(request).enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
e.printStackTrace();
call.cancel();
}
#Override
public void onResponse(Call call, Response response) throws IOException {
final String myResponse = response.body().string();
//...and some code to hold data as JSONArray
//[...]
}
});
this.received = true;
}
}
This Service works; fine. Then, from another class (which purpose will be to handle persistence inserting the received data in a local Room Database), I try to do the following (here's where my mind is blown):
public class DataRepository {
private MyRoomDatabase db;
private MyModelDao mModelDao;
// I'm skipping irrelevant code on purpose
// [...]
public DataRepository(Application application) {
db = MyRoomDatabase.getDatabase(application);
mModelDao = db.myModelDao();
// [...]
// Here I instance a subclass of ContextWrapper(i named it RemoteDataSource) which
// responsability will be handling different Services for making HTTP operations
mRemoteDataSource = new RemoteDataSource(application.getApplicationContext());
// It holds a reference to MyService. It has some public methods, like this one, to
// control the referenced Service from outside with some encaspsulation
mRemoteDataSource.startMyService();
// Instantiating a private nested class...
PopulateDbAsync mPopulateDbAsync = new PopulateDbAsync(db);
mPopulateDbAsync.doInBackground();
}
// [...]
// Here is the failing code
private class PopulateDbAsync extends AsyncTask<Void, Void, Void> {
PopulateDbAsync(MyRoomDatabase db) {}
#Override
protected Void doInBackground(final Void... params) {
MyService mService = mRemoteDataSource.getMyService();
if (mService == null) {
// This doesn't happen at all right now...
Log.e("MY_ERROR","DataRepository.PopulateDbAsync --> MyService from RemoteDataSource is NULL!!!!");
}
List<MyModel> the_list = mService.getResponseAsObject();
if (the_list == null) {
// HERE! I obtain the NullReferenceException here.
// I am confused about how would I avoid this flaw in my code
Log.e("MY_ERROR", "DataRepository.PopulateDbAsync --> error: response isn't ready yet.");
}
for (MyModel i_model : the_list) {
Log.d("MY_LOG", "DataRepository.PopulateDbAsync --> Inserting data in local DB...");
mModelDao.insert(i_model);
}
return null;
}
}
}
Summarizing: my problem is that I will always get NullReferenceException in this line:
for (MyModel i_model : the_list) {
I am not familiar with multithreading, asyncronous operations and concurrent execution. I have been reading, for two weeks, lots of different documents on the Internet both from Android Official Website and from other websites as well, trying to figure it out... "AsyncTask is not good to perform this kind of operations"... so, what infrastructure should I implement, I have been wondering... should I use Handlers, Threads, Messengers, or what? The more I read, the more confused I get. It's like I have an Analysis Paralysis issue...
Most of the examples I find out there provide too verbose code examples on how to implement multithreading and concurrent execution; while I read them and try to figure out how to implement those structures in my code, I just get stuck; also, with so many classes to choose, I get even more confused...
Due to the HTTP call will need to be performed asyncronously (and response time will not always be the same), I am trying to figure out how to make the piece of code that throws the NullReferenceException "wait" for MyService to complete it's job before starting it's execution; while loops will not work due to it would break Main Thread's lifecycle. "Knowing" if the Service completed it's task or not would be as simple as using the boolean method responseReceived. The big idea is, every time new data is obtained through HTTP, updating the RoomDatabase with it, and, meanwhile, MainActivity would be showing the current local data (if any, or an empty list if there's nothing yet).
So, when I get it, I will understand how to refactor the whole code structure properly to start adding more Service child instances into my RemoteDataSource class, which I created with the idea of having all Service childs that will use OkHttp to perform HTTP communications, wrapped together in a single class for better organization.
What would be the proper way to achieve what I am looking for about this? Would someone be able to provide some short example explaining the code structure I will need for something like this? Examples with empty blocks containing comments like "code to execute when ready here" would be great so I can figure it out.
The question exposed here is related with the same project that made me post this other question some weeks ago; I have been reading here and there, performing some trial-and-error and correcting some code issues here-and-there since then; however, I am making a different question here; finding an answer for this would probably be the first step towards figuring out an answer to the other question as well.
URL References to documentation I have been reading
Some of the documentation I have been reading (but not limited to):
AsyncTask class documentation
Handler class documentation
Basics on Multithreading
Introduction to background processing in Android - Tutorial
Thread With Handlers - Android Example
Messenger class documentation

Well problem is with your application logic as follows,
If you are using AsyncTask that is obviously a separate thread from the main thread. But syncing to your database after retrieving data via HTTP call is a process which has a sequence ( Call through HTTP and retreive -> then persist to database ), it cannot perform asynchronously. So when you call,
List<MyModel> the_list = mService.getResponseAsObject();
this call happens in a particular thread and the program flow is in a different thread.
Since these are asynchronous tasks, they work asynchronously. which means you will never know which one will execute first and which one is next. But as per your logic,
if (the_list == null) {
this part essentially need the_list to be initialized to run. But the problem is at that point, service thread has not finished his work to perform your next logic. so its obvious breaking.
Better if you can re-design this so that you wait for the HTTP request to complete and then persist to database. Because suppose if your HTTP request complets first but still it returns you null or whatever not-desired output. So in that case you need to handle it in your logic.
OK so let me tell you a quick workaround.
Lets use just one thread instead of different threads. So consider changing following line
private class PopulateDbAsync extends AsyncTask<Void, Void, Void> {
to
private class PopulateDbAsync
then you will get an error with
#Override
protected Void doInBackground(final Void... params) {
since we no longer extend AsyncTask class.
so change it as follows, by removing #Override
public Void doInBackground(final Void... params) {
This should fix the stated problem here.

I found a solution: creating custom listeners.
Steps to create a custom listener
1. Define an interface as an event contract with methods that define
events and arguments which are relevant event data.
2. Setup a listener member variable and setter in the child object which can be assigned an implementation of the interface.
3. Owner passes in a listener which implements the interface and handles the events from the child object.
4. Trigger events on the defined listener when the object wants to communicate events to it's owner
I got the NullReferenceException because MyService didn't finish it's job yet. So, first I create the listener's structure within MyService class like this (steps 1 and 2):
private MyServiceListener listener;
public interface MyServiceListener {
public void onDataDownloaded();
}
public void setMyServiceListener(MyServiceListener listener) {
this.listener = listener;
}
And, within the HTTP request's callback (step 4):
#Override
public void onResponse(Call call, Response response) throws IOException {
final String myResponse = response.body().string();
//...and some code to hold data as JSONArray
//[...]
// XXX Trigger the custom event
if (listener != null) {
listener.onDataDownloaded();
}
}
Now, I just can wrap the code that triggered the NullReferenceException within the custom listener like this (step 3):
// Within DataRepository class
mService.setMyServiceListener(new MyService.MyServiceListener) {
#Override
public void onDataDownloaded() {
List<MyModel> the_list = mService.getResponseAsObject();
if (the_list == null) {
// HERE! I obtainED the NullReferenceException here.
Log.e("MY_ERROR", "DataRepository.PopulateDbAsync --> error: response isn't ready yet.");
}
for (MyModel i_model : the_list) {
Log.d("MY_LOG", "DataRepository.PopulateDbAsync --> Inserting data in local DB...");
mModelDao.insert(i_model);
}
return null;
}
}
Actually, the real implementation required to nest this code example into another custom listener following similar steps; but this worked for me.

How to stop repeating myself in Java

I use a method for more than one time in JavaScript by using callback method because JavaScript is an async language.
Example:
function missionOne () {
sumCalculation(1, 2, function (result) {
console.log(result) // writes 3
})
}
function sumCalculation (param1, param2, callback) {
let result = param1 + param2
// The things that take long time can be done here
callback(result)
}
I wonder if there is any way to stop myself in Java?
Edit: I remove several sentences that make more complex the question.

I may be reading too much into your question, but it seems that you're looking into how to handle asynchronous code in Android. There are a couple of native options (not considering any library). I'll focus on two, but keep in mind there are other options.
AsyncTasks
From the documentation
AsyncTask enables proper and easy use of the UI thread. This class allows you to perform background operations and publish results on the UI thread without having to manipulate threads and/or handlers.
Before writing one, you need to know which type of parameters it will receive, the type of progress it will publish during computation and what is its return type. These types are define via the AsyncTask generic Parameters AsyncTask<Params,Progress,Result>. If you don't need them any of them, set them to Void
Here's the basic gist of using an AsyncTask to compute the sum of two ints:
public void sumCalculation (int param1, int param2, Callback callback) {
new AsyncTask<Integer, Void, Integer>() {
#Override
public Integer doInBackground(Integer... params) {
int result = 0;
for (Integer param : params) {
result += param;
}
return result;
}
#Override
protected void onPostExecute(Integer integer) {
super.onPostExecute(integer);
callback.onDone(integer);
}
}.execute(param1, param2);
}
doInBackground, as the name says, will execute a certain piece of code in a background thread. Please note that every AsyncTask will run on a ThreadPool of size 1, so they actually get in the way of other AsyncTasks.
onPostExecute brings the result back to the main thread, so you can update any UI componente. If you try to update the UI from a background thread, an exception will be thrown.
The down side of this particular example is the creation of a new AsyncTask every time that function is called.
Also you should use AsyncTask only if the task won't run for a very long time, couple of seconds at most.
Thread and Handler
Another option suggested on the documentation is using a thread and a handler to communicate between the main thread and a background thread. Although this provides greater flexibility, it also requires more responsibility as you will be responsible for managing the communication yourself, picking the right time to kill your threads and how to recover when something goes bad.
As a rule of thumb, you should only go this way if you really need the extra flexibility.
The overall idea is to create your own Handler and override its handleMessage method.
public class MyHandler {
#Override
public void handleMessage(Message inputMessage) {
int messageType = inputMessage.what;
Object extraData = inputMessage.obj;
...
}
}
public class MyTask extends Thread {
public static public int COMPUTATION_DONE = 0;
private MyHandler handler;
public MyTask(MyHandler handler) {
this.handler = handler;
}
#Override
public void run() {
//do your computation
Message message = handler.obtainMessage(COMPUTATION_DONE, your_result);
handler.sendMessage(message);
}
}
As you can see, this requiring parsing inputMessage.what and deciding what to do with it. Additionally, you need to cast inputMessage.obj to the right type and so on.
These are just two examples, but depending on what you're trying to do, you might need to dig deeper into Services or take a look at some reactive approach, such as RxJava2. However I encourage you to start with the basic before diving into something way more complicated.

Yes it is easy in Java. To take your example above you can write it in Java like this:
public static void main(String[] args) {
System.out.println(sumCalc(1,2));
}
private int sumCalc(int first, int second) {
return first + second;
}

How to read Message in netty in other class

I want to read a message at a specific position in an class other than InboundHandler. I can't find a way to read it expect in the channelRead0 method, which is called from the netty framework.
For example:
context.writeMessage("message");
String msg = context.readMessage;
If this is not possible, how can I map a result, which I get in the channelRead0 method to a specific call I made in another class?

The Netty framework is designed to be asynchronously driven. Using this analogy, it can handle large amount of connections with minimal threading usage. I you are creating an api that uses the netty framework to dispatch calls to a remote location, you should use the same analogy for your calls.
Instead of making your api return the value direct, make it return a Future<?> or a Promise<?>. There are different ways of implementing this system in your application, the simplest way is creating a custom handler that maps the incoming requests to the Promises in a FIFO queue.
An example of this could be the following:
This is heavily based on this answer that I submitted in the past.
We start with out handler that maps the requests to requests in our pipeline:
public class MyLastHandler extends SimpleInboundHandler<String> {
private final SynchronousQueue<Promise<String>> queue;
public MyLastHandler (SynchronousQueue<Promise<String>> queue) {
super();
this.queue = queue;
}
// The following is called messageReceived(ChannelHandlerContext, String) in 5.0.
#Override
public void channelRead0(ChannelHandlerContext ctx, String msg) {
this.queue.remove().setSuccss(msg);
// Or setFailure(Throwable)
}
}
We then need to have a method of sending the commands to a remote server:
Channel channel = ....;
SynchronousQueue<Promise<String>> queue = ....;
public Future<String> sendCommandAsync(String command) {
return sendCommandAsync(command, new DefaultPromise<>());
}
public Future<String> sendCommandAsync(String command, Promise<String> promise) {
synchronized(channel) {
queue.offer(promise);
channel.write(command);
}
channel.flush();
}
After we have done our methods, we need a way to call it:
sendCommandAsync("USER anonymous",
new DefaultPromise<>().addListener(
(Future<String> f) -> {
String response = f.get();
if (response.startWidth("331")) {
// do something
}
// etc
}
)
);
If the called would like to use our a api as a blocking call, he can also do that:
String response = sendCommandAsync("USER anonymous").get();
if (response.startWidth("331")) {
// do something
}
// etc
Notice that Future.get() can throw an InterruptedException if the Thread state is interrupted, unlike a socket read operation, who can only be cancelled by some interaction on the socket. This exception should not be a problem in the FutureListener.

Using java futures without requiring local mutable state

I've written some code that essentially is responsible for orchestrating a number of API's in sequence through a library method I provide to my clients called "orchestrate" (yes I know so original). What sits behind this orchestrate method is nothing more than a loop that executes API's in the order they are received, which in turn are delegated to a number of classes that contain some business logic for building a request, calling an API, performing some validation on the response and finally returning the api result. So, if a client sent in a list of apis:
{a1, a2, a3, a4, a5} it would execute each api in sequence in a completely blocking way.
I'm attempting to beef this up to where I'm able to call multiple API's in parallel depending on how I receive the instructions from a client. Think of it as the client sending me a list of lists like: { {a1, a2}, {a3}, {a4, a5} }
This means I'd like to execute a1 and a2 in parallel (which means build their request, call the apis, validate the response). Then wait until i'm sure both of them are done. Then execute a3, and wait until i'm sure it's done. Finally I want to execute a4 and a5 and follow the usual pattern.
Now, I'm tempted to use futures for the simple abstraction they provide to execute methods a wait for the response using the .get() method. But what I noticed that the executorService needs underneath is future's invocation of a call() method. This is fine, but makes me think that the call() method that is implemented per class may need access to "local" data in order to do its job (after all, I can't pass the call() method any particular parameters). I really want to avoid holding any local mutable state because that brings its own side-effects.
Is there a way for me to NOT hold local state and still use futures to handle my multithreading use-case? Or is my understanding of futures completely wrong and I'm missing something obvious? If not, are there any recommendations on a good path forward with some alternatives?

OK, so you have a class which gets you some data from a web page in a blocking way, and takes some arguments:
public class DataFetcher {
public Data fetchData(int param1, int param2) {
// ...
}
}
And now you want to execute this method twice, concurrently, and get back futures. So you just need to create a Callable:
final DataFetcher fetcher = new DataFetcher();
Callable<Data> task1 = new Callable<>() {
#Override
public Data call() {
return fetcher.fetchData(1, 2);
}
};
Callable<Data> task2 = new Callable<>() {
#Override
public Data call() {
return fetcher.fetchData(3, 4);
}
};
Future<Data> result1 = executorService.submit(task1);
Future<Data> result2 = executorService.submit(task2);
I don't see any mutable state here.
To avoid repeating the code and using anonymous classes, you can define a top-level class:
public DataFetcherTask implements Callable<Data> {
private final DataFetcher fetcher;
private final int param1;
private final int param2;
public DataFetcherTask(DataFetcher fetcher, int p1, int p1) {
this.fetcher = fetcher;
this.param1 = p1;
this.param2 = p2;
}
#Override
public Data call() {
return fetcher.fetchData(param1, param2);
}
};
and then use it like this:
Future<Data> result1 = executorService.submit(new DataFetcherTask(fetcher, 1, 2));
Future<Data> result2 = executorService.submit(new DataFetcherTask(fetcher, 3, 4));
Still no trace of mutable state here.

Jersey/JAX-RS 2 AsyncResponse - how to keep track of current long-polling callers

My goal is to support long-polling for multiple web service callers, and to keep track of which callers are currently "parked" on a long poll (i.e., connected). By "long polling," I mean that a caller calls a web service and the server (the web service) does not return immediately, but keeps the caller waiting for some preset period of time (an hour in my application), or returns sooner if the server has a message to send to the caller (in which case the server returns the message by calling asyncResponse.resume("MESSAGE")).
I'll break this into two questions.
First question: is this a reasonable way to "park" the callers who are long-polling?
#GET
#Produces(MediaType.TEXT_PLAIN)
#ManagedAsync
#Path("/poll/{id}")
public Response poller(#Suspended final AsyncResponse asyncResponse, #PathParam("id") String callerId) {
// add this asyncResponse to a HashMap that is persisted across web service calls by Jersey.
// other application components that may have a message to send to a caller will look up the
// caller by callerId in this HashMap and call resume() on its asyncResponse.
callerIdAsyncResponseHashMap.put(callerId, asyncResponse);
asyncResponse.setTimeout(3600, TimeUnit.SECONDS);
asyncResponse.setTimeoutHandler(new TimeoutHandler() {
#Override
public void handleTimeout(AsyncResponse asyncResponse) {
asyncResponse.resume(Response.ok("TIMEOUT").build());
}
});
return Response.ok("COMPLETE").build();
}
This works fine. I'm just not sure if it's following best practices. It seems odd to have the "return Response..." line at the end of the method. This line is executed when the caller first connects, but, as I understand it, the "COMPLETE" result is never actually returned to the caller. The caller either gets "TIMEOUT" response or some other response message sent by the server via asyncResponse.resume(), when the server needs to notify the caller of an event.
Second question: my current challenge is to accurately reflect the population of currently-polling callers in the HashMap. When a caller stops polling, I need to remove its entry from the HashMap. A caller can leave for three reasons: 1) the 3600 seconds elapse and so it times out, 2) another application component looks up the caller in the HashMap and calls asyncResponse.resume("MESSAGE"), and 3) the HTTP connection is broken for some reason, such as somebody turning off the computer running the client application.
So, JAX-RS has two callbacks I can register to be notified of connections ending: CompletionCallback (for my end-poll reasons #1 and #2 above), and ConnectionCallback (for my end-poll reason #3 above).
I can add these to my web service method like this:
#GET
#Produces(MediaType.TEXT_PLAIN)
#ManagedAsync
#Path("/poll/{id}")
public Response poller(#Suspended final AsyncResponse asyncResponse, #PathParam("id") String callerId) {
asyncResponse.register(new CompletionCallback() {
#Override
public void onComplete(Throwable throwable) {
//?
}
});
asyncResponse.register(new ConnectionCallback() {
#Override
public void onDisconnect(AsyncResponse disconnected) {
//?
}
});
// add this asyncResponse to a HashMap that is persisted across web service calls by Jersey.
// other application components that may have a message to send to a caller will look up the
// caller by callerId in this HashMap and call resume() on its asyncResponse.
callerIdAsyncResponseHashMap.put(callerId, asyncResponse);
asyncResponse.setTimeout(3600, TimeUnit.SECONDS);
asyncResponse.setTimeoutHandler(new TimeoutHandler() {
#Override
public void handleTimeout(AsyncResponse asyncResponse) {
asyncResponse.resume(Response.ok("TIMEOUT").build());
}
});
return Response.ok("COMPLETE").build();
}
The challenge, as I said, is to use these two callbacks to remove no-longer-polling callers from the HashMap. The ConnectionCallback is actually the easier of the two. Since it receives an asyncResponse instance as a parameter, I can use that to remove the corresponding entry from the HashMap, like this:
asyncResponse.register(new ConnectionCallback() {
#Override
public void onDisconnect(AsyncResponse disconnected) {
Iterator<Map.Entry<String, AsyncResponse>> iterator = callerIdAsyncResponseHashMap.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, AsyncResponse> entry = iterator.next();
if (entry.getValue().equals(disconnected)) {
iterator.remove();
break;
}
}
}
});
For the CompletionCallback, though, since the asyncResponse is already done or cancelled at the time the callback is triggered, no asyncResponse parameter is passed in. As a result, it seems the only solution is to run through the HashMap entries checking for done/cancelled ones and removing them, like the following. (Note that I don't need to know whether a caller left because resume() was called or because it timed out, so I don't look at the "throwable" parameter).
asyncResponse.register(new CompletionCallback() {
#Override
public void onComplete(Throwable throwable) {
Iterator<Map.Entry<String, AsyncResponse>> iterator = callerIdAsyncResponseHashMap.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, AsyncResponse> entry = iterator.next();
if (entry.getValue().isDone() || entry.getValue().isCancelled()) {
iterator.remove();
}
}
}
});
Any feedback would be appreciated. Does this approach seem reasonable? Is there a better or more Jersey/JAX-RS way to do it?

Your poller() method does not need to return a Response in order to participate in asynchronous processing. It can return void. If you are doing anything complex in the poller however you should consider wrapping the whole method in a try/catch block that resumes your AsyncResponse object with the exception to ensure that any RuntimeExceptions or other unchecked Throwables are not lost. Logging these exceptions in the catch block here also seems like a good idea.
I'm currently researching the question of how to reliably catch an asynchronous request being cancelled by the client and have read at one question that suggests the mechanism isn't working for the questioner[1]. I'll leave it to others to fill out this information for the moment.
[1] AsyncResponse ConnectionCallback does not fire in Jersey