We Keep Coding

How to "fire and forget" with kotlin coroutines in a blocking event?

I am making something similar to minecraft plugin, but the API is all in Java.
At some point, there are events. Every event is fired sequentially, and the API will wait until an event finishes before calling the event listener for the next event. These events are called from the main thread, this means that nothing else is running while your event listener code is being run, and the application will crash eventually if you are blocking the main thread for too long.
Basically, I want that when a player is opening an inventory, some first content is sent directly, and some second content is retrieved from a database. In Java, you, would do something like this:
private static final List<Item> firstContent = ...;
public static void onInventoryOpenEvent(Inventory inventory) {
inventory.addItems(firstContent);
forkJoinPool.submit(() -> { // "fire and forget"
List<Item> secondContent = retrieveContentFromDB(); // long and blocking
inventory.addItems(secondContent);
});
}
And in kotlin, some would say to use the GlobalScope, others would say to not use it, but for now I don't see any other way than using it like this:
fun onInventoryOpenEvent(inventory: Inventory) { // not a suspending function
inventory.addItems(firstContent)
GlobalScope.launch {
val secondContent = retrieveContentFromDB() // long and blocking
inventory.addItems(secondContent)
}
}

If you truly want to fire and forget, such that the coroutine will not be cancelled under any circumstance, GlobalScope is the right way to do it. The need to truly fire and forget is rare enough in practice that Kotlin gives you a warning about a "fragile API" if you use it, I think because so many newbies were using it for everything. This was especially a problem on Android (where Kotlin is primarily used), where long running tasks that should run even off-screen should be handled in services instead of coroutines.
But, referring to your example, if inventory is something that could become obsolete and you want to free it to the GC at some point, you should use your own created CoroutineScope that you store in a property, so you can cancel it to clear any running coroutines and prevent them from hanging onto references that should be freed for the GC. If the scope might be used for more than one coroutine, then you should give it a SupervisorJob().
private val coroutineScope = CoroutineScope(SupervisorJob())
fun onEndOfLifecycle() { // some function called when cleaning up memory
coroutineScope.cancel()
}

How can i modify a var value inside a function?

I wanted to set up this variable called userName, which should get a new value inside the ValueEventListener. However, when setting the new value inside the function, it doesn't change. The output is still ""
private fun getName(){
var userName = ""
val user = fbAuth.currentUser
var uid = user!!.uid
mDatabase = FirebaseDatabase.getInstance().getReference("users")
mDatabase.addValueEventListener(object : ValueEventListener {
override fun onCancelled(p0: DatabaseError) {
TODO("not implemented")
}
override fun onDataChange(snapshot: DataSnapshot) {
userName = snapshot.child(uid).child("name").getValue().toString()
}
})
println(userName)
}
Expected output: John (value of name child),
current output: ""

The listener is asynchronous, if you put the println statement below the username = line, then it will print.
In fact, go ahead and do that; observe the timestamps; which one prints first? The empty one or the one inside the callback?
the var is being modified by the callback, but the println executes first, long before (in computer times, that is) Firebase emits its value.
Additionally, I would invert the order of the mDatabase lines.
You are essentially requesting a value and then listening for results; the result may have already been emitted. You should add the listener first, then request the data.
Update: what if I need the value for another callback?
Welcome to the world of asynchronous programming :-)
What you describe is a set of independent asynchronous operations. You need value A, and value B, but you can't obtain value B, until you have value A. Both are asynchronous and take time, but you don't have time on the main thread, or rather, you have ~16ms to compute, measure, and draw your screen so the OS can keep up with 60 frames per second. That's not a lot of time and part of the reason why asynchronous programming exists!
This other answer already provides a working sample of what you need. This other external link has a more concrete example of the Observer Listener pattern.
In short, what you want is an instance of an object which can be invoked once the an operation completes.
In a regular synchronous function, each statement is executed after the other and no statement will be executed until the previous one is not finished; all statements are therefore, blocking statements.
For example:
var times = 2
var message = "Hello"
var world = "World"
println("The message is $message $times")
println(world)
Will print:
The message is Hello 2
World
This is because the execution point will go from one line to the other, waiting for the previous one to execute. If one operation takes time, the thread will be blocked (from performing anything else) until that operation completes and the execution point can move to the next instruction.
As you can imagine, the Main Thread in iOS and Android (and well, Windows, macOS, Linux, etc) cannot be blocked, or the OS wouldn't be able to respond to your touches and other things happening (for e.g., on a mobile phone, an incoming phone call wouldn't be able to be processed if the UI is not responsive and you cannot tap "answer").
This is why we use other "threads" to off-load things that are not super fast. This comes with a mindset change, as well as correct planning, for things are now more complicated.
Let's see a simple example (some pseudo code, so bear any obvious glaring mistakes, this is just to illustrate the point, not to write a solution).
fun main() {
var hello = "Hello"
var message = thisTakesTime()
println("The message is $hello $message")
println(hello)
}
fun thisTakesTime(): String {
// do something that takes 1 second (1000 ms)
Thread.sleep(1000)
return "World"
}
This will print
The message is Hello World
Hello
As you can see, nothing changed, except that for one entire second, the main thread was unresponsive. If you were to run this on Android, for example, it will work, but your app will not respond for a second, during the Thread.sleep. One second is fast, try 10 seconds; this exceeds the Android Operating System limit of 5 seconds for the main thread to be unresponsive, before deciding the ANR (application not responding) dialog is needed; this is the infamous "It looks like XXX application is not responding, wait or close".
What can you do?
Initially, if you have too many callbacks (where callback A cannot execute until callback B finished, and callback B cannot execute until callback C finished), and you start nesting them like that, you end up in the infamous Callback-Hell (in Javascript, but valid for any language/platform).
Basically tracking all these asynchronous callbacks and ensuring that by the time the response comes, your next callback is ready, and so forth is a pain, and it introduces exponential complexity if, for example, callback C fails in the middle, now you have to let callback B know that C failed and therefore it will have to fail too, which -in turn- has to let callback A (the original!) know that B failed, and therefore A has to do something about it, does A need to know that B failed because of C? or does A only cares for B and B alone and the reasons behind B's failure are irrelevant?
Well, as you can see, even talking about this gets complicated and messy and I didn't even cover other possible scenarios, equally as complex.
What I'm trying to say here is not that you shouldn't use callbacks; it's that you have to carefully plan where, and when to use them.
Kotlin has alternatives to reduce/remove the callback hell by using Coroutines but these are a moderately advanced topic and it also requires a fundamental change in how you design your components and pieces.
All in all, for your use case, remember the golden rule of OOP: Make small concrete classes that do very few things, and do them well. If you need to start adding too many if () all over the place, then chances are you're mixing business logic, random decisions, and "whatabout" cases all over the place.
Imagine you have a class that processes Location data and uploads it to a server.
You may be tempted to:
Write all the code in the Activity/Fragment (or ViewModel); quickly becomes a mess.
Create a LocationUtils with static methods (or singleton pattern); a mess already, but also hard to test, and mock. What if you need more than one type of processing? Or what if you want to store them in a database, are you going to add more static methods?
Create a small LocationProcessor class, that receives two points (lat/long) does the processing in a small function, and returns the processed data, then create another class called LocationUploader, that receives clean input from a Processor, and uploads it to a server. None of these classes should think about "what if I don't have permissions, what if the user turns location off", etc. These are problems that exceed the responsibility of a class whose intention was to process location coordinates, nothing else. There should be other classes responsible for that. Remember, small classes, small responsibilities == less to worry about in a single file.
Conclusion?
Well, at this point there are better answers which will give you the copy-paste version of what you're looking for; I believe the concept you have to take out of this wall of text today, is that in order to write modern, testable, and simple functional code, a change in how you plan things must happen.
Long story short: when things are not synchronous, you need to keep something (an object) ready to be called back (hence the name callback), listening (or observing) (hence why we call them listener or observers), the emission of something (usually called an Observable, because it can be "observed").
Good luck!

Yes, the listener is asynchronous, it will only work if you print the variable inside onDataChange method.
However you can use a callback strategy to wait for Firebase to return the data. Something like this:
interface MyCallback {
fun onCallback(value: String )
}
fun readData(myCallback: MyCallback){
mDatabase.addValueEventListener(object : ValueEventListener {
override fun onDataChange(snapshot: DataSnapshot) {
userName = snapshot.child(uid).child("name").getValue().toString()
myCallback.onCallback(value)
}
})
}
fun test(){
readData(object: MyCallback {
override fun onCallback(value : String) {
println(value)
}
})
}

As Martin says, it's an asynchronous operation, you should handle the text output after the asynchronous process has completed:
mDatabase.addValueEventListener(object : ValueEventListener {
override fun onCancelled(p0: DatabaseError) {
TODO("not implemented")
}
override fun onDataChange(snapshot: DataSnapshot) {
userName = snapshot.child(uid).child("name").getValue().toString()
println(userName) //--> Asynchronous request has ended, show the name
}
})

Why printing all running threads in Java unblocks an application?

I created a GUI Application in JavaFX which communicates with a serial device (I use a jssc connector). When I want to get data I send a communicate, then wait 1s until the wait function terminates. Communicate is also being sent, among other things, when I click one of the buttons (for starting a device, for identification, etc.). During development process I noticed a bug - communication hangs (but only when receiving messages, I still can send one-way communicates for ie. starting a device) when I click too many buttons too fast or I click buttons which send more than one communicate.
Communication is mainly handled by my own class SerialPortDevice. I create an object of a class' type and then invoke particular methods. Here's the method which waits for a message:
private String waitForMessage() throws SerialPortException {
long operationStartTime = System.currentTimeMillis();
long connectionTimeout = SerialPortCommunicationParameters.CONNECTION_TIMEOUT_IN_MILLIS;
String resultMessage = "";
do {
if (readEventOccurred) {
System.out.println();
resultMessage = receiveMessage();
System.out.println("After receiving a message");
messageReceived = true;
}
} while (((System.currentTimeMillis() - operationStartTime) < connectionTimeout) && (!messageReceived));
if (!readEventOccurred) {
resultMessage = NO_RESPONSE;
}
System.out.println("Time elapsed: " + (System.currentTimeMillis() - operationStartTime + "ms"));
return resultMessage;
}
One can notice that message is being received only when a flag readEventOccured is true. It's handled by my implementation of the SerialPortEventListener:
class SerialPortDeviceReader implements SerialPortEventListener {
private SerialPortDevice device;
SerialPortDeviceReader(SerialPortDevice device) {
this.device = device;
}
public void serialEvent(SerialPortEvent event) {
if (event.isRXCHAR()) {
System.out.println("Serial Event Occured!");
device.setReadEventOccurred(true);
}
}
}
readEventOccured is a boolean field in the SerialPortDevice class which contains waitForMessage function. Also, waitForMessage is invoked by another function, singleConversation:
String singleConversation(String testMessage) {
String resultMessage = NO_RESPONSE;
try {
openConnection();
sendMessage(testMessage);
resultMessage = waitForMessage();
closeConnection();
} catch (SerialPortException e) {
e.printStackTrace();
return resultMessage;
}
System.out.println();
readEventOccurred = false;
messageReceived = false;
return resultMessage;
}
... Which is the only function setting readEventOccured to false. And it is a "top level" function in a SerialPortDevice class which handles sending and receiving a communicates to and from a device.
So communication looks like this:
Button click -> Button handler invocation -> device.singleCommunication(buttons_specific_communicate) -> some methods run and then it comes to the waitForMessage -> method waits 1s for the event -> event occurs (everytime - I get the "Serial Event Occured" communicate) -> readEventOccured is being set to true -> if there is some time left (there is always some time left, everything lasts a miliseconds), message is being received in the waitForMessage method.
There is no problem if I click buttons with a short (in a human sense, 2-3s for example) delay or I don't click these which do send more than one communicate inside their handlers. In different situations weird things happen. I still get the message "Serial Event Occured" (so I suppose readEventOccured is also being set to true) but waitForMessage function does not execute
if(readEventOccured)
statement's code. Also, I have to run an application again to be able to communicate with a device (I mean receiving data, sending works perfectly).
What solved my problem was adding "volatile" modifier to the readEventOccured flag (makes sense by the way, things go fast sometimes). But it didn't please me. I wanted to make a code to run correctly without "volatile". My coworker came up with an idea something goes wrong with threads that are being created when I click buttons and invoke communication - maybe something blocks other thing somewhere? First thing I did was to print all the currently runnig threads and... Man, it solved a problem. App was not hanging anymore. Really, I performed "hanging" scenarios 10-20 times with and without
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
System.out.println(threadSet);
at the beginning of the waitForMessage method and result were unambigous - it somehow removed my problem.
I'm almost certainly sure getting and printing threads is not a solution itself. It's something that is being done by the way but I can't find out what is this. Any useful information? Maybe better understanding Threads in Java would help me? Or it is something else?
Cheers

What you're doing is the textbook example of what happens when there are no visibility guarantees. If we distil your code into its essential bits, we get something like this:
boolean flag = false;
void consume() {
while (true) {
if (flag) {
//we received flag, hooray
}
}
}
void produce() {
flag = true;
}
Now, if produce() and consume() run in different threads, there is absolutely no guarantee that consume() will EVER see flag set to true. volatile creates a memory barrier, meaning that all reads/writes of those variable will be fully ordered.
What you also have in the code though, is lots of System.out.println() lines. These complicate the picture, as they themselves are synchronized, and thus create happens-before relationships between different parts of the code. Unfortunately they don't create the correct sequence, but poke in a few more and you can accidentally get it right. The key word here is "accidentally", you shouldn't rely on this behaviour at all.
So marking readEventOccurred as volatile sort of fixes the problem but going a bit further we can see that your waitForMessage() spin-waits, which is rarely a good idea. I would take a look at the CountDownLatch class for example, designed for similar scenarios. (An even better candidate is its close friend, CyclicBarrier.)

What solved my problem was adding "volatile" modifier to the readEventOccured flag (makes sense by the way, things go fast sometimes). But it didn't please me. I wanted to make a code to run correctly without "volatile".
The fact that adding volatile fixes the problem indicates you've run into the lack of guarantees by the Java Memory Model when multiple threads are involved. Simply put there is no guarantee of when a change on one thread is visible on other threads except for specific situations.
The probably reason why printing 'fixes' the problem is the fact that it:
Changes the timing of the program, might mean the situation in which it fails no longer exists.
Temporarily suspends the threads to read out their current state, maybe the JVM also uses that as an opportunity to synchronize memory.
Multithreaded communication is hard, so I recommend you take a look at the classes available in java.util.concurrent, which have guarantees that you might be able to use to solve your problem more elegantly.

How to make (and save in RAM) sparsearray at app launch? (android)

I am making an android app. There is an activity in the app, which when triggered, makes a sparsearray and fills it with data. Now this process takes upto 1 minute on the emulator, which is very long. So I want to make that sparsearray once, right when the app is launched, and save the sparsearray in RAM for the lifetime of the app. And whenever the activity is triggered it should access the sparsearray in RAM instead of making a new one, thus saving time and processing power. Can this be done, if so how? Sorry if this question is dumb, I am new to android. Thanks!
*Edit: This is what the sparsearray making function looks like:
//function which accesses sparsearray making function
public String[] process(){
InputStream is = context.getAssets().open("feedtitlesandaddresses.txt");
InputStreamReader iz=new InputStreamReader(is);
BufferedReader br = new BufferedReader(iz);
String line = null;
while((line=br.readLine())!=null) {
readLine(line);
}}
//sparsearray making function
private void readLine(String line) {
//some string processing(omitted here)
int num1 = Integer.parseInt(firstNumber);
//int num2 = Integer.parseInt(secondNumber);
if(sparseArray.get(num1) == null) {
sparseArray.put(num1, new SparseArray<String>());
}
temporarySparseArray = sparseArray.get(num1);
for(int w=0;w<size;w++){
temporarySparseArray.put(w, array1[w]);
}
sparseArray.put(num1, temporarySparseArray);
temporarySparseArray = null;
}

You can write you object in the android internal or external file system, and you can read the object when you want to use it.

I feel this question deserves an answer, and I was sure someone would come up with one, but, hmm, not really so far; and if I were Tom Wong, it would currently look to me as if StackOverflow was a place where wannabe experts start fighting over questions rather than answering them. ;) So here we go.
Do something at startup, the result of which lasts as long as your App is in memory: The right place to do this is Application.onCreate(), for which you need to subclass Application. It is okay to store the result in the Application object as well in this case, where you can access it from any Activity by getting the Application context.
Do something that will take some time: In general, it's a good idea to use AsyncTask on Android, but it is specialized for doing things which relate to the User Interface but must not block it. Also, without some explicit tuning, its behaviour will vary amongst different Android versions. Currently, without further measures, if you did your computation via AsyncTask, other AsyncTasks would have to wait first. Also, what you want to do is not a classic task for using an AsyncTask, so use a one-time Thread which simply stores its result finally in a volatile reference.
The following code snippet should give you the idea.
final class MyApplication extends Application {
public volatile String[] expensiveToCompute;
#Override
public void onCreate() {
expensiveToCreate = null;
(new Thread() {
#Override
public void run() {
String[] result = computeMyData();
// it is important to assign the locally created data at once here
expensiveToCompute = result;
}
}).start();
}
}
In this way, expensiveToCompute will be null until the asynchronous computation has finished.
It is important to understand that this Thread will run until it has finished, and that it holds a reference to your MyApplication object. This means that even if the user leaves your app, your Thread will still continue if it has not finished until then.
This is in fact nice, because either Android decides to leave your Application object alive just because it can, in which case the user experience improves. Or Android kills the whole process with the Application object, your Thread and any held data, in which case it will start from scratch next time.
One more thing about, umm, suggestions to move expensive computations into AsyncTask etc: Android assigns a default background priority to such threads, and all background threads will (currently) have to live with 10% CPU time altogether no matter whether the Android system is idle otherwise. So many people who move intensive computations to background tasks notice that it will take ten times longer. Hence, you may want to read my suggestions regarding how to handle this here and/or here.

Exchanging messages between UI Thread and Background Networking Thread

I am currently working on an android app just for personal use. The app communicates with a server by TCP sockets. If the user makes an input it needs to be sent immediately. Also there can be messages from the server at any given time which need to be shown on the UI. For all the networking stuff I have a background thread in mind.
Since I need to pass messages From the UI to the networking thread at any user input and also messages from the networking thread to the UI at any given time my question follows: How can I pass the messages? I already read about the Handler class for 3hours and I couldn't figure out anything. How do handlers work? What would be a neat and smooth running implementation of that? I look more for a strategy to accomplish this goal, not necessarily implementation details.
Thank you so much in advance!

http://developer.android.com/reference/android/os/AsyncTask.html Use this I too had a lot of problems with handlers but the async task handles all that annoying work for you. Make sure you use onPrexecute() and onPostExecute() to update the UI. Hope this helps.

You can use AsyncTask in this case.
You can call it on the UI thread and in onPostExecute do what you want to do with the messages.
It is a much cleaner approach than doing all the dirty networking work there.

First, I'll say I'm learning this myself, but I've managed to get this exact setup working in a prototype. If anyone sees something that should be changed to work better in Android, by all means let me know.
Second, I'm using empty messages so that we have code that would actually work. I assume that creating message objects is straight forward enough.
Part 1 - UI to Network Thread
As far as I have seen, Handler is the answer you need, but it's not the whole picture. Deucalion's comment about Looper is where you need to look next.
http://developer.android.com/reference/android/os/Looper.html
class LooperThread extends Thread {
public Handler mHandler;
public void run() {
Looper.prepare();
mHandler = new Handler() {
public void handleMessage(Message msg) {
// process incoming messages here
}
};
Looper.loop();
}
}
This setup is counter intuitive at first, you need to create the Handler object between the Looper.prepare() and Looper.loop() calls, inside your run() method. I don't know the magic behind the scenes, haven't had a chance to get that far down the rabbit hole.
Important Note:
The sample shows an anonymous Handler() being created. I highly recommend subclassing Handler in your own class and then using an instance of that. It's just a matter of reusability, a lot of network code ends up being the same.
So, rename LooperThread to NetworkThread, or whatever you'd like it to be, and put your connection code in the run() method, but outside the Looper.prepare() / Looper.loop() block (again I don't know how the magic works, so I'm just assuming this is better to do than to have it inside).
// process incoming messages here
Tends to look like this:
switch(msg.what)
{
case 1:
break;
default:
break
}
The LooperThread example has public Handler mHandler; but I prefer to make the handler private and create a method to return it. I do this just to prevent the handler from being accessible until after the network connection is established. No connection, nothing to do with the messages.
So, somewhere in your activity code you create the NetworkThread object, run it, and then get the Handler from it.
NetworkThread network = new NetworkThread();
new Thread(network).run();
//Handler networkHandler = network.handler;
Handler networkHandler = network.getHandler();
Then you just send messages to that handler as needed.
networkHandler.sendEmptyMessage(1);
Part 2 - Network Thread to UI
There are a couple of different ways to handle this. You can create a handler inside the Activity class. All activities have a Looper already setup, so calling Looper.prepare and Looper.loop seem to cause problems.
In my code I create a handler in the onCreate() method, and then I pass it into my NetworkThread object. There may be better places to create this Handler, I don't know what will happen if your application is suspended and then started again later with onResume() or onRestart()
public NetworkThread(Handler mainHandler)
{
this.mainHandler = mainHandler;
}
Then anywhere else I need to send a message to the UI:
this.mainHandler.sendEmptyMessage(1);
I prefer this method because the Handler code, that does the actual work, either exists in the classes the actually respond to the message (using the anonymous Handler objects) or in Handler subclasses that may be in the same package.
You could also pass in the Activity or a View to the network thread and from that call runOnUIThread(Runnable), but that requires you pass in a Runnable. So the code that is meant to work on the UI is being written or referenced in the Networking classes.
Update
You could also use a Service for this. The method above works when the Activity and the Networking have the same life span. If we only need network connection while the Activity is active.
If you need network connectivity beyond the life of the Activity, in a music streaming app for example, then going with a Service would be a better choice. It didn't seem like this was specifically what you were looking for the question.