We Keep Coding

Deadlock occurring when using java.nio.file.Paths & jsfml loadFromFile

I've been trying to debug a problem I've had with loading a font from file (a .ttf file) with the java.nio.file.Paths import, using a combination of Paths.get() and loadFromFile(), but can't seem to find a solution.
Here's the problem code:
import java.io.IOException;
import java.nio.file.Paths;
public final Font FONT_UI_BAR = new Font();
public final Font FONT_FREESANS = new Font();
try {
System.out.println("We get here, before loading");
FONT_UI_BAR.loadFromFile(Paths.get("Game/resources/UI/Font.ttf"));
System.out.println("I've loaded the first font");
FONT_FREESANS.loadFromFile(Paths.get("Game/resources/fonts/freesans/freesans.ttf"));
} catch (IOException e2) {
System.out.println("[ERROR] Could not load font");
e.printStackTrace();
}
The program gets to the first print statement but never reaches the second.
I did a thread dump and found there seems to be a deadlock within the code itself that occurs:
"main#1" prio=5 tid=0x1 nid=NA waiting
java.lang.Thread.State: WAITING
at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:318)
at org.jsfml.internal.SFMLErrorCapture.start(Unknown Source:-1)
at org.jsfml.graphics.Font.loadFromFile(Unknown Source:-1)
at assets.FontCatalogue.<init>(FontCatalogue.java:32)
at assets.FontCatalogue.get(FontCatalogue.java:15)
at screens.HomeScreen.<init>(HomeScreen.java:51)
at controllers.Game.<init>(Game.java:74)
at Main.main(Main.java:16)
I'm not exactly sure how to proceed from here. My program won't function how I want it to without loading these fonts. I've tried loading other kinds of fonts and the problem persists.
Weirdly enough the problem didn't occur with loading other files in the past, such as this code:
TEMP_BG_01.loadFromFile(Paths.get("Game/resources/placeholder/full-moon_bg.png"));
It only started once I started trying to load these fonts.
Ideally I'd like to find a solution that still allows me to use this package because otherwise I have a fair amount of code to rewrite. Not the biggest deal but suggesting simply using another package should be a last resort.
Any ideas appreciated.
EDIT: Interesting to note this issue DOES NOT occur on a Windows machine, only my ubuntu-linux one. The rest of my team on Windows have no issues. Obviously one solution is to go and use Windows instead, but who wants to do that :p
EDIT #2: Turns out I'm now getting this error even with loading from the Texture class in JSFML. I have a feeling I updated my JVM when I updated my ubuntu sometime recently and that's suddenly introduced problems. I can't say for sure because I don't recall updating very recently, but it seems as of 21/02/2021 loading from file with JSFML causes a deadlock :/

The first thing you need to do if you want to continue using JSFML is to determine the initial failure that leaves you in a deadlock state.
The code in the SFMLErrorCapture class is not robust. Should SFMLErrorCapture.start() fail in any way, it will leave the semaphore locked. I suspect this is the initial failure that breaks your application and leaves it deadlocked.
I'd recommend adding logging to the class, such as:
public static void start() {
try {
semaphore.acquire();
capturing = true;
nativeStart();
} catch (InterruptedException ex) {
ex.printStackTrace();
} catch (Throwable t) {
t.printStackTrace();
// lots of other logging, probably to a file in /tmp
// rethrow so original program flow isn't changed
throw t;
}
}
You might also want to add more logging to see if you get any InterruptedExceptions. That's another way the semaphore will never get released, but I don't think a simple upgrade is likely to trigger that kind of behavior change.
And, since it's also possible for finish() to fail in the same manner (such as if nativeFinish() returns null, which I'd think is also a likely failure mode...):
public static String finish() {
try {
final String str;
if (capturing) {
str = nativeFinish().trim();
capturing = false;
semaphore.release();
} else {
str = null;
}
return str;
} catch (Throwable t) {
t.printStackTrace();
// lots of logging
throw t;
}
}
You might need to add throws Throwable to both methods.
This might also help:
public static String finish() {
try {
final String str;
if (capturing) {
// chaining calls is BAD CODE!!!!
// Say hello to NPE if you insist cramming
// multiple calls in one line!!
str = nativeFinish();
if ( str != null ) {
str = str.trim();
}
capturing = false;
semaphore.release();
} else {
str = null;
}
return str;
}
}
Limiting asynchronous actions like this to one at a time is fundamentally broken. If only one action can happen at once, the code complexity added to do actions asynchronously is worse than wasted because such complex code is much more bug-prone and when bugs do happen that complexity makes unrecoverable failures much more likely.
If you can only do one at a time, just do the actions serially with one static synchronized method or in one synchronized block on a static final object.

Capturing stdout from java.lang.reflect method invocation

I have an application that, among other things, runs Java methods via java.lang.reflect. It normally functions as normal; however, a user used it with one of their JARs, and it broke somewhat.
As you can see in the below code, I attempt to capture both stdout and stdin from the method. However, when the method is invoked, only the first line of what the method streams to stdout is actually captured.
Here's the relevant code. If you need to see more of the code, let me know, and I'll add some more:
String retVal = "";
ByteArrayOutputStream out = new ByteArrayOutputStream();
ByteArrayOutputStream err = new ByteArrayOutputStream();
PrintStream origOut = System.out;
PrintStream origErr = System.err;
System.setOut(new PrintStream(out));
System.setErr(new PrintStream(err));
Exception myException = null;
try {
Object myRetVal = null;
myRetVal = m.invoke(obj, convertedMethodArguments);
if (myRetVal != null)
retVal = myRetVal.toString();
} catch (Exception e) {
myException = e;
}
returnObj.addProperty("stdout", out.toString());
returnObj.addProperty("stderr", err.toString());
returnObj.addProperty("rv", retVal);
returnObj.addProperty("rt", m.getReturnType().toString());
if (myException != null && myException.getCause() != null)
returnObj.addProperty("exception", myException.getCause().toString());
else
returnObj.addProperty("exception", "");
System.setOut(origOut);
System.setErr(origErr);
System.out.print(new Gson().toJson(returnObj));
// TODO: remove, debug purposes only
// Should use normal stdout
try {
System.out.println();
m.invoke(obj, convertedMethodArguments);
} catch (Exception e) {
System.out.println(e.toString());
}
When I execute the above code, it only prints out the first line of stdout. However, at the bottom of the code block, I invoke the method again, but this time without any redirection, and I get all of the stdout from the method.
Any help would be greatly appreciated!
EDIT #1: OK, get this. For fun, I commented-out the two lines where I redirect the default System streams (e.g. System.setOut and System.setErr). With these gone, I now expect all stdout to be written to the console directly when I run the app.
I added a message (e.g. System.out.println("Testing...");) at the very end of my code, so that it's the last thing that is executed. When I test the app, I get the first line of stdout, followed by my testing message, and THEN the rest of the stdout.
I have no clue what's going on here.
EDIT #2: Per #Titus's suggestion, I looked into whether or not the method I'm invoking is spinning off its own threads. Turns out, it is. Two threads are created, AWT-AppKit and AWT-Shutdown. The former thread seems to stay in RUNNABLE state, whereas the latter thread stays in the TIMED_WAITING state.
Over time, the AWT-Shutdown thread goes away, but the other one stays alive in its RUNNABLE state. Once my application exits, I believe the method I'm invoking also exits, and at that point the extra messages are displayed to the screen (which explains why I can't capture this bit of STDOUT).
What I don't understand is why this method won't terminate within my application.

Try to flush the streams after you call the method.
Here is an example:
PrintStream outPR = new PrintStream(out);
System.setOut(outPR);
....
outPR.flush();
returnObj.addProperty("stdout", out.toString());
You can even do this:
System.setOut(new PrintStream(out, true));
....
System.out.println();
returnObj.addProperty("stdout", out.toString());
The PrintStream is automatically flushed (if you use the constructor that I've used) when a \n (new line) is written to it.
Based on the edits to your question, it is possible that the method you're calling is creating new threads which means that it is possible that this new threads print to the console after the method returns.
If that is the case, you'll have to wait until this threads finish in order to get all the output.

Java thread stuck after join

I have this Transmitter class, which contains one BufferedReader and one PrintWriter. The idea is, on the main class, to use Transmitter.receive() and Transmitter.transmit() to the main socket. The problem is:
public void receive() throws Exception {
// Reads from the socket
Thread listener = new Thread(new Runnable() {
public void run() {
String res;
try {
while((res = input.readLine()) != null) {
System.out.println("message received: " + res);
outputMessage = (res);
if (res.equals("\n")) {
break;
}
}
} catch (IOException e) {
e.printStackTrace();
}
};
});
listener.start();
listener.join();
}
The thread changes the 'outputMessage' value, which I can get using an auxiliary method. The problem is that, without join, my client gets the outputMessage but I want to use it several times on my main class, like this:
trans1.receive();
while(trans1.getOutput() == null);
System.out.println("message: " + trans1.getOutput());
But with join this system.out never executes because trans1.receive() is stuck... any thoughts?
Edit 1: here is the transmitter class https://titanpad.com/puYBvlVery

You might send \n; that doesn't mean that you will see it in your Java code.
As it says in the Javadoc for BufferedReader.readLine() (emphasis mine):
(Returns) A String containing the contents of the line, not including any line-termination characters
so "\n" will never be returned.

Doing this:
{
Thread listener = new Thread(new Runnable() {
public void run() {
doSomeWork();
};
});
listener.start();
listener.join();
}
will create a new thread and then wait for it to do its work and finish. Therefore it's more or less the same as just directly doing:
doSomeWork();
The new thread doesn't serve any real purpose here.
Also, the extra thread introduces synchronization problems because in your code you don't make sure your variables are synchronized.
Thirdly, your thread keeps reading lines from the input in a loop until there's nothing more to read and unless the other side closes the stream, it will block on the readLine() call. What you will see in with getOutput() will be a random line that just happens to be there at the moment you look, the next time you look it might be the same line, or some completely different line; some lines will be read and forgotten immediatelly without you ever noticing it from the main thread.
You can just call input.readLine() directly in your main thread when you actually need to get a new line message from the input, you don't need an extra reader thread. You could store the read messages into a Queue as yshavit suggests, if that's desirable, e.g. for performance reasons it might be better to read the messages as soon as they are available and have them ready in memory. But if you only need to read messages one by one then you can simply call input.readLine() only when you actually need it.

Need advice on synchronization of Java Vector / ConcurrentModificationException

In a legacy application I have a Vector that keeps a chronological list of files to process and multiple threads ask it for the next file to process. (Note that I realize that there are likely better collections to use (feel free to suggest), but I don't have time for a change of that magnitude right now.)
At a scheduled interval, another thread checks the working directory to see if any files appear to have been orphaned because something went wrong. The method called by this thread occasionally throws a ConcurrentModificationException if the system is abnormally busy. So I know that at least two threads are trying to use the Vector at once.
Here is the code. I believe the issue is the use of the clone() on the returned Vector.
private synchronized boolean isFileInDataStore( File fileToCheck ){
boolean inFile = false;
for( File wf : (Vector<File>)m_dataStore.getFileList().clone() ){
File zipName = new File( Tools.replaceFileExtension(fileToCheck.getAbsolutePath(), ZIP_EXTENSION) );
if(wf.getAbsolutePath().equals(zipName.getAbsolutePath()) ){
inFile = true;
break;
}
}
return inFile;
}
The getFileList() method is as follows:
public synchronized Vector<File> getFileList() {
synchronized(fileList){
return fileList;
}
}
As a quick fix, would changing the getFileList method to return a copy of the vector as follows suffice?
public synchronized Vector<File> getFileListCopy() {
synchronized(fileList){
return (Vector<File>)fileList.clone();
}
}
I must admit that I am generally confused by the use of synchronized in Java as it pertains to collections, as simply declaring the method as such is not enough. As a bonus question, is declaring the method as synchronized and wrapping the return call with another synchronized block just crazy coding? Looks redundant.
EDIT: Here are the other methods which touch the list.
public synchronized boolean addFile(File aFile) {
boolean added = false;
synchronized(fileList){
if( !fileList.contains(aFile) ){
added = fileList.add(aFile);
}
}
notifyAll();
return added;
}
public synchronized void removeFile( File dirToImport, File aFile ) {
if(aFile!=null){
synchronized(fileList){
fileList.remove(aFile);
}
// Create a dummy list so I can synchronize it.
List<File> zipFiles = new ArrayList<File>();
synchronized(zipFiles){
// Populate with actual list
zipFiles = (List<File>)diodeTable.get(dirToImport);
if(zipFiles!=null){
zipFiles.remove(aFile);
// Repopulate list if the number falls below the number of importer threads.
if( zipFiles.size()<importerThreadCount ){
diodeTable.put(dirToImport, getFileList( dirToImport ));
}
}
}
notifyAll();
}
}

Basically, there are two separate issues here: sycnhronization and ConcurrentModificationException. Vector in contrast to e.g. ArrayList is synchronized internally so basic operation like add() or get() do not need synchronization. But you can get ConcurrentModificationException even from a single thread if you are iterating over a Vector and modify it in the meantime, e.g. by inserting an element. So, if you performed a modifying operation inside your for loop, you could break the Vector even with a single thread. Now, if you return your Vector outside of your class, you don't prevent anyone from modifyuing it without proper synchronization in their code. Synchronization on fileList in the original version of getFileList() is pointless. Returning a copy instead of original could help, as could using a collection which allows modification while iterating, like CopyOnWriteArrayList (but do note the additional cost of modifications, it may be a showstopper in some cases).

"I am generally confused by the use of synchronized in Java as it
pertains to collections, as simply declaring the method as such is not
enough"
Correct. synchronized on a method means that only one thread at a time may enter the method. But if the same collection is visible from multiple methods, then this doesn't help much.
To prevent two threads accessing the same collection at the same time, they need to synchronize on the same object - e.g. the collection itself. You have done this in some of your methods, but isFileInDataStore appears to access a collection returned by getFileList without synchronizing on it.
Note that obtaining the collection in a synchronized manner, as you have done in getFileList, isn't enough - it's the accessing that needs synchronizing. Cloning the collection would (probably) fix the issue if you only need read-access.
As well as looking at synchronizing, I suggest you track down which threads are involved - e.g. print out the call stack of the exception and/or use a debugger. It's better to really understand what's going on than to just synchronize and clone until the errors go away!

Where does the m_dataStore get updated? That's a likely culprit if it's not synchronized.

First, you should move your logic to whatever class is m_dataStore if you haven't.
Once you've done that, make your list final, and synchronize on it ONLY if you are modifying its elements. Threads that only need to read it, don't need synchronized access. They may end up polling an outdated list, but I suppose that is not a problem. This gets you increased performance.
As far as I can tell, you would only need to synchronize when adding and removing, and only need to lock your list.
e.g.
package answer;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Example {
public static void main(String[] args)
{
Example c = new Example();
c.runit();
}
public void runit()
{
Thread.currentThread().setName("Thread-1");
new Thread("Thread-2")
{
#Override
public void run() {
test1(true);
}
}.start();
// Force a scenario where Thread-1 allows Thread-2 to acquire the lock
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
Logger.getLogger(Example.class.getName()).log(Level.SEVERE, null, ex);
}
// At this point, Thread-2 has acquired the lock, but it has entered its wait() method, releasing the lock
test1(false);
}
public synchronized void test1(boolean wait)
{
System.out.println( Thread.currentThread().getName() + " : Starting...");
try {
if (wait)
{
// Apparently the current thread is supposed to wait for some other thread to do something...
wait();
} else {
// The current thread is supposed to keep running with the lock
doSomeWorkThatRequiresALockLikeRemoveOrAdd();
System.out.println( Thread.currentThread().getName() + " : Our work is done. About to wake up the other thread(s) in 2s...");
Thread.sleep(2000);
// Tell Thread-2 that it we have done our work and that they don't have to spare the CPU anymore.
// This essentially tells it "hey don't wait anymore, start checking if you can get the lock"
// Try commenting this line and you will see that Thread-2 never wakes up...
notifyAll();
// This should show you that Thread-1 will still have the lock at this point (even after calling notifyAll).
//Thread-2 will not print "after wait/notify" for as long as Thread-1 is running this method. The lock is still owned by Thread-1.
Thread.sleep(1000);
}
System.out.println( Thread.currentThread().getName() + " : after wait/notify");
} catch (InterruptedException ex) {
Logger.getLogger(Example.class.getName()).log(Level.SEVERE, null, ex);
}
}
private void doSomeWorkThatRequiresALockLikeRemoveOrAdd()
{
// Do some work that requires a lock like remove or add
}
}

Java MULTITHREADING - when multiple threads accesses the print method - why the execution of while method is by default synched

Have a doubt in multithreading.
Following is my main program to access a file, am creating 10 threads to be accessed on the object.
public class CallTest {
public static void main(String[] args) throws Exception {
Test t = new Test();
for (int i = 0; i < 10; i++) {
Thread t1 = new Thread(t);
t1.start();
}
}
}
Following is my program to read data from file.
public class Test implements Runnable {
static int i;
public void run() {
try {
i++;
System.out.println("####Count" + i);
print();
} catch (Exception e) {}
}
public void print() {
try {
StringBuilder bufData = new StringBuilder();
File fileTest = new File("D:\\Work\\i466477");
BufferedReader bufferedReader1 = new BufferedReader(new FileReader(
fileTest));
String strRecord = new String();
while ((strRecord = bufferedReader1.readLine()) != null) {
bufData.append(strRecord);
bufData.append("\r");
bufData.append("\n");
}
bufferedReader1.close();
System.out.println("########");
System.out.println(bufData);
} catch (Exception exe) {
System.out.println(exe);
}
}
}
Here I could see the code in the while is by default synchronized, is BufferedReader thread safe or because each thread will have their own copy of StringBuilder and BufferedReader? I could see the contents are read and written properly.

No, that code won't be synchronized by default. Several threads could each be in the while loop at the same time. "Synchronized" isn't the same as "working without any problems" - did you think it was synchronized just because you didn't have any issues? In Java, synchronized is about only allowing one thread to execute certain critical pieces of code at a time in relation to a particular monitor.
Note that your access to i in the run method is unsafe, by the way. You should also close the BufferedReader in a finally block, and avoid catching Exception. Finally, your assignment of new String() to strRecord to start with is pointless. Hopefully these are just errors due to it being test code, but it's worth being aware of them.

Actually, System.out.println is synchronized. Try this again without those.

Each thread has its own StringBuilder, BufferedReader and FileReader (and operating system level file descriptor) so there won't be any interference at that level. (None of these classes is thread-safe, but the instances are thread-confined so that doesn't matter.)
When you are writing, the PrintWriter.print(...) and PrintWriter.println(...) methods are synchronized, and that explains why you don't see output from individual println calls mixed together. (PrintWriter is thread-safe ... and needs to be.)
Note: if you changed your code to include the thread number in each println'ed string, you might occasionally see the output appearing in an unexpected order. Separate calls to a thread-safe method on the same object (the PrintWriter) don't necessarily occur in "first come, first served" order.
The code that updates the static variable i is not thread-safe, and might give you unexpected (incorrect) results every now and then ... depending on what hardware / JVM you use. You should either do the update in a synchronized static method, or replace i with an AtomicInteger.

Local variables are thread confined. But the non atomic operations(like i++) on static variable i is not thread safe.

bufferreader and stringbuilder are not shared between threads, so their use is thread safe.
StringBuffer is thread safe to a degree, in that all its methods are synchronized. BufferedReader is not thread safe.