We Keep Coding

Deadlock occurring when using java.nio.file.Paths & jsfml loadFromFile

I've been trying to debug a problem I've had with loading a font from file (a .ttf file) with the java.nio.file.Paths import, using a combination of Paths.get() and loadFromFile(), but can't seem to find a solution.
Here's the problem code:
import java.io.IOException;
import java.nio.file.Paths;
public final Font FONT_UI_BAR = new Font();
public final Font FONT_FREESANS = new Font();
try {
System.out.println("We get here, before loading");
FONT_UI_BAR.loadFromFile(Paths.get("Game/resources/UI/Font.ttf"));
System.out.println("I've loaded the first font");
FONT_FREESANS.loadFromFile(Paths.get("Game/resources/fonts/freesans/freesans.ttf"));
} catch (IOException e2) {
System.out.println("[ERROR] Could not load font");
e.printStackTrace();
}
The program gets to the first print statement but never reaches the second.
I did a thread dump and found there seems to be a deadlock within the code itself that occurs:
"main#1" prio=5 tid=0x1 nid=NA waiting
java.lang.Thread.State: WAITING
at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:318)
at org.jsfml.internal.SFMLErrorCapture.start(Unknown Source:-1)
at org.jsfml.graphics.Font.loadFromFile(Unknown Source:-1)
at assets.FontCatalogue.<init>(FontCatalogue.java:32)
at assets.FontCatalogue.get(FontCatalogue.java:15)
at screens.HomeScreen.<init>(HomeScreen.java:51)
at controllers.Game.<init>(Game.java:74)
at Main.main(Main.java:16)
I'm not exactly sure how to proceed from here. My program won't function how I want it to without loading these fonts. I've tried loading other kinds of fonts and the problem persists.
Weirdly enough the problem didn't occur with loading other files in the past, such as this code:
TEMP_BG_01.loadFromFile(Paths.get("Game/resources/placeholder/full-moon_bg.png"));
It only started once I started trying to load these fonts.
Ideally I'd like to find a solution that still allows me to use this package because otherwise I have a fair amount of code to rewrite. Not the biggest deal but suggesting simply using another package should be a last resort.
Any ideas appreciated.
EDIT: Interesting to note this issue DOES NOT occur on a Windows machine, only my ubuntu-linux one. The rest of my team on Windows have no issues. Obviously one solution is to go and use Windows instead, but who wants to do that :p
EDIT #2: Turns out I'm now getting this error even with loading from the Texture class in JSFML. I have a feeling I updated my JVM when I updated my ubuntu sometime recently and that's suddenly introduced problems. I can't say for sure because I don't recall updating very recently, but it seems as of 21/02/2021 loading from file with JSFML causes a deadlock :/

The first thing you need to do if you want to continue using JSFML is to determine the initial failure that leaves you in a deadlock state.
The code in the SFMLErrorCapture class is not robust. Should SFMLErrorCapture.start() fail in any way, it will leave the semaphore locked. I suspect this is the initial failure that breaks your application and leaves it deadlocked.
I'd recommend adding logging to the class, such as:
public static void start() {
try {
semaphore.acquire();
capturing = true;
nativeStart();
} catch (InterruptedException ex) {
ex.printStackTrace();
} catch (Throwable t) {
t.printStackTrace();
// lots of other logging, probably to a file in /tmp
// rethrow so original program flow isn't changed
throw t;
}
}
You might also want to add more logging to see if you get any InterruptedExceptions. That's another way the semaphore will never get released, but I don't think a simple upgrade is likely to trigger that kind of behavior change.
And, since it's also possible for finish() to fail in the same manner (such as if nativeFinish() returns null, which I'd think is also a likely failure mode...):
public static String finish() {
try {
final String str;
if (capturing) {
str = nativeFinish().trim();
capturing = false;
semaphore.release();
} else {
str = null;
}
return str;
} catch (Throwable t) {
t.printStackTrace();
// lots of logging
throw t;
}
}
You might need to add throws Throwable to both methods.
This might also help:
public static String finish() {
try {
final String str;
if (capturing) {
// chaining calls is BAD CODE!!!!
// Say hello to NPE if you insist cramming
// multiple calls in one line!!
str = nativeFinish();
if ( str != null ) {
str = str.trim();
}
capturing = false;
semaphore.release();
} else {
str = null;
}
return str;
}
}
Limiting asynchronous actions like this to one at a time is fundamentally broken. If only one action can happen at once, the code complexity added to do actions asynchronously is worse than wasted because such complex code is much more bug-prone and when bugs do happen that complexity makes unrecoverable failures much more likely.
If you can only do one at a time, just do the actions serially with one static synchronized method or in one synchronized block on a static final object.

How to solve JNA error "Only part of a ReadProcessMemory or WriteProcessMemory request was made"? [duplicate]

I have been researching this issue pretty extensively and cannot seem to find an answer.
I know that the Only part of a ReadProcessMemory or WriteProcessMemory request was completed exception is thrown when a 32-bit process tries to access a 64-bit process and the same for a 64-bit modifying a 32-bit process.
The solution to that issue is to change the Platform Target to 'Any CPU'. I have tried this and unfortunately this does not solve my issue.
The next block of code is what keeps throwing the exception. The program that runs this code is used to open up applications on remote computers and keeps a list of all the processes that the program itself opened so that I don't have to loop through all the processes.
Process processToRemove = null;
lock (_runningProcesses)
{
foreach (Process p in _runningProcesses)
{
foreach (ProcessModule module in p.Modules)
{
string[] strs = text.Split('\\');
if (module.ModuleName.Equals(strs[strs.Length - 1]))
{
processToRemove = p;
break;
}
}
if (processToRemove != null)
{
break;
}
}
if (processToRemove != null)
{
processToRemove.Kill();
_runningProcesses.Remove(processToRemove);
}
}
These processes can and most likely will be 32-bit and 64-bit, mixed together.
Is there anything I am doing that I shouldn't be doing, or is there just a better way to do all of this?

As detailed in the comments of the MSDN page for Process.Modules and this thread there is a known issue in Process.Modules when enumerating 32 bit processes from a 64 bit process and visa-versa:
Internally .NET's Process.Modules is using function EnumProcessModules
from PSAPI.dll. This function has a known issue that it cannot work
across 32/64 bit process boundary. Therefore enumerating another
64-bit process from 32-bit process or vice versa doesn't work
correctly.
The solution seems to be to use the EnumProcessModulesEx function, (which must be called via P/Invoke), however this function is only available on later versions of Windows.
We fixed this issue by adding
a new function called EnumProcessModulesEx to PSAPI.dll
(http://msdn2.microsoft.com/en-us/library/ms682633.aspx), but we
currently cannot use it in this case:
it only works on Windows Vista or Windows Server 2008
currently .NET 2.0 Framework don't have a service pack or hotfix to make Process.Modules use this new API

There are only some issues regarding the handling of the processes and the locking that I would change:
object lockObject = new object();
List<Process> processesToRemove = new List<Process>();
foreach (Process p in _runningProcesses)
{
foreach (ProcessModule module in p.Modules)
{
string[] strs = text.Split('\\');
if (module.ModuleName.Equals(strs[strs.Length - 1]))
{
processesToRemove.Add(p);
break;
}
}
}
lock (lockObject)
{
foreach (Process p in processesToRemove)
{
p.Kill();
_runningProcesses.Remove(p);
}
}
I'm not answering for the bounty, just wanted to give some ideas. This code isn't tested because I don't exactly know what you are trying to do there.
Just consider not to lock the process-list and to keep the lock as short as possible.

I agree with #sprinter252 that _runningProcesses should not be used as your sync object here.
//Somewhere that is accessible to both the thread getting the process list and the thread the
//code below will be running, declare your sync, lock while adjusting _runningProcesses
public static readonly object Sync = new object();
IList<Process> runningProcesses;
lock(Sync)
{
runningProcesses = _runningProcesses.ToList();
}
Process processToRemove = null;
foreach (Process p in _runningProcesses)
{
foreach (ProcessModule module in p.Modules)
{
string[] strs = text.Split('\\');
if (module.ModuleName.Equals(strs[strs.Length - 1]))
{
processToRemove = p;
break;
}
}
if (processToRemove != null)
{
break;
}
}
if (processToRemove != null)
{
//If we've got a process that needs killing, re-lock on Sync so that we may
//safely modify the shared collection
lock(Sync)
{
processToRemove.Kill();
_runningProcesses.Remove(processToRemove);
}
}
If this code is wrapped in a loop to continue to check _runningProcesses for the process you wish to kill, consider changing processToRemove to processesToRemove and change it's type to a collection, iterate over that list in the bottom block after a check for a non-zero count and lock outside of that loop to decrease the overhead of obtaining and releasing locks per process to kill.

How do I use one Java program, to monitor another Java program's output?

Below is diagram that shows what I'm trying to do : it is just 2 programs. One is a simple Child program that writes out integers every 2 seconds, line-by-line .
The other is a Parent program that monitors the log file ( just a very basic text file). If the log file doesn't get modified within 5 seconds, then it should restart the Child program (via a batch file ); then continue normally.
My code for the child class is here:
package fileiotestapplication;
import java.io.*;
import java.io.IOException;
import java.util.*;
public class WriterClass {
#SuppressWarnings("oracle.jdeveloper.java.insufficient-catch-block")
public WriterClass() {
super();
int[] content = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,};
String[] friends = {"bob",};
File file = new File("/C:/Java_Scratch/someFile.txt");
// if file does not exists, then create it
try {
if (!file.exists()) {
file.createNewFile();
}
for (int i = 0 ; i < content.length; i++)
{
PrintStream bw = new PrintStream( new FileOutputStream(file, true) );
System.out.println("testing " + i);
bw.println( String.valueOf(content[i]) );
bw.close();
Thread.sleep(2500);
}
System.out.println("Done");
} catch (IOException ioe) {
// TODO: Add catch code
ioe.printStackTrace();
}
catch (InterruptedException ioe) {
// TODO: Add catch code
ioe.printStackTrace();
}
//someIS.println(i);
System.out.println("This is OK");
}
public static void main(String[] args) {
WriterClass writerClass = new WriterClass();
}
}
The source code
And I linked here my current code for the Parent class.
What I'm now trying to do is add in some logic that catches when the child class stops writing output. What I'd like to do is count all the lines in the log file; and then compare them every 5 seconds, is this a good way (the alternative would be - to keep checking to see if the file got modified at all)?
EDIT: The suggestion below to use waitFor() indeed helps, though I'm still working out details : it is generally like :
try {
/* StackOverflow code */
for ( ; ; ) {
ProcessBuilder pb = new ProcessBuilder("TheBatchFile.bat");
pb.directory(new File("C://Java_Scratch_//Autonomic_Using_Batch//"));
Process p = pb.start();
p.waitFor();
}
/* end - StackOverflow code */
}
catch (IOException i) {
i.printStackTrace();
}
catch (InterruptedException i) {
i.printStackTrace();
}

This will get very slow as the file keeps growing in size. A simpler way would be to simply check the last modification time of the file. Assuming that the reason the child program might stop writing to the file is that the program terminates (rather than e.g. hanging in an infinite loop), it is probably better to directly monitor the child process itself rather than relying on observing the effects of the process. This is particularly convenient if the parent process can be responsible for starting the program in the first place.
This can be done with the ProcessBuilder and Process classes in Java 8. Copying from the documentation, you can start the process like this (if you only want to monitor whether it's running or not):
ProcessBuilder pb = new ProcessBuilder("TheBatchFile.bat", "Argument1", "Argument2");
pb.directory(new File("/path/to/working/dir"));
Process p = pb.start();
Then, you can simply call p.waitFor(); to wait for the process to terminate. Do this in a loop, and you have your automatic-restarting-of-child behavior.

You can use the directory watch service:
https://docs.oracle.com/javase/tutorial/essential/io/notification.html
You can configure a path or a file and register a watcher.
The watcher gets a notification every time a file is changed. You can store this timestamp of a notification for later use.
For details see my link above.
You may then use a Timer or a Thread to check last modification.

While your method of creating a text file, and using a batch script is feasible, there is a better way to approach it. This is a standard problem to approach with multitasking, and by creating a couple threads, it is not too difficult at all.
Using threads has several advantages over going externally "around" the system with batch files and multiple programs. For starters, these may include:
Keeping everything together makes the project much tidier, cleaner,
and marginally easier to distribute.
It is easier to implement. Sure threads may seem confusing if you have never used them, but they are the lesser evil in my opinion, then all the steps involved in going around them. As I hope to show below, implementing this problem with threads is not hard.
Improved performance, as the very expensive operations of file IO, and spawning the batch file are avoided. Threads also have improved performance over processes in most cases because they are easier to spawn, and multithreading sees performance improvements on a wider range of processors than multiprocessing by being less reliant on having several cores.
No sketchy overlap between when one program is reading the file, while the other is writing to it simultaneously. These kind of situations are best avoided when possible.
Maintains Java's impressive cross platform abilities, because you are not using batch which is not cross platform. This might not be important to you for this project, but you may come across something in the future with a similar problem, where this is more important, and so you will have practice implementing it.
You learn better by using threads the "right way" instead of
developing bad habits by using a more hacky approach. If this is a
learning project, you might as well learn it right.
I went ahead and coded up the approach that I would most likely use to solve the problem. My code has a child thread the counts every two seconds, and a parent thread that monitors the child, and restarts it if the child goes five seconds without counting. Let's examine my program to give you a good idea of how it is working.
First, here is the class for the parent:
public class Parent {
private Child child;
public Parent(){
child = new Child(this);
child.start();
}
public void report(int count){ //Starts a new watchdog timer
Watchdog restartTimer = new Watchdog(this, count);
restartTimer.start();
}
public void restartChild(int currentCount){
if (currentCount == child.getCount()){ //Check if the count has not changed
//If it hasn't
child.kill();
child.start();
}
}
public static void main(String[] args){
//Start up the parent function, it spawns the child
new Parent();
}
}
The main function in there can be put somewhere else if you want, but to start everything up, just instantiate a parent. The parent class has an instance of the child class, and it starts up the child thread. The child will report it's counting to the parent with the report method, which spawns a watchdog timer (more on that in a second) that will call restartChild after five seconds with the current count. RestartChild, restarts the child thread, if the count is still the same as the one provided.
Here is the class for the watchdog timer:
class Watchdog implements Runnable { //A timer that will run after five seconds
private Thread t;
private Parent parent;
private int initialCount;
public Watchdog(Parent parent, int count){ //make a timer with a count, and access to the parent
initialCount = count;
this.parent = parent;
}
public void run() { //Timers logic
try {
Thread.sleep(5000); // If you want to change the time requirement, modify it here
parent.restartChild(initialCount);
} catch (InterruptedException e) {
System.out.println("Error in watchdog thread");
}
}
public void start () // start the timer
{
if (t == null)
{
t = new Thread (this);
t.start ();
}
}
}
This watchdog timer is a thread that the parent will run with the start method. The parent sends itself as a parameter so that we can call the restartChild function of the parent.It stores the count, because when it runs after five seconds, restartChild will check if the count has changed.
And finally, here is the child class
public class Child implements Runnable{
private Thread t;
public int counter = 0;
private boolean running;
private Parent parent; // Record the parent function
public Child(Parent parent){
this.parent = parent;
}
private void initializeAll(){
counter = 0;
running = true;
}
public int getCount(){
return counter;
}
#Override
public void run() {
while((counter <= 100)&&(running)){
//The main logic for child
counter +=1;
System.out.println(counter);
parent.report(counter); // Report a new count every two seconds
try {
Thread.sleep(2000); // Wait two seconds
} catch (InterruptedException e) {
System.out.println("Thread Failed");
}
}
}
public void start(){ //Start the thread
initializeAll();
t = new Thread(this);
t.start();
}
public void kill(){ //Kill the thread
running = false;
}
}
This is also a thread, thus it implements runnable, and in that regard acts a lot like the watchdog. Run() is the main method of the child thread, this is where your logic goes that gets called when you start it. Starting the child with start() sets all the variables to their defaults, and then begins the run() logic. The logic in run is wrapped in if(running), because that lets us kill the thread internally by setting running to false.
Currently, all the child does right now is increment it's counter, output it to console, and then report the activity to the parent, 100 times, every two seconds. You will likely want to remove the condition stopping it after count passes 100, but I included it, so that the parent would eventual have cause to restart the child. To change the behavior, look at the child's run method, that is where all the main action is at.

Profiling single Web session

While searching for a performance-problem in a Java-web-app, I would like to profile (via sampling) a single web session. The web-app is deployed on a Java EE Application-Server (either JBoss or WLS, so no OpenJDK-specific stuff possible)
Using a traditional sampler like JVisualVM or YourKit will only allow me to profile all running threads at once and thus will not reveal the CPU-Usage of said web-session separated from the base-load, which is already on the server. Part of the problem is, that there is (obviously) no technical connection between web-session and a single thread, i.e. every request might get a different thread out of the threadpool of the application-server.
My idea is to implement a sampler and register the thread associated with the observed web-session every time a new request is received from the server (deregistering the thread, as soon as the request is finished).
First question: Do you have to do this by hand or is there a tool already available to do it for you?
Second question (as I've found none and assume that this is a rather specific problem): What's the best approach?
Obviously one would try to minimize impact of the profiling to the application-performance. Also obviously, as I need to connect the (technical) thread info to application-specific data (websession-ID), this appears to be nothing, which can be done via JVMTI.
Leaves the option of coding a in-app-profiler, where a thread is doing the sampling via Thread.stackTrace or ThreadMXBean.getThreadInfo. Which is better? Is there another, better option?

JProfiler can split the call tree for different URLs or query parameters.
If you can modify the URL or the query parameters for that purpose, then you will be able to separate the session from the background load. For example, you might add a query parameter profile=true that you only pass for requests that should be profiled.
Disclaimer: My company develops JProfiler.

Have a look at:
http://messadmin.sourceforge.net/
This should help with what you want to profile.
Also, http://www.appdynamics.com/ and http://newrelic.com/ are very nice tools for profiling a web app.

If you don't have JProfiler, "coding a in-app-profiler" is certainly an option. Simply printing stack traces at specific intervals during a single method call already gives a nice snapshot view into what's going on.
Here's an example helper class that does that:
import java.io.Closeable;
import java.io.IOException;
import java.lang.management.ManagementFactory;
public class StackDumper extends Thread implements Closeable {
static final String pid = ManagementFactory.getRuntimeMXBean().getName().replaceFirst("#.*", "");
final Thread t;
final int step;
public StackDumper(int timeStepMs) {
setName("Profiler");
this.t = Thread.currentThread();
this.step = timeStepMs;
start();
}
#Override
public void run() {
StringBuilder buf = new StringBuilder(4096);
for (int i = 0;; i += step) {
try {
Thread.sleep(step);
} catch (InterruptedException e) {
break;
}
buf.setLength(0);
for (StackTraceElement e : t.getStackTrace()) {
buf.append(" ").append(e).append('\n');
}
System.out.print(pid + " #" + i + "ms: [" + t.getName() + "]:\n" + buf);
}
}
#Override
public void close() throws IOException {
interrupt();
}
}
Use it as:
try (StackDumper p = new StackDumper(100)) { // prints stack every 100 ms ...
methodUnderTest();
}

Perl launched from Java takes forever

I know this is an absolute shot in the dark, but we're absolutely perplexed.
A perl (5.8.6) script run by Java (1.5) is taking more than an hour to complete. The same script, when run manually from the command line takes 12 minutes to complete. This is on a Linux host.
Logging is the same in both cases and the script is run with the same parameters in both cases.
The script does some complex stuff like Oracle DB access, some scp's, etc, but again, it does the exact same actions in both cases.
We're stumped. Has anyone ever run into a similar situation? If not and if you were faced with the same situation, how would you consider debugging it?

Sub-proceses which produce console output can block (and deadlock) if their stdout/stderror streams are not flushed. #gustafc, the code posed will eventually block the sub-process when it tries to write to stdout/stderror, and there is no room in the stream (and the stream is not being serviced by java).
Process p = startProcess();
final InputStream stdout = p.getInputStream();
final InputStream sterr = p.getErrorStream();
new Thread() {
public void run() {
int c;
while ((c = sterr.read()) != -1) {
System.out.print((char)c);
}
}
}.start();
new Thread() {
public void run() {
int c;
while ((c = sterr.read()) != -1) {
System.out.print((char)c);
}
}
}.start();

I assume you've discarded the possibility that the Java wrapper happens to run simultaneously as something else which causes huge contention over some scarce resource? Good.
If you have a simple class like this:
public class Exec {
public static void main(String[] args) throws Throwable{
class Transfer implements Runnable {
private final InputStream in;
private final OutputStream out;
public Transfer(InputStream i, OutputStream o){
in = i;
out = o;
}
public void run(){
try {
for (int i; (i = in.read()) != -1;) out.write(i);
out.close();
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Process proc = new ProcessBuilder(args).start();
new Thread(new Transfer(System.in, proc.getOutputStream())).start();
new Thread(new Transfer(proc.getInputStream(), System.out)).start();
new Thread(new Transfer(proc.getErrorStream(), System.err)).start();
System.exit(proc.waitFor());
}
}
... and you compare time perl script.pl insert args here and time java Exec perl script.pl insert args here, what happens? If the world is sane, they take about the same time (except that the second one needs a few seconds extra for Java to start), and if that's the case, gradually start adapting the Exec class to look more and more like your deployment environment, and see when it starts taking a really long time.
If Exec above really does take longer time, start logging like crazy in the Perl script, so you see which actions take longer time. And btw, log in the Java wrapper, too, so you see if the Perl startup takes a really long time or something.

One possibility is that you are making the system thrash by trying to run a large Java app and a large Perl app on a system that doesn't have enough memory.
It would be a good idea to use monitoring utilities like top vmstat -5 iostat -5 etc to try and figure out if the slowness corresponds to some OS-level pathology.

To bring this thread to a close, the eventual cause was rogue processes consuming too much CPU. When launched from the command-line, the script had normal priority. When launched from Java, the script had low priority and thus took forever to execute. What threw us off was that the Java code was not just executing the script, it was issuing the same commands via SSH that we issued interactively. Thus, we didn't expect the difference in priority.