We Keep Coding

Application stops responding when using Process.waitFor()

I'm launching an external application from within my Java application (8u11). However the application becomes non-responsive to UI input up under Windows XP and Windows 7 with the standard hour glass/spinner.
I've narrowed this problem down to whether or not I use Process.waitFor(). If I call it I see the problems, if I don't it works fine. Also the application un-freezes if I then quit the Java application.
My question is why is this the case - how can calling waitFor() possibly effect the internal runnings of a child process? And how can I avoid this problem?
The application in question is LinPhone.exe but I don't believe the issue is specific to the application - there must be some general way in which it handles standard IO etc which the way I'm interfering with by calling waitFor().
I need to use Process.waitFor() so I can track when the application has exited.
I've simplified the issue to this SCCEE.
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
public class LinphoneTest {
public static void main(String[] args) throws IOException,
InterruptedException {
String phoneAppPath = "C:\\Program Files\\Linphone\\bin\\linphone.exe";
ProcessBuilder processBuilder = new ProcessBuilder(phoneAppPath);
// move up from bin/linephone.exe
File workingDir = new File(phoneAppPath).getParentFile()
.getParentFile();
processBuilder.directory(workingDir);
processBuilder.redirectErrorStream();
Process process = processBuilder.start();
final BufferedReader stdout = new BufferedReader(
new InputStreamReader(process.getInputStream()));
String line = null;
try {
while (((line = stdout.readLine()) != null)) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
}
new Thread(() -> {
try {
process.waitFor();
} catch (InterruptedException e) {
e.printStackTrace();
}
}, "process wait").start();
Thread.sleep(Long.MAX_VALUE);
}
}

To summarise discussion, I was not handling standard error streams causing linphone to lockup as its output buffer for standard error became full especially as linphone produces a lot of verbose output on standard error. When RunTime.exec wont provides excellent summary of the pitfalls involved in calling processes from Java.
There was also a typo, I was calling the non-standard named getter
processBuilder.redirectErrorStream();
when I should have been calling the "setter"
processBuilder.redirectErrorStream(true);

How to input filePath on FileReader

I have this code that Reads a file from the file path given.
I have hardcoded F://dom.txt. I need the user to input that filepath instead.
How should i do it? Thanks
import java.io.BufferedReader;
import java.io.FileReader;
public class BuffReader extends Converter {
private static BufferedReader br() throws FileNotFoundException{
return new BufferedReader(new FileReader("F://dom.txt")); //<--filepath
}
static String strTem;
public static String readData(String Message){
try{
System.out.print(Message);
strTem = br().readLine();
}catch(Exception e){
System.err.println("Muling tingan ang iyong numerong ibinigay");
}
return strTem;
}
}

Well you need to give the filename to the oddly-named br() method. For example:
private static BufferedReader br(String path) throws FileNotFoundException {
return new BufferedReader(new FileReader(path));
}
public static String readData(String message){
try{
System.out.print(message);
strTem = br(message).readLine();
}catch(Exception e){
System.err.println("Muling tingan ang iyong numerong ibinigay");
}
return strTem;
}
That's assuming the parameter to readData is actually the file you want to read from... otherwise, you'll need to work out where you are going to get the file name from.
(It would be a good idea to work on exception handling and naming, by the way.)

Assuming you mean the input should come from the console (it could also be command line parameter or a gui or whatever you like really) then you should be able to use System.console().readLine("prompt for input")

(assuming it is a standalone) Change your main method to read the variable args[0].
public static void main (String[] args) {
readData(Message, args[0]);
}
Then change the method signatures for readData() and so on. Basically the above code tells you how to read a string from command line.
If you are asking in the middle of the program:
You can use
System.console().readLine("Enter file name: )
to get the user input.
Also i would suggest to keep this file in a config file and read it from the file in order for the program to be flexible.

Process output from apache-commons exec

I am at my wits end here. I'm sure this is something simple and I most likely have huge holes in my understanding of java and streams. I think there are so many classes that I'm a bit overwhelmed with trying to poke through the API to figure out when and how I want to use the multitude of input/output streams.
I just learned about the existence of the apache commons library (self teaching java fail), and am currently trying to convert some of my Runtime.getRuntime().exec to use the commons - exec. Already it's fixed some of the once every 6 months this problem crops up then goes away style problems with exec.
The code executes a perl script, and displays the stdout from the script in the GUI as it is running.
The calling code is inside of a swingworker.
I'm getting lost how to use the pumpStreamHandler... anyway here is the old code:
String pl_cmd = "perl script.pl"
Process p_pl = Runtime.getRuntime().exec( pl_cmd );
BufferedReader br_pl = new BufferedReader( new InputStreamReader( p_pl.getInputStream() ) );
stdout = br_pl.readLine();
while ( stdout != null )
{
output.displayln( stdout );
stdout = br_pl.readLine();
}
I guess this is what I get for copy pasting code I don't fully understand a long time ago. The above I assume is executing the process, then grabs the outputstream (via "getInputStream"?), places it into a buffered reader, then will just loop there until the buffer is empty.
What I don't get is why there is no need for a 'waitfor' style command here? Isn't it possible that there will be some time in which the buffer will be empty, exit the loop, and continue on while the process is still going? When I run it, this doesn't seem to be the case.
In any event, I'm trying to get the same behavior using commons exec, basically again going from google found code:
DefaultExecuteResultHandler rh = new DefaultExecuteResultHandler();
ExecuteWatchdog wd = new ExecuteWatchdog( ExecuteWatchdog.INFINITE_TIMEOUT );
Executor exec = new DefaultExecutor();
ByteArrayOutputStream out = new ByteArrayOutputStream();
PumpStreamHandler psh = new PumpStreamHandler( out );
exec.setStreamHandler( psh );
exec.setWatchdog( wd );
exec.execute(cmd, rh );
rh.waitFor();
I'm trying to figure out what pumpstreamhandler is doing. I assume that this will take the output from the exec object, and fill the OutputStream I provide it with the bytes from the perl script's stdout/err?
If so how would you get the above behavior to have it stream the output line by line? In examples people show you call the out.toString() at the end, and I assume this would just give me a dump of all the output from the script once it is done running? How would you do it such that it would show the output as it is running line by line?
------------Future Edit ---------------------
Found this via google and works nice as well:
public static void main(String a[]) throws Exception
{
ByteArrayOutputStream stdout = new ByteArrayOutputStream();
PumpStreamHandler psh = new PumpStreamHandler(stdout);
CommandLine cl = CommandLine.parse("ls -al");
DefaultExecutor exec = new DefaultExecutor();
exec.setStreamHandler(psh);
exec.execute(cl);
System.out.println(stdout.toString());
}

Don't pass a ByteArrayOutputStream to the PumpStreamHandler, use an implementation of the abstract class org.apache.commons.exec.LogOutputStream. From the javadoc:
The implementation parses the incoming data to construct a line and passes the complete line to an user-defined implementation.
Thus the LogOutputStram is preprocessing the output to give you the control of handling individual lines instead of the raw bytes. Something like this:
import java.util.LinkedList;
import java.util.List;
import org.apache.commons.exec.LogOutputStream;
public class CollectingLogOutputStream extends LogOutputStream {
private final List<String> lines = new LinkedList<String>();
#Override protected void processLine(String line, int level) {
lines.add(line);
}
public List<String> getLines() {
return lines;
}
}
Then after the blocking call to exec.execute your getLines() will have the standard out and standard error you are looking for. The ExecutionResultHandler is optional from the perspective of just executing the process, and collecting all the stdOut/stdErr into a list of lines.

What I don't get is why there is no need for a 'waitfor' style command here? Isn't it possible that there will be some time in which the buffer will be empty, exit the loop, and continue on while the process is still going? When I run it, this doesn't seem to be the case.
readLine blocks. That is, your code will wait until a line has been read.
PumpStreamHandler
from Documentation
Copies standard output and error of subprocesses to standard output
and error of the parent process. If output or error stream are set to
null, any feedback from that stream will be lost.

Based on James A Wilson's answer I created the helper class "Execute". It wraps his answer
into a solution that also supplies the exitValue for convenience.
A single line is necessary to execute a command this way:
ExecResult result=Execute.execCmd(cmd,expectedExitCode);
The following Junit Testcase tests and shows how to use it:
Junit4 test case:
package com.bitplan.newsletter;
import static org.junit.Assert.*;
import java.util.List;
import org.junit.Test;
import com.bitplan.cmd.Execute;
import com.bitplan.cmd.Execute.ExecResult;
/**
* test case for the execute class
* #author wf
*
*/
public class TestExecute {
#Test
public void testExecute() throws Exception {
String cmd="/bin/ls";
ExecResult result = Execute.execCmd(cmd,0);
assertEquals(0,result.getExitCode());
List<String> lines = result.getLines();
assertTrue(lines.size()>0);
for (String line:lines) {
System.out.println(line);
}
}
}
Execute Java helper Class:
package com.bitplan.cmd;
import java.util.LinkedList;
import java.util.List;
import java.util.logging.Level;
import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecutor;
import org.apache.commons.exec.LogOutputStream;
import org.apache.commons.exec.PumpStreamHandler;
/**
* Execute helper using apache commons exed
*
* add this dependency to your pom.xml:
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-exec</artifactId>
<version>1.2</version>
</dependency>
* #author wf
*
*/
public class Execute {
protected static java.util.logging.Logger LOGGER = java.util.logging.Logger
.getLogger("com.bitplan.cmd");
protected final static boolean debug=true;
/**
* LogOutputStream
* http://stackoverflow.com/questions/7340452/process-output-from
* -apache-commons-exec
*
* #author wf
*
*/
public static class ExecResult extends LogOutputStream {
private int exitCode;
/**
* #return the exitCode
*/
public int getExitCode() {
return exitCode;
}
/**
* #param exitCode the exitCode to set
*/
public void setExitCode(int exitCode) {
this.exitCode = exitCode;
}
private final List<String> lines = new LinkedList<String>();
#Override
protected void processLine(String line, int level) {
lines.add(line);
}
public List<String> getLines() {
return lines;
}
}
/**
* execute the given command
* #param cmd - the command
* #param exitValue - the expected exit Value
* #return the output as lines and exit Code
* #throws Exception
*/
public static ExecResult execCmd(String cmd, int exitValue) throws Exception {
if (debug)
LOGGER.log(Level.INFO,"running "+cmd);
CommandLine commandLine = CommandLine.parse(cmd);
DefaultExecutor executor = new DefaultExecutor();
executor.setExitValue(exitValue);
ExecResult result =new ExecResult();
executor.setStreamHandler(new PumpStreamHandler(result));
result.setExitCode(executor.execute(commandLine));
return result;
}
}

Its a very old thread but I had to use Apache Commons Exec and had to solve the same problem. I trust with last version of Apache Commons Exec published in 2014, below solution works well both with and without watchdog;
class CollectingLogOutputStream implements ExecuteStreamHandler {
private final List<String> lines = new LinkedList<String>();
public void setProcessInputStream(OutputStream outputStream) throws IOException
{
}
//important - read all output line by line to track errors
public void setProcessErrorStream(InputStream inputStream) throws IOException {
InputStreamReader isr = new InputStreamReader(inputStream);
BufferedReader br = new BufferedReader(isr);
String line="";
while( (line = br.readLine()) != null){
//use lines whereever you want - for now just print on console
System.out.println("error:"+line);
}
}
//important - read all output line by line to track process output
public void setProcessOutputStream(InputStream inputStream) throws IOException
{
InputStreamReader isr = new InputStreamReader(inputStream);
BufferedReader br = new BufferedReader(isr);
String line="";
while( (line = br.readLine()) != null){
//use lines whereever you want - for now just print on console
System.out.println("output:"+line);
}
}
public void start() throws IOException {
}
public void stop() throws IOException {
}
}
Above class can be set as StreamHandler for the executor as below;
//set newly created class stream handler for the executor
executor.setStreamHandler(new CollectingLogOutputStream());
Complete code is available here;
https://github.com/raohammad/externalprocessfromjava

Prevent launching multiple instances of a java application

I want to prevent the user from running my java application multiple times in parallel.
To prevent this, I have created a lock file when am opening the application, and delete the lock file when closing the application.
When the application is running, you can not open an another instance of jar. However, if you kill the application through task manager, the window closing event in the application is not triggered and the lock file is not deleted.
How can I make sure the lock file method works or what other mechanism could I use?

You could use a FileLock, this also works in environments where multiple users share ports:
String userHome = System.getProperty("user.home");
File file = new File(userHome, "my.lock");
try {
FileChannel fc = FileChannel.open(file.toPath(),
StandardOpenOption.CREATE,
StandardOpenOption.WRITE);
FileLock lock = fc.tryLock();
if (lock == null) {
System.out.println("another instance is running");
}
} catch (IOException e) {
throw new Error(e);
}
Also survives Garbage Collection.
The lock is released once your process ends, doesn't matter if regular exit or crash or whatever.

Similar discussion is at
http://www.daniweb.com/software-development/java/threads/83331
Bind a ServerSocket. If it fails to bind then abort the startup. Since a ServerSocket can be bound only once, only single instsances of the program will be able to run.
And before you ask, no. Just because you bind a ServerSocket, does not mean you are open to network traffic. That only comes into effect once the program starts "listening" to the port with accept().

I see two options you can try:
Use a Java shutdown hook
Have your lock file hold the main process number. The process should exist when you lanuch another instance. If it's not found in your system, you can assume that the lock can be dismissed and overwritten.

Creating a server socket, bounds to a specific port with a ServerSocket instance as the application starts is a straight way.
Note that ServerSocket.accept() blocks, so running it in its own thread makes sense to not block the main Thread.
Here is an example with a exception thrown as detected :
public static void main(String[] args) {
assertNoOtherInstanceRunning();
... // application code then
}
public static void assertNoOtherInstanceRunning() {
new Thread(() -> {
try {
new ServerSocket(9000).accept();
} catch (IOException e) {
throw new RuntimeException("the application is probably already started", e);
}
}).start();
}

You could write the process id of the process that created the lock file into the file.
When you encounter an existing lock file, you do not just quit, but you check if the process with that id is still alive. If not, you can go ahead.

You can create a Server socket like
new ServerSocket(65535, 1, InetAddress.getLocalHost());
at very beginning of your code. Then if AddressAlreadyInUse exception caught in main block you can display the appropriate message.

There are already available java methods in File class to achieve the same. The method is deleteOnExit() which ensure the file is automatically deleted when the JVM exits. However, it does not cater to forcible terminations. One should use FileLock in case of forcible termination.
For more details check, https://docs.oracle.com/javase/7/docs/api/java/io/File.html
Thus code snippet which could be used in the main method can be like :
public static void main(String args[]) throws Exception {
File f = new File("checkFile");
if (!f.exists()) {
f.createNewFile();
} else {
System.out.println("App already running" );
return;
}
f.deleteOnExit();
// whatever your app is supposed to do
System.out.println("Blah Blah")
}

..what other mechanism could I use?
If the app. has a GUI it can be launched using Java Web Start. The JNLP API provided to web-start offers the SingleInstanceService. Here is my demo. of the SingleInstanceService.

You can write something like this.
If file exists try to delete it. if it is not able to delete. We can say that application is already running.
Now create the same file again and redirect the sysout and syserr.
This works for me

Simple lock and advanced lock
I developed 2 solutions for this problem. I was also looking for an easy way of doing this without using any libraries and a lot of code.
My solutions are based on: https://stackoverflow.com/a/46705579/10686802 which I have improved upon. Therefore I would like to thank #akshaya pandey and #rbento
Simple file lock
package YOUR_PACKAGE_NAME;
import java.io.File;
import java.io.IOException;
/**
* Minimal reproducible example (MRE) - Example of a simple lock file.
* #author Remzi Cavdar - ict#remzi.info - #Remzi1993
*/
public class Main {
public static void main(String[] args) {
/*
* Prevents the user of starting multiple instances of the application.
* This is done by creating a temporary file in the app directory.
* The temp file should be excluded from git and is called App.lock in this example.
*/
final File FILE = new File("App.lock");
try {
if (FILE.createNewFile()) {
System.out.println("Starting application");
} else {
System.err.println("The application is already running!");
return;
}
} catch (IOException e) {
throw new RuntimeException(e);
}
/*
* Register a shutdown hook to delete the lock file when the application is closed. Even when forcefully closed
* with the task manager. (Tested on Windows 11 with JavaFX 19)
*/
FILE.deleteOnExit();
// Whatever your app is supposed to do
}
}
Advanced lock
package YOUR_PACKAGE_NAME;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
/**
* Minimal reproducible example (MRE) - Example of a more advanced lock system.
* #author Remzi Cavdar - ict#remzi.info - #Remzi1993
*/
public class Main {
public static void main(String[] args) {
/*
* Prevents the user of starting multiple instances of the application.
* This is done by creating a temporary file in the app directory.
* The temp file should be excluded from git and is called App.lock in this example.
*/
final File FILE = new File("App.lock");
if (FILE.exists()) {
System.err.println("The application is already running!");
return;
}
try (
FileOutputStream fileOutputStream = new FileOutputStream(FILE);
FileChannel channel = fileOutputStream.getChannel();
FileLock lock = channel.lock()
) {
System.out.println("Starting application");
} catch (FileNotFoundException e) {
throw new RuntimeException(e);
} catch (IOException e) {
throw new RuntimeException(e);
}
/*
* Register a shutdown hook to delete the lock file when the application is closed. Even when forcefully closed
* with the task manager. (Tested on Windows 11 with JavaFX 19)
*/
FILE.deleteOnExit();
// Whatever your app is supposed to do
}
}
Testing
Tested on: 31-10-2022
Tested OS: Windows 11 - Version 21H2 (OS Build 22000.1098)
Tested with: OpenJDK 19 - Eclipse Temurin JDK with Hotspot 19+36(x64)
I closed the application and also forcefully closed the application with task manager on Windows both times the lock file seems to be deleted upon (force) close.

I struggled with this same problem for a while... none of the ideas presented here worked for me. In all cases, the lock (file, socket or otherwise) did not persist into the 2nd process instance, so the 2nd instance still ran.
So I decided to try an old school approach to simply crate a .pid file with the process id of the first process. Then any 2nd process would quit if it finds the .pid file, and also the process number specified in the file is confirmed to be still running. This approach worked for me.
There is a fair bit of code, which I provide here in full for your use... a complete solution.
package common.environment;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import javax.annotation.Nonnull;
import javax.annotation.Nullable;
import java.io.*;
import java.nio.charset.Charset;
public class SingleAppInstance
{
private static final #Nonnull Logger log = LogManager.getLogger(SingleAppInstance.class.getName());
/**
* Enforces that only a single instance of the given component is running. This
* is resilient to crashes, unexpected reboots and other forceful termination
* scenarios.
*
* #param componentName = Name of this component, for disambiguation with other
* components that may run simultaneously with this one.
* #return = true if the program is the only instance and is allowed to run.
*/
public static boolean isOnlyInstanceOf(#Nonnull String componentName)
{
boolean result = false;
// Make sure the directory exists
String dirPath = getHomePath();
try
{
FileUtil.createDirectories(dirPath);
}
catch (IOException e)
{
throw new RuntimeException(String.format("Unable to create directory: [%s]", dirPath));
}
File pidFile = new File(dirPath, componentName + ".pid");
// Try to read a prior, existing pid from the pid file. Returns null if the file doesn't exist.
String oldPid = FileUtil.readFile(pidFile);
// See if such a process is running.
if (oldPid != null && ProcessChecker.isStillAllive(oldPid))
{
log.error(String.format("An instance of %s is already running", componentName));
}
// If that process isn't running, create a new lock file for the current process.
else
{
// Write current pid to the file.
long thisPid = ProcessHandle.current().pid();
FileUtil.createFile(pidFile.getAbsolutePath(), String.valueOf(thisPid));
// Try to be tidy. Note: This won't happen on exit if forcibly terminated, so we don't depend on it.
pidFile.deleteOnExit();
result = true;
}
return result;
}
public static #Nonnull String getHomePath()
{
// Returns a path like C:/Users/Person/
return System.getProperty("user.home") + "/";
}
}
class ProcessChecker
{
private static final #Nonnull Logger log = LogManager.getLogger(io.cpucoin.core.platform.ProcessChecker.class.getName());
static boolean isStillAllive(#Nonnull String pidStr)
{
String OS = System.getProperty("os.name").toLowerCase();
String command;
if (OS.contains("win"))
{
log.debug("Check alive Windows mode. Pid: [{}]", pidStr);
command = "cmd /c tasklist /FI \"PID eq " + pidStr + "\"";
}
else if (OS.contains("nix") || OS.contains("nux"))
{
log.debug("Check alive Linux/Unix mode. Pid: [{}]", pidStr);
command = "ps -p " + pidStr;
}
else
{
log.warn("Unsupported OS: Check alive for Pid: [{}] return false", pidStr);
return false;
}
return isProcessIdRunning(pidStr, command); // call generic implementation
}
private static boolean isProcessIdRunning(#Nonnull String pid, #Nonnull String command)
{
log.debug("Command [{}]", command);
try
{
Runtime rt = Runtime.getRuntime();
Process pr = rt.exec(command);
InputStreamReader isReader = new InputStreamReader(pr.getInputStream());
BufferedReader bReader = new BufferedReader(isReader);
String strLine;
while ((strLine = bReader.readLine()) != null)
{
if (strLine.contains(" " + pid + " "))
{
return true;
}
}
return false;
}
catch (Exception ex)
{
log.warn("Got exception using system command [{}].", command, ex);
return true;
}
}
}
class FileUtil
{
static void createDirectories(#Nonnull String dirPath) throws IOException
{
File dir = new File(dirPath);
if (dir.mkdirs()) /* If false, directories already exist so nothing to do. */
{
if (!dir.exists())
{
throw new IOException(String.format("Failed to create directory (access permissions problem?): [%s]", dirPath));
}
}
}
static void createFile(#Nonnull String fullPathToFile, #Nonnull String contents)
{
try (PrintWriter writer = new PrintWriter(fullPathToFile, Charset.defaultCharset()))
{
writer.print(contents);
}
catch (IOException e)
{
throw new RuntimeException(String.format("Unable to create file at %s! %s", fullPathToFile, e.getMessage()), e);
}
}
static #Nullable String readFile(#Nonnull File file)
{
try
{
try (BufferedReader fileReader = new BufferedReader(new FileReader(file)))
{
StringBuilder result = new StringBuilder();
String line;
while ((line = fileReader.readLine()) != null)
{
result.append(line);
if (fileReader.ready())
result.append("\n");
}
return result.toString();
}
}
catch (IOException e)
{
return null;
}
}
}
To use it, simply invoke it like this:
if (!SingleAppInstance.isOnlyInstanceOf("my-component"))
{
// quit
}
I hope you find this helpful.

Finally I found really simple library to achieve this. You can you use JUniqe.
The JUnique library can be used to prevent a user to run at the same
time more instances of the same Java application.
This is an example how to use it from the documentation
public static void main(String[] args) {
String appId = "myapplicationid";
boolean alreadyRunning;
try {
JUnique.acquireLock(appId);
alreadyRunning = false;
} catch (AlreadyLockedException e) {
alreadyRunning = true;
}
if (!alreadyRunning) {
// Start sequence here
}
}

here is a pretty rudimental approach.
If your application is launched from a script, check the running java/javaw processes and their command line before launch
In windows
REM check if there is a javaw process running your.main.class
REM if found, go to the end of the script and skip the launch of a new instance
WMIC path win32_process WHERE "Name='javaw.exe'" get CommandLine 2>nul | findstr your.main.class >nul 2>&1
if %ERRORLEVEL% EQU 0 goto:eof
javaw your.main.class

System.out to a file in java

I'm running an application from inside another one for testing purposes. I want to redirect the output for the tested app to a file, so I can have a log after each test.
Is there a way to redirect the output of an app to a file from the command line in java?

You can use the output stream redirector that is supported by the Windows command line, *nix shells , e.g.
java -jar myjar.jar > output.txt
Alternatively, as you are running the app from inside the vm, you could redirect System.out from within java itself. You can use the method
System.setOut(PrintStream ps)
Which replaces the standard output stream, so all subsequent calls to System.out go to the stream you specify. You could do this before running your wrapped application, e.g. calling System.setOut(new PrintStream(new BufferedOutputStream(new FileOutputStream("output.txt"))));
If you are using a wrapper that you can't modify, then create your own wrapper. So you have FEST wrapper -> stream redirector wrapper -> tested app.
For example, you can implement a simple wrapper like this:
public class OutputRedirector
{
/* args[0] - class to launch, args[1]/args[2] file to direct System.out/System.err to */
public static void main(String[] args) throws Exception
{ // error checking omitted for brevity
System.setOut(outputFile(args(1));
System.setErr(outputFile(args(2));
Class app = Class.forName(args[0]);
Method main = app.getDeclaredMethod("main", new Class[] { (new String[1]).getClass()});
String[] appArgs = new String[args.length-3];
System.arraycopy(args, 3, appArgs, 0, appArgs.length);
main.invoke(null, appArgs);
}
protected PrintStream outputFile(String name) {
return new PrintStream(new BufferedOutputStream(new FileOutputStream(name)), true);
}
}
You invoke it with 3 additional params - the Main class to run, and the output/error directs.

When using this constructor:
new PrintStream(new BufferedOutputStream(new FileOutputStream("file.txt")));
remember to set autoflushing to true, i.e.:
new PrintStream(new BufferedOutputStream(new FileOutputStream("file.txt")), true);
otherwise you may get empty files even after your program finishes.

Yes you can set your desired file like this.
try {
System.setOut(new PrintStream(new File("output-file.txt")));
} catch (Exception e) {
e.printStackTrace();
}

System.out.println() is used to print messages on the console.
System is a class defined in the java.lang package. out is an instance of PrintStream, which is a public and static member of the class System. As all instances of PrintStream class have a public method println().
System.out is a static PrintStream that writes to the console. We can redirect the output to a different PrintStream using the System.setOut() method which takes a PrintStream as a parameter.
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.PrintStream;
public class SetPrintStream {
public static void main(String[] args) throws FileNotFoundException{
System.out.println("Print on console");
// Store console print stream.
PrintStream ps_console = System.out;
File file = new File("file.txt");
FileOutputStream fos = new FileOutputStream(file);
// Create new print stream for file.
PrintStream ps = new PrintStream(fos);
// Set file print stream.
System.setOut(ps);
System.out.println("Print in the file !!");
// Set console print stream.
System.setOut(ps_console);
System.out.println("Console again !!");
}
}
Output:
Print on console
Console again !!
new file.txt will be created.
For more information see my blog:
http://javaexplorer03.blogspot.in/2016/02/how-do-i-redirect-standard-output-to.html

In order to improve "vijay.shad" response I used the code bellow to direct the file to the Home Directory in Linux or MyDocuments in Windows.
try {
System.setOut(new PrintStream(new File(FileSystemView.getFileSystemView()
.getDefaultDirectory().toString()
+ File.separator + "output-file.txt")));
} catch (Exception e) {
e.printStackTrace();
}

File file = new File("xyz.txt");
PrintStream printStreamToFile = new PrintStream(file);
System.setOut(printStreamToFile);
System.out.println("Hello I am writing to File xyz.txt");

Or you can use Class FileWriter.