How do I know if a software is done writing a file if I am executing that software from java?For example, I am executing geniatagger.exe with an input file RawText that will produce an output file TAGGEDTEXT.txt. When geniatagger.exe is finished writing the TAGGEDTEXT.txt file, I can do some other staffs with this file. The problem is- how can I know that geniatagger is finished writing the text file?
try{
Runtime rt = Runtime.getRuntime();
Process p = rt.exec("geniatagger.exe -i "+ RawText+ " -o TAGGEDTEXT.txt");
}
You can't, or at least not reliably.
In this particular case your best bet is to watch the Process complete.
You get the process' return code as a bonus, this could tell you if an error occurred.
If you are actually talking about this GENIA tagger, below is a practical example which demonstrates various topics (see explanation about numbered comments beneath the code). The code was tested with v1.0 for Linux and demonstrates how to safely run a process which expects both input and output stream piping to work correctly.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.concurrent.Callable;
import org.apache.commons.io.IOUtils;
public class GeniaTagger {
/**
* #param args
*/
public static void main(String[] args) {
tagFile(new File("inputText.txt"), new File("outputText.txt"));
}
public static void tagFile(File input, File output) {
FileInputStream ifs = null;
FileOutputStream ofs = null;
try {
ifs = new FileInputStream(input);
ofs = new FileOutputStream(output);
final FileInputStream ifsRef = ifs;
final FileOutputStream ofsRef = ofs;
// {1}
ProcessBuilder pb = new ProcessBuilder("geniatagger.exe");
final Process pr = pb.start();
// {2}
runInThread(new Callable<Void>() {
public Void call() throws Exception {
IOUtils.copy(ifsRef, pr.getOutputStream());
IOUtils.closeQuietly(pr.getOutputStream()); // {3}
return null;
}
});
runInThread(new Callable<Void>() {
public Void call() throws Exception {
IOUtils.copy(pr.getInputStream(), ofsRef); // {4}
return null;
}
});
runInThread(new Callable<Void>() {
public Void call() throws Exception {
IOUtils.copy(pr.getErrorStream(), System.err);
return null;
}
});
// {5}
pr.waitFor();
// output file is written at this point.
} catch (Exception e) {
e.printStackTrace();
} finally {
// {6}
IOUtils.closeQuietly(ifs);
IOUtils.closeQuietly(ofs);
}
}
public static void runInThread(final Callable<?> c) {
new Thread() {
public void run() {
try {
c.call();
} catch (Exception e) {
e.printStackTrace();
} finally {
}
}
}.start();
}
}
Use a ProcessBuilder to start your process, it has a better interface than plain-old Runtime.getRuntime().exec(...).
Set up stream piping in different threads, otherwhise the waitFor() call in ({5}) might never complete.
Note that I piped a FileInputStream to the process. According to the afore-mentioned GENIA page, this command expects actual input instead of a -i parameter. The OutputStream which connects to the process must be closed, otherwhise the program will keep running!
Copy the result of the process to a FileOutputStream, the result file your are waiting for.
Let the main thread wait until the process completes.
Clean up all streams.
If the program exits after generating the output file then you can call Process.waitFor() to let it run to completion then you can process the file. Note that you will likely have to drain both the standard output and error streams (at least on Windows) for the process to finish.
[Edit]
Here is an example, untested and likely fraught with problems:
// ...
Process p = rt.exec("geniatagger.exe -i "+ RawText+ " -o TAGGEDTEXT.txt");
drain(p.getInputStream());
drain(p.getErrorStream());
int exitCode = p.waitFor();
// Now you should be able to process the output file.
}
private static void drain(InputStream in) throws IOException {
while (in.read() != -1);
}
Related
I am new in java, a self learner. I came accross the following issue and was stuck. In fact I am trying to sanitize this code against command injection but failed to understand how. I know how to sanitize user input but this specific has to do with command executed in the OS and I am not sure how anyone help please. here is the code:
public class CommandProcessor {
public CommandProcessor() {
// TODO Auto-generated constructor stub
}
public int invokeCommand(String command) throws IOException{
int exitCode =0;
if(command !=null && !command.isEmpty()) {
Process process = null;
try {
process = Runtime.getRuntime().exec(command);
process.waitFor();
exitCode = process.exitValue();
}catch(InterruptedException e) {
}
}
return exitCode;
}
}
The correct answer is to read the documentation as your current code is not safe.
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Runtime.html#exec(java.lang.String%5B%5D)
The "command to execute" should be a constant.
This is the code I'm running:
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
public class Main {
public static void main(String[] args) throws Exception {
String filePath = "D:/temp/file";
RandomAccessFile file = new RandomAccessFile(filePath, "rw");
try {
MappedByteBuffer buffer = file.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, 128);
// Do something
buffer.putInt(4);
} finally {
file.close();
System.out.println("File closed");
}
System.out.println("Press any key...");
System.in.read();
System.out.println("Finished");
}
}
Before pressing a key, I'm trying to delete the file manually in FAR Manager. But FAR says that the file is locked:
The process cannot access the file because it is being used by another process.
Cannot delete the file
D:\temp\file
Object is being opened in:
Java(TM) Platform SE binary (PID: 5768, C:\Program Files\Java\jdk1.8.0_05\bin\javaw.exe)
Only after pressing a key, the application terminates and I can delete the file.
What is wrong with my code?
Try this one.
public class Test
{
public static void main(String[] args) throws Exception {
String filePath = "D:/temp/file";
RandomAccessFile file = new RandomAccessFile(filePath, "rw");
FileChannel chan = file.getChannel();
try {
MappedByteBuffer buffer = chan.map(FileChannel.MapMode.READ_WRITE, 0, 128);
// Do something
buffer.putInt(4);
buffer.force();
Cleaner cleaner = ((sun.nio.ch.DirectBuffer) buffer).cleaner();
if (cleaner != null) {
cleaner.clean();
}
} finally {
chan.close();
file.close();
System.out.println("File closed");
}
System.out.println("Press any key...");
System.in.read();
System.out.println("Finished");
}
}
#SANN3's answer doesn't work on Java 9 anymore. In Java 9 there is a new method sun.misc.Unsafe.invokeCleaner that can be used. Here is a working code:
MappedByteBuffer buffer = ...
// Java 9+ only:
Class<?> unsafeClass = Class.forName("sun.misc.Unsafe");
Field unsafeField = unsafeClass.getDeclaredField("theUnsafe");
unsafeField.setAccessible(true);
Object unsafe = unsafeField.get(null);
Method invokeCleaner = unsafeClass.getMethod("invokeCleaner", ByteBuffer.class);
invokeCleaner.invoke(unsafe, buffer);
If you are using java1.8 and cannot directly use sun.nio.ch.DirectBuffer and Cleaner, you can try:
public void clean(final ByteBuffer buffer) {
AccessController.doPrivileged((PrivilegedAction<Object>) () -> {
try {
Field field = buffer.getClass().getDeclaredField("cleaner");
field.setAccessible(true);
Object cleaner = field.get(buffer);
Method cleanMethod = cleaner.getClass().getMethod("clean");
cleanMethod.invoke(cleaner);
} catch (Exception e) {
e.printStackTrace();
}
return null;
});
}
This is actually a limitation of JDK. Since the JDK-4724038 which tracks this problem (even though it is marked an enhancement) in JDK says that invoking the cleanup method directly is strongly advised against (also, that the Unsafe class might go away in some future version of JDK), the only workaround seems to be to call the GC. If using the try-with-resources for the file, that would look like this:
try (RandomAccessFile file = new RandomAccessFile(filePath, "rw")) {
MappedByteBuffer buffer = file.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, 128);
// Do something
buffer.putInt(4);
}
System.gc(); // has to be called outside the try-with-resources block
I created https://github.com/vladak/RandomAccessFileTrap to demonstrate this - take a look at the detail of a build in the Github actions tab for this repository to see the actual results.
I'm trying to execute terminal command in linux trough Java and i cant get any input from inputStream.
This is my code
ProcessBuilder build = new ProcessBuilder("/usr/bin/xterm", "find /home");
Process pr = null;
BufferedReader buf;
try {
build.redirectErrorStream(true);
pr = build.start();
buf = new BufferedReader(new InputStreamReader( pr.getInputStream()));
String line = buf.readLine();
pr.waitFor();
while (true) {
System.out.println(line + "sadasdas");
line = buf.readLine();
}
} catch (Exception e) {
e.printStackTrace();
}
Process is executed and immediately terminal closes, and no output is catched and printed. On the other hand if i will compose an unknown command i get all the lines with tips how to use commands. Same problem i had with windows cmd. I was trying to use getRuntime.exec(cmd) method but the end is the same.
I've also tried to created separate threads for process and reader which looks like this
public class kurdee
{
public static Thread thread;
public kurdee()
{
List cmd = new LinkedList();
cmd.add(new String("/usr/bin/xterm"));
cmd.add(new String("find"));
thisProc thispr = new thisProc(cmd);
this.thread = new Thread(thispr);
thread.start();
reader rd = new reader(thispr.proc);
Thread thread1 = new Thread(rd);
thread1.start();}
public static void main(String args[])
{
java.awt.EventQueue.invokeLater(new Runnable() {
public void run() {
kurdee kurd = new kurdee();
}
});
}
}
class reader implements Runnable
{
private BufferedReader buf;
private Process proc;
public reader(Process proc)
{
this.proc=proc;
this.buf = new BufferedReader(new InputStreamReader(proc.getInputStream()));
}
public void run()
{
String line="";
System.out.println("Thread is alive");
try{
//Thread.sleep(1000);
line = buf.readLine();
}catch(Exception ex){System.out.println(ex + " before first while started");}
while(kurdee.thread.isAlive())
{
System.out.println("Thread is alive");
while(line!=null)
{
try{
//System.out.println(proc.exitValue());
System.out.println(line + " asd");
line=buf.readLine();
}catch(Exception e){System.out.println(e + " Inner while loop");}
}
}
}
}
class thisProc implements Runnable
{
private ProcessBuilder build;
public static Process proc=null;
public thisProc(List<String> args)
{
this.build = new ProcessBuilder(args);
build.redirectErrorStream(true);
try{
this.proc = build.start();
}catch(Exception ex){System.out.println(ex + " proc class");}
}
public void run()
{
try{
proc.waitFor();
}catch(Exception ex){System.out.println(ex + " proc class");}
}
}
But with any combination of invoking threads etc i make there is still nothing to read.
I'm trying to use command "find /home -xdev -samefile file" to get all hard links to file so maybe there is an easier way.
xterm is not the way to execute processes in unix, it is not a shell. a shell is something like "/bin/sh". however, "find" is a normal unix executable, so you should just execute that directly, e.g. new ProcessBuilder("find", "/home"). and yes, you should always process the streams on separate threads, as recommended by this article.
First, don't try to execute the command with xterm, that's pointless; just do it directly. Secondly, be careful when you compose your array of command strings to put one word into each string; passing, for example "find /home" as a single string among many to ProcessBuilder is going to error out.
I have a Java program that outputs some text into console. It uses print, println, and some other methods to do this.
At the end of the program , I want to read all the text in console and copy it into a String buffer. How could I do this in Java ? I need to read stdout and stderr separately.
Ok, this was a fun problem. Dosen't seem to be an elegant way of solving it for all PrintStream methods at once. (Unfortunately there is no FilterPrintStream.)
I did write up an ugly reflection-based workaround though (not to be used in production code I suppose :)
class LoggedPrintStream extends PrintStream {
final StringBuilder buf;
final PrintStream underlying;
LoggedPrintStream(StringBuilder sb, OutputStream os, PrintStream ul) {
super(os);
this.buf = sb;
this.underlying = ul;
}
public static LoggedPrintStream create(PrintStream toLog) {
try {
final StringBuilder sb = new StringBuilder();
Field f = FilterOutputStream.class.getDeclaredField("out");
f.setAccessible(true);
OutputStream psout = (OutputStream) f.get(toLog);
return new LoggedPrintStream(sb, new FilterOutputStream(psout) {
public void write(int b) throws IOException {
super.write(b);
sb.append((char) b);
}
}, toLog);
} catch (NoSuchFieldException shouldNotHappen) {
} catch (IllegalArgumentException shouldNotHappen) {
} catch (IllegalAccessException shouldNotHappen) {
}
return null;
}
}
...that can be used like this:
public class Test {
public static void main(String[] args) {
// Create logged PrintStreams
LoggedPrintStream lpsOut = LoggedPrintStream.create(System.out);
LoggedPrintStream lpsErr = LoggedPrintStream.create(System.err);
// Set them to stdout / stderr
System.setOut(lpsOut);
System.setErr(lpsErr);
// Print some stuff
System.out.print("hello ");
System.out.println(5);
System.out.flush();
System.err.println("Some error");
System.err.flush();
// Restore System.out / System.err
System.setOut(lpsOut.underlying);
System.setErr(lpsErr.underlying);
// Print the logged output
System.out.println("----- Log for System.out: -----\n" + lpsOut.buf);
System.out.println("----- Log for System.err: -----\n" + lpsErr.buf);
}
}
Resulting output:
hello 5
Some error
----- Log for System.out: -----
hello 5
----- Log for System.err: -----
Some error
(Note though, that the out field in FilterOutputStream is protected and documented, so it is part of the API :-)
You can't do that once the program is finished running. You need to do it before the program starts to write output.
See this article(archive.org) for details on how to replace stdout and stderr. The core calls are System.setOut() and System.setErr().
You can use PipedInputStream and PipedOutputStream.
//create pairs of Piped input and output streasm for std out and std err
final PipedInputStream outPipedInputStream = new PipedInputStream();
final PrintStream outPrintStream = new PrintStream(new PipedOutputStream(
outPipedInputStream));
final BufferedReader outReader = new BufferedReader(
new InputStreamReader(outPipedInputStream));
final PipedInputStream errPipedInputStream = new PipedInputStream();
final PrintStream errPrintStream = new PrintStream(new PipedOutputStream(
errPipedInputStream));
final BufferedReader errReader = new BufferedReader(
new InputStreamReader(errPipedInputStream));
final PrintStream originalOutStream = System.out;
final PrintStream originalErrStream = System.err;
final Thread writingThread = new Thread(new Runnable() {
#Override
public void run() {
try {
System.setOut(outPrintStream);
System.setErr(errPrintStream);
// You could also set the System.in here using a
// PipedInputStream
DoSomething();
// Even better would be to refactor DoSomething to accept
// PrintStream objects as parameters to replace all uses of
// System.out and System.err. DoSomething could also have
// an overload with DoSomething() calling:
DoSomething(outPrintStream, errPrintStream);
} finally {
// may also want to add a catch for exceptions but it is
// essential to restore the original System output and error
// streams since it can be very confusing to not be able to
// find System.out output on your console
System.setOut(originalOutStream);
System.setErr(originalErrStream);
//You must close the streams which will auto flush them
outPrintStream.close();
errPrintStream.close();
}
} // end run()
}); // end writing thread
//Start the code that will write into streams
writingThread.start();
String line;
final List<String> completeOutputStreamContent = new ArrayList<String>();
while ((line = outReader.readLine()) != null) {
completeOutputStreamContent.add(line);
} // end reading output stream
final List<String> completeErrorStreamContent = new ArrayList<String>();
while ((line = errReader.readLine()) != null) {
completeErrorStreamContent.add(line);
} // end reading output stream
Here is a utility Class named ConsoleOutputCapturer. It allows the output to go to the existing console however behind the scene keeps capturing the output text. You can control what to capture with the start/stop methods. In other words call start to start capturing the console output and once you are done capturing you can call the stop method which returns a String value holding the console output for the time window between start-stop calls. This class is not thread-safe though.
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintStream;
import java.util.Arrays;
import java.util.List;
public class ConsoleOutputCapturer {
private ByteArrayOutputStream baos;
private PrintStream previous;
private boolean capturing;
public void start() {
if (capturing) {
return;
}
capturing = true;
previous = System.out;
baos = new ByteArrayOutputStream();
OutputStream outputStreamCombiner =
new OutputStreamCombiner(Arrays.asList(previous, baos));
PrintStream custom = new PrintStream(outputStreamCombiner);
System.setOut(custom);
}
public String stop() {
if (!capturing) {
return "";
}
System.setOut(previous);
String capturedValue = baos.toString();
baos = null;
previous = null;
capturing = false;
return capturedValue;
}
private static class OutputStreamCombiner extends OutputStream {
private List<OutputStream> outputStreams;
public OutputStreamCombiner(List<OutputStream> outputStreams) {
this.outputStreams = outputStreams;
}
public void write(int b) throws IOException {
for (OutputStream os : outputStreams) {
os.write(b);
}
}
public void flush() throws IOException {
for (OutputStream os : outputStreams) {
os.flush();
}
}
public void close() throws IOException {
for (OutputStream os : outputStreams) {
os.close();
}
}
}
}
Don't do it afterwards, create two StringBuilder objects before the first System.out.print() gets called and just append every string you want to save to the appropriate StringBuilder.
These two line of code will put your output in a text file or u can change the destination as u require.
// Create a file:
System.setOut(new PrintStream( new FileOutputStream("D:/MyOutputFile.txt")));
// Redirect the output to the file:
System.out.println("Hello to custom output stream!");
hope its help u .. :)
I'm using Eclipse's external tools functionality to launch my test server (I can't use the normal servers view for it since it's not supported).
That works fine, but it's a bit sad that I can't click the stacktraces to automatically jump to that line in the code (as you could normally do). I always thought eclipse's console automatically recognized lines of code.
Is there any way to make it do that for external tools?
Thanks
You can copy the stack trace to a Java Stack Trace console. In the Console, switch to a new Java Stack Trace console, paste the stack trace and it will be immediately clickable.
Also, check out the LogViewer plugin, as far as I can recall, it can do that with less effort
As a workaround, I created a simple Java wrapper program which executes the command given as arguments. This allows an Eclipse Java run configuration to be used instead of an external tool.
public class Exec {
private final Process process;
private boolean error;
public Exec(Process process) {
this.process = process;
}
public static void main(String[] command) throws Exception {
new Exec(Runtime.getRuntime().exec(command)).run();
}
public void run() throws Exception {
Thread thread = new Thread(() -> copy(process.getInputStream(), System.out));
thread.start();
copy(process.getErrorStream(), System.err);
int status = process.waitFor();
thread.join();
System.err.flush();
System.out.flush();
System.exit(status != 0 ? status : error ? 1 : 0);
}
private void copy(InputStream in, OutputStream out) {
try {
byte[] buffer = new byte[4096];
for (int count; (count = in.read(buffer)) > 0;) {
out.write(buffer, 0, count);
}
} catch (IOException e) {
error = true;
e.printStackTrace();
}
}
}