Accessing the output of a command running in a docker container - java

I'm trying to upgrade from docker-java 0.10.3 to 3.2.7. This line has me completely stumped:
InputStream response =
dockerClient.attachContainerCmd(container.getId())
.withLogs(true)
.withStdErr(true)
.withStdOut(true)
.withFollowStream(true)
.exec();
I have managed to get round one error by changing it to
InputStream response =
dockerClient.attachContainerCmd(container.getId())
.withLogs(true)
.withStdErr(true)
.withStdOut(true)
.withFollowStream(true)
.exec(new AttachContainerResultCallback());
(but my IDE says that AttachContainerResultCallback is deprecated.) The problem is that .exec() used to return an InputStream. Now it returns a void. I need the InputStream, because the output of the commands running in the container needs to find it's way to the screen. This needs to be realtime, because the user needs to see the output of the commands as they are running; I can't just copy a file at the end.
How can I get hold of this InputStream?
The error is:
java: incompatible types: inference variable T has incompatible bounds
lower bounds: java.io.InputStream,com.github.dockerjava.api.async.ResultCallback<com.github.dockerjava.api.model.Frame>
lower bounds: com.github.dockerjava.core.command.AttachContainerResultCallback

Try it:
var outputStream = new PipedOutputStream();
var inputStream = new PipedInputStream(outputStream);
var response =
dockerClient.attachContainerCmd(container.getId())
.withLogs(false)
.withStdErr(true)
.withStdOut(true)
.withFollowStream(true)
.exec(new ResultCallback.Adapter<>() {
#Override
public void onNext(Frame object) {
System.out.println(object); //for example
try {
outputStream.write(object.getPayload());
} catch (IOException e) {
throw new RuntimeException(e);
}
}
});
try {
response.awaitCompletion();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
Variable inputStream will be what you are looking for.
P.S. In my case I do withLogs(false) because it blocks the current output and I get only part of the log. It probably has something to do with the container I'm connecting to.

Related

How to find the next array in serialization [duplicate]

My problem is when it tries to read the object the second time, it throws the exception:
java.io.StreamCorruptedException: invalid type code: AC
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1356)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at Client.run(BaseStaInstance.java:313)
java.io.StreamCorruptedException: invalid type code: AC
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1356)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at Client.run(BaseStaInstance.java:313)
The first time I send the exact same object message; however, when I try doing the same thing the second time, it throws the error above. Do I need to re-intialize the readObject() method? I even printed out the message object that is being received by the line below and its exact the same as the first instance where it works ok.
Object buf = myInput.readObject();
I'm assuming there's some problem with appending, but I really have no use for appending. I just want to read a fresh line everytime.
I'd really appreciate some help in fixing this bug. Thank you.
==================================
Before that one line, I'm just creating the input and output objects for the socket in the run() method. The object declaration is outside the run() method in the class:-
#Override
public void run() {
try {
sleep((int) 1 * 8000);
} catch (Exception e) {
e.printStackTrace();
}
try {
//Creating input and output streams to transfer messages to the server
myOutput = new ObjectOutputStream(skt.getOutputStream());
myInput = new ObjectInputStream(skt.getInputStream());
while (true) {
buf = myInput.readObject();
}
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
} catch (Exception e) {
e.printStackTrace();
}
}
}
You're right; I don't close the object. I'm not sure how to do that.
The underlying problem is that you are using a new ObjectOutputStream to write to a stream that you have already used a prior ObjectOutputStream to write to. These streams have headers which are written and read by the respective constructors, so if you create another ObjectOutputStream you will write a new header, which starts with - guess what? - 0xAC, and the existing ObjectInputStream isn't expecting another header at this point so it barfs.
In the Java Forums thread cited by #trashgod, I should have left out the part about 'anew for each object at both ends': that's just wasteful. Use a single OOS and OIS for the life of the socket, and don't use any other streams on the socket.
If you want to forget what you've written, use ObjectOutputStream.reset().
And don't use any other streams or Readers or Writers on the same socket. The object stream APIs can handle all Java primitive datatypes and all Serializable classes.

Error handling in java

Am running .exe file from java code using ProcessBulider, the code I have written is given below. The .exe file takes Input.txt(placed in same directory) as input and provide 3 output file in same directory.
public void ExeternalFileProcessing() throws IOException, InterruptedException {
String executableFileName = "I:/Rod/test.exe;
ProcessBuilder processBuilderObject=new ProcessBuilder(executableFileName,"Input.txt");
File absoluteDirectory = new File("I:/Rod");
processBuilderObject.directory(absoluteDirectory);
Process process = processBuilderObject.start();
process.waitFor();
}
this process is working fine by call ExeternalFileProcessing(). Now am doing validation process, If there is any crash/.exe file doesn't run, I should get the error message how can I get error message?
Note: It would be better that error message be simple like run successful/doesn't run successful or simply true/false, so that I can put this in If condition to continue the remaining process.
You can add exception handlers to get the error message.
public void externalFileProcessing() {
String executableFileName = "I:/Rod/test.exe";
ProcessBuilder processBuilderObject = new ProcessBuilder(
executableFileName, "Input.txt");
File absoluteDirectory = new File("I:/Rod");
processBuilderObject.directory(absoluteDirectory);
try {
Process process = processBuilderObject.start();
process.waitFor();
// this code will be executed if the process works
System.out.println("works");
} catch (IOException e) {
// this code will be executed if a IOException happens "e.getMessage()" will have an error
e.printStackTrace();
} catch (InterruptedException e) {
// this code will be executed if the thread is interrupted
e.printStackTrace();
}
}
But it would be better to handle it in the calling function by put a try catch handler in the calling function and handling it there.
Is it a third party .exe or do you have access to its sources? If so, you could work with basic System outputs (for example couts to the console).
Those outputs can be redirected to your java app using something like this:
InputStream is = process.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = "";
while ((line = br.readLine()) != null) {
if(line.equals("something")) {
// do something
}
}
br.close();
This is how i do things like that and it works very well in general. But i must admit, that i can not say/garuantee, that this is THE way to do it. A more advanced approach might be the use of StreamGobbler (see Listing 4.5) to handle the outputs of the .exe.
Let me know if it helped you or not.

When I write more than one object I get this IO error "invalid type code: AC" why? [duplicate]

My problem is when it tries to read the object the second time, it throws the exception:
java.io.StreamCorruptedException: invalid type code: AC
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1356)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at Client.run(BaseStaInstance.java:313)
java.io.StreamCorruptedException: invalid type code: AC
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1356)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at Client.run(BaseStaInstance.java:313)
The first time I send the exact same object message; however, when I try doing the same thing the second time, it throws the error above. Do I need to re-intialize the readObject() method? I even printed out the message object that is being received by the line below and its exact the same as the first instance where it works ok.
Object buf = myInput.readObject();
I'm assuming there's some problem with appending, but I really have no use for appending. I just want to read a fresh line everytime.
I'd really appreciate some help in fixing this bug. Thank you.
==================================
Before that one line, I'm just creating the input and output objects for the socket in the run() method. The object declaration is outside the run() method in the class:-
#Override
public void run() {
try {
sleep((int) 1 * 8000);
} catch (Exception e) {
e.printStackTrace();
}
try {
//Creating input and output streams to transfer messages to the server
myOutput = new ObjectOutputStream(skt.getOutputStream());
myInput = new ObjectInputStream(skt.getInputStream());
while (true) {
buf = myInput.readObject();
}
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
} catch (Exception e) {
e.printStackTrace();
}
}
}
You're right; I don't close the object. I'm not sure how to do that.
The underlying problem is that you are using a new ObjectOutputStream to write to a stream that you have already used a prior ObjectOutputStream to write to. These streams have headers which are written and read by the respective constructors, so if you create another ObjectOutputStream you will write a new header, which starts with - guess what? - 0xAC, and the existing ObjectInputStream isn't expecting another header at this point so it barfs.
In the Java Forums thread cited by #trashgod, I should have left out the part about 'anew for each object at both ends': that's just wasteful. Use a single OOS and OIS for the life of the socket, and don't use any other streams on the socket.
If you want to forget what you've written, use ObjectOutputStream.reset().
And don't use any other streams or Readers or Writers on the same socket. The object stream APIs can handle all Java primitive datatypes and all Serializable classes.

I just write a wrapcommand in java but sometimes it do not print the command output. WHY

I just write a simple commandwrapper in java, this is construction function:
Process process;
Thread in;
Thread out;
public CommandWrapper(Process process) {
this.process = process;
final InputStream inputStream = process.getInputStream();
// final BufferedReader
//final BufferedReader r = new BufferedReader(new InputStreamReader(inputStream));
final byte[] buffer = new byte[1024];
out = new Thread() {
// String line;
int lineNumber = 0;
public void run() {
try {
while (true) {
int count = inputStream.read(buffer);
System.out.println(lineNumber + ":"
+ new String(buffer, 0, count - 1));
// line=r.readLine();
// System.out.println(lineNumber+":"+line);
lineNumber++;
}
} catch (Exception e) {
}
}
};
final BufferedReader reader = new BufferedReader(new InputStreamReader(
System.in));
final OutputStream outputStream = process.getOutputStream();
in = new Thread() {
String line;
public void run() {
try {
//while (true) {
outputStream.write((reader.readLine() + "/n")
.getBytes());
outputStream.flush();
//}
} catch (Exception e) {
}
}
};
}
public void startIn() {
in.start();
}
This is when it invoke:
public static void main(String[] args) {
try {
CommandWrapper command = new CommandWrapper(Runtime.getRuntime()
.exec("wget www.google.com"));
//command.startIn();
command.startOut();
} catch (Exception e) {
e.printStackTrace();
}
}
It works OK when I run simple command like ls -l or other local commander, but when I want to run wget command it is print out nothing as output. I do know why.
From the code you've shown and your description of how you use it, the best guess is that an exception occurs, and you silently swallow it. This happens whenever you have an empty catch-block, like this:
catch (Exception e) {
}
You happen to have one in the run() method of your out thread.
Silently swallowing exceptions is extremely bad practice.
You should never ever ever do this! Depending on your application the appropriate solution varies, but since you're writing a console application you probably want to print the stack trace of the exception. In Java, this is done with e.printStackTrace():
catch (Exception e) {
e.printStackTrace();
}
Another option (which might not be appropriate in this specific case) is to rethrow the exception, possibly after wrapping it in another exception (for example one you've written specifically for your application):
catch (Exception e) {
throw e;
}
// or
catch (Exception e) {
throw new MyOwnException(e);
}
Doing either of these two (printing stack trace or rethrowing) will ensure that no exceptions go unnoticed.
However, no rule without exceptions ;)
There are cases when it is appropriate to have empty catch-clauses. If you know that some operation might throw an exception and you just want to proceed when it happens, an empty catch-clause is a good way to do it. However, the cases where this is appropriated are limited to (at least) the following conditions:
You must know the specific type of the exception. You never want to catch a general exception (i.e. catch (Exception e) since that might be thrown for any reason which you cannot possibly predict. If you use empty catch clauses, always catch specific exception type (such as IOException).
You must know why the exception was thrown. You should only swallow exceptions that you know the origin of. If you swallow any other exceptions, you'll end up like in this situation, where your code doesn't do what you expect and you can't understand why. Swallowed exceptions are extremely difficult to debug, since they are, well, swallowed, and thereby hidden.
You must know that you don't care about the exception. The reason to use empty catch-clauses is mainly (read: only) to handle situations where the code you're using treats something as exceptional, while you do not. By exeptional in this context we mean "something that shouldn't really happen, and if it does, something is seriously wrong."
An example of when empty catch-clauses are appropriate:
Say that you are using someone elses code that opens a file for reading, given the absolute path of the file. Most such routines throw exceptions if the file does not exist - it is the job of the client code (i.e. the code that calls the "open file routine") to ensure that the file exists before trying to open it. Exceptions will also be thrown if, for example, the user running the program does not have permissions to read the file.
Now, you might not really care why the file couldn't be opened, but if it couldn't you just want to keep going - in that case, you swallow all exceptions related to reading the file (in Java, likely an IOException of some sort). Note that you do not swallow all exceptions - only the ones related to opening the file!

file.delete() returns false even though file.exists(), file.canRead(), file.canWrite(), file.canExecute() all return true

I'm trying to delete a file, after writing something in it, with FileOutputStream. This is the code I use for writing:
private void writeContent(File file, String fileContent) {
FileOutputStream to;
try {
to = new FileOutputStream(file);
to.write(fileContent.getBytes());
to.flush();
to.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
As it is seen, I flush and close the stream, but when I try to delete, file.delete() returns false.
I checked before deletion to see if the file exists, and: file.exists(), file.canRead(), file.canWrite(), file.canExecute() all return true. Just after calling these methods I try file.delete() and returns false.
Is there anything I've done wrong?
Another bug in Java. I seldom find them, only my second in my 10 year career. This is my solution, as others have mentioned. I have nether used System.gc(). But here, in my case, it is absolutely crucial. Weird? YES!
finally
{
try
{
in.close();
in = null;
out.flush();
out.close();
out = null;
System.gc();
}
catch (IOException e)
{
logger.error(e.getMessage());
e.printStackTrace();
}
}
It was pretty odd the trick that worked. The thing is when I have previously read the content of the file, I used BufferedReader. After reading, I closed the buffer.
Meanwhile I switched and now I'm reading the content using FileInputStream. Also after finishing reading I close the stream. And now it's working.
The problem is I don't have the explanation for this.
I don't know BufferedReader and FileOutputStream to be incompatible.
I tried this simple thing and it seems to be working.
file.setWritable(true);
file.delete();
It works for me.
If this does not work try to run your Java application with sudo if on linux and as administrator when on windows. Just to make sure Java has rights to change the file properties.
Before trying to delete/rename any file, you must ensure that all the readers or writers (for ex: BufferedReader/InputStreamReader/BufferedWriter) are properly closed.
When you try to read/write your data from/to a file, the file is held by the process and not released until the program execution completes. If you want to perform the delete/rename operations before the program ends, then you must use the close() method that comes with the java.io.* classes.
As Jon Skeet commented, you should close your file in the finally {...} block, to ensure that it's always closed. And, instead of swallowing the exceptions with the e.printStackTrace, simply don't catch and add the exception to the method signature. If you can't for any reason, at least do this:
catch(IOException ex) {
throw new RuntimeException("Error processing file XYZ", ex);
}
Now, question number #2:
What if you do this:
...
to.close();
System.out.println("Please delete the file and press <enter> afterwards!");
System.in.read();
...
Would you be able to delete the file?
Also, files are flushed when they're closed. I use IOUtils.closeQuietly(...), so I use the flush method to ensure that the contents of the file are there before I try to close it (IOUtils.closeQuietly doesn't throw exceptions). Something like this:
...
try {
...
to.flush();
} catch(IOException ex) {
throw new CannotProcessFileException("whatever", ex);
} finally {
IOUtils.closeQuietly(to);
}
So I know that the contents of the file are in there. As it usually matters to me that the contents of the file are written and not if the file could be closed or not, it really doesn't matter if the file was closed or not. In your case, as it matters, I would recommend closing the file yourself and treating any exceptions according.
There is no reason you should not be able to delete this file. I would look to see who has a hold on this file. In unix/linux, you can use the lsof utility to check which process has a lock on the file. In windows, you can use process explorer.
for lsof, it's as simple as saying:
lsof /path/and/name/of/the/file
for process explorer you can use the find menu and enter the file name to show you the handle which will point you to the process locking the file.
here is some code that does what I think you need to do:
FileOutputStream to;
try {
String file = "/tmp/will_delete.txt";
to = new FileOutputStream(file );
to.write(new String("blah blah").getBytes());
to.flush();
to.close();
File f = new File(file);
System.out.print(f.delete());
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
It works fine on OS X. I haven't tested it on windows but I suspect it should work on Windows too. I will also admit seeing some unexpected behavior on Windows w.r.t. file handling.
If you are working in Eclipse IDE, that could mean that you haven't close the file in the previous launch of the application. When I had the same error message at trying to delete a file, that was the reason. It seems, Eclipse IDE doesn't close all files after termination of an application.
Hopefully this will help. I came across similar problem where i couldn't delete my file after my java code made a copy of the content to the other folder. After extensive googling, i explicitly declared every single file operation related variables and called the close() method of each file operation object, and set them to NULL. Then, there is a function called System.gc(), which will clear up the file i/o mapping (i'm not sure, i just tell what is given on the web sites).
Here is my example code:
public void start() {
File f = new File(this.archivePath + "\\" + this.currentFile.getName());
this.Copy(this.currentFile, f);
if(!this.currentFile.canWrite()){
System.out.println("Write protected file " +
this.currentFile.getAbsolutePath());
return;
}
boolean ok = this.currentFile.delete();
if(ok == false){
System.out.println("Failed to remove " + this.currentFile.getAbsolutePath());
return;
}
}
private void Copy(File source, File dest) throws IOException {
FileInputStream fin;
FileOutputStream fout;
FileChannel cin = null, cout = null;
try {
fin = new FileInputStream(source);
cin = fin.getChannel();
fout = new FileOutputStream(dest);
cout = fout.getChannel();
long size = cin.size();
MappedByteBuffer buf = cin.map(FileChannel.MapMode.READ_ONLY, 0, size);
cout.write(buf);
buf.clear();
buf = null;
cin.close();
cin = null;
fin.close();
fin = null;
cout.close();
cout = null;
fout.close();
fout = null;
System.gc();
} catch (Exception e){
this.message = e.getMessage();
e.printStackTrace();
}
}
the answer is when you load the file, you need apply the "close" method, in any line of code, works to me
There was a problem once in ruby where files in windows needed an "fsync" to actually be able to turn around and re-read the file after writing it and closing it. Maybe this is a similar manifestation (and if so, I think a windows bug, really).
None of the solutions listed here worked in my situation. My solution was to use a while loop, attempting to delete the file, with a 5 second (configurable) limit for safety.
File f = new File("/path/to/file");
int limit = 20; //Only try for 5 seconds, for safety
while(!f.delete() && limit > 0){
synchronized(this){
try {
this.wait(250); //Wait for 250 milliseconds
} catch (InterruptedException e) {
e.printStackTrace();
}
}
limit--;
}
Using the above loop worked without having to do any manual garbage collecting or setting the stream to null, etc.
The problem could be that the file is still seen as opened and locked by a program; or maybe it is a component from your program that it had been opened in, so you have to ensure you use the dispose() method to solve that problem.
i.e. JFrame frame;
....
frame.dispose();
You have to close all of the streams or use try-with-resource block
static public String head(File file) throws FileNotFoundException, UnsupportedEncodingException, IOException
{
final String readLine;
try (FileInputStream fis = new FileInputStream(file);
InputStreamReader isr = new InputStreamReader(fis, "UTF-8");
LineNumberReader lnr = new LineNumberReader(isr))
{
readLine = lnr.readLine();
}
return readLine;
}
if file.delete() is sending false then in most of the cases your Bufferedreader handle will not be closed. Just close and it seems to work for me normally.
I had the same problem on Windows. I used to read the file in scala line by line with
Source.fromFile(path).getLines()
Now I read it as a whole with
import org.apache.commons.io.FileUtils._
// encoding is null for platform default
val content=readFileToString(new File(path),null.asInstanceOf[String])
which closes the file properly after reading and now
new File(path).delete
works.
FOR Eclipse/NetBeans
Restart your IDE and run your code again this is only trick work for me after one hour long struggle.
Here is my code:
File file = new File("file-path");
if(file.exists()){
if(file.delete()){
System.out.println("Delete");
}
else{
System.out.println("not delete");
}
}
Output:
Delete
Another corner case that this could happen: if you read/write a JAR file through a URL and later try to delete the same file within the same JVM session.
File f = new File("/tmp/foo.jar");
URL j = f.toURI().toURL();
URL u = new URL("jar:" + j + "!/META-INF/MANIFEST.MF");
URLConnection c = u.openConnection();
// open a Jar entry in auto-closing manner
try (InputStream i = c.getInputStream()) {
// just read some stuff; for demonstration purposes only
byte[] first16 = new byte[16];
i.read(first16);
System.out.println(new String(first16));
}
// ...
// i is now closed, so we should be good to delete the jar; but...
System.out.println(f.delete()); // says false!
Reason is that the internal JAR file handling logic of Java, tends to cache JarFile entries:
// inner class of `JarURLConnection` that wraps the actual stream returned by `getInputStream()`
class JarURLInputStream extends FilterInputStream {
JarURLInputStream(InputStream var2) {
super(var2);
}
public void close() throws IOException {
try {
super.close();
} finally {
// if `getUseCaches()` is set, `jarFile` won't get closed!
if (!JarURLConnection.this.getUseCaches()) {
JarURLConnection.this.jarFile.close();
}
}
}
}
And each JarFile (rather, the underlying ZipFile structure) would hold a handle to the file, right from the time of construction up until close() is invoked:
public ZipFile(File file, int mode, Charset charset) throws IOException {
// ...
jzfile = open(name, mode, file.lastModified(), usemmap);
// ...
}
// ...
private static native long open(String name, int mode, long lastModified,
boolean usemmap) throws IOException;
There's a good explanation on this NetBeans issue.
Apparently there are two ways to "fix" this:
You can disable the JAR file caching - for the current URLConnection, or for all future URLConnections (globally) in the current JVM session:
URL u = new URL("jar:" + j + "!/META-INF/MANIFEST.MF");
URLConnection c = u.openConnection();
// for only c
c.setUseCaches(false);
// globally; for some reason this method is not static,
// so we still need to access it through a URLConnection instance :(
c.setDefaultUseCaches(false);
[HACK WARNING!] You can manually purge the JarFile from the cache when you are done with it. The cache manager sun.net.www.protocol.jar.JarFileFactory is package-private, but some reflection magic can get the job done for you:
class JarBridge {
static void closeJar(URL url) throws Exception {
// JarFileFactory jarFactory = JarFileFactory.getInstance();
Class<?> jarFactoryClazz = Class.forName("sun.net.www.protocol.jar.JarFileFactory");
Method getInstance = jarFactoryClazz.getMethod("getInstance");
getInstance.setAccessible(true);
Object jarFactory = getInstance.invoke(jarFactoryClazz);
// JarFile jarFile = jarFactory.get(url);
Method get = jarFactoryClazz.getMethod("get", URL.class);
get.setAccessible(true);
Object jarFile = get.invoke(jarFactory, url);
// jarFactory.close(jarFile);
Method close = jarFactoryClazz.getMethod("close", JarFile.class);
close.setAccessible(true);
//noinspection JavaReflectionInvocation
close.invoke(jarFactory, jarFile);
// jarFile.close();
((JarFile) jarFile).close();
}
}
// and in your code:
// i is now closed, so we should be good to delete the jar
JarBridge.closeJar(j);
System.out.println(f.delete()); // says true, phew.
Please note: All this is based on Java 8 codebase (1.8.0_144); they may not work with other / later versions.

Categories