We Keep Coding

Creating mp4 file doesn't remove tmp files

I'm trying to write an InputStream that is an mp4 that I get from calling an external SOAP service, when I do so, it always generates this tmp files for my chosen temporary directory(java.io.tmpdir) that aren't removable and stay after the writing is done.
Writing images that I also get from the SOAP service works normal without the permanent tmp on the directory. I'm using java 1.8 SpringBoot
tmp files
This is what I'm doing:
File targetFile = new File("D:/archive/video.mp4");
targetFile.getParentFile().mkdirs();
targetFile.setWritable(true);
InputStream inputStream = filesToWrite.getInputStream();
OutputStream outputStream = new FileOutputStream(targetFile);
try {
int byteRead;
while ((byteRead = inputStream.read()) != -1) {
outputStream.write(byteRead);
}
} catch (IOException e) {
logger.fatal("Error# SaveFilesThread for guid: " + guid, e);
}finally {
try {
inputStream.close();
outputStream.flush();
outputStream.close();
}catch (Exception e){
e.printStackTrace();
}
also tried:
byte data[] = IOUtils.toByteArray(inputStream);
Path file = Paths.get("video.mp4");
Files.write(file, data);
And from apache commons IO:
FileUtils.copyInputStreamToFile(initialStream, targetFile);

When your code starts, the damage is already done. Your code is not the source of the temporary files (It's.. a ton of work for something that could be done so much simpler, though, see below), it's the framework that ends up handing you that filesToWrite variable.
It is somewhat likely that you can hook in at an earlier point and get the raw inputstream representing the socket or HTTP connection, and start saving the files straight from there. Alternatively, Perhaps filesToWrite has a way to get at the files themselves, in which case you can just move them into place instead of copying them over.
But, your code to do this is a mess, it has bad exception handling, and leaks memory, and is way too much code for a simple job, and is possibly 2000x to 10000x slower than needed depending on your harddisk (I'm not exaggerating, calling single-byte read() on unbuffered streams is thousands of times slower!)
// add `throws IOException` to your method signature.
// it saves files, it's supposed to throw IOException,
// 'doing I/O' is in the very definition of your method!
try (InputStream in = filesToWrite.getInputStream();
OutputStream out = new FileOutputStream(targetFile)) {
in.transferTo(out);
}
That's it. That solves all the problems - no leaks, no speed loss, tiny amount of code, fixes the deplorable error handling (which, here, is 'log something to the log, then print something to standard out, then potentially leak a bunch of resources, then don't tell the calling code anything went wrong and return exactly as if the copy operation succeeded).

Download .class file and load it into JVM

How can I download .class file and and load it into jvm using class loader , I have write a simple code simulates downloading a .class file the I tried to load it into JVM
public class face {
public static void main(String[] args) throws IOException,
ClassNotFoundException {
File f = new File("Task.class");
int count;
byte[] buffer = new byte[1024];
DataInputStream dis = new DataInputStream(new FileInputStream(f));
StringBuilder all = new StringBuilder();
while ((count = dis.read(buffer)) > 0) {
// System.out.write(buffer, 0, count);
all.append(buffer);
// System.out.flush();
}
File b = new File("Task.class");
FileOutputStream fos = new FileOutputStream(b);
DataOutputStream dos = new DataOutputStream(fos);
dos.write(all.toString().getBytes());
ClassLoader lod = face.class.getClassLoader();
lod.loadClass(b.getAbsolutePath());
}
}

Use Class.forName(<package_qualified_class_name>)

First, I would like to applogies for the long list of suggestion here, but you have managed to cram an impressive number of mistakes into a small piece of code.
I suggest don't do any of these things
don't use DataInputStream or DataOutputStream when it doesn't add anything. You don't use any method which requires it.
don't write binary data to a StringBuilder. A StringBuilder is for text.
don't copy an entire buffer if you only read part of it. i.e. you need to record the length actual read and copy only the amount used.
don't append a byte[] to a StringBuilder. It won't do what you expect.
don't use a String to store binary data.
don't convert a String to byte[] using the default encoding unless you know you have ASCII data (which you don't)
don't write to a file you just read. As this doesn't make sense. You should have tested this works without the file copy and you would have found this didn't work, before you attempted something more complicated.
you can't write to a file which you still have open in windows. I suggest you close() a file when you are finished with it.
don't attempt to load a class using the file name. You load it by package.class name.
I suggest you try a one liner to load a class first and show this works. The class should appear in your class path, and when you write to the file, you should write it to a directory appropriate for the package.
Instead of doing all this, you could add a http://yourserver/basepath to your class path and it will load the classes from a web service. i.e. you might be able to do this without writing any code at all.

How can I access a non-class file inside a jar FROM 'within' the jar

This should be simple but it has cost me hours. Everything I find on this site indicates I am doing it right but the file still cannot be found.
Inside a jar file I have two files 'CDAkeystore.jks' and 'CDAtruststore.jks' at top level.
Yet when I call
securityProps.setProperty("javax.net.ssl.keyStore","CDAkeystore.jks");
I get a system cannot find the file requested error.
The class file calling this method is inside the same jar in the usual package arrangement.
The jar file is as follows:
com ..... (a lot more class files)
org ..... (lots of class files)
META-INF
CDAtruststore.jks
CDAkeystore.jks
How can this be SOOO difficult?!!
---------- Added INfo ------n
Since the object using the path is open source I found the routine they are using to load the file. It is:
InputStream keystoreInputStream = preBufferInputStream(new FileInputStream(keyStoreName));
which according to the documentation of FileInputStream(String name) is
Creates a FileInputStream by opening a connection to an actual file, the file named by the path name 'name' in the file system. So how should this path be expressed?

Use YourClass.class.getResourceAsStream() or this.getClass().getResourceAsStream(). You can also use class loader if you are in multiple class loaders environment.

The answer is, in short, that you can't. At least in this situation. I am stuck with passing a path to a file to a library implementation that I have no control over. So the library method accesses the file on the assumption that the file exists in unzipped form in the OS's file system. It is getting the path from a Property.setProperty(stringKey, stringPath)
So the only solution I found was an ugly hack. I need to take the resource in my jar and copy it to a file on the system. Then I would pass the path to that file in the above setProperty() method. The ugly hack is implemented as follows (if anyone else can come up with a nicer solution I would be happy). It does solve the problem. The library routine is able to find my newly created file.
/* This evil reads a file as a resource inside a jar and dumps the file where ever
* the loader of this jar/application defines as the current directory. This pain is done
* so the SecurityDomian class can load the file; it cannot access the file from
* the jar. This means the 'name' passed in contains the file name inside the jar
* prefixed with "/" so it is not read with an assumed package extension.
*/
private boolean createFileFromResource(String name)
{
// Dont bother if the file already exists
if(new File(name.replace("/", "")).exists())
{
return true;
}
byte[] keystoreFile = new byte[2048];
ByteArrayOutputStream byteArrayOut = new ByteArrayOutputStream(2048);
// Get the resource
InputStream inputStream = this.getClass().getResourceAsStream(name);
try
{
int bytesRead = 0;
while(true)
{
// Read the resource into the buffer keystoreFile in 2048 byte chunks
bytesRead = inputStream.read(keystoreFile);
if(bytesRead < 0)
{
break;
}
// Copy and append the chunks to the ByteArrayOutputStream (this class
// does the auto-extending of the output array as more chunks are
// added so you don't have to do it.
byteArrayOut.write(keystoreFile, 0, bytesRead);
}
inputStream.close();
// Now create a file at the root of where ever the loader happens to think
// the root is. So remove the "/" in front of the file name
FileOutputStream outputStream = new FileOutputStream(name.replace("/", ""));
// Write out the file. Note you will be left with garbage at that location.
byteArrayOut.writeTo(outputStream);
outputStream.flush();
outputStream.close();
}
catch (IOException e)
{
e.printStackTrace();
return false;
}
return true;
}

What is the fastest way to extract 1 file from a zip file which contain a lot of file?

I tried the java.util.zip package, it is too slow.
Then I found LZMA SDK and 7z jbinding but they are also lacking something.
The LZMA SDK does not provide a kind of documentation/tutorial of how-to-use, it is very frustrating. No javadoc.
While the 7z jbinding does not provide a simple way to extract only 1 file, however, it only provide way to extract all the content of the zip file. Moreover, it does not provide a way to specify a location to place the unzipped file.
Any idea please?

What does your code with java.util.zip look like and how big of a zip file are you dealing with?
I'm able to extract a 4MB entry out of a 200MB zip file with 1,800 entries in roughly a second with this:
OutputStream out = new FileOutputStream("your.file");
FileInputStream fin = new FileInputStream("your.zip");
BufferedInputStream bin = new BufferedInputStream(fin);
ZipInputStream zin = new ZipInputStream(bin);
ZipEntry ze = null;
while ((ze = zin.getNextEntry()) != null) {
if (ze.getName().equals("your.file")) {
byte[] buffer = new byte[8192];
int len;
while ((len = zin.read(buffer)) != -1) {
out.write(buffer, 0, len);
}
out.close();
break;
}
}

I have not benchmarked the speed but with java 7 or greater, I extract a file as follows.
I would imagine that it's faster than the ZipFile API:
A short example extracting META-INF/MANIFEST.MF from a zip file test.zip:
// file to extract from zip file
String file = "MANIFEST.MF";
// location to extract the file to
File outputLocation = new File("D:/temp/", file);
// path to the zip file
Path zipFile = Paths.get("D:/temp/test.zip");
// load zip file as filesystem
try (FileSystem fileSystem = FileSystems.newFileSystem(zipFile)) {
// copy file from zip file to output location
Path source = fileSystem.getPath("META-INF/" + file);
Files.copy(source, outputLocation.toPath());
}

Use a ZipFile rather than a ZipInputStream.
Although the documentation does not indicate this (it's in the docs for JarFile), it should use random-access file operations to read the file. Since a ZIPfile contains a directory at a known location, this means a LOT less IO has to happen to find a particular file.
Some caveats: to the best of my knowledge, the Sun implementation uses a memory-mapped file. This means that your virtual address space has to be large enough to hold the file as well as everything else in your JVM. Which may be a problem for a 32-bit server. On the other hand, it may be smart enough to avoid memory-mapping on 32-bit, or memory-map just the directory; I haven't tried.
Also, if you're using multiple files, be sure to use a try/finally to ensure that the file is closed after use.

How should I load files into my Java application?

How should I load files into my Java application?

The short answer
Use one of these two methods:
Class.getResource(String)
Class.getResourceAsStream(String)
For example:
InputStream inputStream = YourClass.class.getResourceAsStream("image.jpg");
--
The long answer
Typically, one would not want to load files using absolute paths. For example, don’t do this if you can help it:
File file = new File("C:\\Users\\Joe\\image.jpg");
This technique is not recommended for at least two reasons. First, it creates a dependency on a particular operating system, which prevents the application from easily moving to another operating system. One of Java’s main benefits is the ability to run the same bytecode on many different platforms. Using an absolute path like this makes the code much less portable.
Second, depending on the relative location of the file, this technique might create an external dependency and limit the application’s mobility. If the file exists outside the application’s current directory, this creates an external dependency and one would have to be aware of the dependency in order to move the application to another machine (error prone).
Instead, use the getResource() methods in the Class class. This makes the application much more portable. It can be moved to different platforms, machines, or directories and still function correctly.

getResource is fine, but using relative paths will work just as well too, as long as you can control where your working directory is (which you usually can).
Furthermore the platform dependence regarding the separator character can be gotten around using File.separator, File.separatorChar, or System.getProperty("file.separator").

What are you loading the files for - configuration or data (like an input file) or as a resource?
If as a resource, follow the suggestion and example given by Will and Justin
If configuration, then you can use a ResourceBundle or Spring (if your configuration is more complex).
If you need to read a file in order to process the data inside, this code snippet may help BufferedReader file = new BufferedReader(new FileReader(filename)) and then read each line of the file using file.readLine(); Don't forget to close the file.

I haven't had a problem just using Unix-style path separators, even on Windows (though it is good practice to check File.separatorChar).
The technique of using ClassLoader.getResource() is best for read-only resources that are going to be loaded from JAR files. Sometimes, you can programmatically determine the application directory, which is useful for admin-configurable files or server applications. (Of course, user-editable files should be stored somewhere in the System.getProperty("user.home") directory.)

public byte[] loadBinaryFile (String name) {
try {
DataInputStream dis = new DataInputStream(new FileInputStream(name));
byte[] theBytes = new byte[dis.available()];
dis.read(theBytes, 0, dis.available());
dis.close();
return theBytes;
} catch (IOException ex) {
}
return null;
} // ()

use docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/ClassLoader.html#getResource(java.lang.String)

public static String loadTextFile(File f) {
try {
BufferedReader r = new BufferedReader(new FileReader(f));
StringWriter w = new StringWriter();
try {
String line = reader.readLine();
while (null != line) {
w.append(line).append("\n");
line = r.readLine();
}
return w.toString();
} finally {
r.close();
w.close();
}
} catch (Exception ex) {
ex.printStackTrace();
return "";
}
}