We Keep Coding

Do I need to delete tmp files created by my java application?

I output several temporary files in my application to tmp directories but was wondering if it is best practise that I delete them on close or should I expect the host OS to handle this for me?
I am pretty new to Java, I can handle the delete but want to keep the application as multi-OS and Linux friendly as possible. I have tried to minimise file deletion if I don't need to do it.
This is the method I am using to output the tmp file:
try {
java.io.InputStream iss = getClass().getResourceAsStream("/nullpdf.pdf");
byte[] data = IOUtils.toByteArray(iss);
iss.read(data);
iss.close();
String tempFile = "file";
File temp = File.createTempFile(tempFile, ".pdf");
FileOutputStream fos = new FileOutputStream(temp);
fos.write(data);
fos.flush();
fos.close();
nopathbrain = temp.getAbsolutePath();
System.out.println(tempFile);
System.out.println(nopathbrain);
} catch (IOException ex) {
ex.printStackTrace();
System.out.println("TEMP FILE NOT CREATED - ERROR ");
}

createTempFile() only creates a new file with a unique name, but does not mark it for deletion. Use deleteOnExit() on the created file to achieve that. Then, if the JVM shuts down properly, the temporary files should be deleted.
edit:
Sample for creating a 'true' temporary file in java:
File temp = File.createTempFile("temporary-", ".pdf");
temp.deleteOnExit();
This will create a file in the default temporary folder with a unique random name (temporary-{randomness}.pdf) and delete it when the JVM exits.
This should be sufficient for programs with a short to medium run time (e.g. scripts, simple GUI applications) that do sth. and then exit. If the program runs longer or indefinitely (server application, a monitoring client, ...) and the JVM won't exit, this method may clog the temporary folder with files. In such a situation the temporary files should be deleted by the application, as soon as they are not needed anymore (see delete() or Files helper class in JDK7).
As Java already abstracts away OS specific file system details, both approaches are as portable as Java. To ensure interoperability have a look at the new Path abstraction for file names in Java7.

Spawned zip process doesn't seem to add file to archive

I am trying to add a text file to a zip archive through a Java program on Linux. The program spawns a process (using java.lang.Process) to execute the commandline "zip -j .zip .txt", reads the output and error streams of the spawned process and waits for the process to complete using waitFor(). Though the program seems to run fine (spawned process exits with exit code 0, indicating that the zip commandline was executed successfully) and the output read from output and error streams do not indicate any errors, at the end of the program the zip archive doesn't always contain the file supposed to have been added. This problem doesn't happen consistently though (even with the same existing-archive and file-to-add) - once in a while (perhaps once in 4 attempts) the zip is found to have been updated correctly. Strangely, the problem doesn't occur at all when the program is run through Eclipse debugger mode. Any pointers on why this problem occurs and how it can be addressed would be helpful. Thanks!
Below is the code snippet. The program calls addFileToZip(File, File, String):
public static void addFileToZip(final File zipFile, final File fileToBeAdded,
final String fileNameToBeAddedAs) throws Exception {
File tempDir = createTempDir();
File fileToBeAddedAs = new File(tempDir, fileNameToBeAddedAs);
try {
FileUtils.copyFile(fileToBeAdded, fileToBeAddedAs);
addFileToZip(zipFile, fileToBeAddedAs);
} finally {
deleteFile(fileToBeAddedAs);
deleteFile(tempDir);
}
}
public static void addFileToZip(final File zipFile, final File fileToBeAdded) throws Exception {
final String[] command = {"zip", "-j", zipFile.getAbsolutePath(), fileToBeAdded.getAbsolutePath()};
ProcessBuilder procBuilder = new ProcessBuilder(command);
Process proc = procBuilder.start();
int exitCode = proc.waitFor();
/*
* Code to read output/error streams of proc and log/print them
*/
if (exitCode != 0) {
throw new Exception("Unable to add file, error: " + errMsg);
}
}

Make sure no other process has the zip file locked for write, or the file being added locked for read. If you're generating the file to be added, make sure the stream is flushed and closed before spawning the zip utility.

I am trying to add a text file to a zip archive through a Java program on Linux.
Use the java.util.zip API, which:
Provides classes for reading and writing the standard ZIP and GZIP file formats.
If you intend to stick with using a Process to do this, be sure to implement all the suggestions of When Runtime.exec() won't.

Hosting an executable within Android application

I am working on an Android application that depends on an ELF binary:
our Java code interacts with this binary to get things done. This
runtime needs to be started and terminated on Application startup and
application exit / on demand.
Questions:
I am assuming that we will be able to execute this binary using the
Runtime.exec() API. Is there any constraints as to where I
need to be putting my library in the folder structure? How would the system runtime locate this executable? Is there some sort of class path setting?
Since the application has dependencies on this Runtime, I was
thinking of wrapping it around a service so that it can be started or
stopped as required. What is the best way to handle such executables
in Android project?
What are other alternatives, assuming that I do not have source code for this executable?
Please advice.
Thanks.

1) No, there should be no constrains, besides those that access system files and thus require root. The best place would be straight to /data/data/[your_package_name] to avoid polluting elsewhere.
2) A very thorough discussion about compiling against native libraries can be found here: http://www.aton.com/android-native-libraries-for-java-applications/ . Another option is a cross-compiler for arm (here is the one used to compile the kernel, it's free: http://www.codesourcery.com/sgpp/lite/arm ). If you plan to maintain a service that executes your cammand, be warned that services can be stopped and restarted by android at any moment.
3) Now, if you don't have the source code, I hope that your file is at least compiled as an arm executable. If not, I don't see how you could even run it.
You will execute the file by running the following commands in your java class:
String myExec = "/data/data/APPNAME/FILENAME";
Process process = Runtime.getRuntime().exec(myExec);
DataOutputStream os = new DataOutputStream(process.getOutputStream());
DataInputStream osRes = new DataInputStream(process.getInputStream());
I know nothing about your executable, so you may or may not need to actually get the inputStream and outputStream.
I am assuming that running adb to push the binary file is out of the question, so
I was looking for a neat way to package it. I found a great post about including an executable in your app. Check it out here:
http://gimite.net/en/index.php?Run%20native%20executable%20in%20Android%20App
The important part is this one (emphasis mine):
From Android Java app, using assets folder
Include the binary in the assets folder.
Use getAssets().open(FILENAME) to get an InputStream.
Write it to /data/data/APPNAME (e.g. /data/data/net.gimite.nativeexe), where your application has access to write files and make it executable.
Run /system/bin/chmod 744 /data/data/APPNAME/FILENAME using the code above.
Run your executable using the code above.
The post uses the assets folder, insted of the raw folder that android suggests for static files:
Tip: If you want to save a static file in your application at compile time, save the file in your project res/raw/ directory. You can open it with openRawResource(), passing the R.raw. resource ID. This method returns an InputStream that you can use to read the file (but you cannot write to the original file).
To access the data folder, you can follow the instructions here:
http://developer.android.com/guide/topics/data/data-storage.html#filesInternal
Also, there's the File#setExecutable(boolean); method that should works instead of the shell command.
So, putting everything together, I would try:
InputStream ins = context.getResources().openRawResource (R.raw.FILENAME)
byte[] buffer = new byte[ins.available()];
ins.read(buffer);
ins.close();
FileOutputStream fos = context.openFileOutput(FILENAME, Context.MODE_PRIVATE);
fos.write(buffer);
fos.close();
File file = context.getFileStreamPath (FILENAME);
file.setExecutable(true);
Of course, all this should be done only once after installation. You can have a quick check inside onCreate() or whatever that checks for the presence of the file and executes all this commands if the file is not there.
Let me know if it works. Good luck!

Here is a complete guide for how to package and run the executable. I based it on what I found here and other links, as well as my own trial and error.
1.) In your SDK project, put the executable file in your /assets folder
2.) Programmatically get the String of that files directory (/data/data/your_app_name/files) like this
String appFileDirectory = getFilesDir().getPath();
String executableFilePath = appFileDirectory + "/executable_file";
3.) In your app's project Java code: copy the executable file from /assets folder into your app's "files" subfolder (usually /data/data/your_app_name/files) with a function like this:
private void copyAssets(String filename) {
AssetManager assetManager = getAssets();
InputStream in = null;
OutputStream out = null;
Log.d(TAG, "Attempting to copy this file: " + filename); // + " to: " + assetCopyDestination);
try {
in = assetManager.open(filename);
Log.d(TAG, "outDir: " + appFileDirectory);
File outFile = new File(appFileDirectory, filename);
out = new FileOutputStream(outFile);
copyFile(in, out);
in.close();
in = null;
out.flush();
out.close();
out = null;
} catch(IOException e) {
Log.e(TAG, "Failed to copy asset file: " + filename, e);
}
Log.d(TAG, "Copy success: " + filename);
}
4.) Change the file permissions on executable_file to actually make it executable. Do it with Java calls:
File execFile = new File(executableFilePath);
execFile.setExecutable(true);
5.) Execute the file like this:
Process process = Runtime.getRuntime().exec(executableFilePath);
Note that any files referred to here (such as input and output files) must have their full path Strings constructed. This is because this is a separate spawned process and it has no concept of what the "pwd" is.
If you want to read the command's stdout you can do this, but so far it's only working for me for system commands (like "ls"), not the executable file:
BufferedReader reader = new BufferedReader(
new InputStreamReader(process.getInputStream()));
int read;
char[] buffer = new char[4096];
StringBuffer output = new StringBuffer();
while ((read = reader.read(buffer)) > 0) {
output.append(buffer, 0, read);
}
reader.close();
process.waitFor();
Log.d(TAG, "output: " + output.toString());

For executing binary file starting from Android 10 it's only possible from read-only folder. It means that you should pack binary with your app. Android doc
Put android:extractNativeLibs="true" into AndroidManifest;
Put your binary to src/main/resources/lib/* directory, where * – stands for architecture of CPU, for instance armeabi-v7a;
Use code like this for executing:
private fun exec(command: String, params: String): String {
try {
val process = ProcessBuilder()
.directory(File(filesDir.parentFile!!, "lib"))
.command(command, params)
.redirectErrorStream(true)
.start()
val reader = BufferedReader(
InputStreamReader(process.inputStream)
)
val text = reader.readText()
reader.close()
process.waitFor()
return text
} catch (e: Exception) {
return e.message ?: "IOException"
}
}
Here is discussion with answer from android team on reddit.

I've done something like this using the NDK. My strategy was to recompile the program using the NDK and write some wrapper JNI code that called into the program's main function.
I'm not sure what the lifecycle of NDK code is like. Even services that are intended to be long-running can be started and stopped by the system when convenient. You would probably have to shutdown your NDK thread and restart it when necessary.

Unable to access Java-created file -- sometimes

In Java, I'm working with code running under WinXP that creates a file like this:
public synchronized void store(Properties props, byte[] data) {
try {
File file = filenameBasedOnProperties(props);
if ( file.exists() ) {
return;
}
File temp = File.createTempFile("tempfile", null);
FileOutputStream out = new FileOutputStream(temp);
out.write(data);
out.flush();
out.close();
file.getParentFile().mkdirs();
temp.renameTo(file);
}
catch (IOException ex) {
// Complain and whine and stuff
}
}
Sometimes, when a file is created this way, it's just about totally inaccessible from outside the code (though the code responsible for opening and reading the file has no problem), even when the application isn't running. When accessed via Windows Explorer, I can't move, rename, delete, or even open the file. Under Cygwin, I get the following when I ls -l the directory:
ls: cannot access [big-honkin-filename]
total 0
?????????? ? ? ? ? ? [big-honkin-filename]
As implied, the filenames are big, but under the 260-character max for XP (though they are slightly over 200 characters).
To further add to the sense that my computer just wants me to feel stupid, sometimes the files created by this code are perfectly normal. The only pattern I've spotted is that once one file in the directory "locks", the rest are screwed.
Anybody ever run into something like this before, or have any insights into what's going on here?

Make sure you always close the stream in a finally block. In your case if an exception is thrown the stream might not get closed and will leak a file handle. You could use procexp from SysInternals to see which process holds the handle to the file.

Although, by definition, NTFS should handle path length up to 2^15-1, in practice the length of paths is limited to 255.
You can create files with a longer path name (filename including parent folder names), but you cannot access them afterwards. The error I get in these cases is that the file could not be found. To get rid of these files, I have to shorten the names of parent folders, until the path length is short enough.

decompress .gz file in batch

I have 100 of .gz files which I need to de-compress.
I have couple of questions
a) I am using the code given at http://www.roseindia.net/java/beginners/JavaUncompress.shtml to decompress the .gz file. Its working fine.
Quest:- is there a way to get the file name of the zipped file. I know that Zip class of Java gives of enumeration of entery file to work upon. This can give me the filename, size etc stored in .zip file. But, do we have the same for .gz files or does the file name is same as filename.gz with .gz removed.
b) is there another elegant way to decompress .gz file by calling the utility function in the java code. Like calling 7-zip application from your java class. Then, I don't have to worry about input/output stream.
Thanks in advance.
Kapil

a) Zip is an archive format, while gzip is not. So an entry iterator does not make much sense unless (for example) your gz-files are compressed tar files. What you want is probably:
File outFile = new File(infile.getParent(), infile.getName().replaceAll("\\.gz$", ""));
b) Do you only want to uncompress the files? If not you may be ok with using GZIPInputStream and read the files directly, i.e. without intermediate decompression.
But ok. Let's say you really only want to uncompress the files. If so, you could probably use this:
public static File unGzip(File infile, boolean deleteGzipfileOnSuccess) throws IOException {
GZIPInputStream gin = new GZIPInputStream(new FileInputStream(infile));
FileOutputStream fos = null;
try {
File outFile = new File(infile.getParent(), infile.getName().replaceAll("\\.gz$", ""));
fos = new FileOutputStream(outFile);
byte[] buf = new byte[100000];
int len;
while ((len = gin.read(buf)) > 0) {
fos.write(buf, 0, len);
}
fos.close();
if (deleteGzipfileOnSuccess) {
infile.delete();
}
return outFile;
} finally {
if (gin != null) {
gin.close();
}
if (fos != null) {
fos.close();
}
}
}

Regarding A, the gunzip command creates an uncompressed file with the original name minus the .gz suffix. See the man page.
Regarding B, Do you need gunzip specifically, or will another compression algorithm do? There's a java port of the LZMA compression algorithm used by 7zip to create .7z files, but it will not handle .gz files.

If you have a fixed number of files to decompress once, why don't you use existing tools for that?
As Paul Morie noticed, gunzip can do that:
for i in *.gz; do gunzip $i; done
And it would automatically name them, stripping .gz$
On windows, try winrar, probably, or gunzip from http://unxutils.sf.net

GZip is normally used only on single files, so it generally does not contain information about individual files. To bundle multiple files into one compressed archive, they are first combined into an uncompressed Tar file (with info about individual contents), and then compressed as a single file. This combination is called a Tarball.
There are libraries to extract the individual file info from a Tar, just as with ZipEntries. One example. You will first have to extract the .gz file into a temporary file in order to use it, or at least feed the GZipInputStream into the Tar library.
You may also call 7-Zip from the command line using Java. 7-Zip command-line syntax is here: 7-Zip Command Line Syntax. Example of calling the command shell from Java: Executing shell commands in Java. You will have to call 7-Zip twice: once to extract the Tar from the .tar.gz or .tgz file, and again to extract the individual files from the Tar.
Or, you could just do the easy thing and write a brief shell script or batch file to do your decompression. There's no reason to hammer a square peg in a round hole -- this is what batch files are made for. As a bonus, you can also feed them parameters, reducing the complexity of a java command line execution considerably, while still letting java control execution.

Have you tried
gunzip *.gz

.gz files (gzipped) can store the filename of a compressed file. So for example FuBar.doc can be saved inside myDocument.gz and with appropriate uncompression, the file can be restored to the filename FuBar.doc. Unfortunately, java.util.zip.GZIPInputStream does not support any way of reading the filename even if it is stored inside the archive.