We Keep Coding

Creating mp4 file doesn't remove tmp files

I'm trying to write an InputStream that is an mp4 that I get from calling an external SOAP service, when I do so, it always generates this tmp files for my chosen temporary directory(java.io.tmpdir) that aren't removable and stay after the writing is done.
Writing images that I also get from the SOAP service works normal without the permanent tmp on the directory. I'm using java 1.8 SpringBoot
tmp files
This is what I'm doing:
File targetFile = new File("D:/archive/video.mp4");
targetFile.getParentFile().mkdirs();
targetFile.setWritable(true);
InputStream inputStream = filesToWrite.getInputStream();
OutputStream outputStream = new FileOutputStream(targetFile);
try {
int byteRead;
while ((byteRead = inputStream.read()) != -1) {
outputStream.write(byteRead);
}
} catch (IOException e) {
logger.fatal("Error# SaveFilesThread for guid: " + guid, e);
}finally {
try {
inputStream.close();
outputStream.flush();
outputStream.close();
}catch (Exception e){
e.printStackTrace();
}
also tried:
byte data[] = IOUtils.toByteArray(inputStream);
Path file = Paths.get("video.mp4");
Files.write(file, data);
And from apache commons IO:
FileUtils.copyInputStreamToFile(initialStream, targetFile);

When your code starts, the damage is already done. Your code is not the source of the temporary files (It's.. a ton of work for something that could be done so much simpler, though, see below), it's the framework that ends up handing you that filesToWrite variable.
It is somewhat likely that you can hook in at an earlier point and get the raw inputstream representing the socket or HTTP connection, and start saving the files straight from there. Alternatively, Perhaps filesToWrite has a way to get at the files themselves, in which case you can just move them into place instead of copying them over.
But, your code to do this is a mess, it has bad exception handling, and leaks memory, and is way too much code for a simple job, and is possibly 2000x to 10000x slower than needed depending on your harddisk (I'm not exaggerating, calling single-byte read() on unbuffered streams is thousands of times slower!)
// add `throws IOException` to your method signature.
// it saves files, it's supposed to throw IOException,
// 'doing I/O' is in the very definition of your method!
try (InputStream in = filesToWrite.getInputStream();
OutputStream out = new FileOutputStream(targetFile)) {
in.transferTo(out);
}
That's it. That solves all the problems - no leaks, no speed loss, tiny amount of code, fixes the deplorable error handling (which, here, is 'log something to the log, then print something to standard out, then potentially leak a bunch of resources, then don't tell the calling code anything went wrong and return exactly as if the copy operation succeeded).

Do I need to close a stream?

Do I need to close a FileOutputStream in the following example? And why?
FileOutputStream fos = new FileOutputStream("bytes.info");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(data);
oos.close();

If there were no exceptions thrown, then the FileOutputStream would be closed by ooo.close().
An exception thrown in writeObject would prevent any of the streams from being closed. So the close call should be in a finally block.
There's the additional problem that the ObjectOutputStream could throw an exception in its constructor. It writes the stream header in the constructor which can cause an exception. In this case, the FileOutputStream needs to be closed, but calling oos.close() is not possible because there's no reference to the ObjectOutputStream. So you really need two separate calls to close, one for each stream, both in finally blocks.
Using try-with-resources takes care of all of this for you:
try(
FileOutputStream fos = new FileOutputStream("bytes.info");
ObjectOutputStream oos = new ObjectOutputStream(fos)
) {
oos.writeObject(data);
}

Yes, you need to close the stream. Leaving FileOutputStream unclosed creates a possibility that some data that has been successfully written to the stream does not get saved to the file. If a program opens multiple file streams, not closing them creates a possibility of failures due to running out of native resources (too many files opened simultaneously).
FileOutputStream manages native resources, which are released by the close method. The class has a finalizer, too, which releases resources as well. As part of releasing native resources, the stream finishes out the writing of buffered data, if there is any. However, since JVM does not guarantee that a finalizer is going to be called on every object, failure to call close creates a risk of leaving buffered data unwritten.

Of course you have to close the FileOutputStream file. If not sometimes the data you save into the file might not get saved and you will end up with an empty file after executing the program. And you might wanna use try with resources so you won't have to close it manually and can do the exception handling part both at once.
try (FileOutputStream fos = new FileOutputStream("bytes.info");
ObjectOutputStream oos = new ObjectOutputStream(fos));{
}catch(){}
Just in case if you are not using try with resources close the file streams in the finally block manually.
FileOutputStream fos = null;
ObjectOutputStream oos = null;
try{
fos = new FileOutputStream("bytes.info");
oos = new ObjectOutputStream(fos));
oos.writeObject(data);
}catch(){
}finally{
if(fos != null){
fos.close();
}
if(oos != null){
oos.close();
}
}
It is a must to check whether those file streams are null or not. Because if they are null then there will be another error. Still it's better to use try with resources.

How to download monthly Treasury Files

Up till early this year the US Treasury web site posted monthly US Receipts and Outlays data in txt format. It was easy to write a program to read and store the info. All I use were:
URL url = new URL("https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts1214.txt")
URLConnection connection.openConnection();
InputStream is = connection.getInputStream();
Then I just read the InputStream into a local file.
Now when I try same code, for May, I get an InputStream with nothing in it.
Just clicking on "https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts0415.xlsx" opens an excel worksheet (the download path has since changed).
Which is great if you don't mind clicking on each link separately ... saving the file somewhere ... opening it manually to enable editing ... then saving it again as a real .xlsx file (because they really hand you an .xls file.)
But when I create a URL from that link, and use it to get an InputStream, the is empty. I also tried url.openStream() directly. No different.
Can anyone see a way I can resume using a program to read the new format?
In case its of interest I now use this code to write the stream to the file bit by bit... but there are no bits, so I don't know if it works.
static void copyInputStreamToFile( InputStream in, File file ) {
try {
OutputStream out = new FileOutputStream(file);
byte[] buf = new byte[1024];
System.out.println("reading: " + in.read(buf));
//This is what tells me it is empty, i.e. the loop below is ignored.
int len;
while((len=in.read(buf))>0){
out.write(buf,0,len);
}
out.close();
in.close();
} catch (Exception e) {
e.printStackTrace();
}
}
Any help is appreciated.

Java5 FileInputStream fails before I can get file lock

I have application (Java5) distributed among several JVMs that reads/writes files from a shared storage (the storage is managed by Windows). I wanted to use exclusive/shared locks for file write/read as following:
FileOutputStream fos = null;
FileLock lock = null;
ObjectOutputStream oos = null;
try {
fos = new FileOutputStream(new File("//share/test.dat")); // HERE IT MAY THROW FileNotFoundException...
lock = fos.getChannel().lock(); // ... and I won't acquire a lock.
oos = new ObjectOutputStream(fos);
oos.writeObject(value);
return true;
} catch (Exception e) {
// Log it.
} finally {
// Close locks and I/O streams.
}
The problem: if JVM1 is currently reading the file test.dat and JVM2 is trying to write to the same file then FileNotFoundException ("The process cannot access the file because it is being used by another process") will be thrown on JVM2 in "new FileOutputStream(new File("//share/test.dat"))" line.
It seems to be Catch22 case: on the one hand I want to acquire a lock to get I/O stream; on the other hand I need to have an I/O stream to acquire a lock.
As I could see, the same situation with RandomAccessFile.
Any ideas?..

As you can see from the error message, Windows just won't let you do what you're trying to do. You can treat the Windows error itself as the lock condition, if you think about it. You would have a cognate problem here anyway because of the timing window between the open and the lock. Windows in a way is doing you a favor here. For once.

Client-Server File Transfer in Java

I'm looking for an efficient way to transfer files between client and server processes using TCP in Java. My server code looks something like this:
socket = serverSocket.accept();
InputStream is = socket.getInputStream();
OutputStream os = socket.getOutputStream();
FileInputStream fis = new FileInputStream(new File(filename));
I'm just unsure of how to proceed. I know I want to read bytes from fis and then write them to os, but I'm unsure about the best way to read and write bytes using byte streams in Java. I'm only familiar with writing/reading text using Writers and Readers. Can anyone tell me the appropriate way to do this? What should I wrap os and fis in (if anything) and how do I keep reading bytes until the end of file without a hasNext() method (or equivalent)

You could do something like:
byte[] contents = new byte[BUFFER_SIZE];
int numBytes =0;
while((numBytes = is.read(contents))>0){
os.write(contents,0,numBytes);
}

You could use Apache's IOUtils.copy(in, out) or
import org.apache.commons.fileupload.util.Streams;
...
Streams.copy(in, out, false);
Inspecting the source might prove interesting. ( http://koders.com ?)
There is the java.nio.Channel with a transferTo method, with mixed opinions in the community wether better for smaller/larger files.
A simple block wise copy between Input/OutputStream would be okay. You could wrap it in buffered streams.