We Keep Coding

EOFException from ObjectInputStream constructor

I have an app that serializes and reads/writes some custom objects in Java.
One of my clients has a particular file (only one) that is throwing a EOFException whenever the file is read into the ObjectInputStream constructor.
java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
java.io.ObjectInputStream.readStreamHeader(Unknown Source)
java.io.ObjectInputStream.(Unknown Source)
EDIT: Sorry, my mistake. I forgot to mention that I am receiving the file through this code:
File folder = new File(path);
File[] files = folder.listFiles();
So, the File does exist as far as File#listFiles() is retrieving it.
So file in the code below is received from the loop:
for(File file : files)
Thus, the IOException shouldn't be from the file being missing (because why would listFiles() return it?).
END-EDIT
I figured this may be due to a glitch from a failed-partial-write of the object, so I added code to delete the problem file if there is a EOFException:
try (InputStream is = new FileInputStream(file); ObjectInputStream ois = new ObjectInputStream(is);) {
// Do stuff...
} catch (IOException e) {
if(e instanceof EOFException) {
file.delete();
}
ErrorHandler.handleError(e);
}
Although this code executes successfully, it does not actually delete the file. (I still see the error in logs constantly). So, I opted to have my client manually search for and delete this file. He searched, found it, and deleted it. He confirmed to me that it successfully deleted the file. However, even after he manually deleted it, this error still pops up!
Although this is a Java program, my suspicion is this is a Windows file-system glitch so Java won't have much to do with this. Does anyone have experience with "ghost" files that seem to be there but aren't? Or that seem to get deleted but don't?
This is a confusing problem. Impossible for me to reproduce.

The file is empty, or doesn't contain a complete object stream header. In either event it is corrupt, and you should have detected that when you wrote it.
Probably you failed to close the ObjectOutputStream when you created the file.

FileOutputStream does not create file

I actually checked other posts that could be related to this and I couldn't find any answer to my question. So, had to create this newly:
The file does not get created in the given location with this code:
File as = new File ("C:\\Documents and Settings\\<user>\\Desktop\\demo1\\One.xls");
if (!as.exists()) {
as.createNewFile();
}
FileOutputStream fod = new FileOutputStream(as);
BufferedOutputStream dob = new BufferedOutputStream(fod);
byte[] asd = {65, 22, 123};
byte a1 = 87;
dob.write(asd);
dob.write(a1);
dob.flush();
if (dob!=null){
dob.close();
}
if(fod!=null){
fod.close();
The code runs fine and I don't get any FileNotFoundException!!
Is there anything that I'm missing out here?

You can rewrite your code like this:
BufferedOutputStream dob = null;
try {
File file = new File("C:\\Documents and Settings\\<user>\\Desktop\\demo1\\One.xls");
System.out.println("file created:" + file.exists());
FileOutputStream fod = new FileOutputStream(file);
System.out.println("file created:" + file.exists());
BufferedOutputStream dob = new BufferedOutputStream(fod);
byte[] asd = {65, 22, 123};
byte a1 = 87;
dob.write(asd);
dob.write(a1);
//dob.flush();
}
catch (Exception ex) {
ex.printStackTrace();
}
finally {
if (dob != null) {
dob.close();
}
}
In this case it is only necessary to call the topmost stream handler close() method - the BufferedOutputStream's one:
Closes this output stream and releases any system resources associated with the stream.
The close method of FilterOutputStream calls its flush method, and then calls the close method of its underlying output stream.
so, the dob.flush() in try block is commented out because the dob.close() line in the finally block flushes the stream. Also, it releases the system resources (e.g. "closes the file") as stated in the apidoc quote above. Using the finally block is a good practice:
The finally block always executes when the try block exits. This ensures that the finally block is executed even if an unexpected exception occurs. But finally is useful for more than just exception handling — it allows the programmer to avoid having cleanup code accidentally bypassed by a return, continue, or break. Putting cleanup code in a finally block is always a good practice, even when no exceptions are anticipated.
The FileOutputStream constructor creates an empty file on the disk:
Creates a file output stream to write to the file represented by the specified File object. A new FileDescriptor object is created to represent this file connection.
First, if there is a security manager, its checkWrite method is called with the path represented by the file argument as its argument.
If the file exists but is a directory rather than a regular file, does not exist but cannot be created, or cannot be opened for any other reason then a FileNotFoundException is thrown.
Where a FileDescriptor is:
Instances of the file descriptor class serve as an opaque handle to the underlying machine-specific structure representing an open file, an open socket, or another source or sink of bytes. The main practical use for a file descriptor is to create a FileInputStream or FileOutputStream to contain it.
Applications should not create their own file descriptors.
This code should either produce a file or throw an exception. You have even confirmed that no conditions for throwing exception are met, e.g. you are replacing the string and the demo1 directory exists. Please, rewrite this to a new empty file and run.
If it still behaving the same, unless I have missed something this might be a bug. In that case, add this line to the code and post output:
System.out.println(System.getProperty("java.vendor")+" "+System.getProperty("java.version"));
Judging from the path, I'd say you are using Win 7, am I right? What version?

Then it means there is a file already in your directory

reduce number of opened files in java code

Hi I have some code that uses block
RandomAccessFile file = new RandomAccessFile("some file", "rw");
FileChannel channel = file.getChannel();
// some code
String line = "some data";
ByteBuffer buf = ByteBuffer.wrap(line.getBytes());
channel.write(buf);
channel.close();
file.close();
but the specific of the application is that I have to generate large number of temporary files, more then 4000 in average (used for Hive inserts to the partitioned table).
The problem is that sometimes I catch exception
Failed with exception Too many open files
during the app running.
I wounder if there any way to tell OS that file is closed already and not used anymore, why the
channel.close();
file.close();
does not reduce the number of opened files. Is there any way to do this in Java code?
I have already increased max number of opened files in
#/etc/sysctl.conf:
kern.maxfiles=204800
kern.maxfilesperproc=200000
kern.ipc.somaxconn=8096
Update:
I tried to eliminate the problem, so I parted the code to investigate each part of it (create files, upload to hive, delete files).
Using class 'File' or 'RandomAccessFile' fails with the exception "Too many open files".
Finally I used the code:
FileOutputStream s = null;
FileChannel c = null;
try {
s = new FileOutputStream(filePath);
c = s.getChannel();
// do writes
c.write("some data");
c.force(true);
s.getFD().sync();
} catch (IOException e) {
// handle exception
} finally {
if (c != null)
c.close();
if (s != null)
s.close();
}
And this works with large amounts of files (tested on 20K with 5KB size each). The code itself does not throw exception as previous two classes.
But production code (with hive) still had the exception. And it appears that the hive connection through the JDBC is the reason of it.
I will investigate further.

The amount of open file handles that can be used by the OS is not the same thing as the number of file handles that can be opened by a process. Most unix systems restrict the number of file handles per process. Most likely it something like 1024 file handles for your JVM.
a) You need to set the ulimit in the shell that launches the JVM to some higher value. (Something like 'ulimit -n 4000')
b) You should verify that you don't have any resource leaks that are preventing your files from being 'finalized'.

Make sure to use a finally{} block. If there is an exception for some reason the close will never happen in the code as written.

Is this the exact code? Because I can think of one scenario where you might be opening all the files in a loop and written the code to close all of them in the end which is causing this problem. Please post the full code.

Do I have to close FileInputStream?

I am working as a trainee in Test Automation.
I am working with creating Junit code with Eclipse and run using Eclipse.
In that I am retriving the datas from excel sheet using FileInputStream function.
FileInputStream fi=new FileInputStream("c:\\search.xls");
Workbook w=Workbook.getWorkbook(fi);
Sheet s=w.getSheet(0);
Is it necessary to close the Inputstream function? If it so please guide me with some codings.

Yes, you need to close the inputstream if you want your system resources released back.
FileInputStream.close() is what you need.

You either need to close(), or end your program.
However you can run into confusing issues if you don't close the file as
sometimes test are run individually or a group of test are run in the same process. (So you could have a test which works one way but not the other)
you cannot rename or delete an open file.
It is best practice to always close your resources which you are finished with them, however I see unit tests as scripts which don't always have to follow best practice.

FileInputStream fi=null;
try {
fi=new FileInputStream("c:\\search.xls");
Workbook w=Workbook.getWorkbook(fi);
Sheet s=w.getSheet(0);
} finally {
if (fi!=null) {
fi.close();
}
}

Yes! you should always release the resources once after you are done with them. Java has a powerful mechanism for Garbage Collection(note that it is different thing compare to resource management/leaks.)
So a Garbage collector can not determine that if you need the resource in future or not? Failing to release resources may cause issues like- Denial of services, poor performance .
As already answered but another effort less way is try with resources
try (FileInputStream fi = new FileInputStream("c:\\search.xls")) {
//do something with fi.
//fi.getChannel() ;
} catch(IOException e) {
// exception handling.
} finally {
// some statements for finally.
}
Now you don't need to explicitly call fi.close() method.

It's always a good idea to close resources you use, BUT:
If you use resource A in resource B, it's sensible to close B instead of A if it has a method for it.
In your case, you use FileInputStream in Workbook, so you'd better to close Workbook and rely on Workbook that it will close FileInputStream.
In this particular case, actually, Workbook will close FileInputStream at the end of the getWorkbook() method but it's still a good idea to close Workbook to be able to be garbage collected.

Recently, when I tried to refactor my code, I had to move the workbook creation to another method, and the FileInputStream is created in that method. That method creates a FileInputStream and returns a Workbook. But FileInputStream is not visible from the main method; so how will I close my FileInputStream at the end of main method? The answer is, you don't have to close FileInputStream, instead you just close the workbook, which internally closes FileInputStream. In short, it is incorrect to say that you must close FileInputStream no matter what.

I do such a way to ensure to close the excel file input stream, this maybe helps
abstract int workWithWorkBook(Workbook workBook);
protected int doWorkBook(Path excelFile) throws IOException {
File f = excelFile.toFile();
try (FileInputStream excelContent = new FileInputStream(excelFile.toFile())){
POIFSFileSystem fileSystem = new POIFSFileSystem(excelContent);
Workbook workBook = null;
if (f.getName().endsWith("xls")) {
workBook = new HSSFWorkbook(fileSystem);
} else if (f.getName().endsWith("xlsx")) {
workBook = new XSSFWorkbook(excelContent);
}
return workWithWorkBook(workBook);
}catch (Exception e){
e.printStackTrace();
throw e;
}
}
9b9ea92b-5b63-47f9-a865-fd40dd602cd5

Do something like this.
FileInputStream fi=null;
try{
fi = new FileInputStream("c:\\search.xls");
Workbook w=Workbook.getWorkbook(fi);
Sheet s=w.getSheet(0);
}catch(IOException ioe){
}finally{
if(fi != null){
fi.close();
}
fi = null;//This will be hint to get finalize() called on fi so that underlying resources used will released like files opened.
}

The Workbook implements the Closeable interface which indicates that you should call the close() method or use a try with resources to free resources acquired by a Workbook object.
You can’t conclude anything on how the InputStream resource is used by the Workbook by simply observing that the Workbook has a close() method.
What really goes on is that the Workbook constructor uses the stream to initialize itself. The constructor does not keep a reference to the stream. The Workbook constructor does not close the InputStream. It is your responsibility to close the InputStream and you can do this immediately after the Workbook object is constructed. The following code is therefore OK:
private static Workbook getWorkbook() throws IOException {
try (InputStream is = ...) {
return WorkbookFactory.create(is); // newer API
}
}
try (Workbook workbook = getWorkbook()) {
Sheet sheet = workbook.getSheet("SheetName");
...
}

Basic CompSci 101 tell us to make sure to close resources that we open, in Java or any language. So yes, you need to close them. Bad juju is bound to happen when you do not do so.
Also, you should learn (and have the inclination) to use the Javadocs. Look up at the Javadoc for FileInputStream and Closeable. The answers are there.

How to find out which thread is locking a file in java?

I'm trying to delete a file that another thread within my program has previously worked with.
I'm unable to delete the file but I'm not sure how to figure out which thread may be using the file.
So how do I find out which thread is locking the file in java?

I don't have a straight answer (and I don't think there's one either, this is controlled at OS-level (native), not at JVM-level) and I also don't really see the value of the answer (you still can't close the file programmatically once you found out which thread it is), but I think you don't know yet that the inability to delete is usually caused when the file is still open. This may happen when you do not explicitly call Closeable#close() on the InputStream, OutputStream, Reader or Writer which is constructed around the File in question.
Basic demo:
public static void main(String[] args) throws Exception {
File file = new File("c:/test.txt"); // Precreate this test file first.
FileOutputStream output = new FileOutputStream(file); // This opens the file!
System.out.println(file.delete()); // false
output.close(); // This explicitly closes the file!
System.out.println(file.delete()); // true
}
In other words, ensure that throughout your entire Java IO stuff the code is properly closing the resources after use. The normal idiom is to do this in the try-with-resources statement, so that you can be certain that the resources will be freed up anyway, even in case of an IOException. E.g.
try (OutputStream output = new FileOutputStream(file)) {
// ...
}
Do it for any InputStream, OutputStream, Reader and Writer, etc whatever implements AutoCloseable, which you're opening yourself (using the new keyword).
This is technically not needed on certain implementations, such as ByteArrayOutputStream, but for the sake of clarity, just adhere the close-in-finally idiom everywhere to avoid misconceptions and refactoring-bugs.
In case you're not on Java 7 or newer yet, then use the below try-finally idiom instead.
OutputStream output = null;
try {
output = new FileOutputStream(file);
// ...
} finally {
if (output != null) try { output.close(); } catch (IOException logOrIgnore) {}
}
Hope this helps to nail down the root cause of your particular problem.

About this question, I also try to find out this answer, and ask this question and find answer:
Every time when JVM thread lock a file exclusively, also JVM lock
some Jave object, for example, I find in my case:
sun.nio.fs.NativeBuffer
sun.nio.ch.Util$BufferCache
So you need just find this locked Java object and analyzed them and
you find what thread locked your file.
I not sure that it work if file just open (without locked exclusively), but I'm sure that is work if file be locked exclusively by Thread (using java.nio.channels.FileLock, java.nio.channels.FileChannel and so on)
More info see this question