I am planning a function that creates and returns an InputStream that in turn reads from another InputStream because the initialization of that InputStream is not trivial and I would like to use it in multiple places. Consider this simple example:
private static InputStream openStream() throws IOException {
Path path = Paths.get("/etc/passwd");
InputStream inputStream = Files.newInputStream(path);
return new BufferedInputStream(inputStream);
}
I will use this function as follows:
public static void main(String[] args) {
try (InputStream stream = openStream()) {
byte[] buffer = new byte[1024];
int numBytes;
while ((numBytes = stream.read(buffer, 0, buffer.length)) > 0) {
System.out.printf("Just read %d bytes from stream!%n", numBytes);
}
} catch (IOException e) {
e.printStackTrace();
}
}
However, I am concerned that closing the BufferedInputStream in this example will not close the InputStream inside it. Will this lead to orphaned file handles and memory leaks if called multiple times? What is a better solution for this?
A simple solution I could think of is to define a closable container class and put both input streams into that class. When calling close(), this class would simply close all its open handles.
class StreamContainer implements Closeable {
private final InputStream[] inputStreams;
public StreamContainer(InputStream... inputStreams) {
this.inputStreams = inputStreams;
}
#Override
public void close() throws IOException {
for (InputStream inputStream : this.inputStreams) {
inputStream.close();
}
}
}
But I suppose, there might be a better solution, built-in mechanic or development pattern. Or maybe these constructs should be avoided?
In this cases you should read the code source of the BufferedInputStream, this is the close definition
public void close() throws IOException {
while(true) {
byte[] buffer;
if ((buffer = this.buf) != null) {
if (!U.compareAndSetObject(this, BUF_OFFSET, buffer, (Object)null)) {
continue;
}
InputStream input = this.in;
this.in = null;
if (input != null) {
input.close();
}
return;
}
return;
}
}
As you can see when closing the BufferedInputStream, the underlying input stream is closed as well.
And this is the documentation of close:
public void close() throws IOException
Closes this input stream and releases any system resources associated with the stream. Once the
stream has been closed, further read(), available(), reset(), or
skip() invocations will throw an IOException. Closing a previously
closed stream has no effect.
Related
I am using a BufferedWriter to write in a File. All the nested streams of the BufferedWriter (FileOutputStream, GzipOutputStream, OuputStreamWriter, ...) should be automatically closed if I close the BufferedWriter. If all of the streams have been instantiated then closing only the outermost is just fine.
The documentation on Closeable interface states that close method:
Closes this stream and releases any system resources associated with
it.
So, I am wondering why I cannot find a out.close(); in the source code of the java.io.BufferedWriter :
// Java 11.0.12_7 source code. Line 262
public void close() throws IOException {
synchronized (lock) {
if (out == null) {
return;
}
try (Writer w = out) {
flushBuffer();
// Why don't we have a out.close() here ?
} finally {
out = null;
cb = null;
}
}
}
How the out Writer is it closed ?
I'm new to Netty. There is a problem about file transfer confusing me for days. I want to send image file from client to server.
The code below is executable. But only I shutdown server forcibly can I open received image file normally. Otherwise, it shows "It looks like you don't have permission to view this file. Check the permissions and try again". So I want to close fileoutputstream when there is no data in ByteBuf using ByteBuf.isReadable(), but the else block in method channelRead in ServerHandler never reach. It's useless.
Besides, if sending text file, it can be open normally when server is alive. I don't want to shutdown server every time after transfer. Please give me some suggestions to solve it.
This is FileClientHandler
public class FileClientHandler extends ChannelInboundHandlerAdapter {
private int readLength = 8;
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
sendFile(ctx.channel());
}
private void sendFile(Channel channel) throws IOException {
File file = new File("C:\\Users\\xxx\\Desktop\\1.png");
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
for (;;) {
byte[] bytes = new byte[readLength];
int readNum = bis.read(bytes, 0, readLength);
// System.out.println(readNum);
if (readNum == -1) {
bis.close();
fis.close();
return;
}
sendToServer(bytes, channel, readNum);
}
}
private void sendToServer(byte[] bytes, Channel channel, int length)
throws IOException {
channel.writeAndFlush(Unpooled.copiedBuffer(bytes, 0, length));
}
}
This is FileServerHandler
public class FileServerHandler extends ChannelInboundHandlerAdapter {
private File file = new File("C:\\Users\\xxx\\Desktop\\2.png");
private FileOutputStream fos;
public FileServerHandler() {
try {
if (!file.exists()) {
file.createNewFile();
} else {
file.delete();
file.createNewFile();
}
fos = new FileOutputStream(file);
} catch (IOException e) {
e.printStackTrace();
}
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg)
throws Exception {
try {
ByteBuf buf = (ByteBuf) msg;
if (buf.isReadable()) {
buf.readBytes(fos, buf.readableBytes());
fos.flush();
} else {
System.out.println("I want to close fileoutputstream!");
buf.release();
fos.flush();
fos.close();
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
Fixing the server side
In the Netty world, there are multiple "events":
channelActive
channelRead
channelReadComplete
channelInactive
exceptionCaught
more...
Of these "events", you probably already knows what channelRead does (since your using it), but another one that you seem to need is the channelInactive. This one is called when the other endpoint shuts the connection down, and you can use it like this:
#Override
public void channelInactive(ctx) {
System.out.println("I want to close fileoutputstream!");
fos.close();
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg)
throws Exception {
try {
ByteBuf buf = (ByteBuf) msg;
// if (buf.isReadable()) { // a buf should always be readable here
buf.readBytes(fos, buf.readableBytes());
// fos.flush(); // flushing is always done when closing
//} else {
// System.out.println("I want to close fileoutputstream!");
// buf.release(); // Should be placed in the finally block
// fos.flush();
// fos.close();
//}
} catch (Exception e) {
e.printStackTrace();
} finally {
buf.release(); // Should always be done, even if writing to the file fails
}
}
However, how does the server know the connection has shutdown? At the moment the client does not shut the server down, and instead keep running in the background forever keeping the connection alive.
Fixing the client side
To properly shutdown the connection from the client, we need to call channel.close(), however, we cannot directly insert this before the return line, as this causes a race condition between sending the data, and closing the connection in the network layer, potentially dropping data.
To properly handle these conditions, Netty uses a Future system that allows code to handle events after asynchronous actions happen.
Lucky for us, Netty already has a build in solution for this, we only need to wire it up. To wire this solution up to our code, we have to keep track of the latest ChannelFuture emitted by Netty's write method.
To properly implement this solution, we change sendToServer to return the result of the write method:
private ChannelFuture sendToServer(byte[] bytes, Channel channel, int length)
throws IOException {
return channel.writeAndFlush(Unpooled.copiedBuffer(bytes, 0, length));
}
Then we keep keep track of this return value, and add a listener containing Netty's build in solution when we want to close the connection:
ChannelFuture lastFuture = null;
for (;;) {
byte[] bytes = new byte[readLength];
int readNum = bis.read(bytes, 0, readLength);
// System.out.println(readNum);
if (readNum == -1) {
bis.close();
fis.close();
if(lastFuture == null) { // When our file is 0 bytes long, this is true
channel.close();
} else {
lastFuture.addListener(ChannelFutureListener.CLOSE);
}
return;
}
lastFuture = sendToServer(bytes, channel, readNum);
}
JavaDoc for InputStreamReader doesn't say anything about closing the underlying InputStream:
https://docs.oracle.com/javase/8/docs/api/java/io/InputStreamReader.html#close--
Description copied from class: Reader
Closes the stream and releases any system resources associated with it. Once the stream has been closed, further read(), ready(), mark(), reset(), or skip() invocations will throw an IOException. Closing a previously closed stream has no effect.
Does closing an InputStreamReader also close the underlying InputStream?
UPDATE In:
InputStreamReader istream = new InputStreamReader(conn.getInputStream(), "UTF-8")
istream.close();
Do I need to close conn.getInputStream()?
InputStreamReader implementation direct close call to StreamDecoder which is a native class.
As other answers and comments said, the answer is yes, it does close the InputStream. You can see for yourself with the following code:
InputStream is = new FileInputStream("D:\\a.txt");
Reader r = new InputStreamReader(is);
r.close();
is.read(); // throws exception: stream is closed.
Therefore, if you close the Reader, you don't need to also close the InputStream. However, I guess you are using try-with-resources everywhere (aren't you? ;) ) and the InputStream as well as the Reader will both be closed at the end of the try block. That doesn't matter, because an InputStream can be closed multiple times; it's a no-op if the stream is already closed.
If you want to avoid closing the InputStream, you can write a simple wrapper that does nothing when it is closed:
class UncloseableInputStream extends FilterInputStream {
public UncloseableInputStream(InputStream is) {
super(is);
}
public void close() {
// Do nothing.
}
}
InputStream is = new FileInputStream("D:\\a.txt");
Reader r = new InputStreamReader(new UncloseableInputStream(is));
r.close();
is.read(); // still works despite closing the reader.
It depends on stream implementation. InputStream is just an "interface" in terms of close(). InputStreamReader will not close an interface. It will close the underlying data resource (like file descriptor) if it is. It will do nothing if close is override and empty in an implementation.
In OpenJdk StreamDecoder has a method
void implClose() throws IOException {
if(this.ch != null) {
this.ch.close();
} else {
this.in.close();
}
}
this.in is a InputStream from decoder constructor:
StreamDecoder(InputStream var1, Object var2, CharsetDecoder var3) {
...
if(this.ch == null) {
this.in = var1;
...
}
...
}
Here are examples of closing actions. ByteArrayInputStream:
Closing a ByteArrayInputStream has no effect. The methods in this class can be called after the stream has been closed without generating an IOException.
public void close() throws IOException {
}
FileInputStream differes:
Closes this file input stream and releases any system resources associated with the stream. If this stream has an associated channel then the channel is closed as well. After you closed the underlying instance it doesn't matter which interfaces were using it, it will be closed.
public void close() throws IOException {
synchronized (closeLock) {
if (closed) {
return;
}
closed = true;
}
if (channel != null) {
channel.close();
}
fd.closeAll(new Closeable() {
public void close() throws IOException {
close0();
}
});
}
I'm experiencing java.lang.OutOfMemoryError: Java heap space whenever I try to execute my code. However, if I close my streams in certain instances the error goes away, but because my streams are closing prematurely I'm missing data.
I'm very new to Java and I'm clearly not understanding how to manage the streams. How and when should I close streams?
private void handleFile(File source)
{
FileInputStream fis = null;
try
{
if(source.isFile())
{
fis = new FileInputStream(source);
handleFile(source.getAbsolutePath(), fis);
}
else if(source.isDirectory())
{
for(File file:source.listFiles())
{
if(file.isFile())
{
fis = new FileInputStream(file);
handleFile(file, fis);
}
else
{
handleFile(file);
}
}
}
}
catch(IOException ioe)
{
ioe.printStackTrace();
}
finally
{
try
{
if(fis != null) { fis.close(); }
}
catch(IOException ioe) { ioe.printStackTrace(); }
}
}
private handleFile(String fileName, InputStream inputStream)
{
try
{
byte[] initialBytes = isToByteArray(inputStream);
byte[] finalBytes = initialBytes;
if(initialBytes.length == 0) return;
if(isBytesTypeB(initialBytes))
{
finalBytes = getBytesTypeB(startingBytes);
}
// Other similar method checks
// .....
map.put(fileName, finalBytes);
}
catch(IOException ioe)
{
ioe.printStackTrace();
}
}
private byte[] isToByteArray(InputStream inputStream)
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int nRead;
while((nRead = inputStream.read(buffer)) != -1)
{
baos.write(buffer, 0, nRead);
}
return baos.toByteArray();
}
private boolean isBytesTypeB(byte[] fileBytes)
{
// Checks if these bytes match a particular type
if(BytesMatcher.matches(fileBytes, fileBytes.length))
{
return true;
}
return false;
}
private byte[] getBytesTypeB(byte[] fileBytes)
{
//decompress bytes
return decompressedBytes;
}
First of all, do not read the entire streams in memory. Use buffers when reading and writing.
Use ByteArrayInputStream and ByteArrayInputStream only if you're sure you'll be reading very small streams (whose data you will need to re-use for some operations) and it really makes sense to keep the data in memory. Otherwise, you will quickly (or unexpectedly) run out of memory.
Define the streams outside a try-catch block and close them in the finally block (if they are not null). For example:
void doSomeIOStuff() throws IOException
{
InputStream is = null;
try
{
is = new MyInputStream(...);
// Do stuff
}
catch (IOException ioExc)
{
// Either just inform (poor decision, but good for illustration):
ioExc.printStackTrace();
// Or re-throw to delegate further on:
throw new IOException(ioExc);
}
finally
{
if (is != null)
{
is.close();
}
}
}
This way your resources are always properly closed after use.
Out of curiosity, what should the handleFile(...) method really be doing?
While running debugger, the program pauses on initializing object streams from server main input output streams. Following is the code :
public TFileReader(Client cli)throws Exception{
this.cli = cli;
fileSock = new Socket(cli.ServerIp(), cli.FilePort());
fobjIn = new ObjectInputStream(fileSock.getInputStream());
fobjOut = new ObjectOutputStream(fileSock.getOutputStream());
fobjOut.flush();
}
#Override
public void run(){
try{
System.out.println("file reader thread online");
fobjOut.writeObject(cli.Name());
fobjOut.flush();
String per = (String) fobjIn.readObject();
System.out.println(per+"video filing...");
if(!per.equals("OKF"))
{
throw new Exception("Error In retriving video.");
}
It pauses on fobjIn and do not go to execute fobjOut although fobjIn it passes from fobjIn breakpoint but do not hit out breakpoint.
I would keep it simple like this
public TFileReader(Client cli) throws IOException {
this.cli = cli;
socket = new Socket(cli.ServerIp(), cli.FilePort());
out = new ObjectOutputStream(socket.getOutputStream());
out.flush();
in = new ObjectInputStream(socket.getInputStream());
}
public void writeObject(Object o) throw IOException {
out.writeObject(o);
out.reset();
out.flush();
}
public <T> T readObject() throw IOException {
return (T) in.readObject();
}
public void close() throws IOException {
in.close();
out.close();
socket.close();
}
The problem is that ObjectInputStream pre-reads data on initialization.
The preferred solution is at Java Creating a new ObjectInputStream Blocks: always initialize your ObjectOutputStream before initializing your ObjectInputStream, so that the "handshake" that the two use internally can be initiated.
When you don't control all the code and cannot change the order, consider delaying the OIS initialization until data is available (InputStream.available or mark/read/reset on a buffered stream wrapping it, etc).