We Keep Coding

Convert DataInput to DataInputStream?

How can I convert DataInput to DataInputStream in java?
I need to know the size of the DataInput.

Since a stream, by definition, really has no begining or end and thus no fool proof way of knowing how much is available, you just have to read from the stream in fixed sized chunks. It almost sounds like you'd be better off with plain old .read() rather than readFully():
DataInputStream dis = new DataInputStream(...);
byte[] buf = new byte[1024];
int lastRead = 0;
do {
lastRead = dis.read(buf);
//do something with 'buf' here
} while (lastRead > 0);

You'll encounter difficulty when you want know how many bytes to be read. Simplest solution is to cast it to a ByteArrayInputStream and use it's available() method to get to know how many bytes are available for reading.
Following example worked for me
DataInput in = (...);
ByteArrayInputStream bis = (ByteArrayInputStream) in;
byte[] buffer = new byte[bis.available()];
in.readFully(buffer);
//use buffer as your wish

Java InputStream reading problem

I have a Java class, where I'm reading data in via an InputStream
byte[] b = null;
try {
b = new byte[in.available()];
in.read(b);
} catch (IOException e) {
e.printStackTrace();
}
It works perfectly when I run my app from the IDE (Eclipse).
But when I export my project and it's packed in a JAR, the read command doesn't read all the data. How could I fix it?
This problem mostly occurs when the InputStream is a File (~10kb).
Thanks!

Usually I prefer using a fixed size buffer when reading from input stream. As evilone pointed out, using available() as buffer size might not be a good idea because, say, if you are reading a remote resource, then you might not know the available bytes in advance. You can read the javadoc of InputStream to get more insight.
Here is the code snippet I usually use for reading input stream:
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = 0;
while ((bytesRead = in.read(buffer)) >= 0){
for (int i = 0; i < bytesRead; i++){
//Do whatever you need with the bytes here
}
}
The version of read() I'm using here will fill the given buffer as much as possible and
return number of bytes actually read. This means there is chance that your buffer may contain trailing garbage data, so it is very important to use bytes only up to bytesRead.
Note the line (bytesRead = in.read(buffer)) >= 0, there is nothing in the InputStream spec saying that read() cannot read 0 bytes. You may need to handle the case when read() reads 0 bytes as special case depending on your case. For local file I never experienced such case; however, when reading remote resources, I actually seen read() reads 0 bytes constantly resulting the above code into an infinite loop. I solved the infinite loop problem by counting the number of times I read 0 bytes, when the counter exceed a threshold I will throw exception. You may not encounter this problem, but just keep this in mind :)
I probably will stay away from creating new byte array for each read for performance reasons.

read() will return -1 when the InputStream is depleted. There is also a version of read which takes an array, this allows you to do chunked reads. It returns the number of bytes actually read or -1 when at the end of the InputStream. Combine this with a dynamic buffer such as ByteArrayOutputStream to get the following:
InputStream in = ...
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int read;
byte[] input = new byte[4096];
while ( -1 != ( read = in.read( input ) ) ) {
buffer.write( input, 0, read );
}
input = buffer.toByteArray()
This cuts down a lot on the number of methods you have to invoke and allows the ByteArrayOutputStream to grow its internal buffer faster.

File file = new File("/path/to/file");
try {
InputStream is = new FileInputStream(file);
byte[] bytes = IOUtils.toByteArray(is);
System.out.println("Byte array size: " + bytes.length);
} catch (IOException e) {
e.printStackTrace();
}

Below is a snippet of code that downloads a file (*. Png, *. Jpeg, *. Gif, ...) and write it in BufferedOutputStream that represents the HttpServletResponse.
BufferedInputStream inputStream = bo.getBufferedInputStream(imageFile);
try {
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int bytesRead = 0;
byte[] input = new byte[DefaultBufferSizeIndicator.getDefaultBufferSize()];
while (-1 != (bytesRead = inputStream.read(input))) {
buffer.write(input, 0, bytesRead);
}
input = buffer.toByteArray();
response.reset();
response.setBufferSize(DefaultBufferSizeIndicator.getDefaultBufferSize());
response.setContentType(mimeType);
// Here's the secret. Content-Length should equal the number of bytes read.
response.setHeader("Content-Length", String.valueOf(buffer.size()));
response.setHeader("Content-Disposition", "inline; filename=\"" + imageFile.getName() + "\"");
BufferedOutputStream outputStream = new BufferedOutputStream(response.getOutputStream(), DefaultBufferSizeIndicator.getDefaultBufferSize());
try {
outputStream.write(input, 0, buffer.size());
} finally {
ImageBO.close(outputStream);
}
} finally {
ImageBO.close(inputStream);
}
Hope this helps.

my code hangs but when i debug it it never hangs

i have a java code that SOMETIMES hangs when i run it.Its a put command that puts through sockets( and input stream etc a file from a server to a client) . Sometimes this works but sometimes it doesnt and i get an error message. i tried every possible path that can be taken through debugging and i can never get it to hang. Is there a way to inspect the thing when its hang through eclipse?`
if (sentence.length() > 3 && sentence.substring(0, 3).equals("put")) {
File checkFile = new File(dir.getCurrentPath(), sentence.substring(4));
if (checkFile.isFile() && checkFile.exists()) {
try {
outToServer.writeBytes(sentence + "\n");
boolean cont = false;
String x;
while (!cont) {
if ((x = inFromServer.readLine()).equals("continue")) {
cont = true;
}
}
String name = sentence.substring(4);
copy.copyFile(name);
// outToServer.writeBytes("continue" + "\n");
this is the client code that recieves the PUT request(i.e. put test.txt takes the file test.txt and puts it in the server's local dir.
Copy file: (the thing that copies the data)
File checkFile = new File(dir.getCurrentPath(), file);
if (checkFile.isFile() && checkFile.exists()) {
DataOutputStream outToClient = new DataOutputStream(socket.getOutputStream());
// byte[] receivedData = new byte[8192];
File inputFile = new File(dir.getCurrentPath(), file);
byte[] receivedData = new byte[(int) inputFile.length()];
// String theLength = "" + inputFile.length();
outToClient.writeBytes("" + inputFile.length() + "\n");
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(getCopyPath(file)));
// if (extension.equals("")) {
// extension = "txt";
// }
// BufferedReader inFromClient = new BufferedReader(new InputStreamReader(socket.getInputStream()));
// boolean cont = false;
// while (!cont) {
// if (inFromClient.readLine().equals("continue")) {
// cont = true;
//
// }
// }
// outToClient.writeBytes("continue" + "\n");
bis.read(receivedData, 0, receivedData.length);
OutputStream os = socket.getOutputStream();
os.write(receivedData, 0, receivedData.length);
// outToClient.writeBytes("finish" + "\n");
os.flush();
Protocol on server(copies stuff in the file)
if (get.equals("put")) {
//so the client sends: the put request
//then sends the length
try {
BufferedReader inFromClient = new BufferedReader(new InputStreamReader(socket.getInputStream()));
DataOutputStream outToClient = new DataOutputStream(socket.getOutputStream());
outToClient.writeBytes("continue" + "\n");
int length = Integer.parseInt(inFromClient.readLine());
// String x = "";
// boolean cont = false;
// while (!cont) {
// String temp = inFromClient.readLine();
// if (temp.equals("finish")) {
// cont = true;
// }
// else {
// x += temp;
// }
// }
byte[] recieveFile = new byte[length];
InputStream is = socket.getInputStream();
FileOutputStream fos = new FileOutputStream("Copy " + input.substring(4));
BufferedOutputStream bos = new BufferedOutputStream(fos);
int bytesRead;
int current = 0;
bytesRead = is.read(recieveFile, 0, recieveFile.length);
current = bytesRead;
If you cant answer the question in specific just tell me how i can debug the code that hangs or how i can debug concurrent code.(by the way the way that the signal passing is done is by passing a token as you can see i.e. the server sents a continue token that tells the client to start sending data, i havent done this with threads methods notify and wait since i cant use , since any single object has only 1 method.

when your process hangs, get the current stack dump for the process. this will show you why the process is hung.
note, you have at least one bug in your program, in that you are not handling the return value from the InputStream.read() method which will, at the very least, cause you to have busted data on the server side. (see #rk2010's answer for more details).
you main bug, though, is that when you wrap the BufferedReader around the socket InputStream, you are probably going to end up "stealing" more bytes from the stream than the just length value. when a BufferedReader reads data from the underlying stream, it can read more data than it actually returns in the readLine() method (so, it may read 1000 chars internally, but the first "line" may only contain 20 chars). if you continue to use the BufferedReader, everything is fine, but if you discard the BufferedReader and try to read more data from the underlying stream, you will have less data there than you expect. so, when you go to read the file contents, there aren't enough bytes available.
instead, you should be using DataOutputStream/DataInputStream exclusively. write the length as a long value (i.e. DataOutputStream.writeLong(long)), then write the bytes after (i.e. DataOutputStream.write(byte[])). then read the data using the corresponding methods in DataInputStream. since you can use the DataInputStream exclusively when reading (first to read the file size and then to read the actual file bytes), you don't risk losing bytes when switching read "modes" (additionally, DataInputStream does not do any internal buffering like BufferedReader). don't use Readers/Writers at all.

add logging after every read and write command. Give each thread a name.
run the program, see how the log matches up with what you expect.
note that you can't always rely on read method like the way you are. Safe way to read is by looping until you get a negative length.
int len = -1;
byte[] buff = new byte[1024]; // for 4KB, use: 4* 1024
while ( (len = is.read(buff, 0, buff.length)) > -1){
// copy len number of bytes from buff array into some other place.
}
Check out IOUtils.copy method
from Apache Commons IOUtils class

read a file byte by byte then perform some operation every n bytes

I would like to know how can I read a file byte by byte then perform some operation every n bytes.
for example:
Say I have a file of size = 50 bytes, I want to divide it into blocks each of n bytes. Then each block is sent to a function for some operations to be done on those bytes. The blocks are to be created during the read process and sent to the function when the block reaches n bytes so that I don`t use much memory for storing all blocks.
I want the output of the function to be written/appended on a new file.
This is what I've reached to read, yet I don't know it it is right:
fc = new JFileChooser();
File f = fc.getSelectedFile();
FileInputStream in = new FileInputStream(f);
byte[] b = new byte[16];
in.read(b);
I haven't done anything yet for the write process.

You're on the right lines. Consider wrapping your FileInputStream with a BufferedInputStream, which improve I/O efficiency by reading the file in chunks.
The next step is to check the number of bytes read (returned by your call to read) and to hand-off the array to the processing function. Obviously you'll need to pass the number of bytes read to this method too in case the array was only partially populated.

So far your code looks OK. For reading binary files (as opposed to text files) you should indeed use FileInputStream (for reading text files, you should use a Reader, such as FileReader).
Note that you should check the return value from in.read(b);, because it might read less than 16 bytes if there are less than 16 bytes left at the end of the file.
Ofcourse you should add a loop to the program that keeps reading blocks of bytes until you reach the end of the file.
To write data to a binary file, use FileOutputStream. That class has a constructor that you can pass a flag to indicate that you want to append to an existing file:
FileOutputStream out = new FileOutputStream("output.bin", true);
Also, don't forget to call close() on the FileInputStream and FileOutputStream when you are done.
See the Java API documentation, especially the classes in the java.io package.

I believe that this will work:
final int blockSize = // some calculation
byte[] block = new byte[blockSize];
InputStream is = new FileInputStream(f);
try {
int ret = -1;
do {
int bytesRead = 0;
while (bytesRead < blockSize) {
ret = is.read(block, bytesRead, blockSize - bytesRead);
if (ret < 0)
break; // no more data
bytesRead += ret;
}
myFunction(block, bytesRead);
} while (0 <= ret);
}
finally {
is.close();
}
This code will call myFunction with blockSize bytes for all but possibly the last invocation.

It's a start.
You should check what read() returns. It can read fewer bytes than the size of the array, and also indicate that the end of the file is reached.
Obviously, you need to read() in a loop...
It might be a good idea to reuse the array, but that requires that the part that reads the array copies what it needs, rather than just keeping a reference to the array.

I think this is what you migth need
void readFile(String path, int n) {
try {
File f = new File(path);
FileInputStream fis = new FileInputStream(f);
int ret = 0;
byte[] array = new byte[n];
while(ret > -1) {
ret = fis.read(array);
doSomething(array, ret);
}
fis.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}

Determine the size of an InputStream

My current situation is: I have to read a file and put the contents into InputStream. Afterwards I need to place the contents of the InputStream into a byte array which requires (as far as I know) the size of the InputStream. Any ideas?
As requested, I will show the input stream that I am creating from an uploaded file
InputStream uploadedStream = null;
FileItemFactory factory = new DiskFileItemFactory();
ServletFileUpload upload = new ServletFileUpload(factory);
java.util.List items = upload.parseRequest(request);
java.util.Iterator iter = items.iterator();
while (iter.hasNext()) {
FileItem item = (FileItem) iter.next();
if (!item.isFormField()) {
uploadedStream = item.getInputStream();
//CHANGE uploadedStreambyte = item.get()
}
}
The request is a HttpServletRequest object, which is like the FileItemFactory and ServletFileUpload is from the Apache Commons FileUpload package.

This is a REALLY old thread, but it was still the first thing to pop up when I googled the issue. So I just wanted to add this:
InputStream inputStream = conn.getInputStream();
int length = inputStream.available();
Worked for me. And MUCH simpler than the other answers here.
Warning This solution does not provide reliable results regarding the total size of a stream. Except from the JavaDoc:
Note that while some implementations of {#code InputStream} will return
* the total number of bytes in the stream, many will not.

I would read into a ByteArrayOutputStream and then call toByteArray() to get the resultant byte array. You don't need to define the size in advance (although it's possibly an optimisation if you know it. In many cases you won't)

You can't determine the amount of data in a stream without reading it; you can, however, ask for the size of a file:
http://java.sun.com/javase/6/docs/api/java/io/File.html#length()
If that isn't possible, you can write the bytes you read from the input stream to a ByteArrayOutputStream which will grow as required.

I just wanted to add, Apache Commons IO has stream support utilities to perform the copy. (Btw, what do you mean by placing the file into an inputstream? Can you show us your code?)
Edit:
Okay, what do you want to do with the contents of the item?
There is an item.get() which returns the entire thing in a byte array.
Edit2
item.getSize() will return the uploaded file size.

For InputStream
org.apache.commons.io.IoUtils.toByteArray(inputStream).length()
For Optional < MultipartFile >
Stream.of(multipartFile.get()).mapToLong(file->file.getSize()).findFirst().getAsLong()

you can get the size of InputStream using getBytes(inputStream) of Utils.java check this following link
Get Bytes from Inputstream

The function below should work with any InputStream. As other answers have hinted, you can't reliably find the length of an InputStream without reading through it, but unlike other answers, you should not attempt to hold the entire stream in memory by reading into a ByteArrayOutputStream, nor is there any reason to. Instead of reading the stream, you should ideally rely on other API for stream sizes, for example getting the size of a file using the File API.
public static int length(InputStream inputStream, int chunkSize) throws IOException {
byte[] buffer = new byte[chunkSize];
int chunkBytesRead = 0;
int length = 0;
while((chunkBytesRead = inputStream.read(buffer)) != -1) {
length += chunkBytesRead;
}
return length;
}
Choose a reasonable value for chunkSize appropriate to the kind of InputStream. E.g. reading from disk it would not be efficient to have too small a value for chunkSize.

When explicitly dealing with a ByteArrayInputStream then contrary to some of the comments on this page you can use the .available() function to get the size. Just have to do it before you start reading from it.
From the JavaDocs:
Returns the number of remaining bytes that can be read (or skipped
over) from this input stream. The value returned is count - pos, which
is the number of bytes remaining to be read from the input buffer.
https://docs.oracle.com/javase/7/docs/api/java/io/ByteArrayInputStream.html#available()

If you need to stream the data to another object that doesn't allow you to directly determine the size (e.g. javax.imageio.ImageIO), then you can wrap your InputStream within a CountingInputStream (Apache Commons IO), and then read the size:
CountingInputStream countingInputStream = new CountingInputStream(inputStream);
// ... process the whole stream ...
int size = countingInputStream.getCount();

If you know that your InputStream is a FileInputStream or a ByteArrayInputStream, you can use a little reflection to get at the stream size without reading the entire contents. Here's an example method:
static long getInputLength(InputStream inputStream) {
try {
if (inputStream instanceof FilterInputStream) {
FilterInputStream filtered = (FilterInputStream)inputStream;
Field field = FilterInputStream.class.getDeclaredField("in");
field.setAccessible(true);
InputStream internal = (InputStream) field.get(filtered);
return getInputLength(internal);
} else if (inputStream instanceof ByteArrayInputStream) {
ByteArrayInputStream wrapper = (ByteArrayInputStream)inputStream;
Field field = ByteArrayInputStream.class.getDeclaredField("buf");
field.setAccessible(true);
byte[] buffer = (byte[])field.get(wrapper);
return buffer.length;
} else if (inputStream instanceof FileInputStream) {
FileInputStream fileStream = (FileInputStream)inputStream;
return fileStream.getChannel().size();
}
} catch (NoSuchFieldException | IllegalAccessException | IOException exception) {
// Ignore all errors and just return -1.
}
return -1;
}
This could be extended to support additional input streams, I am sure.

Add to your pom.xml:
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.5</version>
</dependency>
Use to get the content type lenght (InputStream file):
IOUtils.toByteArray(file).length

Use this method, you just have to pass the InputStream
public String readIt(InputStream is) {
if (is != null) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is, "utf-8"), 8);
StringBuilder sb = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
sb.append(line).append("\n");
}
is.close();
return sb.toString();
}
return "error: ";
}

try {
InputStream connInputStream = connection.getInputStream();
} catch (IOException e) {
e.printStackTrace();
}
int size = connInputStream.available();
int available ()
Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream. The next invocation might be the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.
InputStream - Android SDK | Android Developers