This question already has answers here:
Easy way to write contents of a Java InputStream to an OutputStream
(24 answers)
Closed 3 years ago.
FileInputStream in = new FileInputStream(myFile);
ByteArrayOutputStream out = new ByteArrayOutputStream();
Question: How can I read everything from in into out in a way which is not a hand-crafted loop with my own byte buffer?
Java 9 (and later) answer (docs):
in.transferTo(out);
Seems they finally realized that this functionality is so commonly needed that it’d better be built in. The method returns the number of bytes copied in case you need to know.
Write one method to do this, and call it from everywhere which needs the functionality. Guava already has code for this, in ByteStreams.copy. I'm sure just about any other library with "general" IO functionality has it too, but Guava's my first "go-to" library where possible. It rocks :)
In Apache Commons / IO, you can do it using IOUtils.copy(in, out):
InputStream in = new FileInputStream(myFile);
OutputStream out = new ByteArrayOutputStream();
IOUtils.copy(in, out);
But I agree with Jon Skeet, I'd rather use Guava's ByteStreams.copy(in, out)
So what Guava's ByteStreams.copy(in, out) does:
private static final int BUF_SIZE = 0x1000; // 4K
public static long copy(InputStream from, OutputStream to)
throws IOException {
checkNotNull(from);
checkNotNull(to);
byte[] buf = new byte[BUF_SIZE];
long total = 0;
while (true) {
int r = from.read(buf);
if (r == -1) {
break;
}
to.write(buf, 0, r);
total += r;
}
return total;
}
In my project I used this method:
private static void copyData(InputStream in, OutputStream out) throws Exception {
byte[] buffer = new byte[8 * 1024];
int len;
while ((len = in.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
}
Alternatively to Guava one could use Apache Commons IO (old), and Apache Commons IOUtils (new as advised in the comment).
I'd use the loop, instead of importing new classes, or adding libraries to my project. The library function is probably also implemented with a loop. But that's just my personal taste.
However, my question to you: what are you trying to do? Think of the "big picture", if you want to put the entire contents of a file into a byte array, why not just do that? The size of the arrray is file.length(), and you don't need it to dynamically grow, hidden behind a ByteArrayOutputStream (unless your file is shared, and its contents can change while you read).
Another alternative: could you use a FileChannel and a ByteBuffer (java.nio)?
Related
I have a piece of code which uses the deflate algorithm to compress a file:
public static File compressOld(File rawFile) throws IOException
{
File compressed = new File(rawFile.getCanonicalPath().split("\\.")[0]
+ "_compressed." + rawFile.getName().split("\\.")[1]);
InputStream inputStream = new FileInputStream(rawFile);
OutputStream compressedWriter = new DeflaterOutputStream(new FileOutputStream(compressed));
byte[] buffer = new byte[1000];
int length;
while ((length = inputStream.read(buffer)) > 0)
{
compressedWriter.write(buffer, 0, length);
}
inputStream.close();
compressedWriter.close();
return compressed;
}
However, I'm not happy with the OutputStream copying loop since it's the "outdated" way of writing to streams. Instead, I want to use a Java 7 API method such as Files.copy:
public static File compressNew(File rawFile) throws IOException
{
File compressed = new File(rawFile.getCanonicalPath().split("\\.")[0]
+ "_compressed." + rawFile.getName().split("\\.")[1]);
OutputStream compressedWriter = new DeflaterOutputStream(new FileOutputStream(compressed));
Files.copy(compressed.toPath(), compressedWriter);
compressedWriter.close();
return compressed;
}
The latter method however does not work correctly, the compressed file is messed up and only a few bytes are copied. How come?
I see mainly two problems.
You copy from the target instead of the source. I think the copying has to be changed to Files.copy(rawFile.toPath(), compressedWriter);.
The Javadoc of copy says: "Note that if the given output stream is Flushable then its flush method may need to invoked after this method completes so as to flush any buffered output." So, you have to call the flush-method of the OutputStream after copy.
Additionally there is one more point. The Javadoc of copy says:
It is strongly recommended that the output stream be promptly closed if an I/O error occurs.
You can close the OutputStream in a finally-block to make sure it happens in case of an error. Another possibility is to use try with resources that was introduced in Java 7.
I'm quite new to using Java I/O as I haven't ever before and have written this to download a .mp4 file from www.kissanime.com.
The download is very, very slow at the moment (approximately 70-100kb/s) and was wondering how I could speed it up. I don't really understand the byte buffering so any help with that would be appreciated. That may be my problem, I'm not sure.
Here's my code:
protected static boolean downloadFile(URL source, File dest) {
try {
URLConnection urlConn = source.openConnection();
urlConn.setConnectTimeout(1000);
urlConn.setReadTimeout(5000);
InputStream in = urlConn.getInputStream();
FileOutputStream out = new FileOutputStream(dest);
BufferedOutputStream bout = new BufferedOutputStream(out);
int fileSize = urlConn.getContentLength();
byte[] b = new byte[65536];
int bytesDownloaded = 0, len;
while ((len = in.read(b)) != -1 && bytesDownloaded < fileSize) {
bout.write(b, 0, len);
bytesDownloaded += len;
// System.out.println((double) bytesDownloaded / 1000000.0 + "mb/" + (double) fileSize / 1000000.0 + "mb");
}
bout.close();
} catch (IOException e) {
e.printStackTrace();
}
return true;
}
Thanks. Any further information will be provided upon request.
I can't find any questions on here related to downloading media files, and I'm sorry if this is deemed to be a duplicate.
Try using IOUtils.toByteArray, It takes an inputstream and returns an array with all bytes, in my opinion it's generally a good idea to check the common utility packages like apache-commons and guava and see if what you're trying to do hasn't already been done
If you want to save the file from InputStream then use this bellow method of apache-commons
FileUtils.copyInputStreamToFile ()
public static void copyInputStreamToFile(InputStream source,
File destination)
throws IOException
Copies bytes from an InputStream source to a file destination. The directories up to destination will be created if they don't already exist. destination will be overwritten if it already exists. The source stream is closed.
Always use file and IO related stuff by using library if available.There are also some other utility methods available & you can explore .
IOUtils
FileUtils
Turns out that it was the vast number of redirects from the link that caused the download speed to be throttled. Thanks everyone who answered.
I want to read a file into a byte array. So, I am reading it using:
int len1 = (int)(new File(filename).length());
FileInputStream fis1 = new FileInputStream(filename);
byte buf1[] = new byte[len1];
fis1.read(buf1);
However, it is realy very slow. Can anyone inform me a very fast approach (possibly best one) to read a file into byte array. I can use java library also if needed.
Edit: Is there any benchmark which one is faster (including library approach).
It is not very slow, at least there is not way to make it faster. BUT it is wrong. If file is big enough the method read() will not return all bytes from fist call. This method returns number of bytes it managed to read as return value.
The right way is to call this method in loop:
public static void copy(InputStream input,
OutputStream output,
int bufferSize)
throws IOException {
byte[] buf = new byte[bufferSize];
int bytesRead = input.read(buf);
while (bytesRead != -1) {
output.write(buf, 0, bytesRead);
bytesRead = input.read(buf);
}
output.flush();
}
call this as following:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
copy(new FileInputStream(myfile), baos);
byte[] bytes = baos.toByteArray();
Something like this is implemented in a lot of packages, e.g. FileUtils.readFileToByteArray() mentioned by #Andrey Borisov (+1)
EDIT
I think that reason for slowness in your case is the fact that you create so huge array. Are you sure you really need it? Try to re-think your design. I believe that you do not have to read this file into array and can process data incrementally.
apache commons-io FileUtils.readFileToByteArray
I know how to get the inputstream for a given classpath resource, read from the inputstream until i reach the end, but it looks like a very common problem, and i wonder if there an API that I don't know, or a library that would make things as simple as
byte[] data = ResourceUtils.getResourceAsBytes("/assets/myAsset.bin")
or
byte[] data = StreamUtils.readStreamToEnd(myInputStream)
for example!
Java 9 native implementation:
byte[] data = this.getClass().getClassLoader().getResourceAsStream("/assets/myAsset.bin").readAllBytes();
Have a look at Google guava ByteStreams.toByteArray(INPUTSTREAM), this is might be what you want.
Although i agree with Andrew Thompson, here is a native implementation that works since Java 7 and uses the NIO-API:
byte[] data = Files.readAllBytes(Paths.get(this.getClass().getClassLoader().getResource("/assets/myAsset.bin").toURI()));
Take a look at Apache IOUtils - it has a bunch of methods to work with streams
I usually use the following two approaches to convert Resource into byte[] array.
1 - approach
What you need is to first call getInputStream() on Resource object, and then pass that to convertStreamToByteArray method like below....
InputStream stream = resource.getInputStream();
long size = resource.getFile().lenght();
byte[] byteArr = convertStreamToByteArray(stream, size);
public byte[] convertStreamToByteArray(InputStream stream, long size) throws IOException {
// check to ensure that file size is not larger than Integer.MAX_VALUE.
if (size > Integer.MAX_VALUE) {
return new byte[0];
}
byte[] buffer = new byte[(int)size];
ByteArrayOutputStream os = new ByteArrayOutputStream();
int line = 0;
// read bytes from stream, and store them in buffer
while ((line = stream.read(buffer)) != -1) {
// Writes bytes from byte array (buffer) into output stream.
os.write(buffer, 0, line);
}
stream.close();
os.flush();
os.close();
return os.toByteArray();
}
2 - approach
As Konstantin V. Salikhov suggested, you could use org.apache.commons.io.IOUtils and call its IOUtils.toByteArray(stream) static method and pass to it InputStream object like this...
byte[] byteArr = IOUtils.toByteArray(stream);
Note - Just thought I'll mention this that under the hood toByteArray(...) checks to ensure that file size is not larger than Integer.MAX_VALUE, so you don't have to check for this.
Commonly Java methods will accept an InputStream. In that majority of cases, I would recommend passing the stream directly to the method of interest.
Many of those same methods will also accept an URL (e.g. obtained from getResource(String)). That can sometimes be better, since a variety of the methods will require a repositionable InputStream and there are times that the stream returned from getResourceAsStream(String) will not be repositionable.
This question already has answers here:
Convert InputStream to byte array in Java
(34 answers)
Closed 6 years ago.
My background is .net, I'm fairly new to Java. I'm doing some work for our company's java team and the architect needs me to implement a method that takes an InputStream (java.io) object. In order to fulfill the method's purpose I need to convert that into a byte array. Is there an easy way to do this?
The simplest way is to create a new ByteArrayOutputStream, copy the bytes to that, and then call toByteArray:
public static byte[] readFully(InputStream input) throws IOException
{
byte[] buffer = new byte[8192];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
while ((bytesRead = input.read(buffer)) != -1)
{
output.write(buffer, 0, bytesRead);
}
return output.toByteArray();
}
A simple way would be to use org.apache.commons.io.IOUtils.toByteArray( inputStream ), see apache commons io.