I am seeing a strange behavior with ImageIO.read() method.
I pass the InputStream to this method and when I try to read it for the second time it fails to read and returns null.
I am trying to upload images to the Amazon S3 and I want to create 3 version of the image. The original and 2 thumbnails. My problem is that when I want to create the 2 thumbnails I need to read the InputStream using the ImageIO.read(). If I run this method 2 for the same InputStream I get the null for the second read.
I can circumvent this problem by reading only one and passing the same BufferedImage to the scaling method. However I still need the InputStream that my method gets to pass to the AmazonS3 services in other to upload the original file as well.
So my question is does anyone have any idea what happens to the input stream after ImageIO reads it for the first time?
Code sample below
public String uploadImage(InputStream stream, String filePath, String fileName, String fileExtension) {
try {
String originalKey = filePath + fileName + "." + fileExtension;
String smallThumbKey = filePath + fileName + ImageConst.Path.SMALL_THUMB + "." + fileExtension;
String largetThumbKey = filePath + fileName + ImageConst.Path.LARGE_THUMB + "." + fileExtension;
BufferedImage image = ImageIO.read(stream);
InputStream smallThumb = createSmallThumb(image, fileExtension);
InputStream largeThumb = createLargeThumb(image, fileExtension);
uploadFileToS3(originalKey, stream);
uploadFileToS3(smallThumbKey, smallThumb);
uploadFileToS3(largetThumbKey, largeThumb);
return originalKey;
} catch (IOException ex) {
Logger.getLogger(ManageUser.class.getName()).log(Level.SEVERE, null, ex);
}
return null;
}
ImageIO.read is going to read to the end of the input stream. Meaning there's no data left to read, which is why you're getting null when you try and read more data from it.
If you want to reuse the input stream, you'll have to call reset() on it; but that'll only work if the underlying InputStream implementation supports resetting, see markSupported() of InputStream.
That's the simple, but naive fix.
Keep in mind, you've already read the image into memory, so you don't really need to do that. This is a little clumsy, but you can write it out to a ByteArrayOutputStream, then build a new ByteArrayInputStream off of that.
If I were doing this, I'd probably read it into a byte array to begin with. Check out Commons IOUtils.read() for that. Then I'd build a new ByteArrayInputStream and reset() that as needed since it definitely supports marking.
I suppose, you could wrap your original input stream in a BufferedInputStream and use mark() to save the starting position within the stream, and reset() to return to it.
To fully answer your question: every stream gets read sequentially (often there is some kind of internal pointer pointing to the current position). After the first ImageIO.read(), the stream is read until its end, thus, any further read operation won't return any data (usually -1 to indicate the end of the stream). Using mark() you can save a certain position and later on return to it using reset().
Related
I have a method where I need to read a resource file and pass it to another method as an InputStream. The obvious approach
InputStream is = classLoader.getResourceAsStream("filename.pem");
works fine when actually running the application, but in testing it returns an empty InputStream (filled with zeros). I don't think its a problem with the resource path or anything, because when I use a nonsense path like "filex" (filex does not exist), I get an actual null pointer exception and not an empty stream. Also in debugger the complete file path of the empty Stream points to the correct path, where the file actually is stored (default class root).
Furthermore, with the following workaround it works:
File file = new File(classLoader.getResource("filename.pem").getFile());
String fileS= new String(Files.readAllBytes(file.toPath()), Charset.defaultCharset());
InputStream is = classLoader.getResourceAsStream("filename.pem");
InputStream is2 = new ByteArrayInputStream(fileS.getBytes(StandardCharsets.UTF_8));
In this example is2 has the actual content of the file in the InputStream, while is has an Stream filled with zeros. I can't quite explain that behaviour. I double checked with 'getClass().getClassLoader().getClass()' if we use some modified ClassLoader in the Application, but it is the original one from sun.misc.Launcher$AppClassLoader.
So my questions are:
Why does the workaround work but not the classic approach?
Why does it fail only in test class?
Is there a way to make it work? The workaround is more lines of code and also need to catch IOException because of the Files.readAllBytes() call.
The only idea I had left: the encoding or charset has something to do with it. But to my knowledge there is no parameter in getResourceAsStream() like Charset or StandardCharsets.
If you open a resource file as a stream, you end up with a BufferedInputStream around a FileInputStream.
The call chain is as follows:
java.lang.ClassLoader#getResource returns an URL
url.openStream() is called
this first opens the Stream; sun.net.www.protocol.file.Handler#createFileURLConnection
then the Stream is connected: is = new BufferedInputStream(new FileInputStream(filename)); in sun.net.www.protocol.file.FileURLConnection#connect
finally you get this is back as InputStream
What you're seeing is the empty internal buffer of the BufferedInputStream, which will be used as soon as you start reading from the InputStream.
See is FileInputStream not buffered and why BufferedInputStream is faster?
If you for example read from the InputStream with all zero's, you will see it does actually contain data:
Scanner scanner = new Scanner(inputStream, StandardCharsets.UTF_8.name());
String firstLine = scanner.next();
From https://www.baeldung.com/convert-input-stream-to-string
Your workaround works, because after you've located the file from the resource URL you actually start reading it directly.
So what might be failing in your test; are you not trying to read from the stream in your testcase? How are you using/validating if this inputstream is correct in your test vs the real application? There might be the problem.
I am currently developing a REST service which receives in its request a field where it is passed a file in base 64 format ("n" characters come). What I do within the service logic is to convert that character string to a File to save it in a predetermined path.
But the problem is that when the file is too large (3MB) the service becomes slow and takes a long time to respond.
This is the code I am using:
String filename = "TEXT.DOCX"
BufferedOutputStream stream = null;
// THE FIELD base64file IS WHAT A STRING IN BASE FORMAT COMES FROM THE REQUEST 64
byte [] fileByteArray = java.util.Base64.getDecoder (). decode (base64file);
// VALID FILE SIZE
if ((1 * 1024 * 1024 <fileByteArray.length) {
logger.info ("The file [" + filename + "] is too large");
} else {
stream = new BufferedOutputStream (new FileOutputStream (new File ("C: \" + filename)));
stream.write (fileByteArray);
}
How can I do to avoid this inconvenience. And that my service does not take so long to convert the file to File.
Buffering does not improve your performance here, as all you are trying to do is simply write the file as fast as possible. Generally it looks fine, change your code to directly use the FileOutputStream and see if it betters things:
try (FileOutputStream stream = new FileOutputStream(path)) {
stream.write(bytes);
}
Alternatively you could also try using something like Apache Commons to do the task for you:
FileUtils.writeByteArrayToFile(new File(path), bytes);
Try the following, also for large files.
Path outPath = Paths.get(filename);
try (InputStream in = Base64.getDecoder ().wrap(base64file)) {
Files.copy(in, outPath);
}
This keeps only a buffer in memory. Your code might become slow because of taking more memory.
wrap takes an InputStream which you should provide, not the entire String.
From Network point of view:
Both json and xml can support large amount of data exchange. And, 3MB is not really huge. But, there is a limitation on how much browser can handle (if this call is from a user interface).
Also, web server like Tomcat has property to handle 2MB by default (check maxPostSize http://tomcat.apache.org/tomcat-6.0-doc/config/http.html#Common_Attributes)
You can also try chunking the request payload (although it shouldn't be required for a 3MB file)
From Implementation point of view:
Write operation on your disk could be slow. It also depends on your OS.
If your file size is really large, you can use Java FileChannel class with ByteBuffer.
To know the cause of slowness (network delay or code), check the performance with a simple Java program against the web service call.
Consider the following code snippet getInputStreamForRead() method creates and returns a new input stream for read.
InputStream is = getInputStreamForRead(); //This method creates and returns an input stream for file read
is = getDecompressedStream(is);
Since the orginal file content is compressed and stored it has to be decompressed while reading. Hence getDecompressedStream() method below would provide option to decompress the stream content
public InputStream getDecompressedStream(InputStream is) throws Exception {
return new GZIPInputStream(is);
}
Have the following doubts
Which one is correct for the above snippet
is = getDecompressedStream(is)
or
InputStream newStream = getDecompressedStream(is)
Will reusing the InputStream variable again cause any trouble?
I'm completely new with streams. Kindly help me to know about this.
As long as:
you're not manipulating the original InputStream between the original assignment and the new invocation
you're always closing your streams in a finally statement
... you should be fine re-assigning to the original variable - it's just a new value passed to an existing reference.
In fact, that may be the recommended way, since you get to only close one Closeable programmatically, as GZIPInputStream#close...
Closes this input stream and releases any system resources associated with the stream.
(see here - I read this as, "closes the underlying stream").
Since you want to close the input stream correctly, the best way is to create the input stream using chaining, and using a try-with-resources to handle the close for you.
try (InputStream is = getDecompressedStream(getInputStreamForRead())) {
// code using stream here
}
I tried to use FileChannel.transferFrom to move some content of a file to the begining.
try (RandomAccessFile rafNew = new RandomAccessFile(_fileName, "rw");
RandomAccessFile rafOld = new RandomAccessFile(_fileName, "r");)
{
rafOld.seek(pos);
rafOld.getChannel().transferTo(0, count, rafNew.getChannel());
} catch (IOException e) {
throw new RuntimeException(e.getMessage());
}
The result of this is a file with strange repetitions of data. The example works if I first transfer data to a buffer file and then from buffer file back to the origin file again.
The Java Docs say nothing about the case where source and destination are the same file.
You are transferring 'count' bytes starting from zero from 'rafOld' to 'rafNew', which hasn't had any seeks done on it, so is also at position zero. So at best your code doesn't do what you said it does. The seek() you did on 'rafOld' doesn't affect the transferTo() operation. You should have removed it and written
transferTo(pos, count, rafNew.getChannel());
But there are still two problems with this.
If count > pos you will be overwriting the source region.
transferTo() must be called in a loop, as it isn't guaranteed to compete the entire transfer in a single call. It returns the number of bytes actually transferred,
I have program in which I have to load a PNG as a String and then save it again, but after I save it it becomes unreadable. If I open both the loaded PNG and the saved String in the editor, I can see that java created linebreaks all over the file. If this is is the problem, how can I avoid this?
public static void main(String[] args)
{
try
{
File file1 = new File("C://andim//testFile.png");
StringBuffer content = new StringBuffer();
BufferedReader reader = null;
reader = new BufferedReader(new FileReader(file1));
String s = null;
while ((s = reader.readLine()) != null)
{
content.append(s).append(System.getProperty("line.separator"));
}
reader.close();
String loaded=content.toString();
File file2=new File("C://andim//testString.png");
FileWriter filewriter = new FileWriter(file2);
filewriter.write(loaded);
filewriter.flush();
filewriter.close();
}
catch(Exception exception)
{
exception.printStackTrace();
}
}
I have program in which I have to load a PNG as a String and then save it again, but after I save it it becomes unreadable.
Yes, I'm not surprised. You're treating arbitrary binary data as if it's text data (in whatever your platform default encoding is, to boot). It's not. Don't do that. It's possible that in some encodings you'll get away with it - until you start trying to pass the string elsewhere in a way that strips unprintable characters etc.
If you must convert arbitrary binary data to text, use base64 or hex. If possible, avoid the conversion to text in the first place though. If you just want to copy a file, use InputStream and OutputStream - not Reader and Writer.
This is a big general point: keep data in its "native" representation as long as you possibly can. Only convert data to a different representation when you absolutely have to, and be very careful about it.
Don't use text-based APIs to read binary files. In this case, you don't want a BufferedReader, and you certainly don't want readLine, which may well treat more than just one thing as a line separator. Use an InputStream (for instance, FileInputStream) and an OutputStream (for instance, FileOutputStream), not readers and writers.
Don't do that
PNGs are not textual data.
If you try to read arbitrary bytes into a string, Java will mangle the bytes into actual text, corrupting the data you read.
You need to use byte[]sm not strings.