I have to a class that generates an Excel file, class takes an output stream, intended for a FileOutputWriter.
However I need to upload the file to S3 later, so saving it to a file would be an unnecessary step.
To upload a file to S3 do I need it as an input stream.
I have therefore been looking for a way to convert an output stream to an input stream or transfer the data to an input stream.
I have tried PipeOutputStream, but xlsFile.write(outstr); just blocks forever when using a PipedOutputStream, using a FileOutputStream will it work fine and write the file.
PipedInputStream inputStream = new PipedInputStream();
try (OutputStream outstr = new PipedOutputStream(inputStream)) {
xlsFile.write(outstr);
outstr.flush();
return inputStream;
}
JDK 9 has added InputStream#transferTo(OutputStream out) for this functionality, though JDK 9 is not yet released. Early releases are available here.
Related
I have a web page with an upload feature which lets you upload a excel file, on hitting upload an Ajax call is fired. From there I get the FileItem input stream and using the method fileItem.getInputStream(), I have another class with a method which I need to pass the file to, which has a FileInputStream parameter. So my question is how do I convert the input stream to a FileInputStream?
A detailed solution would be appreciated as I am a junior developer, so I am still learning.
Many thanks.
From JavaDoc
A FileInputStream obtains input bytes from a file in a file system.
I would suggest two solutions:
The proper one is to change the API and to have InputStream as a parameter. I don't see a reason why you have FileInputStream in your API.
If you don't own the API and cannot change it I'm afraid you will need to save the InputStream to temp file and then create FileInputStream giving a path to this file (it's a suboptimal solution as you first write the file to disk - risking out of space - and then read it and streaming API is designed for reading / writing data on the fly)
If you are using org.apache.commons.fileupload.FileItem interface then your class is probably DefaultFileItem which is a subclass of DiskFileItem. So you can cast FileItem to DiskFileItem. then if you look at the source code of DiskFileItem you'll find that getInputStream() is actually returning a FileInputStream or a ByteArrayInputStream If you get a FileInputStream from DiskFileItem you can pass it directly to your other class. But if you get a ByteArrayInputStream you will have to write the contents to your own temporary file and then open another FileInputStream on this temp file. There is also another method DiskFileItem.getStoreLocation() which seem to return the server side File used for upload, but it may return null if the file is cached in memory.
In conclusion: you cannot be sure that there is going to be a server side file because the upload may be cached in memory. Therefore if you need a FileInputStream elsewhere you will have to create it yourself by creating a temp file. There is an example on how to pipe between two streams here.
//Pass file path/name directly to FileInputStream
FileInputStream input1 = new FileInputStream("input.txt");
//Save file path that has been passed in by the user, into a string variable.
String fileName = args[0];
//pass path to File object
File inputFile = new File(fileName);
//pass file object to FileOutputStream
FileOutputStream output = new FileOutputStream(inputFile);
I need to write a file stream to database. The file content must be readable only through
the program. Manual open file should not display the readable content. I decided to use
ObjectOutput stream as it is the binary writing mechanism in java. But I can see the string
content when I open the file.
Writing to stream
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream os = new ObjectOutputStream(baos);
os.writeObject("HIIIIIIIIIIIIIIIIIIIIII HOW ARE YOU");
The created content is look like
’ t #HIIIIIIIIIIIIIIIIIIIIII HOW ARE YOU
How to get complete binary stream output?
The file content must be readable only through the program. Manual open file should not display the readable content.
So you need some security.
I decided to use ObjectOutput stream as it is the binary writing mechanism in java.
That's (a) a non sequitur, and (b) security by obscurity: i.e. it is no security at all.
You should use encryption.
I have written a little program that just reads a files contents and writes it to a new copy. This works perfectly with text files, but with PNGs and video files, it fails to correctly create the file (the image is all black or the video will not play). I know there are APIs that can copy files with one line, but I'd love to know why this isn't working. Here is the code:
import java.io.*;
public class CopyFile
{
public static void main(String[] args) throws Exception
{
File file = new File("test.mp4");
File copy = new File("copy.mp4");
InputStreamReader input = new InputStreamReader(new FileInputStream(file));
OutputStreamWriter out = new OutputStreamWriter(new FileOutputStream(copy));
System.out.println(input.ready());
while(input.ready())
{
int i = input.read();
//System.out.print( (char) ( (byte) i));
out.write(i);
}
input.close();
out.flush();
out.close();
}
}
Don't use Reader and Writer unless you know the input is text. Use InputStream and OutputStream.
Don't use ready(), or, for Sotirios' benefit, available() either. Neither of them is a valid test for end of stream. They both concern whether the input can be read without blocking, which isn't the same thing at all. See the Javadoc.
You're not detecting end of stream correctly. If read() returns -1 you're still copying that to the output.
Copying a single character or single byte at a time is extremely slow.
The canonical way to copy streams in Java is as follows:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
where count is an int, and buffer is a byte[] of any size greater than zero, typically 8192.
Readers and Writers are for reading character streams (i.e., text). Pictures and videos are binary data, not text, and will probably be corrupted if you pass them through character streams. This is because, depending on the character set, there is not necessarily a reversible mapping between bytes and characters. Some byte sequences are gibberish if interpreted as characters, then gibberish gets written back to the file.
Use the InputStream and OutputStream that you open directly, instead of wrapping them up as a Reader and Writer, and it will work correctly. These are byte streams and can handle any type of data.
E.g.,
InputStream input = new FileInputStream(file);
OutputStream out = new FileOutputStream(copy);
P.S. This will still be quite slow. You can wrap the streams in a BufferedInputStream and BufferedOutputStream for a simple way to improve performance, although the one-line copy APIs will probably still be faster.
I'm retrieving a file from a FTP Server. The file is encoded as UTF-8
ftpClient.connect(props.getFtpHost(), props.getFtpPort());
ftpClient.login(props.getUsername(), props.getPassword());
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
inputStream = ftpClient.retrieveFileStream(fileNameBuilder
.toString());
And then somewhere else I'm reading the input stream
bufferedReader = new BufferedReader(new InputStreamReader(
inputStream, "UTF-8"));
But the file is not getting read as UTF-8 Encoded!
I tried ftpClient.setAutodetectUTF8(true); but still doesn't work.
Any ideas?
EDIT:
For example a row in the original file is
...00248090041KENAN SARÐIN 00000000015.993FAC...
After downloading it through FTPClient, I parse it and load in a java object, one of the fields of the java object is name, which for this row is read as "KENAN SAR�IN"
I tried dumping to disk directly:
File file = new File("D:/testencoding/downloaded-file.txt");
FileOutputStream fop = new FileOutputStream(file);
ftpClient.retrieveFile(fileName, fop);
if (!file.exists()) {
file.createNewFile();
}
I compared the MD5 Checksums of the two files(FTP Server one and the and the one dumped to disk), and they're the same.
I would separate out the problems first: dump the file to disk, and compare it with the original. If it's the same as the original, the problem has nothing to do with UTF-8. The FTP code looks okay though, and if you're saying you want the raw binary data, I'd expect it not to mess with anything.
If the file is the same after transfer as before, then the problem has nothing to do with FTP. You say "the file is not getting read as UTF-8 Encoded" but it's not clear what you mean. How certain are you that it's UTF-8 text to start with? If you could edit your question with the binary data, how it's being read as text, and how you'd expect it to be read as text, that would really help.
Try to download the file content as bytes and not as characters using InputStream and OutputStream instead of InputStreamReader. This way you are sure that the file is not changed during transfer.
I was handed some data in a file with an .dat extension. I need to read this data in a java program and build the data into some objects we defined. I tried the following, but it did not work
FileInputStream fstream = new FileInputStream("news.dat");
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
Could someone tell me how to do this in java?
What kind of file is it? Is it a binary file which contains serialized Java objects? If so, then you rather need ObjectInputStream instead of DataInputStream to read it.
FileInputStream fis = new FileInputStream("news.dat");
ObjectInputStream ois = new ObjectInputStream(fis);
Object object = ois.readObject();
// ...
(don't forget to properly handle resources using close() in finally, but that's beyond the scope of this question)
See also:
Basic serialization tutorial
A .dat file is usually a binary file, without any specific associated format. You can read the raw bytes of the file in a manner similar to what you posted - but you will need to interpret these bytes according to the underlying format. In particular, when you say "open" the file, what exactly do you want to happen in Java? What kind of objects do you want to be created? How should the stream of bytes map to these objects?
Once you know this, you can either write this layer yourself or use an existing API (assuming it's a standard format).
For reference, your example doesn't work because it assumes that the binary format is a character representation in the platform's default charset (as per the InputStreamReader constructor). And as you say it's binary, this will fail to convert the binary to a stream of characters (since, after all, it's not).
// BufferedInputStream not strictly needed, but much more efficient than reading
// one byte at a time
BufferedInputStream in = new BufferedInputStream (new FileInputStream("news.dat"));
This will give you a buffered stream which will return the raw bytes of the file; you can now either read and process them yourself, or pass this input stream to some library API that will create appropriate objects for you (if such a library exists).
That entirely depends on what sort of file the .dat is. Unfortunately, .dat is often used as a generic extension for a data file. It could be binary, in which case you could use FileInputStream fstream = new FileInputStream(new File("news.dat")); and call read() to get bytes from the file, or text, in which case you could use BufferedReader buff = new BufferedInputReader(new FileInputStream(new File("news.dat"))); and call readLine() to get each line of text. [edit]Or it could be Java objects in which case what BalusC said.[/edit]
In both cases, you'd then need to know what format the file was in to divide things up and get meaning from it, although this would be much easier if it was text as it could be done by inspection.
Please try the below code:
FileReader file = new FileReader(new File("File.dat"));
BufferedReader br = new BufferedReader(file);
String temp = br.readLine();
while (temp != null) {
temp = br.readLine();
System.out.println(temp);
}
A better way would be to use try-with-resources so that you would not have to worry about closing the resources.
Here is the code.
FileInputStream fis = new FileInputStream("news.dat");
try(ObjectInputStream objectstream = new ObjectInputStream(fis)){
objectstream.readObject();
}
catch(IOException e){
//
}