We Keep Coding

How to log inputstream to file in android and then parse it

I am receiving an InputStream from HttpUrlConnection (connection.getInputStream()), and I am parsing the input stream using DocumentBuilder (documenbtBuilder.parse(inputStream)). Before parsing, I want to write the received data to log file. When I do that, I get org.xml.sax.SAXParseException: Unexpected end of document Exception in the parse method. My code works fine if I don't write to file, but I need to log the data received.
Please find the code that writes to file below :
final InputStream input = connection.getInputStream();
writeLogInfo(input);
Document doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(input);
//Method that writes tito log file.
private void writeLogInfo(InputStream input){
OutputStream os = new FileOutputStream("mylogfile.txt");
byte[] buffer = new byte[1024];
int byteRead;
while((byteRead = input.read(buffer)) != -1){
os.write(buffer,0,byteRead);
}
os.flush();
os.close();
}
I suspect it is because of multiple use of InputStream, since the code works when I don't invode writeLogInfo(). I am not closing the inputstream anywhere in my code. What am I doing wrong here ?

When you are writing the content to a file, you are reaching the end of the inputstream.
So after that, when you are trying to parse, you get the exception.
You need to use mark and reset methods on the inputstream before passing it to documentbuilder.
Also, first you need to check if the input stream supports mark.
Here is the javadoc, for your reference
http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html

Are you sure the size of the file is under 1024 bytes. If not why don't you put your "inputstream" to BufferredInputstream, and create the byte array..
BufferedInputStream bin= new BufferedInputStream(new DataInputStream(input));
byte[] buffer= new byte[bin.available()];
bin.read(buffer);
os.write(buffer);
......
......
bin.close();
......

File from InputStream

Yes, this question has been asked a millions times, and I believe I've looked at them all. They are very "sometimesy", slow, or not what I need.
On one project, I use the following code to use the InputStream received from a GET to turn that into a PDF. This works PERFECTLY, every time, on my physical device and my emulator (Genymotion 2.1.1, Emulator API 18 4.3). Note that some things are edited out, and the PDFs are generally small, less than 1 MB.
public abstract class MyPDFFile extends File implements ApiModel{
public MyPDFFile(InputStream inputStream){
super(context.getExternalFilesDir(
Environment.DIRECTORY_DOWNLOADS), "my_pdf.pdf");
if (externalStorageIsWritable()) {
try {
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
FileOutputStream fileInputStream = new FileOutputStream(this);
BufferedWriter bufferedWriter = new BufferedWriter(new OutputStreamWriter(fileInputStream));
int readLine;
char[] cbuf = new char[1];
do {
readLine = bufferedReader.read(cbuf);
bufferedWriter.write(cbuf);
} while (readLine != -1);
bufferedWriter.close();
}
catch(IOException e){
// Didn't work
}
}
else{
// Cant write
}
}
I figured on this new project, I could use the same code to download an APK from the internet to the device. Nope, definitely not the case. I eventually tried this for Inputstream to File:
FileOutputStream fileOutputStream = new FileOutputStream(file);
byte[] buffer = new byte[1];
while ( (read(buffer)) > 0 ) {
fileOutputStream.write(buffer);
}
fileOutputStream.close();
close();
That works on my emulator, and works fine. I moved to testing on my device... not so much, which is weird, because the working PDF code works on both my emulator and device. I've tried adjusting the size of my buffer to various multiples of 512 (which results in the file being EXTREMELY small, like a few KB, to being EXTREMELY large, about double the apk size, which is about 5.6 MB).
Also, another weird thing: I can NEVER get it to successfully save outside of the constructor. When I do the saving there, the InputStream is fine, my file gets created, yadayada, and when I use successful code, I just rename the file afterwards since all I have access to in the constructor is the InputStream. If I decide "No, I want to name it when I have the proper things" and simply save the InputStream to my object, it NEVER works properly. Can never get above 4KB for the downloaded file. I've tried extends InputStream and extends BufferedInputStream to no avail.
I can post more code if needed. All I would have access to is the InputStream from my GET request; I'm using the browep Android HTTP library and that's all I can get without trying to mess with the library itself (or overriding methods in it).

The problem is that you're reading the file byte by byte. This can take ton of time. Instead, read the file in bigger piece of chunks, like 4 or 8 KBs:
int file_chunk_size = 1024 * 4; //4KBs, written like this to easily change it to 8
byte[] buffer = new byte[file_chunk_size];
int bytesRead = 0;
while ( (bytesRead = read(buffer)) > 0 ) {
fileOutputStream.write(buffer, 0, bytesRead);
}

Convert byte array into a file without writing the file to a disk

I have saved icon size images in a mysql database in bytes, now, what I need to do is retrieve those file bytes from the database and show those images in a swing application, I have a method which gets the bytes from the database and convert it back to a file but I have to write that file in to the disk
this is my method,
public void downloadFile(int FamerId) throws Exception {
String sql = "SELECT * FROM images WHERE famer_id=?";
Connection con = JDBCConnectionPool.getInstance().checkOut();
PreparedStatement ps = con.prepareStatement(sql);
ps.setInt(1, FamerId);
ResultSet resultSet = ps.executeQuery();
int count = 0;
while (resultSet.next()) {
ByteArrayInputStream bais;
ObjectInputStream inputStream;
bais = new ByteArrayInputStream(resultSet.getBytes("image"));
inputStream = new ObjectInputStream(bais);
SaveFile sf = (SaveFile) inputStream.readObject();
FileOutputStream out = new FileOutputStream("fileLocation/" + resultSet.getString("image_name"));
byte[] bytes = sf.getArray();
int c = 0;
while (c < bytes.length) {
out.write(bytes[c]);
c++;
}
out.close();
inputStream.close();
bais.close();
JDBCConnectionPool.getInstance().checkOut();
}
}
but this method doesn't give what I need, please assist me.

You can read images directly from byte streams with the ImageIO class. Assuming of course that you have previously written the image data in a compatible format. Which is hard to say given the fact that in your code you use an intermediary object input stream when reading your byte data. Here's an example of how you can create an image directly from the database without using intermediary files:
bais = new ByteArrayInputStream(resultSet.getBytes("image"));
final BufferedImage image = ImageIO.read(bais);
// pass the image to your Swing layer to be rendered.
And an example of how you would have written the data to the database, in order to be able to use this code:
final ByteArrayOutputStream baos = new ByteArrayOutputStream(64000);
ImageIO.write(image, "PNG", baos);
final byte[] data = baos.toByteArray();
// write data to database

The answer to your question is its platform dependent. From the docs
A file output stream is an output stream for writing data to a File or
to a FileDescriptor. Whether or not a file is available or may be
created depends upon the underlying platform. Some platforms, in
particular, allow a file to be opened for writing by only one
FileOutputStream (or other file-writing object) at a time. In such
situations the constructors in this class will fail if the file
involved is already open.
FileOutputStream is meant for writing streams of raw bytes such as
image data. For writing streams of characters, consider using
FileWriter.
So if you want to write to a file then file may or may not be created.
If you don't want to create the file and you are just interested in byte[] (content of the file) you can then use solution provided by #Perception or can just pass the inputStream that you have already created.

Out of memory when encoding file to base64

Using Base64 from Apache commons
public byte[] encode(File file) throws FileNotFoundException, IOException {
byte[] encoded;
try (FileInputStream fin = new FileInputStream(file)) {
byte fileContent[] = new byte[(int) file.length()];
fin.read(fileContent);
encoded = Base64.encodeBase64(fileContent);
}
return encoded;
}
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.commons.codec.binary.BaseNCodec.encode(BaseNCodec.java:342)
at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:657)
at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:622)
at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:604)
I'm making small app for mobile device.

You cannot just load the whole file into memory, like here:
byte fileContent[] = new byte[(int) file.length()];
fin.read(fileContent);
Instead load the file chunk by chunk and encode it in parts. Base64 is a simple encoding, it is enough to load 3 bytes and encode them at a time (this will produce 4 bytes after encoding). For performance reasons consider loading multiples of 3 bytes, e.g. 3000 bytes - should be just fine. Also consider buffering input file.
An example:
byte fileContent[] = new byte[3000];
try (FileInputStream fin = new FileInputStream(file)) {
while(fin.read(fileContent) >= 0) {
Base64.encodeBase64(fileContent);
}
}
Note that you cannot simply append results of Base64.encodeBase64() to encoded bbyte array. Actually, it is not loading the file but encoding it to Base64 causing the out-of-memory problem. This is understandable because Base64 version is bigger (and you already have a file occupying a lot of memory).
Consider changing your method to:
public void encode(File file, OutputStream base64OutputStream)
and sending Base64-encoded data directly to the base64OutputStream rather than returning it.
UPDATE: Thanks to #StephenC I developed much easier version:
public void encode(File file, OutputStream base64OutputStream) {
InputStream is = new FileInputStream(file);
OutputStream out = new Base64OutputStream(base64OutputStream)
IOUtils.copy(is, out);
is.close();
out.close();
}
It uses Base64OutputStream that translates input to Base64 on-the-fly and IOUtils class from Apache Commons IO.
Note: you must close the FileInputStream and Base64OutputStream explicitly to print = if required but buffering is handled by IOUtils.copy().

Either the file is too big, or your heap is too small, or you've got a memory leak.
If this only happens with really big files, put something into your code to check the file size and reject files that are unreasonably big.
If this happens with small files, increase your heap size by using the -Xmx command line option when you launch the JVM. (If this is in a web container or some other framework, check the documentation on how to do it.)
If the file recurs, especially with small files, the chances are that you've got a memory leak.
The other point that should be made is that your current approach entails holding two complete copies of the file in memory. You should be able to reduce the memory usage, though you'll typically need a stream-based Base64 encoder to do this. (It depends on which flavor of the base64 encoding you are using ...)
This page describes a stream-based Base64 encoder / decoder library, and includes lnks to some alternatives.

Well, do not do it for the whole file at once.
Base64 works on 3 bytes at a time, so you can read your file in batches of "multiple of 3" bytes, encode them and repeat until you finish the file:
// the base64 encoding - acceptable estimation of encoded size
StringBuilder sb = new StringBuilder(file.length() / 3 * 4);
FileInputStream fin = null;
try {
fin = new FileInputStream("some.file");
// Max size of buffer
int bSize = 3 * 512;
// Buffer
byte[] buf = new byte[bSize];
// Actual size of buffer
int len = 0;
while((len = fin.read(buf)) != -1) {
byte[] encoded = Base64.encodeBase64(buf);
// Although you might want to write the encoded bytes to another
// stream, otherwise you'll run into the same problem again.
sb.append(new String(buf, 0, len));
}
} catch(IOException e) {
if(null != fin) {
fin.close();
}
}
String base64EncodedFile = sb.toString();

You are not reading the whole file, just the first few kb. The read method returns how many bytes were actually read. You should call read in a loop until it returns -1 to be sure that you have read everything.
The file is too big for both it and its base64 encoding to fit in memory. Either
process the file in smaller pieces or
increase the memory available to the JVM with the -Xmx switch, e.g.
java -Xmx1024M YourProgram

This is best code to upload image of more size
bitmap=Bitmap.createScaledBitmap(bitmap, 100, 100, true);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream); //compress to which format you want.
byte [] byte_arr = stream.toByteArray();
String image_str = Base64.encodeBytes(byte_arr);

Well, looks like your file is too large to keep the multiple copies necessary for an in-memory Base64 encoding in the available heap memory at the same time. Given that this is for a mobile device, it's probably not possible to increase the heap, so you have two options:
make the file smaller (much smaller)
Do it in a stram-based way so that you're reading from an InputStream one small part of the file at a time, encode it and write it to an OutputStream, without ever keeping the enitre file in memory.

In Manifest in applcation tag write following
android:largeHeap="true"
It worked for me

Java 8 added Base64 methods, so Apache Commons is no longer needed to encode large files.
public static void encodeFileToBase64(String inputFile, String outputFile) {
try (OutputStream out = Base64.getEncoder().wrap(new FileOutputStream(outputFile))) {
Files.copy(Paths.get(inputFile), out);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}

How to play .wav file from URL in web browser embed - Java

I want to play a .wav sound file in embed default media player in IE. Sound file is on some HTTP location. I am unable to sound it in that player.
Following is the code.
URL url = new URL("http://www.concidel.com/upload/myfile.wav");
URLConnection urlc = url.openConnection();
InputStream is = (InputStream)urlc.getInputStream();
fileBytes = new byte[is.available()];
while (is.read(fileBytes,0,fileBytes.length)!=-1){}
BufferedOutputStream out = new BufferedOutputStream(response.getOutputStream());
out.write(fileBytes);
Here is embed code of HTML.
<embed src="CallStatesTreeAction.do?ivrCallId=${requestScope.vo.callId}&agentId=${requestScope.vo.agentId}" type="application/x-mplayer2" autostart="0" playcount="1" style="width: 40%; height: 45" />
If I write in FileOutputStream then it plays well
If I replace my code of getting file from URL to my local hard disk. then it also works fine.
I don't know why I am unable to play file from HTTP. And why it plays well from local hard disk.
Please help.

Make sure you set the correct response type. IE is very picky in that regard.
[EDIT] Your copy loop is broken. Try this code:
URL url = new URL("http://www.concidel.com/upload/myfile.wav");
URLConnection urlc = url.openConnection();
InputStream is = (InputStream)urlc.getInputStream();
fileBytes = new byte[is.available()];
int len;
while ( (len = is.read(fileBytes,0,fileBytes.length)) !=-1){
response.getOutputStream.write(fileBytes, 0, len);
}
The problem with your code is: If the data isn't fetched in a single call to is.read(), it's not appended to fileBytes but instead the first bytes are overwritten.
Also, the output stream which you get from the response is already buffered.