We Keep Coding

InputStream returns unexpected -1/empty

I seem to be hitting a constant unexpected end of my file. My file contains first a couple of strings, then byte data.
The file contains a few separated strings, which my code reads correctly.
However when I begin to read the bytes, it returns nothing. I am pretty sure it has to do with me using the Readers. Does the BufferedReader read the entire stream? If so, how can I solve this?
I have checked the file, and it does contain plenty of data after the strings.
InputStreamReader is = new InputStreamReader(in);
BufferedReader br = new BufferedReader(is);
String line;
{
line = br.readLine();
String split[] = line.split(" ");
if (!split[0].equals("#binvox")) {
ErrorHandler.log("Not a binvox file");
return false;
}
ErrorHandler.log("Binvox version: " + split[1]);
}
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int nRead, cnt = 0;
byte[] data = new byte[16384];
while ((nRead = in.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
cnt += nRead;
}
buffer.flush();
// cnt is always 0
The binvox format is as followed:
#binvox 1
dim 64 40 32
translate -3 0 -2
scale 6.434
data
[byte data]
I'm basically trying to convert the following C code to Java:
http://www.cs.princeton.edu/~min/binvox/read_binvox.html

For reading the whole String you should do this:
ArrayList<String> lines = new ArrayList<String>();
while ((line = br.readLine();) != null) {
lines.add(line);
}
and then you may do a cycle to split each line, or just do what you have to do during the cycle.

As icza has alraedy wrote, you can't create a InputStream and a BufferedReader and user both. The BufferedReader will read from the InputStream as many as he wants, and then you can't access your data from the InputStream.
You have several ways to fix it:
Don't use any Reader. Read the bytes yourself from an InputStream and call new String(bytes) on it.
Store your data encoded (e.g. Base64). Encoded data can be read from a Reader. I would recommend this solution. That'll look like that:
public byte[] readBytes (Reader in) throws IOException
{
String base64 = in.readLine(); // Note that a Base64-representation never contains \n
byte[] data = Base64.getDecoder().decode(base64);
return data
}

You can't wrap an InputStream in a BufferedReader and use both.
As its name hints, BufferedReader might read ahead and buffer data from the underlying InputStream which then will not be available when reading from the underlying InputStream directly.
Suggested solution is not to mix text and binary data in one file. They should be stored in 2 separate files and then they can be read separately. If the remaining data is not binary, then you should not read them via InputStream but via your wrapper BufferedReader just as you read the first lines.

I recommend to create a BinvoxDetectorStream that pre-reads some bytes
public class BinvoxDetectorStream extends InputStream {
private InputStream orig;
private byte[] buffer = new byte[4096];
private int buflen;
private int bufpos = 0;
public BinvoxDetectorStream(InputStream in) {
this.orig = new BufferedInputStream(in);
this.buflen = orig.read(this.buffer, 0, this.buffer.length);
}
public BinvoxInfo getBinvoxVersion() {
// creating a reader for the buffered bytes, to read a line, and compare the header
ByteArrayInputStream bais = new ByteArrayInputStream(buffer);
BufferedReader rdr = new BufferedReader(new InputStreamReader(bais)));
String line = rdr.readLine();
String split[] = line.split(" ");
if (split[0].equals("#binvox")) {
BinvoxInfo info = new BinvoxInfo();
info.version = split[1];
split = rdr.readLine().split(" ");
[... parse all properties ...]
// seek for "data\r\n" in the buffered data
while(!(bufpos>=6 &&
buffer[bufpos-6] == 'd' &&
buffer[bufpos-5] == 'a' &&
buffer[bufpos-4] == 't' &&
buffer[bufpos-3] == 'a' &&
buffer[bufpos-2] == '\r' &&
buffer[bufpos-1] == '\n') ) {
bufpos++;
}
return info;
}
return null;
}
#Override
public int read() throws IOException {
if(bufpos < buflen) {
return buffer[bufpos++];
}
return orig.read();
}
}
Then, you can detect the Binvox version without touching the original stream:
BinvoxDetectorStream bds = new BinvoxDetectorStream(in);
BinvoxInfo info = bds.getBinvoxInfo();
if (info == null) {
return false;
}
...
[moving bytes in the usual way, but using bds!!! ]
This way we preserve the original bytes in bds, so we'll be able to copy it later.

I saw someone else's code that solved exactly this.
He/she used DataInputStream, which can do a readLine (although deprecated) and readByte.

Reading bytes from a file?

I need to read some data until file is opened at different times, but I'm not sure if pointer to data that have not been read yet is automatic increased?
My method:
//method for copy binary data from file to binaryDataBuffer
void readcpy(String fileName, int pos, int len) {
try {
File lxDirectory = new File(Environment.getExternalStorageDirectory().getPath() + "/DATA/EXAMPLE/");
File lxFile = new File(lxDirectory, (fileName);
FileInputStream mFileInputStream = new FileInputStream(lxFile);
mFileInputStream.read(binaryDataBuffer, pos, len);
}
catch (Exception e) {
Log.d("Exception", e.getMessage());
}
}
So, if I call this method first time and read and save 5 bytes for example, will be on next call of the method read out bytes from 5th byte? I don't close file after reading.

When you create an InputStream (because a FileInputStream is an InputStream), the stream is created anew each time, and starts at the beginning of the stream (therefore the file).
If you want to read from where you left off the last time, you need to retain the offset and seek -- or retain the initial input stream you have opened.
While you can seek into a stream (using .skip()), it is in any event NOT recommended to reopen each time, it is costly; also, when you are done with a stream, you should close it:
// with Java 7: in automatically closed
try (InputStream in = ...;) {
// do stuff
} catch (WhateverException e) {
// handle exception
}
// with Java 6
InputStream in = ...;
try {
// do stuff
} catch (WhateverException e) {
// handle exception
} finally {
in.close();
}

Try this code:
public String getStringFromFile (String filePath) throws Exception {
File fl = new File(filePath);
FileInputStream fin = new FileInputStream(fl);
BufferedReader reader = new BufferedReader(new InputStreamReader(fin));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line).append("\n");
}
String ret = sb.toString();
//Make sure you close all streams.
fin.close();
reader.close();
return ret;
}

I find RandomAccessFile, it has offset which I need in my case.

Best way to read an input stream to a buffer

For reading any input stream to a buffer there are two methods. Can someone help me understand which is the better method and why? And in which situation we should use each method?
Reading line by line and appending it to the buffer.
Eg:
public String fileToBuffer(InputStream is, StringBuffer strBuffer) throws IOException{
StringBuffer buffer = strBuffer;
InputStreamReader isr = null;
try {
isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
while ((line = br.readLine()) != null) {
buffer.append(line + "\n");
}
} finally {
if (is != null) {
is.close();
}
if (isr != null) {
isr.close();
}
}
return buffer.toString();
}
Reading up to buffer size ie 1024 bytes in a char array.
Eg:
InputStreamReader isr = new InputStreamReader(is);
final int bufferSize = 1024;
char[] buffer = new char[bufferSize];
StringBuffer strBuffer = new StringBuffer();
/* read the base script into string buffer */
try {
while (true) {
int read = isr.read(buffer, 0, bufferSize);
if (read == -1) {
break;
}
strBuffer.append(buffer, 0, read);
}
} catch (IOException e) {
}

Consider
public String fileToBuffer(InputStream is, StringBuffer strBuffer) throws IOException {
StringBuilder sb = new StringBuilder(strBuffer);
try (BufferedReader rdr = new BufferedReader(new InputStreamReader(is))) {
for (int c; (c = rdr.read()) != -1;) {
sb.append((char) c);
}
}
return sb.toString();
}

Depends on the purpose.
For work with text files read lines (if you need them).
For work with raw binary data use chunks of bytes.
In you examples chunks of bytes are more robust.
What if a line is too long and breaks some of intermediate objects?
If your file is binary, do you know how big a line will be?
May be the size of file.
Trying to "swallow" too big String may cause ErrorOutOfMemory.
With 1024 bytes it (ok - almost) never happens.
Chunking by 1024 bytes may take longer, but its more reliable.

Using 'readLine' isn't so neat. The asker's method 2 is quite standard, but the below method is unique (and likely better):
//read the whole inputstream and put into a string
public String inputstream2str(InputStream stream) {
Scanner s = new Scanner(stream).useDelimiter("\\A");
return s.hasNext()? s.next():"";
}
From a String you can convert to byte array or whatever buffer you want.

Problem: Java-Read socket (Bluetooth) input stream until a string (<!MSG>) is read

I use the following code (from Bluetooth Chat sample app) to read the incoming data and construct a string out of the bytes read. I want to read until this string has arrived <!MSG>. How to insert this condition with read() function?
The whole string looks like this <MSG><N>xxx<!N><V>yyy<!V><!MSG>. But the read() function does not read entire string at once. When I display the characters, I cannot see all the characters in the same line. It looks like:
Sender: <MS
Sender: G><N>xx
Sender: x<V
.
.
.
I display the characters on my phone (HTC Desire) and I send the data using windows hyperterminal.
How to make sure all the characters are displayed in a single line? I have tried using StringBuilder and StringBuffer instead of new String() but the problem is read() function does not read all the characters sent. The length of the input stream (bytes) is not equal to actual length of the string sent. The construction of string from the read bytes is happening alright.
Thank you for any suggestions and time spent on this. Also please feel free to suggest other mistakes or better way of doing below things, if any.
Cheers,
Madhu
public void run() {
Log.i(TAG, "BEGIN mConnectedThread");
//Writer writer = new StringWriter();
byte[] buffer = new byte[1024];
int bytes;
//String end = "<!MSG>";
//byte compare = new Byte(Byte.parseByte(end));
// Keep listening to the InputStream while connected
while (true) {
try {
//boolean result = buffer.equals(compare);
//while(true) {
// Read from the InputStream
bytes = mmInStream.read(buffer);
//Reader reader = new BufferedReader(new InputStreamReader(mmInStream, "UTF-8"));
//int n;
//while ((bytes = reader.read(buffer)) != -1) {
//writer.write(buffer, 0, bytes);
//StringBuffer sb = new StringBuffer();
//sb = sb.append(buffer);
//String readMsg = writer.toString();
String readMsg = new String(buffer, 0, bytes);
//if (readMsg.endsWith(end))
// Send the obtained bytes to the UI Activity
mHandler.obtainMessage(BluetoothChat.MESSAGE_READ, bytes, -1, readMsg)
.sendToTarget();
//}
} catch (IOException e) {
Log.e(TAG, "disconnected", e);
connectionLost();
break;
}
}
}

The read function does not make any guarantee about the number of bytes it returns (it generally tries to return as many bytes from the stream as it can, without blocking). Therefore, you have to buffer the results, and keep them aside until you have your full message. Notice that you could receive something after the "<!MSG>" message, so you have to take care not to throw it away.
You can try something along these lines:
byte[] buffer = new byte[1024];
int bytes;
String end = "<!MSG>";
StringBuilder curMsg = new StringBuilder();
while (-1 != (bytes = mmInStream.read(buffer))) {
curMsg.append(new String(buffer, 0, bytes, Charset.forName("UTF-8")));
int endIdx = curMsg.indexOf(end);
if (endIdx != -1) {
String fullMessage = curMsg.substring(0, endIdx + end.length());
curMsg.delete(0, endIdx + end.length());
// Now send fullMessage
}
}

Prepend lines to file in Java

Is there a way to prepend a line to the File in Java, without creating a temporary file, and writing the needed content to it?

No, there is no way to do that SAFELY in Java. (Or AFAIK, any other programming language.)
No filesystem implementation in any mainstream operating system supports this kind of thing, and you won't find this feature supported in any mainstream programming languages.
Real world file systems are implemented on devices that store data as fixed sized "blocks". It is not possible to implement a file system model where you can insert bytes into the middle of a file without significantly slowing down file I/O, wasting disk space or both.
The solutions that involve an in-place rewrite of the file are inherently unsafe. If your application is killed or the power dies in the middle of the prepend / rewrite process, you are likely to lose data. I would NOT recommend using that approach in practice.
Use a temporary file and renaming. It is safer.

There is a way, it involves rewriting the whole file though (but no temporary file). As others mentioned, no file system supports prepending content to a file. Here is some sample code that uses a RandomAccessFile to write and read content while keeping some content buffered in memory:
public static void main(final String args[]) throws Exception {
File f = File.createTempFile(Main.class.getName(), "tmp");
f.deleteOnExit();
System.out.println(f.getPath());
// put some dummy content into our file
BufferedWriter w = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(f)));
for (int i = 0; i < 1000; i++) {
w.write(UUID.randomUUID().toString());
w.write('\n');
}
w.flush();
w.close();
// append "some uuids" to our file
int bufLength = 4096;
byte[] appendBuf = "some uuids\n".getBytes();
byte[] writeBuf = appendBuf;
byte[] readBuf = new byte[bufLength];
int writeBytes = writeBuf.length;
RandomAccessFile rw = new RandomAccessFile(f, "rw");
int read = 0;
int write = 0;
while (true) {
// seek to read position and read content into read buffer
rw.seek(read);
int bytesRead = rw.read(readBuf, 0, readBuf.length);
// seek to write position and write content from write buffer
rw.seek(write);
rw.write(writeBuf, 0, writeBytes);
// no bytes read - end of file reached
if (bytesRead < 0) {
// end of
break;
}
// update seek positions for write and read
read += bytesRead;
write += writeBytes;
writeBytes = bytesRead;
// reuse buffer, create new one to replace (short) append buf
byte[] nextWrite = writeBuf == appendBuf ? new byte[bufLength] : writeBuf;
writeBuf = readBuf;
readBuf = nextWrite;
};
rw.close();
// now show the content of our file
BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(f)));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
}

You could store the file content in a String and prepend the desired line by using a StringBuilder-Object. You just have to put the desired line first and then append the file-content-String.
No extra temporary file needed.

No. There are no "intra-file shift" operations, only read and write of discrete sizes.

It would be possible to do so by reading a chunk of the file of equal length to what you want to prepend, writing the new content in place of it, reading the later chunk and replacing it with what you read before, and so on, rippling down the to the end of the file.
However, don't do that, because if anything stops (out-of-memory, power outage, rogue thread calling System.exit) in the middle of that process, data will be lost. Use the temporary file instead.

private static void addPreAppnedText(File fileName) {
FileOutputStream fileOutputStream =null;
BufferedReader br = null;
FileReader fr = null;
String newFileName = fileName.getAbsolutePath() + "#";
try {
fileOutputStream = new FileOutputStream(newFileName);
fileOutputStream.write("preappendTextDataHere".getBytes());
fr = new FileReader(fileName);
br = new BufferedReader(fr);
String sCurrentLine;
while ((sCurrentLine = br.readLine()) != null) {
fileOutputStream.write(("\n"+sCurrentLine).getBytes());
}
fileOutputStream.flush();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
fileOutputStream.close();
if (br != null)
br.close();
if (fr != null)
fr.close();
new File(newFileName).renameTo(new File(newFileName.replace("#", "")));
} catch (IOException ex) {
ex.printStackTrace();
}
}
}