How can I convert DataInput to DataInputStream in java?
I need to know the size of the DataInput.
Since a stream, by definition, really has no begining or end and thus no fool proof way of knowing how much is available, you just have to read from the stream in fixed sized chunks. It almost sounds like you'd be better off with plain old .read() rather than readFully():
DataInputStream dis = new DataInputStream(...);
byte[] buf = new byte[1024];
int lastRead = 0;
do {
lastRead = dis.read(buf);
//do something with 'buf' here
} while (lastRead > 0);
You'll encounter difficulty when you want know how many bytes to be read. Simplest solution is to cast it to a ByteArrayInputStream and use it's available() method to get to know how many bytes are available for reading.
Following example worked for me
DataInput in = (...);
ByteArrayInputStream bis = (ByteArrayInputStream) in;
byte[] buffer = new byte[bis.available()];
in.readFully(buffer);
//use buffer as your wish
Related
This question already has an answer here:
java reader vs. stream
(1 answer)
Closed 7 years ago.
I am going over input output section and there are so much classes and interfaces that I am confused a little bit.
As documentation says InputStream is a byte based stream whereas Reader is character-based stream. But as I understood correctly the only difference between them is that Reader reads two bytes per time instead of one byte at a time as InputStream does.
Therefore I don't understand is there any difference:
Using InputStream
byte[] bytes = new byte[2];
InputStream in = new FileInputStream("input.txt");
int bytesRead = in.read(bytes);
while(bytesRead != -1) {
doSomethingWithData(data);
bytesRead = inputstream.read(data);
}
Using Reader
Reader reader = new InputStreamReader(new FileInputStream("input.txt"));
int data = reader.read();
while (data != -1) {
doSomethingWithData(data);
data = reader.read();
}
I am really confusing about these aspects. Explain please in details. Thanks.
You can use a Reader to read text data. And it's supports some character encoding like - ISO, UTF-8. If you want to read a text file with some encoding then you can use Readers like - BufferedReader, StringReader etc.
And you can use Stream (InputStream, OutputStream) to manipulate binary data. For example you want to read a image file then you can use FileInputStream and when you want to save it to disk then you can use FileOutputStream.
I have a serialization code that can serialize an object to a bytebuffer. I want to write the length of the buffer first to the stream followed by the bytebuffer itself. Here is how I am writing to the outputStream:
MyObject = new MyObject();
//fill in MyObject
...
DataOutputStream out = new DataOutputStream(new FileOutputStream("a.txt"));
ByteBuffer buffer = MySerializer.encode(myObject);
int length = buffer.remaining();
out.write(length);
WritableByteChannel channel = Channels.newChannel(out);
channel.write(buffer);
out.close();
I verified this code and it seems to be working fine. But when I try to deserialize, I am not able to do it correctly. Here is the snippet of my deserializer code :
DataInputStream in = new DataInputStream(new FileInputStream("a.txt"));
int objSize = in.readInt();
byte[] byteArray = new byte[objSize];
...
The problem is that the length is not being read properly from stream.
Can anyone help me figure out what am I missing here?
write writes a byte. readInt reads 4 bytes and combines them to make an int.
You probably want to write the length with writeInt (which splits the int into 4 bytes and writes those).
Could I bond readBuf to DataInputStream, I will read some bytes to readBuf, and I will read int and other data from DataInputStream which bond to readBuf. Can I do below code?
byte[] readBuf = new byte[MAX_BYTE_SIZE ];
DataInputStream dis = new DataInputStream(new ByteArrayInputStream(readBuf));
writeBytes( UartCmd[1] );
i_ReadLength = readBytes(readBuf);
uc_Index = dis.readInt();
writeBytes( i_WritedLength );
i_ReadLength = readBytes(readBuf);
uc_Index = dis.readInt();
thinks all your help.
I suggest java.nio.ByteBuffer
...
uc_Index = ByteBuffer.wrap(readBuf).getInt();
writeBytes( i_WritedLength );
i_ReadLength = readBytes(readBuf);
uc_Index = ByteBuffer.wrap(readBuf).getInt();
You already have "bonded" a byte[] buffer to an input stream with this line:
DataInputStream dis = new DataInputStream(new ByteArrayInputStream(readBuf));
That should work with dis.readInt(); just fine. Loop over it for as many int's as you expect to be in the buffer.
I can't tell you what's going to happen with readBytes() and writeBytes() since you haven't defined them. The name i_ReadLength seems to mean something completely different than what I'd expect a method named readBytes() to return. Your compiler won't care but us humans do.
If you're hoping to be able to look inside readBuf and tell how many int's have been added to it you're likely going to be disappointed. A byte array of int's is not going to be null terminated like a string would be. You MUST keep track of how much of the readBuf has been initialized with integers somewhere or you will end up reading garbage (data that was never written and could have any value).
What you could do is make the first int in the byte array the count of all the other int's in the array. Once you read that you know how many times to loop. Make sure to set it to the correct value when you write it. Also don't write in to many int's and overflow the byte array. Remember byte's and int's are different sizes.
If that's what you've done don't reinvent the wheel. Do it like this:
byte[] readBuf = getSizePrefixedByteArrayOfInts();
DataInputStream dis = new DataInputStream(new ByteArrayInputStream(readBuf));
//writeBytes( UartCmd[1] ); //Seriously, I've no idea what this does
int i_ReadLength = dis.readInt();
int[] readInts = new int[i_ReadLength];
for(int i = 0; i < i_ReadLength; i++) {
readInts[i] = dis.readInt();
}
Critical to understand here is that this code expects the length counter itself to NOT be counted. If the counter is the only int initialized in the buffer then it's value should be zero. So here's test some code that does that:
private byte[] getSizePrefixedByteArrayOfInts() {
ByteArrayOutputStream baos = null;
DataOutputStream dos = null;
int[] buf = {65, 66, 67, 68, 69, 70, 71};
try{
// create byte array output stream
baos = new ByteArrayOutputStream();
// create data output stream
dos = new DataOutputStream(baos);
// write number of other ints in array
dos.write(buf.length);
// write to the stream from integer array
for(int i: buf)
{
dos.write(i);
}
// flushes bytes to underlying output stream
dos.flush();
return baos.toByteArray();
}catch(Exception e){
// if any error occurs
e.printStackTrace();
}finally{
// releases all system resources from the streams
if(baos!=null)
baos.close();
if(dos!=null)
dos.close();
}
}
My thanks to tutorialspoint for the bulk of getSizePrefixedByteArrayOfInts() code.
I'm working with Amazon S3 and would like to upload an InputStream (which requires counting the number of bytes I'm sending).
public static boolean uploadDataTo(String bucketName, String key, String fileName, InputStream stream) {
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buffer = new byte[1];
try {
while (stream.read(buffer) != -1) { // copy from stream to buffer
out.write(buffer); // copy from buffer to byte array
}
} catch (Exception e) {
UtilityFunctionsObject.writeLogException(null, e);
}
byte[] result = out.toByteArray(); // we needed all that just for length
int bytes = result.length;
IO.close(out);
InputStream uploadStream = new ByteArrayInputStream(result);
....
}
I was told copying a byte at a time is highly inefficient (obvious for large files). I can't make it more because it will add padding to the ByteArrayOutputStream, which I can't strip out. I can strip it out from result, but how can I do it safely? If I use an 8KB buffer, can I just strip out the right most buffer[i] == 0? Or is there a better way to do this? Thanks!
Using Java 7 on Windows 7 x64.
You can do something like this:
int read = 0;
while ((read = stream.read(buffer)) != -1) {
out.write(buffer, 0, read);
}
stream.read() returns the number of bytes that have been written into buffer. You can pass this information to the len parameter of out.write(). So you make sure that you write only the bytes you have read from the stream.
Use Jakarta Commons IOUtils to copy from the input stream to the byte array stream in a single step. It will use an efficient buffer, and not write any excess bytes.
If you want efficiency you could process the file as you read it. I would replace uploadStream with stream and remove the rest of the code.
If you need some buffering you can do this
InputStream uploadStream = new BufferedInputStream(stream);
the default buffer size is 8 KB.
If you want the length use File.length();
long length = new File(fileName).length();
I have this piece of code which I'm hoping will be able to tell me how much data I have downloaded (and soon put it in a progress bar), and then parse the results through my Sax Parser. If I comment out basically everything above the //xr.parse(new InputSource(request.getInputStream())); line and swap the xr.parse's over, it works fine. But at the moment, my Sax parser tells me I have nothing. Is it something to do with is.read (buffer) section?
Also, just as a note, request is a HttpURLConnection with various signatures.
/*Input stream to read from our connection*/
InputStream is = request.getInputStream();
/*we make a 2 Kb buffer to accelerate the download, instead of reading the file a byte at once*/
byte [ ] buffer = new byte [ 2048 ] ;
/*How many bytes do we have already downloaded*/
int totBytes,bytes,sumBytes = 0;
totBytes = request.getContentLength () ;
while ( true ) {
/*How many bytes we got*/
bytes = is.read (buffer);
/*If no more byte, we're done with the download*/
if ( bytes <= 0 ) break;
sumBytes+= bytes;
Log.v("XML", sumBytes + " of " + totBytes + " " + ( ( float ) sumBytes/ ( float ) totBytes ) *100 + "% done" );
}
/* Parse the xml-data from our URL. */
// OLD, and works if comment all the above
//xr.parse(new InputSource(request.getInputStream()));
xr.parse(new InputSource(is))
/* Parsing has finished. */;
Can anyone help me at all??
Kind regards,
Andy
'I could only find a way to do that
with bytes, unless you know another
method?'.
But you haven't found a method. You've just written code that doesn't work. And you don't want to save the input to a String either. You want to count the bytes while you're parsing them. Otherwise you're just adding latency, i.e. wasting time and slowing everything down. For an example of how to do it right, see javax.swing.ProgressMonitorInputStream. You don't have to use that but you certainly do have to use a FilterInputStream of some sort, probaby one you write yourself, that is wrapped around the request input stream and passed to the parser.
Your while loop is consuming the input stream and leaving nothing for the parser to read.
For what you're trying to do, you might want to look into implementing a FilterInputStream subclass wrapping the input stream.
You are building an InputStream over another InputStream that consumes its data before.
If you want to avoid reading just single bytes you could use a BufferedInputStream or different things like a BufferedReader.
In any case it's better to obtain the whole content before parsing it! Unless you need to dynamically parse it.
If you really want to keep it on like you are doing you should create two piped streams:
PipedOutputStream pipeOut = new PipedOutputStream();
PipedInputStream pipeIn = new PipedInputStream();
pipeIn.connect(pipeOut);
pipeOut.write(yourBytes);
xr.parse(pipeIn);
Streams in Java, like their name suggest you, doesn't have a precise dimension neither you know when they'll finish so whenever you create an InputStream, if you read from them you cannot then pass the same InputStream to another object because data is already being consumed from the former one.
If you want to do both things (downloading and parsing) together you have to hook between the data received from the HTTPUrlConncection you should:
first know the length of the data being downloaded, this can be obtained from HttpUrlConnection header
using a custom InputStream that decorates (this is how streams work in Java, see here) updading the progressbar..
Something like:
class MyInputStream extends InputStream
{
MyInputStream(InputStream is, int total)
{
this.total = total;
}
public int read()
{
stepProgress(1);
return super.read();
}
public int read(byte[] b)
{
int l = super.read(b);
stepProgress(l);
return l;
}
public int read(byte[] b, int off, int len)
{
int l = super.read(b, off, len);
stepProgress(l);
return l
}
}
InputStream mis= new MyInputStream(request.getInputStream(), length);
..
xr.parse(mis);
You can save your data in a file, and then read them out.
InputStream is = request.getInputStream();
if(is!=null){
File file = new File(path, "someFile.txt");
FileOutputStream os = new FileOutputStream(file);
buffer = new byte[2048];
bufferLength = 0;
while ((bufferLength = is.read(buffer)) > 0)
os.write(buffer, 0, bufferLength);
os.flush();
os.close();
XmlPullParserFactory factory = XmlPullParserFactory.newInstance();
factory.setNamespaceAware(true);
XmlPullParser xpp = factory.newPullParser();
FileInputStream fis = new FileInputStream(file);
xpp.setInput(new InputStreamReader(fis));
}