I am writing download servlet that reads a html file and writes to servletOutputStream, the problem right at the of the file transferred it is adding some garbage data any suggestions about this,
below is code I am using for this
int BUFFER_SIZE = 1024 * 8;
servOut = response.getOutputStream();
bos = new BufferedOutputStream(servOut);
fileObj = new File(file);
fileToDownload = new FileInputStream(fileObj);
bis = new BufferedInputStream(fileToDownload);
response.setContentType("application/text/html");
response.setHeader("ContentDisposition","attachment;filename="+dump+".html");
byte[] barray = new byte[BUFFER_SIZE];
while ((bis.read(barray, 0, BUFFER_SIZE)) != -1) {
bos.write(barray, 0, BUFFER_SIZE);
}
bos.flush();
bis.read returns the number of bytes read. You need to take that into account in your write call.
Something like:
int rd;
while ((rd=bis.read(...)) != -1) {
bos.write(..., rd);
}
The problem is with the following part of your code:
while ((bis.read(barray, 0, BUFFER_SIZE)) != -1) {
bos.write(barray, 0, BUFFER_SIZE);
}
You are always writing out a multiple of BUFFER_SIZE bytes, even if the size of your input isn't a multiple of BUFFER_SIZE. This results in garbage being written at the end of the last block.
You can fix it like so:
int read;
while ((read = bis.read(barray, 0, BUFFER_SIZE)) != -1) {
bos.write(barray, 0, read);
}
Related
we are using the below methods to save the inputstream into a file. but taking more time. Need some suggestion to optimise it. Any other method to save the input stream to line very fast.
InputStream stream = null;
stream = httpConnection.getInputStream();
File imageFile = new File(directoryPath, imageFileName);
FileOutputStream imageFileStream = new FileOutputStream(imageFile);
DigestOutputStream digestoutput = new DigestOutputStream(imageFileStream,
md);
byte[] imagebuffer = new byte[1024 * 1024];
long size = 0;
while (true) {
if(stream != null){
int bytesRead = stream.read(imagebuffer);
if (bytesRead < 0){break;
}
digestoutput.write(imagebuffer, 0, bytesRead);
}}
I decompress received GZIPped data to string. Problem when I have BUFFER_SIZE as 512 it breaks unicode characters at buffer limit points. As a result I get text with question marks. It happens with non latin letters.
...во и ��ргуме...
public static String decompress(byte[] compressed) throws IOException {
final int BUFFER_SIZE = 512;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
StringBuilder string = new StringBuilder();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
string.append(new String(data, 0, bytesRead));
}
gis.close();
is.close();
return string.toString();
}
The error is in the algorithm, assuming that the block being read ends (and starts) on a UTF-8 bytes sequence border.
So do it as follows:
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while ((bytesRead = gis.read(data)) != -1) {
baos.write(data, 0, bytesRead);
}
gis.close();
is.close();
return baos.toString("UTF-8");
You can wrap the GZIPInputStream into an InputStreamReader and read characters instead of bytes. By doing so, you don't have the problem of potentially invalid encodings at the buffer boundaries.
I was trying to flush data in a file in my local machine to response. But at some point I get an IndexOutOfBoundsException.
FileInputStream inputStream = new FileInputStream(downloadFile);
OutputStream outStream = response.getOutputStream();
byte[] buffer = new byte[4096];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer, 0, 4096)) != -1) {
outStream.write(buffer, 0, bytesRead);
}
inputStream.close();
outStream.close();
The above code is what I was trying. The downloadFile path given is correct and it works till the while loop. But then the IndexOutOfBoundsException occurs. I tried it with inputStream.read(buffer) but that didn't work.
Give code is working perfectlly; since there is no information is given regarding the response object I have modify the OutputStream to FileOutputStream; just to test.
Below code segment is working perfectly.
public class Test
{
public static void main(String args[]) throws IOException
{
FileInputStream inputStream = new FileInputStream("C:\\readme.txt");
FileOutputStream outputStream = new FileOutputStream("D:\\readme1.txt");
//OutputStream outStream = response.getOutputStream();
byte[] buffer = new byte[4096];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer, 0, 4096)) != -1) {
outputStream.write(buffer, 0, bytesRead);
}
inputStream.close();
outputStream.close();
}
}
I'm trying to handle large files by 10MB byte arrays at a time.
I'm trying to get the byte arrays one at a time (not get entire byte array for the huge file and split the byte array, after all the problem was due to memory)
This is what I have so far:
private byte[] readFile(File file, int offset) throws IOException
{
BufferedInputStream inStream = null;
ByteArrayOutputStream outStream = null;
byte[] buf = new byte[1048576];
int read = 0;
try
{
inStream = new BufferedInputStream(new FileInputStream(file));
outStream = new ByteArrayOutputStream();
long skipped = inStream.skip(offset);
read = inStream.read(buf);
if (read != -1)
{
outStream.write(buf, 0, read);
return outStream.toByteArray();
}
}
finally
{
if (inStream != null) {try {inStream.close();} catch (IOException e) {}}
if (outStream != null) {try {outStream.close();} catch (IOException e) {}}
}
return null;
the parameter offset will be in 10MB increments as well.
So the problem I'm having is that, even tho the skipped long variable gives me 1048576 bytes skipped, the second 10MB i'm suppose to receive from calling readFile(file, 1048576) is the same as the first byte array from the first 10MB. Thus it didn't really skip the first 10MB at all.
What's the problem here? Is there another way of implementing this idea?
Redesign the method. At present you are copying byte arrays like its going out of style: once from the buffer to the ByteArrayOutoutStream and again from there to the return value. So you need three of those at once. Change the signature of the method, so that the caller provides the byte array as well as the offset, and the stream, and have it return the count. In other words get rid of it altogether and just call FileInputStream.read(buffer, offset, length) from wherever you are calling this.
So as per user #EJP I revised the code to work efficiently. I am no longer copying to ByteArrayOutputStream as I realized that .toByteArray actually returns a copy of the read byte array and is very memory inefficient. I also only opens the stream once so the skipping would be unneeded.
int fileLength = (int) file.length();
byte[] buffer = new byte[fileLength < FILE_UPLOAD_CHUNK_SIZE ?
fileLength : FILE_UPLOAD_CHUNK_SIZE];
int bytesRead;
int readTotal = 0;
BufferedInputStream inStream = null;
try
{
inStream = new BufferedInputStream(new FileInputStream(file));
do
{
bytesRead = inStream.read(buffer, 0, buffer.length);
if (bytesRead == -1)
{
continue;
}
byte[] finalBuffer;
if (buffer.length > bytesRead)
{
finalBuffer = Arrays.copyOf(buffer, bytesRead);
}
else
{
finalBuffer = buffer;
}
uploadChunk(
finalBuffer,
mimeType,
uploadPath,
fileLength,
readTotal,
readTotal + bytesRead - 1);
readTotal += bytesRead;
} while (bytesRead != -1);
}
finally
{
if (inStream != null)
{
inStream.close();
}
}
The only blemish I have for this code is the way I have to make a new copy of the byte array when the last chunk is less than 10MB. There should be a more efficient way of doing it but this is working fine for me for now.
I am trying to download/resume file. Resume seems to work, but whole download brings the problem. After executing this code testfile is 5242845. But it should be 5242880! I opened this two files in the hex editor and figured out that testfile missing some bytes at the end (begining is okay). This is the code:
String url = "http://download.thinkbroadband.com/5MB.zip";
String DESTINATION_PATH = "/sdcard/testfile";
URLConnection connection;
connection = (HttpURLConnection) url.openConnection();
File file = new File(DESTINATION_PATH);
if (file.exists()) {
downloaded = (int) file.length();
connection.setRequestProperty("Range", "bytes=" + (file.length()) + "-");
}
connection.setDoInput(true);
connection.setDoOutput(true);
BufferedInputStream in = new BufferedInputStream(connection.getInputStream());
FileOutputStream fos = (downloaded == 0) ? new FileOutputStream(DESTINATION_PATH) : new FileOutputStream(DESTINATION_PATH, true);
BufferedOutputStream bout = new BufferedOutputStream(fos, 1024);
byte[] data = new byte[1024];
int x = 0;
int i = 0;
int lenghtOfFile = connection.getContentLength();
while ((x = in.read(data, 0, 1024)) != -1) {
i++;
bout.write(data, 0, x);
downloaded += x;
}
I think that the problem is here while ((x = in.read(data, 0, 1024)) != -1) {.
For example we have file 1030 bytes long. First write is good, bout.write(data,0,1024); but next time while ((x = in.read(data, 0, 1024)) != -1) { gets -1, because 1030-1024=6 bytes left. And we are trying to write 1024 bytes! I know it should not be so, but it seems that it is how I said. How can I figure this? Thanks.
bout.flush();
and/or
bout.close();
You need to close your BufferedOutputStream to ensure that all that is buffered is sent to the buffered OutputStream.
google told me, there is a "available" method of bufferedinputstream, so you can write like
(I´m not an java guru)
while (in.available() > 0)
{
x = in.read(data, 0, 1024);
bout.write(data, 0, x);
}