I want to use Deflater and Inflater (NOT DeflaterOutputStream and InflaterInputStream) to compress files. The problem is that deflater stops working after mentioned buffer size in this case is 1024. I am using the following code:
public class CompressionUtils {
static String deflateInput = "pic.jpg";
static String deflateOutput = "picDeflate.raw";
static String inflateOutput = "picInflate.jpg";
public static void compress() throws IOException {
Deflater deflater = new Deflater();
byte[] data = new byte[1024];
FileInputStream in = new FileInputStream(new File(deflateInput));
FileOutputStream out = new FileOutputStream(new File(deflateOutput));
long readBytes = 0;
while ((readBytes = in.read(data, 0, 1024)) != -1) {
deflater.setInput(data);
deflater.finish();
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int count = deflater.deflate(buffer); // returns the generated code... index
out.write(buffer, 0, count);
}
}
}
public static void decompress() throws IOException, DataFormatException {
Inflater inflater = new Inflater();
byte[] data = new byte[1024];
FileInputStream in = new FileInputStream(new File(deflateOutput));
FileOutputStream out = new FileOutputStream(new File(inflateOutput));
long readBytesCount = 0;
long readCompressedBytesCount = 0;
long readBytes = 0;
while ((readBytes = in.read(data, 0, 1024)) != -1) {
readBytesCount = readBytesCount + readBytes;
inflater.setInput(data);
byte[] buffer = new byte[1024];
while (!inflater.finished()) {
int count = inflater.inflate(buffer);
System.out.println("Remaining: " + inflater.getRemaining());
out.write(buffer, 0, count);
}
}
System.out.println("readBytesCount: " + readBytesCount);
}
public static void main(String[] args) {
System.out.println("Operation started");
try {
compress();
} catch (Exception e) {
e.printStackTrace();
}
System.out.println("Operation ended");
}
}
And this is output (in windows) of dir:
01-04-2018 16:52 220,173 pic.jpg
28-04-2018 20:50 943 picDeflate.raw
28-04-2018 20:28 1,024 picInflate.jpg
Why does the compress code stops after reading 1024 bytes?
finish() is only for when you're finished. It is the last thing called after you have provided all of the input data to the object.
Related
I have a problem with the time it requires for .gz files to get uncompressed using the following code:
import java.io.*;
import java.util.zip.*;
public class UnGunzipClass{
public static boolean ungunzip(String compressedFile, String decompressedFile){
try{
// in
FileInputStream fileIn = new FileInputStream(compressedFile);
GZIPInputStream gZipIn = new GZIPInputStream(fileIn);
BufferedInputStream in = new BufferedInputStream(gZipIn);
// out
FileOutputStream fileOut = new FileOutputStream(decompressedFile);
BufferedOutputStream out = new BufferedOutputStream(fileOut);
int n = 0;
int len = 1024*1024*1024;
byte[] buffer = new byte[len];
while((n = in.read(buffer,0,len)) > 0){
out.write(buffer,0,n);
}
gZipIn.close();
fileOut.close();
return true;
} catch (IOException e){
e.printStackTrace();
return false;
}
}
}
Note: files are up to 100MB, but it is still taking me tens of minutes to run, so I am trying to get something faster. Speed is good :)
You created your BufferedInputStream from the GZIPInputStream, for performance you would do that in the reverse order. Also, I suggest you shrink your buffer size (and use your buffered streams). Finally, I would use a try-with-resources Statement and that might look something like
public static boolean ungunzip(String compressedFile,
String decompressedFile) {
final int BUFFER_SIZE = 32768;
try (InputStream in = new GZIPInputStream(new BufferedInputStream(
new FileInputStream(compressedFile)), BUFFER_SIZE);
OutputStream out = new BufferedOutputStream(
new FileOutputStream(decompressedFile), BUFFER_SIZE)) {
int n = 0;
byte[] buffer = new byte[BUFFER_SIZE];
while ((n = in.read(buffer, 0, BUFFER_SIZE)) > 0) {
out.write(buffer, 0, n);
}
return true;
} catch (IOException e) {
e.printStackTrace();
}
return false;
}
I have a class that compresses and decompresses a byte array;
public class Compressor
{
public static byte[] compress(final byte[] input) throws IOException
{
try (ByteArrayOutputStream bout = new ByteArrayOutputStream();
GZIPOutputStream gzipper = new GZIPOutputStream(bout))
{
gzipper.write(input, 0, input.length);
gzipper.close();
return bout.toByteArray();
}
}
public static byte[] decompress(final byte[] input) throws IOException
{
try (ByteArrayInputStream bin = new ByteArrayInputStream(input);
GZIPInputStream gzipper = new GZIPInputStream(bin))
{
// Not sure where to go here
}
}
}
How do I decompress the input and return a byte array?
Note: I don't want to do any conversion to strings because of character encoding issues.
your missing code will be something like
byte[] buffer = new byte[1024];
ByteArrayOutputStream out = new ByteArrayOutputStream();
int len;
while ((len = gzipper.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
gzipper.close();
out.close();
return out.toByteArray();
i'm trying to unGzip and unTar an inputStream in java , i have those methods :
public InputStream unTar(InputStream in) throws IOException {
TarInputStream myTarStream = new TarInputStream(in);
TarEntry entry = myTarStream.getNextEntry();
InputStream input = null;
while (entry != null) {
ByteArrayOutputStream output = new ByteArrayOutputStream();
byte[] buff = new byte[1024];
int read;
do {
read = myTarStream.read(buff);
if (read != -1) {
output.write(buff, 0, read);
}
} while (read != -1);
output.flush();
input = new ByteArrayInputStream(output.toByteArray());
entry = myTarStream.getNextEntry();
}
myTarStream.close();
return input;
}
public InputStream unGzipIt(InputStream in) {
byte[] buffer = new byte[1024];
InputStream outGZIPStream = null;
try {
ByteArrayOutputStream bytesOutput = new ByteArrayOutputStream();
GZIPInputStream gis = new GZIPInputStream(in);
int len;
while ((len = gis.read(buffer)) > 0) {
bytesOutput.write(buffer, 0, len);
}
in.close();
bytesOutput.close();
outGZIPStream = new ByteArrayInputStream(bytesOutput.toByteArray());
} catch (IOException e) {
e.printStackTrace();
}
return outGZIPStream;
}
the problem is when i passed an inputStream from a file in my local , it working,
but when i passed the inputStream from my server response , it doesn't work .
should i use reset and mark ? any help ? thank you
this i s how i'm getting the inputStream :
public InputStream getFolder(#PathParam("id") String envId, #PathParam("appName") String appName, #PathParam("imageType") String imageType,
#QueryParam("folderPath") String folderPath) throws EnvAutomationException, IOException {
Environment env = Envs.getEnvironmentManager().findEnvironment(envId);
ApplicationInstance appInst = env.getApplicationInstance(appName);
Container container = appInst.getContainer(imageType);
InputStream folderData = Envs.getContainerizationManager().getFolder(container, folderPath);
ByteArrayOutputStream output = new ByteArrayOutputStream();
byte[] buff = new byte[1024];
int size = 0;
int read;
do {
read = folderData.read(buff);
if (read != -1) {
size = size + read;
output.write(buff, 0, read);
}
} while (read != -1);
output.flush();
byte[] bo = output.toByteArray();
InputStream input = new ByteArrayInputStream(bo);
InputStream inputGZIP = gzipIt(input);
return inputGZIP;
}
since it's a .tar file , i gizip it and this is the method to gzip;
public InputStream gzipIt(InputStream source) {
byte[] buffer = new byte[1024];
InputStream outGZIPStream = null;
try {
ByteArrayOutputStream bytesOutput = new ByteArrayOutputStream();
GZIPOutputStream gzos = new GZIPOutputStream(bytesOutput);
int len;
int size = 0;
while ((len = source.read(buffer)) > 0) {
gzos.write(buffer, 0, len);
size = size + len;
}
source.close();
gzos.close();
byte[] bo = bytesOutput.toByteArray();
outGZIPStream = new ByteArrayInputStream(bytesOutput.toByteArray());
logger.info("folder tar size :" + size + " ; folder gzip size " + bo.length);
} catch (IOException e) {
e.printStackTrace();
}
return outGZIPStream;
}
I know that there's a way of converting a file to byte array in chunks, here's a sample code:
InputStream inputStream = new FileInputStream(videoFile);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] b = new byte[1024];
int bytesRead =0;
while ((bytesRead = inputStream.read(b)) != -1)
{
bos.write(b, 0, bytesRead);
}
I'm looking for the opposite: a way of converting a byte array into a file in chunks. I didn't find any example of doing it in chunks.
You just have to use either the write(byte[]) or write(byte[],int,int) methods from the FileOutputStream class.
byte[] to file:
FileOutputStream fop = null; File file;
try {
file = new File(filePath);
fop = new FileOutputStream(file, true);
fop.write(chunk);
fop.flush();
fop.close();
System.out.println("Done");
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fop != null) {
fop.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
Try this for file to byte[]:
InputStream is = new FileInputStream(file);
int length = (int) file.length();
int take = 262144;//size of your chunk
byte[] bytes = new byte[take];
int offset=0;
int a = 0;
do {
a = is.read(bytes, 0, take);
offset += a;
//And you can add here each chunk created in to a list, etc, etc.
//encode to base 64 this is extra :)
String str = Base64.encodeToString(bytes, Base64.DEFAULT);
} while (offset < length);=
is.close();
is=null;
Consider generalizing the problem.
This method copies data in chunks:
public static <T extends OutputStream> T copy(InputStream in, T out)
throws IOException {
byte[] buffer = new byte[1024];
for (int r = in.read(buffer); r != -1; r = in.read(buffer)) {
out.write(buffer, 0, r);
}
return out;
}
This can then be used in both reading to and from byte arrays:
try (InputStream in = new FileInputStream("original.txt");
OutputStream out = new FileOutputStream("copy.txt")) {
byte[] contents = copy(in, new ByteArrayOutputStream()).toByteArray();
copy(new ByteArrayInputStream(contents), out);
}
I am developing an Android App to send a file via bluetooth to a java server using the BlueCove library version 2.1.0 based on this snippet. At the beginning everything looks fine, but the file will not transfered completly. Only about 7KB of 35KB.
Android
private void sendFileViaBluetooth(byte[] data){
OutputStream outStream = null;
BluetoothDevice device = btAdapter.getRemoteDevice(address);
btSocket = device.createRfcommSocketToServiceRecord(MY_UUID);
btSocket.connect();
try {
outStream = btSocket.getOutputStream();
outStream.write( data );
outStream.write("end of file".getBytes());
outStream.flush();
} catch (IOException e) {
} finally{
try {
outStream.close();
btSocket.close();
device = null;
} catch (IOException e) {
}
}
}
PC Server
InputStream inStream = connection.openInputStream();
byte[] buffer = new byte[1024];
File f = new File("d:\\temp.jpg");
FileOutputStream fos = new FileOutputStream (f);
InputStream bis = new BufferedInputStream(inStream);
int bytes = 0;
boolean eof = false;
while (!eof) {
bytes = bis.read(buffer);
if (bytes > 0){
int offset = bytes - 11;
byte[] eofByte = new byte[11];
eofByte = Arrays.copyOfRange(buffer, offset, bytes);
String message = new String(eofByte, 0, 11);
if(message.equals("end of file")) {
eof = true;
} else {
fos.write (buffer, 0, bytes);
}
}
}
fos.close();
connection.close();
I tried already to split the byte array before writing:
public static byte[][] divideArray(byte[] source, int chunksize) {
byte[][] ret = new byte[(int)Math.ceil(source.length / (double)chunksize)][chunksize];
int start = 0;
for(int i = 0; i < ret.length; i++) {
ret[i] = Arrays.copyOfRange(source,start, start + chunksize);
start += chunksize ;
}
return ret;
}
private void sendFileViaBluetooth(byte[] data){
[...]
byte[][] chunks = divideArray(data, 1024);
for (int i = 0; i < (int)Math.ceil(data.length / 1024.0); i += 1) {
outStream.write( chunks[i][1024] );
}
outStream.write("end of file".getBytes());
outStream.flush();
[...]
}
Every help or ideas are appreciated.
You don't need any of this. The canonical way to copy a stream in Java is this:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
out.close();
Same at both ends. TCP/IP will do all the chunking for you. All you need to do is cope correctly with varying size reads, which this code does.