I have a raw string which is base64 compressed with zlib coming from the server and I need to convert it into JSON using zlib decompression. Here is the code I am using:
public static void decompress(String compressed) throws Exception {
try {
Inflater inflater = new Inflater(true);
inflater.setInput(compressed.getBytes());
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(compressed.getBytes().length);
byte[] buffer = new byte[1024];
while (!inflater.finished()) {
int count = inflater.inflate(buffer);
outputStream.write(buffer, 0, count);
}
outputStream.close();
byte[] output = outputStream.toByteArray();
String str = new String(output, StandardCharsets.UTF_8);
Logger.e("STR", str);
} catch (Exception e) {
e.printStackTrace();
}
}
However, I am getting this error while decompressing:
java.util.zip.DataFormatException: invalid code lengths set
at line int count = inflater.inflate(buffer);
I don't understand what I am doing wrong. Please help me fix this issue.
Related
As per my requirement I want to create checksum value using SHA-256, from InputStream,
As below:
private InputStream createZipInput(List<ResponsePack> aList, byte[] manifestData)
{
final int bufferSize = 2048;
byte buffer[] = new byte[bufferSize];
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
ZipOutputStream zipFileToSend = new ZipOutputStream(byteStream);
LOG.trace("Compressing the file {}");
try
{
for (ResponsePack info : aList)
{
ByteArrayOutputStream byteStreamCheckSum = new ByteArrayOutputStream();
ZipOutputStream zipFileToSendCheckSum = new ZipOutputStream(byteStreamCheckSum);
zipFileToSend.putNextEntry(new ZipEntry(info.getFileName()));
zipFileToSendCheckSum.putNextEntry(new ZipEntry(info.getFileName()));
InputStream in = info.getFileContentStream();
int length;
while ((length = in.read(buffer)) >= 0)
{
zipFileToSend.write(buffer, 0, length);
zipFileToSendCheckSum.write(buffer, 0, length);
}
zipFileToSend.closeEntry();
zipFileToSendCheckSum.closeEntry();
String checksum = validChecksum(byteStreamCheckSum.toByteArray());
LOG.error("Checksum {}", checksum);
zipFileToSendCheckSum.flush();
zipFileToSendCheckSum.close();
}
zipFileToSend.close();
}
catch (IOException e)
{
return e;
}
return new ByteArrayInputStream(byteStream.toByteArray());
}
private static String validChecksum(byte[] dataCopy)
{
printLOG("Byte Array Size {}", dataCopy.length);
try (ZipInputStream zipInputStream = new ZipInputStream(new ByteArrayInputStream(dataCopy)))
{
ZipEntry zipEntry;
MessageDigest digest = DigestUtils.getSha256Digest();
DWriter writer = new DWriter(digest);
while ((zipEntry = zipInputStream.getNextEntry()) != null)
{
org.apache.commons.io.output.ByteArrayOutputStream dest = StreamUtils.extractFileAsByteArrayStream(zipInputStream);
LOG.error("CheckSum Entity creating");
if(zipEntry != null)
{
printLOG("CheckSum Entity file Name {}", zipEntry.getName());
}
LOG.error("Byte array size {}", dest.toByteArray().length);
writer.write(dest.toByteArray());
dest.flush();
dest.close();
}
if (writer.getChecksum() != null)
{
return writer.getChecksum();
}
else
{
return "";
}
}
catch (Exception e)
{
printLOG("Exception encountered while creating checksum: {}", e.getMessage());
return "";
}
}
static class DWriter
{
private final MessageDigest myDigest;
DWriter(MessageDigest digest)
{
myDigest = digest;
}
public void write(byte[] data)
{
myDigest.update(data);
}
public String getChecksum()
{
return new String(Hex.encodeHex(myDigest.digest()));
}
}
But the problem is when I checked the log, found byte array contains value but still checksum always creating for empty string, as below
Byte Array Size 3948
CheckSum Entity creating
CheckSum Entity file Name 20200911104812526.json
Byte array size 20854
Checksum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Help me where I am doing wrong, due to which I am getting checksum for an empty string
I'm not sure what's wrong with the code but it seems overly complicated: you're writing the input into a zipped stream and the dezip it in memory to read it again.
You don't need to do that: storing the input in a (non-zipped) byte array should be enough.
I think you need to make sure that in.read() works as intended (and that there's actually some data to read).
You get the checksum for a null input and your zip entry is also empty, so it looks like the input was empty. Add some logs or use a debugger to investigate what's happening.
private InputStream createZipInput(List<ResponsePack> aList, byte[] manifestData) {
final int bufferSize = 2048;
byte buffer[] = new byte[bufferSize];
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
ZipOutputStream zipFileToSend = new ZipOutputStream(byteStream);
LOG.trace("Compressing the file {}");
try {
for (ResponsePack info : aList) {
ByteArrayOutputStream byteStreamCheckSum = new ByteArrayOutputStream();
zipFileToSend.putNextEntry(new ZipEntry(info.getFileName()));
InputStream in = info.getFileContentStream();
int length;
while ((length = in.read(buffer)) != -1) {
zipFileToSend.write(buffer, 0, length);
byteStreamCheckSum.write(buffer, 0, length);
}
zipFileToSend.closeEntry();
MessageDigest digest = DigestUtils.getSha256Digest();
digest.update(byteStreamCheckSum.toByteArray());
String checksum = new String(Hex.encodeHex(digest.digest()));
LOG.error("Checksum {}", checksum);
}
zipFileToSend.close();
} catch (IOException e) {
throw e;
}
return new ByteArrayInputStream(byteStream.toByteArray());
I am deflating a string with java with this function:
protected static byte[] Compress(String source) {
try {
// deficne start time
long startTime = System.currentTimeMillis();
//get bytes
byte[] bytes = source.getBytes("UTF-8");
Deflater deflater = new Deflater();
deflater.setInput(bytes);
deflater.finish();
ByteArrayOutputStream bos = new ByteArrayOutputStream(bytes.length);
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int bytesCompressed = deflater.deflate(buffer);
bos.write(buffer, 0, bytesCompressed);
}
try {
//close the output stream
bos.close();
} catch (IOException ioe) {
System.out.println("Error while closing the stream : " + ioe);
}
//get the compressed byte array from output stream
byte[] compressedArray = bos.toByteArray();
return compressedArray;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
now, I want to inflate that string with PHP with this function:
$uncompressed = gzinflate($filepath);
echo $uncompressed;
die();
but I got data error in php file, how can I solve this problem?
the error message is :
Warning: gzinflate(): data error
regards
I'm trying (In Java) to get a file document then convert it to bitArray afterwards to a String of similar representation, then back to the original bit array and finally to the original final document.
Here is my code, But the file generated is not viewable in this case the image.
try {
File file = new File("C:/Users/dkimigho/Downloads/kenyapowerlogo.jpg");
FileInputStream fis = null;
try {
fis = new FileInputStream(file);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//create FileInputStream which obtains input bytes from a file in a file system
//FileInputStream is meant for reading streams of raw bytes such as image data. For reading streams of characters, consider using FileReader.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for (int readNum; (readNum = fis.read(buf)) != -1;) {
//Writes to this byte array output stream
bos.write(buf, 0, readNum);
System.out.println("read " + readNum + " bytes,");
}
} catch (IOException ex) {
Logger.getLogger(ARRAYBITStoPDF.class.getName()).log(Level.SEVERE, null, ex);
}
byte[] bytes = bos.toByteArray();
System.out.println("byte1");
for (int i = 0; i < bytes.length; i++) {
System.out.print(bytes[i]);
}
//We have the bytes now convert to String
String stringbytearray=new String(bytes);
System.out.println("stringbytearray: "+stringbytearray);
//We have the bytes now convert to String
byte[] content = stringbytearray.getBytes();
System.out.println("byte2");
for (int i = 0; i < content.length; i++) {
System.out.print(content[i]);
}
int size = bytes.length;
InputStream isfilecontent = null;
byte[] b = new byte[size];
isfilecontent = new ByteArrayInputStream(content);
//writing the downloaded data into a PDF file
FileOutputStream fileOutputpdf = new FileOutputStream("C:/Users/dkimigho/Downloads/mykenyapowerlogo.jpg");
/* use binary I/O to prevent line based operation messing with the encoding.*/
byte[] buf2 = new byte[2048];
int b_read = 0;
while ( (b_read = isfilecontent.read(buf2)) > 0) {
fileOutputpdf.write(buf2, 0, b_read);
}
fileOutputpdf.flush();
//closed the output stream
fileOutputpdf.close();
} catch (IOException e) {
// handle IOException
System.out.println(e.getMessage());
}
Any help on pointing what I'm doing wrong? A correction of my code to a working one may be important.
I found an answer one has to use JAVA 8 java.util.Base64 to encode and decode the bytes without loosing information on the document. I hope it will be of help to someone.
/*
* 1. How to convert an image file to byte array?
*/
try {
File file = new File("C:/Users/qwerty/Downloads/factura.pdf");
FileInputStream fis = null;
try {
fis = new FileInputStream(file);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//create FileInputStream which obtains input bytes from a file in a file system
//FileInputStream is meant for reading streams of raw bytes such as image data. For reading streams of characters, consider using FileReader.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for (int readNum; (readNum = fis.read(buf)) != -1;) {
//Writes to this byte array output stream
bos.write(buf, 0, readNum);
// System.out.println("read " + readNum + " bytes,");
}
} catch (IOException ex) {
Logger.getLogger(ARRAYBITStoPDF.class.getName()).log(Level.SEVERE, null, ex);
}
byte[] bytes = bos.toByteArray();
bos.close(); // should be inside a finally block
//We have the bytes now convert to String
// ENCODING
String encodedDoc= Base64.getEncoder().encodeToString(bytes);
System.out.println(encodedDoc);
// DECODING
int size = bytes.length;
InputStream isfilecontent = null;
//byte[] b = new byte[size];
isfilecontent = new ByteArrayInputStream(Base64.getDecoder().decode(encodedDoc));
//writing the downloaded data into a PDF file
FileOutputStream fileOutputpdf = new FileOutputStream("C:/Users/qwerty/Downloads/myfactura.pdf");
/* use binary I/O to prevent line based operation messing with the encoding.*/
byte[] buf2 = new byte[2048];
int b_read = 0;
while ( (b_read = isfilecontent.read(buf2)) > 0) {
fileOutputpdf.write(buf2, 0, b_read);
}
fileOutputpdf.flush();
//closed the output stream
fileOutputpdf.close();
} catch (IOException e) {
// handle IOException
System.out.println(e.getMessage());
}
I'm using jpountz LZ4 to try and compress files and I want to read in and output files using Java file input and output streams. I've tried to find a solution online but theres nothing, I found a previous stackoverflow question on how to implement LZ4 correctly and I've taken that and tried to modify it to use the streams, but I'm not sure if this is correct or if it's even working.
When running the compression on a text file it outputs a file which has some characters missing or replaced with symbols
ðHello world Heðo world Hello ðrld Hello worlðHello worl
but when running it with a image file it throws an out of bounds error. I've also been unable to get decompression to work as it throws a Error decoding offset 3 of input buffer.
Here is my code any help would be appreciated thanks
public void LZ4Compress(InputStream in, OutputStream out){
int noBytesRead = 0; //number of bytes read from input
int noBytesProcessed = 0; //number of bytes processed
try {
while ((noBytesRead = in.read(inputBuffer)) >= 0) {
noBytesProcessed = inputBuffer.length;
decompressedLength = inputBuffer.length;
outputBuffer = compress(inputBuffer, decompressedLength);
out.write(outputBuffer, 0, noBytesRead);
}
out.flush();
in.close();
out.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void LZ4decompress(InputStream in, OutputStream out){
int noBytesRead = 0; //number of bytes read from input
try {
while((noBytesRead = in.read(inputBuffer)) >= 0){
noBytesProcessed = inputBuffer.length;
outputBuffer = decompress(inputBuffer);
out.write(outputBuffer, 0, noBytesRead);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static byte[] compress(byte[] src, int srcLen) {
decompressedLength = srcLen;
int maxCompressedLength = compressor.maxCompressedLength(decompressedLength);
byte[] compressed = new byte[maxCompressedLength];
int compressLen = compressor.compress(src, 0, decompressedLength, compressed, 0, maxCompressedLength);
byte[] finalCompressedArray = Arrays.copyOf(compressed, compressLen);
return finalCompressedArray;
}
private static LZ4SafeDecompressor decompressor = factory.safeDecompressor();
public static byte[] decompress(byte[] finalCompressedArray) {
byte[] restored = new byte[finalCompressedArray.length];
restored = decompressor.decompress(finalCompressedArray, finalCompressedArray.length);
return restored;
}
So I solved my problem by using LZ4block input/output streams
public static void LZ4compress(String filename, String lz4file){
byte[] buf = new byte[2048];
try {
String outFilename = lz4file;
LZ4BlockOutputStream out = new LZ4BlockOutputStream(new FileOutputStream(outFilename), 32*1024*1024);
FileInputStream in = new FileInputStream(filename);
int len;
while((len = in.read(buf)) > 0){
out.write(buf, 0, len);
}
in.close();
out.close();
} catch (IOException e) {
}
}
public static void LZ4Uncompress(String lz4file, String filename){
byte[] buf = new byte[2048];
try {
String outFilename = filename;
LZ4BlockInputStream in = new LZ4BlockInputStream(new FileInputStream(lz4file));
FileOutputStream out = new FileOutputStream(outFilename);
int len;
while((len = in.read(buf)) > 0){
out.write(buf, 0, len);
}
in.close();
out.close();
} catch (IOException e) {
}
}
Looking only at code, I would say you are going wrong here:
outputBuffer = compress(inputBuffer, decompressedLength);
out.write(outputBuffer, 0, noBytesRead);
You have already trimmed outputBuffer in compress. Try:
out.write(outputBuffer);
How can i decompress a String that was zipped by PHP gzcompress() function?
Any full examples?
thx
I tried it now like this:
public static String unzipString(String zippedText) throws Exception
{
ByteArrayInputStream bais = new ByteArrayInputStream(zippedText.getBytes("UTF-8"));
GZIPInputStream gzis = new GZIPInputStream(bais);
InputStreamReader reader = new InputStreamReader(gzis);
BufferedReader in = new BufferedReader(reader);
String unzipped = "";
while ((unzipped = in.readLine()) != null)
unzipped+=unzipped;
return unzipped;
}
but it's not working if i i'm trying to unzip a PHP gzcompress (-ed) string.
PHP's gzcompress uses Zlib NOT GZIP
public static String unzipString(String zippedText) {
String unzipped = null;
try {
byte[] zbytes = zippedText.getBytes("ISO-8859-1");
// Add extra byte to array when Inflater is set to true
byte[] input = new byte[zbytes.length + 1];
System.arraycopy(zbytes, 0, input, 0, zbytes.length);
input[zbytes.length] = 0;
ByteArrayInputStream bin = new ByteArrayInputStream(input);
InflaterInputStream in = new InflaterInputStream(bin);
ByteArrayOutputStream bout = new ByteArrayOutputStream(512);
int b;
while ((b = in.read()) != -1) {
bout.write(b); }
bout.close();
unzipped = bout.toString();
}
catch (IOException io) { printIoError(io); }
return unzipped;
}
private static void printIoError(IOException io)
{
System.out.println("IO Exception: " + io.getMessage());
}
Try a GZIPInputStream. See this example and this SO question.
See
http://developer.android.com/reference/java/util/zip/InflaterInputStream.html
since the DEFLATE algorithm is gzip.