Hi i am just wondering is it possible to set the deflater used in gzip output stream to use HUFFMAN_ONLY, i have it working with my own deflate method.
public static byte[] deflate(byte[] data) throws IOException {
Deflater deflater = new Deflater();
deflater.setInput(data);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(data.length);
deflater.finish();
deflater.setLevel(Deflater.BEST_COMPRESSION); //*****THESE 2 LINES HERE
deflater.setStrategy(Deflater.HUFFMAN_ONLY); // *******
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int count = deflater.deflate(buffer); // returns the generated code... index
outputStream.write(buffer, 0, count);
}
outputStream.close();
byte[] output = outputStream.toByteArray();
return output;
}
so basically i want to know how to set the deflater used in my gzip method to the same as the deflater above when i use the lines:
deflater.setLevel(Deflater.BEST_COMPRESSION);
deflater.setStrategy(Deflater.HUFFMAN_ONLY);
this is my gzip method:
//GZIP Compression method
public static byte[] compress(String data) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream(data.length());
GZIPOutputStream gzip = new GZIPOutputStream(bos);
gzip.write(data.getBytes());
gzip.close();
byte[] compressed = bos.toByteArray();
bos.close();
return compressed;
}
No, but you can just use Deflater with nowrap set to true, and write your own gzip header and trailer.
Related
I have a byte array and I want to decompress this byte array. When I run below code it gives;
java.util.zip.ZipException: Not in GZIP format
I get this byte array from a soap webservice. When I call this webservice from soap UI it returns;
<size>491520</size>
<studentData>
<dataContent>Uy0xMDAwMF90MTAwMDAtVXNlciBTZWN1cml0eSBB........</dataContent>
</studentData>
Is there a problem with data coming from web service or my decompress method?
public static byte[] decompress(final byte[] input) throws Exception{
try (ByteArrayInputStream bin = new ByteArrayInputStream(input);
GZIPInputStream gzipper = new GZIPInputStream(bin)) {
byte[] buffer = new byte[1024];
ByteArrayOutputStream out = new ByteArrayOutputStream();
int len;
while ((len = gzipper.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
gzipper.close();
out.close();
return out.toByteArray();
}
}
EDIT:
I decoded base64 and write it to a file called "test.gzip". Now I can extract this file with 7zip and I can see all student files without any problem.
String encoded = Base64.getEncoder().encodeToString(studentData.getDataContent());
byte[] decoded = Base64.getDecoder().decode(encoded);
FileOutputStream fos = new FileOutputStream("test.gzip");
fos.write(decoded);
fos.close();
But when I try to decompress this decoded file it still gives same error;
decompress(decoded);
I am currently trying to read in data from a server response. I am using a Socket to connect to a server, creating a http GET request, then am using a Buffered Reader to read in data. Here is what the code looks like compacted:
Socket conn = new Socket(server, 80);
//Request made here
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String response;
while((response = inFromServer.readLine()) != null){
System.out.println(response);
}
I would like to read in the data, instead of as a String, as a byte array, and write it to a file. How is this possible? Any help is greatly appreciated, thank you.
You need to use a ByteArrayOutputStream, do something like the below code:
Socket conn = new Socket(server, 80);
//Request made here
InputStream is = conn.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int readBytes = -1;
while((readBytes = is.read(buffer)) > 1){
baos.write(buffer,0,readBytes);
}
byte[] responseArray = baos.toByteArray();
One way is to use Apache commons-io IOUtils
byte[] bytes = IOUtils.toByteArray(inputstream);
With plain java:
ByteArrayOutputStream output = new ByteArrayOutputStream();
try(InputStream stream = new FileInputStream("myFile")) {
byte[] buffer = new byte[2048];
int numRead;
while((numRead = stream.read(buffer)) != -1) {
output.write(buffer, 0, numRead);
}
} catch(IOException e) {
e.printStackTrace();
}
// and here your bytes
byte[] myDesiredBytes = output.toByteArray();
If you are not using Apache commons-io library in your project,I have pretty simple method to do the same without using it..
/*
* Read bytes from inputStream and writes to OutputStream,
* later converts OutputStream to byte array in Java.
*/
public static byte[] toByteArrayUsingJava(InputStream is)
throws IOException{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int reads = is.read();
while(reads != -1){
baos.write(reads);
reads = is.read();
}
return baos.toByteArray();
}
I want to compress byte array before saving to the file.
When I use Deflator to compress byte array, I get OutOfMemoryError:
ERROR/dalvikvm-heap(16065): Out of memory on a 921616-byte allocation.
I check the code and it is the same as android developer. But I added DeflatorOutputStream to reduce memory usage.
My code:
public static byte[] compress(byte[] data) throws IOException {
Deflater deflater = new Deflater();
deflater.setInput(data);
deflater.finish();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(data.length);
DeflaterOutputStream dos=new DeflaterOutputStream(outputStream);
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int count=deflater.deflate(buffer);
// returns the generated code... index
dos.write(buffer, 0, count);
}
deflater.end();
byte[] output = outputStream.toByteArray();
dos.finish();
dos.close();
outputStream.close();
return output;
}
I checked that error occurs in this line: int count=deflater.deflate(buffer);
I think there is a much simpler solution:
public static byte[] compress(byte[] data) throws IOException {
ByteArrayOutputStream bout = new ByteArrayOutputStream(data.length);
try (DeflaterOutputStream out = new DeflaterOutputStream(bout)) {
out.write(data);
}
return bout.toByteArray();
}
does someone have an idea why this code to create a gzipped string is not working? CLI gzip on a Mac can't open the resulting file: "Not in gz format".
Please note: I need the string, not the file. Creating the gzipped file directly works, so does writing the JSON without zipping it.
The file writing in this example is just for testing purposes.
public someMethod {
String gzippedString = this.gzippedString(finalJSONObject.toJSONString());
OutputStream outputStream = new FileOutputStream(new File(this.jsonOutputPath + "/myfile.gz"));
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(outputStream));
writer.append(gzippedString);
writer.close();
}
private String gzippedString(String inputString) throws IOException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
GZIPOutputStream gzipOutputStream = new GZIPOutputStream(outputStream);
gzipOutputStream.write(inputString.getBytes());
gzipOutputStream.close();
outputStream.close();
String gzippedString = outputStream.toString();
return gzippedString;
}
EDIT:
chrylis showed me the way:
public void someMethod() {
byte[] byteArray = this.gzippedByteArray(finalJSONObject.toJSONString());
FileOutputStream out = new FileOutputStream(this.jsonOutputPath + "/myfile.gz");
out.write(byteArray);
out.close();
}
private byte[] gzippedByteArray(String inputString) throws IOException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
GZIPOutputStream gzipOutputStream = new GZIPOutputStream(outputStream);
gzipOutputStream.write(inputString.getBytes());
gzipOutputStream.close();
outputStream.close();
byte[] gzippedByteArray = outputStream.toByteArray();
return gzippedByteArray;
}
This results in a working gzipped JSON.
Thanks a lot!
You're round-tripping binary data through a String, which has a character encoding and other such mangling. Use the byte[] directly instead.
I need a code snippt for converting DataHandler to byte[].
This data handler contains Image.
It can be done by using below code without much effort using apache IO Commons.
final InputStream in = dataHandler.getInputStream();
byte[] byteArray=org.apache.commons.io.IOUtils.toByteArray(in);
You can do it like this:
public static byte[] toBytes(DataHandler handler) throws IOException {
ByteArrayOutputStream output = new ByteArrayOutputStream();
handler.writeTo(output);
return output.toByteArray();
}
private static final int INITIAL_SIZE = 1024 * 1024;
private static final int BUFFER_SIZE = 1024;
public static byte[] toBytes(DataHandler dh) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream(INITIAL_SIZE);
InputStream in = dh.getInputStream();
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead;
while ( (bytesRead = in.read(buffer)) >= 0 ) {
bos.write(buffer, 0, bytesRead);
}
return bos.toByteArray();
}
Beware that ByteArrayOutputStream.toByteArray() creates a copy of the internal byte array.
I use this code:
public static byte[] getContentAsByteArray(DataHandler handler) throws IOException {
byte[] bytes = null;
ByteArrayOutputStream bos = new ByteArrayOutputStream();
handler.writeTo(bos);
bos.flush();
bos.close();
bytes = bos.toByteArray();
return bytes;
}
Is something like this what you are looking for?
public static byte[] getBytesFromDataHandler(final DataHandler data) throws IOException {
final InputStream in = data.getInputStream();
byte out[] = new byte[0];
if(in != null) {
out = new byte[in.available()];
in.read(out);
}
return out;
}
UPDATE:
Based on dkarp's comment this is incorrect. According to the docs for InputStream:
Returns the number of bytes that can be read (or skipped over) from this input stream without blocking by the next caller of a method for this input stream. The next caller might be the same thread or or another thread.
It looks like Costi has the correct answer here.