I'm trying to compress and decompress some bytes of data with the Deflater and InflaterOutputStream.
The problem is that compression seems to work (I'm not
sure since the compressed data is always the same even if I have random test data).
But the decompression return nothing at all.
What am I doing wrong?
My console output:
Test data: D8A8E00821608F227AE473774E177216
Compressed data: 789C
Decompressed data:
My program:
SecureRandom random = new SecureRandom();
byte[] testdata = new byte[16];
random.nextBytes(testdata);
System.out.println("Test data: " + DatatypeConverter.printHexBinary(testdata));
byte[] compressed = null;
try (ByteArrayOutputStream buffer = new ByteArrayOutputStream())
{
try (DeflaterOutputStream stream = new DeflaterOutputStream(buffer))
{
stream.write(testdata);
stream.flush();
compressed = buffer.toByteArray();
System.out.println("Compressed data: " + DatatypeConverter.printHexBinary(compressed));
}
}
catch (IOException e)
{
System.out.println("IOException during compression.");
}
byte[] decompressed = null;
try (ByteArrayOutputStream buffer = new ByteArrayOutputStream())
{
try (InflaterOutputStream stream = new InflaterOutputStream(buffer))
{
stream.write(compressed);
stream.flush();
decompressed = buffer.toByteArray();
System.out.println("Decompressed data: " + DatatypeConverter.printHexBinary(decompressed));
}
}
catch (IOException e)
{
System.out.println("IOException during decompression.");
}
The problem is that you're only flushing the stream - which doesn't necessarily mean there's no more data to come, which can affect the decompression.
If you change both of your flush() calls to close(), you'll see you get the appropriate data back... or as you're using a try-with-resources statement, just let that close the inner stream, and wait until after that to call toByteArray:
try (ByteArrayOutputStream buffer = new ByteArrayOutputStream())
{
try (DeflaterOutputStream stream = new DeflaterOutputStream(buffer))
{
stream.write(testdata);
}
compressed = buffer.toByteArray();
System.out.println("Compressed data: " + Arrays.toString(compressed));
}
catch (IOException e)
{
System.out.println("IOException during compression.");
}
(Ditto when decompressing.)
Related
With Java:
I have a byte[] that represents a file.
How do I write this to a file (ie. C:\myfile.pdf)
I know it's done with InputStream, but I can't seem to work it out.
Use Apache Commons IO
FileUtils.writeByteArrayToFile(new File("pathname"), myByteArray)
Or, if you insist on making work for yourself...
try (FileOutputStream fos = new FileOutputStream("pathname")) {
fos.write(myByteArray);
//fos.close(); There is no more need for this line since you had created the instance of "fos" inside the try. And this will automatically close the OutputStream
}
Without any libraries:
try (FileOutputStream stream = new FileOutputStream(path)) {
stream.write(bytes);
}
With Google Guava:
Files.write(bytes, new File(path));
With Apache Commons:
FileUtils.writeByteArrayToFile(new File(path), bytes);
All of these strategies require that you catch an IOException at some point too.
Another solution using java.nio.file:
byte[] bytes = ...;
Path path = Paths.get("C:\\myfile.pdf");
Files.write(path, bytes);
Also since Java 7, one line with java.nio.file.Files:
Files.write(new File(filePath).toPath(), data);
Where data is your byte[] and filePath is a String. You can also add multiple file open options with the StandardOpenOptions class. Add throws or surround with try/catch.
From Java 7 onward you can use the try-with-resources statement to avoid leaking resources and make your code easier to read. More on that here.
To write your byteArray to a file you would do:
try (FileOutputStream fos = new FileOutputStream("fullPathToFile")) {
fos.write(byteArray);
} catch (IOException ioe) {
ioe.printStackTrace();
}
Try an OutputStream or more specifically FileOutputStream
Basic example:
String fileName = "file.test";
BufferedOutputStream bs = null;
try {
FileOutputStream fs = new FileOutputStream(new File(fileName));
bs = new BufferedOutputStream(fs);
bs.write(byte_array);
bs.close();
bs = null;
} catch (Exception e) {
e.printStackTrace()
}
if (bs != null) try { bs.close(); } catch (Exception e) {}
File f = new File(fileName);
byte[] fileContent = msg.getByteSequenceContent();
Path path = Paths.get(f.getAbsolutePath());
try {
Files.write(path, fileContent);
} catch (IOException ex) {
Logger.getLogger(Agent2.class.getName()).log(Level.SEVERE, null, ex);
}
////////////////////////// 1] File to Byte [] ///////////////////
Path path = Paths.get(p);
byte[] data = null;
try {
data = Files.readAllBytes(path);
} catch (IOException ex) {
Logger.getLogger(Agent1.class.getName()).log(Level.SEVERE, null, ex);
}
/////////////////////// 2] Byte [] to File ///////////////////////////
File f = new File(fileName);
byte[] fileContent = msg.getByteSequenceContent();
Path path = Paths.get(f.getAbsolutePath());
try {
Files.write(path, fileContent);
} catch (IOException ex) {
Logger.getLogger(Agent2.class.getName()).log(Level.SEVERE, null, ex);
}
I know it's done with InputStream
Actually, you'd be writing to a file output...
This is a program where we are reading and printing array of bytes offset and length using String Builder and Writing the array of bytes offset length to the new file.
`Enter code here
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
//*This is a program where we are reading and printing array of bytes offset and length using StringBuilder and Writing the array of bytes offset length to the new file*//
public class ReadandWriteAByte {
public void readandWriteBytesToFile(){
File file = new File("count.char"); //(abcdefghijk)
File bfile = new File("bytefile.txt");//(New File)
byte[] b;
FileInputStream fis = null;
FileOutputStream fos = null;
try{
fis = new FileInputStream (file);
fos = new FileOutputStream (bfile);
b = new byte [1024];
int i;
StringBuilder sb = new StringBuilder();
while ((i = fis.read(b))!=-1){
sb.append(new String(b,5,5));
fos.write(b, 2, 5);
}
System.out.println(sb.toString());
}catch (IOException e) {
e.printStackTrace();
}finally {
try {
if(fis != null);
fis.close(); //This helps to close the stream
}catch (IOException e){
e.printStackTrace();
}
}
}
public static void main (String args[]){
ReadandWriteAByte rb = new ReadandWriteAByte();
rb.readandWriteBytesToFile();
}
}
O/P in console : fghij
O/P in new file :cdefg
You can try Cactoos:
new LengthOf(new TeeInput(array, new File("a.txt"))).value();
More details: http://www.yegor256.com/2017/06/22/object-oriented-input-output-in-cactoos.html
I'm trying to write compressed data to a file and then read in the data and decompress it using the GZIP library. I've tried changing all formatting to StandardCharsets.UTF-8 and ISO-8859-1 and neither have fixed the GZIP format error. I'm wondering if it could possible have to do with the file I'm reading in? Here's the compression function:
public static byte[] compress(String originalFile, String compressFile) throws IOException {
// read in data from text file
// The name of the file to open.
String fileName = originalFile;
// This will reference one line at a time
String line = null;
String original = "";
try {
// FileReader reads text files in the default encoding.
FileReader fileReader =
new FileReader(fileName);
// Always wrap FileReader in BufferedReader.
BufferedReader bufferedReader =
new BufferedReader(fileReader);
while((line = bufferedReader.readLine()) != null) {
original.concat(line);
}
// Always close files.
bufferedReader.close();
}
catch(FileNotFoundException ex) {
System.out.println(
"Unable to open file '" +
fileName + "'");
}
catch(IOException ex) {
System.out.println(
"Error reading file '"
+ fileName + "'");
// Or we could just do this:
// ex.printStackTrace();
}
// create a new output stream for original string
try (ByteArrayOutputStream out = new ByteArrayOutputStream())
{
try (GZIPOutputStream gzip = new GZIPOutputStream(out))
{
gzip.write(original.getBytes(StandardCharsets.UTF_8));
}
byte[] compressed = out.toByteArray();
out.close();
String compressedFileName = compressFile;
try {
// Assume default encoding.
FileWriter fileWriter =
new FileWriter(compressedFileName);
// Always wrap FileWriter in BufferedWriter.
BufferedWriter bufferedWriter =
new BufferedWriter(fileWriter);
// Note that write() does not automatically
// append a newline character.
String compressedStr = compressed.toString();
bufferedWriter.write(compressedStr);
// Always close files.
bufferedWriter.close();
}
catch(IOException ex) {
System.out.println(
"Error writing to file '"
+ fileName + "'");
// Or we could just do this:
// ex.printStackTrace();
}
return compressed;
}
}
(I'm receiving the error on the line in the following decompression function) -
GZIPInputStream compressedByteArrayStream = new GZIPInputStream(new ByteArrayInputStream(s.getBytes(StandardCharsets.UTF_8)));
Decompression Function:
public static String decompress(String file) throws IOException {
byte[] compressed = {};
String s = "";
File fileName = new File(file);
FileInputStream fin = null;
try {
// create FileInputStream object
fin = new FileInputStream(fileName);
// Reads up to certain bytes of data from this input stream into an array of bytes.
fin.read(compressed);
//create string from byte array
s = new String(compressed);
System.out.println("File content: " + s);
}
catch (FileNotFoundException e) {
System.out.println("File not found" + e);
}
catch (IOException ioe) {
System.out.println("Exception while reading file " + ioe);
}
finally {
// close the streams using close method
try {
if (fin != null) {
fin.close();
}
}
catch (IOException ioe) {
System.out.println("Error while closing stream: " + ioe);
}
}
// create a new input string for compressed byte array
GZIPInputStream compressedByteArrayStream = new GZIPInputStream(new ByteArrayInputStream(s.getBytes(StandardCharsets.UTF_8)));
ByteArrayOutputStream byteOutput = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
// create a string builder and byte reader for the compressed byte array
BufferedReader decompressionBr = new BufferedReader(new InputStreamReader(compressedByteArrayStream, StandardCharsets.UTF_8));
StringBuilder decompressionSb = new StringBuilder();
// write data to decompressed string
String line1;
while((line1 = decompressionBr.readLine()) != null) {
decompressionSb.append(line1);
}
decompressionBr.close();
int len;
String uncompressedStr = "";
while((len = compressedByteArrayStream.read(buffer)) > 0) {
uncompressedStr = byteOutput.toString();
}
compressedByteArrayStream.close();
return uncompressedStr;
}
Here's the error message that i am receiving:
[B#7852e922
File content:
java.io.EOFException
at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268)
at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
at org.kingswoodoxford.Compression.decompress(Compression.java:136)
at org.kingswoodoxford.Compression.main(Compression.java:183)
Any suggestions as to how I might be able to fix this?
When you read the file you discard the new line at the end of each line.
A more efficient option which does do this is to copy a block i.e. char[] at a time. You can also convert the text as you go rather than creating a String or a byte[].
BTW original.concat(line); returns the concatenated string which you are discarding.
The real problem is you write to one stream and close a different one. This means that if there is any buffered data at the end of the file (and this is highly likely) the end of the file will be truncated and when you read it it will complain that your file is incomplete or EOFException.
Here is a shorter example
public static void compress(String originalFile, String compressFile) throws IOException {
char[] buffer = new char[8192];
try (
FileReader reader = new FileReader(originalFile);
Writer writer = new OutputStreamWriter(
new GZIPOutputStream(new FileOutputStream(compressFile)));
) {
for (int len; (len = reader.read(buffer)) > 0; )
writer.write(buffer, 0, len);
}
}
In the decompress, don't encode binary as text and attempt to get back the same data. It will almost certainly be corrupted. Try to use a buffer and a loop like I did for compress. i.e. it shouldn't be any more complicated.
Is there a way I can get netty ByteBuf from a java File object? I am trying to use Custom Snappy class to validate checksum. Below is my code where I'd like to get File object in bytebuf variable.
import io.netty.buffer.ByteBuf;
File file = new File("boot.zip");
ByteBuf byteBuf = Unpooled.buffer(128);
byteBuf.writeBytes(new FileInputStream(file));
int expectedChecksum = Snappy.calculateChecksum(byteBuf);
logger.info("checksum = "+expectedChecksum);
boolean checksumValid = true;
try {
CustomSnappy.validateChecksum(expectedChecksum, byteBuf);
} catch(DecompressionException e) {
checksumValid = false;
e.printStackTrace();
logger.error("DecompressionException in checksum calculation " + e);
} catch(Exception e) {
checksumValid = false;
e.printStackTrace();
logger.error("Exception in checksum calculation " + e);
}
I used Unpooled.buffer(128) but it gives me No signature of method :io.netty.buffer.UnpooledHeapByteBuf.writeBytes() is applicable for argument types: (java.io.FileInputStream) values: [java.io.FileInputStream#28b6520b]
You can created a MappedByteBuf from a FileChannel and wrap it in ByteBuffer via Unpooled.wrappedBuffer(..).
Here is an example of how to "create a MappedByteBuf from a FileChannel and wrap it in ByteBuffer via Unpooled.wrappedBuffer(..)":
try {
File file = new File(filename);
FileInputStream fileInputStream = new FileInputStream(file);
FileChannel fileChannel = fileInputStream.getChannel();
MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, file.length());
ByteBuf byteBuf = Unpooled.wrappedBuffer(mappedByteBuffer);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
You can use the method ByteBuf#writeBytes(InputStream). This should work, but I did not test it.
ByteBuf buffer = // see http://stackoverflow.com/a/15088238/834
buffer.writeBytes(new FileInputStream(file), file.length());
See http://netty.io/4.0/api/io/netty/buffer/ByteBuf.html#writeBytes(java.io.InputStream,%20int)
The documentation shows how to create a new ByteBuf: http://netty.io/4.0/api/io/netty/buffer/Unpooled.html
If you are using JDK 1.7, the class of java.nio.file.Files could make it easy with amount of static methods.
ByteBuf buffer = Unpooled.copiedBuffer(Files.readAllBytes(new File(this.getClass().getResource("/requests/overPackageRequest").getFile()).toPath()));
I have a method like
public void put(#Nonnull final InputStream inputStream, #Nonnull final String uniqueId) throws PersistenceException {
// a.) create gzip of inputStream
final GZIPInputStream zipInputStream;
try {
zipInputStream = new GZIPInputStream(inputStream);
} catch (IOException e) {
e.printStackTrace();
throw new PersistenceException("Persistence Service could not received input stream to persist for " + uniqueId);
}
I wan to convert the inputStream into zipInputStream, what is the way to do that?
The above method is incorrect and throws Exception as "Not a Zip Format"
converting Java Streams to me are really confusing and I do not make them right
The GZIPInputStream is to be used to decompress an incoming InputStream. To compress an incoming InputStream using GZIP, you basically need to write it to a GZIPOutputStream.
You can get a new InputStream out of it if you use ByteArrayOutputStream to write gzipped content to a byte[] and ByteArrayInputStream to turn a byte[] into an InputStream.
So, basically:
public void put(#Nonnull final InputStream inputStream, #Nonnull final String uniqueId) throws PersistenceException {
final InputStream zipInputStream;
try {
ByteArrayOutputStream bytesOutput = new ByteArrayOutputStream();
GZIPOutputStream gzipOutput = new GZIPOutputStream(bytesOutput);
try {
byte[] buffer = new byte[10240];
for (int length = 0; (length = inputStream.read(buffer)) != -1;) {
gzipOutput.write(buffer, 0, length);
}
} finally {
try { inputStream.close(); } catch (IOException ignore) {}
try { gzipOutput.close(); } catch (IOException ignore) {}
}
zipInputStream = new ByteArrayInputStream(bytesOutput.toByteArray());
} catch (IOException e) {
e.printStackTrace();
throw new PersistenceException("Persistence Service could not received input stream to persist for " + uniqueId);
}
// ...
You can if necessary replace the ByteArrayOutputStream/ByteArrayInputStream by a FileOuputStream/FileInputStream on a temporary file as created by File#createTempFile(), especially if those streams can contain large data which might overflow machine's available memory when used concurrently.
GZIPInputStream is for reading gzip-encoding content.
If your goal is to take a regular input stream and compress it in the GZIP format, then you need to write those bytes to a GZIPOutputStream.
See also this answer to a related question.
i am having a problem in reading a file from Flex. The file contains a base64encoded string. when i read the file i get the length as 47856 and the decoded base64 byte array length as 34157.
When i read the same File from java i get the length as 48068 and 35733 respectively.
What is the problem?
private function init():void{
var file:File = File.desktopDirectory.resolvePath("Files/sample.txt");
stream = new FileStream();
stream.open(file, FileMode.READ);
var str:String = stream.readUTFBytes(stream.bytesAvailable);
stream.close();
str = str.replace(File.lineEnding, "\n");
contents.text = str;
fileName.text = file.name;
}
public function playSound(contents:String):void{
try{
var byteData: ByteArray;
byteData = new ByteArray();
byteData.writeUTFBytes(contents);
var dec:Base64Decoder = new Base64Decoder();
dec.decode(contents);
byteData = dec.toByteArray();
Alert.show("byte Array " + byteData.toString().length +" :: " +contents.length);
}
And this is my java code for reading the file...Whatever result i am expecting is achieved in the java side.
private static String readFile(String path) throws IOException {
FileInputStream stream = new FileInputStream(new File(path));
try {
FileChannel fc = stream.getChannel();
MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
return Charset.defaultCharset().decode(bb).toString(); }
finally { stream.close();
}
}
Java Code where i am printing the length
byte[] decodedBase64 = new byte[byteLength];
String speexData = null;
try {
speexData = readFile(userDir +"//" +xmlFileName);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
// System.out.println("sa " + sa);
try{
decodedBase64= Base64.decodeToByteArray(speexData);
System.out.println("decodednase64 length " + decodedBase64.length +" :: " +speexData.length());
}
catch(Exception e){
}
You would have to post your java code to show what you're doing there, as well.
However, without knowing more, I could take a guess and say that when you replace the line ending, you may be removing a byte each time (if it was \r\n and you're making it \n, for example).