We Keep Coding

Not in GZIP Format - JAVA

I'm trying to write compressed data to a file and then read in the data and decompress it using the GZIP library. I've tried changing all formatting to StandardCharsets.UTF-8 and ISO-8859-1 and neither have fixed the GZIP format error. I'm wondering if it could possible have to do with the file I'm reading in? Here's the compression function:
public static byte[] compress(String originalFile, String compressFile) throws IOException {
// read in data from text file
// The name of the file to open.
String fileName = originalFile;
// This will reference one line at a time
String line = null;
String original = "";
try {
// FileReader reads text files in the default encoding.
FileReader fileReader =
new FileReader(fileName);
// Always wrap FileReader in BufferedReader.
BufferedReader bufferedReader =
new BufferedReader(fileReader);
while((line = bufferedReader.readLine()) != null) {
original.concat(line);
}
// Always close files.
bufferedReader.close();
}
catch(FileNotFoundException ex) {
System.out.println(
"Unable to open file '" +
fileName + "'");
}
catch(IOException ex) {
System.out.println(
"Error reading file '"
+ fileName + "'");
// Or we could just do this:
// ex.printStackTrace();
}
// create a new output stream for original string
try (ByteArrayOutputStream out = new ByteArrayOutputStream())
{
try (GZIPOutputStream gzip = new GZIPOutputStream(out))
{
gzip.write(original.getBytes(StandardCharsets.UTF_8));
}
byte[] compressed = out.toByteArray();
out.close();
String compressedFileName = compressFile;
try {
// Assume default encoding.
FileWriter fileWriter =
new FileWriter(compressedFileName);
// Always wrap FileWriter in BufferedWriter.
BufferedWriter bufferedWriter =
new BufferedWriter(fileWriter);
// Note that write() does not automatically
// append a newline character.
String compressedStr = compressed.toString();
bufferedWriter.write(compressedStr);
// Always close files.
bufferedWriter.close();
}
catch(IOException ex) {
System.out.println(
"Error writing to file '"
+ fileName + "'");
// Or we could just do this:
// ex.printStackTrace();
}
return compressed;
}
}
(I'm receiving the error on the line in the following decompression function) -
GZIPInputStream compressedByteArrayStream = new GZIPInputStream(new ByteArrayInputStream(s.getBytes(StandardCharsets.UTF_8)));
Decompression Function:
public static String decompress(String file) throws IOException {
byte[] compressed = {};
String s = "";
File fileName = new File(file);
FileInputStream fin = null;
try {
// create FileInputStream object
fin = new FileInputStream(fileName);
// Reads up to certain bytes of data from this input stream into an array of bytes.
fin.read(compressed);
//create string from byte array
s = new String(compressed);
System.out.println("File content: " + s);
}
catch (FileNotFoundException e) {
System.out.println("File not found" + e);
}
catch (IOException ioe) {
System.out.println("Exception while reading file " + ioe);
}
finally {
// close the streams using close method
try {
if (fin != null) {
fin.close();
}
}
catch (IOException ioe) {
System.out.println("Error while closing stream: " + ioe);
}
}
// create a new input string for compressed byte array
GZIPInputStream compressedByteArrayStream = new GZIPInputStream(new ByteArrayInputStream(s.getBytes(StandardCharsets.UTF_8)));
ByteArrayOutputStream byteOutput = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
// create a string builder and byte reader for the compressed byte array
BufferedReader decompressionBr = new BufferedReader(new InputStreamReader(compressedByteArrayStream, StandardCharsets.UTF_8));
StringBuilder decompressionSb = new StringBuilder();
// write data to decompressed string
String line1;
while((line1 = decompressionBr.readLine()) != null) {
decompressionSb.append(line1);
}
decompressionBr.close();
int len;
String uncompressedStr = "";
while((len = compressedByteArrayStream.read(buffer)) > 0) {
uncompressedStr = byteOutput.toString();
}
compressedByteArrayStream.close();
return uncompressedStr;
}
Here's the error message that i am receiving:
[B#7852e922
File content:
java.io.EOFException
at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268)
at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
at org.kingswoodoxford.Compression.decompress(Compression.java:136)
at org.kingswoodoxford.Compression.main(Compression.java:183)
Any suggestions as to how I might be able to fix this?

When you read the file you discard the new line at the end of each line.
A more efficient option which does do this is to copy a block i.e. char[] at a time. You can also convert the text as you go rather than creating a String or a byte[].
BTW original.concat(line); returns the concatenated string which you are discarding.
The real problem is you write to one stream and close a different one. This means that if there is any buffered data at the end of the file (and this is highly likely) the end of the file will be truncated and when you read it it will complain that your file is incomplete or EOFException.
Here is a shorter example
public static void compress(String originalFile, String compressFile) throws IOException {
char[] buffer = new char[8192];
try (
FileReader reader = new FileReader(originalFile);
Writer writer = new OutputStreamWriter(
new GZIPOutputStream(new FileOutputStream(compressFile)));
) {
for (int len; (len = reader.read(buffer)) > 0; )
writer.write(buffer, 0, len);
}
}
In the decompress, don't encode binary as text and attempt to get back the same data. It will almost certainly be corrupted. Try to use a buffer and a loop like I did for compress. i.e. it shouldn't be any more complicated.

Why I'm getting junk data in the File?

I'm receiving a File from server and in client I want to save this file to a directory. Here is the code that do the job of saving the File
FileDetails obj1= (FileDetails)object;
String str= "A "+obj1.fileExtension+" Received From "+obj1.source;
JOptionPane.showMessageDialog(null,str,"Success", WIDTH, null);
FileOutputStream saveFile = new FileOutputStream("F:\\Download\\"+obj1.fileExtension);
ObjectOutputStream save = new ObjectOutputStream(saveFile);
byte[] buf= convertToByteArray(obj1.file);
save.write(buf);
save.close();
The function convertToByteArray is as following
private byte[] convertToByteArray(File file){
try{
FileInputStream fis = new FileInputStream(file);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
for (int readNum; (readNum = fis.read(buf)) != -1;) {
bos.write(buf, 0, readNum); //no doubt here is 0
bos.flush();
System.out.println("read " + readNum + " bytes,");
}
byte[] bytes = bos.toByteArray();
return bytes;
} catch (IOException ex) {
ex.printStackTrace();
}
return null;
}
FileDetails is a serializable Object which contains the file that server sends and the file name in fileExtension. The class is as following -
public class FileDetails implements Serializable{
public File file;
public String fileExtension;
public String source;
public String destination;
}
now problem is I get Junk Data in the saved file in client side. To test whether I'm receiving pure data from server I've printed the content of the file in console and got expected result. But when I open the saved file in the specified directory I get some Chinese scripts What can I do ? Please help me out.

Part of the problem is, I think, looking at the file in Notepad. If I open the file in another editor, say TextPad using binary mode, I see that the file starts with a serialization header. Using the ObjectOutputStream serializes the binary data of the file as an object. What I think you want to do is remove the ObjectOutputStream line and instead change it to the following:
FileOutputStream saveFile = new FileOutputStream("F:\\Download\\"+obj1.fileExtension);
//ObjectOutputStream save = new ObjectOutputStream(saveFile);
byte[] buf= convertToByteArray(obj1.file);
saveFile.write(buf);
saveFile.close();
That will write out the bytes ONLY of the file.

Take a look at the following:
http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#move(java.nio.file.Path, java.nio.file.Path, java.nio.file.CopyOption...)

How to create new excel document in java

Here I am using poi-jar to export data from database to excel it working fine . But here I want change instead of creating manual path. I wanted make that as to download automatically with out creating any manual path like this:
OutputStream file = new FileOutputStream(new File("D:\\venki1213.xls"));
And this is my code:
Session ses = HibernateUtil.getSessionFactory().openSession();
String query;
query = "from LibraryImportEntity ";
List<LibraryImportEntity> list = ses.createQuery(query).list();
ses.close();
System.out.println("list size" + list.size());
String filename = "D://ranjith//ranjith1213.xls";
OutputStream file = new FileOutputStream(new File("D:\\venki1213.xls"));
HSSFWorkbook hwb = new HSSFWorkbook();
HSSFSheet sheet = hwb.createSheet("new sheet");
HSSFRow rowhead = sheet.createRow((short) 0);
rowhead.createCell(0).setCellValue("Sl.No");
rowhead.createCell(1).setCellValue("Magazine Name");
rowhead.createCell(2).setCellValue("Volume No");
rowhead.createCell(3).setCellValue("Issue No");
rowhead.createCell(4).setCellValue("Cost");
int i = 1;
for (LibraryImportEntity l1 : list) {
System.out.println("sl_no" + l1.getSl_no());
System.out.println("Magazinename" + l1.getMagazinename());
System.out.println("sl_no" + l1.getVolumeno());
System.out.println("sl_no" + l1.getCost());
HSSFRow row = sheet.createRow((short) i);
row.createCell(0).setCellValue(l1.getSl_no());
row.createCell(1).setCellValue(l1.getMagazinename());
row.createCell(2).setCellValue(l1.getVolumeno());
row.createCell(3).setCellValue(l1.getIssueno());
row.createCell(4).setCellValue(l1.getCost());
i++;
}
try {
FileOutputStream fileOut = new FileOutputStream(filename);
hwb.write(file);
fileOut.close();
} catch (IOException ex) {
Logger.getLogger(LibraryExportDAO.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println("Your excel file has been generated!");
return "success";
}

If you are generating the excel in a browser just call the method you want to generate the excel file based on a url and set the response properties like this,
//1.Fill the data from db
//2.Set the response properties
String fileName = "Excel.xls";
response.setHeader("Content-Disposition", "inline; filename=" + fileName);
// Make sure to set the correct content type(the below content type is ok)
response.setContentType("application/vnd.ms-excel");
//3.Write to the output stream
Writer.write();//call write method of Writer class to write the data to o/p stream
Writer Class:
public class Writer {
private static Logger logger = Logger.getLogger("service");
/**
* Writes the report to the output stream
*/
public static void write(HttpServletResponse response, HSSFSheet worksheet) {
logger.debug("Writing excel data to the stream");
try {
// Retrieve the output stream
ServletOutputStream outputStream = response.getOutputStream();
// Write to the output stream
worksheet.getWorkbook().write(outputStream);
// Flush the stream
outputStream.flush();
}
catch (Exception e) {
logger.error("Unable to write excel data to the output stream");
}
}
}
In the response receiving end you'll be prompted to download the file in the browser window..
LINK

Instead of this code,
OutputStream file = new FileOutputStream(new File("D:\\venki1213.xls"));
Use,
OutputStream file = new FileOutputStream(new File("venki1213.xls"));
This will create a file in project folder.
Cheers ...... !

You have to write the file to response
ByteArrayOutputStream bos = null;
try {
File file = new File(filename);
FileInputStream fis = new FileInputStream(file);
bos = new ByteArrayOutputStream();
if (fis != null) {
byte[] buf = new byte[(int) file.length()];
for (int num; (num = fis.read(buf)) != -1;) {
bos.write(buf, 0, num);
}
}
byte[] bytes = bos.toByteArray();
response.setContentType("application/vnd.ms-excel");
final OutputStream fileOutputStream = response.getOutputStream();
fileOutputStream.write(bytes);
fileOutputStream.flush();
fileOutputStream.close();
} catch (Exception e) {
}

What is the best way to calculate a checksum for a Java class?

I have an application where I am generating a "target file" based on a Java "source" class. I want to regenerate the target when the source changes. I have decided the best way to do this would be to get a byte[] of the class contents and calculate a checksum on the byte[].
I am looking for the best way to get the byte[] for a class. This byte[] would be equivalent to the contents of the compiled .class file. Using ObjectOutputStream does not work. The code below generates a byte[] that is much smaller than the byte contents of the class file.
// Incorrect function to calculate the byte[] contents of a Java class
public static final byte[] getClassContents(Class<?> myClass) throws IOException {
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
try( ObjectOutputStream stream = new ObjectOutputStream(buffer) ) {
stream.writeObject(myClass);
}
// This byte array is much smaller than the contents of the *.class file!!!
byte[] contents = buffer.toByteArray();
return contents;
}
Is there a way to get the byte[] with the identical contents of the *.class file? Calculating the checksum is the easy part, the hard part is obtaining the byte[] contents used to calculate an MD5 or CRC32 checksum.

THis is the solution that I ended up using. I don't know if it's the most efficient implementation, but the following code uses the class loader to get the location of the *.class file and reads its contents. For simplicity, I skipped buffering of the read.
// Function to obtain the byte[] contents of a Java class
public static final byte[] getClassContents(Class<?> myClass) throws IOException {
String path = myClass.getName().replace('.', '/');
String fileName = new StringBuffer(path).append(".class").toString();
URL url = myClass.getClassLoader().getResource(fileName);
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
try (InputStream stream = url.openConnection().getInputStream()) {
int datum = stream.read();
while( datum != -1) {
buffer.write(datum);
datum = stream.read();
}
}
return buffer.toByteArray();
}

I don't get what you means, but i think you are looking for this, MD5.
To check MD5 of a file, you can use this code
public String getMd5(File file)
{
DigestInputStream stream = null;
try
{
stream = new DigestInputStream(new FileInputStream(file), MessageDigest.getInstance("MD5"));
byte[] buffer = new byte[65536];
read = stream.read(buffer);
while (read >= 1) {
read = stream.read(buffer);
}
}
catch (Exception ignored)
{
int read;
return null;
}
return String.format("%1$032x", new Object[] { new BigInteger(1, stream.getMessageDigest().digest()) });
}
Then, you can store the md5 of a file in any way for exmaple XML. An exmaple of MD5 is 49e6d7e2967d1a471341335c49f46c6c so once the file name and size change, md5 will change. You can store md5 of each file in XML format and next time your run a code to check md5 and compare the md5 of each file in the xml file.

If you really want the contents of the .class file, you should read the contents of .class file, not the byte[] representation that is in memory. So something like
import java.io.*;
public class ReadSelf {
public static void main(String args[]) throws Exception {
Class classInstance = ReadSelf.class;
byte[] bytes = readClass(classInstance);
}
public static byte[] readClass(Class classInstance) throws Exception {
String name = classInstance.getName();
name = name.replaceAll("[.]", "/") + ".class";
System.out.println("Reading this: " + name);
File file = new File(name);
System.out.println("exists: " + file.exists());
return read(file);
}
public static byte[] read(File file) throws Exception {
byte[] data = new byte[(int)file.length()]; // can only read a file of size INT_MAX
DataInputStream inputStream =
new DataInputStream(
new BufferedInputStream(
new FileInputStream(file)));
int total = 0;
int nRead = 0;
try {
while((nRead = inputStream.read(data)) != -1) {
total += nRead;
}
}
finally {
inputStream.close();
}
System.out.println("Read " + total
+ " characters, which should match file length of "
+ file.length() + " characters");
return data;
}
}

Is it possible to gzip and upload this string to Amazon S3 without ever being written to disk?

I know this is probably possible using Streams, but I wasn't sure the correct syntax.
I would like to pass a string to the Save method and have it gzip the string and upload it to Amazon S3 without ever being written to disk. The current method inefficiently reads/writes to disk in between.
The S3 PutObjectRequest has a constructor with InputStream input as an option.
import java.io.*;
import java.util.zip.GZIPOutputStream;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.PutObjectRequest;
public class FileStore {
public static void Save(String data) throws IOException
{
File file = File.createTempFile("filemaster-", ".htm");
file.deleteOnExit();
Writer writer = new OutputStreamWriter(new FileOutputStream(file));
writer.write(data);
writer.flush();
writer.close();
String zippedFilename = gzipFile(file.getAbsolutePath());
File zippedFile = new File(zippedFilename);
zippedFile.deleteOnExit();
AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials(
new FileInputStream("AwsCredentials.properties")));
String bucketName = "mybucket";
String key = "test/" + zippedFile.getName();
s3.putObject(new PutObjectRequest(bucketName, key, zippedFile));
}
public static String gzipFile(String filename) throws IOException
{
try {
// Create the GZIP output stream
String outFilename = filename + ".gz";
GZIPOutputStream out = new GZIPOutputStream(new FileOutputStream(outFilename));
// Open the input file
FileInputStream in = new FileInputStream(filename);
// Transfer bytes from the input file to the GZIP output stream
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
in.close();
// Complete the GZIP file
out.finish();
out.close();
return outFilename;
} catch (IOException e) {
throw e;
}
}
}

I would use something like the following:
ByteArrayOutputStream byteOut = new ByteArrayOutputStream();
GZipOuputStream gzipOut = new GZipOutputStream(byteOut);
// write your stuff
byte[] bites = byteOut.toByteArray();
//write the bites to the amazon stream
You are writing the zipped values out to the byte stream, then taking the byte values, you can write those to your other stream. You can also wrap the stream to the amazon site (i.e. the output stream from the http connection or something similar) and avoid the whole ByteArrayOutputStream.
Edit: I noticed your last sentence - bleah. You can take the bytes you created, create a ByteArrayInputStream with them, and then pass that in as an input stream:
ByteArrayInputStream byteInStream = new ByteArrayInputStream(bites);
It should read from the input stream to the output stream, if I am understanding what you are describing correctly. Otherwise, you can simply write to the output stream.

This is essentially what aperkins suggested. I don't know the interface to AS3, so his suggestion to create a ByteArrayInputStream over the byte array is probably the way to go.
import java.io.*;
import java.util.zip.GZIPOutputStream;
import com.amazonaws.auth.PropertiesCredentials;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.PutObjectRequest;
public class FileStore {
public static void Save(String data) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
Writer writer = new OutputStreamWriter(baos);
writer.write(data);
writer.flush();
writer.close();
byte[] zippedBytes = gzipFile(baos.toByteArray());
AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials(
new FileInputStream("AwsCredentials.properties")));
String bucketName = "mybucket";
String key = "test/" + zippedFile.getName();
s3.putObject(new PutObjectRequest(bucketName, key,
new ByteArrayInputStream(zippedBytes));
}
public static byte[] gzipFile(byte[] bytes) throws IOException {
try {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream out = new GZIPOutputStream(baos);
out.write(bytes, 0, bytes.length);
// Complete the GZIP file
out.finish();
out.close();
return baos.toByteArray();
} catch (IOException e) {
throw e;
}
}
}