I am a Java bonehead/newbie so please be gentle. I have two functions which I realize are somewhat incompatible:
saveS3toFilesystem - takes a InputStream from AWS S3 and saves it to the local filesystem as a file
decompress - takes a string and decodes the base64 encoding and the decompresses the gzip compression.
I really want these two to work in concert to achieve the result of the file saved to the filesystem being the uncompressed file but I realize that my "decompress" function should probably be changed to receive a stream rather than a string but sadly I'm just a "cutter and paster" in the world of Java these days.
Here are my two functions as they are now:
private void saveS3toFilesystem(String filename, String bucketName, String localFilename) {
S3Object obj = s3.getObject(bucketName, filename);
InputStream in = obj.getObjectContent();
try {
Files.createDirectories(Paths.get(localFilename.replace(this.FILE_EXTENSION, "")));
Files.copy(in, Paths.get(localFilename));
this.logger.log("Input file has been placed in local filesystem for ITMS to pick up: " + localFilename + "\n");
} catch (IOException err) {
this.logger.log("There was a problem saving the file to " + localFilename);
err.printStackTrace();
} finally {
try {
in.close();
} catch (IOException err) {
err.printStackTrace();
}
}
return;
}
and then ...
private String decompress(String compressedZip) {
byte[] decodedBytes = Base64.getDecoder().decode(compressedZip);
String result = null;
GZIPInputStream zip = null;
try {
zip = new GZIPInputStream(new ByteArrayInputStream(decodedBytes));
result = IOUtils.toString(zip);
} catch (IOException e) {
e.printStackTrace();
} finally {
IOUtils.closeQuietly(zip);
}
return result;
}
Can anyone please help me to achieve the dream? Happy to do it with streams, strings, or any method that will work. Sadly I can't afford atm to up my Java skills enough to grok the solution myself.
Many thanks in advance.
Based on the following APIs :
Base64.Decoder and GZIPInputStream (look at the wrap method on the former and the constructors on the latter), the decompress method can be overloaded as follows:
private String decompress(InputStream compressedStream) {
InputStream decodingStream = Base64.getDecoder().wrap(compressedStream);
String result = null;
GZIPInputStream zip = null;
try {
zip = new GZIPInputStream(decodingStream);
result = IOUtils.toString(zip);
} catch (IOException e) {
e.printStackTrace();
} finally {
IOUtils.closeQuietly(zip);
}
return result;
}
And finally, the changes to saveS3toFilesystem are as follows :
private void saveS3toFilesystem(String filename, String bucketName, String localFilename) {
S3Object obj = s3.getObject(bucketName, filename);
InputStream in = obj.getObjectContent();
// decoding the inputstream via decode into a string, which is then
// used in order to create an inputstream of decoded data
InputStream decodedStream =
new ByteArrayInputStream(decompress(in).getBytes(StandardCharsets.UTF_8));
try {
Files.createDirectories(Paths.get(localFilename.replace(this.FILE_EXTENSION, "")));
Files.copy(decodedStream, Paths.get(localFilename));
this.logger.log("Input file has been placed in local filesystem for ITMS to pick up: " + localFilename + "\n");
} catch (IOException err) {
this.logger.log("There was a problem saving the file to " + localFilename);
err.printStackTrace();
} finally {
try {
in.close();
} catch (IOException err) {
err.printStackTrace();
}
}
return;
}
Related
I want to read from a file and store it in a string.
This is my method:
public static String createJsonFileFromNode(String filename, JsonNode root) {
String dirName = "src/test/resources/json/";
File dir = new File (dirName);
File actualFile = new File (dir, filename);
try (Writer writer = new BufferedWriter(new OutputStreamWriter (
new FileOutputStream(actualFile), "utf-8")))
{
writer.write(String.valueOf(root));
log.info(actualFile.getPath());
String updatedJson = FileUtils.readFileToString(actualFile, "UTF-8");
return updatedJson;
}
catch (UnsupportedEncodingException e) {
e.printStackTrace();
return "";
} catch (FileNotFoundException e) {
e.printStackTrace();
return "";
} catch (IOException e) {
e.printStackTrace();
return "";
}
}
I have two problems in the above method:
In String dirName = "src/test/resources/json/" I am passing an entire path, which I dont want to. I want to pass it as "/json/"
updatedJson is retuning null even though the file is getting saved to the particular direction. Not sure what is going on. Can someone please help me?
Thank you.
My scenario is the folowing, I'm getting some data from db, writing it to a file, then giving that file by inputStrem to the client response.
Here is the code doing this:
public void getErrorsFile(UUID jobId, Pageable pageable, HttpServletResponse response) {
List<ValidationErrorsDTO> failedList = getValidationErrors(jobId, pageable);
String fileName = "failedRows.txt";
PrintWriter writer;
Path path = null;
try {
writer = new PrintWriter(fileName, "UTF-8");
for (ValidationErrorsDTO dto : failedList) {
String line = dto.getLineNumber() + ": " + dto.getValidationMessage() + "\n";
writer.println(line);
}
writer.flush();
path = Paths.get(fileName);
} catch (IOException e) {
e.printStackTrace();
}
try (InputStream inputStream = Files.newInputStream(path)) {
writeStreamToResponse(response, inputStream, fileName);
IOUtils.closeQuietly(inputStream);
Files.delete(path);
} catch (IOException e) {
e.printStackTrace();
}
}
private void writeStreamToResponse(HttpServletResponse response, InputStream inputStream, String fileName) throws IOException {
response.addHeader("Content-disposition", "attachment;filename=" + fileName);
response.setContentType("txt/plain");
IOUtils.copy(inputStream, response.getOutputStream());
response.flushBuffer();
}
And when running this code, all the phases go well, except the one, which removes the file at the end.And I get this exception
java.nio.file.FileSystemException: failedRows.txt: The process cannot access the file because it is being used by another process.
The speps I did for solving the issue:
I did put the process of removing file in another thread, thinking that it's a concurrency issue.
No other process is holding this file somehow I think.
I did try another way of removing the file, used java.io.file.delete
method.
I did put the process of removing file in finally block.
Nothing helps.
Will appreciate any useful suggestion.
You should have open the writer instance as try-with-resources similar to opening the input stream:
try (writer = new PrintWriter(fileName, "UTF-8")) {
for (ValidationErrorsDTO dto : failedList) {
String line = dto.getLineNumber() + ": " + dto.getValidationMessage() + "\n";
writer.println(line);
}
writer.flush();
path = Paths.get(fileName);
} catch (IOException e) {
e.printStackTrace();
}
I am trying to create kmz file from kml file on the fly and render it as a stream of bytes in web application.
But when I downloaded generated kmz file, I couldn't open it using archive manager on Ubuntu.
I view similar questions on this site, but it don't work.
Can someone help me and explain what I do wrong?!
This is my code.
#Public public void retrieveKmlInOldFormat() {
File file = new File(Play.applicationPath+"/"+Play.configuration.getProperty("web.content", "../bspb-web")+"/map/map.kml");
String kmlFileContent = null;
try {
String kmlUrl = file.toURI().toURL().toString();
kmlFileContent = BSPBKml2OldFormatConverter.toOldKml(
kmlParserLocal.load(kmlUrl));
} catch (MalformedURLException e) {
e.printStackTrace();
}
String zippedFileName = "old_fmt_map.kmz";
String zippedKml = compressKmlFile(kmlFileContent,zippedFileName);
response.setContentTypeIfNotSet("application/vnd.google-earth.kmz");
renderBinary(new ByteArrayInputStream(zippedKml.getBytes()),zippedFileName);
return;
}
Compress method code:
private String compressKmlFile(String kmlFileContent,String zipEntryName){
String zippedContent = null;
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
ZipOutputStream zipStream = new ZipOutputStream(new
BufferedOutputStream(byteStream));
ZipEntry zipEntry = null;
zipEntry = new ZipEntry("doc.kml");
try {
zipEntry.setSize(kmlFileContent.getBytes("UTF-8").length);
zipStream.putNextEntry(zipEntry);
zipStream.write(kmlFileContent.getBytes("UTF-8"));
zipStream.closeEntry();
zippedContent = new String(byteStream.toByteArray(),"UTF-8");
} catch (IOException e) {
logger.error("Error while zipping kml file content");
}
finally {
try {
byteStream.close();
zipStream.close();
} catch (IOException e) {
logger.error(e.getMessage());
}
}
return zippedContent;
}
The problem is about downloaded corrupted kmz archive. This problem can be resolved by using output stream of http response as constructor argument for ZipOutputStream class.
Solution is in this code.
#Public public void retrieveKmlInOldFormat(){
File file = new File(Play.applicationPath+"/"+Play.configuration.getProperty("web.content", "../bspb-web")+"/map/map.kml");
String kmlFileContent = null;
try {
String kmlUrl = file.toURI().toURL().toString();
kmlFileContent = BSPBKml2OldFormatConverter.toOldKml(kmlParserLocal.load(kmlUrl));
} catch (MalformedURLException e) {
e.printStackTrace();
}
response.setContentTypeIfNotSet("application/vnd.google-earth.kmz");
response.setHeader("Content-Disposition", "attachment; filename=\"old_fmt_map.kmz\"");
renderAsKmz(response, kmlFileContent,"old_fmt_map.kml");
return;
}
private void renderAsKmz(Response response,String kmlFileContent,String zipEntryName){
ZipOutputStream zipStream = new ZipOutputStream(response.out);
ZipEntry zipEntry = new ZipEntry(zipEntryName);
try {
zipStream.putNextEntry(zipEntry);
zipStream.write(kmlFileContent.getBytes());
} catch (IOException e) {
logger.error("Error while zipping kml file content : " + e.getMessage());
}
finally {
try {
zipStream.closeEntry();
zipStream.close();
} catch (IOException e) {
logger.error("Error while closing zipped stream : " + e.getMessage());
}
}
I'm trying to encrypt a large file which is 500MB but my code throws an out of memory error, for small files below 50MB the code works fine. Im using a third party library called JNCryptor for encryption, please have a look at my code and correct me if any mistake. Thanks in advance.
public void encrypt() {
String file = Environment.getExternalStorageDirectory() + "/sai/ravi_enc.exe";
byte[] filedata = null;
try {
filedata = IOUtils.toByteArray(new FileInputStream(file));
} catch (IOException e) {
e.printStackTrace();
}
JNCryptor cryptor = new AES256JNCryptor();
String password = "123456789";
try {
byte[] ciphertext = cryptor.decryptData(filedata, password.toCharArray());
String Outfile = Environment.getExternalStorageDirectory() + "/sai/ravi_dec.exe";
writeFile(ciphertext, Outfile);
System.out.println("Done");
} catch (CryptorException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void writeFile(byte[] data, String fileName) throws IOException {
FileOutputStream out = new FileOutputStream(fileName);
out.write(data);
out.close();
}
This is up to JNCryptor to well manage memory, especially if it uses temp buffer. It is better to work on streams instead of buffers. Just for testing, if you write directly the fileData to your outFile, do you get OutOfMemoryError?
writeFile(filedata, Outfile);
I am trying to use the AndroidImageSlider library and populate it with images that I have downloaded as a base64 string.
The library only accepts URLs, R.drawable values, and the File object as parameters.
I am trying to convert the image string to a File object in order to be passed to the library function. I have been able to decode from base_64 and convert to a byte[] so far.
String imageData;
byte[] imgBytesData = android.util.Base64.decode(imageData, android.util.Base64.DEFAULT);
You'll need to save the File object to disk for that to work. This method will save the imageData string to disk and return the associated File object.
public static File saveImage(final Context context, final String imageData) {
final byte[] imgBytesData = android.util.Base64.decode(imageData,
android.util.Base64.DEFAULT);
final File file = File.createTempFile("image", null, context.getCacheDir());
final FileOutputStream fileOutputStream;
try {
fileOutputStream = new FileOutputStream(file);
} catch (FileNotFoundException e) {
e.printStackTrace();
return null;
}
final BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(
fileOutputStream);
try {
bufferedOutputStream.write(imgBytesData);
} catch (IOException e) {
e.printStackTrace();
return null;
} finally {
try {
bufferedOutputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return file;
}
It creates a temporary file in your applications 'cache' directory. However, you are still responsible for deleting the file once you no longer need it.