We Keep Coding

Java heap out of memory issue while zipping list of nio paths

I am asynchronously running the below method to zip the given set of nio paths. When there are multiple tasks running, java heap out of memory exception is encountered.
public InputStream compressToZip(String s3folderName, Set<Path> paths) throws Exception {
try {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(byteArrayOutputStream);
paths.forEach(path -> {
try {
System.out.println("Zipping " + path.getFileName());
zos.putNextEntry(new ZipEntry(path.getFileName().toString()));
FileInputStream ObjectInputStream = new FileInputStream(path.toFile());
IOUtils.copy(ObjectInputStream, zos);
zos.closeEntry();
} catch (Exception e) {
...
}
});
zos.close();
return new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
} catch (Exception e) {
...
}
}
The input stream returned from this file will be written on sftp location.
org.springframework.integration.file.remote.session.Session session = this.createSession(deliveryType, deliveryLocation);
zippedIpStream = fileCompressionSvc.compressToZip(s3folderName, fileDir);
session.write(zippedIpStream, deliveryLocation.getLocation().getFolder() + "/"
+ getfileNameFormat(fileNameFormat, masterId) + ".zip");
I am not sure what went wrong to occur java heap issue. Could you please assist me.

Changed the implementation to write the file into a file in local path and then send that file to sftp and then delete the temp zip file.
public void compressToZip(String s3folderName, Set<Path> distinctPaths, String efsPathWithFileName) throws Exception {
try(FileOutputStream fos = new FileOutputStream(efsPathWithFileName);
ZipOutputStream zos = new ZipOutputStream(fos)) {
distinctPaths.forEach(path -> {
try {
zos.putNextEntry(new ZipEntry(path.getFileName().toString()));
final FileInputStream fis = new FileInputStream(path.toFile());
IOUtils.copy(fis, zos);
zos.closeEntry();
} catch (IOException e) {
...
}
});
} catch (Exception e) {
...
throw e;
}
}
calling method:
InputStream zippedIpStream = new FileInputStream(tempCompressedFilePath);
session.write(zippedIpStream, deliveryLocation.getLocation().getFolder() + "/" + fileNameToWrite);
...
...
zippedIpStream.close();
...
...
Files.deleteIfExists(Paths.get(tempCompressedFilePath));

can't open genereted .xlsx by Apache POI which i send to frontend and download from browser (message: file damaged and cannot be opened)?

I have some error of open file.xlsx, genereted by librart Apache POI. File is save fine and open fine on my local machine. But after sending stream from java to vue it doesn't work. This is java:
public void sendFileToResponse(HttpServletResponse response, SearchParam searchParam) throws `IOException {`
String type = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
String fileNameAttr = "filename=MyExcel.xlsx";
response.setHeader("Content-Disposition", "attachment;" + fileNameAttr);
response.setHeader("Access-Control-Allow-Origin", "*");
response.setCharacterEncoding("utf-8");
response.setContentType(type + ";charset=UTF-8");
Workbook workbook;
FileOutputStream fileOutputStream = null;
ServletOutputStream outputStream = null;
try {
workbook = getFile(searchParam);
fileOutputStream = new FileOutputStream("MyExcel2.xlsx");
workbook.write(fileOutputStream);
outputStream = response.getOutputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
workbook.write(baos);
byte[] xlsx = baos.toByteArray();
outputStream.write(xlsx);
workbook.close();
fileOutputStream.flush();
fileOutputStream.close();
} catch (Exception e) {
logger.error("Error: {}", e);
}
here i get it on frontend:
saveStatistic () {
client.get('/statistics/statisticUseOfData/download', {responseType: 'blob'})
.then(response => {
console.log(response)
this.downloadFileName = 'test.xlsx'
var aHref = document.createElement('a')
var binaryData = [];
binaryData.push(response.data);
var urlBlob = window.URL.createObjectURL(new Blob(binaryData, {type: "application/octet-stream"}))
aHref.href = urlBlob
aHref.download = this.downloadFileName
document.body.appendChild(aHref)
aHref.click()
window.URL.revokeObjectURL(urlBlob)
})
.catch(error => {
console.log(error)
})
and that part of response, which i have in console.log(response.data):
PKw1Q[Content_Types].xml�S�n�0����*6�PU�C���\{�X�%����]8�R�
q�cfgfW�d�q�ZCB|��|�*�*h㻆},^�{Va�^K<4�6�N�XQ�dž�9�!P��$��҆�d�c�D�j);��ѝP�g��E�M'O�ʕ����H7L�h���R���G��^�'�{��zސʮB��3�˙��h.�h�W�жF�j娄CQՠ똈���}ιL�U:D�����%އ����,�B���[� �� ;˱� �{N��~��X�p�ykOL��kN�V��ܿBZ~����q�� �ar��{O�PKz��q;PKw1Q_rels/.rels���j�0�_���8�`�Q��2�m��4[ILb��ږ���.[K
�($}��v?�I�Q.���uӂ�h���x>=��#��p�H"�~�}� �n����*"�H�׺؁�����8�Z�^'�#��7m{��O�3���G�u�ܓ�'��y|a�����D� ��l_EYȾ����vql3�ML�eh���*���\3�Y0���oJ׏� :��^��}PK��z��IPKw1QdocProps/app.xmlM��

You don't need to write the workbook into a file and then into memory, you can write directly into the response:
try {
workbook = getFile(searchParam);
workbook.write(response.getOutputStream());
} catch (Exception e) {
logger.error("Error: {}", e);
}

I found the problem. It was in one custom layer before axios, which override http requests. And someone didn't add any params for this get, just url
Thank's you

How to download csv file using spring boot controller with contents?

I am facing the issue while downloading the csv file. I have created get api in spring boot and wanted to download the file through that api. Here is my code.
#GetMapping(value = "/citydetails/download")
public ResponseEntity<Object> getCityDetails() throws IOException {
System.out.println("Starting the rest call.");
FileWriter filewriter = null;
service = new LocalityService();
try {
List<CityDetails> details = service.getCityDetailsList();
if (details.isEmpty()) {
System.out.println("List is empty.");
}
StringBuilder fileContent = new StringBuilder("STATE,DISTRICT,CITY,VILLAGE,PIN\n");
for (CityDetails data : details)
fileContent.append(data.getStatename()).append(data.getDistrict()).append(data.getCity())
.append(data.getVillage()).append(data.getPindode());
XSSFWorkbook workbook = service.saveFileContents(details);
String filename = "C:\\Users\\" + System.getProperty("user.name") + "\\Downloads\\cityDetails.csv";
FileOutputStream out = new FileOutputStream(filename);
File file = new File("cityDetails.csv");
workbook.write(out);
out = new FileOutputStream(file);
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType("text/csv"))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + "cityDetails.csv" + "\"")
.body(file);
} catch (Exception ex) {
System.out.println("Failed to execute rest" + ex.getStackTrace() + "Locale: " + ex.getLocalizedMessage());
ErrorDetails errorDetails = new ErrorDetails(new Date(), ex.getMessage(), "false");
return new ResponseEntity<>(errorDetails, HttpStatus.INTERNAL_SERVER_ERROR);
} finally {
if (filewriter != null)
filewriter.close();
}
}
Exception:
Here i have used XSSFWorkbook to save my content to csv. Now i wanted to download the csv file through Rest api which will contain the data.I have tried multiple ways but i am getting empty file.With the above code it will save the csv file in download folder in windows machine, but i want this api to work on every system so I wanted to download the csv file with contents instead of saving it to specific location. I am facing the issue, how to resolve that?

HSSFWorkbook wb = new HSSFWorkbook();
HSSFSheet sheet=wkb.createSheet("sheet1");
HSSFRow row1=sheet.createRow(0);
HSSFCell cell=row1.createCell(0);
cell.setCellValue("table_demo");
sheet.addMergedRegion(new CellRangeAddress(0,0,0,3));
//input Excel date
HSSFRow row2=sheet.createRow(1);
row2.createCell(0).setCellValue("name");
row2.createCell(1).setCellValue("class");
row2.createCell(2).setCellValue("score_1");
row2.createCell(3).setCellValue("score_2");
HSSFRow row3=sheet.createRow(2);
row3.createCell(0).setCellValue("Jeams");
row3.createCell(1).setCellValue("High 3");
row3.createCell(2).setCellValue(87);
row3.createCell(3).setCellValue(78);
//output Excel file
OutputStream output=response.getOutputStream();
response.reset();
response.setHeader("Content-disposition", "attachment; filename=details.xls");
response.setContentType("application/msexcel");
wkb.write(output);
output.close();
This code can get a simple excel file

//image
#GetMapping(value = "/image")
public #ResponseBody byte[] getImage() throws IOException {
InputStream in = getClass()
.getResourceAsStream("/com/baeldung/produceimage/image.jpg");
return IOUtils.toByteArray(in);
}
//normal file
#GetMapping(
value = "/get-file",
produces = MediaType.APPLICATION_OCTET_STREAM_VALUE
)
public #ResponseBody byte[] getFile() throws IOException {
InputStream in = getClass()
.getResourceAsStream("/com/baeldung/produceimage/data.txt");
return IOUtils.toByteArray(in);
}

Create text file and add to zip file and download spring boot with out savind in local server

create and download zip file by adding list of text files. with out creating the file in local server, it should be download at client side direct,
Here i added a code snippet, it was creating in local server, but i dont want that, it should create and download at client side instant. Please help me in this way..
#GetMapping("/download/rawdata")
public void downloadRawdata(#RequestParam("date") String date){
log.info("date : "+date);
List<Rawdata> rawdatas = rawdataRepoisotry.findRawdataByDate(date);
log.info("size of rawdata : "+rawdatas.size());
List<File> files = new ArrayList<File>();
int i = 1;
for(Rawdata rawdata : rawdatas){
log.info("rawdata : "+ rawdata.getRawdata());
File file = new File(i+".txt");
try (Writer writer = new BufferedWriter(new FileWriter(file))) {
String contents = rawdata.getRawdata();
writer.write(contents);
files.add(file);
} catch (IOException e) {
e.printStackTrace();
}
i++;
}
try {
zipFile(files, new File(date+".zip"));
} catch (IOException e) {
e.printStackTrace();
throw new RuntimeException("Failed while creating Zip file");
}
}
public FileOutputStream zipFile(final List<File> files, final File targetZipFile) throws IOException {
try {
FileOutputStream fos = new FileOutputStream(targetZipFile);
ZipOutputStream zos = new ZipOutputStream(fos);
byte[] buffer = new byte[128];
for(File currentFile : files){
if (!currentFile.isDirectory()) {
ZipEntry entry = new ZipEntry(currentFile.getName());
FileInputStream fis = new FileInputStream(currentFile);
zos.putNextEntry(entry);
int read = 0;
while ((read = fis.read(buffer)) != -1) {
zos.write(buffer, 0, read);
}
zos.closeEntry();
fis.close();
}
}
zos.close();
fos.close();
return fos;
} catch (FileNotFoundException e) {
System.out.println("File not found : " + e);
throw new FileNotFoundException();
}
}

Here is an example using FileSystemResource.
What has been modified is (see the numbers in the commented code ) :
1) Declare that the response will be of type application/octet-stream
2) #ResponseBody
Annotation that indicates a method return value should be bound to the
web response body
3) Declare that the method returns a FileSystemResource body
4) Return the FileSystemResource entity based on your created zip file
Note that this will still create the file on the server side first, but you may want to use File.createTempFile and File.deleteOnExit.
#GetMapping("/download/rawdata", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)//1
#ResponseBody //2
public ResponseEntity<FileSystemResource> downloadRawdata(#RequestParam("date") String date){ //3
log.info("date : "+date);
List<Rawdata> rawdatas = rawdataRepoisotry.findRawdataByDate(date);
log.info("size of rawdata : "+rawdatas.size());
List<File> files = new ArrayList<File>();
int i = 1;
for(Rawdata rawdata : rawdatas){
log.info("rawdata : "+ rawdata.getRawdata());
File file = new File(i+".txt");
try (Writer writer = new BufferedWriter(new FileWriter(file))) {
String contents = rawdata.getRawdata();
writer.write(contents);
files.add(file);
} catch (IOException e) {
e.printStackTrace();
}
i++;
}
try {
File resultFile = new File(date+".zip");
zipFile(files, resultFile);
return new ResponseEntity<>(new FileSystemResource(resultFile), HttpStatus.OK); //4
} catch (IOException e) {
e.printStackTrace();
throw new RuntimeException("Failed while creating Zip file");
}
}

why my method retruniing null value?

I have declared variable directory as a global and using that variable below in my returnData method nut it is returning null value.
public void SaveImage(String FileName, Bitmap mBitmap) {
if (!Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)) {
String root = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES).toString();
File directory = new File(root + File.separator + "HMS_BARCODE");
directory.mkdirs();
//create a file to write bitmap data
File f = new File(directory, FileName + ".png");
Log.e("dir", "" + directory);
try {
f.createNewFile();
} catch (IOException e) {
e.printStackTrace();
Log.e("IOException", "IOException");
}
//Convert bitmap to byte array
ByteArrayOutputStream bos = new ByteArrayOutputStream();
mBitmap.compress(Bitmap.CompressFormat.PNG, 0, bos);
byte[] bytearray = bos.toByteArray();
//Write bytes in file
FileOutputStream fos = null;
try {
fos = new FileOutputStream(f);
fos.write(bytearray);
fos.flush();
fos.close();
} catch (Exception e) {
e.printStackTrace();
Log.e("Exception", "" + e);
}
} else {
String root = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES).toString();
directory = new File(root + File.separator + "HMS_BARCODE");
if (!directory.exists()) {
directory.mkdirs();
}
File f = new File(directory, FileName + ".png");
Log.e("dir1", "" + directory);
try {
f.createNewFile();
} catch (IOException e) {
e.printStackTrace();
Log.e("IOException", "IOException");
}
Log.e("dir1", "" + directory);
//Convert bitmap to byte array
ByteArrayOutputStream bos = new ByteArrayOutputStream();
mBitmap.compress(Bitmap.CompressFormat.PNG, 0, bos);
byte[] bytearray = bos.toByteArray();
//Write bytes in file
FileOutputStream fos = null;
try {
fos = new FileOutputStream(f);
fos.write(bytearray);
fos.flush();
fos.close();
} catch (Exception e) {
e.printStackTrace();
Log.e("Exception", "" + e);
}
}
}
// this is method returning null value I want want that directory
// value to pass to another class
public File returnData() {
Log.e("Exception", "" + directory);
return directory;
}

Please format your question correctly, it is hard to read. From what I can see your directory variable is not global, it is local for saveImage method.
If you want to have access to the directory variable from different methods of the same class instance, then you need to declare it as a class variable. For example:
public class MyClass {
private File directory;
public void saveImage(...) {....}
public File returnData(...) {...}
}

You must call SaveImage method in your returnData method :
//this is method returning null value i want want that directory value to pass to another class
public File returnData(){
SaveImage(.../*the parameters*/);
Log.e("Exception", "" + directory);
return directory;
}