We Keep Coding

Workbook takes long time to generate excel file

I'm trying to generate excel file with 200k records. But it is taking almost 2 hours to generate the file.
Here is my code of generating excel file.
Workbook workbook=null;
csvFileName = userId+"_Records_"+new SimpleDateFormat("yyyyMMddHHmmss")
.format(new Date())+".xls";
path = ReadPropertyFile.getProperties("download.reports.path");
misService.insertXLSRecord(ackNo,"-",null, VspCommonConstants.getIpFromRequest(request),
new Date(), userId,"N",userReportRoleId);
workbook = getWorkbook(path+csvFileName);
Sheet sheet = workbook.createSheet(WorkbookUtil.createSafeSheetName(studAppForm.get(0)
.getScheme_Id()+"_"+studAppForm.get(0).getEFP_Scholarship_Name(),'_'));
if(schemeQuestionData.containsKey(currSheetSchemeId))
createXLSHeaders(sheet,schemeQuestionData.get(currSheetSchemeId));
Row row = sheet.createRow(++rowCount);
currAppId=studAppForm.get(j).getApp_Id().toString();
jspTableAppIds.remove(jspTableAppIds.indexOf(new BigInteger(currAppId)));
writeBook(studAppForm.get(j), row);
Here is my createXLSHeaders method to create header
void createXLSHeaders( Sheet sheet, List<SchemeMasterBean> schemeMasterBeanList){
LOGGER.info("Creating XLS SheetHeaders for sheet "+sheet.getSheetName());
// Sheet sheet = workbook.createSheet();
Row header = sheet.createRow(0);
header.createCell(0).setCellValue("APPLICATION ID");
header.createCell(1).setCellValue("APPLICATION STATUS");
header.createCell(2).setCellValue("APPLICATION DATE");
header.createCell(3).setCellValue("SCHEME/SCHOLARSHIP APPLIED");
header.createCell(4).setCellValue("SCHEME ID");
header.createCell(5).setCellValue("STUDENT ID");
header.createCell(6).setCellValue("STUDENT FULL NAME");
.
.
.
62 heading...
int i=73;
if(schemeMasterBeanList!=null)
for(SchemeMasterBean schemeMasterBean :schemeMasterBeanList){
if(!schemeMasterBean.getSmSchemeType().equals("5") &&
!schemeMasterBean.getSmSchemeType().equals("6")){
header.createCell(i).setCellValue(schemeMasterBean.getSmScholarshipName());
i++;
}
}
}
and finally writebook method
private void writeBook(StudentAppFormVsp saf, Row row) throws JSONException {
Cell cell = row.createCell(0);
cell.setCellValue(saf.getApp_Id()!=null?saf.getApp_Id().toString():"");
cell = row.createCell(1);
cell.setCellValue(saf.getApp_Status()!=null?getApplicationStatusMap().get(saf.getApp_Status()):"");
cell = row.createCell(2);
cell.setCellValue(saf.getCrtn_time()!=null?saf.getCrtn_time().toString():"");
cell = row.createCell(3);
cell.setCellValue(saf.getEFP_Scholarship_Name()!=null?saf.getEFP_Scholarship_Name().toString():"");
cell = row.createCell(4);
cell.setCellValue(saf.getScheme_Id()!=null?saf.getScheme_Id().toString():"");
cell = row.createCell(5);
cell.setCellValue(saf.getStud_Id()!=null?saf.getStud_Id().toString():"");
.
.
62 rows
}
How to reduce the excel sheet generation time?

First: play around with memory for the application if possible.
Then: the tip on using a profiler is really worth the effort.
Any DOM, XML, Excel or otherwise often suffer from location references searching from top to the actual position.
Creating a DOM instead of writing sequentially is costly with respect to memory, and can slow things down. Maybe consider this.
You could make two loop: writing to a CSV file, and then creating an XLS(X).
Then you know where the complexity resides.
The following (I rewrote a bit) is slightly suspect: toString + new BigInteger points to a conversion; I hope not from BigInteger to String to BigInteger.
StudentAppFormVsp saf = studAppForm.get(j);
currAppId = saf.getApp_Id().toString();
jspTableAppIds.remove(jspTableAppIds.indexOf(BigInteger.valueOf(currAppId)));
writeBook(saf, row);

Java Apache POI Unable to Get Code Working Correctly

Edit - thanks for responses. Have made changes and as suggested found the problem was a NOW on empty rows.
I am writing a program that loads various excel sheets and provides an output based on certain criteria. See code below. My problem is that the code does not write or save to the DISPLAY sheet. I realise the code below is not particularly neat but have copy pasted to check if I can get anything to save. If I comment out everything within the loop and try to write to the cell at the end before I save, it works and shows in the sheet. If I don't comment out the data formatter part it doesn't work even when trying to write to cell just before the part where I am saving output file. I am guessing the problem is caused by the data formatter, but I can't work out why.
public void checkForChanges() {
try {
FileInputStream fsIP = new FileInputStream("Change.xls");
HSSFWorkbook fWorkbook = new HSSFWorkbook(fsIP);
HSSFSheet recipeSheet = fWorkbook.getSheet("RECIPE STEPS");
HSSFSheet fromSheet = fWorkbook.getSheet("FROM FORMAT");
HSSFSheet toSheet = fWorkbook.getSheet("TO FORMAT");
HSSFSheet displaySheet = fWorkbook.getSheet("DISPLAY");
for (int i = 0; i < 30; i++) {
DataFormatter recipeFormatter = new DataFormatter();
HSSFRow recipeRow = recipeSheet.getRow(i);
HSSFCell recipeCellsColumnA = recipeRow.getCell(0);
String recipeCellValueColumnA = recipeFormatter.formatCellValue(recipeCellsColumnA);
System.out.println(recipeCellValueColumnA);
HSSFCell recipeCellsColumnB = recipeRow.getCell(1);
String recipeCellValueColumnB = recipeFormatter.formatCellValue(recipeCellsColumnB);
System.out.println(recipeCellValueColumnB);
DataFormatter fromFormatter = new DataFormatter();
HSSFRow fromRow = fromSheet.getRow(i);
HSSFCell fromCells = fromRow.getCell(0);
String fromCellValue = fromFormatter.formatCellValue(fromCells);
System.out.println(fromCellValue);
DataFormatter toFormatter = new DataFormatter();
HSSFRow toRow = toSheet.getRow(i);
HSSFCell toCells = toRow.getCell(0);
String toCellValue = toFormatter.formatCellValue(toCells);
System.out.println(toCellValue);
if (recipeCellValueColumnB.equals("YES") && !fromCellValue.equals(toCellValue)) {
System.out.println("PUT VALUE FROM 'TO FORMAT' COLUMN A ROW I INTO 'DISPLAY' SHEET CELL COLUMN B ROW I");
System.out.println("PUT VALUE FROM 'RECIPE STEPS' COLUMN A ROW I INTO 'DISPLAY' SHEET CELL COLUMN A ROW I");
Row row = displaySheet.createRow(0);
Cell cell = row.createCell(0);
cell.setCellValue("TEST");
} else if (recipeCellValueColumnB.equals("YES") && fromCellValue.equals(toCellValue)) {
System.out.println("SET CELL IN 'DISPLAY' SHEET COLUMN A ROW I TO '' ");
System.out.println("SET CELL IN 'DISPLAY' SHEET COLUMN B ROW I TO '' ");
Row row = displaySheet.createRow(0);
Cell cell = row.createCell(0);
cell.setCellValue("TEST");
} else if (recipeCellValueColumnB.equals("NO")) {
System.out.println("PUT VALUE FROM 'TO FORMAT' COLUMN A ROW I INTO 'DISPLAY' SHEET CELL COLUMN B ROW I");
System.out.println("PUT VALUE FROM 'RECIPE STEPS' COLUMN A ROW I INTO 'DISPLAY' SHEET CELL COLUMN A ROW I");
Row row = displaySheet.createRow(0);
Cell cell = row.createCell(0);
cell.setCellValue("TEST");
}else if (recipeCellValueColumnA.equals("Step Name") && recipeCellValueColumnB.equals("Always Compare?")) {
System.out.println("SET CELL IN DISPLAY COLUMN A ROW I TO 'REQUIRED STEPS'");
System.out.println("PUT VALUE FROM 'TO FORMAT' COLUMN A ROW I INTO 'DISPLAY' SHEET CELL COLUMN B ROW I");
Row row = displaySheet.createRow(0);
Cell cell = row.createCell(0);
cell.setCellValue("TEST");
}
}
FileOutputStream output_file = new FileOutputStream(new File("Change.xls"));
BufferedOutputStream bos = new BufferedOutputStream(output_file);
fWorkbook.write(output_file);
fWorkbook.close();
bos.close();
output_file.close();
}catch(Exception e){
}
}
Sorry for any poor explanation, it's late and I'm tired and frustrated!
Thanks

The problem with your code is this:
catch(Exception e)
{
}
This is saying "don't tell me about any exceptions". This is exception squashing ... and it is horrible, and lazy, and just plain wrong.
Somewhere in your code there is probably something that is either throwing an exception directly, or causing POI (or something) to throw an exception. That's a bug. But your horrible exception squashing is throwing away the evidence that will allow you to identify and then fix the bug.
I am guessing the problem is caused by the data formatter, but I can't work out why.
It could be many things ... including something daft like an NPE or a passing an out-of-range index of an incorrect filename.
Solution:
Get rid of the try catch, and allow the exceptions to propagate to the caller.
In the caller (or further up the stack) handle unexpected exceptions by printing or logging a stacktrace and causing the application to fail.
Run the modified program.
When it fails (at it probably will), read the exception message and stacktrace and work out what the underlying problem is; i.e the problam that your horrible exception squashing is hiding.
Then remember to NEVER squash all exceptions like that, in Java or in any other programming language.
(It is sometimes OK to squash a specific exception in a specific context ... but only after carefully analyzing the code to ensure that that you won't squash other (unexpected) exceptions at the same time.)

Apache POI how to apply Workbook Style to all Pivot Tables. how do i exclude sheets?

So i am working on a program which reads a text file and writes this data in an excel workbook. after the data is written i create pivot tables from the data. to get style in the pivot table i've set the whole workbook style which somewhy only applys to the sheets with the pivot tables and not the data sheets. now i want to exclude one sheet with pivot table from the workbook style. is there a way to exclude one sheet or set the style for the sheets which need it?
code for workbook style:
wb.createDataFormat().putFormat((short) 0, "_($* #,##0.00_);_($* (#,##0.00);_($* \"-\"??_);_(#_)");
pivot table:
CellReference cr = new CellReference("A1");
CellReference c1 = new CellReference(0, 0);
CellReference c2 = new CellReference(data.getPhysicalNumberOfRows() - 1, data.getRow(0).getLastCellNum() - 1);
AreaReference ar = new AreaReference(c1, c2);
XSSFPivotTable pivotTable = sheet.createPivotTable(ar, cr, data);
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(4).setAxis(STAxis.AXIS_COL);
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(4).addNewItems();
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(4).getItems().addNewItem()
.setT(STItemType.DEFAULT);
pivotTable.getCTPivotTableDefinition().addNewColFields().addNewField().setX(4);
pivotTable.addRowLabel(10);
pivotTable.addRowLabel(11);
pivotTable.addRowLabel(1);
pivotTable.addColumnLabel(DataConsolidateFunction.COUNT, 0);
i already tried to set the style for the cells in the sheet but didn't work. any suggestions?

so i found an solution here: Apache Poi set data field style for pivot table. this question might also be more accurate on what i wanted to achieve

Apache POI Excel - how to configure columns to be expanded?

I am using Apache POI API to generate excel spreadsheet to output some data.
The problem I am facing is when the spreadsheet is created and opened, columns are not expanded so that some long text like Date formatted text is not showing up on first glance.
I could just double click the column border in excel to expand or drag the border to adjust the column width but there could be 20+ columns and there is no way I want to do that manually every time I open the spreadsheet :(
I found out (though could be wrong method) groupRow() and setColumnGroupCollapsed() might be able to do the trick but no luck. Maybe I'm using it in wrong way.
Sample Code snippet
Workbook wb = new HSSFWorkbook();
CreationHelper createHelper = wb.getCreationHelper();
//create sheet
Sheet sheet = wb.createSheet("masatoSheet");
//not really working yet.... :(
//set group for expand/collapse
//sheet.groupRow(0, 10); //just random fromRow toRow argument values...
//sheet.setColumnGroupCollapsed(0, true);
//create row
Row row = sheet.createRow((short)0);
//put a cell in the row and store long text data
row.createCell(0).setCellValue("Loooooooong text not to show up first");
When this spreadsheet is created, the "Looooooong text not to show up first" string is in the cell but since the column is not expanded only "Loooooooo" is showing up.
How can I configure it so that when I open my spreadsheet, the column is already expanded???

After you have added all your data to the sheet, you can call autoSizeColumn(int column) on your sheet to autofit the columns to the proper size
Here is a link to the API.
See this post for more reference
Problem in fitting the excel cell size to the size of the content when using apache poi

Tip : To make Auto size work , the call to sheet.autoSizeColumn(columnNumber) should be made after populating the data into the excel.
Calling the method before populating the data, will have no effect.

If you want to auto size all columns in a workbook, here is a method that might be useful:
public void autoSizeColumns(Workbook workbook) {
int numberOfSheets = workbook.getNumberOfSheets();
for (int i = 0; i < numberOfSheets; i++) {
Sheet sheet = workbook.getSheetAt(i);
if (sheet.getPhysicalNumberOfRows() > 0) {
Row row = sheet.getRow(sheet.getFirstRowNum());
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext()) {
Cell cell = cellIterator.next();
int columnIndex = cell.getColumnIndex();
sheet.autoSizeColumn(columnIndex);
}
}
}
}

You can try something like this:
HSSFSheet summarySheet = wb.createSheet();
summarySheet.setColumnWidth(short column, short width);
Here params are:column number in sheet and its width
But,the units of width are pretty small, you can try 4000 for example.

For Excel POI:
sheetName.autoSizeColumn(cellnum);

sample code below
HSSFWorkbook wb = new HSSFWorkbook();
HSSFSheet sheet = wb.createSheet("your sheet name");
HSSFRow row = sheet.createRow(0);
cell = row.createCell(0);
cell.setCellValue("A BIG NAME WITH AUTO SIZE FEATURE ENABLED");
//this is crucial
sheet.autoSizeColumn(0);
//argument must be cell number
cell = row.createCell(1);
cell.setCellValue("a big name without auto size feature enabled");
Check the output and go nuts :)

If you know the count of your columns (f.e. it's equal to a collection list). You can simply use this one liner to adjust all columns of one sheet (if you use at least java 8):
IntStream.range(0, columnCount).forEach(sheet::autoSizeColumn)

You can add this, after your loop.
for (int i = 0; i<53;i++) {
sheet.autoSizeColumn(i);
}

I use below simple solution:
This is your workbook and sheet:
XSSFWorkbook workbook = new XSSFWorkbook();
XSSFSheet sheet = workbook.createSheet("YOUR Workshhet");
then add data to your sheet with columns and rows. Once done with adding data to sheet write following code to autoSizeColumn width.
for (int columnIndex = 0; columnIndex < 15; columnIndex++) {
sheet.autoSizeColumn(columnIndex);
}
Here, instead 15, you add the number of columns in your sheet.
Hope someone helps this.

You can use setColumnWidth() if you want to expand your cell more.

Its very simple, use this one line code
dataSheet.autoSizeColumn(0)
or give the number of column in bracket
dataSheet.autoSizeColumn(cell number )

You can wrap the text as well. PFB sample code:
CellStyle wrapCellStyle = new_workbook.createCellStyle();
wrapCellStyle.setWrapText(true);

Find Excel Cell by Text in Apache POI

I'd like to find a cell in an Excel sheet by its text. The text is something like %t:
sheet.findCell("%t"); // pseudo-code, not working
My goal is to enable the user to provide kind of template, in which data is written. Colours and fonts, as well as data's position can be configured by the user in an Excel file. This %t cell is the top-left corner of the data table.
Additional question: Is there a more elegant way to get this job done?
EDIT I'm iterating over the rows and cells to find it. I'm afraid it's not really efficient, but it works so far:
public static Cell findCell(XSSFSheet sheet, String text) {
for(Row row : sheet) {
for(Cell cell : row) {
if(text.equals(cell.getStringCellValue()))
return cell;
}
}
return null;
}

You can iterate through the cells of the sheet and investigate the contents. I don't think there is an easier method.

Its an old post but still i want to publish my code.
You can define a file path.
String inputFile = "src\main\resources\file.xlsx";
XSSFWorkbook xssfWorkbook = new XSSFWorkbook(new FileInputStream(inputFile));
DataFormatter formatter = new DataFormatter();
for (XSSFSheet sheet : xssfWorkbook) {
for (Row row : sheet) {
for (Cell cell : row) {
if (formatter.formatCellValue(cell).contains("name")){
cell.setCellValue("test");
}
}
}
}
xssfWorkbook.write(new FileOutputStream(inputFile));