We Keep Coding

Workbook takes long time to generate excel file

I'm trying to generate excel file with 200k records. But it is taking almost 2 hours to generate the file.
Here is my code of generating excel file.
Workbook workbook=null;
csvFileName = userId+"_Records_"+new SimpleDateFormat("yyyyMMddHHmmss")
.format(new Date())+".xls";
path = ReadPropertyFile.getProperties("download.reports.path");
misService.insertXLSRecord(ackNo,"-",null, VspCommonConstants.getIpFromRequest(request),
new Date(), userId,"N",userReportRoleId);
workbook = getWorkbook(path+csvFileName);
Sheet sheet = workbook.createSheet(WorkbookUtil.createSafeSheetName(studAppForm.get(0)
.getScheme_Id()+"_"+studAppForm.get(0).getEFP_Scholarship_Name(),'_'));
if(schemeQuestionData.containsKey(currSheetSchemeId))
createXLSHeaders(sheet,schemeQuestionData.get(currSheetSchemeId));
Row row = sheet.createRow(++rowCount);
currAppId=studAppForm.get(j).getApp_Id().toString();
jspTableAppIds.remove(jspTableAppIds.indexOf(new BigInteger(currAppId)));
writeBook(studAppForm.get(j), row);
Here is my createXLSHeaders method to create header
void createXLSHeaders( Sheet sheet, List<SchemeMasterBean> schemeMasterBeanList){
LOGGER.info("Creating XLS SheetHeaders for sheet "+sheet.getSheetName());
// Sheet sheet = workbook.createSheet();
Row header = sheet.createRow(0);
header.createCell(0).setCellValue("APPLICATION ID");
header.createCell(1).setCellValue("APPLICATION STATUS");
header.createCell(2).setCellValue("APPLICATION DATE");
header.createCell(3).setCellValue("SCHEME/SCHOLARSHIP APPLIED");
header.createCell(4).setCellValue("SCHEME ID");
header.createCell(5).setCellValue("STUDENT ID");
header.createCell(6).setCellValue("STUDENT FULL NAME");
.
.
.
62 heading...
int i=73;
if(schemeMasterBeanList!=null)
for(SchemeMasterBean schemeMasterBean :schemeMasterBeanList){
if(!schemeMasterBean.getSmSchemeType().equals("5") &&
!schemeMasterBean.getSmSchemeType().equals("6")){
header.createCell(i).setCellValue(schemeMasterBean.getSmScholarshipName());
i++;
}
}
}
and finally writebook method
private void writeBook(StudentAppFormVsp saf, Row row) throws JSONException {
Cell cell = row.createCell(0);
cell.setCellValue(saf.getApp_Id()!=null?saf.getApp_Id().toString():"");
cell = row.createCell(1);
cell.setCellValue(saf.getApp_Status()!=null?getApplicationStatusMap().get(saf.getApp_Status()):"");
cell = row.createCell(2);
cell.setCellValue(saf.getCrtn_time()!=null?saf.getCrtn_time().toString():"");
cell = row.createCell(3);
cell.setCellValue(saf.getEFP_Scholarship_Name()!=null?saf.getEFP_Scholarship_Name().toString():"");
cell = row.createCell(4);
cell.setCellValue(saf.getScheme_Id()!=null?saf.getScheme_Id().toString():"");
cell = row.createCell(5);
cell.setCellValue(saf.getStud_Id()!=null?saf.getStud_Id().toString():"");
.
.
62 rows
}
How to reduce the excel sheet generation time?

First: play around with memory for the application if possible.
Then: the tip on using a profiler is really worth the effort.
Any DOM, XML, Excel or otherwise often suffer from location references searching from top to the actual position.
Creating a DOM instead of writing sequentially is costly with respect to memory, and can slow things down. Maybe consider this.
You could make two loop: writing to a CSV file, and then creating an XLS(X).
Then you know where the complexity resides.
The following (I rewrote a bit) is slightly suspect: toString + new BigInteger points to a conversion; I hope not from BigInteger to String to BigInteger.
StudentAppFormVsp saf = studAppForm.get(j);
currAppId = saf.getApp_Id().toString();
jspTableAppIds.remove(jspTableAppIds.indexOf(BigInteger.valueOf(currAppId)));
writeBook(saf, row);

How to set Multiple default value in Apache POI pivot table report filter

I need some help, I'm working on a project where I need to create a pivot table with report filter. Now I want to set Multiple default value to the report filter. I've gone through this previously posted question Click here!
I couldn't find anything online related this. Thanks in advance.

There has not much changed at creating pivot tables using apache poi since my last answer. So my code in the linked Q/A in your question only can extended a little bit for fulfilling the requirement selecting multiple items as default in page filter. A code which would be general able creating pivot tables from all kind of data, as Excel can do, is further on not possible.
If the need is selecting multiple items, first MultipleItemSelectionAllowed needs to be set in appropriate pivot field.
Then for each item which not shall be selected H(idden) needs set true.
Example:
import org.apache.poi.xssf.usermodel.*;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.ss.util.*;
import org.apache.poi.ss.SpreadsheetVersion;
import java.util.Random;
import java.io.*;
class PivotTableTest4 {
private static void setCellData(Sheet sheet) {
Row row = sheet.createRow(0);
Cell cell = row.createCell(0);
cell.setCellValue("Name");
cell = row.createCell(1);
cell.setCellValue("Value1");
cell = row.createCell(2);
cell.setCellValue("Value2");
cell = row.createCell(3);
cell.setCellValue("City");
for (int r = 1; r < 15; r++) {
row = sheet.createRow(r);
cell = row.createCell(0);
cell.setCellValue("Name " + ((r-1) % 5 + 1));
cell = row.createCell(1);
cell.setCellValue(r * new java.util.Random().nextDouble());
cell = row.createCell(2);
cell.setCellValue(r * new java.util.Random().nextDouble());
cell = row.createCell(3);
cell.setCellValue("City " + ((r-1) % 4 + 1));
}
}
public static void main(String[] args) {
try {
XSSFWorkbook wb = new XSSFWorkbook();
XSSFSheet sheet = wb.createSheet();
//Create some data to build the pivot table on
setCellData(sheet);
XSSFPivotTable pivotTable = sheet.createPivotTable(
new AreaReference(new CellReference("A1"), new CellReference("D15"), SpreadsheetVersion.EXCEL2007), new CellReference("H5"));
//Configure the pivot table
//Use first column as row label
pivotTable.addRowLabel(0);
//Sum up the second column
pivotTable.addColumnLabel(DataConsolidateFunction.SUM, 1);
//Avarage the third column
pivotTable.addColumnLabel(DataConsolidateFunction.AVERAGE, 2);
//Add fourth column as page filter
pivotTable.addReportFilter(3);
/*
Apache poi adds 15 pivot field items of type "default" (<item t="default"/>) here.
This is because there are 15 rows (A1:D15) and, because they don't have a look at the data,
they are assuming max 15 different values. This is fine because Excel will rebuild its pivot cache while opening.
But if we want preselect items, then this is not fine. Then we must know what items there are that can be preselected.
So we need at least as much items as we want preselecting as numbered items: <item x="0"/><item x="1"/><item x="2"/>...
And we must build a cache definition which has shared elements for those items.
*/
for (int i = 0; i < 4; i++) {
//take the first 4 items as numbered items: <item x="0"/><item x="1"/><item x="2"/><item x="3"/>
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(3).getItems().getItemArray(i).unsetT();
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(3).getItems().getItemArray(i).setX((long)i);
//build a cache definition which has shared elements for those items
//<sharedItems><s v="City 1"/><s v="City 2"/><s v="City 3"/><s v="City 4"/></sharedItems>
pivotTable.getPivotCacheDefinition().getCTPivotCacheDefinition().getCacheFields().getCacheFieldArray(3).getSharedItems().addNewS().setV("City " + (i+1));
}
//Now we can predefinite a page filter. Second item, which is "City 2", in this case.
//pivotTable.getCTPivotTableDefinition().getPageFields().getPageFieldArray(0).setItem(1);
//If the need is selecting multiple items, first MultipleItemSelectionAllowed needs to be set.
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(3).setMultipleItemSelectionAllowed(true);
//Then set H(idden) true for all items which not shall be selected. First ("City 1") and fourth ("City 4") in this case.
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(3).getItems().getItemArray(0).setH(true);
pivotTable.getCTPivotTableDefinition().getPivotFields().getPivotFieldArray(3).getItems().getItemArray(3).setH(true);
FileOutputStream fileOut = new FileOutputStream("PivotTableTest4.xlsx");
wb.write(fileOut);
fileOut.close();
wb.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
This needs the full jar of all of the schemas, ooxml-schemas-1.3.jar, as mentioned in the FAQ.

How to get the Excel sheet cell data by column name in using apache POI library

I want to get the cell data of a particular sheet with column name rather than column index,
Currently i can get the cell data using
fis = new FileInputStream(ExcelPath);
wb = new XSSFWorkbook(fis);
sh = wb.getSheet(SheetName);
cell = sh.getRow(rownum).getCell(colnum);
but I am looking for a way where I can get the cell data by,
cell = sh.getRow(rownum).getCell(ColName); //not working

Excel columns don't have names. They only have letters, eg A, D, BB
To work out how to map an Excel-style column letter into a POI-style 0-based index, you need to use the method convertColStringToIndex(java.lang.String ref) which does it for you
If you mean you want to lookup a column based on the value held in the first row, you should do something like:
DataFormatter fmt = new DataFormatter();
Map<String,Integer> headings = new HashMap<String,Integer>();
Row headings = sheet.getRow(0);
if (headings == null) throw new IllegalArgumentException("Empty headings row");
for (Cell c : headings) {
headings.put(fmt.formatCellValue(c), c.getColumnIndex());
}
If you have more complex use cases, such as a heading running over multiple rows, you'll need to write more complex code... It's all driven by your use-case

Apache POI blank values

I am using Apache POI to import data from excel file to database.(newbie to APACHE POI)
In which I am allowing user to select columns from excel sheet and Map those columns to the Database columns. After mapping the columns, when I try to insert the records from Excel to Database then:
If Columns with NO blank values in them are Mapped then Proper data is inserted into the database
If columns are Mapped with BLANK values in them, then if a Excel Cell has blank value then previous value of that column is assigned.
Source Code:
FileInputStream file = new FileInputStream(new File("C:/Temp.xls"));
HSSFWorkbook workbook = new HSSFWorkbook(file); //Get the workbook instance for XLS file
HSSFSheet sheet = workbook.getSheetAt(0); //Get first sheet from the workbook
Iterator<Row> rowIterator = sheet.iterator(); //Iterate through each rows from first sheet
while (rowIterator.hasNext())
{
HSSFRow hssfRow = (HSSFRow) rowIterator.next();
Iterator<Cell> iterator = hssfRow.cellIterator();
int current = 0, next = 1;
while (iterator.hasNext())
{
HSSFCell hssfCell = (HSSFCell) iterator.next();
current = hssfCell.getColumnIndex();
for(int i=0;i<arrIndex.length;i++) //arrayIndex is array of Excel cell Indexes selected by the user
{
if(arrIndex[i] == hssfCell.getColumnIndex())
{
if(current<next)
{
//System.out.println("Condition Satisfied");
}
else
{
System.out.println( "pstmt.setString("+next+",null);");
pstmt.setString(next,null);
next = next + 1;
}
System.out.println( "pstmt.setString("+next+","+((Object)hssfCell).toString()+");");
pstmt.setString(next,((Object)hssfCell).toString());
next = next + 1;
}
}
}
pstmt.addBatch();
}
I have look for similar questions on SO, but still not able to solve the issue.. So any help will be appreciated.
Thanks in advance..

You've made a very common mistake, which has been covered in rather a lot of past StackOverflow questions
As the Apache POI documentation on cell iterating says
In some cases, when iterating, you need full control over how missing or blank cells are treated, and you need to ensure you visit every cell and not just those defined in the file. (The CellIterator will only return the cells defined in the file, which is largely those with values or stylings, but it depends on Excel).
It sounds like you are in that situation, where you need to care about hitting every row/cell, and not just grabbing all the available cells without worrying about the gaps
You'll want to change you code to look somewhat like the example in the POI docs:
// Decide which rows to process
int rowStart = Math.min(15, sheet.getFirstRowNum());
int rowEnd = Math.max(1400, sheet.getLastRowNum());
for (int rowNum = rowStart; rowNum < rowEnd; rowNum++) {
Row r = sheet.getRow(rowNum);
int lastColumn = Math.max(r.getLastCellNum(), MY_MINIMUM_COLUMN_COUNT);
for (int cn = 0; cn < lastColumn; cn++) {
Cell c = r.getCell(cn, Row.RETURN_BLANK_AS_NULL);
if (c == null) {
// The spreadsheet is empty in this cell
// Mark it as blank in the database if needed
} else {
// Do something useful with the cell's contents
}
}
}

Apache POI Excel - how to configure columns to be expanded?

I am using Apache POI API to generate excel spreadsheet to output some data.
The problem I am facing is when the spreadsheet is created and opened, columns are not expanded so that some long text like Date formatted text is not showing up on first glance.
I could just double click the column border in excel to expand or drag the border to adjust the column width but there could be 20+ columns and there is no way I want to do that manually every time I open the spreadsheet :(
I found out (though could be wrong method) groupRow() and setColumnGroupCollapsed() might be able to do the trick but no luck. Maybe I'm using it in wrong way.
Sample Code snippet
Workbook wb = new HSSFWorkbook();
CreationHelper createHelper = wb.getCreationHelper();
//create sheet
Sheet sheet = wb.createSheet("masatoSheet");
//not really working yet.... :(
//set group for expand/collapse
//sheet.groupRow(0, 10); //just random fromRow toRow argument values...
//sheet.setColumnGroupCollapsed(0, true);
//create row
Row row = sheet.createRow((short)0);
//put a cell in the row and store long text data
row.createCell(0).setCellValue("Loooooooong text not to show up first");
When this spreadsheet is created, the "Looooooong text not to show up first" string is in the cell but since the column is not expanded only "Loooooooo" is showing up.
How can I configure it so that when I open my spreadsheet, the column is already expanded???

After you have added all your data to the sheet, you can call autoSizeColumn(int column) on your sheet to autofit the columns to the proper size
Here is a link to the API.
See this post for more reference
Problem in fitting the excel cell size to the size of the content when using apache poi

Tip : To make Auto size work , the call to sheet.autoSizeColumn(columnNumber) should be made after populating the data into the excel.
Calling the method before populating the data, will have no effect.

If you want to auto size all columns in a workbook, here is a method that might be useful:
public void autoSizeColumns(Workbook workbook) {
int numberOfSheets = workbook.getNumberOfSheets();
for (int i = 0; i < numberOfSheets; i++) {
Sheet sheet = workbook.getSheetAt(i);
if (sheet.getPhysicalNumberOfRows() > 0) {
Row row = sheet.getRow(sheet.getFirstRowNum());
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext()) {
Cell cell = cellIterator.next();
int columnIndex = cell.getColumnIndex();
sheet.autoSizeColumn(columnIndex);
}
}
}
}

You can try something like this:
HSSFSheet summarySheet = wb.createSheet();
summarySheet.setColumnWidth(short column, short width);
Here params are:column number in sheet and its width
But,the units of width are pretty small, you can try 4000 for example.

For Excel POI:
sheetName.autoSizeColumn(cellnum);

sample code below
HSSFWorkbook wb = new HSSFWorkbook();
HSSFSheet sheet = wb.createSheet("your sheet name");
HSSFRow row = sheet.createRow(0);
cell = row.createCell(0);
cell.setCellValue("A BIG NAME WITH AUTO SIZE FEATURE ENABLED");
//this is crucial
sheet.autoSizeColumn(0);
//argument must be cell number
cell = row.createCell(1);
cell.setCellValue("a big name without auto size feature enabled");
Check the output and go nuts :)

If you know the count of your columns (f.e. it's equal to a collection list). You can simply use this one liner to adjust all columns of one sheet (if you use at least java 8):
IntStream.range(0, columnCount).forEach(sheet::autoSizeColumn)

You can add this, after your loop.
for (int i = 0; i<53;i++) {
sheet.autoSizeColumn(i);
}

I use below simple solution:
This is your workbook and sheet:
XSSFWorkbook workbook = new XSSFWorkbook();
XSSFSheet sheet = workbook.createSheet("YOUR Workshhet");
then add data to your sheet with columns and rows. Once done with adding data to sheet write following code to autoSizeColumn width.
for (int columnIndex = 0; columnIndex < 15; columnIndex++) {
sheet.autoSizeColumn(columnIndex);
}
Here, instead 15, you add the number of columns in your sheet.
Hope someone helps this.

You can use setColumnWidth() if you want to expand your cell more.

Its very simple, use this one line code
dataSheet.autoSizeColumn(0)
or give the number of column in bracket
dataSheet.autoSizeColumn(cell number )

You can wrap the text as well. PFB sample code:
CellStyle wrapCellStyle = new_workbook.createCellStyle();
wrapCellStyle.setWrapText(true);