I have a JSON which has entries that look like
[{ "key": { "keyLabel": "Label1" }, "specs": [{ "specKey":
"spec1", "specValue": ["s11"] }, { "specKey": "spec2",
"specValue": ["s12"] }] },
{ "key": { "keyLabel":
"Label2" }, "specs": [{ "specKey": "spec1", "specValue":
["s21"] }, { "specKey": "spec3", "specValue":
["s22"] }] }]
Spec Keys present changes on the basis of KeyLabel value. As you can see above if if KeyLabel = Label1, spec1 and spec2 are present. If KeyLabel = Label2, spec1 and spec3 are present
I want to create a CSV/Excel using this such that header/top row as following columns
KeyLabel, spec1, spec2, spec3 (basically union of all specKeys)
Label1, s11,s12
Label2, s21, ,s22
So, the challenging part is that at the time of writing to file, I need to write under in the appropriate column.
Any thoughts on if there are any csv/excel libraries which make this easier. Naive way does seem very elegant - which is to store the ordered list of headers and basic on key write commas and values so that values are in write column
I think you don't need specific library for doing that.
You should only need some common library jackson poi-ooxml
First, you should read json string to POJO object (ref: http://www.jsonschema2pojo.org/)
List examples = new ObjectMapper().readValue(json, new TypeReference>() {});
Seconds, collect header
List header = examples
.stream()
.map(Example::getSpecs)
.flatMap(Collection::stream)
.map(Spec::getSpecKey)
.distinct()
.collect(Collectors.toList());
header.add(0, "Key");
Next, use POI library write excel file
final XSSFWorkbook workbook = new XSSFWorkbook();
final XSSFSheet sheet = workbook.createSheet(WorkbookUtil.createSafeSheetName("sheetname"));
// Headers
Row row = sheet.createRow(0);
for (int index = 0; index < headers.size(); index++) {
row.createCell(index).setCellValue(headers.get(index));
}
// Content
int startRow = 1;
for (Example object : objects) {
row = sheet.createRow(startRow++);
// key label
row.createCell(0).setCellValue(object.getKey().getKeyLabel());
// spec
for (int index = 1; index < headers.size(); index++) {
String speckey = headers.get(index);
String value = object.getSpecs().stream().filter(e -> e.getSpecKey().equals(speckey)).findAny().map(Spec::getSpecValue).orElse(Collections.emptyList()).toString();
row.createCell(index).setCellValue(value);
}
}
FileOutputStream fileOut = new FileOutputStream("D:\\poi-generated-file.xlsx");
try {
workbook.write(fileOut);
} finally {
fileOut.close();
workbook.close();
}
It only example code. You should modify and update to better.
Related
Need to get "value" based on given "key" from Excel file
I have excel file
File name Test xlsx
and sheet name sheet1
And sheet contains following key and value pairs and. JIRA ticket is unique .
Test case description
testdata key
Testdatavalue
testdata2 key
Testdata2 Value
testdata3 key
Testdata3 value
Sampiletest description1
Testcase-jira-1
user1id
Harshadh
Password
123ggg
Sampiletest2 discription
Testcase-jira-2
user2
Ramu
Password123
333ggg
Sampiletest3 discription
Test case jira-3
user3
latha
Password556
73hhh
Up to N number of rows
Here, I needs to get the data in following way by using Java Selenium Cucumber. I am going to use above test data to pass in Cucumber step definition class file by BDD way.
How can we get the data in definition file for following way
1)If pass Key value from current row how can we get the value of value for provide test input for webSeleinum element
Example 4th row data
Sampiletest3 discription|Test case jira-3| user3|latha|Password556|73hhh
.....
If I call the "user3" that should return "Password556"
Same way any row I need to get the value.
Please guide me
You can try the below code.
Feature file:
In examples, you can give the row numbers and sheet name to use the data for itterations.
Scenario Outline: Login to the application with multiple users.
Given get data from datasheet with "<test_id>" and "<sheetName>"
And login to the application
Examples:
| test_id | sheetName |
| 1 | Login |
| 2 | Login |
Excel data:
Read the data from excel and store it in a hashmap:
Create a class to read the data (Example: ExcelReader)
Use org.apache.poi.ss.usermodel and org.apache.poi.xssf.usermodel imports
public class ExcelReader {
private File file;
private FileInputStream inputStream;
private String testID;
private String sheetName;
private int testIdColumn;
private int numberOfColumns;
private XSSFCell cell;
public HashMap<String, String> fieldsAndValues;
public ExcelReader(String testId, String sheetName) {
file = new File(System.getProperty("user.dir") + "Excel location path");
try {
inputStream = new FileInputStream(file);
} catch (FileNotFoundException e) {
System.out.println("File not found at given location: " + e);
}
this.testID = testId;
this.sheetName = sheetName;
this.readExcelAndCreateHashMapForData();
}
public HashMap<String, String> readExcelAndCreateHashMapForData() {
try {
fieldsAndValues = new HashMap<String, String>();
XSSFWorkbook workBook = new XSSFWorkbook(inputStream);
XSSFSheet sheet = workBook.getSheet(sheetName);
/* Get number of rows */
int lastRow = sheet.getLastRowNum();
int firstRow = sheet.getFirstRowNum();
int numberOfRows = lastRow - firstRow;
/*
* Get test_Id column number.
*/
outerloop: for (int row = 0; row < numberOfRows; row++) {
numberOfColumns = sheet.getRow(row).getLastCellNum();
for (int cellNumber = 0; cellNumber < numberOfColumns; cellNumber++) {
cell = sheet.getRow(row).getCell(cellNumber);
cell.setCellType(Cell.CELL_TYPE_STRING);
if (sheet.getRow(row).getCell(cellNumber).getStringCellValue().equalsIgnoreCase("test_ID")) {
testIdColumn = sheet.getRow(row).getCell(cellNumber).getColumnIndex();
break outerloop;
}
}
}
/*
* Search for the test id value.
*/
outerloop: for (int i = 0; i <= numberOfRows; i++) {
cell = sheet.getRow(i).getCell(testIdColumn);
cell.setCellType(Cell.CELL_TYPE_STRING);
if (testID.equals(sheet.getRow(i).getCell(testIdColumn).getStringCellValue())) {
for (int j = 0; j < numberOfColumns; j++) {
XSSFCell key = sheet.getRow(testIdColumn).getCell(j);
XSSFCell value = sheet.getRow(i).getCell(j);
key.setCellType(Cell.CELL_TYPE_STRING);
if (value == null) {
// Not capturing blank cells.
} else if (value.getCellType() == XSSFCell.CELL_TYPE_BLANK) {
// Not capturing blank cells.
} else {
value.setCellType(Cell.CELL_TYPE_STRING);
String fieldName = sheet.getRow(testIdColumn).getCell(j).getStringCellValue().trim();
String fieldValue = sheet.getRow(i).getCell(j).getStringCellValue().trim();
fieldsAndValues.put(fieldName, fieldValue);
}
}
System.out.println("Fields and values: " + Arrays.toString(fieldsAndValues.entrySet().toArray()));
break outerloop;
}
}
} catch (Exception e) {
System.out.println("Exception occurred at getting the sheet: " + e);
}
/* Return the hash map */
return fieldsAndValues;
}
}
StepDefinition:
ExcelReader excelReader;
#Given("get data from datasheet with \"(.*)\" and \"(.*)\"$")
public void get_data_from_datasheet(String testId, String sheetName) {
excelReader = new ExcelReader(testId, sheetName);
}
#And("login to the application")
public void loginApplication(){
driver.findElement(By.xpath("element")).sendKeys(excelReader.fieldsAndValues.get("UserName"));
driver.findElement(By.xpath("element")).sendKeys(excelReader.fieldsAndValues.get("PassWord"));
driver.findElement(By.xpath("element")).click();
}
I would recommend putting all the data for a scenario inside of Gherkin documents, but you might have a valid use cases for pulling data from excel. However, in my experience, these type of requirements are rare. The reason why it is not recommended is, your BDD feature files are your requirements and should contain the right level of information to document the expected behavior of the system. If your data comes from an excel, then it just makes the requirement reading bit more difficult and makes it difficult to maintain.
Saying that if there is a strong reason for you to have these data stored in excel, you could easily achieve this using NoCodeBDD. All you have to do is map the column names and upload the excel and the tool take care of the rest. Please check this .gif to see how it is done. https://nocodebdd.live/examples-using-excel
Disclaimer: I am the founder of NoCodeBDD.
If you are using Junit5 here is an example on how it is done https://newbedev.com/data-driven-testing-in-cucumber-using-excel-files-code-example
You can use external data-source to provide examples using qaf-cucumber. It will enable to provide data-file to be used to provide examples from external data-source, which includes csv, json, xml, excel file or database query.
We cannot directly integrete Excel file data to Gerkin file
.
Instead write separate method in step file to get data from excel and do your cases.
I use following code get the data - common code
public static JSONArray Read_Excel_Data(String filename, String sheetname) throws IOException {
FileInputStream fileIn = null;
Workbook workbookout = null;
JSONArray totalData = new JSONArray();
try{
log.info("Filename and Sheet name : "+filename+", "+ sheetname );
fileIn = new FileInputStream(new File(filename));
workbookout = new XSSFWorkbook(fileIn);
Sheet sh = workbookout.getSheet(sheetname);
int totRows = sh.getLastRowNum();
Row hearderRow = sh.getRow(0);
int totCols = hearderRow.getLastCellNum();
log.info("Total [ Rows and Colums ] : [ "+totRows+" and "+ totCols +" ] ");
for(int i=1; i <= totRows; i++ ){
log.info("Progressing row : "+i);
Row tempRw = sh.getRow(i);
JSONObject jo = new JSONObject();
for(int j=0; j<totCols; j++ ){
Cell tempCell = tempRw.getCell(j);
Cell HeaderCell = hearderRow.getCell(j);
try{
jo.put(HeaderCell.getStringCellValue(), tempCell.getStringCellValue());
log.info("Value in "+i+" / "+j+" :::::::::::: > "+tempCell.getStringCellValue() );
}catch (NullPointerException npe){
log.warn(":::::::::::: > Null Value in [ "+i+" / "+j+" ] ");
}
}
totalData.add(jo);
}
workbookout.close();
fileIn.close();
System.out.println("Total data :::::::: "+totalData.toJSONString());
}catch(Exception e){
e.printStackTrace();
log.error("Error Occured !!"+e.toString());
workbookout.close();
fileIn.close();
}
return totalData;
}
I am reading an excel data file from Java using apache POI API and populating an HashMap collection with excel Headers as Key and specified Row Data as Value of the Map. All Headers are always present but data corresponding to some headers may or may not be present.
Below is my code logic:
First I populate all the headers to an ArrayList.
Then I Iterate through the cells of the specified excel data row and I add the header value from the ArrayList, populated previously, and Data cell value from the row as key-value to an HashMap.
Below is my code:
ArrayList<String> headerList = new ArrayList<String>();
Map<String, String> dataMap = new LinkedHashMap<String, String>();
DateFormat df = new SimpleDateFormat("dd/MM/yyyy");
// Create object of XSSFWorkbook to get hold of excel file
FileInputStream fis = new FileInputStream(System.getProperty("user.dir") + "\\Resources\\TestData.xlsx");
XSSFWorkbook workbook = new XSSFWorkbook(fis);
try
{
int noOfSheets = workbook.getNumberOfSheets();
for(int i=0; i<noOfSheets; i++)
{
if(workbook.getSheetName(i).equalsIgnoreCase(workSheet))
{
//Get access to sheet
XSSFSheet sheet = workbook.getSheetAt(i);
//Get access to all rows of sheet
Iterator<Row> rows = sheet.iterator();
Row headerRow = rows.next();
Iterator<Cell> headerCells = headerRow.cellIterator();
while(headerCells.hasNext())
{
headerList.add(headerCells.next().getStringCellValue());
}
// Get access to specific row
while(rows.hasNext())
{
Row dataRow = rows.next();
if(dataRow.getCell(0).getStringCellValue().equalsIgnoreCase(testCase))
{
int j = 0;
//Get access to collection of cells of the identified rows
Iterator<Cell> dataCells = dataRow.cellIterator();
//loop through all the cells of the row and add cell data to arraylist.
while(dataCells.hasNext())
{
Cell dataCell = dataCells.next();
if(dataCell.getCellType()==CellType.STRING)
{
//arrList.add(dataCell.getStringCellValue());
dataMap.put(headerList.get(j), dataCell.getStringCellValue());
}
else if(dataCell.getCellType()==CellType.NUMERIC)
{
if(DateUtil.isCellDateFormatted(dataCell))
{
//arrList.add(df.format(dataCell.getDateCellValue()));
dataMap.put(headerList.get(j), df.format(dataCell.getDateCellValue()));
}
else
{
//arrList.add(NumberToTextConverter.toText(dataCell.getNumericCellValue()));
dataMap.put(headerList.get(j), NumberToTextConverter.toText(dataCell.getNumericCellValue()));
}
}
else if(dataCell.getCellType()==CellType.BOOLEAN)
{
//arrList.add(Boolean.toString(dataCell.getBooleanCellValue()));
dataMap.put(headerList.get(j), Boolean.toString(dataCell.getBooleanCellValue()));
}
else
dataMap.put(headerList.get(j), null);
j++;
}
}
}
}
}
If there is no data in any cell then I do not want the corresponding header to be added in the Map. But when I iterate through the Data cells(dataCells.hasNext()), then the iterator does not return me null for that blank cell, instead it totally skips the Blank cell. So all headers are added but those cells where is no data are not added hence there is mismatch of Header-Data key-values.
Example: If data cell of Column 5 is blank then, "Value" of the column 5 header is mapped with the value of Data Column 6 as "Key-Value" in the HashMap. What I want is column 5 header should be skipped being added to the Map when column 5 data is blank. How do I resolve this logical mismatch issue?
Excel file screenshot
Debug scenario screenshot
I have to check if certain columns header names are present in an excel sheet in a particular order. The names are : First Name, Last Name(optional), Email Id, Phone No, Address(optional).
I need to check if I the input excel sheet has the mandatory columns as well as any combination of the optional ones.
Ex: First Name, Email Id, Phone No. (All mandatory ones)
First Name, Last Name, Email Id, Phone
A total of 4 cases are possible.
I have stored the column headers in an arraylist. (If there is an efficient way for my problem statement please let me know!).
public void readAndUpload() throws FileNotFoundException, IOException
{
FileInputStream file = new FileInputStream(new File("path.."));
HSSFWorkbook workbook = new HSSFWorkbook(file);
HSSFSheet sheet = workbook.getSheetAt(0);
HSSFRow row = sheet.getRow(0);
Iterator<Cell> cellIterator = row.cellIterator();
ArrayList<String> headerCell = new ArrayList<String>(5);
while(cellIterator.hasNext())
{
Cell cell = cellIterator.next();
headerCell.add(cell.getStringCellValue());
}
ArrayList<String> validHeaders = new ArrayList<>(Arrays.asList("First Name", "Email Id","Phone Number"));
if(headerCell.contains(validHeaders))
{
//validation logic here
}
else
{
System.out.println("Mandatory fields not present");
}
file.close();
}
What I understood from your requirement is that you would like to validate the following things,
1. If the headers in the excel are permitted headers (mandatory and optional).
2. If the mandatory headers are present.
3. If the mandatory headers are in order.
Here is one solution.
public class HeaderProblem {
public static void main(String[] args) {
List<String> headersFromExcel = Arrays.asList("EMail", "Phone", "Address", "Last Name", "First Name");
System.out.println(validate(headersFromExcel));
}
private static boolean validate(List<String> headers) {
List<String> mandatoryHeaders = Arrays.asList("First Name", "EMail", "Phone");
List<String> optionalHeaders = Arrays.asList("Last Name", "Address");
List<String> allHeaders = new ArrayList<>(mandatoryHeaders);
allHeaders.addAll(optionalHeaders);
Map<String, Integer> headerIndexMap = IntStream.range(0, headers.size())
.boxed()
.collect(Collectors.toMap(i -> headers.get(i), i -> i));
if(!allHeaders.containsAll(headers)) {
System.out.println("Some headers exist which are not allowed");
return false;
}
if (!headers.containsAll(mandatoryHeaders)) {
System.out.println("Mandatory headers are not present");
return false;
}
System.out.println(mandatoryHeaders.stream()
.map(headerIndexMap::get)
.collect(Collectors.toList()));
// Check if the manadatory headers are in order
Integer result = mandatoryHeaders.stream()
.map(headerIndexMap::get)
.reduce(-1, (x, hi) -> x < hi ? hi : headers.size());
if (result == headers.size()) {
System.out.println("Mandatory headers are not in order");
return false;
}
return true;
}
}
Here, I have assumed that the headersFromExcel field is the list of headers extracted from your excel.
One more thing, in your code you have used,
if(headerCell.contains(validHeaders))
Here, you should use containsAll() instead of contains(), because contains will consider the passed argument as one object and will compare with every individual cell of the ArrayList and will return always false in your case.
I am trying to get the column values for a specific row in a excel using poi methods.
I am able to get the values but the problem is I want the values only from second column.
public static ArrayList<String> GetBusinessComponentList() throws IOException{
String Tcname = "TC02_AggregateAutoByPassRO_CT";
ArrayList<String> arrayListBusinessFlow ;
arrayListBusinessFlow = new ArrayList<String>();
FileInputStream fileInput = new FileInputStream(oFile);
wb = new HSSFWorkbook(fileInput);
sheet = wb.getSheet("Business Flow");
int rownr = findRow(sheet, Tcname);
row = sheet.getRow(rownr);
for (Cell cell : row) {
String arr = cell.getStringCellValue();
arrayListBusinessFlow.add(arr);
}
return arrayListBusinessFlow;
}
private static int findRow(HSSFSheet sheet, String cellContent){
for (Row row : sheet) {
for (Cell cell : row) {
if (cell.getCellType() == Cell.CELL_TYPE_STRING) {
if (cell.getRichStringCellValue().getString().trim().equals(cellContent)) {
return row.getRowNum();
}
}
}
}
return 0;
}
}
OUTPUT:
[TC02_AggregateAutoByPassRO_CT,
StrategicUINewBusiness.Login,
StrategicUINewBusiness.CustomerSearch,
StrategicUINewBusiness.NamedInsured,
StrategicUINewBusiness.InsuranceScoreByPass,
StrategicUINewBusiness.VehiclePage,
StrategicUINewBusiness.DriverPage,
StrategicUINewBusiness.ViolationPage,
StrategicUINewBusiness.UnderwritingPage,
StrategicUINewBusiness.CoveragePage,
StrategicUINewBusiness.Portfolio,
StrategicUINewBusiness.BillingPage,
StrategicUINewBusiness.FinalSalePage,
StrategicUINewBusiness.PolicyConfirmation, , , ]
But I do not want my test case name when I am getting.
Please help me what changes i needed to do. thanks!
Currently, the code you're using to iterate over cells only returns cells with content or styling, and skips totally empty ones. You need to change to one of the other ways of iterating over cells, so you can control it to read from the second column onwards.
If you look at the Apache POI Documentation on iterating over rows and cells, you'll see a lot more details on the two main ways to iterate.
For your case, you'll want something like:
// We want to read from the 2nd column onwards, zero based
int firstColumn = 1;
// Always fetch at least 4 columns
int MY_MINIMUM_COLUMN_COUNT = 5;
// Work out the last column to go to
int lastColumn = Math.max(r.getLastCellNum(), MY_MINIMUM_COLUMN_COUNT);
// To format cells into strings
DataFormatter df = new DataFormatter();
// Iterate over the cells
for (int cn = firstColumn; cn < lastColumn; cn++) {
Cell c = r.getCell(cn, Row.RETURN_BLANK_AS_NULL);
if (c == null) {
// The spreadsheet is empty in this cell
} else {
// Do something useful with the cell's contents
// eg get the cells value as a string
String cellAsString = df.formatCellValue(c);
}
}
Use Cell cell=row.getCell(1); and also you can use sheet.getLastRowNum() to get the number last row on the sheet.
for (int i=0;i<=row.getLastCellNum();i++) {
if (i!=1){
//your stuff
}
}
I am attempting to write an excel sheet using POI and hashMaps. My code successfully creates and populates an excel sheet but not all of the information is written to the sheet. In debug mode it seems to write 5 key, value pairs and then loops to the start at key [0]. Can someone tell me where the mistake is in my logic?
HSSFWorkbook workbook = new HSSFWorkbook();
HSSFSheet sheet = workbook.createSheet(make + " sheet");
int rowNum = 0;
ConcurrentHashMap <Integer, String[] > data = new
ConcurrentHashMap<Integer, String[]>();
data.put(rowNum++, new String[] {"VIN", "Make", "Model", "Year",
"Major Group", "Caption", "Cap Filter", "Illustration",
"Illus Filter", "PNC", "Part Name", "Part Number", "Quantity",
"Part Filter"});
for (Part p : plist){
String PNC = p.getPNC();
String quantity = p.getQuantity();
if(vFilterId.contains(p.getId())) {
data.put(rowNum++ , new String[] {vinId, make,
model, year, group, caption, capFilter, illusName,
iFilter, PNC, p.getName(), p.getPartNumber(),
quantity, "NONFILTERED"});
} else {
data.put(rowNum++ , new String[] {vinId, make,
model, year, group, caption, capFilter, illusName,
iFilter, PNC, p.getName(), p.getPartNumber(),
quantity, "NONFILTERED"});
}
}
int rowIndex = 0;
Set<Integer> keyset = data.keySet();
for (Integer key : keyset) {
Row row = sheet.createRow(rowIndex++);
Object[] objArr = data.get(key);
int cellIndex = 0;
for (Object obj : objArr) {
Cell cell = row.createCell(cellIndex++);
cell.setCellValue((String) obj);
}
}
try {
FileOutputStream fos = new FileOutputStream(outputFile);
workbook.write(fos);
fos.close();
System.out.println("XLS sheet written.");
} catch (IOException e) {
e.printStackTrace();
}
}
return ReturnConstants.SUCCESS;
}
Thanks in advance!
You're using a map to store data representing rows in a table. You'd be better off using a List since maps are generally not sorted (i.e. the order of the entries might change over time).
HSSFWorkbook workbook = new HSSFWorkbook();
HSSFSheet sheet = workbook.createSheet(make + " sheet");
List<String[]> data = new ArrayList<String[]>();
data.add(new String[] {"VIN", "Make", "Model", "Year",
"Major Group", "Caption", "Cap Filter", "Illustration",
"Illus Filter", "PNC", "Part Name", "Part Number", "Quantity",
"Part Filter"});
for (Part p : plist){
String PNC = p.getPNC();
String quantity = p.getQuantity();
if(vFilterId.contains(p.getId())) {
data.add(new String[] {vinId, make,
model, year, group, caption, capFilter, illusName,
iFilter, PNC, p.getName(), p.getPartNumber(),
quantity, "NONFILTERED"});
} else {
data.add(new String[] {vinId, make,
model, year, group, caption, capFilter, illusName,
iFilter, PNC, p.getName(), p.getPartNumber(),
quantity, "NONFILTERED"});
}
}
for (int rowIndex=0; rowIndex<data.size(); rowIndex++) {
Row row = sheet.createRow(rowIndex);
Object[] objArr = data.get(rowIndex);
int cellIndex = 0;
for (Object obj : objArr) {
Cell cell = row.createCell(cellIndex++);
cell.setCellValue((String) obj);
}
}
It's hard to determine without debugging your code, but from what you described it seems like your keyset variable contains only 3 entries, which is why you are outputting only 3 rows. So the first thing I would do would be to display (using System.out.println) data.size() and keyset.size() to see how big they are. I am guessing one or both are only of size 3.
Did you show all the relevant code? I think perhaps not, because, for example, your if statement seems redundant as you output the same thing regardless of whether you execute the if code or else clause.
I do agree with the first answer that there is no need to use a HashMap; an ArrayList will work just as well. But, as you discovered, that has nothing to do with why you are only seeing 3 rows.
Sorry I can't be more helpful, but I think some debugging on your part with System.out.println will go a long way in discovering what the problem is.