I have 3000 lines of code which will perform copy the data from Excel and insert into MySQL db, and I written the code without spring framework. And everything looks fine but now I want to them as a Spring Boot REST service. If I call the endpoint URL It should return that the record is inserted as response message if not according to the code It should say "No record inserted."
I will post the sample code base below to get an idea that what I have tried so far.
package com.online.amazon.asinhunt.feature;
import java.io.File;
import java.io.FileInputStream;
import java.nio.file.Paths;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.ss.usermodel.Row;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import com.online.amazon.asinhunt.dto.DBCloneDTO1;
public class GITJapi {
#DataProvider(name = "data")
public static Object[][] getVAlueFromExcel() throws Exception {
HSSFSheet sheet = TestJapi.getActiveSheet();
Object[][] dataObj = new Object[sheet.getLastRowNum()
- JDBCUtils.getRecordCounts(38)][sheet.getRow(0).getLastCellNum()];
boolean isCondition = false;
int count = 0;
for (int i = 1; i <= sheet.getLastRowNum(); i++) {
HSSFRow row = sheet.getRow(i);
for (int j = 0; j < row.getLastCellNum(); j++) {
row.getCell(j, Row.CREATE_NULL_AS_BLANK);
if (JDBCUtils.getRecordCounts(38) < i) {
System.out.println(row.getCell(j).getStringCellValue());
isCondition = true;
dataObj[count][j] = row.getCell(j).getStringCellValue();
}
}
if (isCondition == true) {
count++;
}
}
return dataObj;
}
public static HSSFSheet getActiveSheet() throws Exception {
File f = new File(
".//testOCT_US.xls");
FileInputStream fis = new FileInputStream(f);
HSSFWorkbook book = new HSSFWorkbook(fis);
HSSFSheet sheet = book.getSheetAt(0);
fis.close();
return sheet;
}
#Test(dataProvider = "data")
public static void insertRecords(String rowNu, String testcaseId,
String description, String priority, String buyer,
String transactionData, String dbValidation) throws Exception {
DBCloneDTO1 setValueToPojo = new DBCloneDTO1(rowNu, testcaseId,
description, priority, buyer, transactionData, dbValidation);
JDBCUtils.insertQuery(setValueToPojo);
System.out.println("Cloned successfully....");
System.out.println(description + "\t Description.....");
}
#AfterSuite
public static void cleanUp() throws Exception {
boolean condition = isDeleteDirectory(new File(".//clone1//"));
if (condition) {
Sysout("");
}
}
public static boolean isDeleteDirectory(File directory) {
if (directory.exists()) {
File[] files = directory.listFiles();
if (null != files) {
for (int i = 0; i < files.length; i++) {
if (files[i].isDirectory()) {
isDeleteDirectory(files[i]);
} else {
files[i].delete();
}
}
}
}
return (directory.delete());
}
}
The above Java file is executable one which has #Test annotation and there was a lot of calls in this file which is just POJO and replica of Excel and Backend. The whole code purpose to insert the record into back end which is available in Excel sheet.
Now I need to convert this whole project in a single REST Call. If I hit REST spring end point URL It should call above class and return an OK message or data inserted message as output.
Is really easy to do that, you need to take in mind .this things, first as a rest .api you will add a "controller" this is in order to add the necessaries endpoints, .the your actual code will be part of a "service" the services will have the business logic that you need in this case will be the excel logic. (Second thing is not a good practice return the object that you are using in the services layer so you will need a transform object to do that. https://spring.io/guides/gs/rest-service/
Related
I finally got IntelliJ to work. I'm using the code below. It works perfect. I need it to loop over and over and pull links from a spreadsheet to find the price over and over again on different items. I have a spreadsheet with a few sample URLs located in column C starting at row 2. How can I have JSOUP use the URLs in this spreadsheet then output to column D?
public class Scraper {
public static void main(String[] args) throws Exception {
final Document document = Jsoup.connect("examplesite.com").get();
for (Element row : document.select("#price")) {
final String price = row.select("#price").text();
System.out.println(price);
}
}
Thanks in advance for any help!
Eric
You can use JExcel library to read and edit sheets: https://sourceforge.net/projects/jexcelapi/ .
When you download the zip file with library there's also very useful tutorial.html.
Explanation in comments:
import java.io.File;
import java.io.IOException;
import jxl.Cell;
import jxl.CellType;
import jxl.Workbook;
import jxl.write.Label;
import jxl.write.WritableSheet;
import jxl.write.WritableWorkbook;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
public class StackoverflowQuestion51577491 {
private static final int URL_COLUMN = 2; // Column C
private static final int PRICE_COLUMN = 3; // Column D
public static void main(final String[] args) throws Exception {
// open worksheet with URLs
Workbook originalWorkbook = Workbook.getWorkbook(new File("O:/original.xls"));
// create editable copy
WritableWorkbook workbook = Workbook.createWorkbook(new File("O:/updated.xls"), originalWorkbook);
// close read-only workbook as it's not needed anymore
originalWorkbook.close();
// get first available sheet
WritableSheet sheet = workbook.getSheet(0);
// skip title row 0
int currentRow = 1;
Cell cell;
// iterate each cell from column C until we find an empty one
while (!(cell = sheet.getCell(URL_COLUMN, currentRow)).getType().equals(CellType.EMPTY)) {
// raed cell contents
String url = cell.getContents();
System.out.println("parsing URL: " + url);
// parse and get the price
String price = parseUrlWithJsoupAndGetProductPrice(url);
System.out.println("found price: " + price);
// create new cell with price
Label cellWithPrice = new Label(PRICE_COLUMN, currentRow, price);
sheet.addCell(cellWithPrice);
// go to next row
currentRow++;
}
// save and close file
workbook.write();
workbook.close();
}
private static String parseUrlWithJsoupAndGetProductPrice(String url) throws IOException {
// download page and parse it to Document
Document doc = Jsoup.connect(url).get();
// get the price from html
return doc.select("#priceblock_ourprice").text();
}
}
before:
after:
I would like to find a comment in Docx document (somehow, by author or ID...), then create new content. I was able to create a comment, with the help of this answer, but had no luck with manipulation.
As said in my answer linked in your question, until now the XWPFdocument will only read that package part while creating. There is neither write access nor a possibility to create that package part. This is mentioned in XWPFDocument.java - protected void onDocumentRead(): code line 210: "// TODO Create according XWPFComment class, extending POIXMLDocumentPart".
So we need doing this ourself until now. We need providing class extending POIXMLDocumentPart for comments and registering this relation instead of only relation to the simple POIXMLDocumentPart. So that and changings can be made which were committed while writing the XWPFDocument.
Example:
import java.io.*;
import org.apache.poi.*;
import org.apache.poi.openxml4j.opc.*;
import org.apache.xmlbeans.*;
import org.apache.poi.xwpf.usermodel.*;
import static org.apache.poi.POIXMLTypeLoader.DEFAULT_XML_OPTIONS;
import org.openxmlformats.schemas.wordprocessingml.x2006.main.*;
import javax.xml.namespace.QName;
import java.math.BigInteger;
import java.util.GregorianCalendar;
import java.util.Locale;
public class WordChangeComments {
public static void main(String[] args) throws Exception {
XWPFDocument document = new XWPFDocument(new FileInputStream("WordDocumentHavingComments.docx"));
for (POIXMLDocumentPart.RelationPart rpart : document.getRelationParts()) {
String relation = rpart.getRelationship().getRelationshipType();
if (relation.equals(XWPFRelation.COMMENT.getRelation())) {
POIXMLDocumentPart part = rpart.getDocumentPart(); //this is only POIXMLDocumentPart, not a high level class extending POIXMLDocumentPart
//provide class extending POIXMLDocumentPart for comments
MyXWPFCommentsDocument myXWPFCommentsDocument = new MyXWPFCommentsDocument(part.getPackagePart());
//and registering this relation instead of only relation to POIXMLDocumentPart
String rId = document.getRelationId(part);
document.addRelation(rId, XWPFRelation.COMMENT, myXWPFCommentsDocument);
//now the comments are available from the new MyXWPFCommentsDocument
for (CTComment ctComment : myXWPFCommentsDocument.getComments().getCommentArray()) {
System.out.print("Comment: Id: " + ctComment.getId());
System.out.print(", Author: " + ctComment.getAuthor());
System.out.print(", Date: " + ctComment.getDate());
System.out.print(", Text: ");
for (CTP ctp : ctComment.getPArray()) {
System.out.print(ctp.newCursor().getTextValue());
}
System.out.println();
//and changings can be made which were committed while writing the XWPFDocument
if (BigInteger.ONE.equals(ctComment.getId())) { //the second comment (Id 0 = first)
ctComment.setAuthor("New Author");
ctComment.setInitials("NA");
ctComment.setDate(new GregorianCalendar(Locale.US));
CTP newCTP = CTP.Factory.newInstance();
newCTP.addNewR().addNewT().setStringValue("The new Text for Comment with Id 1.");
ctComment.setPArray(new CTP[]{newCTP });
}
}
}
}
document.write(new FileOutputStream("WordDocumentHavingComments.docx"));
document.close();
}
//a wrapper class for the CommentsDocument /word/comments.xml in the *.docx ZIP archive
private static class MyXWPFCommentsDocument extends POIXMLDocumentPart {
private CTComments comments;
private MyXWPFCommentsDocument(PackagePart part) throws Exception {
super(part);
comments = CommentsDocument.Factory.parse(part.getInputStream(), DEFAULT_XML_OPTIONS).getComments();
}
private CTComments getComments() {
return comments;
}
#Override
protected void commit() throws IOException {
System.out.println("============MyXWPFCommentsDocument is committed=================");
XmlOptions xmlOptions = new XmlOptions(DEFAULT_XML_OPTIONS);
xmlOptions.setSaveSyntheticDocumentElement(new QName(CTComments.type.getName().getNamespaceURI(), "comments"));
PackagePart part = getPackagePart();
OutputStream out = part.getOutputStream();
comments.save(out, xmlOptions);
out.close();
}
}
}
This works for apache poi 3.17. Since apache poi 4.0.0 the ooxml part is separated. So there must be:
...
import org.apache.poi.ooxml.*;
...
import static org.apache.poi.ooxml.POIXMLTypeLoader.DEFAULT_XML_OPTIONS;
...
I have written a data driven framework Java program using below 2 classes. Inthis prgram am trying to log into system by taking login ids from external excel file.
DDF_ExcelClass - library java class to take data from external excel
file and will be called in SnapDealLogin_2
SnapDealLogin_2 - datadrivenframework java program which is failed. In
this class am taking login data from external excel file using
DDF_ExcelClass.
The error message shown are java.lang.RuntimeException: and
java.lang.ArrayIndexOutOfBoundsException: 5
Please note if login ids are given inside the program (not taking from external excel file) then SnapDealLogin_2 works good. failing when try to
take login ids from excel file.
I have tried to identify the cause of fail but could not. please help.
// Data Driver Framework - Excel Class stored as Library to reuse
package datadrivenframework;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import java.io.File;
import java.io.FileInputStream;
public class DDF_ExcelClass{
XSSFWorkbook Snap_WB;
XSSFSheet Snap_Sheet;
// NAVIGATING TO THE EXCEL FILE
public DDF_ExcelClass(String Path) throws Exception{
File Snap_Excel = new File(Path);
FileInputStream Snap_Input = new FileInputStream(Snap_Excel);
Snap_WB = new XSSFWorkbook(Snap_Input);
Snap_WB.close();
}
// NAVIGATING TO THE EXCEL SHEET AND GETTING THE CELL VALUE THEN RETURNING THE SAME
public String Snap_ok(int SheetIndex, int Row, int Column){
Snap_Sheet = Snap_WB.getSheetAt(SheetIndex);
String Snap_DataR = Snap_Sheet.getRow(Row).getCell(Column).getStringCellValue();
//System.out.println("THE VALUE IN THE ROW " + Row + " IS " + Snap_DataR);
return Snap_DataR;
}
// FINDING THE TOTAL ROW COUNT IN THE DATA SHEET AND RETURNING THE SAME
public int filerowcount(int SheetID){
int rowcount = Snap_WB.getSheetAt(SheetID).getLastRowNum();
//rowcount = rowcount+1;
return rowcount;
}
}
This is a simple data driver framework program where data is taken from external excel file
package datadrivenframework;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Test;
import org.testng.annotations.DataProvider;
import datadrivenframework.DDF_ExcelClass;
public class SnapDealLogin_2 {
#Test(dataProvider = "ABN") // declaring that # Test annotation should get value from dataProvider annotation named "ABN" to proceed further
public void Snap_Login(String UserID) throws InterruptedException{
System.setProperty("webdriver.chrome.driver",
"C:\\Users\\hp\\workspace\\Selenium\\browser\\chromedriver.exe"); // declaring the location of Chromedriver
WebDriver Snap = new ChromeDriver(); // initializing webdriver Snap as ChromeDriver
Snap.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); // adding implicit wait
Snap.get("https://www.snapdeal.com/");// entering snapdeal web url
Thread.sleep(3000);// adding sleep
Snap.manage().window().maximize(); // maximizing the browser
Thread.sleep(2000);// adding sleep
Snap.findElement(By.xpath(".//*[#id='sdHeader']/div[4]/div[2]/div/div[3]/div[3]/div/span[1]")).click();// clicking Sign-On button
Snap.findElement(By.xpath("//a [#href='https://www.snapdeal.com/login']")).click(); // Clicking Login button
Snap.switchTo().frame("loginIframe"); // Switching the webdriver control to new frame "Loginframe"
Snap.manage().timeouts().implicitlyWait(5,TimeUnit.SECONDS);// adding implicit wait
Snap.findElement(By.id("userName")).sendKeys(UserID); // entering User id
Snap.findElement(By.id("checkUser")).click(); //clicking "Continue"button
Snap.quit(); // closing the window
}
#DataProvider(name = "ABN") // adding annotation as DataProvider and naming it as "ABN"
public Object [][] Snap_DP() throws Exception{
DDF_ExcelClass DEC = new DDF_ExcelClass("C:\\Users\\hp\\Desktop\\User_Credentials.xlsx");
int rowcount = DEC.filerowcount(0);
Object[][] xlobject = new Object[rowcount][1];
for(int I = 0; I<=rowcount; I++){
xlobject[I][0] = DEC.Snap_ok(0, I, 0);
}
return xlobject;
}
}
I am working on a requirement where I need to parse CSV record fields against multiple validations. I am using supercsv which has support for field level processors to validate data.
My requirement is to validate each record/row field against multiple validations and save them to the database with success/failure status. for failure records I have to display all the failed validations using some codes.
Super CSV is working file but it is checking only first validation for a filed and if it is failed , ignoring second validation for the same field.Please look at below code and help me on this.
package com.demo.supercsv;
import java.io.FileReader;
import java.io.IOException;
import java.io.StringWriter;
import java.util.ArrayList;
import java.util.List;
import org.supercsv.cellprocessor.Optional;
import org.supercsv.cellprocessor.constraint.NotNull;
import org.supercsv.cellprocessor.constraint.StrMinMax;
import org.supercsv.cellprocessor.constraint.StrRegEx;
import org.supercsv.cellprocessor.constraint.UniqueHashCode;
import org.supercsv.cellprocessor.ift.CellProcessor;
import org.supercsv.exception.SuperCsvCellProcessorException;
import org.supercsv.io.CsvBeanReader;
import org.supercsv.io.CsvBeanWriter;
import org.supercsv.io.ICsvBeanReader;
import org.supercsv.io.ICsvBeanWriter;
import org.supercsv.prefs.CsvPreference;
public class ParserDemo {
public static void main(String[] args) throws IOException {
List<Employee> emps = readCSVToBean();
System.out.println(emps);
System.out.println("******");
writeCSVData(emps);
}
private static void writeCSVData(List<Employee> emps) throws IOException {
ICsvBeanWriter beanWriter = null;
StringWriter writer = new StringWriter();
try{
beanWriter = new CsvBeanWriter(writer, CsvPreference.STANDARD_PREFERENCE);
final String[] header = new String[]{"id","name","role","salary"};
final CellProcessor[] processors = getProcessors();
// write the header
beanWriter.writeHeader(header);
//write the beans data
for(Employee emp : emps){
beanWriter.write(emp, header, processors);
}
}finally{
if( beanWriter != null ) {
beanWriter.close();
}
}
System.out.println("CSV Data\n"+writer.toString());
}
private static List<Employee> readCSVToBean() throws IOException {
ICsvBeanReader beanReader = null;
List<Employee> emps = new ArrayList<Employee>();
try {
beanReader = new CsvBeanReader(new FileReader("src/employees.csv"),
CsvPreference.STANDARD_PREFERENCE);
// the name mapping provide the basis for bean setters
final String[] nameMapping = new String[]{"id","name","role","salary"};
//just read the header, so that it don't get mapped to Employee object
final String[] header = beanReader.getHeader(true);
final CellProcessor[] processors = getProcessors();
Employee emp;
while ((emp = beanReader.read(Employee.class, nameMapping,
processors)) != null) {
emps.add(emp);
if (!CaptureExceptions.SUPPRESSED_EXCEPTIONS.isEmpty()) {
System.out.println("Suppressed exceptions for row "
+ beanReader.getRowNumber() + ":");
for (SuperCsvCellProcessorException e :
CaptureExceptions.SUPPRESSED_EXCEPTIONS) {
System.out.println(e);
}
// for processing next row clearing validation list
CaptureExceptions.SUPPRESSED_EXCEPTIONS.clear();
}
}
} finally {
if (beanReader != null) {
beanReader.close();
}
}
return emps;
}
private static CellProcessor[] getProcessors() {
final CellProcessor[] processors = new CellProcessor[] {
new CaptureExceptions(new NotNull(new StrRegEx("\\d+",new StrMinMax(0, 2)))),//id must be in digits and should not be more than two charecters
new CaptureExceptions(new Optional()),
new CaptureExceptions(new Optional()),
new CaptureExceptions(new NotNull()),
// Salary
};
return processors;
}
}
Exception Handler:
package com.demo.supercsv;
import java.util.ArrayList;
import java.util.List;
import org.supercsv.cellprocessor.CellProcessorAdaptor;
import org.supercsv.cellprocessor.ift.CellProcessor;
import org.supercsv.exception.SuperCsvCellProcessorException;
import org.supercsv.util.CsvContext;
public class CaptureExceptions extends CellProcessorAdaptor {
public static List<SuperCsvCellProcessorException> SUPPRESSED_EXCEPTIONS =
new ArrayList<SuperCsvCellProcessorException>();
public CaptureExceptions(CellProcessor next) {
super(next);
}
public Object execute(Object value, CsvContext context) {
try {
return next.execute(value, context);
} catch (SuperCsvCellProcessorException e) {
// save the exception
SUPPRESSED_EXCEPTIONS.add(e);
if(value!=null)
return value.toString();
else
return "";
}
}
}
sample csv file
ID,Name,Role,Salary
a123,kiran,CEO,"5000USD"
2,Kumar,Manager,2000USD
3,David,developer,1000USD
when I run my program supercsv exception handler displaying this message for the ID value in the first row
Suppressed exceptions for row 2:
org.supercsv.exception.SuperCsvConstraintViolationException: 'a123' does not match the regular expression '\d+'
processor=org.supercsv.cellprocessor.constraint.StrRegEx
context={lineNo=2, rowNo=2, columnNo=1, rowSource=[a123, kiran, CEO, 5000USD]}
[com.demo.supercsv.Employee#23bf011e, com.demo.supercsv.Employee#50e26ae7, com.demo.supercsv.Employee#40d88d2d]
for field Id length should not be null and more than two and it should be neumeric...I have defined field processor like this.
new CaptureExceptions(new NotNull(new StrRegEx("\\d+",new StrMinMax(0, 2))))
but super csv ignoring second validation (maxlenght 2) if given input is not neumeric...if my input is 100 then its validating max lenght..but how to get two validations for wrong input.plese help me on this
SuperCSV cell processors will work in sequence. So, if it passes the previous constraint validation then it will check next one.
To achieve your goal, you need to write a custom CellProcessor, which will check whether the input is a number (digit) and length is between 0 to 2.
So, that both of those checks are done in a single step.
I'm fairly new to Selenium webdriver and Java, previously I used Selenium IDE. I'm trying to read testdata from an Excel sheet. Which is then written to Eclipse console, which works, and should be used to execute the actual test, which doesn't work. The actual test is not executed because I get an error argument type mismatch. The code looks like this:
package Test_Excel;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Arrays;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
import jxl.Sheet;
import jxl.Workbook;
import jxl.read.biff.BiffException;
#RunWith(Parameterized.class)
public class TestWithExcelData {
// Our two parameters
private final int input;
private final int resultExpected;
// Constructor
public TestWithExcelData(int input, int result) {
this.input = input;
this.resultExpected = result;
}
#Parameters
public static Iterable<Object []> data() throws BiffException, IOException
{
String FilePath = "C://Selenium//workspace//testproject//src//testdata//TestData.xls";
FileInputStream fs = new FileInputStream(FilePath);
Object[][] object = new Object[6][2];
Workbook wb = Workbook.getWorkbook(fs);
//locate the excel file in the local machine
Sheet sheet = wb.getSheet("IOResult");
int i=1; //avoid header row
while(!(sheet.getCell(0, i).getContents().equals("end"))) //read data till it reaches the cell whose text is ‘end’
{
object[i-1][0]=sheet.getCell(0, i).getContents();
object[i-1][1]=sheet.getCell(1, i).getContents();
System.out.print(sheet.getCell(0, i).getContents() + "\t");
System.out.print(sheet.getCell(1, i).getContents() + "\t");
System.out.println();
i++;
}
return Arrays.asList(object);
}
#Test
public void testSquareOff(){
Assert.assertEquals(resultExpected, MathUtils.square(input));
}
}
Is there somebody who can point me in the right direction?
Thanks in advance
The method sheet.getCell(column, row).getContents() is returning a String, but your constructor expects int.
You may modify the two lines in the data() method:
object[i-1][0] = Integer.valueOf(sheet.getCell(0, i).getContents());
object[i-1][1] = Integer.valueOf(sheet.getCell(1, i).getContents());