I'm using openCsv library cause it's very easy to use and I got all necessary functional. But now I need to use some custom logic except checking for correct types, columns and other usual things. I have a csv with columns firstName, secondName, firstSalary, lastSalary and others. I want to check that firstDayOfWork is less than lastDayOfWork in the process of parcing and add a new csvException if it is false. So, if now I'm parcing file
firstName,secondName,firstSalary,lastSalary
John, Doe, testtext, 5000
Alice, , 100, 5000
Harry, Smith, 400, 200
and processing a list of csvExcpetions I can get results of parcing like
Number of mistakes: 2
Line 1: Conversion of testtext to java.lang.Integer failed.
Line 2: Field 'secondName' is mandatory but no value was provided.
I want to something like
Number of mistakes: 3
Line 1: Conversion of testtext to java.lang.Integer failed.
Line 2: Field 'secondName' is mandatory but no value was provided.
Line 3: firstSalary cannot be more than lastSalary
Or some custom parcing logic like check if some field catch regexp, two fields at the same time greater than 0, etc.
I can firstly parce and convert it to beans and by the second cycle check my beans to these rules, but there are can be a lot of lines and it will take longer, so, I want to check it in the one process.
Can I get it by openCsv? And if yes, how? If no, what another tool I can use?
Thank you.
based on How to create a list of valid CSV records by logging the error of invalid record using OpenCsv?, instead of throwing an IllegalArgumentException in the setters, you could:
create a BeanVerifier to verify the whole bean after creation;
throw a CsvConstraintViolationException on error;
add .withThrowExceptions(false) to the CsvToBeanBuilder;
Invoke getCapturedExceptions() to collect the validation errors.
The advantage of this solution is that you can collect the list of CSV exceptions as you wanted:
Example
final CsvToBean<EmployeeBean> csvToBean =
new CsvToBeanBuilder<EmployeeBean>(new FileReader("c:\\test.csv"))
.withType(EmployeeBean.class)
.withVerifier(new EmployeeSalaryVerifier())
.withThrowExceptions(false)
.build();
final List<EmployeeBeans> employees = csvToBean.parse();
List<CsvException> exceptions = parser.getCapturedExceptions();
// logging number of mistakes and, for each exception, its line number and message
logger.error("Number of Mistakes: {}", exceptions.size());
employees.getCapturedExceptions().stream().forEach(ex ->
logger.error("Line {}: {}", ex.getLineNumber(), ex.getMessage(), ex));
EmployeeBean
public class EmployeeBean {
#CsvBindByName(column = "First Name", required = true)
private String firstName;
#CsvBindByName(column = "Last Name", required = true)
private String lastName;
#CsvBindByName(column = "First Salary" required = true)
private Long firstSalary;
#CsvBindByName(column = "Last Salary")
private Long lastSalary;
// regular getters and setters, no validation here
}
BeanVerifier
public class EmployeeSalaryVerifier implements BeanVerifier<EmployeeBean> {
#Override
public boolean verifyBean(Employee bean) throws CsvConstraintViolationException {
// check salary
if (bean.getLastSalary() != null && bean.getFirstSalary().compareTo(bean.getLastSalary()) > 0) {
throw new CsvConstraintViolationException("First Salary cannot be greater than Last Salary.");
}
return true;
}
While there is a request on the todo/wish list for OpenCSV I don't think it will help you because it's about pre parsing validation
For what you want you should just add a check in your setters to throw an exception when the values are bad. Here is my Bean and test.
public class SetterBean {
#CsvBindByName(column = "First Name", required = true)
private String firstName;
#CsvBindByName(column = "Last Name", required = true)
private String lastName;
#CsvBindByName(column = "First Salary")
private Long firstSalary;
#CsvBindByName(column = "Last Salary")
private Long lastSalary;
public void setFirstName(String firstName) {
this.firstName = firstName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public void setFirstSalary(Long firstSalary) {
if (lastSalary != null && (firstSalary.compareTo(lastSalary) > 0)) {
throw new IllegalArgumentException("First Salary cannot be greater than Last Salary.");
}
this.firstSalary = firstSalary;
}
public void setLastSalary(Long lastSalary) {
if (firstSalary != null && (lastSalary.compareTo(firstSalary) < 0)) {
throw new IllegalArgumentException("Last Salary cannot be less than First Salary.");
}
this.lastSalary = lastSalary;
}
}
public class SetterValidationTest {
private static final String HEADER = "First Name,Last Name,First Salary,Last Salary\n";
CsvToBean<SetterBean> csvToBean;
#Test
public void normalData() {
String data = HEADER + "First, Last, 1, 2";
csvToBean = new CsvToBeanBuilder<SetterBean>(new StringReader(data))
.withType(SetterBean.class)
.build();
List<SetterBean> beans = csvToBean.parse();
assertEquals(1, beans.size());
}
#Test(expected = Exception.class)
public void firstGTLast() {
String data = HEADER + "First, Last, 2, 1";
csvToBean = new CsvToBeanBuilder<SetterBean>(new StringReader(data))
.withType(SetterBean.class)
.build();
List<SetterBean> beans = csvToBean.parse();
}
}
And here is the result.
Exception in thread "pool-1-thread-1" java.lang.RuntimeException: com.opencsv.exceptions.CsvDataTypeMismatchException
at com.opencsv.bean.concurrent.ProcessCsvLine.run(ProcessCsvLine.java:91)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: com.opencsv.exceptions.CsvDataTypeMismatchException
at com.opencsv.bean.AbstractBeanField.assignValueToField(AbstractBeanField.java:192)
at com.opencsv.bean.AbstractBeanField.setFieldValue(AbstractBeanField.java:159)
at com.opencsv.bean.concurrent.ProcessCsvLine.processField(ProcessCsvLine.java:140)
at com.opencsv.bean.concurrent.ProcessCsvLine.processLine(ProcessCsvLine.java:126)
at com.opencsv.bean.concurrent.ProcessCsvLine.run(ProcessCsvLine.java:82)
... 3 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.opencsv.bean.AbstractBeanField.assignValueToField(AbstractBeanField.java:187)
... 7 more
Caused by: java.lang.IllegalArgumentException: Last Salary cannot be less than First Salary.
at integrationTest.BeanTests.SetterBean.setLastSalary(SetterBean.java:33)
... 12 more
Hope that helps.
Scott Conway :)
Related
I tried to rebuild the sample from the XDocReport documentation, but when i try to create tables or fill them. It always fails due to the follwing error. I cannot figure out what the issue is...
I have updated all the dependencies and tried different syntax in the .odt file.
ERROR:
Apr. 12, 2022 10:21:05 VORM. freemarker.log._JULLoggerFactory$JULLogger error
SEVERE: Error executing FreeMarker template
FreeMarker template error:
The following has evaluated to null or missing:
==> devs.name [in template "fr.opensagres.xdocreport.document.odt.ODTReport#57855c9a!content.xml" at line 5, column 11]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use [#if myOptionalVar??]when-present[#else]when-missing[/#if]. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: ${devs.name} auto-escaped [in template "fr.opensagres.xdocreport.document.odt.ODTReport#57855c9a!content.xml" at line 5, column 9]
----
Java stack trace (for programmers):
----
freemarker.core.InvalidReferenceException: [... Exception message was already printed; see it above ...]
at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:134)
at freemarker.core.EvalUtil.coerceModelToTextualCommon(EvalUtil.java:481)
at freemarker.core.EvalUtil.coerceModelToStringOrMarkup(EvalUtil.java:401)
at freemarker.core.EvalUtil.coerceModelToStringOrMarkup(EvalUtil.java:370)
at freemarker.core.BuiltInForLegacyEscaping._eval(BuiltInForLegacyEscaping.java:34)
at freemarker.core.Expression.eval(Expression.java:101)
at freemarker.core.Expression.evalAndCoerceToStringOrUnsupportedMarkup(Expression.java:139)
at freemarker.core.BuiltInForString.getTargetString(BuiltInForString.java:34)
at freemarker.core.BuiltInForString._eval(BuiltInForString.java:29)
at freemarker.core.Expression.eval(Expression.java:101)
at freemarker.core.MethodCall._eval(MethodCall.java:55)
at freemarker.core.Expression.eval(Expression.java:101)
at freemarker.core.Expression.evalAndCoerceToStringOrUnsupportedMarkup(Expression.java:139)
at freemarker.core.BuiltInForString.getTargetString(BuiltInForString.java:34)
at freemarker.core.BuiltInForString._eval(BuiltInForString.java:29)
at freemarker.core.Expression.eval(Expression.java:101)
at freemarker.core.MethodCall._eval(MethodCall.java:55)
at freemarker.core.Expression.eval(Expression.java:101)
at freemarker.core.Expression.evalAndCoerceToStringOrUnsupportedMarkup(Expression.java:139)
at freemarker.core.BuiltInForString.getTargetString(BuiltInForString.java:34)
at freemarker.core.BuiltInForString._eval(BuiltInForString.java:29)
at freemarker.core.Expression.eval(Expression.java:101)
at freemarker.core.MethodCall._eval(MethodCall.java:55)
at freemarker.core.Expression.eval(Expression.java:101)
at freemarker.core.DollarVariable.calculateInterpolatedStringOrMarkup(DollarVariable.java:100)
at freemarker.core.DollarVariable.accept(DollarVariable.java:63)
at freemarker.core.Environment.visit(Environment.java:383)
at freemarker.core.IteratorBlock$IterationContext.executedNestedContentForCollOrSeqListing(IteratorBlock.java:321)
at freemarker.core.IteratorBlock$IterationContext.executeNestedContent(IteratorBlock.java:271)
at freemarker.core.IteratorBlock$IterationContext.accept(IteratorBlock.java:244)
at freemarker.core.Environment.visitIteratorBlock(Environment.java:657)
at freemarker.core.IteratorBlock.acceptWithResult(IteratorBlock.java:108)
at freemarker.core.IteratorBlock.accept(IteratorBlock.java:94)
at freemarker.core.Environment.visit(Environment.java:347)
at freemarker.core.Environment.visit(Environment.java:353)
at freemarker.core.Environment.visit(Environment.java:353)
at freemarker.core.Environment.process(Environment.java:326)
at fr.opensagres.xdocreport.template.freemarker.FreemarkerTemplateEngine.process(FreemarkerTemplateEngine.java:163)
at fr.opensagres.xdocreport.template.freemarker.FreemarkerTemplateEngine.processNoCache(FreemarkerTemplateEngine.java:122)
at fr.opensagres.xdocreport.template.AbstractTemplateEngine.process(AbstractTemplateEngine.java:118)
at fr.opensagres.xdocreport.template.AbstractTemplateEngine.process(AbstractTemplateEngine.java:83)
at fr.opensagres.xdocreport.document.AbstractXDocReport.processTemplateEngine(AbstractXDocReport.java:772)
at fr.opensagres.xdocreport.document.AbstractXDocReport.process(AbstractXDocReport.java:518)
at fr.opensagres.xdocreport.document.AbstractXDocReport.process(AbstractXDocReport.java:484)
at report_creation.fillList.main(fillList.java:51)
I am currently using this code:
fillList.java
public class fillList {
public static void main(String[] args) {
try {
// 1) Load ODT file by filling Velocity template engine and cache
// it to the registry
InputStream in = fillList.class.getResourceAsStream("Test.odt");
IXDocReport report = XDocReportRegistry.getRegistry().loadReport(in, TemplateEngineKind.Freemarker);
// 2) Create fields metadata to manage lazy loop (#foreach velocity)
// for table row.
FieldsMetadata metadata = new FieldsMetadata();
metadata.addFieldAsList("developers.name");
metadata.addFieldAsList("developers.lastName");
metadata.addFieldAsList("developers.mail");
report.setFieldsMetadata(metadata);
// 3) Create context Java model
IContext context = report.createContext();
Project project = new Project("Create Dynamic Reports", 101);
context.put("project", project);
// Register developers list
List<Developer> developers = new ArrayList<>();
developers.add(new Developer("ZERR", "Angelo", "angelo.zerr#gmail.com"));
developers.add(new Developer("Leclercq", "Pascal", "pascal.leclercq#gmail.com"));
context.put("developers", developers);
// 4) Generate report by merging Java model with the ODT
OutputStream out = new FileOutputStream(new File("Test2_out.odt"));
report.process(context, out);
System.out.println("Report creation successfully completed!");
} catch (IOException e) {
System.err.println("[Debug Assistent]: No report created! >> IOException");
e.printStackTrace();
} catch (XDocReportException e) {
System.err.println("[Debug Assistent]: No report created! >> XDocReportException");
e.printStackTrace();
}
}
}
Project.java
public class Project {
private final String name;
private final Integer ID;
public Project(String name, Integer id) {
this.name = name;
this.ID = id;
}
public String getName() {
return name;
}
public Integer getID() {
return ID;
}
}
Developer.java
public class Developer {
private final String name;
private final String lastName;
private final String mail;
public Developer(String name, String lastName, String mail) {
this.name = name;
this.lastName = lastName;
this.mail = mail;
}
public String getFirstName() {
return name;
}
public String getLastName() {
return lastName;
}
public String getMail() {
return mail;
}
}
This is the .odt file i am using.
In the Developer class you have getFirstName instead of getName. Hence dev.name won't work.
I'm getting problem with accessing variables with getter & setter on multiple classes. I looked up this one but I'm too confused.
I have 3 type users: Admin (Position No: 0 in mysql table), Manager (Position No: 1), Clerk (Position No:2).
I have SeeReportsAndFeedbacks class. I want to show all reports by selecting rows with position_no = 0 and 1 to admin and manager, 2 to clerk. It's already done with if statement.
So clerk can see only see reports that with position_no=2
manager can see only see reports that with position_no=0 and 1
admin can see only see reports that with position_no=0 and 1
Please help me. I'm stucked here for a long time. What are wrong with my getter setters?
If i set on Login_Form, and call get it shows correct in girisyap() function but if i call get in other class named SeeReportsAndFeedbacks it shows first initial value from Users () constructor instead of set value on girisyap() function on Login_Form.
tip value takes position_no from mysql db as string, new1 value is parsing (converting) string to int for if statement
screenshot
GIST
Users Class
public class Users {
private int id;
private String username;
private String fullname;
private String password;
private String phone;
private String gender;
private byte[] image;
private int position_no;
public Users () {
setPno(1); //firsst initialize
//getFullname();
}
public Users (int uid ,String uname, String fname, String upassword, String uphone, String ugender, byte[] uimage, int pno){
this.id = uid;
this.username = uname;
this.fullname = fname;
this.password = upassword;
this.phone = uphone;
this.gender =ugender;
this.image =uimage;
this.position_no = pno;
}
public Users (int pno){
setPno(pno);
}
public int getPno(){
return position_no;
}
public void setPno(int pno){
this.position_no = pno;
}}
SeeReportsAndFeedbacks class (i removed not-related funcs or some other gui things for the question.
public class SeeReportsAndFeedbacks { // extends javax.swing.JFrame
//CLIENT client = new CLIENT();
int new1 = 9999; //testing something
int PositionNoGetiren;
//sers loginf = new Users(0, null,null,null,null,null,null,new1);
public SeeReportsAndFeedbacks() {
//initComponents();
Users loginf = new Users();
PositionNoGetiren = loginf.getPno(); //gets initial value instead of set value on login_form
System.out.println("Babakingg " + PositionNoGetiren);
//int ananas = loginf.getPno();
//fillFeedbackJTable(jTable2);
}
public void fillReportJTable(){//JTable table
//loginf.setPno(2); it works if i manually set but it's useless
//System.out.println("Loginfvalue in see reports: " + loginf.getPno() + loginf.getUsername());
//new1 = loginf.getPno(); //not works shows 0
//see.getNo();
new1=PositionNoGetiren;
String selectQuery = "SELECT * FROM `users` WHERE `position_no` = ?";
if(new1==0){//admin
selectQuery = "SELECT * FROM `reports`";
}
if(new1==1){//manager
selectQuery = "SELECT * FROM `reports` WHERE `position_no` = 1";
}
if(new1==2){//clerk
selectQuery = "SELECT * FROM `reports` WHERE `position_no` = 2";
}
//}
}}
Login_Form
public class Login_Form {
int positionNoGetiren;
/**
* Creates new form Login_Form
*/
public Login_Form() {
//initComponents();
//positionNoGetiren = 9999;
}
private void girisyap() {
//I DELETED ALL DATABASE RELATED THINGS FOR QUESTION.
//String tip = rs.getString("position_no"); //DETECTS CORRECTLY POSITION NO FROM DATABASE
String tip = "IT'S rs.getString(\"position_no\")"; //for posting question
System.out.println(tip);
int new1 = Integer.parseInt(tip);
//Users loginf = new Users(new1); //welcome yazisi icin
Users loginf = new Users(); //ONLY WORKS IN THIS CLASS.
loginf.setPno(new1); //set user type for reports class BUT IT'S NOT WORKING
System.out.println("Loginf degeri login_formdaki: " + loginf.getPno());
//THIS IF IS WORKING CORRECTLY.
if(new1==0){
//Admin form = new Admin();
//form.setVisible(true);
//form.pack();
//form.setLocationRelativeTo(null);
// form.setExtendedState(JFrame.MAXIMIZED_BOTH);
}
if(new1==1){
//Manager form = new Manager();
//form.setVisible(true);
//form.pack();
//form.setLocationRelativeTo(null);
}
if(new1==2){
//Clerk form = new Clerk();
//form.setVisible(true);
//form.pack();
//form.setLocationRelativeTo(null);
// form.setExtendedState(JFrame.MAXIMIZED_BOTH);
}
//this.dispose();
}
private void jButton_LoginActionPerformed(java.awt.event.ActionEvent evt) {
girisyap();}
}
There is nothing wrong with your getters and setters.
This is most likely an issue with parsing the value of TIP, that could not be parsed as an INT, maybe its a float with a weird value of like 2.00000004, or simply null. Try writing a test or log the value which your query return and check if this is the value you are looking for.
I'm attempting to convert the JSON output from my session and map it to a class that I've created using JAVA's ObjectMapper. When I run my tests on Lambda I get a Deserialisation error:
Deserialization error: com.amazon.ask.exception.AskSdkException
com.amazon.ask.exception.AskSdkException: Deserialization error
at com.amazon.ask.util.impl.JacksonJsonUnmarshaller.unmarshall(JacksonJsonUnmarshaller.java:50)
at com.amazon.ask.impl.AbstractSkill.execute(AbstractSkill.java:44)
at com.amazon.ask.AlexaSkill.execute(AlexaSkill.java:22)
at com.amazon.ask.SkillStreamHandler.handleRequest(SkillStreamHandler.java:71)
Caused by: com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve type id 'AnswerIntent' as a subtype of [simple type, class com.amazon.ask.model.Request]: known type ids = [Alexa.Presentation.APL.UserEvent, AlexaHouseholdListEvent.ItemsCreated, AlexaHouseholdListEvent.ItemsDeleted, AlexaHouseholdListEvent.ItemsUpdated, AlexaHouseholdListEvent.ListCreated, AlexaHouseholdListEvent.ListDeleted, AlexaHouseholdListEvent.ListUpdated, AlexaSkillEvent.SkillAccountLinked, AlexaSkillEvent.SkillDisabled, AlexaSkillEvent.SkillEnabled, AlexaSkillEvent.SkillPermissionAccepted, AlexaSkillEvent.SkillPermissionChanged, AudioPlayer.PlaybackFailed, AudioPlayer.PlaybackFinished, AudioPlayer.PlaybackNearlyFinished, AudioPlayer.PlaybackStarted, AudioPlayer.PlaybackStopped, Connections.Request, Connections.Response, Display.ElementSelected, GameEngine.InputHandlerEvent, IntentRequest, LaunchRequest, Messaging.MessageReceived, PlaybackController.NextCommandIssued, PlaybackController.PauseCommandIssued, PlaybackController.PlayCommandIssued, PlaybackController.PreviousCommandIssued, SessionEndedRequest, System.ExceptionEncountered] (for POJO property 'request')
at [Source: UNKNOWN; line: -1, column: -1] (through reference chain: com.amazon.ask.model.RequestEnvelope$Builder["request"])
at com.fasterxml.jackson.databind.exc.InvalidTypeIdException.from(InvalidTypeIdException.java:43)
at com.fasterxml.jackson.databind.DeserializationContext.invalidTypeIdException(DeserializationContext.java:1628)
at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownTypeId(DeserializationContext.java:1186)
at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._handleUnknownTypeId(TypeDeserializerBase.java:291)
at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:162)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:97)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:254)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:151)
at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:269)
at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:193)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3972)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2264)
at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:2746)
at com.amazon.ask.util.impl.JacksonJsonUnmarshaller.unmarshall(JacksonJsonUnmarshaller.java:48)
... 3 more
I did checks to ensure that my "riddleItem" variable is not null. The JSON values are being mapped to the Person class which just returns properties of a person. The code is shown below and I've highlighted the line which the error occurs on:
public Optional<Response> handle(HandlerInput input) {
Map<String, Object> sessionAttributes = input.getAttributesManager().getSessionAttributes();
System.out.println("This a FIRST debug");
LOG.debug("This a FIRST debug");
boolean correctAnswer;
String speechText = null, response;
System.out.println("This a SECOND debug");
Map<String, String> riddleItem = (LinkedHashMap<String, String>)sessionAttributes.get(Attributes.RIDDLE_ITEM_KEY);
Person person;
// System.out.println("riddleItem " + riddleItem);
if(riddleItem != null)
{
person = MAPPER.convertValue(riddleItem, Person.class); // ERROR OCCURS ON THIS LINE
}
System.out.println("This a THIRD debug");
PersonProperty personProperty = PersonProperty.valueOf((String) sessionAttributes.get(Attributes.RIDDLE_PROPERTY_KEY));
int counter = (int) sessionAttributes.get(Attributes.COUNTER_KEY);
int riddleGameScore = (int) sessionAttributes.get(Attributes.RIDDLE_SCORE_KEY);
System.out.println("This a FOURTH debug");
IntentRequest intentRequest = (IntentRequest) input.getRequestEnvelope().getRequest();
correctAnswer = compareSlots(intentRequest.getIntent().getSlots(), getPropertyOfPerson(personProperty, person));
System.out.println("This a FIFTH debug " + correctAnswer);
if(correctAnswer)
{
riddleGameScore++;
response = getSpeechExpressionCon(true);
System.out.println("This a SIXTH debug " + response);
sessionAttributes.put(Attributes.RIDDLE_SCORE_KEY, riddleGameScore);
}
else
{
response = getSpeechExpressionCon(false);
System.out.println("This a SEVENTH debug " + response);
}
AnswerIntentHandler setup = new AnswerIntentHandler();
//
if(riddle.getAnswer() != null)
{
speechText = "Hello " + riddle.getAnswer();
}
return input.getResponseBuilder()
.withSimpleCard("RiddleSession", speechText)
.withSpeech(speechText)
.withShouldEndSession(true)
.build();
}
[Json Output of properties under "riddleItem" during Session]1
I know my the values being mapped aren't empty thus I'm at a complete loss of ideas as to what's going on as I've come up short with possible ideas as to what the problem might be.
I solved the problem as I came to realise that when mapping from JSON to a class, methods ('set' methods) for assigning the JSON values to the variables in the class must be created. A sample structure for example:
public class State {
public State() {}
public State(String name, String abbreviation, String capital, String statehoodYear, String statehoodOrder) {
this.name = name;
this.abbreviation = abbreviation;
this.capital = capital;
this.statehoodYear = statehoodYear;
this.statehoodOrder = statehoodOrder;
}
public String getName() {
return name;
}
public String getAbbreviation() {
return abbreviation;
}
public String getCapital() {
return capital;
}
public String getStatehoodYear() { return statehoodYear; }
public String getStatehoodOrder() {
return statehoodOrder;
}
public void setName(String name) {
this.name = name;
}
public void setAbbreviation(String abbreviation) {
this.abbreviation = abbreviation;
}
public void setCapital(String capital) {
this.capital = capital;
}
public void setStatehoodYear(String statehoodYear) {
this.statehoodYear = statehoodYear;
}
public void setStatehoodOrder(String statehoodOrder) {
this.statehoodOrder = statehoodOrder;
}
}
The declaration of an empty constructor is necessary when making use of multiple constructors where, one is parametric. In some cases without the inclusion of such constructor an error may be thrown so, to avoid the possibility of said error, adding the constructor as a "Dummy" so to say, is essential.
I have a Java Spring based web application and I want to insert a record to a table only if the table does not contain any rows that are "similar" (according to some specific, irrelevant criteria) to the new row.
Because this is a multi-threaded environment, I cannot use a SELECT+INSERT two-step combination as it would expose me to a race condition.
The same question was first asked and answered here and here several years ago. Unfortunately, the questions have got only a little attention and the provided answer is not sufficient to my needs.
Here's the code I currently have and it's not working:
#Component("userActionsManager")
#Transactional
public class UserActionsManager implements UserActionsManagerInterface {
#PersistenceContext(unitName = "itsadDB")
private EntityManager manager;
#Resource(name = "databaseManager")
private DB db;
...
#SuppressWarnings("unchecked")
#Override
#PreAuthorize("hasRole('ROLE_USER') && #username == authentication.name")
public String giveAnswer(String username, String courseCode, String missionName, String taskCode, String answer) {
...
List<Submission> submissions = getAllCorrectSubmissions(newSubmission);
List<Result> results = getAllCorrectResults(result);
if (submissions.size() > 0
|| results.size() > 0) throw new SessionAuthenticationException("foo");
manager.persist(newSubmission);
manager.persist(result);
submissions = getAllCorrectSubmissions(newSubmission);
results = getAllCorrectResults(result);
for (Submission s : submissions) manager.lock(s, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
for (Result r : results ) manager.lock(r, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
manager.flush();
...
}
#SuppressWarnings("unchecked")
private List<Submission> getAllCorrectSubmissions(Submission newSubmission) {
Query q = manager.createQuery("SELECT s FROM Submission AS s WHERE s.missionTask = ?1 AND s.course = ?2 AND s.user = ?3 AND s.correct = true");
q.setParameter(1, newSubmission.getMissionTask());
q.setParameter(2, newSubmission.getCourse());
q.setParameter(3, newSubmission.getUser());
return (List<Submission>) q.getResultList();
}
#SuppressWarnings("unchecked")
private List<Result> getAllCorrectResults(Result result) {
Query q = manager.createQuery("SELECT r FROM Result AS r WHERE r.missionTask = ?1 AND r.course = ?2 AND r.user = ?3");
q.setParameter(1, result.getMissionTask());
q.setParameter(2, result.getCourse());
q.setParameter(3, result.getUser());
return (List<Result>) q.getResultList();
}
...
}
According to the answer provided here I am supposed to somehow use OPTIMISTIC_FORCE_INCREMENT but it's not working. I suspect that the provided answer is erroneous so I need a better one.
edit:
Added more context related code. Right now this code still has a race condition. When I make 10 simultaneous HTTP POST requests approximately 5 rows will get erroneously inserted. Other 5 requests are rejected with HTTP error code 409 (conflict). The correct code would guarantee that only 1 row would get inserted to the database no matter how many concurrent requests I make. Making the method synchronous is not a solution since the race condition still manifests for some unknown reason (I tested it).
Unfortunately after several days of research I was unable to find a short and simple solution to my problem. Since my time budget is not unlimited I had to come up with a workaround. Call it a kludge if you may.
Since the whole HTTP request is a transaction, it will be rolled back at the sight of any conflicts. I am using this for my advantage by locking a special entity within the context of the whole HTTP request. Should multiple HTTP requests be received at the same time, all but one will result in some PersistenceException.
In the beginning of the transaction I am checking whether no other correct answers have been submitted yet. During that check the lock is already effective so no race condition could happen. The lock is effective until the answer is submitted. This basically simulates a critical section as a SELECT+INSERT two step query on the application level (in pure MySQL I would have used the INSERT IF NOT EXISTS construct).
This approach has some drawbacks. Whenever two students submit an answer at the same time, one of them will be thrown an exception. This is sort of bad for performance and bandwidth because the student who received HTTP STATUS 409 has to resubmit their answer.
To compensate the latter, I am automatically retrying to submit the answer on the server side a couple of times between randomly chosen time intervals. See the according HTTP request controller code is below:
#Controller
#RequestMapping("/users")
public class UserActionsController {
#Autowired
private SessionRegistry sessionRegistry;
#Autowired
#Qualifier("authenticationManager")
private AuthenticationManager authenticationManager;
#Resource(name = "userActionsManager")
private UserActionsManagerInterface userManager;
#Resource(name = "databaseManager")
private DB db;
.
.
.
#RequestMapping(value = "/{username}/{courseCode}/missions/{missionName}/tasks/{taskCode}/submitAnswer", method = RequestMethod.POST)
public #ResponseBody
Map<String, Object> giveAnswer(#PathVariable String username,
#PathVariable String courseCode, #PathVariable String missionName,
#PathVariable String taskCode, #RequestParam("answer") String answer, HttpServletRequest request) {
init(request);
db.log("Submitting an answer to task `"+taskCode+"` of mission `"+missionName+
"` in course `"+courseCode+"` as student `"+username+"`.");
String str = null;
boolean conflict = true;
for (int i=0; i<10; i++) {
Random rand = new Random();
int ms = rand.nextInt(1000);
try {
str = userManager.giveAnswer(username, courseCode, missionName, taskCode, answer);
conflict = false;
break;
}
catch (EntityExistsException e) {throw new EntityExistsException();}
catch (PersistenceException e) {}
catch (UnexpectedRollbackException e) {}
try {
Thread.sleep(ms);
} catch(InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
if (conflict) str = userManager.giveAnswer(username, courseCode, missionName, taskCode, answer);
if (str == null) db.log("Answer accepted: `"+answer+"`.");
else db.log("Answer rejected: `"+answer+"`.");
Map<String, Object> hm = new HashMap<String, Object>();
hm.put("success", str == null);
hm.put("message", str);
return hm;
}
}
If for some reason the controller is unable to commit the transaction 10 times in a row then it will try one more time but will not attempt to catch the possible exceptions. When an exception is thrown on the 11th try then it will be processed by the global exception controller and the client will receive HTTP STATUS 409. The global exception controller is defined below.
#ControllerAdvice
public class GlobalExceptionController {
#Resource(name = "staticDatabaseManager")
private StaticDB db;
#ExceptionHandler(SessionAuthenticationException.class)
#ResponseStatus(value=HttpStatus.FORBIDDEN, reason="session has expired") //403
public ModelAndView expiredException(HttpServletRequest request, Exception e) {
ModelAndView mav = new ModelAndView("exception");
mav.addObject("name", e.getClass().getSimpleName());
mav.addObject("message", e.getMessage());
return mav;
}
#ExceptionHandler({UnexpectedRollbackException.class,
EntityExistsException.class,
OptimisticLockException.class,
PersistenceException.class})
#ResponseStatus(value=HttpStatus.CONFLICT, reason="conflicting requests") //409
public ModelAndView conflictException(HttpServletRequest request, Exception e) {
ModelAndView mav = new ModelAndView("exception");
mav.addObject("name", e.getClass().getSimpleName());
mav.addObject("message", e.getMessage());
synchronized (db) {
db.setUserInfo(request);
db.log("Conflicting "+request.getMethod()+" request to "+request.getRequestURI()+" ("+e.getClass().getSimpleName()+").", Log.LVL_SECURITY);
}
return mav;
}
//ResponseEntity<String> customHandler(Exception ex) {
// return new ResponseEntity<String>("Conflicting requests, try again.", HttpStatus.CONFLICT);
//}
}
Finally, the giveAnswer method itself utilizes a special entity with a primary key lock_addCorrectAnswer. I lock that special entity with the OPTIMISTIC_FORCE_INCREMENT flag which makes sure that no two transactions can have overlapping execution times for the giveAnswer method. The respective code can be seen below:
#Component("userActionsManager")
#Transactional
public class UserActionsManager implements UserActionsManagerInterface {
#PersistenceContext(unitName = "itsadDB")
private EntityManager manager;
#Resource(name = "databaseManager")
private DB db;
.
.
.
#SuppressWarnings("unchecked")
#Override
#PreAuthorize("hasRole('ROLE_USER') && #username == authentication.name")
public String giveAnswer(String username, String courseCode, String missionName, String taskCode, String answer) {
.
.
.
if (!userCanGiveAnswer(user, course, missionTask)) {
error = "It is forbidden to submit an answer to this task.";
db.log(error, Log.LVL_MAJOR);
return error;
}
.
.
.
if (correctAnswer) {
.
.
.
addCorrectAnswer(newSubmission, result);
return null;
}
newSubmission = new Submission(user, course, missionTask, answer, false);
manager.persist(newSubmission);
return error;
}
private void addCorrectAnswer(Submission submission, Result result) {
String var = "lock_addCorrectAnswer";
Global global = manager.find(Global.class, var);
if (global == null) {
global = new Global(var, 0);
manager.persist(global);
manager.flush();
}
manager.lock(global, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
manager.persist(submission);
manager.persist(result);
manager.flush();
long submissions = getCorrectSubmissionCount(submission);
long results = getResultCount(result);
if (submissions > 1 || results > 1) throw new EntityExistsException();
}
private long getCorrectSubmissionCount(Submission newSubmission) {
Query q = manager.createQuery("SELECT count(s) FROM Submission AS s WHERE s.missionTask = ?1 AND s.course = ?2 AND s.user = ?3 AND s.correct = true");
q.setParameter(1, newSubmission.getMissionTask());
q.setParameter(2, newSubmission.getCourse());
q.setParameter(3, newSubmission.getUser());
return (Long) q.getSingleResult();
}
private long getResultCount(Result result) {
Query q = manager.createQuery("SELECT count(r) FROM Result AS r WHERE r.missionTask = ?1 AND r.course = ?2 AND r.user = ?3");
q.setParameter(1, result.getMissionTask());
q.setParameter(2, result.getCourse());
q.setParameter(3, result.getUser());
return (Long) q.getSingleResult();
}
}
It is important to note that the entity Global has to have a version annotation in its class for the OPTIMISTIC_FORCE_INCREMENT to work (see code below).
#Entity
#Table(name = "GLOBALS")
public class Global implements Serializable {
.
.
.
#Id
#Column(name = "NAME", length = 32)
private String key;
#Column(name = "INTVAL")
private int intVal;
#Column(name = "STRVAL", length = 4096)
private String strVal;
#Version
private Long version;
.
.
.
}
Such an approach can be optimized even further. Instead of using the same lock name lock_addCorrectAnswer for all giveAnswer calls, I could generate the lock name deterministically from the name of the submitting user. For example, if the student's username is Hyena then the primary key for the lock entity would be lock_Hyena_addCorrectAnswer. That way multiple students could submit answers at the same time without receiving any conflicts. However, if a malicious user spams the HTTP POST method for submitAnswer 10x in parallel they will be prevented by the this locking mechanism.
I have a simple entity class and it is supposed to include unique names on it.
#Entity
class Package {
#PrimaryKey(sequence = "ID")
public Long id;
#SecondaryKey(relate = Relationship.ONE_TO_ONE)
public String name;
private Package() {}
public Package(String name) { this.name = name; }
#Override
public String toString() { return id + " : " + name; }
}
I want to use deferred writing option because of extensive modification. Here is the test i tried and its output.
final String dbfilename = "test01";
new File(dbfilename).mkdirs();
EnvironmentConfig config = new EnvironmentConfig().setAllowCreate(true);
Environment environment = new Environment(new File(dbfilename), config);
StoreConfig storeConfig = new StoreConfig().setAllowCreate(true).setDeferredWrite(true);
EntityStore store = new EntityStore(environment, "", storeConfig);
PrimaryIndex<Long, Package> primaryIndex = store.getPrimaryIndex(Long.class, Package.class);
try {
primaryIndex.put(new Package("package01")); // will be put.
primaryIndex.put(new Package("package01")); // throws exception.
} catch (UniqueConstraintException ex) {
System.out.println(ex.getMessage());
}
store.sync(); // flush them all
// expecting to find one element
SortedMap<Long,Package> sortedMap = primaryIndex.sortedMap();
for (Package entity : sortedMap.values()) {
System.out.println(entity);
}
Output
(JE 5.0.73) Unique secondary key is already present
1 : package01
2 : package01
So my question is that even if it throws exception while putting second package, why does it lists two packages. Any way to avoid this without using transactions?
Thanks.