I am getting conditionalcheckfailed exception when trying to save/update items using dynamodb mapper.
Can anyone please share snippet of code using java that can demonstrate how versioning and optimistic locking can be implemented successfully?
Tried not setting version at all!!
Tried adding a record to table, and then doing read before save.
Nothing woked!! I continue to get ConditionalCheckFailed Exception.
Only thing works is if I set the config to COBBLER!! but that's not what I want as I need optimistic locking for my data.
DB item class---
#DynamoDBTable(tableName="Funds")
public class FundsItem {
private String id;
private String auditId;
private Long version;
private String shopId;
private String terminalId;
private String txId;
#DynamoDBHashKey(attributeName = "Id")
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
#DynamoDBRangeKey(attributeName = "AuditId")
public String getAuditId() {
return auditId;
}
public void setAuditId(String auditId) {
this.auditId = auditId;
}
#DynamoDBVersionAttribute(attributeName = "Version")
public Long getVersion() { return version; }
public void setVersion(Long version) { this.version = version; }
#DynamoDBAttribute(attributeName = "ShopId")
public String getShopId() {
return shopId;
}
public void setShopId(String shopId) {
this.shopId = shopId;
}
#DynamoDBAttribute(attributeName = "TerminalId")
public String getTerminalId() { return terminalId; }
public void setTerminalId(String terminalId) {
this.terminalId = terminalId;
}
#DynamoDBAttribute(attributeName = "TxId")
public String getTxId() {
return txId;
}
public void setTxId(String txId) {
this.txId = txId;
}
}
Code to save new item -----
public long addFunds(FundsRequest request){
FundsItem dbItem = new FundsItem();
String Id = request.getShopId().trim() + request.getTerminalId().trim();
String V0_Audit_Rec = "V0_Audit_" + Id;
//save V0 item.
dbItem.setVersion((long) 1);
dbItem.setId(Id);
dbItem.setAuditId(V0_Audit_Rec);
dbItem.setShopId(request.getShopId().trim());
dbItem.setTerminalId(request.getTerminalId().trim());
dbItem.setTxId(request.getTxId().trim());
mapper.save(dbItem);
}
Pls check the snippet above - This is a new empty table.
hashkey - id, rangekey - auditId, VersionField - version.
I just want to be able to add a new record that's why not doing any read before saving a new item. If I can get this simple case i.e. adding a new /first record to the dynamodb table work, I can implement rest of the use cases too.
In general:
Never set your version, the SDK will initialise this if required.
Always try and load an item with your key first. If null is returned, create the item and save it. Else update the returned item and save it.
I know you mentioned you've tried the above. If its truely an empty table your code should work OK (minus the setting of the version).
A couple of things I would also do:
Don't set your version field with a custom attribute name. In theory this should be fine, but for the sake of making your code the same as the AWS examples, I would remove this, at least until you have it working.
Although I think you need to remove the setting of the version, I note you are casting to a long, not a Long. Again, unlikely to be an issue but just something to eliminate at least. i.e. if you insist of setting version use new Long(l).
Related
I'm not sure how to phrase the question title to be honest, if someone has a suggestion, please let me know.
My use case is this, I have an entity with an account property like so (this is cleaned up to avoid clutter):
#Entity
#Table(name = "report_line", schema = "public")
public class ReportLine extends BaseReportLine {
#ManyToOne
#JoinColumn(name = "report_id")
private Report report;
#ManyToOne
#JoinColumn(name = "account_id")
private Account account;
}
But a DTO that only has an account id / different properties:
public class ImportLineDto {
public String groupName;
public Integer position;
public Integer parentPosition;
public String accountId;
public String name;
public BigDecimal amount;
public List<ImportLineDto> lines = new ArrayList<>();
}
I need to go through / flatten all lines so I can save it to a JPA repository, but there are 2 issues:
Is there a way to create the table line object using the accountId only, without having to look up the account for each line, as that will add a massive amount of unnecessary db calls.
What should I do with the 'lines' on each table object after flattening? Should I set them to null / empty list?
Is there a better way to do this? For once I can actually make changes to the code
Here is what I have so far:
private void saveReport(ImportedResult result) {
Report report = new Report();
...
report.setLines(getReportLinesFromDtoLines(result.lineItems.lines));
ReportRepository.saveAndFlush(report);
}
private List<ReportLine> getReportLinesFromDtoLines(ImportLineDto lines) {
List<ImportLineDto> flatLines = flatMapRecursive(lines).collect(Collectors.toList());
List<ReportLine> reportLines = new ArrayList<>();
for(ImportLineDto line: flatLines) {
ReportLine reportLine = new ReportLine();
reportLine.setItemText(line.name);
reportLine.setAmount(line.amount);
reportLine.setAccount(???);
// how do I set the 'Account' property using the id only, without looking up each account?
reportLines.add(reportLine);
}
return ReportLines;
}
public Stream<ImportLineDto> flatMapRecursive(ImportLineDto item) {
if (item.lines == null) {
return Stream.empty();
}
return Stream.concat(Stream.of(item), item.lines.stream()
.flatMap(this::flatMapRecursive));
}
Follow up:
Just to throw a wrench in there, what if the DTO accountId was not the actual "id" field in the table, but another custom field, I have another situation like that, would it even be possible? I still need the answer the the 1st question however with a standard id.
you may use entityManager.getReference as explained here
reportLine.setAccount(entityManager.getReference(Account.class, line.accountId));
I'm terribly sorry if I can't start another post which is connected to my previous one but my question is somewhat different.
I noticed that I really can save new data in my database as long as I never added data to the database by using the line spring.datasource.initialization-mode=always in my application.properties and made a data.sql file with a few insert statements. Once I insert the data using that file, I can access the data and show it to the user, but I can't create any new data because I get the following error
ERROR: duplicate key value violates unique constraint "joke_pkey"
Detail: Key (id)=(1) already exists.
Does anyone know how to help me with this? I'm doing an interview task and I am meant to first import data using the data.sql file and then later add some more data.
The post with my code is here:
Spring Boot using save never inserts a row inside of a Postgresql table
EDIT - someone recommended adding my code here directly and saying what I've tried.
I have tried to initialize the database with the application properties the way they are, then restarting the app but without the last line, and setting the spring.jpa.hibernate.ddl-auto to none. But even so, it didn't work. I genuinely expected it to work like that. Because if the table is empty and I fill it in using the functions I created, everything works like a charm, even after restarting the server (id keep the ring.jpa.hibernate.ddl-auto to none again to keep the data from being deleted)
I have also tried simply changing the GenerationType.AUTO to GenerationType.TABLE strategy in my Joke class, but that didn't seem to change anything either.
application.properties :
spring.datasource.url=jdbc:postgresql://localhost:5432/flyway_demo
spring.datasource.username=bob
spring.datasource.password=bob123
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.hibernate.ddl-auto=create
spring.datasource.initialization-mode=always
My Web Controller that has the post function:
#PostMapping("/post")
public String insertJoke(JokeForm jokeForm) {
int categoryid = jokeForm.getCategoryId();
String content = jokeForm.getContent();
databasController.insert(categoryid, content);
return "redirect:/";
}
My DBController whose insert function is being called
public Joke insert(int categoryid, String content) {
return jokeRepository.save(new Joke(categoryid, content));
}
Most of my Joke data class:
#Entity
public class Joke {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(columnDefinition = "serial")
private Long id;
#NotNull
#Column(name = "category_id_FK")
private long categoryId;
#NotBlank
private String content;
#Column(columnDefinition = "integer default 0")
private int likes = 0;
#Column(columnDefinition = "integer default 0")
private int dislikes = 0;
public Joke() {
}
public Joke(long categoryid, String content) {
this.setCategoryid(categoryid);
this.setContent(content);
}
// id
public Long getId() {
return this.id;
}
// id
public void setId(Long id) {
this.id = id;
}
// categoryid
public long getCategoryid() {
return this.categoryId;
}
public void setCategoryid(long categoryid) {
this.categoryId = categoryid;
}
// content
public String getContent() {
return this.content;
}
public void setContent(String content) {
this.content = content;
}
// likes
public int getLikes() {
return this.likes;
}
public void setLikes(int likes) {
this.likes = likes;
}
// dislikes
public int getDislikes() {
return this.dislikes;
}
public void setDislikes(int dislikes) {
this.dislikes = dislikes;
}
}
Joke Repository:
#Repository
public interface JokeRepository extends JpaRepository<Joke, Integer> {
Joke findById(long id);
List<Joke> findByCategoryid(int categoryid);
}
It seems that all you need to do is change GenerationType.AUTO to GenerationType.IDENTITY.
Reason behind this is the sequence, which might be out of sync if you use AUTO. Because then hibernate uses its own sequence instead of the one postgres creates when using serial.
This is maybe a beginner question on hibernate. I am doing my first steps, I designed a simple datamodel consisting of about 10 entities and I use hibernate to persist them to my Oracle XE database. Now I am facing the following problem: First time, when I do a transaction to persist some entities, they are persisted properly. I verify, that the data exists in the database and then I delete all the entries from all database tables. I verify that all tables are empty again. Then I run my program again to persist some new entities - and here happens something really strange: Afterwards I find in my databse the new entries as well as the old ones, which were persisted last time and which I had deleted! They contained the old IDs and the old data fields! How can this be? This happens even if I shut down my computer after the first time the program runs! How does it remember the old entries and where are they saved? Do you have any ideas?
Some information, that might be useful:
I am using annotations (instead of config files) for the mapping.
Following you see the classes used for persisting as well as one example of an entity (I am showing only one entity to avoid making the question too long).
As you see, I am using FetchType = EAGER on my MANY to MANY mappings (as I understand, this makes sure, that all related entities are loaded immediately together with any loaded entity). Can this have any impact?
Thanks for any help!
public class PersistenceManager {
private static final SessionFactory factory = new Configuration().configure().buildSessionFactory();
public static void sampleData() {
try(Session session = factory.openSession()) {
SampleDataLoader.loadSampleData(session);
} catch(HibernateException e) {
System.out.println("Exception during persisting! Message: " + e.getMessage());
e.printStackTrace();
}
}
}
public class SampleDataLoader {
static void loadSampleData(Session session) {
Language french = new Language("French");
Language german = new Language("German");
Noun garcon = new Noun(french, "garcon", false);
Noun junge = new Noun(german, "Junge", false);
junge.addTranslation(garcon);
ZUser user = new ZUser("Daniel", "password");
user.setOwnLanguage(german);
user.setEmail("abc#somemail.de");
user.setDateRegistered(LocalDateTime.now());
user.addForeignLanguage(french);
Transaction transaction = session.beginTransaction();
session.save(user);
session.save(french);
session.save(german);
session.save(junge);
transaction.commit();
}
}
#Entity
public class ZUser {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
#Column(name = "id")
private int id;
#Column
private String name;
#Column
private String password;
#Column
private String email;
#Column
private String picturePath;
#Column
private LocalDateTime dateRegistered;
#ManyToOne(fetch=FetchType.EAGER)
#JoinColumn(name="OWNLANGUAGE_ID")
private Language ownLanguage;
#ManyToMany(cascade = { CascadeType.ALL })
#JoinTable(name="USER_LANGUAGE",
joinColumns=#JoinColumn(name="USER_ID"),
inverseJoinColumns=#JoinColumn(name="LANGUAGE_ID")
)
private Set<Language> foreignLanguages = new HashSet<>();
public ZUser() { }
public ZUser(String n, String p) {
name = n;
password = p;
}
public int getId() { return id; }
public void setId(int id) { this.id = id; }
public String getName() { return name; }
public void setName(String name) { this.name = name; }
public String getPassword() { return password; }
public void setPassword(String password) { this.password = password; }
public String getEmail() { return email; }
public void setEmail(String email) { this.email = email; }
public String getPicturePath() { return picturePath; }
public void setPicturePath(String picturePath) { this.picturePath = picturePath; }
public LocalDateTime getDateRegistered() { return dateRegistered; }
public void setDateRegistered(LocalDateTime dateRegistered) { this.dateRegistered = dateRegistered; }
public Language getOwnLanguage() { return ownLanguage; }
public void setOwnLanguage(Language ownLanguage) { this.ownLanguage = ownLanguage; }
public void addForeignLanguage(Language language) {foreignLanguages.add(language);}
public Set<Language> getForeignLanguages() {return Collections.unmodifiableSet(foreignLanguages); }
}
Clarified by the comment of Jagger (see comments). Indeed, I was using Oracle SQL command line to delete the entries and I had rgotten, that I need to explicitely commit after deleting. The solution can be so easy :)
I want to use Objectify to query Google Cloud Datastore. What is an appropriate way to find a record based on a known key-value pair? The record is in the database, I verified this by Google's Datastore viewer.
Here is my method stub, which triggers the NotFoundException:
#ApiMethod(name="getUser")
public User getUser() throws NotFoundException {
String filterKey = "googleId";
String filterVal = "jochen.bauer#gmail.com";
User user = OfyService.ofy().load().type(User.class).filter(filterKey, filterVal).first().now();
if (user == null) {
throw new NotFoundException("User Record does not exist");
}
return user;
}
Here is the User class:
#Entity
public class User {
#Id
Long id;
private HealthVault healthVault;
private String googleId;
public User(String googleId){
this.googleId = googleId;
this.healthVault = new HealthVault();
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public HealthVault getHealthVault() {
return healthVault;
}
public void setHealthVault(HealthVault healthVault) {
this.healthVault = healthVault;
}
public String getGoogleId() {
return googleId;
}
public void setGoogleId(String googleId) {
this.googleId = googleId;
}
}
I think it fails because of transaction. You need to make a transctionless call like:
User user = OfyService.ofy().transactionless().load().type(User.class).filter(filterKey, filterVal).first().now();
More info about transactions on App Engine:
https://cloud.google.com/appengine/docs/java/datastore/transactions
https://github.com/objectify/objectify/wiki/Transactions
EDIT
Your object needs #Index annotation. It will add field to datastore index. Only properties that are in the index can be searchable. Filter method is one of them.
#Id
Long id;
#Index
private HealthVault healthVault;
#Index
private String googleId;
P.S. delete your object with googleId jochen.bauer#gmail.com and write it again to database after you updated your entity. And objectify will find it.
First add #Index in your fields model. I didn't see filterVal as an email in your model. Even so, to get the entity based in your filterVal assuming that is googleId is the field of your entity.
User user = OfyService.ofy().load().type(User.class).filter("googleId", filterVal).now();
And so if your filterKey is the id of your entity.
User user = OfyService.ofy().load().key(Key.create(User.class, filterKey)).now();
Im using the entity manager JPA with eclipse link 2.3 &Derby db and I have model with 10 entities and for each entity I need to store 1000 records ,this process takes about 70 sec. I have test it for the same model with 10 entities but with 100 records the all process with the commit take about 1.2 sec which is great.
the bottle neck is the entityManager.getTransaction().commit(); which I do just one time after i persist all the data, the commit take more 95% from the all process.
when I use the JVM monitor I dive in to the commit and I see that one of the class is responsible for almost all commit time ,the class is org.eclipse.persistence.mappings.ManyToManyMapping
http://www.eclipse.org/eclipselink/api/1.0/org/eclipse/persistence/mappings/ManyToManyMapping.html
my entities and the model doesn't have any many to many relationship or use any many to many annotation what could be the reason for the Exponential behavior ?
I have noticed that when I remove this two entities the time was saved by 85%
what is wrong with this entities
The navigation is from person which have cardinality 1 to title award that have cardinality of N i.e one person can have many awards...
#javax.persistence.Entity
#javax.persistence.Table(name = "a3_Person")
public class Person {
#javax.persistence.Id
#javax.persistence.Column(length = 20)
//// Id;
private String person_id;
public String getPerson_id() { return this.person_id; }
public void setPerson_id(String person_id) { this.person_id = person_id; }
#javax.persistence.Column(length = 30)
//// Name;
private String person_name;
public String getPerson_name() { return this.person_name; }
public void setPerson_name(String person_name) { this.person_name = person_name; }
//// Awards;
private List<TitleAward> person_awards;
public List<TitleAward> getPerson_awards() { return this.person_awards; }
public void setPerson_awards(List<TitleAward> person_awards) { this.person_awards = person_awards; }
}
#javax.persistence.Entity
#javax.persistence.Table(name = "a3_TitleAward")
public class TitleAward {
#javax.persistence.Id
#javax.persistence.Column(length = 20)
//// Id;
private String titleaward_id;
public String getTitleaward_id() { return this.titleaward_id; }
public void setTitleaward_id(String titleaward_id) { this.titleaward_id = titleaward_id; }
#javax.persistence.Column(length = 30)
//// Type;
private String titleaward_type;
public String getTitleaward_type() { return this.titleaward_type; }
public void setTitleaward_type(String titleaward_type) { this.titleaward_type = titleaward_type; }
#javax.persistence.Column(length = 30)
//// Category;
private String Cateeheihbadc;
public String getCateeheihbadc() { return this.Cateeheihbadc; }
public void setCateeheihbadc(String Cateeheihbadc) { this.Cateeheihbadc = Cateeheihbadc; }
#javax.persistence.Column()
//// Year;
private String titleaward_year;
public String getTitleaward_year() { return this.titleaward_year; }
public void setTitleaward_year(String titleaward_year) { this.titleaward_year = titleaward_year; }
#javax.persistence.Column()
//// Won;
private Boolean titleaward_won;
public Boolean getTitleaward_won() { return this.titleaward_won; }
public void setTitleaward_won(Boolean titleaward_won) { this.titleaward_won = titleaward_won; }
//// Person;
private Person Pers_fhfgdcjef;
public Person getPers_fhfgdcjef() { return this.Pers_fhfgdcjef; }
public void setPers_fhfgdcjef(Person Pers_fhfgdcjef) { this.Pers_fhfgdcjef = Pers_fhfgdcjef; }
}
There are a number of performance optimization outlined here,
http://java-persistence-performance.blogspot.com/2011/06/how-to-improve-jpa-performance-by-1825.html
ManyToManyMapping is also used for the #OneToMany annotation that uses a #JoinTable, are you using this? In general correctly profiling and understanding a profile can be difficult, so your profile may not be valid.
Please include your code, and a sample of the SQL log, and profile. You can also enable the EclipseLink PerformanceMonitor,
see,
http://www.eclipse.org/eclipselink/documentation/2.4/concepts/monitoring003.htm
If 100 records only takes 1.2s then you could probably break your process into batches of 100 and get 12s instead of 70s. 70s sounds like you have some sort of n^2 issue going on.