We Keep Coding

JPA not fetching data that reflects state of database

I have encountered a curious bug or feature while writing code. Here's the situation:
We are using a PostgreSQL database, EclipseLink in a JavaEE project.
What I am doing in this part of the project is fetching an entity from the database i.e.:
User user = userController.get(userId);
Which then goes to our controller and fetches the user via a TypedQuery:
#Stateless
#LocalBean
public class UserController {
private EntityManager em;
public User get(Integer userId){
User retval = null;
TypedQuery<User> = em.createNamedQuery("User.findByUserId", User.class);
q.setParameter("userId", userId);
retval = q.getSingleResult();
}
public User update(final User modified){...}
}
And in my User class I have:
#NamedQuery(name = "User.findByUserId", query = "SELECT u FROM User u WHERE u.id = :userId"),
So the call goes, I get my user object with its respective data from the database.
In the class where I called the userController.get method I continue to modify the data on this object, and call our controller again to update this data on the database
user.setServiceId(1); //any id (integer) pointing to an existing service, this is a ManyToOne relationship
userController.update(user);
And here is where it gets funny. In our update method inside the controller class I have my modified User object and using this object I get the primary key userId and fetch the data again from the database to get the original:
#Stateless
#LocalBean
public class userController {
private EntityManager em;
public User get(Integer userId){...}
public User update(final User modified){
User retval = null;
if(modified != null){
try {
User original = get(modified.getId()); //Here I fetch the current state of the DB
if(original != null){
Set<Modifications> modifications = apply(original, modified); //Method to apply modifications
retval = em.merge(original); //Merge changes into database
em.flush(); //Force data to be persisted
catch(Exception e){
}
return retval;
}
}
However, the fields in the original object do not reflect the state of the database but instead contains the same data as the modified object. In this case, the serviceId on the database is null, and in the modified I set it to an ID. The original has its serviceId set to the same value as the modified object even though it should contain the fetched data from the database, in this case null
My current solution is to construct a new User object, after fetching the user from the database, and modify the data on that new object:
User user = userController.get(userId);
User newUser = new User(user);
newUser.setService(service);
userController.update(newUser);
Now when I do the update method, the original reflects the state of the database.
Or maybe it reflects the state of the user object that already exists in the persistence context?
But why does this happen? Since I do make a new get call with a SELECT statement to the database in my update method.

You are using the same EntityManager for everything, both the read and the 'merge', which in this case is then a no-op. Everything read in through an EM is managed, so that if you read it back again, you get the same instance back. As long as the User isn't being serialized, it is 'managed' by the EntityManager it was read from, and so that same instance, and its changes, are visible on any get calls on that ID.
You didn't show how you are getting EntityManagers, but I would guess is isn't container managed, as they would inject a new one for these calls, and then close them for you when done. You haven't shown any transaction logic on how the update and the em context it is using are hooked up, but I would suggest you create a new EntityManager for these calls. Flush also seems unnecessary, as if update is wrapped in a transaction, should handle flushing the update statement to the database without this extra call.

If user.setServiceId(1); is called when the "user" entity is managed, the call is going to update the database row.
you can check the manage entity lifecycle

You need to refresh the data after saving it to the database and to get the latest state of the object, as em.refresh(retval)
You can find the code added below.
#Stateless
#LocalBean
public class userController {
private EntityManager em;
public User get(Integer userId){...}
public User update(final User modified){
User retval = null;
if(modified != null){
try {
User original = get(modified.getId()); //Here I fetch the current state of the DB
if(original != null){
Set<Modifications> modifications = apply(original, modified); //Method to apply modifications
retval = em.merge(original); //Merge changes into database
em.flush(); //Force data to be persisted
em.refresh(retval); // This will fetch the updated data from database
catch(Exception e){
}
return retval;
}
}

Update/notify other User in Spring Web

I have some design/implementation issue that I just can't wrap my head around it. I am currently working on a text-based game with multiple players. I kind of understand how it works for Player-to-Server, I meant that Server sees every individual Player as the same.
I'm using spring-boot 2, spring-web, thymeleaf, hibernate.
I implemented a custom UserDetails that returns after the user login.
#Entity
#Table(name = "USER")
public class User implements Serializable {
#Id
private long userId;
#Column(unique = true, nullable = false)
private String userName;
#OneToOne(cascade = CascadeType.ALL)
#JoinColumn(name = "playerStatsId")
private PlayerStats stats;
}
public class CurrentUserDetailsService implements UserDetailsService {
#Override
public CurrentUser loadUserByUsername(String userName) {
User user = this.accountRepository.findByUserName(userName)
.orElseThrow(() ->
new UsernameNotFoundException("User details not found with the provided username: " + userName));
return new CurrentUser(user);
}
}
public class CurrentUser implements UserDetails {
private static final long serialVersionUID = 1L;
private User user = new User();
public CurrentUser(User user) {
this.user = user;
}
public PlayerStats getPlayerStats() {
return this.user.getStats();
}
// removed the rest for brevity
}
Hence, in my controller, I can do this to get the CurrentUser.
*Note each User is also a player.
#GetMapping("/attackpage")
public String viewAttackPage(#AuthenticationPrincipal CurrentUser currentUser) {
// return the page view for list of attacks
return "someview";
}
The currentUser here would reflect to the current user per say (Player 1 or 2 or 3 and so on). Which works fine for most of the stuff happening to themselves such as purchasing some stuff, updating profile and so on.
But what I can't get or know how to achieve is when 2 players interact.
For example, Player 1 attacks Player 2. If I am Player 1, what I'll do is to click the "Attack" on the View and select the Player 2, and submit the command. Hence, in the controller, it will be something like this.
#GetMapping("/attack")
public String launchAttack(#AuthenticationPrincipal CurrentUser currentUser, #RequestParam("playername") String player2) {
updatePlayerState(player2);
return "someview";
}
public void updatePlayerState(String player) {
User user = getUserByPlayername(player);
// perform some update to player state (say health, etc)
// update back to db?
}
Here's is what really got me confused.
As seen previously, when each User/Player logs in, a set of user (player) current state will be pulled from the DB and store "in-memory".
Hence, when Player 1 attacks Player 2,
How do I "notify" or update Player 2 that the stats has changed, and thus, Player 2 should pull updated stats from db to memory.
How to tackle the possible concurrency issue here? For example, Player 2 health is 50 in DB. Player 2 then perform some action (say purchase health potion + 30), which then update the DB (health to 80). However, just before the DB is updated, Player 1 has already launch the attack and grab from DB the state of Player 2 where it will return 50 since DB has yet to be updated. So now, whatever changes made in getUserByPlayername() and update to the DB will be wrong, and the entire state of the Player will be "de-sync". I hope I am making sense here.
I understand that there is #Version in hibernate for optimistic locking but I'm not sure if it's applicable in this case. And would spring-session be useful in such case?
Should I not store the any data in memory when user login? Should I always be retrieving data from DB only when some action is performed? Like when viewProfile, then I pull from accountRepository. or when viewStats then I pull from statsRepository and on so.
Do point me in the right direction. Would appreciate for any concrete example of sort, or some kind of video/articles. If there is any additional information required, do let me know and I'll try to explain my case better.
Thank you.

I think that you should not be updating the currentUser in your Controller methods, and should not be relying on the data in that object to represent a player's current state. There are probably ways to get that to work, but you'd need to mess around with updating the security context.
I also recommend that you lookup Users by id instead of userName, so will write the rest of this answer with that approach. If you insist on finding Users by userName, adjust where necessary.
So, keeping it simple, I would have a reference to the accountRepository in the Controller, and then, whenever you need to get or update a player's state, use
User user = accountRepository.findById(currentUser.getId())
Yes, #Version and optimistic locking will help with the concurrency issues that you're concerned about. You can reload the Entity from the database, and retry the operation if you catch an #OptimisticLockException. Or, you may want to respond to player 1 with something like "Player 2 has just purchased a potion of healing, and is now 80 heath, do you still want to attack?"

I'm not a spring user, but I think that the problem is more conceptual than technical.
I'll try to provide an answer which uses a general approach, while writing the examples in a JavaEE style so that they should be understandable, and hopefully, portable to spring.
First of all: every single DETACHED entity is stale data. And stale data is not "trustable".
So:
each method that modify the state of an object should re-fetch the object from DB inside the transaction:
updatePlayerState() should be a transaction-boundary method (or called inside a tx), and getUserByPlayername(player) should fetch the target object from the DB.
JPA speaking: em.merge() is forbidden (without proper locking, i.e. #Version).
if you (or spring) are doing this already, there's little to add.
WRT the "lost update problem" you mention in your 2. be aware that this covers the application server side (JPA/Hibernate), but the very same problem could be present on DB side, which should be properly configured for, at least, repeatable read isolation. Take a look at MySQL does not conform to Repeatable Read really, if you are using it.
you have to handle controller fields that refer stale Players/Users/Objects. You have, at least, two options.
re-fetch for each request: suppose Player1 has attacked Player2 and diminished Player2 HP by 30. When Player2 goes to a view that shows his HP, the controller behind that view should have re-fetched the Player2/User2 entity before rendering the view.
In other words, all of your presentation (detached) entities should be, sort of, request-scoped.
i.e you can use a #WebListener to reload your Player/User:
#WebListener
public class CurrentUserListener implements ServletRequestListener {
#Override
public void requestInitialized(ServletRequestEvent sre) {
CurrentUser currentUser = getCurrentUser();
currentUser.reload();
}
#Override
public void requestDestroyed(ServletRequestEvent sre) {
// nothing to do
}
public CurrentUser getCurrentUser() {
// return the CurrentUser
}
}
or a request-scoped bean (or whatever-spring-equivalent):
#RequestScoped
public class RefresherBean {
#Inject
private CurrentUser currentUser;
#PostConstruct
public void init()
{
currentUser.reload();
}
}
notify other controller instances: if the update succeeded a notification should be sent to other controllers.
i.e. using CDI #Observe (if you have CDI available):
public class CurrentUser implements UserDetails {
private static final long serialVersionUID = 1L;
private User user = new User();
public CurrentUser(User user) {
this.user = user;
}
public PlayerStats getPlayerStats() {
return this.user.getStats();
}
public void onUpdate(#Observes(during = TransactionPhase.AFTER_SUCCESS) User user) {
if(this.user.getId() == user.getId()) {
this.user = user;
}
}
// removed the rest for brevity
}
Note that CurrentUser should be a server-managed object.

foreign keys in objectify / google app engine

I'm trying to write an application for the google app engine using Objectify and am having some trouble. I'm new to noSQL datastores so it's probably a conceptual problem.
I have an entity called a message, and each message has a from User - the user who created the message.
#Entity
public class Message {
#Index private Key<User> fromUserKey;
#IgnoreSave private User fromUser;
Annoyingly, I have to have both a User and a Key field in the message. JSON needs the User object to populate the response with useful fields, and the google app engine needs the Key to be able to store the reference to the user. The #IgnoreSave annotation is used to stop Objectify asking the google app engine to try to store the User object (which will fail).
When fetching messages, the from user key is populated, but the from User object is not. Here's what the DAO code looks like for my "getMessages" operation:
public static List<Message> getSentMessages(long userId) {
List<Message> result;
result= ofy().load().type(Message.class).filter("from", Key.create(User.class, userId)).limit(1000).list();
return result;
}
But the fromUser object is not populated on each Message, only the fromUserKey. How do I get the actual User object populated in the response? I need to access such fields as forename, surname, etc - not just the ID of the User. I could get the from User from the DB, loop through the results, then call setFromUSer on each message, but it's ugly. And it also causes a ConcurrentModificationException...
I'm sure I'm doing something wrong here, but I can't work out what.

What you can do is have a property Ref<User> on the Message entity and annotate it with '#Parent'. This means that each Message entity will become part of the user's entity group.
The Ref<?> works like a key but allows you to directly access the actual entity object; that way you can easily get the forename, surname etc.
Change your class as follows:
#Entity
#Cache
public class Message
{
#Id Long id;
#Load #Parent private Ref<User> user;
public User getUser() { return this.user.get(); }
public void setUser(User value) { this.user = Ref.Create(value); }
}
Having done that, you will be able to perform ancestor queries to retrieve all the message entities associated with a particular user:
public static List<Message> getSentMessages(long userId)
{
User parent = ofy().load().type(User.class).id(userId).now();
List<Message> results = ofy().load().type(Message.class).ancestor(parent).limit(1000).list();
return results;
}
You should now be able to do what you wanted, and hopefully not get any more errors.

adding static property data when creating entities

Suppose I have two entity types: User and Country.
Country entities are never created and aren't mutable. They're keyed by ISO alpha3 country codes, e.g. "USA". They reside in the table COUNTRY with PK column ID.
Users have a many-to-one mapping to Country w/ cascade = none. Users reside in the USER table with a FK column COUNTRY_ID that references COUNTRY(ID).
When creating a new User I might do something like:
User user = new User();
user.setCountry(em.find(Country.class, "USA"));
em.persist(user);
I gather that's wasteful, though, since it requires a query against COUNTRY first. So I noticed I can also do this:
Country usa = new Country();
usa.setId("USA");
User user = new User();
user.setCountry(usa);
em.persist(user);
Since cascade = "none" this shouldn't need to query the COUNTRY table; it should just insert "USA" directly into USER as COUNTRY_ID. Is that correct?
Now suppose there's some code that only ever creates Users with COUNTRY_ID = "USA". In that case, it occurred to me to store a static instance of Country with ID = "USA" and use it for every new user? For example:
public class UsaUserFactory implements Factory<User> {
private static final Country USA = new Country();
static { USA.setId("USA"); }
public User newInstance() {
User user = new User();
user.setCountry(USA);
return user;
}
}
public SomeOtherClass {
public void persistUser(EntityManager em, Factory<User> uf, ...) {
User user = uf.newInstance();
// set some other properties
em.persist(user);
}
}
Assume persistUser() is called concurrently from multiple threads and from within multiple persistence contexts.
My questions:
Will persisting User entities mutate my singleton "USA" Country instance in any way?
Is this inadvisable for some other reasons entirely?
The two classes above are just to illustrate the question; I'm not actually doing anything quite that silly.

I would be more inclined to try using a cache for the read-only data to reduce the actual database calls. See here for help setting up a cache that might help.

many-to-many JPA mapping inserting but not fething the child collections

i've hit a block once again with hibernate.I've posted numerous times on different aspects of the user and contact management that i've been building.
The sad thing is that i didn't really have the time to play with it and understand it better before actually starting working with it. Sorry but English is not my native language, i rather speak french. And again i've started coding in java in an autodidact way.i'm doing all of this by reading books and haven't gone to school for it. with time constraints it's hard to read a book from beginning to the end.
I'm not sure i should put every of my codes dealing with an issue here and from what i've learned from other forum is to post just the necessary and being concise.
So in my User model i have UserAccount class, Profile that holds details like name, preferences etc , AccountSession and Phone.
my contact management model have Contact and Group.
UserAccount has one-to-one association with Profile, one-to-many with AccountSession,contact and group, all bidirectional.the one-to-many association with phone is unidirectional because contact also has and unidirectional with Phone.
Contact has a bidirectional many-o-many with group and one-to-many with phone that i said earlier.
Group also has a many-to-many bedirectional with contact.
here are the mappings
// UserAccount
......
#OneToOne(targetEntity=UserProfileImpl.class,cascade={CascadeType.ALL})
#org.hibernate.annotations.Cascade(value=org.hibernate.annotations.CascadeType.DELETE_ORPHAN)
#JoinColumn(name="USER_PROFILE_ID")
private UserProfile profile;
#OneToMany(targetEntity=ContactImpl.class, cascade={CascadeType.ALL}, mappedBy="userAccount")
#org.hibernate.annotations.Cascade(value=org.hibernate.annotations.CascadeType.DELETE_ORPHAN)
private Set<Contact> contacts = new HashSet<Contact>();
#OneToMany(targetEntity=GroupImpl.class, cascade={CascadeType.ALL}, mappedBy="userAccount")
#org.hibernate.annotations.Cascade(value=org.hibernate.annotations.CascadeType.DELETE_ORPHAN)
private Set<Group> groups = new HashSet<Group>();
.......
//Group
#ManyToOne(targetEntity=UserAccountImpl.class)
#JoinColumn(name="USER_ACCOUNT_ID",nullable=false)
private UserAccount userAccount;
#ManyToMany(targetEntity=ContactImpl.class,cascade={CascadeType.PERSIST, CascadeType.MERGE})
#JoinTable(name="GROUP_CONTACT_MAP", joinColumns={#JoinColumn(name="GROUP_ID")},
inverseJoinColumns={#JoinColumn(name="CONTACT_ID")})
private Set<Contact> contacts = new HashSet<Contact>();
//Contact
....
#ManyToOne(targetEntity=UserAccountImpl.class)
#JoinColumn(name="USER_ACCOUNT_ID",nullable=false)
private UserAccount userAccount;
#ManyToMany(targetEntity=GroupImpl.class, mappedBy="contacts")
private Set<Group> groups=new HashSet<Group>();
....
// helper methods from group
public void addContact(Contact contact) {
try{
this.getContacts().add(contact);
contact.getGroups().add(this);
}catch(Exception e) {
}
}
//helper method from group
public void removeContact(Contact contact) {
contact.getGroups().remove(contact);
this.getContacts().remove(contact);
}
//helper method from contact
public void addGroup(Group group) {
try{
this.getGroups().add(group);
group.getContacts().add(this);
} catch(Exception e) {
e.printStackTrace();
}
}
//Helper method from group
public void removeGroup(Group group){
try{
group.getContacts().remove(this);
this.getGroups().remove(group);
} catch(Exception e) {
e.printStackTrace();
}
}
//UserAccount setter from Contact.All the children with many-to-one have the same
/**
* #param userAccount the userAccount to set
*/
public void setUserAccount(UserAccount userAccount) {
this.userAccount = userAccount;
}
I'ld like to pull the UserAccount by its email field which is an unique field in the UserAccount table.
In the UserAccountDAO the method i call to get the UserAccount is getUserAccountByEmail here below.So i expect this method to load all the children collections of the UserAccount namely its Contact collection, group collection.I want it in such a way that when UserAccount is loaded with Contacts collection each of the contact object has its reference with its belonging groups collection if any etc and vice versa.
public UserAccount getUserAccountByEmail(String email) {
// try {
logger.info("inside getUserAccountByEmail");
logger.debug(email);
Session session = (Session) this.getDBSession().getSession();
UserAccount user = (UserAccount) session.createCriteria(this.getPersistentClass())
.setFetchMode("contacts", FetchMode.SELECT) //recently added
.setFetchMode("groups", FetchMode.SELECT) // recently added
.add(Restrictions.eq("email", email))
.uniqueResult();
logger.debug(user);
return user;
// } catch(NonUniqueResultException ne) {
// logger.debug("Exception Occured: getUserAccountByEmail returns more than one result ", ne);
// return null;
// } catch(HibernateException he){
// logger.debug("Exception Occured: Persistence or JDBC exception in method getUserAccountByEmail ",he);
// return null;
// }catch(Exception e) {
// logger.debug("Exception Occured: Exception in method getUserAccountByEmail", e);
// return null;
// }
Since there has to be an UserAccount before any contact and groups, in my unit test when testing the saving of a contact object for which there must be an existing group i do this in order
a create userAccount object ua.
b create group object g1;
c create contact object c1;
d ua.addGroup(g1);
e c1.setUserAccount(ua);
f c1.addGroup(g1);
g uaDao.save(ua); // which saves the group because of the cascade
h cDao.save(c1);
Most of the time i use the session.get() from hibernate to pull c1 by its it id generated by hibernate and do all the assertions which works actually.
but in Integration test when i call getUserAccountByEmail with and without the setFetchMode and it returns the right object but then all the children collections are empty. i've tried the JOIN and the SELECT.the query string changes but then the result set is still the same. So this arises some questions :
1. What should i do to fix this?
2. the helper method works fine but it's on the parent side(i do it in the test).What i've been wondering about is that doing c1.setUserAccount(ua); is enough to create a strong relationship between UserAccount and contact.most of the time there will not be cases where i save the userAccount with contact but yet the helper method that set the association in both side and which is in UserAccount will not been called before i save the contact for a particular userAccount.So i'm little confused about that and suspecting that setting of the association is part of the why something is not working properly.and then calling session.get(UserAccount.class, ua.getID()) i think goes what i want and i'ld like getUserAccountByEmail to do the same.
3. ChssPly76 thinks the mapping has to be rewrite.So i'm willing to let you guide me through this.I really need to know the proper way to do this because we can't lean everything from a good book.So i you think i should change the mapping just show me how.and probable i'm doing things the wrong way without even been aware of that so don't forget i'm still learning java itself.THanks for the advise and remarks and thanks for reading this

I agree with you that it seems likely that the associations between your parent objects and their child collections are not getting persisted properly. I always like to start out by looking at what is in the database to figure out what's going on. After you run your test what do you see in the actual database?
It seems likely that one of two things is happening (using UserAccount as an example):
The items in the child collection are not getting saved to the database at all, in which case you'll be able to see in the database that there are no records associated with your UserAccount. This could be caused by saving the UserAccount object before you've added the child object to the UserAccount's collection.
The items in the child collection are getting saved to the database, but without the needed association to the parent object, in which case you'll see rows for your child items but the join column (ie 'userAccount' will be null). This could be caused by not setting the userAccount() property on the child object.
These are the two scenarios that I've run into where I've seen the problem you describe. Start by taking a look at what goes into your database and see if that leads you farther.