I am dusting off my google app-engine / datastore skills ... and getting stuck on something very simple.
As per the example on the GAE documentation I am trying to update an entity as follows:
// persistence and business logic
PersistenceManager pm = PMF.get().getPersistenceManager();
// get it
NickName n = pm.getObjectById(NickName.class, nicknameId);
// update fields
n.givenName = "new name";
n.nickName = "new nickname";
n.timeStamp = new Date();
// close manager to persist changes
pm.close();
This doesn't work (as in the changes are not persisted, but no errors or anything else)!
At the same time I found that if I create a new entity with the same ID the changes get persisted:
// persistence and business logic
PersistenceManager pm = PMF.get().getPersistenceManager();
NickName n = new NickName("new name", "new nickname", new Date());
// set id
n.id = nicknameId;
pm.makePersistent(n);
pm.close();
I have the feeling I already solved this the 1st time I approached app engine and the data-store.
This is what my entity looks like:
#PersistenceCapable
public class NickName {
public NickName(String name, String nickname, Date timestamp) {
this.givenName = name;
this.nickName = nickname;
this.timeStamp = timestamp;
}
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
public String id;
#Persistent
public String givenName;
#Persistent
public String nickName;
#Persistent
public Date timeStamp;
}
Any help appreciated!
One issue may be that you are setting the fields directly instead of going through the setter methods. I'm fairly certain that JDO works by instrumenting the field setters so that they notify the persistence layer of any changes that occur. It has no way of directly monitoring changes to the backing field values themselves. So maybe try:
n.setGivenName("new name");
n.setNickName("new nickname");
n.setTimeStamp(new Date());
You're able to get away with setting the field directly when you create the object because the makePersistent() call tells the persistence manager that is needs to inspect the field values and save them. Though it's worth noting that setting field values directly like this is generally considered to be poor coding style.
Also, have you tried using the JPA interface instead of the JDO interface? In GAE they should be interchangeable:
EntityManager em = EMF.get();
NickName n = em.find(NickName.class, nicknameId);
n.givenName = "new name";
n.nickName = "new nickname";
n.timeStamp = new Date();
em.merge(n);
em.close();
This gives you an explicit merge() call which should work even with setting the field values directly.
Related
Hi considering the following example:
Resource:
#PUT
#Path("{id}")
public Response update(#PathParam(value = "id") final String id, final Person person) {
final Person person = service.getPerson(id);
final EntityTag etag = new EntityTag(Integer.toString(person.hashCode()));
// If-Match is required
ResponseBuilder builder = request.evaluatePreconditions(etag);
if (builder != null) {
throw new DataHasChangedException("Person data has changed: " + id);
}
service.updatePerson(id, person.getName());
....
}
Service:
public void updatePerson(final String id, final String name) {
final Query<Person> findQuery = morphiaDataStore.createQuery(Person.class).filter("id ==", id);
UpdateOperations<Person> operation = morphiaDataStore.createUpdateOperations(Person.class).set("name", name);
morphiaDataStore.findAndModify(findQuery, operation );
}
Person:
#Entity("person")
public class Person {
#Id
private ObjectId id;
#Version
private Long version;
private String name;
...
}
I do check if the etag provided is the same of the person within the database. However this check is been done on the resource itself. I don't think that this is safe since the update happens after the check and another thread could have gone threw the check in the meantime. How can this be solved correctly? Any example or advise is appreciated.
Morphia already implements optimistic-locking via #Version annotation.
http://mongodb.github.io/morphia/1.3/guides/annotations/#version
#Version marks a field in an entity to control optimistic locking. If the versions change in the database while modifying an entity (including deletes) a ConcurrentModificationException will be thrown. This field will be automatically managed for you – there is no need to set a value and you should not do so. If another name beside the Java field name is desired, a name can be passed to this annotation to change the document’s field name.
I see you have already use the annotation in your example. Make sure the clients include the version of the document as part of the request so you can also pass it to morphia.
Not sure if findAndModify will be able to handle it (I would think it does). but at least I'm sure save does handle it.
Assuming the object person contains the new name and version that the client was looking at, you can do directly something like this to update the record:
morphiaDataStore.save(person);
If there was another save before this client could pick it up the versions will no longer match and a ConcurrentModificationException will be issued with this message:
Entity of class %s (id='%s',version='%d') was concurrently updated
I have an issue i'm struggling with for some time now. Im trying to implement a news feed feature in my app using GAE cloud endpoints and java. The common concept is of followers and followees, where an action of a followee can be seen by his followers. A new follower should also see his followees past actions, not only from the time he started following.
I made a few tries with the following components. Each try worked great but was lacking something:
On each user action i added a 'log' entity into the datastore with the user id included. When a user was displaying his news feed i just queried for all those entities by their user ids according to the user's followees list. Everything was fine until i realized that a 'IN' query cannot be cursored. So this option was gone.
On this try, which is also the current state of the application, im using the Search API. Upon every user action im not storing a 'log' entity into the datastore anymore but a document into a search index. Complex queries can be cursored here and the world is smiling again. But... im not too sure that, billing wise, this is a smart descision. It seems that the costs of searching/adding/deleting documents along side the documented daily limitations is making the whole thing a bit too sketchy.
The next try should be Prospective Search API. From what i'm reading in the documents it seems the right component to pick for that purpose. Unfortunately, the documentation is really poor and give very little examples. Also the billing information is unclear.
So im asking for the advice of the stackoverflow community. Can you please advise me about this matter ? and if Prospective Search is the right option to choose, can you please provide some clear sample java code that uses cloud endpoints?
EDIT : Just to emphasize the main design requirement here - The news feed feature need to have the ability to fetch sorted followees actions using a cursor (in order avoid querying the whole batch).
Use a pull-aggregate-per-follower model: periodically (or on demand) query all followees actions once and then cache them inside a dedicated per-follower entity. Remember the time of last query, so next time you just query from that point on (assuming actions can not be added/changed to the past times).
This will give you the following features (and limitations):
If query is on-demand, than you will not need to query for users that are inactive.
Since the query is "new-only" (looks for new actions only), it would cost you nothing if it returned zero results.
You will only query each followee actions per follower once. After that all recent actions would be cached inside one entity and loaded into memory with one get. This should be a substantial cost and time saving.
You could sort/filter actions in memory any way you wish.
Limitations:
Entities have a 1MB limit, so there is a max no of actions that you can cache in one entity. So you will either need to limit caching of recent actions per user or spread out action caching over multiple entities.
You will need to use IN query over followees (max 30) and also use parallel threads to achieve decent performance. This could easily hit 3-5 seconds when querying over 1000-2000 followees. Also, you could easily hit RPC limit (aka max concurrent API calls) per instance when serving multiple users at the same time.
I hope I understand the question correctly - you want to implement a news feed into your application and allow users to follow each other. The new followers need to be able to see the users actions. I am sure there are multiple other ways of solving this problem, but I will attempt to help you out by providing a solution that makes use of JAVA JDO to access the datastore.
I would first design the entity relationships in JDO as follows:
1 User to many actions.
1 User to many followers (User).
1 User to many following (User).
Here are simple JDO classes:
User Class:
#PersistenceCapable(identityType=IdentityType.APPLICATION)
public class User {
#PrimaryKey
#Persistent(valueStrategy=IdGeneratorStrategy.IDENTITY)
private Key key;
#Persistent
private String userId; // Google unique user ID, could also store user email.
#Persistent
private Set<Key> actions;
#Persistent
private Set<Key> followers;
#Persistent
private List<Key> following;
public User(Key key, String userId) {
this.key = key;
this.userId = userId;
this.actions = new HashSet<Key>();
this.followers = new HashSet<Key>();
this.following = new HashSet<Key>();
}
public Key getKey() {
return this.key;
}
public void addAction(Key actionKey) {
this.actions.add(actionKey);
}
public void addActions(Set<Key> actionKeys) {
this.actions.addAll(actionKeys);
}
public Set<Key> getActions() {
return this.actions;
}
public void addFollower(Key followerKey) {
this.followers.add(followerKey);
}
public void addFollowers(Set<Key> followerKeys) {
this.followers.addAll(followerKeys);
}
public Set<Key> getFollowers() {
return this.followers;
}
public void addFollowing(Key followingKey) {
this.following.add(followingKey);
}
public void addAllFollowing(Set<Key> followingKeys) {
this.following.addAll(followingKeys);
}
public Set<Key> getFollowing() {
return this.following;
}
}
Action Class:
#PersistenceCapable(identityType=IdentityType.APPLICATION)
public class Action {
#PrimaryKey
#Persistent(valueStrategy=IdGeneratorStrategy.IDENTITY)
private Key key;
#Persistent
Date date;
#Persistent
private String title;
public Action(Key key, String title) {
this.key = key;
this.title = title;
this.date = new Date(); // date of creation (now).
}
public Key getKey() {
return this.key;
}
public void setTitle(String title) {
this.title = title;
}
public String getTitle() {
return this.title;
}
}
The Action class makes use of a Date property, you can refer to the documentation for applicable data types in the datastore. When an action is created, a Date object is allocated and initialized so that it represents the time at which it was allocated, measured to the nearest millisecond.
In my example above I linked the entities by their Keys, you could instead link them by their classes as follows:
List<Action> actions;
The relationship in my example is one of an unowned one-to-many relationship, perhaps it should be owned one-to-many. More information here for your to take a look and perhaps decide which would be best for your solution.
Once the relationships have been defined, you can create your endpoint classes around the JDO model classes. This will create basic api methods. You might want to change the endpoint class methods to suit your needs, for example change the way an action is created. A basic example would be to create the key from the actions title as follows (ActionEnpoint.java):
...
#ApiMethod(name = "insertAction")
public Action insertAction( #Named("title") String title ) {
PersistenceManager pm = getPersistenceManager();
Key key = KeyFactory.createKey(Action.class.getSimpleName(), title);
Action action = null;
try {
action = new Action(key, title);
pm.makePersistent(action);
} finally {
pm.close();
}
return action;
}
...
If you want to, you can add a method to your UserEndpoint class to query the datastore and return all actions belonging to that user and per date using the datastore query objects.
You need to add a method to your UserEndpoint class that allows you to add an action to that user, here is a simple example:
...
#ApiMethod(name = "addActionToUser")
public Achiever addActionToUser(
#Named("userId") String userId,
#Named("actionTitle") String actionTitle) {
PersistenceManager pm = getPersistenceManager();
Key userKey = KeyFactory.createKey(User.class.getSimpleName(), userId);
Key actionKey = KeyFactory.createKey(Action.class.getSimpleName(), actionTitle);
User user = null;
try {
user = (User) pm.getObjectById(User.class, userKey);
user.addAction(actionKey);
pm.makePersistent(user);
} catch (Exception e) {
}
return user;
}
...
Once all of the above is complete you can easily get the list of actions per user by calling the getUser method in your UserEndpoint class, which returns a User object. You can then call [ReturnedUserObject].getActions(). A new follower can now view all of the "followees" actions by just calling the api method to get that "followees" object and get his/her actions. You can then just sort the actions by date or however you envision it.
I hope I understood your question correctly, I was unsure about the first component you mentioned, but it seemed as though you got your relationships mixed up. I hope this solution points you in the right direction at least :).
If you need any additional help or clarification, or my answer was completely off point to what you were looking for then please let me know.
Kind regards,
Miki
I'm having a problem storing an object in the datastore. I have an object, MyObject, that I'm trying to store but when the code is executed nothing happens. I go to look at the datastore dashboard and MyObject isn't there. No exceptions are thrown and there are no errors.
Here's my object
#PersistenceCapable(identityType = IdentityType.APPLICATION)
public class MyObject{
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;
#Persistent
String name;
#Persistent
String beta;
#Persistent
double loc;
public MyObject(String name1){
name = name1;
}
//getters and setters
}
and here's the code to store the object
public static void saveMyObject(MyObject a)throws Exception{
PersistenceManager pm = PMF.get().getPersistenceManager();
try{
pm.makePersistent(a);
}
catch(Exception e){
throw e;
}
finally{
pm.close();
}
}
Can anyone see what I'm missing?
It looks like you are using JDO, so you might want either to add a JDO tag or else mention that somewhere....
I would replace
try
{
pm.makePersistent(a);
}
with
try
{
MyObject myObj = new MyObject(a.getName()); // or whatever the getter is
myObj.setField2(a.getField2()); // Copy 1 data member from a
... // Make a MyObject.copy(...) method?
pm.makePersistent(myObj);
}
The key thing is that JDO uses enhancement: magic bytecode that is inserted after main Java compilation. I manipulate my persistent entity objects within the lifecycle ("scope") of enhancement to get JDO to work.
I also use transactions for writing (I don't know your JDO auto-transaction setting(s)). I always use transactions when creating and persisting a new persistent entity. You might want to try that if the change above does not work.
I am trying to learn JDO (and at the same time its GAE and Spring intricacies) by creating a small web app, and am having trouble getting updated domain objects to persist back to the database. I initially grab the entity from the DB and detach it so that I can show it to the user and allow them to change it. Once the user has made the changes and posts the form back to the app, I again grab the entity from the DB (detached), update its properties, and then call a pm.makePersistent(). The abbreviated code is as follows:
User Domain Object:
#PersistenceCapable(detachable="true")
public class User extends BaseEntity {
#Persistent
private String firstName = "";
#Persistent
private String middleInitial = "";
#Persistent
private String lastName = "";
}
DAO Read Method:
public User read(Key key) throws DataException {
PersistenceManager pm = PMF.get().getPersistenceManager();
User pkg, detached = null;
try {
pkg = (User) pm.getObjectById(User.class, key);
detached = pm.detachCopy(pkg);
detached.setIsAlreadyInDB(true);
}
catch (Exception e) {
throw new DataException("An error occured trying to read the User object. Details:\n" + e.getMessage());
}
finally {
pm.close();
}
return detached;
}
DAO Update Method:
private void update(User pkg) throws DataException {
PersistenceManager pm = PMF.get().getPersistenceManager();
Transaction tx = pm.currentTransaction();
try {
tx.begin();
pm.makePersistent(pkg);
tx.commit();
}
finally {
if (tx.isActive()) tx.rollback();
pm.close();
}
}
Now when I get down into the update method, I've proven to myself that I'm working with just the same object from my read via inspecting its hashCode(), I've changed a value using the domain object's setter method, I've even printed the changed value to the console to make sure it's getting done, and JDOHelper.isDirty() still returns false, and therefore none of the changes get persisted back to the database.
Any thoughts on what I'm missing or if I'm approaching this from the wrong angle? Thank you for helping out a JDO beginner!
JDOHelper.isDirty is for managed objects. A detached object is not managed. DataNucleus provides a helper method of its own to get the dirty fields while detached since the logic is implementation-specific
String[] dirtyFieldNames = NucleusJDOHelper.getDetachedObjectDirtyFields(obj, pm);
I am using the Goole app engine datastore with Java and trying to load an Object with a List of Enums. Every time I load the object the List is null. The object is
#PersistenceCapable(identityType = IdentityType.APPLICATION)
public class ObjectToSave {
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long id;
#Persistent
private List<AnEnum> anEnumList;
//public getters and setters
}
The enum is simple
public enum AnEnum {
VALUE_1,
VALUE_2;
}
The code to save it is
ObjectToSave objectToSave = new ObjectToSave();
List<AnEnum> anEnumList = new ArrayList<AnEnum>();
anEnumList.add(AnEnum.VALUE_1);
objectToSave.setAnEnumList(anEnumList);
PersistenceManager pm = pmfInstance.getPersistenceManager();
try {
pm.makePersistent(objectToSave);
} finally {
pm.close();
}
The code to load it is
PersistenceManager pm = pmfInstance.getPersistenceManager();
try {
Key key = KeyFactory.createKey(ObjectToSave.class.getSimpleName(), id);
ObjectToSave objectToSave = pm.getObjectById(ObjectToSave.class, key);
} finally {
pm.close();
}
I can view the data in the datastore using http://localhost:8080/_ah/admin and can see my List has been saved but it is not there when the object is loaded.
I created my project with the Eclipse plugin and haven't made any changes to the datastore settings as far as I know. So why id my Enum list null?
Yes but your List field is not in the default fetch group at loading so hence is not loaded.
Read JDO Fetch Groups. You could add it to the DFG, or enable a custom fetch group, or just "touch" the field before closing the PM.
--Andy (DataNucleus)
How are you creating an instance of ObjectToSave? The default value of all instance variable reference types is null, so unless you have (additional) code to create an instance of List<AnEnum> and assign it to anEnumList, null would be expected.