I have a GAE project written in Java and I have some thoughts about the HRD and a problem that I'm not sure how to solve.
Basically I have users in my system. A user consists of a userid, a username, an email and a password. Each time I create a new user, I want to check that there isn't already a user with the same userid (should never happen), username or email.
The userid is the key, so I think that doing a get with this will be consistent. However, when I do a query (and use a filter) to find possible users with the same username or email, I can't be sure that the results are consistent. So if someone has created a user with the same username or email a couple of seconds ago, I might not find it with my query. I understand that ancestors are used to work around this problem, but what if I don't have an ancestor to use for the query? The user does not have a parent.
I'd be happy to hear your thoughts on this, and what is considered to be best practice in situations like these. I'm using Objectify for GAE if that changes anything.
I wouldn't recommend using email or any other natural key for your User entity. Users change their email addresses and you don't want to end up rewriting all the foreign key references in your database whenever someone changes their email.
Here's a short blurb on how I solve this issue:
https://groups.google.com/d/msg/google-appengine/NdUAY0crVjg/3fJX3Gn3cOYJ
Create a separate EmailLookup entity whose #Id is the normalized form of an email address (I just lowercase everything - technically incorrect but saves a lot of pain when users accidentally capitalize Joe#example.com). My EmailLookup looks like this:
#Entity(name="Email")
public class EmailLookup {
/** Use this method to normalize email addresses for lookup */
public static String normalize(String email) {
return email.toLowerCase();
}
#Id String email;
#Index long personId;
public EmailLookup(String email, long personId) {
this.email = normalize(email);
this.personId = personId;
}
}
There is also a (not-normalized) email field in my User entity, which I use when sending outbound emails (preserve case just in case it matters for someone). When someone creates an account with a particular email, I load/create the EmailLookup and the User entities by key in a XG transaction. This guarantees that any individual email address will be unique.
The same strategy applies for any other kind of unique value; facebook id, username, etc.
A way around the HRD's eventual consistency, is to use get instead of query. To be able to do this is you need to generate natural IDs, e.g. generate IDs that consists of data you receive in request: email and username.
Since get in HRD has strong consistency, you will be able to reliably check if user already exists.
For example a readable natural ID would be:
String naturalUserId = userEmail + "-" + userName;
Note: in practice emails are unique. So this is a good natural ID on it's own. No need to add a made-up username to it.
You may also enable cross-group transactions (see https://developers.google.com/appengine/docs/java/datastore/overview#Cross_Group_Transactions) and then in one transaction look for the user and create a new one, if that helps.
Recommend avoiding an indexed field and query unless you have other uses for it. Here is what I have done before (Python) using key_name (since entity ids need to be ints). Easy to use either the key_name or id for other entities that need to link to user:
username = self.request.get('username')
usernameLower = username.lower()
rec = user.get_by_key_name(usernameLower)
if rec is None:
U = user(
key_name = usernameLower,
username = username,
etc...)
U.put()
else:
self.response.out.write(yourMessageHere)
Related
I've been trying to get the email of the user that submitted a response to a short answer assignment. In all of my other experiences using the Google Classroom API, the "userId" parameter has been the student's email address, but when I call getUserId on the StudentSubmission object, I get a weird string of numbers. How do I get the email associated with the submitted response?
Here's my code for reference:
ListStudentSubmissionsResponse submissionResponse = service.courses().courseWork().studentSubmissions().list(courseId, assignmentId).execute();
List<StudentSubmission> submissions = submissionResponse.getStudentSubmissions();
for (StudentSubmission sub : submissions)
{
System.out.println(sub.getId() + "\t" + sub.getUserId() + "\t" + sub.getState());
}
And this is the response that I am getting:
Cg4I2vWq_8IDEIWck4DDAw 108878473486432178050 CREATED
Does anyone know what is going on here?
There is published documentation on this API... no need to ask here: https://developers.google.com/classroom/reference/rest
More specifically, there are userProfile APIs that differentiate an "id" from an "emailAddress".
https://developers.google.com/classroom/reference/rest/v1/userProfiles
Now that said, there are no guarantees according to the JSON schema, being just a "String" type ... as to what the value might be.
"emailAddress" should be just that but "id" could be whatever the local system uses to identify principals - which is almost definitely NOT going to be the email address. It will instead be some global identifier for whatever principal management system is being used.
Don't not read the docs: retrieve_student_responses
Students are identified by the unique ID or email address of the user, as returned by the Google Admin SDK.
So there's clearly no guarantee it will return the email..
If you read the docs you can find how to properly retrieve the email for a student id: retrieve_a_users_profile
To retrieve the abridged profile, including ID and name, for a user, call userProfiles.get() with the user's ID, email, or "me" for the requesting user.
To retrieve the emailAddress field, you must include the classroom.profile.emails scope.
I was always taught to use IDs in my code to refer to records into the database.
But let's take the case we have same roles in the table Role. Now I want to query only the records related to the role Player:
ID ROLE
1 Admin
2 Organizer
3 Player
I don't know in my code the ID of Player, but I want to retrieve all the players, so with Hibernate I wrote:
String queryString = "from User u where u.role.role = ?";
Query queryObject = getSession().createQuery(queryString);
queryObject.setParameter(0, "player");
return queryObject.list();
As you can see I wrote "player" in the code. I think this is not the best way and I should use an ID instead. But I don't know the ID and it may change depending on the server on which I run the application. A second problem with my solution is that "player" can be capitalized into the database and this may be changed over time.
So, what should be the solution to all these problems? Is there any way to use the ID instead? Or any other way to improve this code?
In this case it seems that role should be an enum and your query would look something like:
queryObject.setParameter(0, Role.PLAYER);
Also, you might take a look at the criteria API which will help you create more type-safe queries that are more robust vs. refactoring.
You should create a enum class like this.
public enum Role {
Admin(1),
Organizer(2),
Player(3);
}
And change your code to
String queryString = "from User u where u.id= ?";
Query queryObject = getSession().createQuery(queryString);
queryObject .setParameter(0, Role.PLAYER);
return queryObject.list();
You can do using create a mapping table like ,
UserRoleMapping
ID - Incremental,
UserId - Refers to user table,
RoleId - Refers to role table
As one user can have more than one role so it will satisfy that thing too.
to Get the roles using query
select role.id from userrolemapping urm innerjoin on user u.id = urm.id where u.id = ?
using IDs or string/vachar etc. lookup is all dependent on the data that you have in the database. Some organization keep the id static and some keep the name/description static. So, make sure you have good understanding of your DB data and what will stay static . However, if the role is static you can use HQL ignore case like the example I provided for you below (I'm not adding information about the ID static path because others have already provided information about it for and don't want to duplicate it ).
--note you can take the percentages out if you only want "player"
String queryString = "from User u where lower( u.role.role ) like lower('%"+ ? +"%')";
Query queryObject = getSession().createQuery(queryString);
queryObject.setParameter(0, "player");
return queryObject.list();
My DB schema is
INTEGER id
TIME_STAMP last_updated // indexed
VARCHAR url // indexed
// Place attributes
VARCHAR name // indexed
VARCHAR address // indexed
VARCHAR phone // indexed
INTEGER score
Out of name, address, and phone fields, one of them must be non-empty, and the other two empty.
Valid examples:
{name="Pizza Hut", address=null, phone=null, score=40}
Invalid examples:
{name="Pizza Hut", address="some address", phone=null, score=40}
I'm trying to write this function.
// return a map/dictionary of String keys and Integer scores of latest data in DB that matches given url, and is not older than last_updated.
// example: {name=40, address=50, phone=66}
public Map getPlaceAnalysis(String name_, String address_, String phone_, String url, DateTime last_updated_);
Scaling assumptions:
This function will be getting called very frequently.
DB has millions of rows
name, phone, address may not be my only fields. I may have 20+ attributes about a place.
There are two approaches:
3 (or the number of attributes about a Place stored in the DB) queries:
SELECT score FROM db_name WHERE name=name_ AND url = url_ AND last_updated > last_updated_
SELECT score FROM db_name WHERE phone=phone_ AND url = url_ AND last_updated > last_updated_
SELECT score FROM db_name WHERE address=address_ AND url = url_ AND last_updated > last_updated_
After each query, I can select the latest row and write the score into my map/dictionary to be returned for the appropriate key.
1 query:
SELECT name, phone, address, score FROM db_name WHERE (name=name_ OR phone=phone_ OR address=address_ ) AND url = url_ AND last_updated > last_updated_
I then loop through the query results and figure out the most updated row for name, phone, and address, and return the appropriate map/dictionary.
Does it matter which approach I take? I'm guessing the second one is better, but I have very little experience with databases. Sorry if any explanation is unclear. I'd imagine this is a pretty common problem but couldn't figure out how to search for this problem.
When running a query to a remote database there is a considerable amount of time spent in establishing the connection and stuff like that, that are more related to the communication with the database and not so much with the query itself. From this point of view you want to minimize the number of queries you issue to the database.
As explained in MYSQL reference manual for Multiple-Column Indexes, it seems that it would be to your benefit to create a composite index on the three fields you are using in your query. This index would also be used when you search for name only, name and address and all the three fields together (see what it says about leftmost prefixes) so pay attention to the order you are going to declare the index.
The idea of the hash of the three fields seems also interesting, if it can be used in your case.
Finally you should read about Understanding the Query Execution Plan and/or have a database administrator to help you with that.
The key thing to understand when talking about query performance is to design a repeatable test case (i.e. with the same set of data, a repeatable database load etc) and then try your different approaches. And always watch out for differences between the environment you test and your production environment.
I hope that helps.
In my project, I send scheduled mails to all registered users. Up to now, there were no restrictions, so I just got all the Users from the database and sent the mails without any security checks. Now, I have to make sure, that only users with a certain role receive those mails.
The only thing I found so far are several ways to check the roles of the currently logged in user. But I need to check the roles of a whole list of users, that are currently not logged in.
I can't figure out how to access these roles. I don't even know if this can be done with spring security at all. So, does anyone know about this kind of problem?
Thanks for your help!
I think that you did not understand what the 'role' means for a user. Usually the role is implemented in the database part of the application. You may have the User table with attributes (id, name, sname, ...., role) (1st case) or you may have it as a separate table ex 'Roles' with attributes (id, name, desc) and then have a FK in your User table that will link to the Roles table (2nd case).
So, when you want to send an email to all those with role = admin then you do it like this:
sql = "select * from users where role = 'admin'"; //First case
sql = "select * from users , roles where users.role_id = roles.id and roles.desc = 'admin' "; //second case
As you have guessed the implementation of this, has nothing to do with spring or whatever security...
How do I select from a table by another unique column other than id
im looking a replacement for
String fbid=<some facebookId>
User user = User.findBy("facebookId",fbid).first();
I'm using Java + Ebean and not scala!
thanks
User user = User.find.where().eq("facebookId", fbid).findUnique();
Of course you can create a 'shortcut' by adding a method into your User model:
public static User findByFacebookId(String fbid){
return find.where().eq("facebookId", fbid).findUnique();
}
and using it in controller:
User user = User.findByFacebookId(fbid);
It's pretty good solution as you can keep your controllers' code clean and use custom finders as sophisticated as required at the case.