We Keep Coding

Modelling data and accessing it using the DAO pattern

I'm creating a very simple application in Java that will be storing questions in an embedded Derby database. I've decided to use the DAO pattern for accessing the data in the database. I cannot make use of an ORM for this project.
A question will have data that I would normally model using a many to one relationship in a relational database. An example of this data would be:
A question will have one category. One category will have multiple questions.
A question will have a score of 1000, 2000 or 3000. A score will have many questions.
With the above in mind, I would create three tables (brackets indicate columns):
Question (id, question, scoreId, categoryId)
Score (id, score)
Category (id, category)
My first question is:
Would modelling my data across three tables like I suggest above be bad practice/the wrong way to go about this? Is there any benefit in storing score and category in separate tables? Or would it be better to combine them into the Question table? A many to one relationship that links to a table with a single column (with the exception of id) seems redundant to me, as instead of storing an id referencing the Score/Category table, we can simply store the value of the category/score (since the category/score table does not store any additional information).
My second question is:
If modelling my data across separate tables is the correct approach, then how would I access the data using the DAO pattern? My confusion comes from the following:
I would create a DAO to populate a Question model object that would look a little something like this:
public class Question {
String question;
String category;
Integer score;
}
I would create a concrete implementation of the DAO interface like this:
public class QuestionAccessObject implements QuestionDao {
private static final String TABLE_1 = "QUESTION";
private static final String TABLE_2 = "SCORE";
private static final String TABLE_3 = "CATEGORY";
#Override
public List<Question> getAllQuestions() {
List<Question> questions = new ArrayList<>();
//Run a query with joins across the three tables and iterate over the result to populate the list
return questions;
}
}
Shouldn't each DAO object only be concerned with a single table in the database? My approach listed above doesn't seem like the most correct way to go about this. Seperate tables would also make inserting data into the database very messy (I don't understand how I could take clean approach using the DAO pattern and multiple tables). Creating a DAO for the Score and Category tables just wouldn't really make sense.. (and if I did this, how would I populate my model?)

Would modelling my data across three tables like I suggest above be bad practice/the wrong way to go about this? Is there any benefit in storing score and category in separate tables....?
It's a matter of discussion. In case of score I rather stick this information with the question. On the other hand, the category would be in the separated table since more of the question would share the same category, so it makes a perfect sense.
Shouldn't each DAO object only be concerned with a single table in the database?
Yes, DAO, an object should be concerned with a single source of data - as you say. I would certainly try to avoid any ComplexDao since those classes tend to get more complex and the number of methods increases over the time.
There exist a service layer to combine those results together and provide an output to the controller using the very same service.

Modeling the data across separate tables is A correct approach (not necessarily the best).
Separating tables helps database normalization: https://en.wikipedia.org/wiki/Database_normalization.
One could argue that the DAO pattern implies that each DAO object is concerned with a single entity . Similar to how ORMs work, an entity could easily reference other entities.
When you query for a question you could also just return the category and score ids inside the question object and force the library user to fetch the score value and category value (lazy fetch) using those id values with their respective DAOs (score and category).
So I believe that what you're doing seems fine
Hope this helps

How to build a relational Java object model over a not-that-relational data model?

I am currently working on several solutions to redesign a web application based on a database that is shared between this application and other running services.
The actual application uses NodeJS, and basically processes data after having exctracted too much information from a table in the database. I think this is wrong, because we should only ask the database for the data we need, and limit processing on the server as these processes imply long loading time for end users.
So I thought I would build a Java API instead, that would use objects that can be easily used by the front part of the application, and use Hibernate or an equivalent component to extract necessary information from database.
But the problem is that the database over which I would build this API, while supposed to be relational (it's postgreSQL), is actually composed of tables that do not communicate between each other : there are no joins between tables, hence no cascade on modifiying related objects. Furthermore, the tables do not correspond directly to objects used in the application. They are more like "super objects" that would contain other sub-objects in a Java model.
Problem is, I cannot recreate a "cleaner" database as it is used by other services (which is probably why it is so weirdly organised, by the way). I should use it as it is, and map the redesigned application to it. I could be allowed to redesign some tables to make them "more relational", but this should have no implications for the other services. That would imply to drastically rewrite some SQL functions, and I am not sure that it is a very efficient solution. That being say, I am not very experienced on writing SQL functions, so maybe that could be a solution in the end.
So, to put it all in a nutshell, what could I do to build an object oriented model using Java, or any other technology that you could think of, to redesign properly the application, without throwing the database or modifying it drastically ?
Any suggestion or lead of research would be greatly appreciated. Also, please tell me if I am not being precise enough, and I will try to improve my question.

Use JPA to map your database records into entities.
This wiki page gives quite a few samples of advanced uses :
multiple tables for an entity :
#Entity
#Table(name="EMPLOYEE")
#SecondaryTable(name="EMP_DATA",
pkJoinColumns = #PrimaryKeyJoinColumn(name="EMP_ID", referencedColumnName="ID")
)
public class Employee {
...
#Column(name="YEAR_OF_SERV", table="EMP_DATA")
private int yearsOfService;
#OneToOne
#JoinColumn(name="MGR_ID", table="EMP_DATA", referencedColumnName="ID")
private Employee manager;
...
}
multiple kind of entities in single table (this sample is mine, there was no code on the wiki ; let's hope I didn't mess it up !) :
#Inheritance
#DiscriminatorColumn(name="DISCRIMINATING_COLUMN")
#Table(name="SOME_TABLE")
public class Generic { ... }
// only maps records from SOME_TABLE where DISCRIMINATING_COLUMN=SOME_VALUE
#Entity
#DiscriminatorValue("SOME_VALUE")
public class firstSpecificType { ... }
// only maps records from SOME_TABLE where DISCRIMINATING_COLUMN=OTHER_VALUE
#Entity
#DiscriminatorValue("OTHER_VALUE")
public class secondSpecificType { ... }
entities from proc calls :
// This stored procedure returns a result set and has one input parameter.
#NamedStoredProcedureQuery(
name = "ReadAddressById",
resultClasses = Address.class,
procedureName = "READ_ADDRESS",
parameters = {
#StoredProcedureParameter(mode=javax.persistence.ParameterMode.IN, name="P_ADDRESS_ID", type=Long.class)
}
)
#Entity
public class Address {
...
}
and many others.

JPA setting referenced property without retrieving it. Best practices

Let's assume I have Entity that have nested Entity inside it.
For example (please, ignore missing annotations, getters/setters, etc):
#Entity
class User {
private String userId;
private Set<UserOperation> userOperations;
}
#Entity
class UserOperation {
private String someString;
// This is nested referenced entity
private User user;
}
Let's assume that I want to insert new UserOperation and all that I have is userId.
Can I do something like:
// We just create new user. There is no interaction to get existing from DB. Totally only 1 insert
User user = new User();
user.setId("someId")
UserOperation uOp = new UserOperation();
uOp.setUser(user);
uOp.setSomeString("just op");
em.persist(uOp);
Or I should go that way only:
// We retrieve existing user. There is interaction to get it from DB. Totally 1 select and 1 insert
User user = em.find("someId")
UserOperation uOp = new UserOperation();
uOp.setUser(user);
uOp.setSomeString("just op");
em.persist(uOp);
What is the right way of doing it?
Because from DB perspective userOperation table just have String user reference, so ID should be enough. Java requires an object.
When call "new User" I would like to avoid, properties of existing user be flushed (as they are all not set) or JPA trying to insert new user and operation failing due to primary key violation.
Some examples are welcomed.

For your use case, there is particularly method getReference() in EntityManager. It gives you an entity object for id, but does not access DB to create it. Therefore the best solution is a slightly modified 2nd solution of yours:
// We retrieve a stub user for given id. There is no interaction with DB
User user = em.getReference("someId", User.class);
UserOperation uOp = new UserOperation();
uOp.setUser(user);
uOp.setSomeString("just op");
em.persist(uOp);
Explanation:
getReference() has the same logical meaning as find(), with the exception that it does call DB. The consequence is that it does not check if there is a row in DB table with the given id, and that the object you get does not yet contain the data. However, the object is fully capable to load additinal data when get method is called. Therefore the object is fully usable even if retrieved by getReference() - in fact it works the same way as lazy loading.
A side note to your first solution:
The first solution would not work, as it would create a new entity user and then it would fail either when storing the entity to DB if it is cascaded (persist always calls insert and it would try to insert user with the same ID as exists in DB), or it would fail that UserOperation is to be persisted while user is not. In order to fix this solution, you would need to call em.merge(user) before you call em.persist(userOperation). But again, this would call a select to DB in the same way as em.find().

The best way to do this is using the second example. We should always try to use the actual object direct from db. Working with only the db reference will be way worse to mantain.
Now speaking specifically about Hibernate, it makes even more sense to work with whole objects, especially because of Hibernate's cascade, that can and will (if cascade is set) update the child entities of the one you are persisting to database.
Well, I have to admit that always fetching objects from database may cause some performance issues especially after the database gets a huge amount of data, so it's always important to implement nice and coherent model entities, and also keep in track of database hits from your application, and try to keep it the less possible queries being generated.
As for example, your own example (the second) is clean and easy to understand, I would stick with this approach, since it's really simple.
Hope it can solve your questons :)

What goes into Spring Security's ACL_OBJECT_IDENTITY table?

I'm attempting to integrate Spring Security with ACL support into an existing GWT application, and if I click another unhelpful link I swear will need a new mouse and keyboard. I've worked through what will be needed to use Spring to authenticate against Active Directory via LDAP, and I've worked out how we can assign custom permissions based on AD attributes (i.e. group membership), and I've even worked out how to perform a custom check against permissions (a true bitmask operation) using a custom ACL schema. What I haven't figured out is just what goes into the ACL tables.
ACL_SID
id:bigint (pk)
principal:boolean (ak)
sid:varchar (ak)
This table is pretty self-explanatory; we'll be using non-principal entries here only.
ACL_CLASS
id:bigint (pk)
class:varchar (ak)
This table is also pretty self-explanatory. As I understand it, we simply create an entry for every class/interface we wish to secure.
ACL_ENTRY
id:bigint (pk)
acl_object_identity:bigint (fak)
ace_order:int (ak)
sid:bigint (fk)
mask:bigint
granting:boolean
audit_success:boolean
audit_failure:boolean
This table is also mostly self-explanatory; we've customized the schema with a bigint/long in the mask field, but the question stems from just what acl_object_identity is referencing. Obviously, it points to the field in ACL_OBJECT_IDENTITY, but...
ACL_OBJECT_IDENTITY
id:bigint (pk)
object_id_class:bigint (fak)
object_id_identity:bigint (ak)
parent_object_identity:bigint (fk)
owner_sid:bigint (fk)
entries_inheriting:boolean
The object_id_identity is, what? A method? How is it referenced by way of Spring Security's annotations?
MyClass.java
#PreAuthorize("hasPermission(#someInput, 'READ')")
public boolean myMethod(String someInput) {
return true;
}
Presumably, #someInput refers somehow to the ACL_OBJECT_IDENTITY.object_id_identity field, but how?

First you need some kind of domain object class to store your data. This class must have a getId() method. For example:
public class DomainObject {
private Long id;
private String data;
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getData() {
return data;
}
public void setData(String data) {
this.data = data;
}
}
Change your secured method to:
#PreAuthorize("hasPermission(#someInput, 'READ')")
public boolean myMethod(DomainObject someInput) {
return true;
}
Now, someInput.getId() is the same as ACL_OBJECT_IDENTITY.object_id_identity.

have a look at the Grails Spring Security ACL Plugin Documentation.
It explains the domain classes used for the specific plugin, but it might help you. Look for AclObjectIdentity and AbstractAclObjectIdentity.
The ACL_OBJECT_IDENTITY.object_id_identity field is the instance id and uniquely identify the instance (together with object_id_class)
In your example, 'someInput' would be that instance.
this piece of code is taken from the beforementioned documentation:
#PreAuthorize("hasPermission(#report, delete) or hasPermission(#report, admin)")
void deleteReport(Report report) {
report.delete()
}
It explains itself immo. The report is deleted if the currently authenticated user has permission 'admin' or 'delete' for the given Report instance. Report is someInput in your code snippet.
Hope this helps.

The question seems to suggest a slight misunderstanding of Spring ACL. In the question, we are asked how to register an ACL for a String object, so that this method protection will work:
#PreAuthorize("hasPermission(#someInput, 'READ')")
public boolean myMethod(String someInput) {
return true;
}
As mentioned in this answer, it doesn't really make sense to protect a String object. When you think about it, this makes sense. Loosely speaking, we can divide all objects into two categories:
Objects that are persisted to the DB - let's call them entities
Objects that are not persisted to the DB - let's call them transients
In any real-life use case I can think of, it only makes sense to protect access to an entity, not a transient; I'll argue for this case in general later. To start with though, let's stick with a use case related to that of the question to see why protection of a transient probably isn't what's desired in this case.
Use Case
It's not 100% clear what the use case is in the question e.g. what someInput represents. But I'm assuming the use case is similar to the following. Suppose there are BankAccount entities and some operation on those entities e.g. readAccount. Only users who have read access to a BankAccount should be allowed to call readAccount. Now, BankAccount entities may be uniquely identifiable by their accountNumber, which is of type String. So we may be mistakenly led to try something like this, which is similar to the code in the question:
#PreAuthorize("hasPermission(#accountNumber, 'READ')")
public Account readAccount(String accountNumber) {
//CODE THAT RETRIEVES THE ENTITY FROM THE DB AND RETURNS IT
}
OK, it's not a bad assumption to make. I suppose the idea in our minds at this stage is that Spring ACL stores a table of account numbers, and for each account number, a list of people who have READ access to it. The problem is, Spring ACL doesn't work like that. As mentioned in this answer, Spring ACL identifies an objects in the following way:
What is the class of the object? In this case, it would be java.lang.String
What is the ID of the object? In this case, Spring ACL requires that the object needs a getId() method. Luckily, if you're using Hibernate, all your entities will have this by default, so you don't need to do anything extra to implement it. But what about String? Well, this doesn't have a getId() method. So Spring ACL won't be able to register an ACL for it and you won't be able to set up any permissions for Strings.
When you think about it, it actually makes sense that Spring ACL is designed this way. The getId() method allows us to associate the persisted ACL permissions entries with the persisted entities. This is the typical use case. So in the above example, what we're really trying to do is restrict access to Account objects, not account numbers. In that case, we have two options: preauthorize or postuathorize.
With pre-authorize, we'd need the fully-qualified path to the Account class. So let's say it's in the package X.Y, we'd have:
#PreAuthorize("hasPermission(#accountId, 'X.Y.Account', 'READ')")
public Account readAccount(Long accountId) {
//CODE THAT RETRIEVES THE ENTITY FROM THE DB AND RETURNS IT
}
Notice that in the above, we are using the ID, not the account number, to identify the account. This is the only way you are allowed identify an entity with Spring ACL, because the getId() is the link between the ACLs and their associated objects. Of course, you're free to write whatever code you like to retrieve the object by the given ID e.g. you could do something silly like increment the ID before retrieving it. So the object returned isn't guaranteed to be the same one that's authorized in this case: that's up to you to write the correct retrieval code.
The other way we can protect the method is with post authorize, as follows:
#PostAuthorize("hasPermission(returnObject, 'READ')")
public Account readAccount(String accountNumber) {
//CODE THAT RETRIEVES THE ENTITY FROM THE DB AND RETURNS IT
}
In this case, the code that retrieves the account entity is actually called, and then only after it's retrieved, the account, which is the object returned, is checked by the ACL framework against the current user and the READ permission. An advantage here is that we can retrieve the account any way we like e.g. by accountNumber in this case. Another advantage is that the object that's authorized is guaranteed to be the same as the one returned. The disadvantage is that we have to actually do the retrieval before we can make the call as to whether the user has permission or not. And if they don't have permission, then the retrieved object is effectively thrown away and so it may be a bit less performant than #PreAuthorize if the retrieval code is expensive to run.
Why Protecting Strings with Spring ACL Doesn't Make Sense Anyway
Technically speaking, I suppose you might be able to protect Strings, or indeed any other transient as long as it has a getId() method. With Strings, we could maybe add an extension function getId() for example. But I can't think of a practical use case for why we'd want to do that. For example, imagine we not only have Account objects but we also have Customer objects. And let's say Customer objects are uniquely identifiable by a customerNumber field, which is a String. And suppose we want to restrict access to customers in a similar way to accounts. Then what if coincidentally a customerNumber matched an accountNumber? In Spring ACL, we are only allowed one entry in the object identity table for every combination of object class plus ID, as per the Spring docs:
CONSTRAINT uk_acl_object_identity UNIQUE (object_id_class, object_id_identity)
So, suppose the String "fadfads389" happens to be both a customerNumber for some Customer and an accountNumber for some Account. If we were to restrict access to it via Spring ACL, what would that mean? Would it mean that a user has access to the account? The customer? Both?
Hopefully this example demonstrates why it doesn't really make sense to protect some transient class such as String with Spring ACL when that transient is used to identify entities: when we're interested in protecting entities we just protect the entities themselves, using the implicit ID of those entities e.g. the ID stored by Hibernate.
Transients Could Still be protected
Of course, there's nothing stopping you from adding object identity entries into the Spring ACL acl_object_identity table for any class you want, as long as that class has a getId() method. So it is certainly possible for you to add permissions relating to transients, and should those transients appear again in memory then Spring ACL will kick in. But this isn't really what Spring ACL was designed for - it's really meant to protect entities, not transients, which are linked to the ACL logic by getId().
Strings Could Still be Used in PreAuthorize
Now, although we shouldn't really use Spring ACL to protect Strings, that's not to say that #PreAuthorize is completely off the table when it comes to Strings. We note that #PreAuthorize can handle any SpEL expression, the same for #PostAuthorize and the other method annotations as pointed out here. So, for example, you would be able to do something like this:
#PreAuthorize("#user.accountNumbers.contains(#accountNumber)")
public Account readAccount(String accountNumber, User user) {
//CODE THAT RETRIEVES THE ENTITY FROM THE DB AND RETURNS IT
}
The above assumes that the User class maintains a list of account numbers to which that user has access, so presumably a User is an entity, or at least is backed by some DB-persisted data.
However, beware if you do want to go down this path. Not only do you risk entangling your access control logic with the rest of your business logic, you also might lose out on the performance front; Spring ACL uses caching to make permissions lookups fast whereas the above code presumably needs to fetch User data from the DB in order to do the authorization.

Extending JPA entity data at runtime

I need to allow client users to extend the data contained by a JPA entity at runtime. In other words I need to add a virtual column to the entity table at runtime. This virtual column will only be applicable to certain data rows and there could possibly be quite a few of these virtual columns. As such I don't want to create an actual additional column in the database, but rather I want to make use of additional entities that represent these virtual columns.
As an example, consider the following situation. I have a Company entity which has a field labelled Owner, which contains a reference to the Owner of the Company. At runtime a client user decides that all Companies that belong to a specific Owner should have the extra field labelled ContactDetails.
My preliminary design uses two additional entities to accomplish this. The first basically represents the virtual column and contains information such as the field name and type of value expected. The other represents the actual data and connects an entity row to a virtual column. For example, the first entity might contain the data "ContactDetails" while the second entity contains say "555-5555."
Is this the right way to go about doing this? Is there a better alternative? Also, what would be the easiest way to automatically load this data when the original entity is loaded? I want my DAO call to return the entity together with its extensions.
EDIT: I changed the example from a field labelled Type which could be a Partner or a Customer to the present version as it was confusing.

Perhaps a simpler alternative could be to add a CLOB column to each Company and store the extensions as an XML. There is a different set of tradeoffs here compared to your solution but as long as the extra data doesn't need to be SQL accessible (no indexes, fkeys and so on) it will probably be simple than what you do now.
It also means that if you have some fancy logic regarding the extra data you would need to implement it differently. For example if you need a list of all possible extension types you would have to maintain it separately. Or if you need searching capabilities (find customer by phone number) you will require lucene or similar solution.
I can elaborate more if you are interested.
EDIT:
To enable searching you would want something like lucene which is a great engine for doing free text search on arbitrary data. There is also hibernate-search which integrates lucene directly with hibernate using annotations and such - I haven't used it but I heard good things about it.
For fetching/writing/accessing data you are basically dealing with XML so any XML technique should apply. The best approach really depends on the actual content and how it is going to be used. I would suggest looking into XPath for data access, and maybe look into defining your own hibernate usertype so that all the access is encapsulated into a class and not just plain String.

I've run into more problems than I hoped I would and as such I decided to dumb down the requirements for my first iteration. I'm currently trying to allow such Extensions only on the entire Company entity, in other words, I'm dropping the whole Owner requirement. So the problem could be rephrased as "How can I add virtual columns (entries in another entity that act like an additional column) to an entity at runtime?"
My current implementation is as follow (irrelevant parts filtered out):
#Entity
class Company {
// The set of Extension definitions, for example "Location"
#Transient
public Set<Extension> getExtensions { .. }
// The actual entry, for example "Atlanta"
#OneToMany(fetch = FetchType.EAGER)
#JoinColumn(name = "companyId")
public Set<ExtensionEntry> getExtensionEntries { .. }
}
#Entity
class Extension {
public String getLabel() { .. }
public ValueType getValueType() { .. } // String, Boolean, Date, etc.
}
#Entity
class ExtensionEntry {
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "extensionId")
public Extension getExtension() { .. }
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "companyId", insertable = false, updatable = false)
public Company getCompany() { .. }
public String getValueAsString() { .. }
}
The implementation as is allows me to load a Company entity and Hibernate will ensure that all its ExtensionEntries are also loaded and that I can access the Extensions corresponding to those ExtensionEntries. In other words, if I wanted to, for example, display this additional information on a web page, I could access all of the required information as follow:
Company company = findCompany();
for (ExtensionEntry extensionEntry : company.getExtensionEntries()) {
String label = extensionEntry.getExtension().getLabel();
String value = extensionEntry.getValueAsString();
}
There are a number of problems with this, however. Firstly, when using FetchType.EAGER with an #OneToMany, Hibernate uses an outer join and as such will return duplicate Companies (one for each ExtensionEntry). This can be solved by using Criteria.DISTINCT_ROOT_ENTITY, but that in turn will cause errors in my pagination and as such is an unacceptable answer. The alternative is to change the FetchType to LAZY, but that means that I will always "manually" have to load ExtensionEntries. As far as I understand, if, for example, I loaded a List of 100 Companies, I'd have to loop over and query each of those, generating a 100 SQL statements which isn't acceptable performance-wise.
The other problem which I have is that ideally I'd like to load all the Extensions whenever a Company is loaded. With that I mean that I'd like that #Transient getter named getExtensions() to return all the Extensions for any Company. The problem here is that there is no foreign key relation between Company and Extension, as Extension isn't applicable to any single Company instance, but rather to all of them. Currently I can get past that with code like I present below, but this will not work when accessing referenced entities (if for example I have an entity Employee which has a reference to Company, the Company which I retrieve through employee.getCompany() won't have the Extensions loaded):
List<Company> companies = findAllCompanies();
List<Extension> extensions = findAllExtensions();
for (Company company : companies) {
// Extensions are the same for all Companies, but I need them client side
company.setExtensions(extensions);
}
So that's were I'm at currently, and I have no idea how to proceed in order to get past these problems. I'm thinking that my entire design might be flawed, but I'm unsure of how else to try and approach it.
Any and all ideas and suggestions are welcome!

The example with Company, Partner, and Customer is actually good application for polymorphism which is supported by means of inheritance with JPA: you will have one the following 3 strategies to choose from: single table, table per class, and joined. Your description sounds more like joined strategy but not necessarily.
You may also consider just one-to-one( or zero) relationship instead. Then you will need to have such relationship for each value of your virtual column since its values represent different entities. Hence, you'll have a relationship with Partner entity and another relationship with Customer entity and either, both or none can be null.

Use pattern decorator and hide your entity inside decoratorClass bye

Using EAV pattern is IMHO bad choice, because of performance problems and problems with reporting (many joins). Digging for solution I've found something else here: http://www.infoq.com/articles/hibernate-custom-fields