There's an enterprise application using Java + Hibernate + PostgreSQL. Hibernate is configured via annotations in the Java source code. So far the database schema is fixed, but I faced the problem that it needs to be dynamic:I can receive data from different locations and I have to store these in different tables. This means that I have to create tables run-time.
Fortunately, it seems that all of these data coming from the different institutes can have the same schema. But I still don't know how to do that using Hibernate. There are two main problems:
How to tell to Hibernate that many different tables have the same structure? For example the "Patient" class can be mapped to not just the "patient" table, but the "patient_mayo_clinic" table, "patient_northwestern" table, etc. I can feel that this causes ambiguity: how Hibernate knows which table to access when I do operations on the Patient class? It can be any (but only one) of the former listed tables.
How can I dynamically create tables with Hibernate and bind a class to them?
Response to suggestions:
Thanks for all of the suggestions. So far all of the answers discouraged the dynamic creation of tables. I'll mark Axel's answer, since it achieves certain goals, and it is a supported solution. More specifically it's called multi-tenancy. Sometimes it's important to know some important phrases which describes our problem (or part of our problem).
Here are some links about multi-tenancy:
Multi-tenancy in Hibernate
Hibernate Chapter 16. Multi-tenancy
Multi-tenancy Design
EclipseLink JPA multi-tenancy
In real world scenario multi-tenancy also involves the area of isolating the sets of data from each other (also in terms of access and authorization by different credentials) once they are shoved into one table.
You can't do this with Hibernate.
Why not extend your patient table with an institute column?
This way you'll be able to differentiate, without running into mapping issues.
I am afraid you can't do this easily in Hibernate. You would have to generate the Java source, compile it, add it to your classpath and load it dynamically with java.reflection package. If that works, which I doubt it, it will be an ugly solution (IMHO).
Have you consider using a schema less database i.e: NoSQL databases or RDF
databases. They are much more flexible in terms of what you can store in them , basically things are not tight up against a relational schema.
In most environments it is not a good idea to create tables dynamically simply because dbas will not give you the rights to create tables in production.
Axel's answer may be right for you. Also look into Inheritance Mapping for Hibernate.
I agree that its not advisable to create tables dynamically nevertheless it's doable.
Personally i would do as Axel Fontaine proposed but if dynamic tables is a must-have for you I would consider using Partitioning.
PostgreSQL allows you to create ona main table and few child tables (partitions), records are disjunctive between child tables, but every record from any child table is visible in parent table. This means that you can insert rows into any child table you want using just simple insert statement (its not cool but has the same level of complexity as composing and persisting an entity, so its acceptable in your case) and query database using HQL
Related
We have a (possibly large) custom data structure implemented in Java (8+). It has a simple and optimal API for querying pieces of data. The logical structure is roughly similar to an RDMS (it has e. g. relations, columns, primary keys, and foreign keys), but there is no SQL driver.
The main goal is to access the data via ORM (mapping logical entities to JPA annotated beans). It would be nice if we could use JPQL. Hibernate is preferred but other alternatives are welcome too.
What is the simplest way to achieve this? Which are the key parts of such an implementation?
(P. S. Directly implementing SessionImplementor, EntityManagerImplementor etc. seems to be too complicated.)
You have two possibilities.
Implement a JDBC compliant driver for your system, so you can use a JPA implementation such as Hibernate "directly" (although you may need to create a custom dialect for your system).
Program directly against the JPA specification like ObjectDB does, which bypasses the need to go through SQL and JPA implementations completely.
The latter one is probably easier, but you'd still need to implement the full JPA API. If it's a custom in-house-only system, there's very little sense in doing either one.
One idea I thought up just now, that I feel may work is this:
Use an existing database implementation like H2 and use the JPA integration with that. H2 already has a JPA integration libraries, so it should be easy.
In this database, create a Java stored procedure or function and call it from your current application through JPA. See this H2 documentation on how to create a Java stored procedure or function. (You may want to explore the section "Using a Function as a Table" also.)
Define a protocol for the service methods and encapsulate it in a model class. An instance of this model class may be passed to the function/SP and responses retrieved.
Caveat: I have never done this myself but I think it will work.
Edit: Here is a diagram representing the thought. Though the diagram show H2 separately, it will most probably be in the same JVM as "Your Java/JEE application". However, since it is not necessary to use H2, I have shown it as as separate entity.
I'm using java play to make a web application and I'm having some trouble persisting data using ebeans and jpa annotations.
I've searched a lot, and I'm a little confused, I think that this is something that should be common, I want to have tables named like this:
company 1_users
company 2_users
company N_users
but I want to have only one entity named user, can be possible to have this schema ?
I've searched about using one entity with multiple tables, about using table name prefixes and about using dynamic table name but I got nothing helpful.
Some ideas ??
As #GlennLane said in comment, creating separate tables isn't good idea, instead use some field as a company id/discriminator. You will save tons of nerves, and you won't need to update your code each time when new company joins the branch.
If you really want separate tables create separate model for each company, at least you won't mishmash that in code.
In our project we have a constraint of not having the luxury to alter the table structure already in place. The tables are highly denormalized in nature.
We have come up with good POJOs for the application. We have the Entity beans generated out of the exiting tables. Now we have to map the POJOs to the entities so that we can persist.
Ultimately, we combine a good POJO with a bad table. Any thoughts on options/alternatives/suggestions to this approach?
Hibernate/JPA(2) has a rich set of functionality to manipulate the mapping (so that your objects can differ from the tables), so that many (NOT ALL) old tables can be mapped to normal object. -- May you should have a look at this first, any use your pojo/table-"solution" only if this mapping is not powerful enough.
If you have a read only application, you can think of using views to make your table/views more like you objects. This may reduse the amount of strange mapping.
I don't know your mapping, size of the application or use case, but have you considered not to use Hibernate? I ask this, because I can imagine (how I said: I don't know you application), that in a architecture like this, no Hibernate feature is used and so Hibernate will add only a not needed complexity.
If you are using Hibernate you should be able to map your POJOs to the table structure using only XML files, without creating new Java beans. This would allow you to easily change the mapping if all of a sudden you can change the tables structures and make the economy of intermediary beans. That's the best you can do.
I'm working on a project using Play Framework that requires me to create a multi-user application. I've a central panel where we add a certain workshop for a team. Thing is, I don't know if this is the best way, but I want to generate the tables like
team1_tablename
team1_secondtable..
Then when a certain request hits using the virtual host (e.x. http://teamawesome.workshop.com) I would need to maneuver the query to THAT certain table.
The problem is not generating the tables, but working with the models. All the workshops are going to have the same generic tables. In the model I would have to state the table, etc but then if this was PHP with doctrine I would have a template created them after creating the workshop team1, but in java even if I generate them I would have to compile them too which requires me to do more research.
My question is more Hibernate oriented before jumping the gun here and giving up on possible solutions. I'm all ears
I've thought of using NamedQueries, I don't know if I misread but I read in a hibernate book that you could query then add the result to a generic model so then I use that model to retain all my results...
If there are any doubts let me know, thanks (note this is not a multi database question, just using different sets of tables with unique prefixes)
I wonder if you could use one single set of tables, but have something like TEAM_ID as a foreign key in each table.
You would need one single TEAM table, where TEAM_ID will be the primary key. This will get migrated to tables and become part of foreign keys.
For instance, if you have a Player entity, having a collection of HighScores, then in the DB the Player table will have a TEAM_ID (foreign key from the Team table) and the HighScores table will have a composed foreign key (Player_id, Team_id) coming from the Player table..
So, bottom line, I am suggesting a logical partitioning of your database rather then a physical one (as you've considered initially).
Hope this makes sense, it definitely needs more thought, but if you think it's an interesting idea, I can think it through in more detail.
I am familiar with Hibernate and another web framework, here is how I would handle it:
I would create a single set of tables for one team that would address all my needs. Then I would:
Using DB2: Create a schema for each team copying the set of tables into each schema.
Using MySQL: Create a new Database for each copying the set of tables into each one.
Note: A 'database' in MySQL is more like a schema in other databases. (Sorry I'd rather keep things too simple than miss the point)
Now you can set up a separate hibernate.cfg.xml file for each connection (this isn't exactly the best way but perhaps best to start because it's so easy). Now you can specify the connection parameters... including the schema/db. Now your entity table, lets say it's called "team" will use the "team" table where ever it is connected...
To get started very quickly, when a user logs on create a user object in their session.
The user object will have a Hibernate SessionFactory which will be used for all database requests built from the correct hibernate.cfg.xml file as determined by parsing the URL used in the login.
Once the above is working... There are some serious efficiency concerns to address. That being that each logged on user is creating a SessionFactory... Maybe it isn't an issue if there isn't a lot of concurrent use but you probably want to look into Spring at that point and use a connection pool per team. This way there is one Session factory per team and there is no major object creation when a user signs in.
The benefits of this solution is that it should be easier to create new sets of tables because each table set lives in it's own world. There will only be one set of Entity Classes as opposed to the product of one for every team and table. The database schema stays rather simple not being complicated by adding team names and then the required constraints. If the teams require data ownership and privacy it will be rather easy to move the database to a different location.
The down side is that if the model needs to be changed for a team it must be done for each team (as opposed to a single table set using teamName as a foreign key).
The idea of using different tables for each team (despite what successful apps may use it) is honestly quite naïve, and has serious pitfalls when you take maintenance into account...
Just think what you will be forced to do if you discover you need a new table or even just an index... you'll end up needing to write DML scripts as templates and to use some (custom) software to run them on all the teams...
As mentioned in the other answers (Quaternion's and Octav's), I think you have two viable options:
Bring the "team" into your data model
Split the data in different databases/schemas
To choose the option that works best for you, you must decide if the "team" is really something you can partition your dataset into, or if it is really one more entity you want to bring into your datamodel.
You may have noticed that I'm using "splitting" here instead of "partitioning" - that's because the latter term is generally used by DBAs to indicate what we could call "sharding" - "splitting" is intended to be a stronger term.
Splitting is only viable if:
entities in different partitions do not ever need to reference each other
no query will ever need to access data from different partitions (this applies to queries used for reporting too)
As you might well see, splitting in this sense is not very attractive (maybe it could be ok now, but what when you find yourself wanting to add new features?), so my advice is to go for the "the Team is an entity" solution.
Also note that maintaining a set of databases/schemas is actually harder than maintaining a single (albeit maybe a bit more complex) database... again, think of what steps you should take to add an index in a production system...
The only downside of the single-databse solution manifests if you end up having multiple front-ends (maybe due to customizations for particular customers): changes to a shared database have the potential to affect all the applications using it, so you may need to coordinate upgrades to the different webapps to minimize risks (note, however, that in most cases you'll be able to change the database without breaking compatibility).
After all it's a little bit frustrating to get no information just shoot into the dark. Nevertheless now I have start the work, I try to finish.
I think you could do you job with following solution:
Wrote a PlayPlugin and make sure you add to every request the team to the request args. Then you wrote your own NamingStrategy. In the NamingStrategy you could read the request.args and put the team into your table name. Depending on how you add it Team_ or Team. it will be your preferred solution or something with schema. It sounds that you have an db-schema so it would be probably the best solution to stay with this tables and don't migrate.
Please make the next time your request more abstract so that you can provide some information like how many tables, is team an entity and how much records a table has (max, avg, min). How stable is your table model? This are all questions which helps to give a clear recommendation with arguments.
You can try the module vhost, but it seems not very good maintained. But I think the idea to put the name of the team into the table name is really weired. Postgres and Oracle has schemas for that. So you use myTeam.myTable. But then you must do the persistence by your selves.
Another approach would be different databases, but again you don't have good support by play. I would try this
Run for each team a separate play-server, if you don't have to much teams.
Put a reference to a Team-table for every model. Then you can use hibernate-filters or add it manually as additional parameter to each query. Of course this increase your performance. You can fix this issue with oracle partitions.
I haven't worked with hibernate. I have little bit of experience in java. I was going through source of a beast of an java application created by Oracle(Retail Price Management). I was expecting a lot of sql code embedded in there as the application makes heavy use of database. But to my surprise, NO embedded SQL code! so far. I found that it was using what is called as "Hibernate" from the lot of .hbm.xml files. Is it a trademark for java programs using hibernate or maybe I haven't seen the complete codebase?. Could someone enlighten me how this is possible?. Thanks.
Hibernate, as all ORM tools, indeed lessens or eliminates the need to use raw SQL in Java code, due to the following:
many associations between various entities are recorded in the Hibernate mapping, so these are fetched automatically by Hibernate - i.e. if you have an aggregation relationshiop between two classes on the Java side, this may be mapped as a foreign key relationship in the DB, and Hibernate, whenever an instance of class A is loaded, can automatically load the associated instances of class B too,
many queries can be done in Hibernate's own HQL query language, or using its Criteria API.
Under the hood Hibernate does generate SQL to communicate with the DB, but this is not visible on the Java side. It can be seen in the logs though, if it is enabled.
Due to this, programs using Hibernate very rarely need to use JDBC or SQL directly. The exceptions are typically ralted to "tricky" legacy DB schemas which can't be fully handled by Hibernate.
Because that's the whole purpose of using Hibernate or any other object-relational mapping framework.
Hibernate solves object-relational impedance mismatch problems by replacing direct persistence-related database accesses with high-level object handling functions.
Hibernate generates SQL for all its standard database operations. It understands different SQL dialects, and the mapping files (.hbm.xml) tell it about the database structure so it knows how to construct its queries. There is a showSql setting you can turn on if you want to see it outputting its generated SQL as it runs.
Hibernate is an Object-Relational Mapper (ORM). ORMs are used to hide the ugly details of SQL incompatibility[sic] between databases from your program -- you define your tables and map them to an object hierarchy (the .hbm.xml files) and then Hibernate does the rest. Thus most programs that use Hibernate won't see a single phrase of SQL, unless there's a specific reason to execute a complicated query.
Hibernate is a tool, or technology that takes care of the interaction between the database and application for you. You have to tell the structure of the application and the database to it, this is what is in the .hbm.xml files.
The SQL is generated by Hibernate at runtime (kind of)
Say you have an Fruit class, and objects of this is persisted into a T_FRUIT table.
You say this to hibernate, via the .hbm.xml files. That there is a table T_FRUIT, this table is represented by the Fruit class, and which fields in the Fruit class correspond to which columns in th T_FRUIT table.
And then it knows whenever you are trying to save a fruit, it should insert/update to the T_FRUIT table.
When you want to create an Apple, you create an object of fruit corresponding to apple and save "save this fruit".
Hibernate takes care of persisting it.
You can have relationships defined between tables, and Hibernate is intelligent enough to persist in multiple tables.
When you fetch a fruit, hibernate fetches the details of the fruit and its children also(data from referencing tables). And you can say whether you want fetch all the children
at once, or as and when required.
And so on. Aim is to make your life easier, and code maintainable, easy to read, portable,...
With this info, let me redirect you.