We Keep Coding

Creating a Jooq Caching Layer

So, I'm working on using Jooq to create a caching layer over Postgres. I've been using the MockConnection/MockDataProvider objects to intercept every query, and this is working, but I'm having a few issues.
First, how do I determine between reads and writes? That is, how do I tell whether a query is an insert/update/etc or a select, given only the MockExecuteContext that's passed into the execute method in MockDataProvider?
And I'm a bit confused on how I can do invalidations. The basic scheme I'm implementing right now is that whenever a "write" query is made to a table, I invalidate all cached queries that involve that table. This goes back to my first question, on telling different types of queries from each other, but also brings up another issue: how would I identify the tables used in a query given only the sql string and the bindings (both are attributes of MockExecuteContext)?
Also, is this a correct approach at caching? My first thought was to override the fetch() method, but that method is final, and I'd rather not change something already embedded in Jooq itself. This is the only other way I could think of to intercept all requests made so I could create a separate, persistent caching layer.
I have seen this (https://groups.google.com/forum/#!topic/jooq-user/xSjrvnmcDHw) question, but I'm still not clear on how Lukas recommended to identify tables from the object. I can try to implement a Postgres NOTIFY, but I wanted something native in Jooq first. I've seen this issue (https://github.com/jOOQ/jOOQ/issues/2665) pop up a lot too, but I'm not sure how it applies.
Keep in mind that I'm new to Jooq, so it's quite possible that I'm missing something obvious.
Thanks!

How would I audit the changes to a list of JPA entities?

I've got two lists of entities: One that is the current state of the rows in the DB, the other is the changes that were made to the list. How do I audit the rows that were deleted, added, and the changes made to the entities? My audit table is used by all the entities.
Entity listeners and Callback methods look like a perfect fit, until you notice the sentence that says: A callback method must not invoke EntityManager or Query methods! Because of this restriction, I can collect audits, but I can't persist them to the database :(
My solution has been a complex algorithm to discover the audits.
If the entity is in the change list and has no key, it's an add
If the entity is in the db but not the changes list, it's a delete
If the entity is in both list, recursively compare their fields to find differences to audit (if any)
I collect these and insert them into the DB in the same transaction I merge the changes list. But I hate the fact that I'm writing this by hand. It seems like JPA should be able to do this logic for me.
One solution we've come up with is to use an Entity Listener that posts the audits to a JMS queue. The queue then inserts the audits into the database. But I don't like this solution because I think setting up a JMS queue is a pain. It's currently the best solution we've got though.
I'm using eclipselink (ideally, that's not relevant) and have found these two things that look helpful but the JMS queue is a better solution than them:
http://wiki.eclipse.org/EclipseLink/FAQ/JPA#How_to_access_what_changed_in_an_object_or_transaction.3F This looks really difficult to use. You search for the fields by a string. So if I refactor my entity and forget to update this, it'll throw a runtime error.
http://wiki.eclipse.org/EclipseLink/Examples/JPA/History This isn't consistent with the way we currently audit. It expects a special entity_history table.

The EntityListener looks like a good approach since you are able to collect the audit information.
Have you tried persisting the information in a different transaction than the one persisting the changes? perhaps obtaining a reference to a Stateless EJB (assuming you are using EJBs) and using methods marked with #TransactionAttribute(TransactionAttributeType.REQUIRES_NEW). In this way the transaction persisting the original changes is put on hold while the transaction of the audit completes. Note that you will not be able to access the updated information in this separate audit transaction, since the original one has not committed yet.

Is removing an entity from a hibernate changeset as simple as calling EntityManger.detach()?

Background
I have a java/spring system where transactions are managed manually via a custom HandlerInterceptor. That is to say:
at the begining of every request a transaction is opened (an unfortunate part of the system is that any request might result in a write to the db)
an EntityManager instance joins the transaction
the entity manager is used to load entities which are modified. The EntityManager tracks all changes
at the end of every request the EntityManager is flushed and committed
Yes this is not ideal, but I did not create this system and it's simple enough to allow us to work within it's confines - I'm not looking to change it without good reason.
I am not used to commit-all-tracked-entities-on-flush behavior and so have been doing something like:
//change entity
if(ovalValidator.isValid(entity))
em.persist(entity);
I need to fix this to work with my new understanding and switching the above to this seems to work:
//change entity
if(!ovalValidator.isValid(entity))
em.detach(entity);
My question
It is my understanding that this just removes the entity from the flush queue even if it IS marked as dirty. Is this correct? Is there a better way to achieve what I am trying to (don't save changes to that entity)? Is there anything I need to look out for if I'm doing this?

detache removes the entity from the session (changeTracking, lazyloading, ...) it does what you want. You could also implement en interceptor removing the dirty mark of the invalid entities but i think your solution would work as well

EntityManager refresh

I have web application using JPA. This entity manager keeps bunch of entites and suddenly I update the database from other side. I use MySQL and I use PhpMyAdmin and change some row.
How to tell entity manager to re-synchronize, e.g. to forgot all the entites in cache?
I know there is refresh(Object) method, but is there any possibility how to do refreshAll() or something what results in this?
It is sure this is expensive operation but if it has to be done.

entityManager.getEntityManagerFactory().getCache().evictAll()
Refresh is something different since it modifies your object. This line will just empty the cache, so if you fetch objects changed outside the entity manager, it will do an actual database query instead of using the outdated cached value.

I had a similar issue and the evictAll() line above worked for me.
Alternatively, the #Cache annotation on the entity class worked too, with the benefit of being able to control caching parameters:
#Cache(coordinationType=CacheCoordinationType.INVALIDATE_CHANGED_OBJECTS)
See: http://wiki.eclipse.org/EclipseLink/Examples/JPA/Caching

If you are using EclipseLink instead of Hibernate the hint is:
em.createNamedQuery("SomeEntity.SomeNamedQuery")
.setHint(QueryHints.REFRESH, true)
.getResultList();

Well, for some people (like me) that tried to add factory.getCache().evictAll(); and doesn't work, and are used JPA + Hibernate, to refresh a query add the hint org.hibernate.cacheMode to IGNORE. Example:
em.createNamedQuery("SomeEntity.SomeNamedQuery")
.setHint("org.hibernate.cacheMode", "IGNORE")
.getResultList();

cache.evictAll is not working for me. So to retrieve data pushed from another app, I peform :
em.getTransaction().begin();
em.getTransaction().commit();
After that, my find query retrieves refreshed data. I don't know if it's very safe solution but it works properly.

When you read an object into an EntityManager, it becomes part of the persistence context, and the same object will remain in the EntityManager until you either clear() it and get a new EntityManager.
So if you update the database, the EntityManager will not see the change unless you call refresh() on the object, or clear() the EntityManager. This has nothing to do with the shared cache (L2) or the persistence context (L1). If you also also using a shared cache, and updating the database directly, then your shared cache will be out of date. You need to refresh() the object, or mark it as invalid to be refreshed the next time it is queried.
Code must follow the way like.
DETACH
REFRESH
MERGE
FLUSH

Pre-update and post-update in Hibernate

I want to use preupdate and postupdate concepts in my application.
The purpose is i want to get all values of a particulra record which is being updated and insert these values as a record along with new values in a table named audit.
Can any one give some suggestions or pointers on this.
Thanks,
Narendra

There are 2 ways to achieve this:
Hibernate Interceptors. See http://docs.jboss.org/hibernate/core/3.6/reference/en-US/html/events.html
Database level triggers/stored procedures.
You use the first if the pre/post functionality is business logic related -- i.e. your application is going to act on the update.
You use the second if you are only doing data level changes and the application doesn't need to know about it.

0 You can use HibernateInterceptors
1 You can do it programatically at service layer
2 You can either implement this things by database TRIGGERS then your DB will handle this.

You should have a look to Hibernate Envers. It does the Audit for you.
http://docs.jboss.org/envers/docs/index.html