We Keep Coding

CSV file dynamic map to Database Tables using Hibernate

We have a requirement where around 25 CSV files would come each day & stored the Database in equivalent table structure.
Any of CSV file column structure could change in future by add new /remove columns & underlying DB table would align to the new format, without code change or redeployment.
Here are the choice of tech.
SpringBoot as Run time
Hibernate as JPA/DB Inetraction
Oracle DB as database
If using Hibernate, how to achieving this dynamic column management of the table as per the incoming CSV?
As far as I know, Hibernate would have Java Entity classes equivalent to the Table , which will be used to persists data. Any table change need Entity class change too.
Possible solution could be
just define basic JPA Entity & table structure (like ids & FKs linking to other tables etc) for CSV equivalent table,
then on arrival of CSV files, add the columns to the table by running the ALTER table command from application
In future 1st CSVs, if column added/removed , use similar alter commands
Is this achievable by Hibernate?
Or any other product better suited for this kind of tasks.

Task definition
We will have to implement a mechanism allowing for creating/deleting custom fields in real time avoiding the application restart, add a value into it and make sure the value is present in the application database. Besides we will have to make sure that the custom field can be used in queries.
Solution
Domain Model
We will first need a business entity class which we will experiment with. Let is be Contact class. There will be two persistent fields: id and name.
However besides these permanent and unchangeable fields the class should be some sort of construction to store values of custom fields. Map would be an ideal construction for this.
Let's create a base class for all business entities supporting custom fields - CustomizableEntity, that contains Map CustomProperties to work with custom fields:
package com.enterra.customfieldsdemo.domain;
import java.util.Map;
import java.util.HashMap;
public abstract class CustomizableEntity {
private Map customProperties;
public Map getCustomProperties() {
if (customProperties == null)
customProperties = new HashMap();
return customProperties;
}
public void setCustomProperties(Map customProperties) {
this.customProperties = customProperties;
}
public Object getValueOfCustomField(String name) {
return getCustomProperties().get(name);
}
public void setValueOfCustomField(String name, Object value) {
getCustomProperties().put(name, value);
}
}
Step 1 - base class CustomizableEntity
Inherit our class Contact from this base class:
package com.enterra.customfieldsdemo.domain;
import com.enterra.customfieldsdemo.domain.CustomizableEntity;
public class Contact extends CustomizableEntity {
private int id;
private String name;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
Step 2 - Class Contact inherited from CustomizableEntity.
We should not forget about the mapping file for this class:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
<hibernate-mapping auto-import="true" default-access="property" default-cascade="none" default-lazy="true">
<class abstract="false" name="com.enterra.customfieldsdemo.domain.Contact" table="tbl_contact">
<id column="fld_id" name="id">
<generator class="native"/>
</id>
<property name="name" column="fld_name" type="string"/>
<dynamic-component insert="true" name="customProperties" optimistic-lock="true" unique="false" update="true">
</dynamic-component>
</class>
</hibernate-mapping>
Step 3 - Mapping Class Contact.
Please note that properties id and name are done as all ordinary properties, however for customProperties we use a tag . Documentation on Hibernate 3.2.0GA says that the point of a dynamic-component is:
"The semantics of a mapping are identical to . The advantage of this kind of mapping is the ability to determine the actual properties of the bean at deployment time, just by editing the mapping document. Runtime manipulation of the mapping document is also possible, using a DOM parser. Even better, you can access (and change) Hibernate's configuration-time metamodel via the Configuration object."
Based on this regulation from Hibernate documentation we will be building this function mechanism.
HibernateUtil and hibernate.cfg.xml
After we are defined with the domain model of our application we have to create necessary conditions for Hibernate framework functioning. For this we have to create a configuration file hibernate.cfg.xml and class to work with the core Hibernate functions.
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE hibernate-configuration
PUBLIC "-//Hibernate/Hibernate Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="show_sql">true</property>
<property name="dialect">
org.hibernate.dialect.MySQLDialect</property>
<property name="cglib.use_reflection_optimizer">true</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/custom_fields_test</property>
<property name="hibernate.connection.username">root</property>
<property name="hibernate.connection.password"></property>
<property name="hibernate.c3p0.max_size">50</property>
<property name="hibernate.c3p0.min_size">0</property>
<property name="hibernate.c3p0.timeout">120</property>
<property name="hibernate.c3p0.max_statements">100</property>
<property name="hibernate.c3p0.idle_test_period">0</property>
<property name="hibernate.c3p0.acquire_increment">2</property>
<property name="hibernate.jdbc.batch_size">20</property>
<property name="hibernate.hbm2ddl.auto">update</property>
</session-factory>
</hibernate-configuration>
Step 4 - Hibernate configuration file.
The file hibernate.cfg.xml does not contain anything noticeable except for this string:
<property name="hibernate.hbm2ddl.auto">update</property>
Step 5 - using auto-update.
Later we will explain in details on its purpose and tell more how we can go without it. There are several ways to implement class HibernateUtil. Our implementation will differ a bit from well known due to changes into Hibernate configuration.
package com.enterra.customfieldsdemo;
import org.hibernate.*;
import org.hibernate.mapping.PersistentClass;
import org.hibernate.tool.hbm2ddl.SchemaUpdate;
import org.hibernate.cfg.Configuration;
import com.enterra.customfieldsdemo.domain.Contact;
public class HibernateUtil {
private static HibernateUtil instance;
private Configuration configuration;
private SessionFactory sessionFactory;
private Session session;
public synchronized static HibernateUtil getInstance() {
if (instance == null) {
instance = new HibernateUtil();
}
return instance;
}
private synchronized SessionFactory getSessionFactory() {
if (sessionFactory == null) {
sessionFactory = getConfiguration().buildSessionFactory();
}
return sessionFactory;
}
public synchronized Session getCurrentSession() {
if (session == null) {
session = getSessionFactory().openSession();
session.setFlushMode(FlushMode.COMMIT);
System.out.println("session opened.");
}
return session;
}
private synchronized Configuration getConfiguration() {
if (configuration == null) {
System.out.print("configuring Hibernate ... ");
try {
configuration = new Configuration().configure();
configuration.addClass(Contact.class);
System.out.println("ok");
} catch (HibernateException e) {
System.out.println("failure");
e.printStackTrace();
}
}
return configuration;
}
public void reset() {
Session session = getCurrentSession();
if (session != null) {
session.flush();
if (session.isOpen()) {
System.out.print("closing session ... ");
session.close();
System.out.println("ok");
}
}
SessionFactory sf = getSessionFactory();
if (sf != null) {
System.out.print("closing session factory ... ");
sf.close();
System.out.println("ok");
}
this.configuration = null;
this.sessionFactory = null;
this.session = null;
}
public PersistentClass getClassMapping(Class entityClass){
return getConfiguration().getClassMapping(entityClass.getName());
}
}
Step 6 - HibernateUtils class.
Alongside with usual methods like getCurrentSession(), getConfiguration(), which is necessary for regular work of the application based on Hibernate, we also have implemented such methods as: reset() and getClassMapping(Class entityClass). In the method getConfiguration(), we configure Hibernate and add class Contact into the configuration.
Method reset() has been used to close all used by Hibernate resources and clearing all of its settings:
public void reset() {
Session session = getCurrentSession();
if (session != null) {
session.flush();
if (session.isOpen()) {
System.out.print("closing session ... ");
session.close();
System.out.println("ok");
}
}
SessionFactory sf = getSessionFactory();
if (sf != null) {
System.out.print("closing session factory ... "); sf.close();
System.out.println("ok");
}
this.configuration = null;
this.sessionFactory = null;
this.session = null;
}
Step 7 - method reset()
Method getClassMapping(Class entityClass) returns object PersistentClass, that contains full information on mapping the related entity. In particular the manipulations with the object PersistentClass allow modifying the set of attributes of the entity class in the run-time.
public PersistentClass getClassMapping(Class entityClass){
return
getConfiguration().getClassMapping(entityClass.getName());
}
Step 8 - method getClassMapping(Class entityClass).
Manipulations with mapping
Once we have the business entity class (Contact) available and the main class to interact with Hibernate we can start working. We can create and save samples of the Contact class. We can even place some data into our Map customProperties, however we should be aware that this data (stored in Map customProperties) are not saved to the DB.
To have the data saved we should provide for the mechanism of creating custom fields in our classs and make it the way Hibernate knows how to work with them.
To provide for class mapping manipulation we should create some interface. Let's call it CustomizableEntityManager. Its name should reflect the purpose of the interface managing a business entity, its contents and attributes:
package com.enterra.customfieldsdemo;
import org.hibernate.mapping.Component;
public interface CustomizableEntityManager {
public static String CUSTOM_COMPONENT_NAME = "customProperties";
void addCustomField(String name);
void removeCustomField(String name);
Component getCustomProperties();
Class getEntityClass();
}
Step 9 - Interface CustomizableEntityManager
The main methods for the interface are: void addCustomField(String name) and void removeCustomField(String name). These should created and remove our custom field in the mapping of the corresponding class.
Below is the way to implement the interface:
package com.enterra.customfieldsdemo;
import org.hibernate.cfg.Configuration;
import org.hibernate.mapping.*;
import java.util.Iterator;
public class CustomizableEntityManagerImpl implements CustomizableEntityManager {
private Component customProperties;
private Class entityClass;
public CustomizableEntityManagerImpl(Class entityClass) {
this.entityClass = entityClass;
}
public Class getEntityClass() {
return entityClass;
}
public Component getCustomProperties() {
if (customProperties == null) {
Property property = getPersistentClass().getProperty(CUSTOM_COMPONENT_NAME);
customProperties = (Component) property.getValue();
}
return customProperties;
}
public void addCustomField(String name) {
SimpleValue simpleValue = new SimpleValue();
simpleValue.addColumn(new Column("fld_" + name));
simpleValue.setTypeName(String.class.getName());
PersistentClass persistentClass = getPersistentClass();
simpleValue.setTable(persistentClass.getTable());
Property property = new Property();
property.setName(name);
property.setValue(simpleValue);
getCustomProperties().addProperty(property);
updateMapping();
}
public void removeCustomField(String name) {
Iterator propertyIterator = customProperties.getPropertyIterator();
while (propertyIterator.hasNext()) {
Property property = (Property) propertyIterator.next();
if (property.getName().equals(name)) {
propertyIterator.remove();
updateMapping();
return;
}
}
}
private synchronized void updateMapping() {
MappingManager.updateClassMapping(this);
HibernateUtil.getInstance().reset();
// updateDBSchema();
}
private PersistentClass getPersistentClass() {
return HibernateUtil.getInstance().getClassMapping(this.entityClass);
}
}
Step 10 - implementing interface CustomizableEntityManager
First of all we should point out that when creating class CustomizableEntityManager we specify the business entity class the manager will operate. This class is passed as a parameter to designer CustomizableEntityManager:
private Class entityClass;
public CustomizableEntityManagerImpl(Class entityClass) {
this.entityClass = entityClass;
}
public Class getEntityClass() {
return entityClass;
}
Step 11 - class designer CustomizableEntityManagerImpl
Now we should get more interested in how to implement method void addCustomField(String name):
public void addCustomField(String name) {
SimpleValue simpleValue = new SimpleValue();
simpleValue.addColumn(new Column("fld_" + name));
simpleValue.setTypeName(String.class.getName());
PersistentClass persistentClass = getPersistentClass();
simpleValue.setTable(persistentClass.getTable());
Property property = new Property();
property.setName(name);
property.setValue(simpleValue);
getCustomProperties().addProperty(property);
updateMapping();
}
Step 12 - creating custom field.
As we can see from the implementation, Hibernate offers more options in working with properties of persistent objects and their representation in the DB. As per the essence of the method:
1) We create class SimpleValue that allow us to denote how the value of this custom field will be stored in the DB in which field and table of the DB:
SimpleValue simpleValue = new SimpleValue();
simpleValue.addColumn(new Column("fld_" + name));
simpleValue.setTypeName(String.class.getName());
PersistentClass persistentClass = getPersistentClass();
simpleValue.setTable(persistentClass.getTable());
Step 13 - creating new column of the table.
2) We create a property of the persistent object and add a dynamic component into it (!), that we have planned to be used for this purpose:
Property property = new Property()
property.setName(name)
property.setValue(simpleValue)
getCustomProperties().addProperty(property)
Step 14 - creating object property.
3) And finally we should make our application perform certain changes in the xml files and update the Hibernate configuration. This can be done via method updateMapping();
It is necessary to clarify the purpose of another two get-methods which have been used in the code above. The first method is getCustomProperties():
public Component getCustomProperties() {
if (customProperties == null) {
Property property = getPersistentClass().getProperty(CUSTOM_COMPONENT_NAME);
customProperties = (Component) property.getValue();
}
return customProperties;
}
Step 15 - getting CustomProperties as Component.
This method finds and returns object Component corresponding to the tag in the mapping of our business entity.
The second method is updateMapping():
private synchronized void updateMapping() {
MappingManager.updateClassMapping(this);
HibernateUtil.getInstance().reset();
// updateDBSchema();
}
Step 16 - method updateMapping().
The method is in charge for storing the updated mapping of the persistent class and updates the configuration status of Hibernate to make further changes that we make valid when the changes take effect.
By the way we should get back to the string:
<property name="hibernate.hbm2ddl.auto">update</property>
of the Hibernate configuration. If this string was missing we would have to launch executing updates of the DB schema using hibernate utilities. However using the setting allows us to avoid this.
Saving mapping
Modifications to mapping made in run-time do not save by themselves into the corresponding xml mapping file and to make the changes to get activated at next launch of the application we need to manually save changes to the corresponding mapping file.
To do this we will be using class MappingManager the main purpose of which is to save mapping of the designated business entity to its xml mapping file:
package com.enterra.customfieldsdemo;
import com.enterra.customfieldsdemo.domain.CustomizableEntity;
import org.hibernate.Session;
import org.hibernate.mapping.Column;
import org.hibernate.mapping.Property;
import org.hibernate.type.Type;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import java.util.Iterator;
public class MappingManager {
public static void updateClassMapping(CustomizableEntityManager entityManager) {
try {
Session session = HibernateUtil.getInstance().getCurrentSession();
Class<? extends CustomizableEntity> entityClass = entityManager.getEntityClass();
String file = entityClass.getResource(entityClass.getSimpleName() + ".hbm.xml").getPath();
Document document = XMLUtil.loadDocument(file);
NodeList componentTags = document.getElementsByTagName("dynamic-component");
Node node = componentTags.item(0);
XMLUtil.removeChildren(node);
Iterator propertyIterator = entityManager.getCustomProperties().getPropertyIterator();
while (propertyIterator.hasNext()) {
Property property = (Property) propertyIterator.next();
Element element = createPropertyElement(document, property);
node.appendChild(element);
}
XMLUtil.saveDocument(document, file);
} catch (Exception e) {
e.printStackTrace();
}
}
private static Element createPropertyElement(Document document, Property property) {
Element element = document.createElement("property");
Type type = property.getType();
element.setAttribute("name", property.getName());
element.setAttribute("column", ((Column)
property.getColumnIterator().next()).getName());
element.setAttribute("type",
type.getReturnedClass().getName());
element.setAttribute("not-null", String.valueOf(false));
return element;
}
}
Step 17 - the utility to update mapping of the persistent class.
The class literally performs the following:
Defines a location and loads xml mapping for the designated business entity into the DOM Document object for further manipulations with it;
Finds the element of this document . In particular here we store the custom fields and its contents we change;
Delete (!) all embedded elements from this element;
For any persistent property contained in our component that is in charge for the custom fields storage, we create a specific document element and define attributes for the element from the corresponding property;
Save this newly created mapping file.
When manipulating XML we use (as we can see from the code) class XMLUtil, that in general can be implemented in any way though it should correctly load and save the xml file.
Our implementation is given at the Step below:
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.w3c.dom.Document;
import org.xml.sax.SAXException;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.dom.DOMSource;
import java.io.IOException;
import java.io.FileOutputStream;
public class XMLUtil {
public static void removeChildren(Node node) {
NodeList childNodes = node.getChildNodes();
int length = childNodes.getLength();
for (int i = length - 1; i > -1; i--)
node.removeChild(childNodes.item(i));
}
public static Document loadDocument(String file)
throws ParserConfigurationException, SAXException, IOException {
DocumentBuilderFactory factory =DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
return builder.parse(file);
}
public static void saveDocument(Document dom, String file)
throws TransformerException, IOException {
TransformerFactory tf = TransformerFactory.newInstance();
Transformer transformer = tf.newTransformer();
transformer.setOutputProperty(OutputKeys.DOCTYPE_PUBLIC, dom.getDoctype().getPublicId());
transformer.setOutputProperty(OutputKeys.DOCTYPE_SYSTEM, dom.getDoctype().getSystemId());
DOMSource source = new DOMSource(dom);
StreamResult result = new StreamResult();
FileOutputStream outputStream = new FileOutputStream(file);
result.setOutputStream(outputStream);
transformer.transform(source, result);
outputStream.flush();
outputStream.close();
}
}
Source: Please refer this article for more detail

org.dom4j.DocumentException: hibernate.org Nested exception: hibernate.org

I am new to hibernate.I am developing a struts application where i want to integerate hibernate.I am using SQL Server 2008 R2 as database.I have configured my hibernate configuration file and used annotations based entities.When i try to run my hibernate configuration file,I get this below error :
Error
12:49:41.752 [main] DEBUG org.hibernate.util.DTDEntityResolver - trying to resol
ve system-id [http://hibernate.org/dtd/hibernate-configuration-3.0.dtd]
Initial SessionFactory creation failed.org.hibernate.HibernateException: Could n
ot parse configuration: hibernate.cfg.xml
I have given my files below :
DAOImpl
package com.myProj.dao.impl;
import java.util.logging.Logger;
import com.googlecode.s2hibernate.struts2.plugin.annotations.SessionTarget;
import com.googlecode.s2hibernate.struts2.plugin.annotations.TransactionTarget;
import com.myProj.dao.StudentDetailsDAO;
import com.myProj.entity.StudentDetails;
import org.hibernate.Session;
import org.hibernate.Transaction;
public class StudentDetailsDAOImpl implements StudentDetailsDAO{
Logger LOGGER;
#SessionTarget
Session session;
#TransactionTarget
Transaction transaction;
#Override
public void saveOrUpdateStudentDetail(StudentDetails studDetails) {
try{
session.saveOrUpdate(studDetails);
}catch(Exception e){
transaction.rollback();
LOGGER.info("StudentDetailsDAOImpl : saveOrUpdateStudentDetail : Exception "+e.toString());
}
}
}
StudentDetails.java
package com.myProj.entity;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.Table;
#Entity
#Table(name="dbo.student_details")
public class StudentDetails {
#Id
#GeneratedValue
#Column(name="id")
private int id;
#Column(name="name")
private String name;
#Column(name="grade")
private String grade;
#Column(name="dob")
private String dob;
#Column(name="stud_address")
private String stud_address;
#Column(name="stud_language")
private String stud_language;
#Column(name="student_marks_id")
private StudentMarks studentMarks;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getGrade() {
return grade;
}
public void setGrade(String grade) {
this.grade = grade;
}
public String getDob() {
return dob;
}
public void setDob(String dob) {
this.dob = dob;
}
public String getStud_address() {
return stud_address;
}
public void setStud_address(String stud_address) {
this.stud_address = stud_address;
}
public String getStud_language() {
return stud_language;
}
public void setStud_language(String stud_language) {
this.stud_language = stud_language;
}
public StudentMarks getStudentMarks() {
return studentMarks;
}
public void setStudentMarks(StudentMarks studentMarks) {
this.studentMarks = studentMarks;
}
}
StudentMark.java
package com.myProj.entity;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.Table;
#Entity
#Table(name="dbo.student_marks")
public class StudentMarks {
#Id
#GeneratedValue
#Column(name="id")
private int id;
#Column(name="english")
private int english;
#Column(name="physics")
private int physics;
#Column(name="chemistry")
private int chemistry;
#Column(name="biology")
private int biology;
#Column(name="maths")
private int maths;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getEnglish() {
return english;
}
public void setEnglish(int english) {
this.english = english;
}
public int getPhysics() {
return physics;
}
public void setPhysics(int physics) {
this.physics = physics;
}
public int getChemistry() {
return chemistry;
}
public void setChemistry(int chemistry) {
this.chemistry = chemistry;
}
public int getBiology() {
return biology;
}
public void setBiology(int biology) {
this.biology = biology;
}
public int getMaths() {
return maths;
}
public void setMaths(int maths) {
this.maths = maths;
}
}
HibernateUtil.java
package com.myProj.util;
import org.hibernate.SessionFactory;
import org.hibernate.cfg.AnnotationConfiguration;
import com.myProj.entity.StudentDetails;
public class HibernateUtil {
private static final SessionFactory sessionFactory;
static {
try {
AnnotationConfiguration config = new AnnotationConfiguration();
config.addAnnotatedClass(StudentDetails.class);
config.configure("hibernate.cfg.xml");
//new SchemaExport(config).create(true,true);
sessionFactory = config.buildSessionFactory();
} catch (Throwable ex) {
// Log the exception.
System.err.println("Initial SessionFactory creation failed." + ex);
throw new ExceptionInInitializerError(ex);
}
}
public static SessionFactory getSessionFactory() {
return sessionFactory;
}
}
hibernate.cfg.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://hibernate.org/dtd/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.driver_class">com.microsoft.sqlserver.jdbc.SQLServerDriver</property>
<property name="hibernate.connection.url"> jdbc:sqlserver://<myIp>:1433;databaseName=<myDatabase></property>
<property name="hibernate.connection.username">root</property>
<property name="connection.password">root</property>
<property name="connection.pool_size">1</property>
<property name="hibernate.dialect">org.hibernate.dialect.SQLServer2008Dialect</property>
<property name="show_sql">true</property>
<property name="hbm2ddl.auto">create</property>
<mapping class="com.myProj.entity.StudentDetails" ></mapping>
<mapping class="com.myProj.entity.StudentMarks" ></mapping>
</session-factory>
</hibernate-configuration>
I have given below the list of jar files that i am using :
commons-beanutils-1.8.0.jar commons-chain-1.2.jar
commons-digester-2.1.jar hibernate-3.5.3.jar
javax.servlet-3.0.jar logback-classic-0.9.6.jar
logback-core-0.9.6.jar ognl-3.0.6.jar
slf4j-api-2.0.99.jar slf4j-log4j13-1.0.1.jar
sqljdbc-1.2.0.jar struts-core-1.3.10.jar
struts-taglib-1.3.10-sources.jar struts-taglib-1.3.5.jar
struts2-core-2.3.8.jar struts2-json-plugin-2.3.8.jar
struts2-tiles-plugin-2.1.8.1.jar xwork-2.0.4.jar
xwork-core-2.3.12.jar
Also when i check my apache server i get the following error in it
Apache Server log
java.lang.ExceptionInInitializerError
at com.myProj.util.HibernateUtil.<clinit>(HibernateUtil.java:27)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3744)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4252)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1014)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:736)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1014)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:448)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:700)
at org.apache.catalina.startup.Catalina.start(Catalina.java:552)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:295)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:433)
Caused by: org.hibernate.HibernateException: Could not parse configuration: hibernate.cfg.xml
at org.hibernate.cfg.Configuration.doConfigure(Configuration.java:1586)
at org.hibernate.cfg.AnnotationConfiguration.doConfigure(AnnotationConfiguration.java:1212)
at org.hibernate.cfg.AnnotationConfiguration.doConfigure(AnnotationConfiguration.java:107)
at org.hibernate.cfg.Configuration.configure(Configuration.java:1520)
at org.hibernate.cfg.AnnotationConfiguration.configure(AnnotationConfiguration.java:1194)
at com.myProj.util.HibernateUtil.<clinit>(HibernateUtil.java:19)
... 21 more
Caused by: org.dom4j.DocumentException: hibernate.org Nested exception: hibernate.org
at org.dom4j.io.SAXReader.read(SAXReader.java:484)
at org.hibernate.cfg.Configuration.doConfigure(Configuration.java:1576)
... 26 more

I have finally found out what is wrong.But not sure why it didnt work.I changed the below line in my configuration file
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">
to
<!DOCTYPE hibernate-configuration SYSTEM "classpath://com/myProj/util/hibernate-configuration-3.0.dtd">
and had placed the file hibernate-configuration-3.0.dtd in the package com.myProj.util.Now this configuration file is running as gem and the session is created.

I have given below the list of jar files that i am using :
commons-beanutils-1.8.0.jar
commons-chain-1.2.jar
commons-digester-2.1.jar
hibernate-3.5.3.jar
javax.servlet-3.0.jar
logback-classic-0.9.6.jar
logback-core-0.9.6.jar
ognl-3.0.6.jar
slf4j-api-2.0.99.jar
slf4j-log4j13-1.0.1.jar
sqljdbc-1.2.0.jar
struts-core-1.3.10.jar
struts-taglib-1.3.10-sources.jar
struts-taglib-1.3.5.jar
struts2-core-2.3.8.jar
struts2-json-plugin-2.3.8.jar
struts2-tiles-plugin-2.1.8.1.jar
xwork-2.0.4.jar
xwork-core-2.3.12.jar
This is the biggest pile of garbage I've ever seen in years.
Only put what you need (and before knowing if you need it or not, you need to know what every jar does), beginning by dropping every Struts 1 jar;
Preserve the consistency of the versions: every Struts2 jar must have the same version, you can't have 2.3.8 with 2.3.12 with 2.1.8.1... what-the-heck ?
Always use the latest version (2.3.32), or you're vulnerable to CRITICAL issues and your server will most likely be hacked in minutes. Small refactors could be needed, but believe me, it's your best option.
Use Maven that will do it for you, or do it manually cum grano salis, or do it manually, but at the end you should have the following lsit of libraries in your project:
struts2-core-2.3.32.jar
xwork-core-2.3.32.jar
struts2-tiles-plugin-2.3.32.jar
struts2-json-plugin-2.3.32.jar
ognl-3.0.19.jar
freemarker-2.3.22.jar
log4j-core-2.3.jar
log4j-slf4j-impl-2.3.jar <-- optional
slf4j-api-1.7.25.jar <-- optional
Plus the ones you've not shown us (asm3.3m, asm5.0.2, etc...).
But seriously, stop a minute and run a Maven archetype for 2.3.32, it will generate an empty project for you and import all the right libraries; then you can take those libraries and put them in your ant-based project (or whatever it is), but don't do this annoying and highly unproductive work manually.
After that, I guess your Hibernate will work.
At that point, however, if you're runnign in a Java EE container (Jboss, Wildfly, Weblogic, Glassfish, TomEE), you should consider migrating from mixed Hibernate / JPA2 (javax.persistence annotations), to full JPA2.
JPA2 is the standard persistence layer in Java EE 6+. It's described in JSR-317 (JPA 2.0) and JSR-338 (JPA 2.1), and it defines a standard and reliable way of handling the persistence. It does not provide an implementation, so you are free to choose the implementation you prefer (eg. Hibernate).
Just use only javax.persistence annotations, and not a single org.hibernate annotation, and you'll be going full JPA2, even if using Hibernate.
That way, many things (like session factories) will not even be needed anymore: almost everything will be handled by the container automatically (but you'll be able to choose to handle transactions manually, if you want), and simply injecting an EntityManager with the #PersistenceContext annotation, through an automatically handled EntityManagerFactory.
Note 1: you don't need to specify the column name if it is identical to the variable name:
#Column(name="the_name") // needed
private String name;
//#Column(name="name") // not needed
#Column // this is enough
private String name;
Note 2: you can automate the camelCase to snake_case variable-to-column conversion by simply inserting a property in the persistence.xml (the JPA version of hibernate.cfg.xml, almost identical), so fullName variable will be translated to full_name on the database without the need to specify it in the #Column annotation.

The problem is hibernate.cfg.xml
<property name="hibernate.connection.url">
jdbc:sqlserver://<myIp>:1433;databaseName=<myDatabase>
</property>
Replace <myIp> and <myDatabase> with proper values

As StanislavL mentioned,the problem is with how properties [ myIp , myDatabase ] are mentioned as xml tags. That causes xml parsing exception.
since you are not using spring, it seems the best bet you have is to set properties at run time. please refer these links - hibernate.cfg.xml - Set parameters from a properties file
check out the answer "You can do it programmatically" - How to include properties from external file to hibernate.cfg.xml?

The onFlushDirty Hibernate Interceptor method is never called

Question: Why MyInterceptor#onFlushDirty is never called?
I extend AbstractEntityManagerFactoryBean in xml configs like
<bean id="myEntityManagerFactory" parent="abstractEntityManagerFactoryBean" abstract="true">
<property name="entityInterceptor">
<bean class="xxxx.MyInterceptor"/>
</property>
</bean>
<bean id="abstractEntityManagerFactoryBean" class="xxxx.MyEntityManagerFactoryBean"/>
MyEntityManagerFactoryBean
public class MyEntityManagerFactoryBean extends AbstractEntityManagerFactoryBean implements LoadTimeWeaverAware {
private Interceptor entityInterceptor;
public Interceptor getEntityInterceptor() {
return entityInterceptor;
}
public void setEntityInterceptor(Interceptor interceptor) {
entityInterceptor = interceptor;
}
}
MyInterceptor:
public class MyInterceptor extends EmptyInterceptor {
public MyInterceptor() {
System.out.println("init"); // Works well
}
// PROBLEM - is never called
#Override
public boolean onFlushDirty(Object entity,
Serializable id,
Object[] currentState,
Object[] previousState,
String[] propertyNames,
Type[] types) {
if (entity instanceof File) {
.....
}
return false;
}
}
UPDATE: [explanation why custom dirty policy looks like not my way]
I want update modified timestamp each time I change something in Folder entity EXCEPT folderPosition. In the same time folderPosition should be persistent and not transient (means cause entity to be dirty).
Due I use Spring Transactional and Hibernate Templates, there is some nuances:
1) I can't update modified timestamp at the end of each setter like:
public void setXXX(XXX xxx) {
//PROBLEM: Hibernate templates collect object via setters,
//means simple get query will cause multiple 'modified' timestamp updates
this.xxx = xxx;
this.modified = new Date();
}
2) I can't call setModified manually, because it has about 25 fields, and setXXX for each field is scattered across whole app. And I have no power to make refactoring.
#Entity
public class Folder {
/**
* GOAL: Changing of each of these fields except 'folderPosition' should cause
* 'modified' timestamp update
*/
private long id;
private String name;
private Date created;
private Date modified;
private Integer folderLocation;
#PreUpdate
public void preUpdate() {
//PROBLEM : change modified even if only location field has been changed!
//PROBLEM: need to know which fields have been updated!
modified = new Date();
}
....
}

You need to extend the findDirty method not onFlushDirty. Check this tutorial for a detail explanation with a reference to a GitHub working example.

How do I prevent EclipseLink (JPA 2.0) from clearing my database/table?

I'm using EclipseLink (JPA 2.0) on Netbeans 7.0 with a MySQL database - which I configured according to this screencast. But sometimes when I persist an Object/Entity my table get's cleared before that object persists, and I get a table with only this object (like it has been cleared). I'm only simply adding new entries in (populating) the database.
How can I prevent this?
Here's my code:
Movie movie = new Movie(
jsondata.path("Title").getTextValue(),
jsondata.path("Year").getValueAsInt(),
jsondata.path("Genre").getTextValue(),
jsondata.path("Plot").getTextValue(),
jsondata.path("Poster").getTextValue());
persist(movie);
My persist() function:
public static void persist(Object object) {
EntityManagerFactory emf = Persistence.createEntityManagerFactory("AllmoviesPU");
EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
try {
em.persist(object);
em.getTransaction().commit();
} catch (Exception e) {
e.printStackTrace();
em.getTransaction().rollback();
}
em.close();
}
My Movie object/entity:
#Entity
public class Movie implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String title;
#Column(name="MOVIEYEAR")
private int year;
private String genre;
private String synopsis;
private String image;
public Movie(String title, int year, String genre, String synopsis, String image) {
this.title = title;
this.year = year;
this.genre = genre;
this.synopsis = synopsis;
this.image = image;
}
public Movie() {
}
/* ... Getters and Setters stuff ... */
}
My persistence.xml:
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="AllmoviesPU" transaction-type="RESOURCE_LOCAL">
<provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
<class>allmovies.data.Movie</class>
<properties>
<property name="javax.persistence.jdbc.url" value="jdbc:mysql://123.123.123.123:3306/psanta_allmovies"/>
<property name="javax.persistence.jdbc.password" value="XXXXXX"/>
<property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver"/>
<property name="javax.persistence.jdbc.user" value="psanta_allmovies"/>
</properties>
</persistence-unit>
</persistence>

Try turning on logging to FINEST or FINE to see what is occurring. ("eclipselink.logging.level"="finest").
EclipseLink does have an option to recreate tables on startup ("eclipselink.ddl-generation"="drop-and-create-tables"), ensure you are not using that. NetBeans or Glassfish may be setting this by default, I think they may do this when in development mode.

There are bunch of things that need to be addressed in your code, although I cannot guarantee they are related to your problem.
The first thing is that creating your entity manager factory and your entity manager every time you simply want to persist an entity is way too expensive. Also, you are not closing your entity manager factory.
The right way to do it is, create an entity manager factory and keep it available for the entire life of your application. Every time you start a conversation with your user (i.e. a transaction) then request from it a entity manager, use it to persist everything you want to persist and then close the entity manager.
When your application finishes, close the entity manager factory.
A second aspect that you might like to take into account is that certain exceptions already imply a rollback, for instance, any exeception thrown during commit execution. Therefore if you attempt a second rollback, then you will get a second exception which you are not handling.
I cannot guarantee anything of these changes would solve your problem, but they are certainly a good start.
That being said, it is not evident for me, from your code, where you are persisting more than one entity. I guess, more context is required.
I'd suggest somewhat like this:
public class DataAccessLayer {
private EntityManagerFactory emf;
public DataAccessLayer (){
}
public synchronized void start(){
if(isStarted())
throw new IllegalStateException("Already started");
this.emf = Persistence.createEntityManagerFactory("AllmoviesPU");
}
public synchronized void stop(){
if(!isStarted())
throw new IllegalStateException("Not started");
this.emf.close();
this.emf = null;
}
public synchronized boolean isStarted(){
return this.emf != null;
}
public void doPersistenceStuff(InputData o){
if(!isStarted())
throw new IllegalStateException("Not started");
EntityManager em = emf.getEntityManager();
EntityTransaction t = entityManager.getTransaction();
try {
t.begin();
//do persistence stuff here
t.commit();
}
catch (RollbackException e) {
//transaction already rolledback
throw new MyPersistenceException(e);
}
catch (Exception e) {
t.rollback(); //manual rollback
throw new MyPersistenceException(e);
}
em.close()
}
}
And you can make sure the emf is closed when your application is finished (like when the user clicks the exit button, or when your server stops or shuts down) by invoking the stop() method in the data access layer instance.

Postgresql UUID supported by Hibernate?

I can't get Hibernate working with java.util.UUID for PostgreSQL.
Here is the mapping using javax.persistence.* annotations:
private UUID itemUuid;
#Column(name="item_uuid",columnDefinition="uuid NOT NULL")
public UUID getItemUuid() {
return itemUuid;
}
public void setItemUuid(UUID itemUuid) {
this.itemUuid = itemUuid;
}
When persisting a transient object I get a SQLGrammarException:
column "item_uuid" is of type uuid but expression is of type bytea at character 149
PostgreSQL version is 8.4.4
JDBC driver - 8.4.4-702 (also tried 9.0 - same thing)
Hibernate version is 3.6, main configuration properties:
<property name="hibernate.dialect">org.hibernate.dialect.PostgreSQLDialect</property>
<property name="hibernate.connection.driver_class">org.postgresql.Driver</property>
<property name="hibernate.connection.url">jdbc:postgresql://192.168.1.1/db_test</property>

This can be solved by adding the following annotation to the UUID:
import org.hibernate.annotations.Type;
...
#Type(type="pg-uuid")
private java.util.UUID itemUuid;
As to why Hibernate doesn't just make this the default setting, I couldn't tell you...
UPDATE:
There still seem to be issues using the createNativeQuery method to open objects that have UUID fields. Fortunately, the createQuery method so far has worked fine for me.

Now you can also use the UUID class provided by java.util.UUID which gets mapped to uuid datatype of Postgres by Hibernate without any conversions required while reading/writing from the database.
#Id
#GeneratedValue
private UUID id;
The generated value is auto by default this lets your JVM define the UUID. This also allows hibernate to use the batch insert optimisation.
You can configure the database to set the UUID value. More information can be found here https://vladmihalcea.com/uuid-identifier-jpa-hibernate/

You try to persist object of type UUID, which is not hibernate-annotated entity. So the hibernate wants to serialize it to byte array (blob type). This is why you get this message 'expression of type bytea'.
You can either store UUID as blobs in database (not elegant), or provide your custom serializer (much work) or manually convert that object. UUID class has methods fromString and toString, so I would store it as String.

As others mentioned, the solution to this issue is to add a #Type(type = "pg-uuid") annotation. However, this type is not compatible with UUID types of other vendors, so this ties your Hibernate classes to Postgres. To work around this, it is possible to insert this annotation at runtime. The below works for Hibernate 4.3.7.
First, you need to insert a custom metadata provider to insert the annotations. Do this as the first step after creating an instance of the Configuration class:
// Perform some test to verify that the current database is Postgres.
if (connectionString.startsWith("jdbc:postgresql:")) {
// Replace the metadata provider with our custom metadata provider.
MetadataProviderInjector reflectionManager = MetadataProviderInjector)cfg.getReflectionManager();
reflectionManager.setMetadataProvider(new UUIDTypeInsertingMetadataProvider(reflectionManager.getMetadataProvider()));
}
This custom metadata provider finds fields and methods of type UUID. If it finds one, it inserts an instance of the org.hibernate.annotations.Type annotation stating that the type should be "pg-uuid":
package nl.gmt.data;
import org.hibernate.annotations.Parameter;
import org.hibernate.annotations.Type;
import org.hibernate.annotations.common.reflection.AnnotationReader;
import org.hibernate.annotations.common.reflection.MetadataProvider;
import java.lang.annotation.Annotation;
import java.lang.reflect.AnnotatedElement;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
class UUIDTypeInsertingMetadataProvider implements MetadataProvider {
private final Map<AnnotatedElement, AnnotationReader> cache = new HashMap<>();
private final MetadataProvider delegate;
public UUIDTypeInsertingMetadataProvider(MetadataProvider delegate) {
this.delegate = delegate;
}
#Override
public Map<Object, Object> getDefaults() {
return delegate.getDefaults();
}
#Override
public AnnotationReader getAnnotationReader(AnnotatedElement annotatedElement) {
// This method is called a lot of times on the same element, so annotation
// readers are cached. We only cache our readers because the provider
// we delegate to also caches them.
AnnotationReader reader = cache.get(annotatedElement);
if (reader != null) {
return reader;
}
reader = delegate.getAnnotationReader(annotatedElement);
// If this element is a method that returns a UUID, or a field of type UUID,
// wrap the returned reader in a new reader that inserts the "pg-uuid" Type
// annotation.
boolean isUuid = false;
if (annotatedElement instanceof Method) {
isUuid = ((Method)annotatedElement).getReturnType() == UUID.class;
} else if (annotatedElement instanceof Field) {
isUuid = ((Field)annotatedElement).getType() == UUID.class;
}
if (isUuid) {
reader = new UUIDTypeInserter(reader);
cache.put(annotatedElement, reader);
}
return reader;
}
private static class UUIDTypeInserter implements AnnotationReader {
private static final Type INSTANCE = new Type() {
#Override
public Class<? extends Annotation> annotationType() {
return Type.class;
}
#Override
public String type() {
return "pg-uuid";
}
#Override
public Parameter[] parameters() {
return new Parameter[0];
}
};
private final AnnotationReader delegate;
public UUIDTypeInserter(AnnotationReader delegate) {
this.delegate = delegate;
}
#Override
#SuppressWarnings("unchecked")
public <T extends Annotation> T getAnnotation(Class<T> annotationType) {
if (annotationType == Type.class) {
return (T)INSTANCE;
}
return delegate.getAnnotation(annotationType);
}
#Override
public <T extends Annotation> boolean isAnnotationPresent(Class<T> annotationType) {
return annotationType == Type.class || delegate.isAnnotationPresent(annotationType);
}
#Override
public Annotation[] getAnnotations() {
Annotation[] annotations = delegate.getAnnotations();
Annotation[] result = Arrays.copyOf(annotations, annotations.length + 1);
result[result.length - 1] = INSTANCE;
return result;
}
}
}

Solution for someone who don't use JPA.
Before:
<property name="testId" >
<column name="test_id" sql-type="uuid" not-null="true"/>
</property>
After:
<property name="testId" column="test_id" type="org.hibernate.type.PostgresUUIDType">
</property>

I had a smililar issue while having Sprint Data and jsonb in postgres. Thank you Sri for the solution!
In the Model, replacing
#Type(type="pg-uuid")
with
#Type(type="org.hibernate.type.PostgresUUIDType")
solved the issue running JUnit Tests with #SpringBootTest.
Example in Entity (Kotlin):
#Type(type="org.hibernate.type.PostgresUUIDType")
#Column(
nullable = false,
unique = true,
updatable = false,
columnDefinition = "CHAR(36)"
)
var uuid: UUID = UUID.randomUUID()