We Keep Coding

CSV file dynamic map to Database Tables using Hibernate

We have a requirement where around 25 CSV files would come each day & stored the Database in equivalent table structure.
Any of CSV file column structure could change in future by add new /remove columns & underlying DB table would align to the new format, without code change or redeployment.
Here are the choice of tech.
SpringBoot as Run time
Hibernate as JPA/DB Inetraction
Oracle DB as database
If using Hibernate, how to achieving this dynamic column management of the table as per the incoming CSV?
As far as I know, Hibernate would have Java Entity classes equivalent to the Table , which will be used to persists data. Any table change need Entity class change too.
Possible solution could be
just define basic JPA Entity & table structure (like ids & FKs linking to other tables etc) for CSV equivalent table,
then on arrival of CSV files, add the columns to the table by running the ALTER table command from application
In future 1st CSVs, if column added/removed , use similar alter commands
Is this achievable by Hibernate?
Or any other product better suited for this kind of tasks.

Task definition
We will have to implement a mechanism allowing for creating/deleting custom fields in real time avoiding the application restart, add a value into it and make sure the value is present in the application database. Besides we will have to make sure that the custom field can be used in queries.
Solution
Domain Model
We will first need a business entity class which we will experiment with. Let is be Contact class. There will be two persistent fields: id and name.
However besides these permanent and unchangeable fields the class should be some sort of construction to store values of custom fields. Map would be an ideal construction for this.
Let's create a base class for all business entities supporting custom fields - CustomizableEntity, that contains Map CustomProperties to work with custom fields:
package com.enterra.customfieldsdemo.domain;
import java.util.Map;
import java.util.HashMap;
public abstract class CustomizableEntity {
private Map customProperties;
public Map getCustomProperties() {
if (customProperties == null)
customProperties = new HashMap();
return customProperties;
}
public void setCustomProperties(Map customProperties) {
this.customProperties = customProperties;
}
public Object getValueOfCustomField(String name) {
return getCustomProperties().get(name);
}
public void setValueOfCustomField(String name, Object value) {
getCustomProperties().put(name, value);
}
}
Step 1 - base class CustomizableEntity
Inherit our class Contact from this base class:
package com.enterra.customfieldsdemo.domain;
import com.enterra.customfieldsdemo.domain.CustomizableEntity;
public class Contact extends CustomizableEntity {
private int id;
private String name;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
Step 2 - Class Contact inherited from CustomizableEntity.
We should not forget about the mapping file for this class:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
<hibernate-mapping auto-import="true" default-access="property" default-cascade="none" default-lazy="true">
<class abstract="false" name="com.enterra.customfieldsdemo.domain.Contact" table="tbl_contact">
<id column="fld_id" name="id">
<generator class="native"/>
</id>
<property name="name" column="fld_name" type="string"/>
<dynamic-component insert="true" name="customProperties" optimistic-lock="true" unique="false" update="true">
</dynamic-component>
</class>
</hibernate-mapping>
Step 3 - Mapping Class Contact.
Please note that properties id and name are done as all ordinary properties, however for customProperties we use a tag . Documentation on Hibernate 3.2.0GA says that the point of a dynamic-component is:
"The semantics of a mapping are identical to . The advantage of this kind of mapping is the ability to determine the actual properties of the bean at deployment time, just by editing the mapping document. Runtime manipulation of the mapping document is also possible, using a DOM parser. Even better, you can access (and change) Hibernate's configuration-time metamodel via the Configuration object."
Based on this regulation from Hibernate documentation we will be building this function mechanism.
HibernateUtil and hibernate.cfg.xml
After we are defined with the domain model of our application we have to create necessary conditions for Hibernate framework functioning. For this we have to create a configuration file hibernate.cfg.xml and class to work with the core Hibernate functions.
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE hibernate-configuration
PUBLIC "-//Hibernate/Hibernate Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="show_sql">true</property>
<property name="dialect">
org.hibernate.dialect.MySQLDialect</property>
<property name="cglib.use_reflection_optimizer">true</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/custom_fields_test</property>
<property name="hibernate.connection.username">root</property>
<property name="hibernate.connection.password"></property>
<property name="hibernate.c3p0.max_size">50</property>
<property name="hibernate.c3p0.min_size">0</property>
<property name="hibernate.c3p0.timeout">120</property>
<property name="hibernate.c3p0.max_statements">100</property>
<property name="hibernate.c3p0.idle_test_period">0</property>
<property name="hibernate.c3p0.acquire_increment">2</property>
<property name="hibernate.jdbc.batch_size">20</property>
<property name="hibernate.hbm2ddl.auto">update</property>
</session-factory>
</hibernate-configuration>
Step 4 - Hibernate configuration file.
The file hibernate.cfg.xml does not contain anything noticeable except for this string:
<property name="hibernate.hbm2ddl.auto">update</property>
Step 5 - using auto-update.
Later we will explain in details on its purpose and tell more how we can go without it. There are several ways to implement class HibernateUtil. Our implementation will differ a bit from well known due to changes into Hibernate configuration.
package com.enterra.customfieldsdemo;
import org.hibernate.*;
import org.hibernate.mapping.PersistentClass;
import org.hibernate.tool.hbm2ddl.SchemaUpdate;
import org.hibernate.cfg.Configuration;
import com.enterra.customfieldsdemo.domain.Contact;
public class HibernateUtil {
private static HibernateUtil instance;
private Configuration configuration;
private SessionFactory sessionFactory;
private Session session;
public synchronized static HibernateUtil getInstance() {
if (instance == null) {
instance = new HibernateUtil();
}
return instance;
}
private synchronized SessionFactory getSessionFactory() {
if (sessionFactory == null) {
sessionFactory = getConfiguration().buildSessionFactory();
}
return sessionFactory;
}
public synchronized Session getCurrentSession() {
if (session == null) {
session = getSessionFactory().openSession();
session.setFlushMode(FlushMode.COMMIT);
System.out.println("session opened.");
}
return session;
}
private synchronized Configuration getConfiguration() {
if (configuration == null) {
System.out.print("configuring Hibernate ... ");
try {
configuration = new Configuration().configure();
configuration.addClass(Contact.class);
System.out.println("ok");
} catch (HibernateException e) {
System.out.println("failure");
e.printStackTrace();
}
}
return configuration;
}
public void reset() {
Session session = getCurrentSession();
if (session != null) {
session.flush();
if (session.isOpen()) {
System.out.print("closing session ... ");
session.close();
System.out.println("ok");
}
}
SessionFactory sf = getSessionFactory();
if (sf != null) {
System.out.print("closing session factory ... ");
sf.close();
System.out.println("ok");
}
this.configuration = null;
this.sessionFactory = null;
this.session = null;
}
public PersistentClass getClassMapping(Class entityClass){
return getConfiguration().getClassMapping(entityClass.getName());
}
}
Step 6 - HibernateUtils class.
Alongside with usual methods like getCurrentSession(), getConfiguration(), which is necessary for regular work of the application based on Hibernate, we also have implemented such methods as: reset() and getClassMapping(Class entityClass). In the method getConfiguration(), we configure Hibernate and add class Contact into the configuration.
Method reset() has been used to close all used by Hibernate resources and clearing all of its settings:
public void reset() {
Session session = getCurrentSession();
if (session != null) {
session.flush();
if (session.isOpen()) {
System.out.print("closing session ... ");
session.close();
System.out.println("ok");
}
}
SessionFactory sf = getSessionFactory();
if (sf != null) {
System.out.print("closing session factory ... "); sf.close();
System.out.println("ok");
}
this.configuration = null;
this.sessionFactory = null;
this.session = null;
}
Step 7 - method reset()
Method getClassMapping(Class entityClass) returns object PersistentClass, that contains full information on mapping the related entity. In particular the manipulations with the object PersistentClass allow modifying the set of attributes of the entity class in the run-time.
public PersistentClass getClassMapping(Class entityClass){
return
getConfiguration().getClassMapping(entityClass.getName());
}
Step 8 - method getClassMapping(Class entityClass).
Manipulations with mapping
Once we have the business entity class (Contact) available and the main class to interact with Hibernate we can start working. We can create and save samples of the Contact class. We can even place some data into our Map customProperties, however we should be aware that this data (stored in Map customProperties) are not saved to the DB.
To have the data saved we should provide for the mechanism of creating custom fields in our classs and make it the way Hibernate knows how to work with them.
To provide for class mapping manipulation we should create some interface. Let's call it CustomizableEntityManager. Its name should reflect the purpose of the interface managing a business entity, its contents and attributes:
package com.enterra.customfieldsdemo;
import org.hibernate.mapping.Component;
public interface CustomizableEntityManager {
public static String CUSTOM_COMPONENT_NAME = "customProperties";
void addCustomField(String name);
void removeCustomField(String name);
Component getCustomProperties();
Class getEntityClass();
}
Step 9 - Interface CustomizableEntityManager
The main methods for the interface are: void addCustomField(String name) and void removeCustomField(String name). These should created and remove our custom field in the mapping of the corresponding class.
Below is the way to implement the interface:
package com.enterra.customfieldsdemo;
import org.hibernate.cfg.Configuration;
import org.hibernate.mapping.*;
import java.util.Iterator;
public class CustomizableEntityManagerImpl implements CustomizableEntityManager {
private Component customProperties;
private Class entityClass;
public CustomizableEntityManagerImpl(Class entityClass) {
this.entityClass = entityClass;
}
public Class getEntityClass() {
return entityClass;
}
public Component getCustomProperties() {
if (customProperties == null) {
Property property = getPersistentClass().getProperty(CUSTOM_COMPONENT_NAME);
customProperties = (Component) property.getValue();
}
return customProperties;
}
public void addCustomField(String name) {
SimpleValue simpleValue = new SimpleValue();
simpleValue.addColumn(new Column("fld_" + name));
simpleValue.setTypeName(String.class.getName());
PersistentClass persistentClass = getPersistentClass();
simpleValue.setTable(persistentClass.getTable());
Property property = new Property();
property.setName(name);
property.setValue(simpleValue);
getCustomProperties().addProperty(property);
updateMapping();
}
public void removeCustomField(String name) {
Iterator propertyIterator = customProperties.getPropertyIterator();
while (propertyIterator.hasNext()) {
Property property = (Property) propertyIterator.next();
if (property.getName().equals(name)) {
propertyIterator.remove();
updateMapping();
return;
}
}
}
private synchronized void updateMapping() {
MappingManager.updateClassMapping(this);
HibernateUtil.getInstance().reset();
// updateDBSchema();
}
private PersistentClass getPersistentClass() {
return HibernateUtil.getInstance().getClassMapping(this.entityClass);
}
}
Step 10 - implementing interface CustomizableEntityManager
First of all we should point out that when creating class CustomizableEntityManager we specify the business entity class the manager will operate. This class is passed as a parameter to designer CustomizableEntityManager:
private Class entityClass;
public CustomizableEntityManagerImpl(Class entityClass) {
this.entityClass = entityClass;
}
public Class getEntityClass() {
return entityClass;
}
Step 11 - class designer CustomizableEntityManagerImpl
Now we should get more interested in how to implement method void addCustomField(String name):
public void addCustomField(String name) {
SimpleValue simpleValue = new SimpleValue();
simpleValue.addColumn(new Column("fld_" + name));
simpleValue.setTypeName(String.class.getName());
PersistentClass persistentClass = getPersistentClass();
simpleValue.setTable(persistentClass.getTable());
Property property = new Property();
property.setName(name);
property.setValue(simpleValue);
getCustomProperties().addProperty(property);
updateMapping();
}
Step 12 - creating custom field.
As we can see from the implementation, Hibernate offers more options in working with properties of persistent objects and their representation in the DB. As per the essence of the method:
1) We create class SimpleValue that allow us to denote how the value of this custom field will be stored in the DB in which field and table of the DB:
SimpleValue simpleValue = new SimpleValue();
simpleValue.addColumn(new Column("fld_" + name));
simpleValue.setTypeName(String.class.getName());
PersistentClass persistentClass = getPersistentClass();
simpleValue.setTable(persistentClass.getTable());
Step 13 - creating new column of the table.
2) We create a property of the persistent object and add a dynamic component into it (!), that we have planned to be used for this purpose:
Property property = new Property()
property.setName(name)
property.setValue(simpleValue)
getCustomProperties().addProperty(property)
Step 14 - creating object property.
3) And finally we should make our application perform certain changes in the xml files and update the Hibernate configuration. This can be done via method updateMapping();
It is necessary to clarify the purpose of another two get-methods which have been used in the code above. The first method is getCustomProperties():
public Component getCustomProperties() {
if (customProperties == null) {
Property property = getPersistentClass().getProperty(CUSTOM_COMPONENT_NAME);
customProperties = (Component) property.getValue();
}
return customProperties;
}
Step 15 - getting CustomProperties as Component.
This method finds and returns object Component corresponding to the tag in the mapping of our business entity.
The second method is updateMapping():
private synchronized void updateMapping() {
MappingManager.updateClassMapping(this);
HibernateUtil.getInstance().reset();
// updateDBSchema();
}
Step 16 - method updateMapping().
The method is in charge for storing the updated mapping of the persistent class and updates the configuration status of Hibernate to make further changes that we make valid when the changes take effect.
By the way we should get back to the string:
<property name="hibernate.hbm2ddl.auto">update</property>
of the Hibernate configuration. If this string was missing we would have to launch executing updates of the DB schema using hibernate utilities. However using the setting allows us to avoid this.
Saving mapping
Modifications to mapping made in run-time do not save by themselves into the corresponding xml mapping file and to make the changes to get activated at next launch of the application we need to manually save changes to the corresponding mapping file.
To do this we will be using class MappingManager the main purpose of which is to save mapping of the designated business entity to its xml mapping file:
package com.enterra.customfieldsdemo;
import com.enterra.customfieldsdemo.domain.CustomizableEntity;
import org.hibernate.Session;
import org.hibernate.mapping.Column;
import org.hibernate.mapping.Property;
import org.hibernate.type.Type;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import java.util.Iterator;
public class MappingManager {
public static void updateClassMapping(CustomizableEntityManager entityManager) {
try {
Session session = HibernateUtil.getInstance().getCurrentSession();
Class<? extends CustomizableEntity> entityClass = entityManager.getEntityClass();
String file = entityClass.getResource(entityClass.getSimpleName() + ".hbm.xml").getPath();
Document document = XMLUtil.loadDocument(file);
NodeList componentTags = document.getElementsByTagName("dynamic-component");
Node node = componentTags.item(0);
XMLUtil.removeChildren(node);
Iterator propertyIterator = entityManager.getCustomProperties().getPropertyIterator();
while (propertyIterator.hasNext()) {
Property property = (Property) propertyIterator.next();
Element element = createPropertyElement(document, property);
node.appendChild(element);
}
XMLUtil.saveDocument(document, file);
} catch (Exception e) {
e.printStackTrace();
}
}
private static Element createPropertyElement(Document document, Property property) {
Element element = document.createElement("property");
Type type = property.getType();
element.setAttribute("name", property.getName());
element.setAttribute("column", ((Column)
property.getColumnIterator().next()).getName());
element.setAttribute("type",
type.getReturnedClass().getName());
element.setAttribute("not-null", String.valueOf(false));
return element;
}
}
Step 17 - the utility to update mapping of the persistent class.
The class literally performs the following:
Defines a location and loads xml mapping for the designated business entity into the DOM Document object for further manipulations with it;
Finds the element of this document . In particular here we store the custom fields and its contents we change;
Delete (!) all embedded elements from this element;
For any persistent property contained in our component that is in charge for the custom fields storage, we create a specific document element and define attributes for the element from the corresponding property;
Save this newly created mapping file.
When manipulating XML we use (as we can see from the code) class XMLUtil, that in general can be implemented in any way though it should correctly load and save the xml file.
Our implementation is given at the Step below:
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.w3c.dom.Document;
import org.xml.sax.SAXException;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.dom.DOMSource;
import java.io.IOException;
import java.io.FileOutputStream;
public class XMLUtil {
public static void removeChildren(Node node) {
NodeList childNodes = node.getChildNodes();
int length = childNodes.getLength();
for (int i = length - 1; i > -1; i--)
node.removeChild(childNodes.item(i));
}
public static Document loadDocument(String file)
throws ParserConfigurationException, SAXException, IOException {
DocumentBuilderFactory factory =DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
return builder.parse(file);
}
public static void saveDocument(Document dom, String file)
throws TransformerException, IOException {
TransformerFactory tf = TransformerFactory.newInstance();
Transformer transformer = tf.newTransformer();
transformer.setOutputProperty(OutputKeys.DOCTYPE_PUBLIC, dom.getDoctype().getPublicId());
transformer.setOutputProperty(OutputKeys.DOCTYPE_SYSTEM, dom.getDoctype().getSystemId());
DOMSource source = new DOMSource(dom);
StreamResult result = new StreamResult();
FileOutputStream outputStream = new FileOutputStream(file);
result.setOutputStream(outputStream);
transformer.transform(source, result);
outputStream.flush();
outputStream.close();
}
}
Source: Please refer this article for more detail

Hibernate : Objects are getting saved without transaction Object

I am having following piece of code for saving the object in the database,
import java.util.HashMap;
import java.util.Map;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import org.hibernate.Session;
import org.hibernate.Transaction;
import model.Hibernatetest;
public class Test
{
static EntityManagerFactory objEntityManagerFactory;
static Session objSession;
static Transaction objTransaction;
public static void main(String[] args)
{
objSession = (Session)
objEntityManagerFactory.createEntityManager().getDelegate();
//objTransaction = objSession.getTransaction();
//objTransaction.begin();
Hibernatetest obj = new Hibernatetest();
obj.setName("Nobal");
obj.setAddress("wlfjegtjwdfhdg");
objSession.save(obj);
obj.setName("235611111");
objSession.flush();
//objTransaction.commit();
objSession.close();
}
public static void getConnection()
{
Map<String,String> properties = new HashMap<String,String>();
properties.put("hibernate.connection.username", "root");
properties.put("hibernate.connection.password", "root");
properties.put("hibernate.connection.driver_class", "com.mysql.jdbc.Driver");
properties.put("hibernate.connection.url", "jdbc:mysql://localhost:3306/hibernatetest");
properties.put("hibernate.dialect", "org.hibernate.dialect.MySQLDialect");
objEntityManagerFactory = Persistence.createEntityManagerFactory("Hibereg", properties);
}
}
and the Persistence file is as follows:
<persistence xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/persistence" version="2.0">
<persistence-unit name="Hibereg">
<class>entities.Hibernatetest</class>
</persistence-unit>
</persistence>
Case 1:
The problem is when I run the above code using the hibernate-core-5.2.8.jar, and don't create any transaction object(you can see I commented the code related to transaction) then the above code gives me following exception.
'TransactionRequiredException' no transaction is in progress.'
However if I uncomment the code for the transaction, everything works well.
Case 2:
But when I run the code using hibernate-core-4.1.4.jar, by commenting the code for the transaction in code snippet provided above, It gives no such exception. That is , it never complains for the absence of transaction object and saves the object in the database..
What is going on in here?
I read over internet that transaction object is must for performing inserts and updates, (I am not sure about selects) but how the second scenario is executing successfully?

Since Hibernate 5.2 this functionality is included inline to JPA specification that does not allow any update flush outside of an transaction boundary. In order to override it pls use following on your hibernate settings:-
properties.put("hibernate.allow_update_outside_transaction", "true");

Hibernate performance tuning tips

I'm developing a small application, which consists of a single table.
I am using technologies are:
NetBeans 8.1
Java 8
Hibernate 4.3.x
Informix
Primefaces 5
I had to investigate a time to connect with Informix Hibernate, but I got it, and the application displays the list with the requested data correctly.
The problem arises with the performance of Hibernate, which is very poor, especially considering that the table contains only 36000 records.
On each page change takes about 6 or 7 seconds.
I have been researching in the official documentation of Hibernate, but can not find concrete examples to improve performance.
Herewith the application code:
hibernate.cfg.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="hibernate.dialect">org.hibernate.dialect.InformixDialect</property>
<property name="hibernate.connection.driver_class">com.informix.jdbc.IfxDriver</property>
<property name="hibernate.connection.url">jdbc:informix-sqli://127.0.0.1:1526/chicho:INFORMIXSERVER=ol_chicho</property>
<!--<property name="hibernate.connection.datasource">jdbc/votacion</property>-->
<property name="hibernate.connection.username">informix</property>
<property name="hibernate.connection.password">informix</property>
<property name="hibernate.connection.autocommit">true</property>
<property name="hibernate.current_session_context_class">thread</property>
<property name="hibernate.default_schema">informix</property>
<property name="hibernate.show_sql">true</property>
<property name="hibernate.hbm2ddl.auto">validate</property>
<property name="hibernate.cache.use_second_level_cache">true</property>
<mapping resource="pojos/Xxpuedenvotar1.hbm.xml"/>
</session-factory>
Pojo:
package pojos;
// Generated 23/08/2016 22:07:42 by Hibernate Tools 4.3.1
/**
* Xxpuedenvotar1 generated by hbm2java
*/
public class Xxpuedenvotar1 implements java.io.Serializable {
private Integer nroaccionista;
private Short estado;
private Integer idcliente;
private String razonSocial;
private Short idlocalidad;
private Short zona;
private String calle;
private String puerta;
private String localidad;
public Xxpuedenvotar1() {
}
public Xxpuedenvotar1(Integer nroaccionista) {
this.nroaccionista = nroaccionista;
}
public Xxpuedenvotar1(Integer nroaccionista, Short estado, Integer idcliente, String razonSocial, Short idlocalidad, Short zona, String calle, String puerta, String localidad) {
this.nroaccionista = nroaccionista;
this.estado = estado;
this.idcliente = idcliente;
this.razonSocial = razonSocial;
this.idlocalidad = idlocalidad;
this.zona = zona;
this.calle = calle;
this.puerta = puerta;
this.localidad = localidad;
}
public Integer getNroaccionista() {
return this.nroaccionista;
}
public void setNroaccionista(Integer nroaccionista) {
this.nroaccionista = nroaccionista;
}
public Short getEstado() {
return this.estado;
}
public void setEstado(Short estado) {
this.estado = estado;
}
public Integer getIdcliente() {
return this.idcliente;
}
public void setIdcliente(Integer idcliente) {
this.idcliente = idcliente;
}
public String getRazonSocial() {
return this.razonSocial;
}
public void setRazonSocial(String razonSocial) {
this.razonSocial = razonSocial;
}
public Short getIdlocalidad() {
return this.idlocalidad;
}
public void setIdlocalidad(Short idlocalidad) {
this.idlocalidad = idlocalidad;
}
public Short getZona() {
return this.zona;
}
public void setZona(Short zona) {
this.zona = zona;
}
public String getCalle() {
return this.calle;
}
public void setCalle(String calle) {
this.calle = calle;
}
public String getPuerta() {
return this.puerta;
}
public void setPuerta(String puerta) {
this.puerta = puerta;
}
public String getLocalidad() {
return this.localidad;
}
public void setLocalidad(String localidad) {
this.localidad = localidad;
}
}
DAO:
package Dao;
import Interfaces.InterfazSocios;
import java.util.List;
import org.hibernate.Query;
import org.hibernate.Session;
import pojos.Xxpuedenvotar1;
/**
*
* #author Gustavo
*/
public class DaoSocios implements InterfazSocios {
private List<Xxpuedenvotar1> listaSocios;
#Override
public List<Xxpuedenvotar1> verTodos(Session sesion) throws Exception {
String hql = "FROM Xxpuedenvotar1 ORDER BY NroAccionista";
//Query consulta = sesion.createQuery(hql).setCacheable(true);
this.listaSocios = sesion.createCriteria(Xxpuedenvotar1.class).list();
//this.listaSocios = consulta.list();
return this.listaSocios;
}
}
I think with these files is sufficient for analysis, since the application works well, with the exception of its slowness.
Thank in advance for your kind attention.

First of all, there are many things you can do to speed up Hibernate. Check out these High-Performance Hibernate Tips, or this High-Performance Hibernate video presentation.
Now, back to your question. You are using the DriverManagerConnectionProvider which only provides a rudimentary connection pooling solution. Better use HikariCP since it's the fastest one, and it's also available via the hibernate-hikaricp Maven dependency.
Related to your statement:
The problem arises with the performance of Hibernate, which is very
poor, especially considering that the table contains only 36000
records.
On each page change takes about 6 or 7 seconds.
Why would you want to fetch 36k records in a single shot?
You can't display them into a UI? That's why we have data pagination after all.
Even for a batch processor, you are better off splitting the whole workload into multiple smaller data sets that allow you to avoid long-running transactions, and even split the load among multiple worker threads.

Can try any of these or all of these depending on the configuration, availability of modifying the configuration properties.
hibernate.show_sql mark this to false, once you are done with your analysis of query formation. This takes time to log.
You can change connection pool size configuration, if you are using one
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
This helps in attaining the connection quickly from the pool. Avoids time to get the connection.
If you have huge data, Please mark the respective column in JPA class with #OrderBy.
Hibernate search provides #SortableField to annotate the field, This indexes internally and helps in retrieving the sorted results.
[EDIT]
If Integer nroaccionista is primary key, it is indexed by default. Mark the column in JPA class with #Id and the DDL creation statement generated with create the index by default. Indexed column helps in better search and sort results.
Hope this helps.

jdbc update query no working on unit test with derby database and SpringJUnit4ClassRunner

I am having problems trying to do an unit test.
The test class is simple like:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {"/job-runner-context.xml})
public class AtualizarDataServiceTest {
#Autowired
DataSource testDataSource;
#Autowired
Service service;
#Before
public void setUp() throws DatabaseUnitException, SQLException{
service.setDataSource(testDataSource);
}
#Test
public final void testUpdateDate() throws SQLException {
assertTrue(verifyDate());
service.updateDate();
assertFalse(verifyDate()); //Assert brokes here
}
private boolean verifyDate(){
SimpleJdbcTemplate consultaTemplate = new SimpleJdbcTemplate(testDataSource);
int count = consultaTemplate.queryForInt("SELECT COUNT(*) FROM MY_TABLE");
return count == 0;
}
}
The service:
public class Service {
private DataSource dataSource;
public void updateDate(){
SimpleJdbcTemplate jdbcTemplate = new SimpleJdbcTemplate(getDataSource());
String query = "UPDATE MY_TABLE SET DT_UPDATE_OPERATION = ?";
jdbcTemplate.update(query, new Object[]{new java.sql.Date(Calendar.getInstance().getTime().getTime())});
}
public DataSource getDataSource() {
return dataSource;
}
public void setDataSource(DataSource dataSource) {
this.dataSource = dataSource;
}
}
job-runner-context.xml important pieces:
<context:annotation-config/>
<context:component-scan base-package="my.package"/>
<bean class="my.package.Service"/>
<bean id="testDataSource" class="com.read.only.MyBasicDataSource" destroy-method="close" lazy-init="true">
<property name="jdbcReference" value="derby" />
</bean>
the jdbc connection properties:
<com:jdbcReference name="derby" type="DATABASE">
<com:credential user="" password="" />
<com:autocommit arg="false" />
<com:databaseConfig driverClassName="org.apache.derby.jdbc.EmbeddedDriver"
url="jdbc:derby:dbderby;create=true" databaseName="ANY" />
</com:jdbcReference>
At fist I thought the problem was related to commit issues but I tried set the value of autocommit properties to "true" and also manually call testDataSource.getConnection().commit() but it didn't work. The code and methods are working fine but the test isn't updating the derby database. In other test classes data is preset with dbUnit in the same database and the code works. This answer here gives an general list of possible problems I checked and I am reading and writing to the same tables in the same schemas. Am I missing something?

Try setup <com:autocommit arg="true" />.

As the answer to the question I posted I verified the autocommit and if I was writing to the right database, but didn't check the obvious: you can't update a table with no registers! The query UPDATE MY_TABLE SET DT_UPDATE_OPERATION = ? was applied to an empty table and the count query would always return 0. I just configured the test to make DbUnit import an state to the database from a xml file. Sorry for the trouble.

How to fetch a <map> in Hibernate

Update: I've created an example on GitHub to demonstrate my problem; HibernateMapTest currently fails due to the fact the HashMap key is a proxy object. I'm hoping someone can suggest a way I can query for the entity and fetch the map so that the test passes...
I'm simply trying to fetch the contents of a HashMap persisted in Hibernate, but I'm having some trouble finding the correct way to do it...
The HBM mapping is as follows, I did not create this but from my research it appears to be a ternary association mapping with a many-to-many relation. (Update: to simplify my question I've forced the map to lazy="false" to avoid my join):
<hibernate-mapping>
<class name="database.Document" table="document">
...
<map name="documentbundles" table="document_bundles" lazy="false">
<key column="id"/>
<index-many-to-many column="pkgitemid" class="database.PkgItem"/>
<many-to-many column="child" class="database.Document" />
</map>
</class>
</hibernate-mapping>
For simplicity I'm just currently just attempting to fetch all the records with this map data populated:
DetachedCriteria criteria = DetachedCriteria.forClass(Document.class);
criteria.add(Restrictions.eq("id", 1));
List<Document> result = hibernateTemplate.findByCriteria(criteria);
After last to false, I now get the contents of the Map without throwing a LazyInitializationException; but none of the key objects have been initialised properly. I've dumped a screenshot to clarify what I mean:
I know that the fields are populated in the database, and I suspect my fetching strategy is still to blame. How do you fetch a <map> in Hibernate correctly?

The error is due to the HibernateTemplate opening a Hibernate session to execute this query:
List results = hibernateTemplate.find("from database.Document d where d.name = 'doc1'");
and then immediately closing the session after the query is run. Then when looping through the keys, the session to which the map was linked is closed so the data cannot be loaded anymore, causing the proxy to throw the LazyInitializationException.
This exception means that the proxy can no longer load the data transparently because the session to which is linked too is now closed.
One of the main goals of the HibernateTemplate is to know when to open and close sessions. The template will keep the session open if there is an ongoing transaction.
So the key here is to wrap the unit test in a TransactionTemplate (the template equivalent to #Transactional), which causes the session to be kept open by the HibernateTemplate. Because the session is kept open, no more lazy initialization exceptions occur.
Modifying the test like this will solve the problem (notice the use of TransactionTemplate):
import database.Document;
import database.PkgItem;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.orm.hibernate3.HibernateTemplate;
import org.springframework.orm.hibernate3.HibernateTransactionManager;
import org.springframework.orm.hibernate3.LocalSessionFactoryBean;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.TransactionStatus;
import org.springframework.transaction.support.TransactionCallbackWithoutResult;
import org.springframework.transaction.support.TransactionTemplate;
import java.util.HashMap;
import java.util.List;
import java.util.Set;
public class HibernateMapTest {
private static final String TEST_DIALECT = "org.hibernate.dialect.HSQLDialect";
private static final String TEST_DRIVER = "org.hsqldb.jdbcDriver";
private static final String TEST_URL = "jdbc:hsqldb:mem:adportal";
private static final String TEST_USER = "sa";
private static final String TEST_PASSWORD = "";
private HibernateTemplate hibernateTemplate;
private TransactionTemplate transactionTemplate;
#Before
public void setUp() throws Exception {
hibernateTemplate = new HibernateTemplate();
LocalSessionFactoryBean sessionFactory = new LocalSessionFactoryBean();
sessionFactory.getHibernateProperties().put("hibernate.dialect", TEST_DIALECT);
sessionFactory.getHibernateProperties().put("hibernate.connection.driver_class", TEST_DRIVER);
sessionFactory.getHibernateProperties().put("hibernate.connection.password", TEST_PASSWORD);
sessionFactory.getHibernateProperties().put("hibernate.connection.url", TEST_URL);
sessionFactory.getHibernateProperties().put("hibernate.connection.username", TEST_USER);
sessionFactory.getHibernateProperties().put("hibernate.hbm2ddl.auto", "create");
sessionFactory.getHibernateProperties().put("hibernate.show_sql", "true");
sessionFactory.getHibernateProperties().put("hibernate.jdbc.batch_size", "0");
sessionFactory.getHibernateProperties().put("hibernate.cache.use_second_level_cache", "false");
sessionFactory.setMappingDirectoryLocations(new Resource[]{new ClassPathResource("database")});
sessionFactory.afterPropertiesSet();
hibernateTemplate.setSessionFactory(sessionFactory.getObject());
transactionTemplate = new TransactionTemplate(new HibernateTransactionManager(sessionFactory.getObject()));
}
#After
public void tearDown() throws Exception {
hibernateTemplate.getSessionFactory().close();
}
#Test
public void testFetchEntityWithMap() throws Exception {
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
protected void doInTransactionWithoutResult(TransactionStatus status) {
// Store the entities and mapping
PkgItem key = new PkgItem();
key.setName("pkgitem1");
hibernateTemplate.persist(key);
Document doc2 = new Document();
doc2.setName("doc2");
hibernateTemplate.persist(doc2);
Document doc1 = new Document();
doc1.setName("doc1");
HashMap<PkgItem, Document> documentbundles = new HashMap<PkgItem, Document>();
documentbundles.put(key, doc2);
doc1.setDocumentbundles(documentbundles);
hibernateTemplate.persist(doc1);
// Now attempt a query
List results = hibernateTemplate.find("from database.Document d where d.name = 'doc1'");
Document result = (Document)results.get(0);
// Check the doc was returned
Assert.assertEquals("doc1", result.getName());
key = (PkgItem)hibernateTemplate.find("from database.PkgItem").get(0);
Set<PkgItem> bundleKeys = result.getDocumentbundles().keySet();
// Check the key is still present in the map. At this point the test fails because
// the map contains a proxy object of the key...
Assert.assertEquals(key, bundleKeys.iterator().next());
}
});
}
}
and these are the test results and the log after the change:

This is a supplement to jhadesdev's answer, since I needed to do a little more work to get exactly what I was looking for.
In summary, you can't fetch a PersistedMap with a Hibernate query and immediately start using it like a typical Java hash map. The keys are always proxies; eager fetching / joining only fetches the map values, not the keys.
This means any code that deals with the hash map needs to be wrapped in a Hibernate transaction, which caused me some architectural problems as my data and service layers are separate.
I worked around this by iterating the hash map within a single transaction and replacing the keys with the ones originally passed in. I kept performance up by batching up the keys I want to fetch and retrieving them in one go:
// Build a list of keys we want to fetch in one go
final List<PkgItem> pkgItems = Arrays.asList(pkgItem1, pkgItem2, ...);
Map<PkgItem, Document> bundles = transactionTemplate.execute(new TransactionCallback< Map<PkgItem, Document> >() {
#Override
public Map<PkgItem, Document> doInTransaction(TransactionStatus transactionStatus) {
if (doc1.getId() == null) return null;
// Merge the parent document into this transaction
Document container = hibernateTemplate.merge(doc1);
// Copy the original package items into the key set
Map<PkgItem, Document> out = new HashMap<PkgItem, Document>();
for (PkgItem dbKey : container.getDocumentbundles().keySet()) {
int keyIndex = pkgItems.indexOf(dbKey);
if (keyIndex > -1) out.put(pkgItems.get(keyIndex), container.getDocumentbundles().get(dbKey));
}
return out;
}
});
// Now we can perform a standard lookup
assertEquals("doc2", result.get(pkgItem1).getName());
I can now use the map without Hibernate in the resulting code, with only a minimal performance hit. I've also updated the test in my example GitHub project to demonstrate how this can work.