In one of our projects user can attach file to his account. We store these files in the MS-SQL database. So, we have the following piece of code:
#Entity
public class File extends AbstractEntity {
#Lob
#Basic
private byte[] data;
#Nullable
public byte[] getData() {
return data;
}
public void setData(byte[] data) {
this.data = data;
}
public File() {
}
public File(byte[] data) {
this.data = data;
}
}
public class SomeBean {
#PersistenceContext
protected EntityManager em;
public Long uploadFile(#NotNull byte[] data) {
final PhysicalFile physicalFile = new PhysicalFile();
physicalFile.setData(data);
em.persist(physicalFile);
return physicalFile.getId();
}
}
And all was nice and pretty, before we tried to upload 40 MB file, and got java.lang.RuntimeException: javax.transaction.RollbackException: [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] Can't commit because the transaction is in aborted state, which was caused by java.lang.OutOfMemoryError: Java heap space inside the uploadFile() method.
I made a heap dump and looked at it in VisualVM.
400+ MB of char[] and 100+ MB of byte[]. On the start our application, including JBoss, was using something about 60-65 MB of heap. So, the question is, why EntityManager consumes heap memory like crazy?
My understaning to your issue as follows.
All the entities that load/persist through an EntityManager stay in memory until you explicitly detach the entities from it (via EntityManager.detach() or EntityManager.clear() or EntityManager.close()). So it's better to have short-lived EntityManagers.
As far as a RuntimeException occurs in the business logic, the em
EntityManager remains open! You'll always want to avoid this sort of
code. you can think of creating and closing the EntityManager as
follows:
public Customer getBestCustomerOfMonth() {
EntityManagerFactory emf = ... ;
EntityManager em = emf.createEntityManager();
// business logic
em.close();
}
You can nest the line for closing the EntityManager em.close(); inside a finally
block
when using transactions outside an enterprise application server
because you'll have to close (commit or rollback) the transaction in
the same way you do for EntityMangers. In order for these resources
(both EntityManager and underlying transaction) to be closed you'll
need to make an additional level of nesting and write your code
similar to this one:
public Customer updateCustomer(Customer cust) {
EntityManagerFactory emf = ... ; EntityManager em =
emf.createEntityManager(); try {
EntityTransaction t = em.getTransaction();
try {
t.begin();
// business logic to update the customer
em.merge(cust);
t.commit();
} finally {
if (t.isActive()) t.rollback();
} } finally {
em.close();
}
}
You may think this nested structure could looks like a bit of a mess, but it is really needed in precence of transactions.
Related
While I have Java Batch jobs that read data, process it and store it in other places in the database, now I need a step to actually remove data from the database. All I need to run is a delete query via JPA.
The chunk based Reader/Processor/Writer pattern does not make sense here. But the Batchlet alternative is giving me a headache either. What did I do?
I created a Batchlet that gets invoked via CDI. At that moment it is easy to inject my JPA EntityManager. What is not easy is to run the update query. Code looks like this:
package ...;
import javax.batch.api.BatchProperty;
import javax.inject.Inject;
import javax.inject.Named;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
#Named("CleanerBatchlet")
public class CleanerBatchlet extends AbstractBatchlet {
public static final Logger log = LogManager.getLogger(CleanerBatchlet.class);
#PersistenceContext(unitName = "...")
private EntityManager entityManager;
#Inject
#BatchProperty(name = "technologyIds")
private String technologyIds;
private void clearQueue(long technologyId) {
//EntityManager entityManager = ...getEntityManager();
//entityManager.getTransaction().begin();
Query q = entityManager.createQuery("delete from Record r where r.technologyId=:technologyId");
q.setParameter("technologyId", technologyId);
int count = q.executeUpdate();
//entityManager.getTransaction().commit();
log.debug("Deleted {} entries from queue {}", count, technologyId);
//entityManager.close();
}
#Override
public String doProcess() throws Exception {
log.debug("doProcess()");
out.println("technologyIds=" + technologyIds);
log.info("technologyIds=" + technologyIds);
try {
String[] parts = technologyIds.split(",");
for (String part: parts) {
long technologyId = Long.parseLong(part);
clearQueue(technologyId);
}
} catch (NullPointerException | NumberFormatException e) {
throw new IllegalStateException("technologyIds must be set to a string of comma-separated numbers.", e);
}
return "COMPLETED";
}
}
As you can see some lines are commented out - these are the ones I am experimenting with.
So if I run the code as-is, I get an exception telling me that the update query requires a transaction. This is regardless of which of the two persistence units in my project I use (one is configured for JTA, the other is not).
javax.persistence.TransactionRequiredException: Executing an update/delete query
It also does not matter whether I uncomment the transaction handling code begin/commit. I still get the same error that a transaction is required to run the update query.
Even when I try to circumvent CDI and JTA completely by creating my own EntityManager via the Persistence API (and close it afterwards, respectively) I do get the very same exception.
So how can I run this delete query or other update queryies from within the batch job?
I'd suggest using plain jdbc to run this delete query, with either auto commit or manual transaction commit.
During the batchlet processing, the incoming transaction is suspended. So the entity manager does not have a transaction context.
Ultimately I made it work by following this tutorial: https://dzone.com/articles/resource-local-vs-jta-transaction-types-and-payara
and going for the Classic RESOURCE_LOCAL Application pattern.
It involves injecting the nonJTA EntityManagerFactory, using that to create the entitymanager and closing it after use. Of course the transaction has to be managed manually but after all now it works.
The essential excerpt of my code looke like this:
#PersistenceUnit(unitName = "...")
private EntityManagerFactory emf;
#Inject
#BatchProperty(name = "technologyIds")
private String technologyIds;
private void clearQueue(long technologyId) {
EntityManager entityManager = emf.createEntityManager();
entityManager.getTransaction().begin();
Query q = entityManager.createQuery("delete from Record r where r.technologyId=:technologyId");
q.setParameter("technologyId", technologyId);
q.executeUpdate();
entityManager.getTransaction().commit();
entityManager.close();
}
I am facing some issue with transactions configured in my code.
Below is the code with transactions which writes data to DB.
Writer.java
class Writer {
#Inject
private SomeDAO someDAO;
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void write(){
this.batchWrite();
}
private void batchWrite () {
try {
someDAO.writeToTable1(true);
} catch(Exception ex) {
someDAO.writeToTable1(false);
}
someDAO.writeToTable2();
}
}
SomeDAO.java
class SomeDAO {
#Inject
private JdbcTemplate JdbcTemplate;
public void writeToTable1(boolean flag) {
// Writes data to table 1 using jdbcTemplate
jdbcTemplate.update();
}
pulic void writeToTable2() {
// Writes data to table 2 using jdbcTemplate
jdbcTemplate.update();
}
}
Here data is getting stored into table 1 properly but sometimes, table 2 is getting skipped.
I am not sure how this is happening as both the tables have been written within same transaction.
Either transaction is partially committing the data or partially rolling back.
I have doubt that in the SomeDAO class I am injecting JdbcTemplate object which is creating new connection instead of using existing connection of transaction.
Can anyone please help me here?
Try binding a Transaction Manager bean having your jdbcTemplate inside #Transactional:
//Create a bean
#Bean
public PlatformTransactionManager txnManager() throws Exception {
return new DataSourceTransactionManager(jdbcTemplate().getDataSource());
}
And then use this transaction manager in #Transactional("txnManager").
I have JSF web application. I'm using JSF 2.1.9, Hibernate 4.1.4, GlassFish 3.1.2, PrimeFaces 3.4.1. The problem is, used heap size is increasing slowly and after 2-3 days reaches max heap size. Then I must restart GlassFish.
Heap dumps:
At the beginning, i clicked all web pages in application and used heap size was 100 MB:
A fter 1-2 days, used heap size is increased to 300 MB (same web pages used during this time):
I took screenshot of most used classes in heap.
In char[] class instances, there are too much SQL query string like that:
Maybe there is not only one problem but I might start to solve from this one. In my web pages, I generally select some objects from database and render it. here are some beans:
images (index controller):
#Named("indexController")
#SessionScoped
public class IndexController implements Serializable {
EntityManagerFactory emf = Persistence.createEntityManagerFactory("imagePU");
public List<Image> getImages() {
EntityManager em = emf.createEntityManager();
List<Image> result;
try {
EntityTransaction entr = em.getTransaction();
boolean committed = false;
entr.begin();
try {
Query query = em.createQuery("SELECT i FROM Image i ORDER BY i.imageId DESC").setMaxResults(12);
result = query.getResultList();
entr.commit();
committed = true;
} finally {
if (!committed) {
entr.rollback();
}
}
} finally {
em.close();
}
return result;
}
}
Tagged images:
#Named("galleryBean")
#SessionScoped
public class GalleryBean implements Serializable {
EntityManagerFactory emf = Persistence.createEntityManagerFactory("imagePU");
public List<TaggedImage> getTaggedImages() {
EntityManager em = emf.createEntityManager();
List<TaggedImage> result;
try {
EntityTransaction entr = em.getTransaction();
boolean committed = false;
entr.begin();
try {
Query query = em.createQuery("SELECT ti FROM TaggedImage ti GROUP BY ti.tag ORDER BY ti.taggedImagesId DESC");
result = query.getResultList();
entr.commit();
committed = true;
} finally {
if (!committed) {
entr.rollback();
}
}
} finally {
em.close();
}
return result;
}
}
By the way, I should not perform business logic in getters, but I think it is not the main reason of my problem. I need help and some advices. I can provide more information if needed.
Thanks for your help.
I had a similar problem with JSF pages once and after a lot of research turned out the problem was number of views which JSF was keeping in session. Not sure if this is the problem in your case but have a look at this to configure number of views. Hope this helps.
Not sure if this will solve your problem, however you are using session scope backing beans and based on the snippets above, could really be request scoped beans as i dont really see much in them that needs to be session saved. I would also look at using perhaps a view scope if it is available and restructure the bean so you could benefit from not having to make multiple calls to the database, as jsf is notorious for calling backing beans multiple times, like so.
#Named("galleryBean")
#RequestScoped // or #ViewScoped
public class GalleryBean implements Serializable {
EntityManagerFactory emf = Persistence.createEntityManagerFactory("imagePU");
private List<TaggedImage> images = null;
public List<TaggedImage> getTaggedImages() {
if (this.images != null) {
return this.images;
}
EntityManager em = emf.createEntityManager();
try {
EntityTransaction entr = em.getTransaction();
boolean committed = false;
entr.begin();
try {
Query query = em.createQuery("SELECT ti FROM TaggedImage ti GROUP BY ti.tag ORDER BY ti.taggedImagesId DESC");
images = query.getResultList();
entr.commit();
committed = true;
} finally {
if (!committed) {
entr.rollback();
}
}
} finally {
em.close();
}
return images;
}
}
Better yet you would separate out the actual fetching of the TaggedImage from the database and put it somewhere it is called as part of the bean construction phase with a #PostConstruct or #URLAction if using pretty faces and getTaggedImages() really just becomes another getter/setter
I am moving a JPA-Hibernate application from a Java EE 6 environment to a Tomcat 7 one.
The application has several DAO classes making queries on the EntityManager. In the Java EE environment I could just inject it using the #PersistenceContext annotation, and let the container manage the EntityManager. Now that I have to do it manually, I was wondering what is the way to go.
Should the entity manager be unique? If so can it be a static final field, created on startup and that every DAO class uses? Does it have a lifecycle that involves closing it and then re-opening it?
Unfortunately, you'll have to do it manually. The way I usually do it is to define a special class:
public class EMF {
private static EntityManagerFactory factory = Persistence.createEntityManagerFactory("name");
public static EntityManager getEntityManager() {
return factory.createEntityManager();
}
}
So, every time you need EntityManager, you have to create it manually. You need to handle transactions as well:
EntityManager em = EMF.getEntityManager();
EntityTransaction et = em.getTransaction();
try {
MyEntity my = new MyEntity();
et.begin();
try {
em.persist(my);
et.commit();
} catch (Exception ex) {
if (et.isActive())
et.rollback();
}
} finally {
em.close();
}
I am doing tests on an ejb3-project using ejb3unit session bean test. The following test will fail with the last assertNotSame() check.
public void testSave() {
Entity myEntity = new Entity();
myEntity.setName("name1");
myEntity = getBeanToTest().save(myEntity);
assertNotSame("id should be set", 0l, myEntity.getId());
// now the problem itself ...
int count = getBeanToTest().findAll().size();
assertNotSame("should find at least 1 entity", 0, count);
}
So, what is happening. The save(entity) method delivers my "persisted" object with an id set. But when I'll try to find the object using findAll() it won't deliver a single result. How can I get my ServiceBean.save method to work, so the persisted entity can be found?
Edit
My ServiceBean looks like this
#Stateless
#Local(IMyServiceBean.class)
public class MyServiceBean implements IMyServiceBean {
#PersistenceContext(unitName = "appDataBase")
private EntityManager em;
public Entity save(Entity entity) {
em.merge(entity);
}
public List<Entity> findAll() {
... uses Query to find all Entities ..
}
}
and for ejb3unit the ejb3unit.properties:
ejb3unit_jndi.1.isSessionBean=false
ejb3unit_jndi.1.jndiName=project/MyServiceBean/local
ejb3unit_jndi.1.className=de.prj.MyServiceBean
Here we go..
public void testSave() {
Entity myEntity = .. // create some valid Instance
// ...
EntityTransaction tx = this.getEntityManager().getTransaction();
try {
tx.begin();
myEntity = getBeanToTest().save(myEntity);
tx.commit();
} catch (Exception e) {
tx.rollback();
fail("saving failed");
}
// ...
}
maybe this'll help some of you.
Perhaps you don't have a running transaction, hence your entity isn't saved.
One option is to manually start the transaction by injecting a #PersistenceContext in the test, but better look for automatic transaction management in ejb3unit.