We Keep Coding

How to do Streaming at controller layer Spring

I am developing one simple Java Spring application and trying to get all the saved Order from database. Application flow is simple though have around 100k record in database. I used streaming at JPA layer from DB to application data is coming fine.
But when this start transferring from Controller to client something it throws 502 response. Following is piece of code for all three layer.
Could someone help to solve this problem? Requirement all this data should come without using any filter or pagination.
#GetMapping(produces = "application/json")
#ResponseStatus(HttpStatus.OK)
public Resources getOrders() {
return userService.getOrders();
}
#Transactional
public Resources getOrders() {
Stream<Orders> orders = streamAll();
List<Order> orderList = new ArrayList<>();
orderList.addAll(orderConverter.createFromEntities(orders));
Resource resources = new Resource();
resources.setResources(orderList);
return resources;
}
#Query("select u from Order")
Stream<Orders> streamAll();

You are getting a 502 response because your JVM is exhausted in memory. Use streaming detached objects at the repository layer.
First, check if your database supports fetching a result set through streaming data. For example, MYSQL supports it using a read-only session and fetch size with value Integer.MIN_VALUE [1].
Later, customize your repository to open a stateless session with StatelessSession interface [2]. Thanks to this interface you are getting objects detached, so Hibernate releases it immediately and no cost memory in cache.
If you need to make changes, remember again, your objects are detached. You need open another session (read-write). Your objects will be saved with these session, and after saving it, you must detach it manually.
This example covers the first requirement:
#PersistenceContext
private EntityManager entityManager;
public void stackoverflowQuestion() {
Session currentSession = entityManager.unwrap(Session.class);
ScrollableResults rows = currentSession
.createQuery("SELECT u FROM Order", Orders.class)
.setFetchSize(Integer.MIN_VALUE)
.scroll(ScrollMode.FORWARD_ONLY);
while (rows.next()) {
Orders order = (Orders) rows.get(0);
...
}
}
[1] https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-implementation-notes.html
[2] https://docs.jboss.org/hibernate/orm/3.5/reference/en-US/html/batch.html

Pass value to injected EJB

I have an EJB application that consists of two beans, ServiceEJB (web tier) and BusinessEJB (business tier), where BusinessEJBis injected in ServiceEJB.
ServiceEJBreceives HTTP requests from the browser, calls a method in BusinessEJB, gets the result, and sends the HTTP response.
Also, ServiceEJB has access to the HttpSession object, where the userId of the user that logged in is stored. BusinessEJBdoes NOT have access to the HttpSession object.
The application needs to log messages (using sl4j/logback, for example). It could log the message in ServiceEJBor BusinessEJB methods, and when it logs a message, it has to include the userId of the session in the log entry.
Since BusinessEJB doesn't have the userId, it needs to get it from ServiceEJB. The question is what is the best way to achieve that. What I DON'T want to do is to add a userId field to each method in BusinessEJB as a parameter, as there are many ServiceEJBs and BusinessEJBs in the application (and other beans called by BusinessEJB that also generate log entries), and I don't want to pollute the application with the userId field. Instead, I could have a userId field at the EJB level, but how to populate them? Is there a way to achieve this with annotations? Any suggestions will be welcome.
#Produces({MediaType.APPLICATION_JSON})
#Consumes({MediaType.APPLICATION_JSON})
#Stateless
public class ServiceEJB {
#Context
HttpServletRequest httpRequest;
#Inject
private BusinessEJB bean;
private String userId;
#Path("someurl")
public Response someMethod1() {
final HttpSession session = httpRequest.getSession();
// get the userId from the session
String s = bean.someMethod2();
// return Response
}
}
#Stateless
public class BusinessEJB {
private String userId;
public String someMethod2() {
// .... log an entry with userId
return "something";
}
}

A few pointers/comments:
If you integrate with application server security, then the user name is available at any component. EJBs can get it by calling getCallerPrincipal() on the injected variant of the EJBContext, here the javax.ejb.SessionContext:
#Resource
private SessionContext sessionCtx;
Servlets can retrieve the principal from the HttpServletRequest.getUserPrincipal(). JAX-RS components (the ServiceEJB) can retrieve it from the javax.ws.rs.core.SecurityContext.getUserPrincipal().
Is there any reason why you are NOT integrating with the application server security?
If you have a good reason NOT to integrate with application server security, I would propose a variation of the solution from the previous answer. The variation is to set the user data from a filter applied to all resources (either servlet filter or JAX-RS ContainerRequestFilter), so that you do not have to worry about setting it in multiple places.
If you ONLY NEED THE USER ID FOR LOGGING, I'd suggest you take a look at the concept of Mapped Diagnostic Contexts (MDC) in slf4j. With it you can set the user id early at the beginning of the request and make it available to all logging statements thereafter.

Create a request scoped CDI bean i.e. UserContext.
Inject it into both EJBs.
In ServiceEJB set user's id and in BusinessEJB read it.

Open Session In View (OSIV) and Hibernate Session flush

Following is a hypothetical situation on Spring 3.x and Hibernate3.x
I have a service layer in spring which invokes 3 DAOs to construct a model.
The DAOs are transactional(#Transactional) and have lazy loaded hibernate collections.
The service method causes a few updates ,along with the fetch of data.
A typical DAO method will be as follows -
public O create(I entity) throws GenericException {
getOrCreateSession().save(entity);
return (O)entity;
}
I have the following questions around OSIV -
1.How many times is this session flushed(database update) in the default AUTO mode?
2.Can OSIV be made to extend the session beyond a single request (to a conversation)?

The AUTO flush mode will execute the pending DML statements when:
the current transaction is committed
when a query might target an entity table, that's current enqueued for flushing
Spring Webflow has support for long conversations.

Spring transaction management

I am working on spring4 mvc to introduce in our new web application and currently we are using struts1.x and wanted to use new framework for support html5/ajax request as simple as possible and wanted to use the power of DI and spring webflow support.
Currently in our struts1.x application db transaction management is done at our custom GenericAction which is subclass of Action, In GenericAction we are getting the connection from data source and handover to the subclasses then any exception raised then caught and rollback, otherwise commit such that database atomicity (transaction management )is done at one place. All the module action classes should extends GenericAction such that database connection will be available and performs the module related stuff and after completing connection will be either rollback or commit in GenericAction as said above.
In spring, scope of the Transaction is started with #Transactional annotation and then ends with a method in Service Class since the service class marked as #Transactional. This is not feasible solution for me. I have read several documents related to the spring transaction before starting and below are my questions.
I am using HibernateTransactionManager as transaction manager
Transaction should start from interceptors in case web request or any class (in case of unit testing).
Transaction should ends with the after interceptor is executed in case of web request or any class in case of unit testing.
In case of any exception raised then our HandlerExceptionResolverImpl handler will execute then connection should be rollback.
Any workaround or best practices would be greatly appreciate.
Thanks
Dhorrairaajj

In my opinion, you can achieve this with minimal change on the current application by following what Spring does in its org.springframework.jdbc.datasource.DataSourceTransactionManager.doBegin(Object, TransactionDefinition). In detail, I think something like following should work:
On the GenericAction, get connection from data source (as you've already done)
Bind the connection via its holder with the current data source:
TransactionSynchronizationManager.bindResource(getDataSource(), txObject.getConnectionHolder());
Eventually, commit the transaction as org.springframework.jdbc.datasource.DataSourceTransactionManager.doCommit(DefaultTransactionStatus).
In any case, Spring's http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/transaction/support/TransactionTemplate.html can help to achieve any programmatic transactional scenario that you want. Just need to ensure that the right transactional resource is bound to the opening transaction.
Hope this helps.

Thanks for you reply. Your post/suggestion drives to do the following
I written an interceptor called TxtManager and below are methods to perform the transaction management.
public boolean preHandle(HttpServletRequest request,
HttpServletResponse response, Object handler) throws Exception
{
Session session = sessionFactory.openSession();
SessionHolder hold = new SessionHolder(
session);
TransactionSynchronizationManager.bindResource(sessionFactory, hold);
// passing null would accept the default transaction definition.
status = transactionManager.getTransaction(null);
return true;
}
public void afterCompletion(HttpServletRequest request,
HttpServletResponse response, Object handler, Exception ex)
throws Exception
{
Exception hanlderEx = (Exception) request
.getAttribute(HandlerExceptionResolverImpl.EXCEPTION_KEY);
if (hanlderEx != null)
{
transactionManager.rollback(status);
}
else
{
transactionManager.commit(status);
}
}
In HandlerExceptionResolverImpl class which responsible for handling things upon exception, put an exception object in request and read the same exception to rollback or commit in afterCompletion method of the interceptor.
In between(preHandler and afterCompletion) we will going to use the standard practices to perform the module related stuff including any other interceptos. Unit testing may not be supported, i will check other alternative for unit testing.
Finally we were able to simulate the existing frame work with your suggestion Thanks
Any suggestion to improves this would be much appreciate !!..
Thanks
Dhorrairaajj

org.hibernate.lazyinitialization exception

org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: pojo.Person.address, no session or session was closed.
I am getting this exception and I'm using Spring 3.0 and Hibernate 3.6.

It looks like you have an Entity called Person which has a lazily loaded mapped collection of Addresses? You have loaded the Person and the session it was loaded in has now been closed.
After the session was closed you then attempted to access that collection of address and Hibernate attempted to load them. However, that is not possible if the original session is no longer available.
In order to access the address property you have a few options:
Use the OpenSessionInView pattern to ensure that the Hibernate session is held open for the duration of the request/response cycle (Since you've tagged Spring MVC I'll assume this is a web based operation). This essentially scopes your Hibernate session to the HTTP request.
Ensure that all required properties are loaded before the session is closed (transaction committed). You can do this using
Hibernate.initialize(person.address)
or by writing HQL that uses a left join fetch. This could be something like:
createQuery("from Person as person left join fetch person.address")
This will override any lazy loading configuration for this query only and ensure that any collections are initialized.

Most probably, you don't have transaction management set up. That is, Spring uses default transaction scope, which is transaction per HibernateTemplate call, and closes session right after return from HibernateTemplate.
You can do one of three things:
set up transactions,
switch to explicit session handling,
use Criteria API or fetch join in order to prefetch the details you need.

I was seeing this issue because I had failed to annotate a method in a service with #Transactional. It seems that Hibernate closes the session when a call to another method is made (even within the same class) unless the caller is annotated appropriately.