Scenario:
I'm supporting an Enterprise application that runs in Wildfly10. The application (.war) uses J2EE technologies (EJBs, JPA, JAX-RS) and SpringBoot features (like, SpringMVC, SpringRest, SpringData, SpringRestData) ... Both stacks co-exists "happily" because they don't interact between them; however, they do share common classes, like utility or Entity Classes (the stacks map to the same database model). Why the application uses those stacks is out the scope of the question.
Currently, I'm trying to improve the performance of a #RestController that pulls some data from the database using a JPA Spring Repository. I found that we're suffering the N + 1 queries problem when calling the #RestController. In past projects (where there were only J2EE technologies), I have used the #BatchSize hibernate annotation to mitigate this problem with total success.
But, in this project, Spring seems to be skipping such annotation. How do I know that? Because I turned on the hibernate SQL logging (hibernate.show_sql) and I can see the N + 1 queries is still happening ...
Key Points:
Here are some insights about the application that you must know before providing (or trying to guess) any answer:
The application has many sub-modules encapsulated as libraries inside WAR file (/WEB-INF/lib) ... Some of these libraries are the jars that encapsulate the entity classes; others are the jars that encapsulate the REST Services (that could be JAX-RS services or Spring Controllers).
The Spring configuration is done in the classes defined in the WAR artifact: in there, we have a class (that extends from SpringBootServletInitializer) annotated with #SpringBootApplication and another class (that extends from RepositoryRestConfigurerAdapter) annotated with #Configuration. Spring customization is done is such class.
The application works with multiple datasources, which are defined in the Wildly server. Spring DATA JPA must address any query pointing to the right datasource. To accomplish this requirement, the application (Spring) was configured like this:
#Bean(destroyMethod="")
#ConfigurationProperties(prefix="app.datasource")
public DataSource dataSource() {
// the following class extends from AbstractRoutingDataSource
// and resolve datasources using JNDI names (the wildfly mode!)
return new DataSourceRouter();
}
#Bean("entityManagerFactory")
public LocalContainerEntityManagerFactoryBean getEntityManagerFactoryBean() {
LocalContainerEntityManagerFactoryBean lemfb;
lemfb = new LocalContainerEntityManagerFactoryBean();
lemfb.setPersistenceUnitName("abcd-pu");
lemfb.setDataSource(dataSource());
return lemfb;
}
The last #Bean declaration favors the use of a persistence.xml file, which we do have in the route /WEB-INF/classes/META-INF/ (i.e. Spring does find this file!) ... In such file, we define our domain classes, so that Spring JPA can see such entities. Also, we can define special JPA properties like: hibernate.show_sql and hibernate.use_sql_comments without issues (this is how I detected the N + 1 queries problem in the first place) ...
What I have done so far?
I tried to add the #BatchSize annotation to the problematic collection. No luck!
I created a new JAX-RS Service whose purpose was to mimic the behavior of the #RestController. I confirmed that the #BatchSize annotation does work in the application's deployment, at least, in JAX-RS Services! (NOTE: the service uses it own persistence.xml) ...
Test details (Updated 2020/07/30): What I did here was to create a new JAX-RS Service and deployed it inside the WAR application, next to the #RestController that presents the problem (I mean, it is the same WAR and the same physical JVM). Both services pull from database the same entity (same class - same classloader), which has a lazy Collection annotated with #BatchSize! ... If I invoke both services, the JAX-RS honors the #BatchSize and pulls the collection using the expected strategy, the #RestController does not ... So, what it is happening here? The only thing different between the services is that each one has a different persistence.xml: the persistence.xml for the JAX-RS is picked by Wildfly directly, the other one is picked by Spring and delegated to Wildfly (I guess) ...
I tried to add the properties: hibernate.batch_fetch_style (=dynamic) and hibernate.default_batch_fetch_size (=10), to the persistence.xml read by Spring ... No luck. I debug the Spring startup process and I saw that such properties are passed to the Spring Engine, but Spring does not care about them. The weird thing here is that properties like: hibernate.show_sql, Spring does honor them ... For those who are asking: "What does these properties do?" Well, they are global equivalent to apply #BatchSize to any JPA lazy collection or proxy without declaring such annotation in any entity.
I setup a small SpringBoot Project using the same Spring version as enterprise application (which is 1.5.8.RELEASE, by the way) and both the annotation and properties approach worked as supposed to.
I've been stuck with this issue for two days, any help to fix this will be appreciated ... thanks!
There are 2-3 possible issues that I can think off.
For some reason, whatever you modify isnt picked up by wildfly - Wildfly classpath resolution is a separate Topic and some missing configuration can cause you a nightmare. This you can identify if you have access to debug the query, and in if you put a breakpoint in the constructor of your Entity class, you will get a chance to evaluate the entity configuration being used, somewhere in the execution conetxt.
BatchSize doesnt work on OneToOne, It only works on OneToMany relationships.
A typical way to define BatchSize is to do along with Lazy load as mentioned in the example here. If you are not using Lazy fetch, hibernate assumes that you are willing to make an eager load and makes another select query to fetch all the details.Please confirm you are using the same syntax as given in the example above.
New Addition:
Put Conditional Breakpoints in PropertyBinder#setLazy() function, and may be backtrace it and put relavent breakpoints in CollectionBinder and AnnotationBinder. then restart/redeploy the server and see what data you are getting for the relavent properties. That will give you fair idea where it is failing..
Why conditional breakpoint? Its because you will have thousands of properties and if you do not add condition to the breakpoint, you will take 1 hour to reach your actual breakpoint
What should be the condition - If its property binder, the condition shoud be like `this.name == . For other classes also you can use the same approach.
Sorry for too detailed description on conditional breakpoints, you might find it redundent.
Looks like the only way to debug your problem is to debug hibernate framework from server startup, then only we will be able to find out the rootcause
Related
Once I #EnableDataFlowServer my SpringBoot application, my own custom entities do not load. (I get the 'type not managed' exception which occurs when JPA isn't finding your entities).
These entities are found within another Spring module that I import, like
#Import({MyDomainsModule.class})
I'm using 2.0.0.m2 of Spring Cloud DataFlow.
Some debugging I've done:
If I add this to my Spring Boot application main class:
#EntityScan({
"com.company.mydomain.entities"
})
Then my entities start to load as usual, but then Spring DataFlow breaks. For example, any time I try to load the UI, I'll get:
|ne.jdbc.spi.SqlExceptionHelper| Table 'dataflow.appregistration' doesn't exist
That makes me thinking by simply adding the EntityScan, I broke some naming strategy since the actual name of the table is of course app_registration
I think this is mostly a 'how do I do multiple locations of JPA-based code in one project', rather than a Spring Cloud DataFlow question. But knowing the fix might require a better understanding of how SCDF. I've checked out the project and reading up both Spring Boot and how SCDF configures itself.
Any help is greatly appreciated!
I had a bad strategy coming in from one of my properties, overriding what SCDF added this to my application.properties.
So to be explicit, I set this in my properties:
spring.jpa.hibernate.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
spring.jpa.hibernate.naming.implicit-strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
And then my SpringBoot application looks like
#SpringBootApplication(exclude = LocalDataFlowServerAutoConfiguration.class)
#Import({MyDomainModule.class})
#EnableDataFlowServer
// EnableDataFlowServer has an EntityScan, which causes ours to not be picked up!
// Look in DataFlowControllerAutoConfiguration for more information
#EntityScan({
"com.company.mydomain.entities"
})
I'm working on a system's back end that uses Spring Boot, REST, HATEOAS, Hibernate and PostgreSQL. For validation, I started using classes that extend org.springframework.validation.Validator. It works well, but only for calls made by the front end. For calls made in the back end, such as by using EntityManager, they don't fire. I've managed to have another validator being called in this situation by using #Constraint for ElementType.TYPE, but it only gets called for create and save methods.
Is it possible to use this validator to validate on delete methods too? There's a project here that's a non operational subset of the project I'm working on, containing the validators I mentioned.
Thanks in advance.
P.S.: I'd rather avoid manually calling the validators whenever I call a repository method in the back end.
P.P.S.: This answer makes me believe it's possible, but I couldn't translate the XML configuration to JavaConfig.
I finally found the answer. In application.properties, add:
spring.jpa.properties.javax.persistence.validation.group.pre-remove=javax.validation.groups.Default
The linked question told me which property I needed, but I didn't know where to place it. I tried to use custom Java configuration and even persistence.xml configuration, but several other things failed.
Here, I learned that "[...] all properties in spring.jpa.properties.* are passed through as normal JPA properties (with the prefix stripped) when the local EntityManagerFactory is created." So I just added that prefix and it worked.
I am assessing whether spring-boot and how I could migrate to using it.
One question I have is whether a project that uses spring boot can be converted easily back to a regular spring project which uses the traditional spring configuration files if that is required. This would be useful in my mind for a few reasons.
1) merging with legacy projects, because as I have read moving from legacy spring to spring-boot is somewhat tedious.
2) Obtaining a view of the spring application context file and webapp configuration files to understand what the actual configurations being used are.
Another question I have is regarding the lack of application-context file, is there a way to have some kind of hybrid where there is still an application-context file that can be seen? Part of my concern is that spring-boot auto configures components without us knowing and learning how they are configured and work together.
Spring Boot provides auto-configuration.
When #SpringBootApplication is encountered, it triggers a search of the CLASSPATH for a file called META-INF/spring.factories which is just a regular text file that enumerates a list of Java configuration classes. Java configuration was introduced in 2006 and then merged into Spring 3 in 2009. So it's been around for a long time. These Java configuration classes define beans in the same way that XML does. Each class is annotated with #Configuration and therein you find beans defined using methods (factory methods, or provider methods) whose return value is managed and exposed via Spring. Each such provider method is annotated with #Bean. This tells Spring to consider the method and its return value the definition of the bean.
Spring Boot tries to launch all the Java configurations it sees enumerated in that text file. It tries to launch RabbitAutoConfiguration.class, which in turn provides beans for connecting to RabbitMQ and so on. Of course, you don't want those beans in certain cases, so Spring Boot takes advantage of Spring framework 4's #Conditional mechanism to conditionally register those beans only if certain conditions are met: is a type on the CLASSPATH, is a property exposed through the environment, has there been another bean of the same type defined by the user, etc. Spring boot uses these to only create the RabbitMQ-specific beans if, for example, the dependencies that you would get from org.springframework.boot:spring-boot-starter-amqp are on the CLASSPATH. It also considers that the user may have provided a different implementation of RabbitTemplate in some othe rbean definition (be it XML or Java configuration) so it uses that if it's there.
These java configuration classes are the same sort of Java configuration classes you would write without Spring Boot. BUT... WHY? 80% of the time, the auto-configuration that Spring Boot provides is going to be as good or better than the configuration you would write yourself. There are only so many ways to configure Spring MVC, Spring Data, Spring Batch, etc., and the wager you take using Spring Boot is that the leaders and engineers on those various projects can provide the most sensible 80%-case configuration that you probably don't care to write, anyway.
So, yes you could use Spring Boot with existing applications, but you'd have to move those existing applications to Spring 4 (which is easy to do if you're using the spring-boot-starter-* dependencies) to take advantage of #Conditional. Spring Boot prefers: NO configuration, Java configuration, XML configuration, in that order.
If you have an existing application, I'd do the following:
find out what dependencies you can remove from your Gradle/Maven build and just have taken care of for you with the various spring-boot-starter- dependencies.
add #SpringBootApplication to a root component class. Eg, if your package is a.b.c, put a class Application in a.Application and annotate that with #SpringBootApplication
You can run it as a Servlet 3 application or in an embedded servlet container. It might be easier to just run in a standard servlet container as you take baby steps. Go to http://start.spring.io and make sure to choose war in the packaging drop down. Copy the ServletInitializer class and the specification from the pom.xml to ensure that your application is a .war, not a .jar. Then, once everything works on Spring Boot, rm -rf the Initializer and then revert the Maven build to a simpler .jar using the Spring Boot plugin for extra win.
If your application has lots of existing configuration, import it using #Import(OldConfig.class) or #ImportResource("old-config.xml") on the a.Application configuration class. The auto-configuration will kick in but it will see, for example, that you may have already defined a DataSource so it'll plug that in in the right places. What I like to do now is just start the application up, see if everything's OK, then start removing code in my old Java or XML configuration. Don't remove your business code, but remove things related to turning on parts of Spring. Anything that looks like #Enable..* or ..:annotation-driven/>. Restart and verify things still work. The goal is to let Spring Boot do as much of the heavy lifting as possible. Viewing the source is very handy here so you can see what Spring Boot will try to do for you. Spring Boot will log information on what #Conditional conditions evaluated to true for you. Simply provide --Ddebug=true wen you start the application to see the output. You could also export DEBUG=true then restart your IDE to run it as long as the environment variable is ivsible in your shell.
I am working on a JAX-RPC webservice that is already built. This is a huge service which consists of around 25-30 operations. A large number of Spring config files (around 50) are also present. A number of test classes are developed which loads all the spring references associated with the service. This takes a long time to load all the spring config files and associated bean references before executing the specific test method. I am trying to see if I can load only the specific spring config files / bean references that pertain to the operation.
I was told that setting default lazy load parameter in spring config might not help as it works only if it is run within a container. Would it be possible to help in suggesting in options that I can follow to decrease the initial loading time?
I don't know, but may be this help you. You can use annotations in you test classes to specify the config files you need to include to the context. Also, you can add spring profiles and activate them from the test class.
#RunWith(SpringJUnit4ClassRunner.class)
#ActiveProfiles({ "dev-test"})
#ContextConfiguration({"/application-context-persistence.xml", "/another-context.xml"})
public class Test{
}
we are developing a set of components that can be used by our applications which currently run on Jboss 4.2.3.GA.
However, since JBoss 4 is quite old now, we'd like to migrate to JBoss 5+ (most likely JBoss 6.1).
In that case, we have a problems with our components containing stateless session beans annotated with the JBoss specific #PoolClass annotion, which has been renamed to #Pool as of JBoss 5.
As you can see, using that annotation in the components introduces a dependency not only on the application server but on specific versions as well. Thus I'd like to get rid of that annotation and use configuration via XML.
In that case, I'd like not to have to define a pool class per session bean, since almost all of them use the StrictMaxPool class with the same settings. If any session bean needs a different pool or different setting, it's ok to provide a specific entry, but I'd rather not do that for the majority.
The default pool class is generally defined in ejb3-interceptors-aop.xl. However, that configuration applies to all applications deployed on that server.
What I'd like to do now, is to define a default pool class (with default settings) per application, if possible without adding anything specific to the session beans in our components (like the JBoss specific #AspectDomain annotation).
Is that possible and if yes, how?
I couldn't manage to find any useful information yet, so I'd be glad if someone could point me in the right direction.
Edit: If you have information on how to do that in JBoss 5+, that would be great too, but JBoss 4 is of higher priority for now.
Seems like I found a solution:
There's a problem with JBoss AOP 1.5.6 which is used by JBoss 4.2.3.GA.
Although there's no bug filed it seems there is one preventing the scoped use of pool definitions (it looks like the source of the problem is an inverted if-condition :) ).
Upgrading to JBoss AOP 2.1.8 works like a charm.
Here's what I do:
Put a jboss-aop.xml file into the root of an ejb jar in the ear (we already have one that contains our persistence.xml).
In the jboss-aop.xml create a domain Stateless Bean that extends from the default Stateless Bean domain and inherits all bindings and definitions
Inside the extended domain override the pool class definition.