We Keep Coding

Spring Security, JDBC, c3p0 connection problem

To improve my skills I've decided to grab an Udemy course about Spring framework. During this course there is a task to create logging into app using JDBC and C3P0.
I've followed all course details and instructor tips. After I deploy my app on Tomcat it is running, but I am unable to log in using the data that has been put in DB. My goal is to log into the app, but for now I am unable to make it. I have even dowloaded solution code, but even after firing it I am unable to log into the further app part.
Whole code structure is good (at least i do believe it). Below I have placed screen and code of most important classes and files.
I have tried to remove {noop} from the DB, but without any result. I have also tried to use an empty line as a password - without any success. When I was doing logging with auth from memory everything was fine, but when i try to parse it from DB i am unable to log in ( there are no error logs, I just got information from app that user and password are wrong.
DemoAppConfig.java
package com.luv2code.springsecurity.demo.config;
import java.beans.PropertyVetoException;
import java.util.logging.Logger;
import javax.sql.DataSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.core.env.Environment;
import org.springframework.web.servlet.ViewResolver;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
import org.springframework.web.servlet.view.InternalResourceViewResolver;
import com.mchange.v2.c3p0.ComboPooledDataSource;
#Configuration
#EnableWebMvc
#ComponentScan(basePackages="com.luv2code.springsecurity.demo")
#PropertySource("classpath:persistence-mysql.properties")
public class DemoAppConfig {
// set up variable to hold the properties
#Autowired
private Environment env;
// set up a logger for diagnostics
private Logger logger = Logger.getLogger(getClass().getName());
// define a bean for ViewResolver
#Bean
public ViewResolver viewResolver() {
InternalResourceViewResolver viewResolver = new InternalResourceViewResolver();
viewResolver.setPrefix("/WEB-INF/view/");
viewResolver.setSuffix(".jsp");
return viewResolver;
}
// define a bean for our security datasource
#Bean
public DataSource securityDataSource() {
// create connection pool
ComboPooledDataSource securityDataSource
= new ComboPooledDataSource();
// set the jdbc driver class
try {
securityDataSource.setDriverClass(env.getProperty("jdbc.driver"));
} catch (PropertyVetoException exc) {
throw new RuntimeException(exc);
}
// log the connection props
// for sanity's sake, log this info
// just to make sure we are REALLY reading data from properties file
logger.info(">>> jdbc.url=" + env.getProperty("jdbc.url"));
logger.info(">>> jdbc.user=" + env.getProperty("jdbc.user"));
// set database connection props
securityDataSource.setJdbcUrl(env.getProperty("jdbc.url"));
securityDataSource.setUser(env.getProperty("jdbc.user"));
securityDataSource.setPassword(env.getProperty("jdbc.password"));
// set connection pool props
securityDataSource.setInitialPoolSize(
getIntProperty("connection.pool.initialPoolSize"));
securityDataSource.setMinPoolSize(
getIntProperty("connection.pool.minPoolSize"));
securityDataSource.setMaxPoolSize(
getIntProperty("connection.pool.maxPoolSize"));
securityDataSource.setMaxIdleTime(
getIntProperty("connection.pool.maxIdleTime"));
return securityDataSource;
}
// need a helper method
// read environment property and convert to int
private int getIntProperty(String propName) {
String propVal = env.getProperty(propName);
// now convert to int
int intPropVal = Integer.parseInt(propVal);
return intPropVal;
}
}
DemoSecurityConfig.java
package com.luv2code.springsecurity.demo.config;
import javax.sql.DataSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.core.userdetails.User;
import org.springframework.security.core.userdetails.User.UserBuilder;
#Configuration
#EnableWebSecurity
public class DemoSecurityConfig extends WebSecurityConfigurerAdapter {
// add a reference to our security data source
#Autowired
private DataSource securityDataSource;
#Override
protected void configure(AuthenticationManagerBuilder auth) throws Exception {
// use jdbc authentication ... oh yeah!!!
auth.jdbcAuthentication().dataSource(securityDataSource);
}
#Override
protected void configure(HttpSecurity http) throws Exception {
http.authorizeRequests()
.antMatchers("/").hasRole("EMPLOYEE")
.antMatchers("/leaders/**").hasRole("MANAGER")
.antMatchers("/systems/**").hasRole("ADMIN")
.and()
.formLogin()
.loginPage("/showMyLoginPage")
.loginProcessingUrl("/authenticateTheUser")
.permitAll()
.and()
.logout().permitAll()
.and()
.exceptionHandling().accessDeniedPage("/access-denied");
}
}
MySpringMvcDispatcherServletInitializer.java
package com.luv2code.springsecurity.demo.config;
import org.springframework.web.servlet.support.AbstractAnnotationConfigDispatcherServletInitializer;
public class MySpringMvcDispatcherServletInitializer extends AbstractAnnotationConfigDispatcherServletInitializer {
#Override
protected Class<?>[] getRootConfigClasses() {
// TODO Auto-generated method stub
return null;
}
#Override
protected Class<?>[] getServletConfigClasses() {
return new Class[] { DemoAppConfig.class };
}
#Override
protected String[] getServletMappings() {
return new String[] { "/" };
}
}
persistence-mysql.properties
#
# JDBC connection properties
#
jdbc.driver=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://localhost:3306/spring_security_demo_plaintext?useSSL=false
jdbc.user=hbstudent
jdbc.password=hbstudent
#
# Connection pool properties
#
connection.pool.initialPoolSize=5
connection.pool.minPoolSize=5
connection.pool.maxPoolSize=20
connection.pool.maxIdleTime=3000
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.luv2code</groupId>
<artifactId>spring-security-demo</artifactId>
<version>1.0</version>
<packaging>war</packaging>
<name>spring-security-demo</name>
<properties>
<springframework.version>5.0.2.RELEASE</springframework.version>
<springsecurity.version>5.0.0.RELEASE</springsecurity.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<!-- Spring MVC support -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<version>${springframework.version}</version>
</dependency>
<!-- Spring Security -->
<!-- spring-security-web and spring-security-config -->
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
<version>${springsecurity.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
<version>${springsecurity.version}</version>
</dependency>
<!-- Add Spring Security Taglibs support -->
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-taglibs</artifactId>
<version>${springsecurity.version}</version>
</dependency>
<!-- Add MySQL and C3P0 support -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.45</version>
</dependency>
<dependency>
<groupId>com.mchange</groupId>
<artifactId>c3p0</artifactId>
<version>0.9.5.2</version>
</dependency>
<!-- Servlet, JSP and JSTL support -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>javax.servlet.jsp</groupId>
<artifactId>javax.servlet.jsp-api</artifactId>
<version>2.3.1</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<!-- TO DO: Add support for Maven WAR Plugin -->
<build>
<finalName>spring-security-demo</finalName>
<pluginManagement>
<plugins>
<plugin>
<!-- Add Maven coordinates (GAV) for: maven-war-plugin -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.0</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
SQL code responsible for generating DB test data
DROP DATABASE IF EXISTS `spring_security_demo_plaintext`;
CREATE DATABASE IF NOT EXISTS `spring_security_demo_plaintext`;
USE `spring_security_demo_plaintext`;
--
-- Table structure for table `users`
--
DROP TABLE IF EXISTS `users`;
CREATE TABLE `users` (
`username` varchar(50) NOT NULL,
`password` varchar(50) NOT NULL,
`enabled` tinyint(1) NOT NULL,
PRIMARY KEY (`username`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Inserting data for table `users`
--
INSERT INTO `users`
VALUES
('john','{noop}test123',1),
('mary','{noop}test123',1),
('susan','{noop}test123',1);
--
-- Table structure for table `authorities`
--
DROP TABLE IF EXISTS `authorities`;
CREATE TABLE `authorities` (
`username` varchar(50) NOT NULL,
`authority` varchar(50) NOT NULL,
UNIQUE KEY `authorities_idx_1` (`username`,`authority`),
CONSTRAINT `authorities_ibfk_1` FOREIGN KEY (`username`) REFERENCES `users` (`username`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Inserting data for table `authorities`
--
INSERT INTO `authorities`
VALUES
('john','ROLE_EMPLOYEE'),
('mary','ROLE_EMPLOYEE'),
('mary','ROLE_MANAGER'),
('susan','ROLE_EMPLOYEE'),
('susan','ROLE_ADMIN');

Type it as below:
username = "john"
password = "{noop}test123"
yes, exactly same password shown in the database, need to include the {noop} also for the password.

Producer#initTransactions doesn't work with KafkaContainer

I try to send messages to Kafka with a transaction. So, I use this code:
try (Producer<Void, String> producer = createProducer(kafkaContainerBootstrapServers)) {
producer.initTransactions();
producer.beginTransaction();
Arrays.stream(messages).forEach(
message -> producer.send(new ProducerRecord<>(KAFKA_INPUT_TOPIC, message)));
producer.commitTransaction();
}
...
private static Producer<Void, String> createProducer(String kafkaContainerBootstrapServers) {
return new KafkaProducer<>(
ImmutableMap.of(
ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaContainerBootstrapServers,
ProducerConfig.CLIENT_ID_CONFIG, UUID.randomUUID().toString(),
ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true,
ProducerConfig.TRANSACTIONAL_ID_CONFIG, UUID.randomUUID().toString()
),
new VoidSerializer(),
new StringSerializer());
}
If I use local Kafka, it works well.
But if I use Kafka TestContainers, it freezes on producer.initTransactions():
private static final String KAFKA_VERSION = "4.1.1";
#Rule
public KafkaContainer kafka = new KafkaContainer(KAFKA_VERSION)
.withEmbeddedZookeeper();
How can I configure KafkaContainer to work with transactions?

Try using Kafka for JUnit instead of Kafka testcontainers. I had the same problem with transactions and made them alive in this way.
Maven dependency that I used:
<dependency>
<groupId>net.mguenther.kafka</groupId>
<artifactId>kafka-junit</artifactId>
<version>2.1.0</version>
<scope>test</scope>
</dependency>

I got an exception using Kafka for JUnit as #AntonLitvinenko suggested. My question about it here.
I added this dependency to fix it (see the issue):
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-test</artifactId>
<version>2.12.0</version>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
</exclusions>
<scope>test</scope>
</dependency>
Also, I used 2.0.1 version for kafka-junit and kafka_2.11:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>${kafkaVersion}</version>
<scope>test</scope>
</dependency>

Cannot resolve this error: Failed to configure a DataSource

I am learning Spring Boot and in my first set up I got this problem:
Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured
I looked up solutions on the web and did find some, including those from Stackoverflow, but none of them works.
My simple code:
#SpringBootApplication
#RestController
public class SpringBootDemoApplication {
public static void main(String[] args) {
SpringApplication.run(SpringBootDemoApplication.class, args);
}
#RequestMapping(value = "/")
public String response(){
return "You made it!";
}
}
My POM:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
My application.properties:
spring.datasource.url=jdbc:mysql://localhost:3306/hrdb?autoReconnect=true
spring.datasource.username=root
spring.datasource.password=passw0rd
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
I also created a file as the following:
#Configuration
public class DBConfig {
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/hr?autoReconnect=true");
dataSource.setUsername("root");
dataSource.setPassword("passw0rd");
return dataSource;
}
}
I got the mentioned error either in Unit Test or starting the service. The only way I find out to make it work is to disable data source auto config:
#SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
Need your help. Thanks.

Check your DB config - it seems to overwrite Spring's default data source, but the url is different than in application.properties: "jdbc:mysql://localhost:3306/hr?autoReconnect=true"
In the properties you have:
"jdbc:mysql://localhost:3306/hrdb?autoReconnect=true"
"hr" vs "hrdb".
I would get rid of this class altogether and instead declare the relevant JDBC driver in your pom.xml and let Spring take care of the connection.

Hadoop distcp from a spring boot application - ClassNotFoundException

I am trying to submit distCP job from a spring boot application on a REST API call.
version of spring: 1.5.13.RELEASE
hadoop version: 2.7.3
below is the code I am using to instantiate the DistCP:
List<Path> srcPathList = new ArrayList<Path>();
srcPathList.add(new Path("hdfs://<cluster>/tmp/<user>/source"));
Path targetPath = new Path("hdfs://<cluster>/tmp/<user>/destination");
DistCpOptions distCpOptions = new DistCpOptions(srcPathList,targetPath);
DistCp distCp = new DistCp(configuration,distCpOptions);
Job job = distCp.execute();
The job is submitted successfully to the cluster, however the job fails due to ClassNotFoundException on the cluster. Below is the exception:
INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED;
cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
Class org.apache.hadoop.tools.mapred.CopyOutputFormat not found
Why does this happen? Any pointers around this would be very helpful!! Thanks!

I found the reason via viewing the job.jar on the NodeManager machine. The structure of job.jar is：
BOOT-INF/class/xxx
this is unreasonable.
I tried to replace the jar package with war，it works!
<packaging>war</packaging>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<!--exclude inner tomcat-->
<exclusions>
<exclusion>
<artifactId>spring-boot-starter-tomcat</artifactId>
<groupId>org.springframework.boot</groupId>
</exclusion>
</exclusions>
</dependency>
<!-- include tomcat-->
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-servlet-api</artifactId>
<version>7.0.47</version>
<scope>provided</scope>
</dependency>
...
and then add start class:
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.support.SpringBootServletInitializer;
public class SpringBootStartApplication extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
//
return builder.sources(xxxPortalApplication.class);
}
}

spring jpa - At least one JPA metamodel must be present*

Anybody know why it doesn't work?
Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
06/04/2017 14:11:24.732 ERROR [main] - org.springframework.boot.SpringApplication: Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jpaMappingContext': Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: At least one JPA metamodel must be present!
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:742)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:866)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1162)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1151)
at com.cadit.web.WebApplicationAware.main(WebApplicationAware.java:19)
Caused by: java.lang.IllegalArgumentException: At least one JPA metamodel must be present!
at org.springframework.util.Assert.notEmpty(Assert.java:277)
at org.springframework.data.jpa.mapping.JpaMetamodelMappingContext.<init>(JpaMetamodelMappingContext.java:52)
at org.springframework.data.jpa.repository.config.JpaMetamodelMappingContextFactoryBean.createInstance(JpaMetamodelMappingContextFactoryBean.java:71)
at org.springframework.data.jpa.repository.config.JpaMetamodelMappingContextFactoryBean.createInstance(JpaMetamodelMappingContextFactoryBean.java:26)
at org.springframework.beans.factory.config.AbstractFactoryBean.afterPropertiesSet(AbstractFactoryBean.java:134)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624)
... 16 common frames omitted
I defined entities in com.cadit.entities:
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;
#Entity
#Table(name="TEST")
public class GenericBeans implements BeanType, IEntity<Long> {
/**
*
*/
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
#Column(name = "TEST_PAID")
protected Long id;
#Column(name = "SOCIETA")
private String SocietaCod;
#Column(name = "CONTO_INTERMEDIARIO")
private String contoInt;
#Column(name = "TIPO_OPERAZIONE")
private String tipoOpe;
public GenericBeans(String societaCod, String contoInt, String tipoOpe) {
SocietaCod = societaCod;
this.contoInt = contoInt;
this.tipoOpe = tipoOpe;
}
public GenericBeans() {
}
public String getSocietaCod() {
return SocietaCod;
}
public void setSocietaCod(String societaCod) {
SocietaCod = societaCod;
}
public String getContoInt() {
return contoInt;
}
public void setContoInt(String contoInt) {
this.contoInt = contoInt;
}
public String getTipoOpe() {
return tipoOpe;
}
public void setTipoOpe(String tipoOpe) {
this.tipoOpe = tipoOpe;
}
#Override
public String toString() {
return "CSV [SocietaCod=" + SocietaCod + ", contoInt=" + contoInt + ", tipoOpe=" + tipoOpe + "]";
}
#Override
public Long getId() {
return this.id;
}
#Override
public void setId(Long id) {
this.id=id;
}
}
I definied my datasource entry definition for spring:
import org.apache.log4j.Logger;
import org.springframework.boot.autoconfigure.domain.EntityScan;
import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.transaction.annotation.EnableTransactionManagement;
#Configuration
#ComponentScan
#EntityScan("com.cadit.entities")
//#EnableJpaRepositories("com.cadit.entities")
#EnableTransactionManagement
#PropertySource("classpath:db-config.properties")
public class DbAutoConfiguration {
static final Logger logger = Logger.getLogger(DbAutoConfiguration.class);
public DbAutoConfiguration() {
}
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource(){
//DataSource ds =new EmbeddedDatabaseBuilder().addScript("classpath:sql/schema.sql").addScript("classpath:testdb/data.sql").build();
DataSourceBuilder ds = DataSourceBuilder.create();
logger.info("dataSource = " + ds);
return ds.build();
}
}
My db-config.properties is:
spring.jpa.hibernate.ddl-auto: validate
spring.jpa.hibernate.naming_strategy: org.hibernate.cfg.ImprovedNamingStrategy
#spring.jpa.database: SQL
spring.jpa.show-sql: true
spring.datasource.driverClassName=net.sourceforge.jtds.jdbc.Driver
spring.datasource.url=jdbc:jtds:sqlserver://localhost:1433;databaseName=example
spring.datasource.username=xxx
spring.datasource.password=xxx
IEntity is:
public interface IEntity <I extends Serializable> extends Serializable{
/**
* Property rappresenta la primary key.
*/
String P_ID = "id";
/**
* restituisce la primary key
* #return
*/
I getId();
/**
* imposta la primary key
* #param id
*/
void setId(I id);
}
I try to write CSV file to database using CrudRepository interface of spring:
import java.io.File;
import java.util.Collections;
import java.util.LinkedList;
import java.util.List;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.repository.CrudRepository;
import com.cadit.entities.GenericBeans;
import com.csvreader.CsvReader;
public class CsvReaders {
static final Logger logger = Logger.getLogger(CsvReader.class);
#Autowired
public CrudRepository<GenericBeans,Long> _entitymanager;
public List loadDataFromCsv(String fileName) {
try {
File file = new ClassPathResource(fileName).getFile();
CsvReader csv = new CsvReader(file.getAbsoluteFile().getPath(),';');
csv.readHeaders();
List l = new LinkedList();
GenericBeans b = new GenericBeans ();
while (csv.readRecord())
{
b.setSocietaCod(csv.get(0));
b.setContoInt(csv.get(1));
b.setTipoOpe(csv.get(2));
_entitymanager.save(b); //persist on db
l.add(b);
b = new GenericBeans();
}
b=null;
return l;
} catch (Exception e) {
logger.error("Error occurred while loading object list from file " + fileName, e);
return Collections.emptyList();
}
}
}
I DO NOT use main class but a class which extend SpringBootServletInitializer because i want to run it on both standalone tomcat and Tomcat installation as WAR application
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.support.SpringBootServletInitializer;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
#Configuration
#ComponentScan(basePackages={"com.cadit.entities","com.cadit.beans"})
#EnableAutoConfiguration
public class WebApplicationAware extends SpringBootServletInitializer {
private static Class<WebApplicationAware> applicationClass = WebApplicationAware.class;
public static void main(String[] args) {
SpringApplication.run(applicationClass, args);
}
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(applicationClass);
}
}
All properties file are in classpath resources because it's a maven project.
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.springframework</groupId>
<artifactId>xxxx</artifactId>
<version>0.1.0</version>
<packaging>war</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.2.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-jpa</artifactId>
<version>1.11.1.RELEASE</version>
</dependency>
<dependency>
<groupId>javax.persistence</groupId>
<artifactId>persistence-api</artifactId>
<version>1.0.2</version>
</dependency>
<!-- altre dipendenze non spring -->
<!-- https://mvnrepository.com/artifact/net.sourceforge.javacsv/javacsv -->
<dependency>
<groupId>net.sourceforge.javacsv</groupId>
<artifactId>javacsv</artifactId>
<version>2.0</version>
</dependency>
<!-- per jpa solo se si usa il Tomcat embedded -->
<dependency>
<groupId>net.sourceforge.jtds</groupId>
<artifactId>jtds</artifactId>
<version>1.3.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-dbcp2</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
<version>2.0</version>
<scope>provided</scope>
</dependency>
<!-- end -->
<!-- dipendenze logback -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.7.5</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.1.7</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>1.1.7</version>
</dependency>
<!-- fine dip logback -->
</dependencies>
<properties>
<start-class>hello.WebApplicationAware</start-class>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
<repositories>
<repository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</pluginRepository>
</pluginRepositories>
</project>
What's the problem, why doesn't it find JPA entities when I run WebApplicationAware class?

Spring does not find any JPA Entities, so no JPA Meta Model is created, that is why you face the exception.
The cause of this problem may be a wrong persistence-api version on your class path.
You are using
<dependency>
<groupId>javax.persistence</groupId>
<artifactId>persistence-api</artifactId>
<version>1.0.2</version>
</dependency>
but I am pretty shure your spring version uses persistence-api version 2.
Could it be, you are using #Entity annotation from version 1 ?
At runtime spring uses version 2, and this is searching for Entites using #Entity from version 2 only !
Remove the dependencies
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-jpa</artifactId>
<version>1.11.1.RELEASE</version>
</dependency>
Instead add
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
This will give you all JPA dependencies in the right version.

I solved it by adding 2 annotations
#EnableAutoConfiguration
#EntityScan(basePackages = { "com.wt.rds" })
and my dependency was in gradle
compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa', version: '2.0.4.RELEASE'

Unfortunately, most of the springboot guides on JPA integration test often lack a piece of configuration here and there.
So here is an example that hopefully should just work for you.
Point 1.
My local environment is currently setup to use springboot version:
<version.spring.boot>1.5.9.RELEASE</version.spring.boot>
That being said, I am currently setting up my local environment to be able to run integration tests against multiple databases (e.g. postgres, hsql, h2).
Therefore, I start by googling any random toturial that approaches this problem.
The next link is one such example:
https://www.baeldung.com/spring-testing-separate-data-source
The above example is a good starting point. It allows you to scoop up a valid Entity and a Valid repository. The springboot test class itself, on the other hand, leaves a lot ot be desired.
With the above example, you will immediately struggle with the integration test. You will get the usuable problems about the example not giving you the application.class to configure the integration test, and you are left hanging clueless as to what springboot annotations you need to put "where" to make the test to finally run without explosions.
So now I give you a MINIMAL set of 3 classes (Entity + Repository + SpringbootTest) that should hopefully have 100 percent of the configuration you need. This will serve as a basis of any JPA based integration test you will need to do in the future, then you can swap your entities and repositories, and continue testing with the same type of srpingboot configuration.
I start by giving you the IRRELEVANT classes. The stuff that is always the same, the stuff that you want to test, and that has nothing to do with configuration.
I am referring to REPOSITORY + ENTITY.
In eclipse create your java package:
tutorial.www.baeldung.com.tutorial001jpa.separateDS
Dump into this package the following trivial entity and repository classes, that are based on the tutorial reference I gave above.
Tutorial001GenericEntity
package tutorial.www.baeldung.com.tutorial001jpa.separateDS;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;
#Entity
#Table(name = "TUTORIAL_001_GENERIC_ENTITY")
public class Tutorial001GenericEntity {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
private String value;
public Tutorial001GenericEntity() {
super();
}
public Tutorial001GenericEntity(String value) {
super();
this.value = value;
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
// standard constructors, getters, setters
}
Then we go for the second trivial code snippet.
The spring repository boiler plate code.
Tutorial001GenericEntityRepository
package tutorial.www.baeldung.com.tutorial001jpa.separateDS;
import org.springframework.data.jpa.repository.JpaRepository;
public interface Tutorial001GenericEntityRepository extends JpaRepository<Tutorial001GenericEntity, Long> {
}
At this point your maven project, src/test/java has a total of two classes. The basic stuff.
An entity and a repository, that serve as an example of any integration test you will ever need to do.
So now you go to the only important class in the example, the stuff that always gives a lot of problems, and that is the springboot test class which more then being responsible to test your business logic also has the complex task of CONFIGURING your test.
In this case, this test class has ALL IN ONE the annotations that allow springboot to disocver your entities, repositories, etc...
package tutorial.www.baeldung.com.tutorial001jpa.separateDS;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.domain.EntityScan;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringRunner;
#RunWith(SpringRunner.class)
#ContextConfiguration(classes = {
tutorial.www.baeldung.com.tutorial001jpa.separateDS.Tutorial001GenericEntityIntegrationTest.ConfigureJpa.class })
#SpringBootTest()
public class Tutorial001GenericEntityIntegrationTest {
#EntityScan(basePackageClasses = { Tutorial001GenericEntity.class })
#EnableJpaRepositories(basePackageClasses = Tutorial001GenericEntity.class)
#EnableAutoConfiguration()
public static class ConfigureJpa {
}
#Autowired
private Tutorial001GenericEntityRepository genericEntityRepository;
#Test
public void givenTutorial001GenericEntityRepository_whenSaveAndRetreiveEntity_thenOK() {
Tutorial001GenericEntity genericEntity = genericEntityRepository.save(new Tutorial001GenericEntity("test"));
Tutorial001GenericEntity foundEntity = genericEntityRepository.findOne(genericEntity.getId());
assertNotNull(foundEntity);
assertEquals(genericEntity.getValue(), foundEntity.getValue());
}
}
The important thing, you see, is that this spring boot test has a class level annotation to provide to the springboot test the configuration context.
What we are doing is dumping one and only one class reference that represents our test configuration.
tutorial.www.baeldung.com.tutorial001jpa.separateDS.Tutorial001GenericEntityIntegrationTest.ConfigureJpa.class
And then on this little guy, you put all of the additional annotations in the world you need that springboot offers to configure applications.
In this case we have a dedicated annotation to mention entities.
Another to mention repositories.
And another to tell springboot to activate its auto configuration.
This springboot auto configuration annotation then does additional vodoo, like looking at your classpath and seeing that you have in the classpath say:
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<scope>test</scope>
<version>2.3.4</version>
</dependency>
And it will immediately know how to configure an in memory data source for this database.
Behind the scenes, there might be additional configuration that is getting used.
For example, if you create an application.properties file in your src/test/resources that file will be considered.
It is very to see that the appliction.properties is considered by your running test.
If you want to verify this, make sure that in your test setup you do not have, for example, any dependency on the JDBC driver for postgres.
And then put into your application.properties something liek this:
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
This dialect is not compatible with HSQL or H2, so it will immediately make your green passing integration test blow up.
To be honest, I do not know if there is a simpler combo of annotations to properly configure the springboot scanning for an integration test.
As a rule, I would recommend that you try avoiding having hundreds of thousands of configuration classes in your src/test/resources.
Because if at some point you want to toggle all of your integration tests from using applicat-postgres.proeprties to application-hsql.properties, you might find yourself needing to tweak multiple configuration classes instead of just one.
So as rule, per maven component you write, I would try to have the tests that check repositories extend some sort of MyBaseINtegrationTestClass, and in there put this
#ContextConfiguration(classes = {
tutorial.www.baeldung.com.tutorial001jpa.separateDS.Tutorial001GenericEntityIntegrationTest.ConfigureJpa.class })
So that you only need to play with one configuration for testing for the hole project.
IN any case, hopefully the triplet of classes given here helps you.
One finel thing, for maven dependencies for integration testing, here is what I am using:
<!-- Test Dependencies JPA REPOSITORY TESTS -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>test</scope>
</dependency>
The reason why i am using hsql and h2 is beacuse I want my integration tests to be able to be tunned to either use application-hsql or application-h2.properties.