I have the access key, secret key, region and DynamoDB endpoint through which I am able to connect to the database and I have kept the configurations in the application.properties file. Now the requirement is to not to keep the access key and secret keys in the application.properties file and follow the Web identity token from AWS STS. I have removed the access key and secret from the application.properties file and have the below configurations. When I start the application I am getting the below message
Consider defining a bean of type 'com.amazonaws.services.dynamodbv2.AmazonDynamoDB' in your configuration.
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig.DefaultTableNameResolver;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig.TableNameOverride;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverterFactory;
import org.socialsignin.spring.data.dynamodb.repository.config.EnableDynamoDBRepositories;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.auth.credentials.WebIdentityTokenFileCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
#Configuration
#EnableDynamoDBRepositories(dynamoDBMapperConfigRef = "dynamoDBMapperConfig", basePackages = "com.xyz.repository")
public class DynamoDBConfiguration {
#Value("${aws.dynamodb.endpoint}")
private String endpoint;
#Value("${aws.region}")
private String region;
#Value("${aws.dynamodb.table.name}")
private String tableName;
#Bean
public DynamoDBMapperConfig dynamoDBMapperConfig(TableNameOverride tableNameOverrider) {
DynamoDBMapperConfig.Builder builder = new DynamoDBMapperConfig.Builder();
builder.withTypeConverterFactory(DynamoDBTypeConverterFactory.standard());
builder.withTableNameResolver(DefaultTableNameResolver.INSTANCE);
builder.withTableNameOverride(tableNameOverrider());
return builder.build();
}
#Bean
public TableNameOverride tableNameOverrider() {
return TableNameOverride.withTableNamePrefix(tableName);
}
#Bean
public DynamoDbClient amazonDynamoDB() {
return DynamoDbClient.builder()
.region(Region.of(region))
.credentialsProvider(WebIdentityTokenFileCredentialsProvider.create())
.build();
}
}
In pom.xml I have the below dependency related to AWS.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>2.15.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependency>
<groupId>io.github.boostchicken</groupId>
<artifactId>spring-data-dynamodb</artifactId>
<version>5.2.5</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>dynamodb</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>sts</artifactId>
</dependency>
You have more then creds issues in your Java code. The bigger issue is you are mixing up V1 and V2.
The Java DynamoDB V1 API is:
com.amazonaws.services.dynamodbv2.*
The Java DyanamoDB V2 API is:
software.amazon.awssdk.services.dynamodb.*
Now your POM file references V2 API:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>2.15.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
You should not mix up V1 and V2. You are using V2 for the Service Client, and V1 for the DynamoDBMapperConfig object. This is not good practice and will not work.
Get rid of V1 API and replace with only V2 API. You seem to want to use DynamoDBMapper functionality. For V2, this is now part of Enhanced Client.
Using the DynamoDB Enhanced Client in the AWS SDK for Java 2.x
To see an AWS Tutorial that shows you how to build a Spring BOOT app using the AWS SDK for Java V2 (including the Enhanced Client), see:
Creating the Amazon DynamoDB web application item tracker
In this tutorial, the EnvironmentVariableCredentialsProvider is used to handle the AWS key values.
Related
I'm trying to build a Backend Service project using the example from the site
Using Spring Cloud Gateway with OAuth 2.0 Patterns
Here is the repository itself
backend
Added dependencies
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.5</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>ru.test.gw.oauth.resource</groupId>
<artifactId>backresource</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>backresource</name>
<description>Demo project for Spring Boot</description>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>14</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
I moved the properties from the quotes-application.properties file to this project
server.port=11002
# Resource server settings
spring.security.oauth2.resourceserver.opaquetoken.introspection-uri=http://localhost:8484/auth/realms/demo/protocol/openid-connect/token/introspect
spring.security.oauth2.resourceserver.opaquetoken.client-id=gateway
spring.security.oauth2.resourceserver.opaquetoken.client-secret=dfdslksfkljweewrfsd
Added a class
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.oauth2.core.DefaultOAuth2AuthenticatedPrincipal;
import org.springframework.security.oauth2.core.OAuth2AuthenticatedPrincipal;
import org.springframework.security.oauth2.server.resource.introspection.ReactiveOpaqueTokenIntrospector;
import reactor.core.publisher.Mono;
// Custom ReactiveTokenIntrospector to map realm roles into Spring GrantedAuthorities
public class KeycloakReactiveTokenInstrospector implements ReactiveOpaqueTokenIntrospector {
private final ReactiveOpaqueTokenIntrospector delegate;
public KeycloakReactiveTokenInstrospector(ReactiveOpaqueTokenIntrospector delegate) {
this.delegate = delegate;
}
#Override
public Mono<OAuth2AuthenticatedPrincipal> introspect(String token) {
return delegate.introspect(token)
.map( this::mapPrincipal);
}
protected OAuth2AuthenticatedPrincipal mapPrincipal(OAuth2AuthenticatedPrincipal principal) {
return new DefaultOAuth2AuthenticatedPrincipal(
principal.getName(),
principal.getAttributes(),
extractAuthorities(principal));
}
protected Collection<GrantedAuthority> extractAuthorities(OAuth2AuthenticatedPrincipal principal) {
//
Map<String,List<String>> realm_access = principal.getAttribute("realm_access");
List<String> roles = realm_access.getOrDefault("roles", Collections.emptyList());
List<GrantedAuthority> rolesAuthorities = roles.stream()
.map(SimpleGrantedAuthority::new)
.collect(Collectors.toList());
Set<GrantedAuthority> allAuthorities = new HashSet<>();
allAuthorities.addAll(principal.getAuthorities());
allAuthorities.addAll(rolesAuthorities);
return allAuthorities;
}
}
And the main class of the project
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.security.oauth2.resource.OAuth2ResourceServerProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.PropertySource;
import org.springframework.security.config.annotation.web.reactive.EnableWebFluxSecurity;
import org.springframework.security.oauth2.server.resource.introspection.NimbusReactiveOpaqueTokenIntrospector;
import org.springframework.security.oauth2.server.resource.introspection.ReactiveOpaqueTokenIntrospector;
import ru.test.gw.oauth.resource.backresource.security.KeycloakReactiveTokenInstrospector;
#SpringBootApplication
//#PropertySource("classpath:quotes-application.properties")
#EnableWebFluxSecurity
public class BackresourceApplication {
public static void main(String[] args) {
SpringApplication.run(BackresourceApplication.class, args);
}
#Bean
public SpringOpaqueTokenIntrospector keycloakIntrospector(OAuth2ResourceServerProperties props) {
NimbusReactiveOpaqueTokenIntrospector delegate = new NimbusReactiveOpaqueTokenIntrospector(
props.getOpaquetoken().getIntrospectionUri(),
props.getOpaquetoken().getClientId(),
props.getOpaquetoken().getClientSecret());
return new KeycloakReactiveTokenInstrospector(delegate);
}
}
And in this class I get an error on SpringOpaqueTokenIntrospector, writes that it is not defined. Although all the imports completely coincide with the training example.
If I add a dependency that the IDE tells me to
import org.springframework.security.oauth2.server.resource.introspection.SpringOpaqueTokenIntrospector;
, then I get an error
Type mismatch: cannot convert from KeycloakReactiveTokenInstrospector to SpringOpaqueTokenIntrospector
What's the problem here? Is there some kind of dependency missing?
I completely repeated the structure of the project from the training material.
So far, I would like to build a project without errors.
Marcus is right in his comment, your keycloakIntrospector #Bean type should be ReactiveOpaqueTokenIntrospector (and not SpringOpaqueTokenIntrospector as declared in your conf)
Few facts:
SpringReactiveOpaqueTokenIntrospector is a ReactiveOpaqueTokenIntrospector but SpringOpaqueTokenIntrospector isn't
your KeycloakReactiveTokenInstrospector is (implements) a ReactiveOpaqueTokenIntrospector too but is neither a SpringReactiveOpaqueTokenIntrospector, SpringOpaqueTokenIntrospector nor OpaqueTokenIntrospector
Side notes
Introspection VS JWT decoding
Keycloak issues JWTs. JWT decoding is far more efficient than introspection: resource-server needs to fetch public-key only once from authorization-server to validate all incoming JWTs when introspection requires to submit access-token to authorization-server for each and every incoming request.
Also, you might not be able to implement multi-tenant scenarios with introspection: how to figure out by which issuer (Keycloak instance or realm) an opaque token was emitted? => you would have to "try" introspection on each issuer until one responds positively :/
Overriding introspector VS providing an authentication converter
If you switch to spring-security 5.8 or higher, customizing introspection is easier: you don't have to override the all introspector but can just provide a ReactiveOpaqueTokenAuthenticationConverter bean instead:
http.oauth2ResourceServer().opaqueToken().authenticationConverter(
(String introspectedToken, OAuth2AuthenticatedPrincipal authenticatedPrincipal) ->
new BearerTokenAuthentication(...));
This bean is called after introspection was successfuly completed (and token attributes retrieved) but before Authentication is instanciated and put in security-context which allows you to just map authorities from any attribute you like or completely switch the authentication implementation.
Simplifying your resource-server configuration
I host a set of libs to ease OAuth2 resource-server testing and configuration. There are various spring-boot starters depending on introspection or JWT decoding is used into servlet or reactive apps.
According to your case (reactive app with introspection), you should have a look at this sample with BearerTokenAuthentication and this other one with a custom authentication.
Configuration can be as simple as:
#EnableReactiveMethodSecurity
#Configuration
public class SecurityConfig {
}
spring.security.oauth2.resourceserver.opaquetoken.introspection-uri=https://localhost:8443/realms/master/protocol/openid-connect/token/introspect
spring.security.oauth2.resourceserver.opaquetoken.client-id=spring-addons-confidential
spring.security.oauth2.resourceserver.opaquetoken.client-secret=change-me
com.c4-soft.springaddons.security.issuers[0].location=https://localhost:8443/realms/master
com.c4-soft.springaddons.security.issuers[0].authorities.claims=realm_access.roles,resource_access.spring-addons-public.roles,resource_access.spring-addons-confidential.roles
# this is probably too permissive, addapt to your needs
com.c4-soft.springaddons.security.cors[0]=/**
<dependency>
<groupId>com.c4-soft.springaddons</groupId>
<artifactId>spring-addons-webflux-introspecting-resource-server</artifactId>
<version>6.0.3</version><!-- warning, this version goes with spring-boot 3.0.0-RC1 -->
</dependency>
I create a real time notification functionality using spring quartz library. I create two services as bellow :
1) quartz-service : Which is used to set schedule a for real time notification.
2) task-service : Which is used to create a task and remind through quartz-service.
When task-service call quartz-service through feign client I'm not get any response. But If I call through Rest Template it's working find.
Actually we are used spring boot microservice architecture, In using Rest Template we need to specify URL Hard coded, So we can't achieved Ribbon concept in this case that's why we not interest to use Rest Template.
So please help me if any once face this problem.
quartz-service :
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-quartz</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
Rest Controller :
#RestController
#RequestMapping(value = "/quartz/taks", produces = "application/hal+json")
public class QuartzTaskController
{
#Autowired
private QuartzTaskServices quartzTaskServices;
#PostMapping("/reminder")
public ResponseEntity<Object> saveTaskReminder(#RequestBody Task task)
{
quartzTaskServices.saveTaskReminderScheduler(task);
return ResponseEntity.ok().build();
}
}
task-service
Dependency :
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>
Feign Client :
#RibbonClient(name="quartz-services")
#FeignClient(name="quartz-services")
public interface QuartzProxy
{
#PostMapping("/quartz/taks/reminder")
public ResponseEntity<Object> saveTaskReminder(#RequestBody Task task);
}
Call Feign Client :
#Autowired
private QuartzProxy quartzProxy;
...
.....
......
quartzProxy.saveTaskReminder(task);
I am attempting to connect to an S3 bucket using the access key and secret key credentials.
This works correctly on my local machine. However, when I try to run it on an EC2 instance the execution seems to stop at the line result = s3Client.listObjectsV2(request);. There are no exceptions. There is simply no response. I would really appreciate any help.
Java code
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(accesskey, secretkey)))
.withRegion(region).build();
ListObjectsV2Result result = null;
List<S3ObjectSummary> objects = null;
String continuationToken = null;
System.out.println("Starting loop to request information");
int count = 1;
do {
ListObjectsV2Request request = new ListObjectsV2Request();
request.setBucketName(bucket);
request.setContinuationToken(continuationToken);
System.out.println("Placing request information #" + count);
result = s3Client.listObjectsV2(request);
System.out.println("Got response for request #" + count++);
continuationToken = result.getNextContinuationToken();
objects = result.getObjectSummaries();
for (S3ObjectSummary os : objects) {
System.out.println(os.getKey());
}
} while (continuationToken != null);
pom.xml
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.466</version>
</dependency>
S3 Bucket policy
{
"Version": "2012-10-17",
"Id": "Policy1563965234895",
"Statement": [
{
"Sid": "Stmt1563965231235",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::xxxxxxxxxxxx:user/xyz_dev",
]
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::xxxx-yyy-bucket",
"arn:aws:s3:::xxxx-yyy-bucket/*"
]
}
]
}
Thank for your responses. I had multiple issues with the code (it was not an issue with the Amazon S3)
It was the infamous error java.lang.NoSuchFieldError: SIGNING_REGION but only occurring on EC2. It was not caught in the try-catch block surrounding the code but was in the HTTP response.
My project is on Spring Boot it had incorrectly imported different versions of aws-sdk modules
I had another POM entry hadoop-aws that had its own version of aws-sdk
Fix:
Added individual aws-sdk module entries instead of the full aws-java-sdk
com.amazonaws
aws-java-sdk-cognitoidp
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-core -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-cognitoidentity -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-cognitoidentity</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-cognitoidp -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-cognitoidp</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-kms -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-kms</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3 -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
</dependency>
Added exclusions to hadoop-aws
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-aws</artifactId>
<version>3.1.1</version>
<exclusions>
<exclusion>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bundle</artifactId>
</exclusion>
</exclusions>
</dependency>
You mention that your project is a Spring BOOT project.
We are working on a document that will show you how to write a Spring BOOT application that invokes AWS Services (in the document, DynamoDB is used as an example) and deploy it to AWS Elastic Beanstalk.
When you do so, there are a few things you need to do to make a Spring BOOT app work, such as:
Set the port that Spring Boot listens on by adding a new environment variable named SERVER_PORT, with the value 5000.
Add a new variable named AWS_ACCESS_KEY_ID and specify your access
key value.
Add a new variable named AWS_SECRET_ACCESS_KEY and specify your secret key value.
To create an AWS Service client, use a EnvironmentVariableCredentialsProvider - like this to use the environment variables.
Region region = Region.US_EAST_1; DynamoDbClient ddb =
DynamoDbClient.builder()
.region(region)
.credentialsProvider(EnvironmentVariableCredentialsProvider.create())
.build();
When the document is done, I will post it here.
Hope this helps...
Anybody know why it doesn't work?
Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
06/04/2017 14:11:24.732 ERROR [main] - org.springframework.boot.SpringApplication: Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jpaMappingContext': Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: At least one JPA metamodel must be present!
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:742)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:866)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1162)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1151)
at com.cadit.web.WebApplicationAware.main(WebApplicationAware.java:19)
Caused by: java.lang.IllegalArgumentException: At least one JPA metamodel must be present!
at org.springframework.util.Assert.notEmpty(Assert.java:277)
at org.springframework.data.jpa.mapping.JpaMetamodelMappingContext.<init>(JpaMetamodelMappingContext.java:52)
at org.springframework.data.jpa.repository.config.JpaMetamodelMappingContextFactoryBean.createInstance(JpaMetamodelMappingContextFactoryBean.java:71)
at org.springframework.data.jpa.repository.config.JpaMetamodelMappingContextFactoryBean.createInstance(JpaMetamodelMappingContextFactoryBean.java:26)
at org.springframework.beans.factory.config.AbstractFactoryBean.afterPropertiesSet(AbstractFactoryBean.java:134)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624)
... 16 common frames omitted
I defined entities in com.cadit.entities:
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;
#Entity
#Table(name="TEST")
public class GenericBeans implements BeanType, IEntity<Long> {
/**
*
*/
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
#Column(name = "TEST_PAID")
protected Long id;
#Column(name = "SOCIETA")
private String SocietaCod;
#Column(name = "CONTO_INTERMEDIARIO")
private String contoInt;
#Column(name = "TIPO_OPERAZIONE")
private String tipoOpe;
public GenericBeans(String societaCod, String contoInt, String tipoOpe) {
SocietaCod = societaCod;
this.contoInt = contoInt;
this.tipoOpe = tipoOpe;
}
public GenericBeans() {
}
public String getSocietaCod() {
return SocietaCod;
}
public void setSocietaCod(String societaCod) {
SocietaCod = societaCod;
}
public String getContoInt() {
return contoInt;
}
public void setContoInt(String contoInt) {
this.contoInt = contoInt;
}
public String getTipoOpe() {
return tipoOpe;
}
public void setTipoOpe(String tipoOpe) {
this.tipoOpe = tipoOpe;
}
#Override
public String toString() {
return "CSV [SocietaCod=" + SocietaCod + ", contoInt=" + contoInt + ", tipoOpe=" + tipoOpe + "]";
}
#Override
public Long getId() {
return this.id;
}
#Override
public void setId(Long id) {
this.id=id;
}
}
I definied my datasource entry definition for spring:
import org.apache.log4j.Logger;
import org.springframework.boot.autoconfigure.domain.EntityScan;
import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.transaction.annotation.EnableTransactionManagement;
#Configuration
#ComponentScan
#EntityScan("com.cadit.entities")
//#EnableJpaRepositories("com.cadit.entities")
#EnableTransactionManagement
#PropertySource("classpath:db-config.properties")
public class DbAutoConfiguration {
static final Logger logger = Logger.getLogger(DbAutoConfiguration.class);
public DbAutoConfiguration() {
}
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource(){
//DataSource ds =new EmbeddedDatabaseBuilder().addScript("classpath:sql/schema.sql").addScript("classpath:testdb/data.sql").build();
DataSourceBuilder ds = DataSourceBuilder.create();
logger.info("dataSource = " + ds);
return ds.build();
}
}
My db-config.properties is:
spring.jpa.hibernate.ddl-auto: validate
spring.jpa.hibernate.naming_strategy: org.hibernate.cfg.ImprovedNamingStrategy
#spring.jpa.database: SQL
spring.jpa.show-sql: true
spring.datasource.driverClassName=net.sourceforge.jtds.jdbc.Driver
spring.datasource.url=jdbc:jtds:sqlserver://localhost:1433;databaseName=example
spring.datasource.username=xxx
spring.datasource.password=xxx
IEntity is:
public interface IEntity <I extends Serializable> extends Serializable{
/**
* Property rappresenta la primary key.
*/
String P_ID = "id";
/**
* restituisce la primary key
* #return
*/
I getId();
/**
* imposta la primary key
* #param id
*/
void setId(I id);
}
I try to write CSV file to database using CrudRepository interface of spring:
import java.io.File;
import java.util.Collections;
import java.util.LinkedList;
import java.util.List;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.repository.CrudRepository;
import com.cadit.entities.GenericBeans;
import com.csvreader.CsvReader;
public class CsvReaders {
static final Logger logger = Logger.getLogger(CsvReader.class);
#Autowired
public CrudRepository<GenericBeans,Long> _entitymanager;
public List loadDataFromCsv(String fileName) {
try {
File file = new ClassPathResource(fileName).getFile();
CsvReader csv = new CsvReader(file.getAbsoluteFile().getPath(),';');
csv.readHeaders();
List l = new LinkedList();
GenericBeans b = new GenericBeans ();
while (csv.readRecord())
{
b.setSocietaCod(csv.get(0));
b.setContoInt(csv.get(1));
b.setTipoOpe(csv.get(2));
_entitymanager.save(b); //persist on db
l.add(b);
b = new GenericBeans();
}
b=null;
return l;
} catch (Exception e) {
logger.error("Error occurred while loading object list from file " + fileName, e);
return Collections.emptyList();
}
}
}
I DO NOT use main class but a class which extend SpringBootServletInitializer because i want to run it on both standalone tomcat and Tomcat installation as WAR application
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.support.SpringBootServletInitializer;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
#Configuration
#ComponentScan(basePackages={"com.cadit.entities","com.cadit.beans"})
#EnableAutoConfiguration
public class WebApplicationAware extends SpringBootServletInitializer {
private static Class<WebApplicationAware> applicationClass = WebApplicationAware.class;
public static void main(String[] args) {
SpringApplication.run(applicationClass, args);
}
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(applicationClass);
}
}
All properties file are in classpath resources because it's a maven project.
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.springframework</groupId>
<artifactId>xxxx</artifactId>
<version>0.1.0</version>
<packaging>war</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.2.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-jpa</artifactId>
<version>1.11.1.RELEASE</version>
</dependency>
<dependency>
<groupId>javax.persistence</groupId>
<artifactId>persistence-api</artifactId>
<version>1.0.2</version>
</dependency>
<!-- altre dipendenze non spring -->
<!-- https://mvnrepository.com/artifact/net.sourceforge.javacsv/javacsv -->
<dependency>
<groupId>net.sourceforge.javacsv</groupId>
<artifactId>javacsv</artifactId>
<version>2.0</version>
</dependency>
<!-- per jpa solo se si usa il Tomcat embedded -->
<dependency>
<groupId>net.sourceforge.jtds</groupId>
<artifactId>jtds</artifactId>
<version>1.3.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-dbcp2</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
<version>2.0</version>
<scope>provided</scope>
</dependency>
<!-- end -->
<!-- dipendenze logback -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.7.5</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.1.7</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>1.1.7</version>
</dependency>
<!-- fine dip logback -->
</dependencies>
<properties>
<start-class>hello.WebApplicationAware</start-class>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
<repositories>
<repository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</pluginRepository>
</pluginRepositories>
</project>
What's the problem, why doesn't it find JPA entities when I run WebApplicationAware class?
Spring does not find any JPA Entities, so no JPA Meta Model is created, that is why you face the exception.
The cause of this problem may be a wrong persistence-api version on your class path.
You are using
<dependency>
<groupId>javax.persistence</groupId>
<artifactId>persistence-api</artifactId>
<version>1.0.2</version>
</dependency>
but I am pretty shure your spring version uses persistence-api version 2.
Could it be, you are using #Entity annotation from version 1 ?
At runtime spring uses version 2, and this is searching for Entites using #Entity from version 2 only !
Remove the dependencies
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-jpa</artifactId>
<version>1.11.1.RELEASE</version>
</dependency>
Instead add
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
This will give you all JPA dependencies in the right version.
I solved it by adding 2 annotations
#EnableAutoConfiguration
#EntityScan(basePackages = { "com.wt.rds" })
and my dependency was in gradle
compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa', version: '2.0.4.RELEASE'
Unfortunately, most of the springboot guides on JPA integration test often lack a piece of configuration here and there.
So here is an example that hopefully should just work for you.
Point 1.
My local environment is currently setup to use springboot version:
<version.spring.boot>1.5.9.RELEASE</version.spring.boot>
That being said, I am currently setting up my local environment to be able to run integration tests against multiple databases (e.g. postgres, hsql, h2).
Therefore, I start by googling any random toturial that approaches this problem.
The next link is one such example:
https://www.baeldung.com/spring-testing-separate-data-source
The above example is a good starting point. It allows you to scoop up a valid Entity and a Valid repository. The springboot test class itself, on the other hand, leaves a lot ot be desired.
With the above example, you will immediately struggle with the integration test. You will get the usuable problems about the example not giving you the application.class to configure the integration test, and you are left hanging clueless as to what springboot annotations you need to put "where" to make the test to finally run without explosions.
So now I give you a MINIMAL set of 3 classes (Entity + Repository + SpringbootTest) that should hopefully have 100 percent of the configuration you need. This will serve as a basis of any JPA based integration test you will need to do in the future, then you can swap your entities and repositories, and continue testing with the same type of srpingboot configuration.
I start by giving you the IRRELEVANT classes. The stuff that is always the same, the stuff that you want to test, and that has nothing to do with configuration.
I am referring to REPOSITORY + ENTITY.
In eclipse create your java package:
tutorial.www.baeldung.com.tutorial001jpa.separateDS
Dump into this package the following trivial entity and repository classes, that are based on the tutorial reference I gave above.
Tutorial001GenericEntity
package tutorial.www.baeldung.com.tutorial001jpa.separateDS;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;
#Entity
#Table(name = "TUTORIAL_001_GENERIC_ENTITY")
public class Tutorial001GenericEntity {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
private String value;
public Tutorial001GenericEntity() {
super();
}
public Tutorial001GenericEntity(String value) {
super();
this.value = value;
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
// standard constructors, getters, setters
}
Then we go for the second trivial code snippet.
The spring repository boiler plate code.
Tutorial001GenericEntityRepository
package tutorial.www.baeldung.com.tutorial001jpa.separateDS;
import org.springframework.data.jpa.repository.JpaRepository;
public interface Tutorial001GenericEntityRepository extends JpaRepository<Tutorial001GenericEntity, Long> {
}
At this point your maven project, src/test/java has a total of two classes. The basic stuff.
An entity and a repository, that serve as an example of any integration test you will ever need to do.
So now you go to the only important class in the example, the stuff that always gives a lot of problems, and that is the springboot test class which more then being responsible to test your business logic also has the complex task of CONFIGURING your test.
In this case, this test class has ALL IN ONE the annotations that allow springboot to disocver your entities, repositories, etc...
package tutorial.www.baeldung.com.tutorial001jpa.separateDS;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.domain.EntityScan;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringRunner;
#RunWith(SpringRunner.class)
#ContextConfiguration(classes = {
tutorial.www.baeldung.com.tutorial001jpa.separateDS.Tutorial001GenericEntityIntegrationTest.ConfigureJpa.class })
#SpringBootTest()
public class Tutorial001GenericEntityIntegrationTest {
#EntityScan(basePackageClasses = { Tutorial001GenericEntity.class })
#EnableJpaRepositories(basePackageClasses = Tutorial001GenericEntity.class)
#EnableAutoConfiguration()
public static class ConfigureJpa {
}
#Autowired
private Tutorial001GenericEntityRepository genericEntityRepository;
#Test
public void givenTutorial001GenericEntityRepository_whenSaveAndRetreiveEntity_thenOK() {
Tutorial001GenericEntity genericEntity = genericEntityRepository.save(new Tutorial001GenericEntity("test"));
Tutorial001GenericEntity foundEntity = genericEntityRepository.findOne(genericEntity.getId());
assertNotNull(foundEntity);
assertEquals(genericEntity.getValue(), foundEntity.getValue());
}
}
The important thing, you see, is that this spring boot test has a class level annotation to provide to the springboot test the configuration context.
What we are doing is dumping one and only one class reference that represents our test configuration.
tutorial.www.baeldung.com.tutorial001jpa.separateDS.Tutorial001GenericEntityIntegrationTest.ConfigureJpa.class
And then on this little guy, you put all of the additional annotations in the world you need that springboot offers to configure applications.
In this case we have a dedicated annotation to mention entities.
Another to mention repositories.
And another to tell springboot to activate its auto configuration.
This springboot auto configuration annotation then does additional vodoo, like looking at your classpath and seeing that you have in the classpath say:
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<scope>test</scope>
<version>2.3.4</version>
</dependency>
And it will immediately know how to configure an in memory data source for this database.
Behind the scenes, there might be additional configuration that is getting used.
For example, if you create an application.properties file in your src/test/resources that file will be considered.
It is very to see that the appliction.properties is considered by your running test.
If you want to verify this, make sure that in your test setup you do not have, for example, any dependency on the JDBC driver for postgres.
And then put into your application.properties something liek this:
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
This dialect is not compatible with HSQL or H2, so it will immediately make your green passing integration test blow up.
To be honest, I do not know if there is a simpler combo of annotations to properly configure the springboot scanning for an integration test.
As a rule, I would recommend that you try avoiding having hundreds of thousands of configuration classes in your src/test/resources.
Because if at some point you want to toggle all of your integration tests from using applicat-postgres.proeprties to application-hsql.properties, you might find yourself needing to tweak multiple configuration classes instead of just one.
So as rule, per maven component you write, I would try to have the tests that check repositories extend some sort of MyBaseINtegrationTestClass, and in there put this
#ContextConfiguration(classes = {
tutorial.www.baeldung.com.tutorial001jpa.separateDS.Tutorial001GenericEntityIntegrationTest.ConfigureJpa.class })
So that you only need to play with one configuration for testing for the hole project.
IN any case, hopefully the triplet of classes given here helps you.
One finel thing, for maven dependencies for integration testing, here is what I am using:
<!-- Test Dependencies JPA REPOSITORY TESTS -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>test</scope>
</dependency>
The reason why i am using hsql and h2 is beacuse I want my integration tests to be able to be tunned to either use application-hsql or application-h2.properties.
Dependencies
org.springframework.cloud:spring-cloud-starter-feign:jar:1.2.2.RELEASE:compile
com.netflix.feign:feign-core:jar:8.16.2:compile
com.netflix.feign:feign-slf4j:jar:8.16.2:compile
com.netflix.feign:feign-jackson:jar:8.15.1:compile
Enabling Feign on SpringBootAppilication
#EnableFeignClients(basePackages = "com.vett.services.bucket.restclient")
Feign interface Client
#FeignClient(name = "myClient", configuration = ClientConfigs.class, url = "https://my-endpoint");
public interface MyClient {
Results in this error
org.springframework.core.annotation.AnnotationConfigurationException: Attribute 'value' in annotation [org.springframework.cloud.netflix.feign.FeignClient] must be declared as an #AliasFor [serviceId], not [name]
So far I have
As its unclear to me what the issue is i have used the value instead of name, my searching has not been successful i have see a few issues with feign annotation but not appear to be similar to this at all
I was getting the same issue, Once I added the below dependency , it started working :
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:Brixton.SR7"}
}
I am using Spring boot 1.4 but Spring 4.3.6. Also Spring feign 1.2.5.RELEASE
This error may occur when using multiple feign clients or bad package architecture. Sometimes this error occurs due to version incompatibilities, but in some projects we may not be able to change the versions. Therefore, you can solve the problem with the following codes. This codes worked for me.
Use this annotation in ApplicationStarter class:
#EnableFeignClients
Feign Client Interface:
import org.springframework.cloud.netflix.feign.FeignClient;
#FeignClient(value = "account-service", url = "${feign.client.account-service}", path = "/account/api/v1")
public interface AccountServiceClient {
#RequestLine("POST /customer/{email}/?name={accountName}")
Long registerCustomer(#Param("email") String email, #Param("accountName") String accountName);
}
Define bean for multiple feign usage:
#Bean
#Qualifier("account-feign-client")
public AccountServiceClient accountServiceClient() {
return Feign.builder().target( AccountServiceClient.class,"${feign.client.account-service}");
}
#Bean
#Qualifier("mail-feign-client")
public MailServiceClient mailServiceClient() {
return Feign.builder().target( MailServiceClient.class,"${feign.client.mail-service}");
}
Autowire in service:
#Autowired
#Qualifier("account-feign-client")
private AccountServiceClient accountServiceClient;
pom.xml:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring.boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Brixton.SR7</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-feign</artifactId>
<version>1.4.7.RELEASE</version>
</dependency>
...
</dependencies>