Spring cloud data flow consumer failing to load consumer configuration properties - java

I have a spring cloud data flow processor that I need to increase the fetch.max.bytes value. Based documentation from here and responses in the following questions Question 1 and Question 2.
My yml contains:
spring:
cloud:
stream:
kafka:
bindings:
splitIncomingFile-in-0:
consumer:
configuration.fetch.max.bytes: 2147483647
Upon start up, the application fails to start, with the below error:
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to bind properties under 'spring.cloud.stream.kafka.bindings.splitincomingfile-out-0' to org.springframework.cloud.stream.binder.kafka.properties.KafkaBindingProperties:
Reason: org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.String] to type [org.springframework.cloud.stream.binder.kafka.properties.KafkaBindingProperties]
Action:
Update your application's configuration
Everything I have tried is not remedying the situation does anyone have a solution.

Related

Spring cloud Stream Kafka, Inject value from Environment variables ie retryable-exceptions section

I am using Spring cloud Stream Kafka Binder and I have a requirement to inject retryable-exceptions values from environment variable, I tried below but I am getting error "No Convertor found capable of converting from type [java.lang.String] to type [java.util.Map<java.lang.Class<java.lang.Throwable>,java.lang.Boolean>]"
spring:
cloud:
stream:
bindings:
consumer-in:
consumer:
retryable-exceptions: {env.map.retryableExceptions:{}}
What is a correct syntax to inject Map from environment variable?
Thanks

Spring Boot with GKE: Retry loading ConfigMap not working as expected by spring cloud kubernetes documentation

I have a spring boot (2.7.0) with spring-cloud-depependencies (2021.0.3) under GKE, where since I noticed Kubernetes api client is failing to connect but just few milliseconds later succeeding (And I can't figure out the reason why yet, although one detail is GKE cluster was upgraded, but that is another story), I rather on the meantime to fail fast the spring boot application and try to wait till the configmap is ready to be read.
In order to do so, I configured my bootstrap as follows:
profiles: some-profile
cloud:
kubernetes:
config:
retry:
initial-interval: 1000
max-attempts: 6
max-interval: 2000
multiplier: 1.1
fail-fast: true
enabled: true
enableApi: true
namespace: some-namespace
name: someConfigmapName
reload:
enabled: true
monitoring-config-maps: true
mode: event
strategy: refresh
I added the respective dependencies as well:
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
As those are the steps declared here: https://docs.spring.io/spring-cloud-kubernetes/docs/current/reference/html/#configmap-propertysource
You can also make your application retry loading ConfigMap property sources on a failure. First, you need to set spring.cloud.kubernetes.config.fail-fast=true. Then you need to add spring-retry and spring-boot-starter-aop to your classpath. You can configure retry properties such as the maximum number of attempts, backoff options like initial interval, multiplier, max interval by setting the spring.cloud.kubernetes.config.retry.* properties.
But I'm getting as an error:
APPLICATION FAILED TO START
***************************
Description:
A component required a bean named 'configServerRetryInterceptor' that could not be found.
Action:
Consider defining a bean named 'configServerRetryInterceptor' in your configuration.
Why should I add the configServerRetryInterceptor bean? shouldn't that one instantiated automatically based on the steps declared in the the documentation? Regardless if that, if I create such a #bean in a class annotated with #Configuration it doesn't work either.
What step am I missing?

The URL 'redis://123456#cruise-redis-master.reddwarf-cache.svc.cluster.local :6379' is not valid for configuring Spring Data Redis

When I upgrade the spring boot to 2.6.6 and Jedis version to 4.0.0. The spring boot application show error like this:
23:24:20:970] [ERROR] - org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter.report(LoggingFailureAnalysisReporter.java:40) -
***************************
APPLICATION FAILED TO START
***************************
Description:
The URL 'redis://123456#cruise-redis-master.reddwarf-cache.svc.cluster.local
:6379' is not valid for configuring Spring Data Redis.
Action:
Review the value of the property 'spring.redis.url'.
I am sure the Redis password was right and the connection string did not change, the connections thing works fine in the lower version of Jedis. why did this happened? what should I do to fix this problem?

How to implement the Redis Rate Limiting in spring cloud gateway?

I am trying to set up the rate-limiting in a spring cloud gateway.
I tried to configure the filter in the application.yaml as below
filters:
- name: RequestRateLimiter
args:
redis-rate-limiter.replenishRate: 1
redis-rate-limiter.burstCapacity: 5
I am getting the below error
java.lang.IllegalArgumentException: Unable to find GatewayFilterFactory with name RequestRateLimiter
How to implement the rate limit in the spring cloud gateway?
One of the possibilities is not including the following dependency (the mistake which I did first time)
org.springframework.boot:spring-boot-starter-data-redis-reactive
Since the args contain redis-rate-limiter this dependency is must.
Below is a screenshot from the IDE showing RequestRateLimiterGatewayFilterFactory getting added when the above dependency is included.

Start Spring Boot context with remote config from JUnit Test

I am writing integration tests in a microservice environment. I want to fully initialize my Spring Boot application context once when running my JUnit test class. I can do this easily enough under normal circumstances, but in this case I need to load remote configuration from a Git repository (I am using spring-cloud-config). Under normal startup conditions, this Spring Boot application does successfully load the configuration from the Git repo. I am just trying to replicate the same context initialization during the test phase. Here is how I have written the test class:
#RunWith(SpringRunner.class)
#SpringBootTest(properties={"ENV=local","spring.cloud.config.enabled=true","spring.cloud.config.uri=http://config.server.com/config","ENCRYPT_KEY=secret"})
#ContextConfiguration(classes ={MyAppConfig.class,MyDBConfig.class})
#PropertySource({"classpath:spring-cloud-config.properties"})
public class MyIntegrationTest {
#Test
public void testFindUsersForCorp() {
System.out.println("Test is running ");
}
}
spring-cloud-config.properties:
spring.profiles.active=${ENV}
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration
spring.cloud.consul.host=${CONSUL.HOST:localhost}
spring.cloud.consul.port=${CONSUL.PORT:8500}
spring.cloud.consul.discovery.healthCheckPath=/custom/health
spring.cloud.consul.discovery.healthCheckInterval=30s
spring.cloud.config.uri=${CONFIG.SERVER.URI:http://config.server:80/config}
spring.cloud.config.username=user
spring.cloud.config.password=${PASSWORD}
Logging output during Spring context initialization:
main WARN: Could not locate PropertySource: I/O error on GET request for "http://localhost:8888/userdata-service/default": Connection refused: connect; nested exception is java.net.ConnectException: Connection refused: connect
main WARN: Cannot enhance #Configuration bean definition 'refreshScope' since its singleton instance has been created too early. The typical cause is a non-static #Bean method with a BeanDefinitionRegistryPostProcessor return type: Consider declaring such methods as 'static'.
main ERROR:
***************************
APPLICATION FAILED TO START
***************************
Description:
Cannot determine embedded database driver class for database type NONE
[OMITTED]
As seen above, the startup fails because no driver class for my persistance layer is defined. Since the relevant config is in Git, this error indicates that the remote configuration is not being fetched.
I appreciate any help anyone can offer. I will update this question with more information if needed.
Spring Cloud Config client should be initialized in bootstrap phase, so you should move corresponding properties from spring-cloud-config.properties to bootstrap.properties
See https://cloud.spring.io/spring-cloud-config/multi/multi__spring_cloud_config_client.html#config-first-bootstrap

Categories