We Keep Coding

Spring Boot Kafka StreamsConfig or ConsumerConfig from application.yaml not applying

I have a very simple spring boot project with a KTable and I want to customize my configuration in application.yml, but the config seems to not be applied. This is my configuration file application.yml
spring:
kafka:
bootstrap-servers: ${KAFKA_BOOTSTRAP_SERVERS:localhost:9092}
streams:
application-id: ${APPLICATION_ID:train-builder-processor}
buffered-records-per-partition: 50
consumer:
auto-offset-reset: earliest
max-poll-records: ${MAX_POLL_RECORDS:50}
max-poll-interval-ms: ${KAFKA_CONSUMER_MAX_POLL_INTERVAL_MS:1000}
properties:
spring:
json:
trusted:
packages:
- com.example.kafkastream
However, when starting the application the log outputs the following:
2022-03-03 08:20:06.992 INFO 32989 --- [ main] s.r.s.m.t.TrainBuilderApplication : Starting TrainBuilderApplication using Java 16.0.2 on MAPFVFG90ZQQ05P with PID 32989 (/Users/xxx/dev/train-builder-processor/target/classes started by xxx in /Users/xxx/dev/train-builder-processor)
2022-03-03 08:20:06.995 DEBUG 32989 --- [ main] s.r.s.m.t.TrainBuilderApplication : Running with Spring Boot v2.6.3, Spring v5.3.15
2022-03-03 08:20:06.995 INFO 32989 --- [ main] s.r.s.m.t.TrainBuilderApplication : No active profile set, falling back to default profiles: default
2022-03-03 08:20:08.856 INFO 32989 --- [ main] org.apache.kafka.streams.StreamsConfig : StreamsConfig values:
acceptable.recovery.lag = 10000
application.id = test.train-builder-processor
application.server =
bootstrap.servers = [localhost:9092]
buffered.records.per.partition = 1000
... (a bunch of other configs)
ConsumerConfig:
...
max.poll.interval.ms = 300000
max.poll.records = 1000
...
Below is the simple application class I'm using:
#EnableKafka
#EnableKafkaStreams
#SpringBootApplication
public class TrainBuilderApplication {
...
#Autowired
private TrainIdMapper trainIdMapper;
#Autowired
private TrainBuilder trainBuilder;
public static void main(String[] args) {
SpringApplication.run(TrainBuilderApplication.class, args);
}
#Bean
public KTable<String, Train> trainTable(StreamsBuilder kStreamBuilder) {
return kStreamBuilder
.stream(Pattern.compile(sourceTopicsPattern), Consumed.with(Serdes.String(), myJsonSerde))
.map(trainIdMapper)
.filter((key, value) -> key != null)
.groupByKey(Grouped.with(Serdes.String(), mySerde))
.aggregate(() -> null, trainBuilder, trainStore);
}
}
The values from my application.yml seems to be ignored. What could be the cause of this? What am I missing? Thanks in advance!

So I figured it out with the help of How do I properly externalize spring-boot kafka-streams configuration in a properties file?.
Apparently, consumer and producer configs are completely separated from streams config when using a KStream. To set specific properties for the consumer of the kafka stream one must use "additional properties" like so:
spring:
kafka:
bootstrap-servers: ${KAFKA_BOOTSTRAP_SERVERS,localhost:9092}
streams:
application-id: ${APPLICATION_ID:train-builder-processor}
cache-max-size-buffering: 1048576
cleanup.on-shutdown: ${CLEANUP_ON_SHUTDOWN:false}
properties:
max:
poll:
records: 50
which was a bit unintuitive, but it works. Hope this can help someone in the future!

spring webclient load balance

Never used webclient with load balancing before and I fallowed https://spring.io/guides/gs/spring-cloud-loadbalancer/ and implemented webclient load balancer, now I am trying to use helthchecks and having problem.
#Bean
#Primary
ServiceInstanceListSupplier serviceInstanceListSupplier(ConfigurableApplicationContext ctx) {
return ServiceInstanceListSupplier
.builder()
.withRetryAwareness()
.withHealthChecks()
.withBase(new RestCaller("restCaller"))
.build(ctx);
}
and I got the error below
2021-06-27 17:32:01.562 WARN 12252 --- [ parallel-4] o.s.c.l.core.RoundRobinLoadBalancer : No servers available for service: httpbin.org
2021-06-27 17:32:01.564 WARN 12252 --- [ parallel-4] eactorLoadBalancerExchangeFilterFunction : LoadBalancer does not contain an instance for the service httpbin.org
2021-06-27 17:32:01.606 WARN 12252 --- [ parallel-4] o.s.c.l.core.RoundRobinLoadBalancer : No servers available for service: httpbin.org
2021-06-27 17:32:01.606 WARN 12252 --- [ parallel-4] eactorLoadBalancerExchangeFilterFunction : LoadBalancer does not contain an instance for the service httpbin.org
2021-06-27 17:32:01.607 WARN 12252 --- [ parallel-4] o.s.c.l.core.RoundRobinLoadBalancer : No servers available for service: httpbin.org
2021-06-27 17:32:01.607 WARN 12252 --- [ parallel-4] eactorLoadBalancerExchangeFilterFunction : LoadBalancer does not contain an instance for the service httpbin.org
2021-06-27 17:32:01.607 WARN 12252 --- [ parallel-4] o.s.c.l.core.RoundRobinLoadBalancer : No servers available for service: restCaller
2021-06-27 17:32:01.608 WARN 12252 --- [ parallel-4] eactorLoadBalancerExchangeFilterFunction : LoadBalancer does not contain an instance for the service restCaller
when I comment "withHealthChecks()" everything works as expected. My main target is to disable the "DefaultServiceInstance" in case it is failing (means http status 503 or 404 or any error).
I prepared a reproducer at https://github.com/ozkanpakdil/spring-examples/tree/master/web-client-loadbalancer just run "mvn test" you will see the error. you can see the configuration at fhttps://github.com/ozkanpakdil/spring-examples/tree/master/web-client-loadbalancer.

Thanks for providing the sample. Have gone through it. There are 2 issues:
The same #LoadBalanced WebClient.Builder instance is used both for handling the original request and sending health-check requests, so the calls coming out from HealthCheckServiceInstanceListSupplier are done with a load-balanced Webclient instead of a non-load-balanced one. Since at this stage the real hosts are being used, a non-load-balanced Webclient instance should be used for that. You can achieve it by instantiating 2 separate Webclient.Builder beans in your configuration and using qualifier to pass a non-loadbalanced one to the HealthCheckServiceInstanceListSupplier, like so:
#Configuration
#LoadBalancerClient(name = "restCaller", configuration = RestCallerConfiguration.class)
public class WebClientConfig {
#LoadBalanced
#Bean
#Qualifier("loadBalancedWebClientBuilder")
WebClient.Builder loadBalancedWebClientBuilder() {
return WebClient.builder();
}
#Bean
#Qualifier("webClientBuilder")
WebClient.Builder webClientBuilder() {
return WebClient.builder();
}
}
#Configuration
public class RestCallerConfiguration {
#Autowired
#Qualifier("webClientBuilder")
WebClient.Builder webClientBuilder;
#Bean
#Primary
ServiceInstanceListSupplier serviceInstanceListSupplier(ConfigurableApplicationContext ctx) {
return ServiceInstanceListSupplier
.builder()
.withRetryAwareness()
.withHealthChecks(webClientBuilder.build())
.withBase(new RestCaller("restCaller"))
.build(ctx);
}
The HealthCheckServiceInstanceListSupplier sends requests at a health-check URL to verify that the service instance is alive. By default, we assume that the collaborating services have spring-boot-starter-actuator in their dependencies and the request is being sent at th/actuator/health endpoint. Since this endpoint is not configured in httpbin, which the tests use, we get a 404. Changing the health-check path in properties will fix that:
spring.cloud.loadbalancer.health-check.path.default=/
I have pushed a branch with a fixed config here. If you run the test with this setup, it passes.

RabbitMQ trying to connect to localhost

I have a Spring boot application running on embedded tomcat with rabbit listener which I configure like this
#Configuration
public class RabbitConfiguration {
public static final String REQUEST_QUEUE = "from-beeline-req";
public static final String REPLY_QUEUE = "from-beeline-reply";
#Bean
public Queue beelineRpcReqQueue() {
return new Queue(REQUEST_QUEUE);
}
#Bean
public Queue beelineRpcReplyQueue() {
return new Queue(REPLY_QUEUE);
}
#Bean
public RabbitTemplate rabbitTemplate(RabbitTemplateConfigurer configurer, ConnectionFactory connectionFactory) {
RabbitTemplate template = new RabbitTemplate();
configurer.configure(template, connectionFactory);
template.setDefaultReceiveQueue(REQUEST_QUEUE);
template.setReplyAddress(REPLY_QUEUE);
template.setUseDirectReplyToContainer(false);
return template;
}
#Bean
public SimpleMessageListenerContainer replyListenerContainer(ConnectionFactory connectionFactory, RabbitTemplate rabbitTemplate) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueues(beelineRpcReplyQueue());
container.setMessageListener(rabbitTemplate);
return container;
}
}
And my application.yml file looks like this
spring:
main:
banner-mode: LOG
rabbitmq:
host: 172.29.14.45
port: 5672
username: guest
password: guest
template:
reply-timeout: 15000
server:
port: 8888
So the main point is I want to connect to Rabbit server located at exact address (172.29.14.45). Created listener container is trying to connect to localhost instead. It ignores rabbit port property as well.
2021-02-23 23:04:59.715 [replyListenerContainer-1] INFO (AbstractConnectionFactory.java:636) - Attempting to connect to: [localhost:5672]
2021-02-23 23:05:01.721 [replyListenerContainer-1] ERROR (AbstractMessageListenerContainer.java:1877) - Failed to check/redeclare auto-delete queue(s).
org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused: connect
and continues to restart consumer after that
2021-02-23 23:17:49.069 [replyListenerContainer-1] INFO (SimpleMessageListenerContainer.java:1428) - Restarting Consumer#2a140ce5: tags=[[]], channel=null, acknowledgeMode=AUTO local queue size=0
2021-02-23 23:17:49.069 [replyListenerContainer-1] DEBUG (BlockingQueueConsumer.java:758) - Closing Rabbit Channel: null
2021-02-23 23:17:49.071 [replyListenerContainer-2] INFO (AbstractConnectionFactory.java:636) - Attempting to connect to: [localhost:5672]
What should I do to tell spring to use my host property instead of localhost?

I always use the addresses property in the application.properties file
spring.rabbitmq.addresses=amqp://username:password#host:port/vhost
The name of the "virtual host" (or vhost) specifies the namespace for entities (such as exchanges and queues) referred to by the protocol. Note that this is not virtual hosting in the HTTP sense.
https://www.rabbitmq.com/uri-spec.html
example:
spring.rabbitmq.addresses=amqp://ihrpsvpp:In4etuiIkgu7FVBr0tr6wYGvGcGyJ9Ja#lion.rmq.cloudamqp.com/ihrpsvpp

Ok, it turned out it was a bean refreshing context of the application, what caused autoconfiguration to fail

Spring boot 2 connection to rabbitmq via Apache Camel

I have problem with connection to rabbitmq via Apache Camel on Spring Boot 2.
I did following steps:
My dependencies:
implementation "org.apache.camel:camel-spring-boot-starter:${camelVersion}"
implementation "org.apache.camel:camel-jackson-starter:${camelVersion}"
implementation "org.apache.camel:camel-core:${camelVersion}"
implementation "org.apache.camel:camel-rabbitmq-starter:${camelVersion}"
implementation "org.springframework.boot:spring-boot-starter-amqp"
Application.yaml
spring:
rabbitmq:
dynamic: true
host: 192.168.1.1
port: 5672
username: X
password: Y
And I have following route:
#Component
public class BasicRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("direct:loggerQueue")
.id("loggerQueue")
.to("rabbitmq://TEST-QUEUE.exchange?queue=TEST-QUEUE.queue&autoDelete=false&connectionFactory=#rabbitConnectionFactory")
.end();
}
}
Finnaly I have still following issue:
2019-03-06 12:46:05.766 WARN 19464 --- [ restartedMain] o.a.c.c.rabbitmq.RabbitMQProducer : Failed to create connection. It will attempt to connect again when publishing a message.
java.net.ConnectException: Connection refused: connect
Connection seems ok, I tested it. Something is bad with rabbitConnectionFactory.
I don't know what I have bad.

The problem appears to be that RabbitMQComponent is expecting to find a connection factory of type com.rabbitmq.client.ConnectionFactory.
However, the springboot auto-configure is creating a connection factory of type org.springframework.amqp.rabbit.connection.CachingConnectionFactory.
So, whenever the RabbitMQComponent attempts to find the appropriate connection factory, because it is looking for the specific type, and because it does not subclass the rabbitmq ConnectionFactory, it returns a null value, and fails to use the appropriate host name and configuration parameters specified in your application.yml.
You should also see the following in your log if you have debug level set:
2019-12-15 17:58:53.631 DEBUG 48710 --- [ main] o.a.c.c.rabbitmq.RabbitMQComponent : Creating RabbitMQEndpoint with host null:0 and exchangeName: asterix
2019-12-15 17:58:55.927 DEBUG 48710 --- [ main] o.a.c.c.rabbitmq.RabbitMQComponent : Creating RabbitMQEndpoint with host null:0 and exchangeName: asterix-sink
EDIT:
The CachingConnectionFactory is configured with the required Rabbit connection factory as part of the autoconfiguration. However, you need to provide a link to the correct factory.
Therefore, you need to add a #Bean to disambiguate.
#Configuration
#RequiredArgsConstructor
public class CamelConfig {
private final CachingConnectionFactory rabbitConnectionFactory;
#Bean
com.rabbitmq.client.ConnectionFactory rabbitSourceConnectionFactory() {
return rabbitConnectionFactory.getRabbitConnectionFactory();
}
}
and in your endpoint configuration:
rabbitmq:asterix?connectionFactory=#rabbitSourceConnectionFactory
Note that the # is optional, as it gets stripped out within the code when it is trying to find the rabbit connection factory bean.
In your application.yml, configure the connection parameters (the url is no longer included in the endpoint URI).
spring:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest

How rollback transaction after timeout in spring boot application in same way as on weblogic

So in my weblogic application we are you using some jtaWeblogicTransactionManager. There is some default timeout which can be override in annotation #Transactional(timeout = 60). I created some infinity loop to read data from db which correctly timeout:
29 Apr 2018 20:44:55,458 WARN [[ACTIVE] ExecuteThread: '9' for queue: 'weblogic.kernel.Default (self-tuning)'] org.springframework.jdbc.support.SQLErrorCodesFactory : Error while extracting database name - falli
ng back to empty error codes
org.springframework.jdbc.support.MetaDataAccessException: Error while extracting DatabaseMetaData; nested exception is java.sql.SQLException: Unexpected exception while enlisting XAConnection java.sql.SQLExceptio
n: Transaction rolled back: Transaction timed out after 240 seconds
BEA1-2C705D7476A3E21D0AB1
at weblogic.jdbc.jta.DataSource.enlist(DataSource.java:1760)
at weblogic.jdbc.jta.DataSource.refreshXAConnAndEnlist(DataSource.java:1645)
at weblogic.jdbc.wrapper.JTAConnection.getXAConn(JTAConnection.java:232)
at weblogic.jdbc.wrapper.JTAConnection.checkConnection(JTAConnection.java:94)
at weblogic.jdbc.wrapper.JTAConnection.checkConnection(JTAConnection.java:77)
at weblogic.jdbc.wrapper.Connection.preInvocationHandler(Connection.java:107)
at weblogic.jdbc.wrapper.Connection.getMetaData(Connection.java:560)
at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:331)
at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:366)
at org.springframework.jdbc.support.SQLErrorCodesFactory.getErrorCodes(SQLErrorCodesFactory.java:212)
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.setDataSource(SQLErrorCodeSQLExceptionTranslator.java:134)
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.<init>(SQLErrorCodeSQLExceptionTranslator.java:97)
at org.springframework.jdbc.support.JdbcAccessor.getExceptionTranslator(JdbcAccessor.java:99)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:655)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:690)
now I would like to make same behavior in my spring boot application so I tried this:
#EnableTransactionManagement
.
.
.
#Bean(name = "ds1")
#ConfigurationProperties(prefix = "datasource.ds1")
public DataSource logDataSource() {
AtomikosDataSourceBean ds = new AtomikosDataSourceBean();
return ds;
}
#Bean(name = "ds2")
#ConfigurationProperties(prefix = "datasource.ds2")
public DataSource refDataSource() {
AtomikosDataSourceBean ds = new AtomikosDataSourceBean();
return ds;
}
tm:
#Bean(name = "userTransaction")
public UserTransaction userTransaction() throws Throwable {
UserTransactionImp userTransactionImp = new UserTransactionImp();
userTransactionImp.setTransactionTimeout(120);
return userTransactionImp;
}
#Bean(name = "atomikosTransactionManager", initMethod = "init", destroyMethod = "close")
public TransactionManager atomikosTransactionManager() throws Throwable {
UserTransactionManager userTransactionManager = new UserTransactionManager();
userTransactionManager.setForceShutdown(false);
userTransactionManager.setTransactionTimeout(120);
return userTransactionManager;
}
#Bean(name = "transactionManager")
#DependsOn({ "userTransaction", "atomikosTransactionManager" })
public JtaTransactionManager transactionManager() throws Throwable {
UserTransaction userTransaction = userTransaction();
TransactionManager atomikosTransactionManager = atomikosTransactionManager();
return new JtaTransactionManager(userTransaction, atomikosTransactionManager);
}
and application.properties:
datasource.ref.xa-data-source-class-name=oracle.jdbc.xa.client.OracleXADataSource
datasource.ref.unique-resource-name=ref
datasource.ref.xa-properties.URL=jdbc:oracle:thin:#...
datasource.ref.xa-properties.user=...
#datasource.ref.xa-properties.databaseName=...
datasource.ref.password=301d24ae7d0d69614734a499df85f1e2
datasource.ref.test-query=SELECT 1 FROM DUAL
datasource.ref.max-pool-size=5
datasource.log.xa-data-source-class-name=oracle.jdbc.xa.client.OracleXADataSource
datasource.log.unique-resource-name=log
datasource.log.xa-properties.URL=jdbc:oracle:thin:#...
datasource.log.xa-properties.user=...
#datasource.log.xa-properties.databaseName=...
datasource.log.password=e58605c2a0b840b7c6d5b20b3692c5db
datasource.log.test-query=SELECT 1 FROM DUAL
datasource.log.max-pool-size=5
spring.jta.atomikos.properties.log-base-dir=target/transaction-logs/
spring.jta.enabled=true
spring.jta.atomikos.properties.service=com.atomikos.icatch.standalone.UserTransactionServiceFactory
spring.jta.atomikos.properties.max-timeout=600000
spring.jta.atomikos.properties.default-jta-timeout=10000
spring.transaction.default-timeout=900
but with no success. My infinity loop never ends (I wait about 15 minutes and then I stop my app). The only time when I saw rollback was when I tried Thread.sleep and after sleep this transaction timeout with rollback but this is not what I want to. So is there some way how to interrupt process after timeout(use timeout in annotation or use default) in same way how in my weblogic application ?
UPDATE
I tested it like this:
public class MyService {
public void customMethod(){
customDao.readSomething();
}
}
public class CustomDao {
#Transactional(timeout = 120)
public void readSomething()
while(true){
//read data from db. app on weblogic throw timeout, spring boot app in docker did nothing and after 15 I give it up and kill it
}
}
}
UPDATE2
When I turn on atomikos debug I can see there is warning during init and some atomikos timer:
2018-05-03 14:00:54.833 [main] WARN c.a.r.xa.XaResourceRecoveryManager - Error while retrieving xids from resource - will retry later...
javax.transaction.xa.XAException: null
at oracle.jdbc.xa.OracleXAResource.recover(OracleXAResource.java:730)
at com.atomikos.datasource.xa.RecoveryScan.recoverXids(RecoveryScan.java:32)
at com.atomikos.recovery.xa.XaResourceRecoveryManager.retrievePreparedXidsFromXaResource(XaResourceRecoveryManager.java:158)
at com.atomikos.recovery.xa.XaResourceRecoveryManager.recover(XaResourceRecoveryManager.java:67)
at com.atomikos.datasource.xa.XATransactionalResource.recover(XATransactionalResource.java:449)
at com.atomikos.datasource.xa.XATransactionalResource.setRecoveryService(XATransactionalResource.java:416)
at com.atomikos.icatch.config.Configuration.notifyAfterInit(Configuration.java:466)
at com.atomikos.icatch.config.Configuration.init(Configuration.java:450)
at com.atomikos.icatch.config.UserTransactionServiceImp.initialize(UserTransactionServiceImp.java:105)
at com.atomikos.icatch.config.UserTransactionServiceImp.init(UserTransactionServiceImp.java:219)
at com.atomikos.icatch.jta.UserTransactionImp.checkSetup(UserTransactionImp.java:59)
at com.atomikos.icatch.jta.UserTransactionImp.setTransactionTimeout(UserTransactionImp.java:127)
maybe this is the reason. How I can fix this ? I am using oracle 12 with ojdbc8 driver
UPDATE 3
after fix UPDATE2 to grant user permission to db I can see in log warning:
2018-05-03 15:16:30.207 [Atomikos:4] WARN c.a.icatch.imp.ActiveStateHandler - Transaction 127.0.1.1.tm152535336001600001 has timed out and will rollback.
problem is that app is still reading data from db after this timeout. Why it is not rollbacked ?
UPDATE 4
so I found in ActiveStateHandler when timeout occurs there is code:
...
setState ( TxState.ACTIVE );
...
and AtomikosConnectionProxy is checking timeout this way
if ( ct.getState().equals(TxState.ACTIVE) ) ct.registerSynchronization(new JdbcRequeueSynchronization( this , ct ));
else AtomikosSQLException.throwAtomikosSQLException("The transaction has timed out - try increasing the timeout if needed");
so why timeout is set state which not cause exception in AtomikosConnectionProxy ?
UPDATE 5
so I found that property
com.atomikos.icatch.threaded_2pc
will solve my problem and now it starts rollback how I want. But I still dont understand why I should set this to true because now I am testing it on some task which should run in single thread

set com.atomikos.icatch.threaded_2pc=true in jta.properties fixed my problem. Idk why this default value was change to false in web application.
* #param single_threaded_2pc (!com.atomikos.icatch.threaded_2pc)
* If true then commit is done in the same thread as the one that
* started the tx.

XA transactions are horribly complicated and you really want to have a very good reason for using them (ie it's literally impossible to add some business process that removes the need for XA), because you are going to get into trouble out in the wild...
That said, My guess is that it's about timeout discrepancies between XA phases.
With XA there are 2 timeouts - a timeout for the 1st phase, known as the Voting phase (which is typically the one set by the #Transactional annotation, but this depends on the JTA provider) and another timeout for the 2nd phase, known as the commit phase, which is typically a lot longer, because the Transaction Manager has already got the agreement from all parties that the commit is ready to go, and therefore provide greater leeway for things like transient network failures and so on.
My guess is that the WebLogic JTA is simply behaving differently to Atomikos with how it's handling the 2nd phase notifications back from the participants, until atomikos is changed to use the multithreaded ack.
If you application is just you and the database, then you can probably get away without an XA Transaction Manager. I'd expect this would behave the way you want for timeouts.
Good Luck!