JedisConnectionException - Read timed out error - java

I'm trying to connect to AWS ElastiCache Redis using Spring Data Redis + Jedis combination. [Redis Cluster enabled, so it has Cluster Config endpoint, with 3 shard - each shard has 1 primary node + 2 replica nodes ]
I'm getting Read timed out error.
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
AWS Redis Server Version : 5.0.3 /
Cluster Mode : Enabled /
SSL : Enabled /
Auth : Enabled ( by password )
Library --
Spring-data-redis : 2.1.6.Release /
jedis : 2.9.0
Telnet works to AWS Redis all nodes and cluster config endpoint at 6379 ports.
I tried Redisson by itself, it connects to AWS Redis, with out any issue.
So, no issues with Redis itself, issue with Spring Data Redis in combination with Jedis.
My Code looks like this -
RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration();
redisClusterConfiguration.setClusterNodes(listOfRedisNode);
redisClusterConfiguration.setPassword(passwordString);
JedisClientConfiguration.JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();
jedisClientConfiguration.connectTimeout(Duration.ofSeconds(60));
jedisClientConfiguration.useSsl();
jedisClientConfiguration.usePooling();
JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(redisClusterConfiguration, jedisClientConfiguration.build() );
jedisConnectionFactory.afterPropertiesSet();
final RedisTemplate<String, Serializable> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(jedisConnectionFactory);
redisTemplate.setKeySerializer(new JdkSerializationRedisSerializer());
redisTemplate.setValueSerializer(new JdkSerializationRedisSerializer());
redisTemplate.afterPropertiesSet();
System.out.println(redisTemplate.getClientList().size());
StringRedisConnection stringRedisConnectionlettuce = new DefaultStringRedisConnection(redisTemplate.getConnectionFactory().getConnection());
final String message2 = stringRedisConnectionlettuce.echo("Hello");
System.out.println("Hello".equals(message2));
Read time out error -
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
at redis.clients.jedis.Protocol.process(Protocol.java:151)
at redis.clients.jedis.Protocol.read(Protocol.java:215)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:239)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:96)
at redis.clients.jedis.Connection.sendCommand(Connection.java:126)
at redis.clients.jedis.Connection.sendCommand(Connection.java:117)
at redis.clients.jedis.BinaryClient.auth(BinaryClient.java:564)
at redis.clients.jedis.BinaryJedis.auth(BinaryJedis.java:2138)
at redis.clients.jedis.JedisClusterConnectionHandler.initializeSlotsCache(JedisClusterConnectionHandler.java:36)
at redis.clients.jedis.JedisClusterConnectionHandler.<init>(JedisClusterConnectionHandler.java:17)
at redis.clients.jedis.JedisSlotBasedConnectionHandler.<init>(JedisSlotBasedConnectionHandler.java:24)
at redis.clients.jedis.BinaryJedisCluster.<init>(BinaryJedisCluster.java:54)
at redis.clients.jedis.JedisCluster.<init>(JedisCluster.java:93)
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(JedisConnectionFactory.java:418)
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(JedisConnectionFactory.java:388)
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.afterPropertiesSet(JedisConnectionFactory.java:345)
at io.github.deepshiv126.springdataredis.example.MySpringBootApplication.main(MySpringBootApplication.java:306)
... 8 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:127)
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:196)
... 27 more
I looked into Spring Source Code and Jedis Source Code -- My assumption its not using SSL Connection ;
JedisConnectionFactory - afterPropertiesSet()-- trying to create Cluster -- under that it's trying initializeSlotsCache, which issued AUTH command to Redis Server, with password -- This is where "Read timed out" is occuring;
I understand local redis - you can go inside and run auth command to get authenticate. But I guess AWS Redis may not able to do that , its needs to have SSL Connection even before it runs AUTH command - Why Jedis is not using SSL Connection ?
This another thread "Cannot get Jedis connection" when using SSL with Redis and Spring Data Redis
says, use something like JedisPool - but spring-data-redis' JedisConnectionFactory doesn't accepts JedisPool. Is there any other way to do that ?
JedisPool jedisPool = new JedisPool("rediss://" + clusterConfigEndPoint + ":6379");
Another question - other libraries use redis ssl connection as rediss:// - how to Jedis Client to use SSL connection,
Any help will be really appreciated!!
Thanks!

I was facing this for quite a while, however with below configuration it works perfectly with AWS ElasticCache - Redis with SSL Enabled (Encryption-in-transit & Encryption-at-rest enabled).
Below Maven dependencies are used as part of springboot application
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<!-- Latest jedis with SSL support -->
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>3.6.0</version>
</dependency>
#Component
public class RedisConfig {
#Bean
public JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();
redisStandaloneConfiguration.setHostName("PrimaryEndpoint of AWS Elastic Cache Cluster");
redisStandaloneConfiguration.setPort(6379);
redisStandaloneConfiguration.setUsername("userName");
redisStandaloneConfiguration.setPassword(new String("password").toCharArray());
JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();
jedisClientConfiguration.connectTimeout(Duration.ofSeconds(60));// 60s connection timeout
jedisClientConfiguration.useSsl();
jedisClientConfiguration.usePooling();
JedisConnectionFactory jedisConFactory = new JedisConnectionFactory(redisStandaloneConfiguration,
jedisClientConfiguration.build());
jedisConFactory.afterPropertiesSet();
return jedisConFactory;
}
#Bean(value = "redisTemplate")
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(redisConnectionFactory);
return redisTemplate;
}
}

Related

JMS client to ActiveMQ at server on Red Hat JBoss EAP 7.3 in Java

I am writing a Java client to connect to my JBoss EAP 7.3 server running ActiveMQ, and I am getting various connection responses as I alter the parameters. Please help me correct the parameters/code. I get:
09:46:57.227 [main] INFO org.xnio.nio - XNIO NIO Implementation Version 3.4.6.Final
09:46:57.606 [Remoting "config-based-naming-client-endpoint" I/O-1] DEBUG org.xnio.nio - Started channel thread 'Remoting "config-based-naming-client-endpoint" I/O-1', selector sun.nio.ch.WindowsSelectorImpl#17ab1d7e ...
jboss.naming.client.connect.options. has the following options {}
09:46:57.763 [main] DEBUG org.jboss.naming.remote.client.HaRemoteNamingStore - Failed to connect to server http-remoting://127.0.0.1:8080
java.lang.RuntimeException: java.io.IOException: For now upgrade responses must have a content length of zero.
at org.jboss.naming.remote.protocol.IoFutureHelper.get(IoFutureHelper.java:95)
at org.jboss.naming.remote.client.HaRemoteNamingStore.failOverSequence(HaRemoteNamingStore.java:198)
... at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:146)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at com.goprecise.ams.demo.SendJmsToProcess.main(SendJmsToProcess.java:46)
Caused by: java.io.IOException: For now upgrade responses must have a content length of zero.
... at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:571)
... at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:335)
at org.jboss.naming.remote.client.EndpointCache$EndpointWrapper.connect(EndpointCache.java:122)
at org.jboss.naming.remote.client.HaRemoteNamingStore.failOverSequence(HaRemoteNamingStore.java:197)
... 8 common frames omitted
javax.naming.CommunicationException: Failed to connect to any server. Servers tried: [http-remoting://127.0.0.1:8080 (java.io.IOException: For now upgrade responses must have a content length of zero.)]
at org.jboss.naming.remote.client.HaRemoteNamingStore.failOverSequence(HaRemoteNamingStore.java:244)
at org.jboss.naming.remote.client.HaRemoteNamingStore.namingStore(HaRemoteNamingStore.java:149)
This is the Java client code in a try catch block attempting to connect:
Properties env = new Properties();
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
env.put(Context.PROVIDER_URL, "http-remoting://127.0.0.1:8080");
env.put(Context.SECURITY_PRINCIPAL,adminUser);
env.put(Context.SECURITY_CREDENTIALS, adminPassword);
Context namingContext = new InitialContext(env);
String CONNECTION_FACTORY = "java:jboss/exported/jms/RemoteConnectionFactory";
ConnectionFactory connectionFactory = (ConnectionFactory) namingContext.lookup(CONNECTION_FACTORY);
System.out.println("Got ConnectionFactory");
Destination destination = (Destination) namingContext.lookup(QUEUE); // Sure QUEUE is correct
System.out.println("Got JMS Endpoint " + QUEUE);
JMSContext context = connectionFactory.createContext(adminUser, adminPassword);
context.createProducer().send(destination, xmlContent);
System.out.println("Got JMS destination");
And these are my JNDI tree values in the EAP management console for java:jboss/exported >> JMS >>
URI java:jboss/exported/jms/RemoteConnectionFactory
Class Name org.apache.activemq.artemis.jms.client.ActiveMQJMSConnectionFactory
Value ActiveMQConnectionFactory [serverLocator=ServerLocatorImpl
[initialConnectors=[TransportConfiguration(name=http-connector,
factory=org-apache-activemq-artemis-core-remoting-impl-netty-
NettyConnectorFactory) ?httpUpgradeEndpoint=http-
acceptor&activemqServerName=default&httpUpgradeEnabled=true&port=
8080&host=kubernetes-docker-internal], discoveryGroupConfiguration=null],
clientID=null, consumerWindowSize = 1048576, dupsOKBatchSize=1048576,
transactionBatchSize=1048576, readOnly=falseEnableSharedClientID=true]
It looks to me like you're using the wrong InitialContextFactory implementation. Try using org.wildfly.naming.client.WildFlyInitialContextFactory instead of org.jboss.naming.remote.client.InitialContextFactory.
You can find a full JMS client example for JBoss EAP 7.3 here.

Error invoking scheduled task Error instantiating bean of type [io.micronaut.configuration.lettuce.health.RedisHealthIndicator]

I have the following problem when running this schedule.
#Singleton
public class TaskScheduler {
private static final Logger LOG = LoggerFactory.getLogger(TaskScheduler.class);
#Inject
private BuildLayerJob buildLayerJob;
#Scheduled(fixedDelay = "30s", initialDelay = "30s")
public void loadRegistriesDescriptions(){
try {
LOG.info("Cargando lista de registries cada 30s.");
buildLayerJob.getBuildLayer().loadRegistries();
}
catch(Exception exception) {
LOG.error("Error cargando lista de registries cada 30s: " +exception.getMessage());
//exception.printStackTrace();
}
}
}
In the first execution there is no problem, but when the time expires and it is executed again it throws me the following error.
20:26:59.291 [pool-1-thread-6] ERROR i.m.s.DefaultTaskExceptionHandler - Error invoking scheduled task Error instantiating bean of type [io.micronaut.configuration.lettuce.health.RedisHealthIndicator]
Message: Unable to connect to localhost:6379
Path Taken: new HealthMonitorTask(CurrentHealthStatus currentHealthStatus,[List healthIndicators]) --> new RedisHealthIndicator(BeanContext beanContext,HealthAggregator healthAggregator,[StatefulRedisConnection[] connections])
io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type [io.micronaut.configuration.lettuce.health.RedisHealthIndicator]
Message: Unable to connect to localhost:6379
Path Taken: new HealthMonitorTask(CurrentHealthStatus currentHealthStatus,[List healthIndicators]) --> new RedisHealthIndicator(BeanContext beanContext,HealthAggregator healthAggregator,[StatefulRedisConnection[] connections])
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1719)
at io.micronaut.context.DefaultBeanContext.addCandidateToList(DefaultBeanContext.java:2727)
at io.micronaut.context.DefaultBeanContext.getBeansOfTypeInternal(DefaultBeanContext.java:2639)
at io.micronaut.context.DefaultBeanContext.getBeansOfType(DefaultBeanContext.java:924)
at io.micronaut.context.AbstractBeanDefinition.lambda$getBeansOfTypeForConstructorArgument$9(AbstractBeanDefinition.java:1124)
at io.micronaut.context.AbstractBeanDefinition.resolveBeanWithGenericsFromConstructorArgument(AbstractBeanDefinition.java:1762)
at io.micronaut.context.AbstractBeanDefinition.getBeansOfTypeForConstructorArgument(AbstractBeanDefinition.java:1119)
at io.micronaut.context.AbstractBeanDefinition.getBeanForConstructorArgument(AbstractBeanDefinition.java:981)
at io.micronaut.configuration.lettuce.health.$RedisHealthIndicatorDefinition.build(Unknown Source)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1693)
at io.micronaut.context.DefaultBeanContext.addCandidateToList(DefaultBeanContext.java:2727)
at io.micronaut.context.DefaultBeanContext.getBeansOfTypeInternal(DefaultBeanContext.java:2639)
at io.micronaut.context.DefaultBeanContext.getBeansOfType(DefaultBeanContext.java:924)
at io.micronaut.context.AbstractBeanDefinition.lambda$getBeansOfTypeForConstructorArgument$9(AbstractBeanDefinition.java:1124)
at io.micronaut.context.AbstractBeanDefinition.resolveBeanWithGenericsFromConstructorArgument(AbstractBeanDefinition.java:1762)
at io.micronaut.context.AbstractBeanDefinition.getBeansOfTypeForConstructorArgument(AbstractBeanDefinition.java:1119)
at io.micronaut.context.AbstractBeanDefinition.getBeanForConstructorArgument(AbstractBeanDefinition.java:984)
at io.micronaut.management.health.monitor.$HealthMonitorTaskDefinition.build(Unknown Source)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1693)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2407)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2393)
at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2084)
at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2058)
at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:618)
at io.micronaut.scheduling.processor.ScheduledMethodProcessor.lambda$process$5(ScheduledMethodProcessor.java:123)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:305)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to localhost:6379
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78)
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56)
at io.lettuce.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:234)
at io.lettuce.core.RedisClient.connect(RedisClient.java:207)
at io.lettuce.core.RedisClient.connect(RedisClient.java:192)
at io.micronaut.configuration.lettuce.AbstractRedisClientFactory.redisConnection(AbstractRedisClientFactory.java:51)
at io.micronaut.configuration.lettuce.DefaultRedisClientFactory.redisConnection(DefaultRedisClientFactory.java:52)
at io.micronaut.configuration.lettuce.$DefaultRedisClientFactory$RedisConnection1Definition.build(Unknown Source)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1693)
... 31 common frames omitted
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:6379
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
I understand that there are problems with the connection to redis, but in the microservice deployed in GCP it continues to generate the same problem.
app.yaml
runtime: java11
service: default
instance_class: B2
env_variables:
LAYERS_SERVER_PORT: 8080
REDIS_FIXEDDELAY: 1s
REDISA_URL: "redis://A"
REDISB_URL: "redis://B"
REDISC_URL: "redis://C"
REDISD_URL: "redis://D"
basic_scaling:
max_instances: 1
idle_timeout: 270s
vpc_access_connector:
name: "projects/example/locations/us-central1/connectors/example"
Local settings. application.yml:
micronaut:
application:
name: example
server:
port: ${EXAMPLE_SERVER_PORT:3000}
cors:
enabled: true
---
redis:
servers:
REDISA:
uri: redis://IP_A
REDISB:
uri: redis://IP_B
REDISC:
uri: redis://IP_C
REDISD:
uri: redis://IP_D
Repository layers.server.repo.InfoRepositoryImpl:
#Singleton
public class InfoRepositoryImpl implements InfoRepository {
private BuildLayerJob buildLayerJob;
#Inject #Named("REDISB") RedisAsyncCommands<String, String> reddisConnectionB;
#Inject #Named("REDISA") RedisAsyncCommands<String, String> reddisConnectionA;
private static final Logger LOG = LoggerFactory.getLogger(InfoRepositoryImpl.class);
public InfoRepositoryImpl(BuildLayerJob buildLayerJob) {
this.buildLayerJob = buildLayerJob;
}
... implementation of methods to process information with redis
}
Can you please check if you are having io.micronaut.redis:micronaut-redis-lettuce dependency added to your class path/ build file.
By default Micronaut will assume redis server to be at localhost:6379, as health checks are by default enabled when redis-lettuce is being activated. It will keep probing for health checks.
If you are using micronaut application.yml, you need to provide the server url which will be accessible from the running app.
Micronaut redis
Example - application.yml
redis:
uri: redis://localhost
ssl: true
timeout: 30s
You can also use below connection string pattern to provide details about redis server.
Redis Standalone
redis :// [[username :] password#] host [: port] [/ database][?
[timeout=timeout[d|h|m|s|ms|us|ns]] [&_database=database_]]
Redis Standalone (SSL)
rediss :// [[username :] password#] host [: port] [/ database][?
[timeout=timeout[d|h|m|s|ms|us|ns]] [&_database=database_]]
Redis Standalone (Unix Domain Sockets)
redis-socket :// [[username :] password#]path
[?[timeout=timeout[d|h|m|s|ms|us|ns]][&_database=database_]]
for more details on connection string - Redis connections string
Micronaut redis configuration properties
Such errors can occur when the said data source is autoconfigured. You can disable Redis autoconfiguration if you're not using it in the application. If you need Redis for the application then you should set spring.redis.host and spring.redis.port.

HikariCP on AppEngine

When deploying a Spring Boot application using HikariCP as the connection pool on Appengine, I get some errors related to database (threads), when performing some requests:
Caused by: java.sql.SQLNonTransientConnectionException: Could not create connection to database server.
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:1008)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:825)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:455)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:207)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:136)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:706)
at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:692)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call memcache.Get in a thread that is neither the original request thread nor a thread created by ThreadManager
Then I discovered that AppEngine only allow applications to create threads using its ThreadFactory. So I made sure to configure my Hikari to use AppEngine's Thread factory like the following :
DataSource ds = new HikariDataSource();
try {
final HikariConfig dataSourceConfig = new HikariConfig();
dataSourceConfig.setDriverClassName(applicationProperties.getDatasource()
.getDriverClassName());
dataSourceConfig.setJdbcUrl(applicationProperties.getDatasource()
.getUrl());
dataSourceConfig.setUsername(applicationProperties.getDatasource()
.getUsername());
dataSourceConfig.setPassword(applicationProperties.getDatasource()
.getPassword());
dataSourceConfig.setRegisterMbeans(false);
if (Objects.equal(ProfileResolver.getActiveCloudPlatform(env), ProfileConstants.SPRING_PROFILE_GCP)) {
log.info("[GCP] Set 'com.google.appengine.api.ThreadManager.backgroundThreadFactory()' "
+ "as the instance of the java.util.concurrent.ThreadFactory");
dataSourceConfig.setThreadFactory(ThreadManager.backgroundThreadFactory());
}
ds = new HikariDataSource(dataSourceConfig);
} catch (final Exception e) {
throw new IllegalStateException(e);
}
return ds;
It works on my local appengine (DevServer), But when deployed I get an exception at datasource initialisation, since Appengine's autoscaling modules doesn't allow the use of Background threads.
Is it possible to keep the "autoscaling" ability while using HikariCP on AppEngine ?
The Java 8 runtime doesn't have the same restrictions around threads like prior versions of App Engine. For example, this sample app uses HikariCP to connect to Cloud SQL, and works without a custom thread manager.

Connect to multiple Kafka servers using springboot

In Spring boot application, I want to connect to 2 different kafka servers simultaneously. I am using KafkaAdmin and AdminClient to make the connection and perform CRUD Operations.
#Bean
public KafkaAdmin kafkaAdmin() {
Map<String, Object> configs = new HashMap<>();
String krb5location = krb5Location;
System.setProperty("java.security.krb5.conf", krb5location);
System.setProperty("java.security.auth.login.config", jaasConfigLocation);
configs.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, server);
configs.put("security.protocol", "SASL_SSL");
configs.put("ssl.truststore.location", sslTruststoreLocation);
configs.put("ssl.truststore.password", sslTruststorePassowrd);
return new KafkaAdmin(configs);
}
#Bean
#PostConstruct
public AdminClient config() {
return AdminClient.create(kafkaAdmin.getConfig());
}
Similarly server 2 is configured in same springboot application.
If I load configuration of both kafka server at once during app initialization following error is displayed
>>>KRBError:
cTime is Sun Jun 03 14:23:02 IST 2001 991558382000
sTime is Tue Nov 20 10:46:53 IST 2018 1542691013000
suSec is 512097
error code is 7
error Message is Server not found in Kerberos database
cname is config1#servername.com
sname is config2#servernname.com
msgType is 30
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:251)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:262)
at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:308)
at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:126)
at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458)
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:361)
at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$2.run(SaslClientAuthenticator.java:359)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslToken(SaslClientAuthenticator.java:359)
at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendSaslClientToken(SaslClientAuthenticator.java:269)
at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:206)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:81)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:474)
at org.apache.kafka.common.network.Selector.poll(Selector.java:412)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:460)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1006)
at java.lang.Thread.run(Thread.java:748)
Caused by: KrbException: Identifier doesn't match expected value (906)
at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140)
at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65)
at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60)
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55)
... 22 more
2018-11-20 10:46:53.605 ERROR 8672 --- [| adminclient-4] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-4] Connection to node -1 failed authentication due to: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - UNKNOWN_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. This may be caused by Java's being unable to resolve the Kafka Broker's hostname correctly. You may want to try to adding '-Dsun.net.spi.nameservice.provider.1=dns,sun' to your client's JVMFLAGS environment. Users must configure FQDN of kafka brokers when authenticating using SASL and `socketChannel.socket().getInetAddress().getHostName()` must match the hostname in `principal/hostname#realm` Kafka Client will go to AUTHENTICATION_FAILED state.

Socket timeout with mongo DB after 30 mins of inactivity

After 30 mins of inactivity or so I start getting the below error when I try to insert into mongo, When I try again it starts to work. Error Below. I'm on Azure:
[INFO ] 2018-09-10T12:00:43,188 [http-nio-8080-exec-6] connection - Closed connection [connectionId{localValue:3, serverValue:26}] to XX.XX.XX.XX:27017 because there was a socket exception raised by this connection.
[ERROR] 2018-09-10T12:00:43,189 [http-nio-8080-exec-6] [dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.data.mongodb.UncategorizedMongoDbException: Timeout while receiving message; nested exception is com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message] with root cause
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_181]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_181]
ava.net.SocketTimeoutException: Read timed out
t java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_181]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_181]
Here is how I initialize my mongo Template:
#Bean
public MongoDbFactory mongoDbFactory() {
String[] addresses = mongoUri.split(",");
List<ServerAddress> servers = new ArrayList<>();
for (String address : addresses) {
String[] split = address.trim().split(":");
servers.add(new ServerAddress(split[0].trim(), Integer.parseInt(split[1].trim())));
}
MongoClientOptions.Builder mongoOperations = MongoClientOptions.builder();
mongoOperations.socketTimeout(1000 * 20); // I tried to increase the socket timeout to see if it helps but no help either
mongoOperations.connectTimeout(1000 * 10);
MongoClient mongoClient = new MongoClient(servers, MongoCredential.createCredential(userName, dbName, password.toCharArray()), mongoOperations.build());
return new SimpleMongoDbFactory(mongoClient, dbName);
}
#Bean
public MongoTemplate getMongoTemplate() {
return new MongoTemplate(mongoDbFactory());
}
My mongod version is 3.6.4 and I'm using the same version of java driver.
I tried to increase/decrease the tcp_keepalive_time setting as provided in the docs using
sudo sysctl -w net.ipv4.tcp_keepalive_time=120 but no help either.
ohk. so we found that the mongo java driver jar was older than the mongo server we were using which was causing this issue. make sure that the driver supports the version of mongo server.

Categories