After 30 mins of inactivity or so I start getting the below error when I try to insert into mongo, When I try again it starts to work. Error Below. I'm on Azure:
[INFO ] 2018-09-10T12:00:43,188 [http-nio-8080-exec-6] connection - Closed connection [connectionId{localValue:3, serverValue:26}] to XX.XX.XX.XX:27017 because there was a socket exception raised by this connection.
[ERROR] 2018-09-10T12:00:43,189 [http-nio-8080-exec-6] [dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.data.mongodb.UncategorizedMongoDbException: Timeout while receiving message; nested exception is com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message] with root cause
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_181]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_181]
ava.net.SocketTimeoutException: Read timed out
t java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_181]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_181]
Here is how I initialize my mongo Template:
#Bean
public MongoDbFactory mongoDbFactory() {
String[] addresses = mongoUri.split(",");
List<ServerAddress> servers = new ArrayList<>();
for (String address : addresses) {
String[] split = address.trim().split(":");
servers.add(new ServerAddress(split[0].trim(), Integer.parseInt(split[1].trim())));
}
MongoClientOptions.Builder mongoOperations = MongoClientOptions.builder();
mongoOperations.socketTimeout(1000 * 20); // I tried to increase the socket timeout to see if it helps but no help either
mongoOperations.connectTimeout(1000 * 10);
MongoClient mongoClient = new MongoClient(servers, MongoCredential.createCredential(userName, dbName, password.toCharArray()), mongoOperations.build());
return new SimpleMongoDbFactory(mongoClient, dbName);
}
#Bean
public MongoTemplate getMongoTemplate() {
return new MongoTemplate(mongoDbFactory());
}
My mongod version is 3.6.4 and I'm using the same version of java driver.
I tried to increase/decrease the tcp_keepalive_time setting as provided in the docs using
sudo sysctl -w net.ipv4.tcp_keepalive_time=120 but no help either.
ohk. so we found that the mongo java driver jar was older than the mongo server we were using which was causing this issue. make sure that the driver supports the version of mongo server.
Related
I'm trying to connect to AWS ElastiCache Redis using Spring Data Redis + Jedis combination. [Redis Cluster enabled, so it has Cluster Config endpoint, with 3 shard - each shard has 1 primary node + 2 replica nodes ]
I'm getting Read timed out error.
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
AWS Redis Server Version : 5.0.3 /
Cluster Mode : Enabled /
SSL : Enabled /
Auth : Enabled ( by password )
Library --
Spring-data-redis : 2.1.6.Release /
jedis : 2.9.0
Telnet works to AWS Redis all nodes and cluster config endpoint at 6379 ports.
I tried Redisson by itself, it connects to AWS Redis, with out any issue.
So, no issues with Redis itself, issue with Spring Data Redis in combination with Jedis.
My Code looks like this -
RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration();
redisClusterConfiguration.setClusterNodes(listOfRedisNode);
redisClusterConfiguration.setPassword(passwordString);
JedisClientConfiguration.JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();
jedisClientConfiguration.connectTimeout(Duration.ofSeconds(60));
jedisClientConfiguration.useSsl();
jedisClientConfiguration.usePooling();
JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(redisClusterConfiguration, jedisClientConfiguration.build() );
jedisConnectionFactory.afterPropertiesSet();
final RedisTemplate<String, Serializable> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(jedisConnectionFactory);
redisTemplate.setKeySerializer(new JdkSerializationRedisSerializer());
redisTemplate.setValueSerializer(new JdkSerializationRedisSerializer());
redisTemplate.afterPropertiesSet();
System.out.println(redisTemplate.getClientList().size());
StringRedisConnection stringRedisConnectionlettuce = new DefaultStringRedisConnection(redisTemplate.getConnectionFactory().getConnection());
final String message2 = stringRedisConnectionlettuce.echo("Hello");
System.out.println("Hello".equals(message2));
Read time out error -
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
at redis.clients.jedis.Protocol.process(Protocol.java:151)
at redis.clients.jedis.Protocol.read(Protocol.java:215)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:239)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:96)
at redis.clients.jedis.Connection.sendCommand(Connection.java:126)
at redis.clients.jedis.Connection.sendCommand(Connection.java:117)
at redis.clients.jedis.BinaryClient.auth(BinaryClient.java:564)
at redis.clients.jedis.BinaryJedis.auth(BinaryJedis.java:2138)
at redis.clients.jedis.JedisClusterConnectionHandler.initializeSlotsCache(JedisClusterConnectionHandler.java:36)
at redis.clients.jedis.JedisClusterConnectionHandler.<init>(JedisClusterConnectionHandler.java:17)
at redis.clients.jedis.JedisSlotBasedConnectionHandler.<init>(JedisSlotBasedConnectionHandler.java:24)
at redis.clients.jedis.BinaryJedisCluster.<init>(BinaryJedisCluster.java:54)
at redis.clients.jedis.JedisCluster.<init>(JedisCluster.java:93)
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(JedisConnectionFactory.java:418)
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(JedisConnectionFactory.java:388)
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.afterPropertiesSet(JedisConnectionFactory.java:345)
at io.github.deepshiv126.springdataredis.example.MySpringBootApplication.main(MySpringBootApplication.java:306)
... 8 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:127)
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:196)
... 27 more
I looked into Spring Source Code and Jedis Source Code -- My assumption its not using SSL Connection ;
JedisConnectionFactory - afterPropertiesSet()-- trying to create Cluster -- under that it's trying initializeSlotsCache, which issued AUTH command to Redis Server, with password -- This is where "Read timed out" is occuring;
I understand local redis - you can go inside and run auth command to get authenticate. But I guess AWS Redis may not able to do that , its needs to have SSL Connection even before it runs AUTH command - Why Jedis is not using SSL Connection ?
This another thread "Cannot get Jedis connection" when using SSL with Redis and Spring Data Redis
says, use something like JedisPool - but spring-data-redis' JedisConnectionFactory doesn't accepts JedisPool. Is there any other way to do that ?
JedisPool jedisPool = new JedisPool("rediss://" + clusterConfigEndPoint + ":6379");
Another question - other libraries use redis ssl connection as rediss:// - how to Jedis Client to use SSL connection,
Any help will be really appreciated!!
Thanks!
I was facing this for quite a while, however with below configuration it works perfectly with AWS ElasticCache - Redis with SSL Enabled (Encryption-in-transit & Encryption-at-rest enabled).
Below Maven dependencies are used as part of springboot application
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<!-- Latest jedis with SSL support -->
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>3.6.0</version>
</dependency>
#Component
public class RedisConfig {
#Bean
public JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();
redisStandaloneConfiguration.setHostName("PrimaryEndpoint of AWS Elastic Cache Cluster");
redisStandaloneConfiguration.setPort(6379);
redisStandaloneConfiguration.setUsername("userName");
redisStandaloneConfiguration.setPassword(new String("password").toCharArray());
JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();
jedisClientConfiguration.connectTimeout(Duration.ofSeconds(60));// 60s connection timeout
jedisClientConfiguration.useSsl();
jedisClientConfiguration.usePooling();
JedisConnectionFactory jedisConFactory = new JedisConnectionFactory(redisStandaloneConfiguration,
jedisClientConfiguration.build());
jedisConFactory.afterPropertiesSet();
return jedisConFactory;
}
#Bean(value = "redisTemplate")
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(redisConnectionFactory);
return redisTemplate;
}
}
When I try to make a .newConnection() to a rabbitMQ server I get the follwing error:
java.io.IOException
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:126)
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:122)
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:373)
at com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection(RecoveryAwareAMQConnectionFactory.java:64)
at com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init(AutorecoveringConnection.java:134)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:997)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:956)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:914)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1068)
... ...
Caused by: com.rabbitmq.client.ShutdownSignalException: connection
error at
com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)
at
com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)
at
com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:494)
at
com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:315)
at
com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory.newConnection(RecoveryAwareAMQConnectionFactory.java:64)
at
com.rabbitmq.client.impl.recovery.AutorecoveringConnection.init(AutorecoveringConnection.java:134)
My code snippet is:
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setUsername("myUser");
connectionFactory.setUsername("myPass");
connectionFactory.setHost("rabbitmq.test.com"); //also tried with the IP
connectionFactory.setPort(customPort);
connectionFactory.setVirtualHost("/myVHost");
Connection connection = connectionFactory.newConnection();
I also tried the compact setup:
connectionFactory.setUri("amqp://myUser:myPass#rabbitmq.test.com:customPort/myVHost");
Now...the strange part is that if I log into GUI, I can access the queues for that vhost...
Any ideas on this situation please? Thanks in advance.
You seem to use the setUsername() function for setting the password.
full code
public class SolrToMongodb {
private static final Logger LOGGER = LoggerFactory.getLogger(SolrToMongodb.class);
public static void main(String[] args) throws IOException, SolrServerException {
SolrToMongodb main = new SolrToMongodb();
main.run();
}
public void run() throws IOException, SolrServerException {
SparkConfig config = new SparkConfig();
JavaSparkContext jsc = new JavaSparkContext(config.sparkConf("admiralty-stream"));
SolrClient client = new HttpSolrClient(Constant.SOLR_STREAMING);
SolrQuery q = new SolrQuery();
q.set("q","*:*");
q.set("indent","on");
q.set("wt", "json");
client.query(q);
try {
CloudSolrClient cloudSolrClient = new CloudSolrClient(Constant.ZOOKEEPER_SOLR);
SolrJavaRDD solrRDD = SolrJavaRDD.get(cloudSolrClient.getZkHost(), "admiraltyStream", jsc.sc());
JavaRDD<SolrDocument> resultsRDD = solrRDD.queryShards(q);
JavaRDD<Object> objectJavaRDD = resultsRDD.map(new Function<SolrDocument, Object>() {
#Override
public Object call(SolrDocument v1) throws Exception {
System.out.println(v1.getFieldValueMap());
return v1.getFieldValueMap();
}
});
}
catch (Exception e){
System.out.println("Exception here : "+e.getMessage());
}
}}
ERROR LOG :
2017-08-02 10:02:58,709 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 0
2017-08-02 10:02:59,688 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 1
2017-08-02 10:03:01,630 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 2
2017-08-02 10:03:02,579 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 3
2017-08-02 10:03:03,540 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 4
2017-08-02 10:03:04,484 [main] ERROR CloudSolrClient - Request to collection admiraltyStream failed due to (0) java.net.ConnectException: Connection refused (Connection refused), retry? 5
Exception :
Exception here : No live SolrServers available to handle this request:[http://xxx.xxx.ph:8983/solr/admiraltyStream, http://xxx.xxx.ph:8983/solr/admiraltyStream, http://xxx.xxx.ph:8983/solr/admiraltyStream]
Using the CloudSolrClient instead of HttpSolrClient allows solrj to do a round-robin load balancing between the available solr servers, and of course is recommended in a SolrCloud context. The "No live SolrServers” message indicates a problem with the collection admiraltyStream.
Specifically, behind the scenes, SolrJ is using LBHttpSolrClient (which is using a set of HttpSolrClient instances) for round-robin requests between shards. I think that your problem is actually this: some shard is not available (i.e. the leader and the replicas).
I would review the currently online replicas (http://solr.server:8983/#/~cloud): there you should see if all the replicas for your collection online.
There must be at least one replica per shard; and I think that in your case:
the node you’re trying to connect directly using HttpSolrClient is up and running
when using CloudSolrClient (i.e. Zookeeper -> Solr), there’s something wrong in your cluster state: Solr believes that there are no replicas for at least one shard, so the list of the available HttpSolrClient instances within a given instance of LBHttpSolrClient is empty
I want to test if my MongoClient has authenticated correctly using default authentication mechanism and if it's not I want to try connecting with different one.
I have following code used for openning mongo connection which uses MONGODB-CR:
MongoCredential credential = MongoCredential.createMongoCRCredential(this.user, this.db, this.pass.toCharArray());
ServerAddress serverAddress = new ServerAddress(this.host, this.port.intValue());
MongoClient mongoClient = new MongoClient(serverAddress, Arrays.asList(new MongoCredential[]{credential}));
I need to find a way to test if authentication was complete. If it was not I want to try with different mechanism - SCRAM-SHA-1. The problem is I dont know how to test connection. Calling most of mongoClient methods leads to MongoCommandException:
com.mongodb.MongoSecurityException: Exception authenticating
at com.mongodb.connection.NativeAuthenticator.authenticate(NativeAuthenticator.java:48)
at com.mongodb.connection.InternalStreamConnectionInitializer.authenticateAll(InternalStreamConnectionInitializer.java:99)
at com.mongodb.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:44)
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115)
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:128)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.mongodb.MongoCommandException: Command failed with error 18: 'auth failed' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "auth failed", "code" : 18 }
at com.mongodb.connection.CommandHelper.createCommandFailureException(CommandHelper.java:170)
at com.mongodb.connection.CommandHelper.receiveCommandResult(CommandHelper.java:123)
at com.mongodb.connection.CommandHelper.executeCommand(CommandHelper.java:32)
at com.mongodb.connection.NativeAuthenticator.authenticate(NativeAuthenticator.java:46)
... 5 common frames omitted
But this is actually not an Exception It's just information in logs, so I'm not able to catch it.
Any ideas how to test if MongoClient has authenticated correctly?
I’m trying to deploy a simple Spring app and getting a “connection timed out ” error. My app tries to take a text input from the user in one jsp, insert that value under username in the db and then display that name in another jsp along with a greeting eg: "hello, "
My environment:
OS: Windows XP professional
Server : Tomcat 6
IDE: Eclipse
DB: MS Access 2007
I am getting the error below:
SEVERE: Context initialization failed
org.springframework.beans.factory.BeanDefinitionStoreException:
IOException parsing XML document from
ServletContext resource
[/WEB-INF/applicationContext.xml];
nested exception is
java.net.ConnectException: Connection
timed out: connect
java.net.ConnectException: Connection
timed out: connect
at java.net.PlainSocketImpl.socketConnect(Native
Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.Socket.connect(Socket.java:520)
at java.net.Socket.connect(Socket.java:470)
at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:388)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:523)
at sun.net.www.http.HttpClient.(HttpClient.java:231)
at sun.net.www.http.HttpClient.New(HttpClient.java:304)
at sun.net.www.http.HttpClient.New(HttpClient.java:321)
SEVERE: Servlet /SpringExample threw
load() exception
java.net.ConnectException: Connection
timed out: connect
at java.net.PlainSocketImpl.socketConnect(Native
Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.Socket.connect(Socket.java:520)
at java.net.Socket.connect(Socket.java:470)
at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
The code to access the db is as below:
//in the profile.java class
public void setUsername(String username) {
int rowsInserted;
setDataSource(dataSource);
jt = new JdbcTemplate(dataSource);
rowsInserted = jt.update("insert into usernamedb (username) values(?)",new Object[] { username });
System.out.println(rowsInserted);
}
in the profileFormController.java class
protected ModelAndView onSubmit(Object command)
{
Profile profile = (Profile) command;
String greeting = "Hello," + profile.getUsername() + "!";
//System.out.println(greeting);
profile.setUsername(profile.getUsername());
return new ModelAndView("greetingDisplay", "greeting", greeting);
}
To set up the DNS, in the ODBC sources I have set “usernamedb” as a DNS source by the user. I am not able to figure out the root cause for this error.
First I would check to see if your db server is running. If it is, make sure you are trying to connect to it at the right address at the right port number, and if you are giving it the right username and password.
If all of those seem to be working you may also need to check that your db server will accept connections from the your address.
In case, any one is still looking for a resolution, take a look at the DTD definition in your /WEB-INF/applicationContext.xml including the dtd version
If you can't find anything wrong, try moving to the XML Schema style of configuration. Please see http://docs.spring.io/spring/docs/current/spring-framework-reference/html/xsd-config.html#xsd-config-body