I have a small spring boot application that I am trying to integrate with Redis using Jedis client.
I set up the Jedis pool to point towards a Redis docker container running locally.
#Bean
public JedisPool jedisPool() {
JedisPool jedisPool = null;
try {
URI redisUri = new URI("redis://localhost:6379");
jedisPool = new JedisPool(poolConfig(), redisUri, 20000, null, null, null);
} catch (URISyntaxException e) {
logger.error("Failed to create the redis pool: ", e);
}
return jedisPool;
}
I have a RedisService that makes use of the JedisPool and perform transactions against the Redis Server.
#Autowired
JedisPool jedisPool;
#Override
public String retrieve(String key) {
Jedis jedis = jedisPool.getResource();
String json = jedis.get(key);
jedis.close();
return json;
}
#Override
public void store(String key, String value) {
Jedis jedis = jedisPool.getResource();
jedis.set(key, value);
jedis.expire(key, keyExpire);
jedis.close();
}
When running the app I get this exception
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.ConnectException: Connection refused (Connection refused)
I am able to connect to the Redis Server via redis-cli but not from my spring boot app.
Any ideas on how to solve this issue?
Try to use http://localhost:6379 instead your current URI
Related
Issue:
1. I am not using Spring Boot MongoAutoConfiguration because we need mongo to be optional.
2. Other applications in same namespace can access MongoDB and network namespace is same for Application namespace and database namespace.
3. When I try to connect I get timeout exception.
4. Same worked on my local
POINTS ALREADY VERIFIED:
1. Checked mongod is up and running. other app in same namepsace is able to access same but its using Spring Mongo implementation.
2. no network issue.
3. There are post in stackoverflow for same exception, Its already tested. Not working.
a) application.properties:
mongo.hosts = mongo-node-1.database, mongo-node-2.database, mongo-node-3.database
mongo.port = 27017
mongo.database = database
isMongoEnabled = true
b) MongoClient bean :
#Configuration
public class MongoConfiguration {
#Value("#{'${mongo.hosts}'.split(',')}")
private List<String> hosts;
#Value("${mongo.port}")
private int port;
#Value("${isMongoEnabled}")
private boolean isMongoEnabled;
#Value("${mongo.database}")
private String database;
private Mongo createMongo() throws Exception {
final List<ServerAddress> serverList = new ArrayList<>();
for (final String host : hosts) {
serverList.add(new ServerAddress(host, port));
}
return new MongoClient(serverList);
}
#Bean
public Mongo mongoClient() throws Exception {
final Mongo mongo = createMongo();
return mongo;
}
}
c) Template bean:
#Configuration
#EnableMongoRepositories(
basePackages = "com.abc.test",
mongoTemplateRef = "customMongoNodeTemplate"
)
#Import(MongoConfiguration.class)
public class TemplateConfiguration {
#Value("${mongo.database}")
private String database;
#Bean
public MongoTemplate customMongoNodeTemplate(#Qualifier ("mongoClient") Mongo mongo) {
final MongoDbFactory factory = new SimpleMongoDbFactory( (MongoClient) mongo, database);
return new MongoTemplate(factory);
}
}
d) Exception:
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=127.0.0.1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]
UPDATE:
After trying SimpleMongoClientDBFactory(uri) got below exception :
Application.properties:
mongo.uri= mongodb://mongo-node-1.database:27017,mongo-node-2.database:27017,mongo-node-3.database:27017/database
Exception :
#com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=127.0.0.1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]
SPRING boot version :
2.1.6
Mongo version :
4.0.6
Mongo Driver Version:
3.8.2
Spring-boot:2.3.0.RELEASE
Disable #SpringBootApplication(exclude = MongoAutoConfiguration.class)
To initiate MongoTemplate we can do with:
MongoDatabaseFactory
com.mongodb.client.MongoClient [interface] (Do not confuse with com.mongodb.MongoClient [class])
Implementation with MongoDatabaseFactory
application.properties:
mongo.uri=mongodb://mongo-node-1.database:27017,mongo-node-2.database:27017,mongo-node-3.database:27017/database
isMongoEnabled=true
Template bean
#Configuration
#EnableMongoRepositories(basePackages = "com.abc.test", mongoTemplateRef = "customMongoNodeTemplate")
public class TemplateConfiguration {
#Value("${mongo.uri}")
private String uri;
#Bean
public MongoTemplate customMongoNodeTemplate() {
return new MongoTemplate(new SimpleMongoClientDatabaseFactory(uri));
}
}
So this was a issue from my end , Turns out some one committed a mongoClient recently in git and when i took pull , that was a new MongoClient() and on local went to localhost but on cluster failed. Code was fine from the start. Thank you very much for baring with me
I'm trying to connect to Redis using Jedis in my java application. I'm instantiating a JedisPool object, and when I get the resource, it throws an exception saying it cannot return the resource. What is weird though is if I just instantiate a Jedis object, it connects without problems, and I can change data.
Here's my RedisDatabase class:
package me.joeleoli.proxylink.database;
import me.joeleoli.proxylink.ProxyLink;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
public class RedisDatabase {
private JedisPool pool;
public RedisDatabase(String host, int port, String password) {
ProxyLink.getInstance().getLogger().info("Attempting to establish Redis connection " + host + ":" + port);
this.pool = new JedisPool(host, port);
try (Jedis jedis = this.pool.getResource()) {
if (password != null && !password.equals("")) {
jedis.auth(password);
}
jedis.select(0);
jedis.close();
}
}
public JedisPool getPool() {
return this.pool;
}
}
Here's my error:
22:16:15 [INFO] [ProxyLink] Attempting to establish Redis connection 127.0.0.1:6379
22:16:15 [WARNING] Exception encountered when loading plugin: ProxyLink
redis.clients.jedis.exceptions.JedisException: Could not return the resource to the pool
at redis.clients.jedis.JedisPool.returnResource(JedisPool.java:106)
at redis.clients.jedis.JedisPool.returnResource(JedisPool.java:12)
at redis.clients.jedis.Jedis.close(Jedis.java:3206)
at me.joeleoli.proxylink.database.RedisDatabase.<init>(RedisDatabase.java:23)
at me.joeleoli.proxylink.ProxyLink.onEnable(ProxyLink.java:71)
at net.md_5.bungee.api.plugin.PluginManager.enablePlugins(PluginManager.java:227)
at net.md_5.bungee.BungeeCord.start(BungeeCord.java:273)
at net.md_5.bungee.BungeeCordLauncher.main(BungeeCordLauncher.java:111)
at net.md_5.bungee.Bootstrap.main(Bootstrap.java:15)
Caused by: redis.clients.jedis.exceptions.JedisException: Could not return the resource to the pool
at redis.clients.util.Pool.returnResourceObject(Pool.java:61)
at redis.clients.jedis.JedisPool.returnResource(JedisPool.java:103)
... 8 more
Caused by: java.lang.IllegalStateException: Object has already been returned to this pool or is invalid
at org.apache.commons.pool2.impl.GenericObjectPool.returnObject(GenericObjectPool.java:551)
at redis.clients.util.Pool.returnResourceObject(Pool.java:59)
... 9 more
The problem with your code is the call to jedis.close() within the try-with-resource block. The try-with-resource block with closes your resource when the block exits. Since you already closed the resource, prior to the block exiting, you end up calling close twice.
Remove the call to jedis.close within the block and just use the try-with-resource functionality.
In Spring Boot, does jdbcTemplate not close the connection automatically once after the it executes the query?
In this case, I am executing a query using jdbcTemplate(where it connects to teradata) but the session is not closing after the query is executed. How can I close the session?
This is my dao file -
#Component
public class DDLReviewDao {
#Autowired
#Qualifier("devbJdbc")
private JdbcTemplate jdbcTemplate;
public static final Logger logger = LogManager.getLogger(DDLReviewDao.class);
public List<DDLObject> getDDLReviewData(DDLQuery ddlQuery) {
String selectSql = MacroGenerator.generateMacro(ddlQuery);
List<DDLObject> ddlObject = jdbcTemplate.query(selectSql, new DDLMapper());
logger.info(ddlObject);
return ddlObject;
}
}
JdbcTemplate gets its connections from javax.sql.DataSource implementation - which is passed to its constructor link.
The DataSources can be basic (creates Connection object for each request) or pooling (has pool of connections and just 'borrows' one for given request's use).
So, it appears that the connection is not closing because you have passed some pooling datasource to JdbcTemplate named devbJdbc. If you realy want to close every connection opened to do the JdbcTemplate job, you can use a basic DataSource implementation: org.springframework.jdbc.datasource.SimpleDriverDataSource just like that:
#Configuration
class DevbConfig {
#Bean(name = "devbDataSource")
DataSource devbDataSource() {
try {
return new SimpleDriverDataSource(DriverManager.getDriver("jdbc:..."), "jdbc:...", "username", "password");
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
#Bean(name = "devbJdbc")
JdbcTemplate devbJdbc(#Qualifier("devbDataSource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
}
In Spring Boot, does jdbcTemplate not close the connection
automatically once after the it executes the query?
Should it close the connection or return it to the connection pool (in case the DataSource is pooled)?
If you read the source code at http://grepcode.com/file/repo1.maven.org/maven2/org.springframework/spring-jdbc/4.1.7.RELEASE/org/springframework/jdbc/core/JdbcTemplate.java#JdbcTemplate.execute%28org.springframework.jdbc.core.StatementCallback%29it boils down to:
public static void doReleaseConnection(Connection con, DataSource dataSource) throws SQLException {
if (con == null) {
return;
}
if (dataSource != null) {
ConnectionHolder conHolder = (ConnectionHolder) TransactionSynchronizationManager.getResource(dataSource);
if (conHolder != null && connectionEquals(conHolder, con)) {
// It's the transactional Connection: Don't close it.
conHolder.released();
return;
}
}
logger.debug("Returning JDBC Connection to DataSource");
doCloseConnection(con, dataSource);
}
and
public static void doCloseConnection(Connection con, DataSource dataSource) throws SQLException {
if (!(dataSource instanceof SmartDataSource) || ((SmartDataSource) dataSource).shouldClose(con)) {
con.close();
}
}
Most-likely, if the DataSource instance is pooled, connections are release back for reuse and not closed.
According to the spring boot docs you can allocate a maximum number of connections to a spring pool like so:
spring.datasource.tomcat.max-active=50
This will obviously only work with the embedded webserver. If you are deploying it to something like a Jboss you'll have to configure that property in the Server config file.
I was trying to run the following example in my machine:
https://examples.javacodegeeks.com/enterprise-java/spring/spring-data-redis-example/
public static void main(String[] args) {
ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext(new ClassPathResource("/spring/spring-config.xml").getPath());
RedisRepo redisRepo = (RedisRepo)context.getBean(“redisRepo");
try{
JedisPool pool = new JedisPool(new JedisPoolConfig(), "x.x.x.x”);
Jedis jedis = pool.getResource();
System.out.println("Connected to Redis”);//connected to Redis
System.out.println("server is running: "+jedis.ping());//PONG
System.out.println("current keys are :"+jedis.keys(“*”));//[ ]
Employee s = new Employee();
s.setId(1);
s.setName(“abc”);
redisRepo.saveState(s);
System.out.println("server is running: "+jedis.ping());//PONG
System.out.println("Finding the One : "+redisRepo.getState(1);//Finding the one :null
}
catch(Exception e){
logger.error(e.getMessage(), e);
}
}
But getting the following error while trying to save the model object:
Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
I would like to add that I can test the connection to redis server by pinging.How can I resolve this issue to persist data in Redis?
I could resolve the issue to save data on local instance by running redis locally.While to save data on remote server I had import resource in the main class.Like:
#ImportResource({classpath:path-to-youXML})
I have a support website that I would like to show some stats gathered from another Java app via JMX. We have noticed the support app sometimes cannot get the stats after the other app has been restarted. I guess this is because the support app has opening a JMX connection to the other app and keeps hold of it. Then every time you go to the page to display the JMX stats it tries to gather them using the connection and it fails.
My question is, is it better to have a single JMX connection and try and work out when we should reconnect it?
Or each time we load the page with JMX stats on it should we create a new JMX connection then close it once we have the values we need?
As per my knowledge,
JMX connections are RMI Connector objects and hence can be held in the client app. + use a heartbeat approach to reconnect.
This way we can avoid overhead of re-establishing RMI connections which are not light weight.
Refer: javax.management.remote.rmi.RMIConnector
We didn't end up using a heartbeat but after reading Girish's answer came up with the following
public class JmxMetricsRetriever {
private final JMXServiceURL jmxUrl;
private final Map<String, Object> env;
private MBeanServerConnection connection;
private JmxMetricsRetriever(JMXServiceURL jmxUrl, Map<String, Object> env) {
this.jmxUrl = jmxUrl;
this.env = env;
reconnect();
}
public synchronized Object getAttributeValue(String jmxObjectName, String attributeName) {
try {
if (connection == null) {
reconnect();
}
try {
return getAttributeValuePrivate(jmxObjectName, attributeName);
} catch (ConnectException exc) {
//This is to reconnect after the Server has been restarted.
reconnect();
return getAttributeValuePrivate(jmxObjectName, attributeName);
}
} catch (MalformedObjectNameException |
AttributeNotFoundException |
MBeanException |
ReflectionException |
InstanceNotFoundException |
IOException ex) {
throw new RuntimeException(ex);
}
}
private synchronized Object getAttributeValuePrivate(String jmxObjectName, String attributeName) throws MalformedObjectNameException, MBeanException, AttributeNotFoundException, InstanceNotFoundException, ReflectionException, IOException {
ObjectName replication = new ObjectName(jmxObjectName);
return connection.getAttribute(replication, attributeName);
}
private synchronized void reconnect() {
logger.info(String.format("Reconnecting to [%s] via JMX", jmxUrl.toString()));
try {
JMXConnector jmxConnector = JMXConnectorFactory.connect(jmxUrl, env);
this.connection = jmxConnector.getMBeanServerConnection();
jmxConnector.connect();
} catch (IOException e) {
//Log something but don't throw an exception otherwise our app will fail to start.
}
}
public static JmxMetricsRetriever build(String url, String port, String user, String password) {
try {
JMXServiceURL jmxUrl = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + url + ":" + port + "/jmxrmi");
Map<String, Object> env = new HashMap<>();
env.put(JMXConnector.CREDENTIALS, new String[]{user, password});
return new JmxMetricsRetriever(jmxUrl, env);
} catch (MalformedURLException ex) {
throw new RuntimeException(ex);
}
}
}
When we start our app we try and create a JMX connect an hold on to it. Every time we get a JMX attribute we check the connection has been created (might not of been if the server we are connecting to was not up when we started our service). Then try and retrieve our attribute. If it failed try and reconnect and get the attribute value. We could not find a better way to test of a JMX connect was still usable so had to catch the exception.