Connections keep piling up in Redis from Java client - java

I have Redis installed on my machine and I am using a Jedis client within my Spring app to write some data onto Redis. For certain reasons, with each write, the number of connections to Redis keep piling up until I get a JedisConnectionFailureException.
My Spring Jedis Config file looks like this :
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<!-- Redis conection factory configurations -->
<bean id="redisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"
p:host-name="${redis.host}"
p:port="${redis.port}"
p:password="${redis.password}"
p:use-pool="true" />
<!-- Redis String Serializer -->
<bean id="stringRedisSerializer" class="org.springframework.data.redis.serializer.StringRedisSerializer"/>
<!-- JDK Serialization Redis Serializer -->
<bean id="jdkSerializationRedisSerializer" class="org.springframework.data.redis.serializer.JdkSerializationRedisSerializer"/>
<!-- Configuring the Redis Template -->
<bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate"
p:connection-factory-ref="redisConnectionFactory"
p:keySerializer-ref="stringRedisSerializer"
p:valueSerializer-ref="jdkSerializationRedisSerializer" />
Within my entire app, this is the only place where I'm creating connections.
In my spring-app, I'm using the connections as follows :
redisTemplate.opsForValue().set("key", "value");

The issue is around wrong Spring Data Redis TX Sync algorithm - it has bugs.
It has been fixed and is available since Spring Data Redis 1.3 GA

Related

spring hbase properties : where to specify apache config timeout?

Where do I set properties like 'hbase.client.retries.number' when using hbase.version : 1.1.1
Using xml to configure spring. In my xml:
<context:property-placeholder location="classpath:webr/hbase.properties" />
<context:component-scan base-package="com.mine.xx" />
<hdp:configuration id="hadoopConfiguration">
fs.defaultFS=file:///usr/local/XX
</hdp:configuration>
<!-- configuration-ref="hadoopConfiguration" -->
<hdp:hbase-configuration configuration-ref="hadoopConfiguration"
zk-quorum="${hbase.zk.host}" zk-port="${hbase.zk.port}" />
But when I set (in webr/hbase.properties) :
hbase.client.retries.number=0
zookeeper.session.timeout=38000
zookeeper.recovery.retry=0
hbase.rpc.timeout=29000
zookeeper.recovery.retry=0
They do not get picked up (the program hangs for more than 3 minutes) after a scan is sent.
The hadoop install is on my lan.

ActiveMQ configuration with Spring Boot

I use ActiveMQ as Embedded with Spring Boot.
It seems the Broker is created trough an ActiveMQConnectionFactory.
I understand that the way to configure the broker is to set parameters in the query with broker. as described here : http://activemq.apache.org/how-do-i-embed-a-broker-inside-a-connection.html
I would like to setup some features about the DLQ, so it's in the destinationPolicy attribute, but the attribute type is not a simple type but a complex type, how can I write the query parameter to disable DLQ, please ?
Complementing #Petter and #April answers, below the same solutions but with more complete samples:
1. Petter solution, import activemq.xml at connnection factory url
build.gradle
ext {
springBootVersion = "1.5.3.RELEASE"
activeMQVersion = "5.14.5"
}
dependencies {
compile("org.springframework.boot:spring-boot-starter-activemq:${springBootVersion}")
compile("org.apache.activemq:activemq-broker:${activeMQVersion}")
testCompile("org.springframework.boot:spring-boot-starter-test:${springBootVersion}")
testCompile group: 'org.apache.activemq', name: 'activemq-spring', version: "${activeMQVersion}"
testCompile("junit:junit:4.12")
}
src/main/resources/activemq.xml
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core-5.4.0.xsd
">
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="broker1" persistent="false" >
<transportConnectors>
<transportConnector name="vm" uri="vm://broker1"/>
</transportConnectors>
</broker>
</beans>
Config.java
#EnableJms
#SpringBootApplication
#EnableAutoConfiguration
#Configuration
public class Config {}
application.properties
spring.activemq.broker-url=vm://broker1?brokerConfig=xbean:activemq.xml
2. April solution, import activemq.xml at Spring Configuration
Just remove application.properties then add #ImportResource("classpath:activemq.xml") entry to Config.java
Config.java
#EnableJms
#SpringBootApplication
#EnableAutoConfiguration
#Configuration
#ImportResource("classpath:activemq.xml")
public class Config {}
Good question. The properties on the vm-transport for auto-broker creation are great, but only up to a point which I think you have hit.
My suggestion is that you define the broker configuration as you normally would have done in XML and then just refer to this xml in the URI. Destination policies are indeed a complex structure and I don't see how it would be a good idea to define them with simple query params even if it was possible.
vm://localhost?brokerConfig=xbean:activemq.xml
I had this problem and solved it by using a spring configuration file. In my case, I wanted to configure my broker to persist.
I added the needed libs in my pom: including activemq-broker, activemq-spring, spring-jms (and in my case, activemq-leveldb-store).
My spring xml file looked like this:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="xyz">
<persistenceAdapter>
<levelDB directory="activemq-data"/>
</persistenceAdapter>
<transportConnectors>
<transportConnector uri="vm:localhost?persistent=true" />
</transportConnectors>
</broker>
</beans>
And I registered the spring file in one of my configuration classes:
#ImportResource("activemq-spring.xml")
That did the job.
I tried the xbeans solution first, but I got stuck because I was missing some xbeans classes, and I didn't know if it was a version thing or what. I'm using activemq 5.12.1

Spring: Nested beans reference to another bean is not working

I am trying to use nested beans with a profile to setup an embedded jms broker when I am in dev mode. Here is my xml:
<beans ...>
...
other bean definition
...
<beans profile="embeddedBroker">
<!-- Configure an embedded ActiveMQ Broker with a TCP connector.
This broker is never set to start automatically by default, it is the broker service wrapper that
will take care to start it if configured to do so.
-->
<amq:broker id="jmsBroker" brokerId="EmbeddedBroker" brokerName="EmbeddedBroker"
persistent="false" useJmx="true" start="false">
<amq:plugins>
<!-- lets enable detailed logging in the broker but ignore ConnectionEvents -->
<amq:loggingBrokerPlugin logMessageEvents="true" logConnectionEvents="false"/>
<amq:timeStampingBrokerPlugin zeroExpirationOverride="1000" ttlCeiling="60000" futureOnly="true"/>
<amq:traceBrokerPathPlugin />
</amq:plugins>
<!-- Create a connector to give an external remote access to this broker. -->
<amq:transportConnectors>
<amq:transportConnector uri="tcp://127.0.0.1:35000" />
</amq:transportConnectors>
</amq:broker>
<!-- This broker service (wrapper) control the lifecycle of the jms broker defined above.
By default the internal broker will be initialized, but not started. It will usually be started only for
internal integration and unit tests.
-->
<bean id="jmsBrokerService" class="com.imetrik.global.common.jms.AMQBrokerService"
init-method="start"
destroy-method="stop">
<property name="broker" ref="jmsBroker" />
<property name="enabled" value="true" />
</bean>
</beans>
</beans>
But when I run my application with the correct profile, spring complain that the jmsBroker reference cannot be found!
Error creating bean with name 'jmsBrokerService' defined in class path resource [jms-beans.xml]: Cannot resolve reference to bean 'jmsBroker' while setting bean property 'broker';
Is there a way to fix that? Is it because the reference bean is not in the same namespace?
I know that if I put it out of the nested beans, it is working fine.
I am using Spring 4.1

spring java xml configuration

im using spring for my java web app. the site has got bigger and i would like to set some configurations.
i have been researching and came across things like document builder factory, replacing spring xml with java config and others. i dunno where to start.
im thinking of implementing the configurations in xml (WEB/newConfig.xml) and have it read by the java beans. basically i wanna input my cofiguration values into xml and have it load by a java bean so that i can use it in controllers and jstl.
im just giving some examples here. for example xml configurations:
<property name="numberOfCars" value="3" />
<property name="webSiteName" value="New Spring Web App" />
....
and i read it in my java class:
class Config {
public getNumberOfCars() {
return numOfCars;
}
public getWebSiteName() {
return webSiteName;
}
}
where should i start and what online materials can i read?
==============================
update
here is what i have created.
applicationContext.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:aop="http://www.springframework.org/schema/aop" xmlns:context="http://www.springframework.org/schema/context"
xmlns:jee="http://www.springframework.org/schema/jee" xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd">
<context:property-placeholder location="/WEB-INF/your_prop_file.properties" />
<bean id="ConfigMgr" class="org.domain.class.ConfigMgr">
<property name="username" value="${username}">
</bean>
</beans>
you_prop_file.properties
username=hello world name
ConfigMgr.java
public class ConfigMgr {
private String username;
...getter
...setter
}
in my controller, here is what i did:
ConfigMgr config = new ConfigMgr();
sysout.out.println(config.getUsername());
i am getting null and i am sure im missing something here. where should i set the username value to the ConfigMgr class?
Spring Java configuration is a newer feature that allows you to configure your Spring application using Java classes instead of XML files. Its just an alternative for XML configuration. XML way is equally feature rich.
From what I could figure out from your problem, you want to move the hardcoded values of params (numberOfCars,webSiteName.. ) outisde your configuration file.
If that is the case, you don't have to go that far.
Just use :-
<context:property-placeholder location="classpath:your_prop_file.properties" />
in your spring xml file and replace the param values like:-
<property name="webSiteName" value="${website.name}" />
You need to have a your_prop_file.properties file in your classpath with enteries like:-
website.name=New Spring Web App
You are not injecting the ConfigMgr bean that you created in XML file.
What you are doing is you are creating a new Object in controller which does not have a clue about properties file.
Now you can inject it using either #Autowired inside your controller or through xml configuration.
There are plenty of examples available in google on basic spring dependency injection.

Integrating ehcache with spring 3.0

I have an application in which I use spring 3.0.2 and ibatis. Now, I need to integrate ehcache with my code. I tried this link but couldnt get it working. I would prefer someone to give me the details of the jars required, xml configurations to be done and code changes if required.
Upgrade to the latest spring 3.1 milestone - it has built-in cache support through annotations - see here
Apart from that, you can always use the EhCacheFactoryBean
To implement this in your application, follow these steps:
Step 1:
Add the jars to your application as listed on the Ehcache Annotations for Spring project site.
Step 2:
Add the Annotation to methods you would like to cache. Lets assume you are using the Dog getDog(String name) method from above:
#Cacheable(name="getDog")
Dog getDog(String name)
{
....
}
Step 3:
Configure Spring. You must add the following to your Spring configuration file in the beans declaration section:
<ehcache:annotation-driven cache-manager="ehCacheManager" />
Please refer to Ehcache site for complete details.
To integrate Ehcache just follow below steps
1 - Add Dependency in pom XML file
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-core</artifactId>
<version>2.6.9</version>
</dependency>
2 - create an xml file called spring-cache.xml put it in the resources folder
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:p="http://www.springframework.org/schema/p" xmlns:cache="http://www.springframework.org/schema/cache"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/cache
http://www.springframework.org/schema/cache/spring-cache.xsd">
<cache:annotation-driven/>
<bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheCacheManager">
<property name="cacheManager" ref="ehcache" />
</bean>
<bean id="ehcache" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
<property name="configLocation" value="classpath:ehcache.xml" />
</bean>
</beans>
3 - as you can see we are using reference of ehcache.xml so create file and put it in resources folder
<?xml version="1.0" encoding="UTF-8"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd" updateCheck="true"
monitoring="autodetect" dynamicConfig="true">
<cache name="users" maxEntriesLocalHeap="5000"
maxEntriesLocalDisk="1000" eternal="false" diskSpoolBufferSizeMB="20"
timeToIdleSeconds="200" timeToLiveSeconds="500"
memoryStoreEvictionPolicy="LFU" transactionalMode="off">
<persistence strategy="localTempSwap" />
</cache>
</ehcache>
so can see create a cache for "users" so that can use wherever user list is queried from the database
4 - use it like below code
#Cacheable(value="users")
public List<User> userList() {
return userDao.findAll();
}
so that's it the same way you can implement cache wherever is required
still have some doubt or confusion see live demo
Integrate EhCache in Spring MVC

Categories