We Keep Coding

Is it possible to define when camel JMS Component must connect to brokerURL or just log something

The problem is that I need to have the possibility to control through property value when the Camel must connect to Artemis URL and when just Log a message that this connection is disabled.
I know this easily could be done in route configuration, but I'm curious is it possible to do it in the Camel JMS component or its configuration process?
Property values:
activemq:
url: tcp://localhost:61616
password: admin
user: admin
connect: true
My Configuration:
#Bean("amqConnFact")
public ActiveMQConnectionFactory getAmqCF(#Value("${activemq.url}") String url,
#Value("${activemq.user}") String user,
#Value("${activemq.password}") String password) {
ActiveMQConnectionFactory amqCF = new ActiveMQConnectionFactory();
amqCF.setBrokerURL(url);
amqCF.setUserName(user);
amqCF.setPassword(password);
return amqCF ;
}
#Bean("cachingConnFact")
public CachingConnectionFactory getCachingConnectionFactory(#Qualifier("amqConnFact") ActiveMQConnectionFactory amqCF) {
CachingConnectionFactory cachingCF = new CachingConnectionFactory();
cachingCF.setTargetConnectionFactory(amqCF);
return cachingCF;
}
I'd like to have something like this:
#Bean("mycomp")
public JmsComponent activeMqComponent(#Qualifier("cachingConnFact", #Value("${activemq.connect}") boolean connect) CachingConnectionFactory cachingCF) {
if(connect){
return JmsComponent.setConnectionFactory(cachingCF);
}else{
-- LOG "This connection is disabled in properties"--
}
}
And in route it could look just like this:
...
.to("mycomp:topic:WOW.IT.WORKS");

Aws Elastic Cache (Redis) failed to connect (jedis connection error) when acessed locally through spring boot java

I am working on a spring boot application where I have to store OTP in Elastic cache (Redis).
Is elastic cache right choice to store OTP?
Using Redis to store OTP
To connect to Redis locally I used "sudo apt-get install Redis-server". It installed and successfully run.
I created a Redisconfig where I asked the application config file for port and hostname. Here I thought I will use this hostname and port to connect to aws elastic cache but Right now I am running locally.
public class RedisConfig {
#Value("${redis.hostname}")
private String redisHostName;
#Value("${redis.port}")
private int redisPort;
#Bean
protected JedisConnectionFactory jedisConnectionFactory() {
return new JedisConnectionFactory();
}
#Bean
public RedisTemplate<String,Integer> redisTemplate() {
final RedisTemplate<String, Integer> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(jedisConnectionFactory());
return redisTemplate;
}
Now I used the RedisTemplate and valueOperation to put, read the data in Redis cache
public class MyService {
private RedisTemplate<String, Integer> redisTemplate;
private ValueOperations<String, Integer> valueOperations;
public OtpService(RedisTemplate<String, Integer> redisTemplate) {
super();
this.redisTemplate = redisTemplate;
valueOperations = redisTemplate.opsForValue();
}
public int generateOTP(String key) throws Exception {
try {
Random random = new Random();
int otp = 1000 + random.nextInt(9000);
valueOperations.set(key, otp, 120, TimeUnit.SECONDS);
return otp;
} catch (Exception e) {
throw new Exception("Exception while setting otp" + e.getMessage()) ;
}
}
public int getOtp(String key) {
try {
return valueOperations.get(key);
} catch (Exception e) {
return 0;
}
}
}
Now This is what I have done and which is running perfectly in local.
Questions I have :
What changes do I need when I am deploying the application in EC2 instance. Do we need to configure hostname and port in the code?
If we need to configure, Is there a way to test locally what would happen when we deploy? Can we simulate that environment somehow?
I have read that to access aws elastic cache (Redis) locally we have to set up proxy server, which is not a good practice, so how can we easily build the app locally and deploy on the cloud?
Why did ValueOperations don't have "delete" method when it has set, put methods? How can I invalidate cache once its usage is done before the expiry time?
Accessing the AWS cache locally:
When I tried to access the aws elastic cache (Redis) by putting the post and hostname in the creation of JedisConnectionFactory instance
#Bean
protected JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration(redisHostName, redisPort);
JedisConnectionFactory factory = new JedisConnectionFactory(configuration);
return factory;
}
I got an error while setting the key value:
Cannot get Jedis connection; nested exception is
redis.clients.jedis.exceptions.JedisConnectionException: Could not get
a resource from the pool
I tried to explain what I have done and what I needed to know?
If anybody knows any blog, resources where things are mentioned in detail please direct me there.

After posting the question, I tried things myself.
As per amazon,
Your Amazon ElastiCache instances are designed to be accessed through
an Amazon EC2 instance.
To connect to Redis locally on Linux,
Run "sudo apt-get install Redis-server". It will install redis server.
Run "redis-cli". It will run Redis on localhost:6379 successfully run.
To connect to server in java(spring boot)
Redisconfig
For local in application.properties: redis.hostname = localhost, redis.port = 6379
For cloud or when deployed to ec2: redis.hostname = "amazon Elastic cache endpoint", redis.port = 6379
public class RedisConfig {
#Value("${redis.hostname}")
private String redisHostName;
#Value("${redis.port}")
private int redisPort;
#Bean
protected JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration(redisHostName, redisPort);
JedisConnectionFactory factory = new JedisConnectionFactory(configuration);
return factory;
}
#Bean
public RedisTemplate<String,Integer> redisTemplate() {
final RedisTemplate<String, Integer> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(jedisConnectionFactory());
return redisTemplate;
}
With this whether you are running locally or on cloud just need to change the URL and things will work perfectly.
After this use RedisTemplate and valueOperation to put, read the data in Redis cache. Same as I mentioned in the question above. No need for any changes.
Answers to the questions:
We need to change the hostname when deploying in the EC2 instance.
Running Redis server locally is exactly same as running Redis when the application is deployed on EC2, no need for changes, use the Redis config I am using.
Yes, don't create a proxy server, this beats the very idea of the cache. run locally with Redis server and change hostname
I still need to find a way to invalidate the cache when using valueOperations

ActiveMQ RedeliveryPolicy not being set

I´m using:
SpringBoot 2.0.4
ActiveMQ 5.15.5
Apache Camel 2.22.0
Java 1.8
Groovy
Maven
Basically, I have a SpringBoot application with an Apache Camel route which consumes messages from ActiveMQ with transactions. I need to set a RedeliveryPolicy on ActiveMQ, so when an error in processing happens, the message is retried a number of times.
I have made a configuration class with beans for ActiveMQ, the transactions work as intended but the RedeliveryPolicy does not work. Can anybody please help me understand what's wrong with this?
Here's the log output for a message that produces an error:
2018-10-23 10:35:28.005 DEBUG 10524 --- [mer[entryQueue]] o.a.c.s.spi.TransactionErrorHandler : Transaction begin (0x35d60381) redelivered(false) for (MessageId: ID:EPIC-LAP-25-50304-1540306817804-4:3:1:1:2 on ExchangeId: ID-EPIC-LAP-25-1540312510586-0-1))
2018-10-23 10:35:28.020 DEBUG 10524 --- [mer[entryQueue]] o.apache.camel.processor.SendProcessor : >>>> direct://middle Exchange[ID-EPIC-LAP-25-1540312510586-0-1]
2018-10-23 10:35:28.375 DEBUG 10524 --- [mer[entryQueue]] o.a.camel.processor.DefaultErrorHandler : Failed delivery for (MessageId: ID:EPIC-LAP-25-50304-1540306817804-4:3:1:1:2 on ExchangeId: ID-EPIC-LAP-25-1540312510586-0-1). On delivery attempt: 0 caught: java.lang.RuntimeException: ExceptionTest: Order Failed
2018-10-23 10:35:28.390 ERROR 10524 --- [mer[entryQueue]] o.a.camel.processor.DefaultErrorHandler : Failed delivery for (MessageId: ID:EPIC-LAP-25-50304-1540306817804-4:3:1:1:2 on ExchangeId: ID-EPIC-LAP-25-1540312510586-0-1). Exhausted after delivery attempt: 1 caught: java.lang.RuntimeException: ExceptionTest: Order Failed
Here's my config class for ActiveMQ:
import org.apache.activemq.ActiveMQConnectionFactory
import org.apache.activemq.RedeliveryPolicy
import org.apache.activemq.camel.component.ActiveMQComponent
import org.springframework.beans.factory.annotation.Qualifier
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.jms.connection.JmsTransactionManager
import javax.jms.DeliveryMode
#Configuration
class ActiveMQConfiguration {
#Bean
ActiveMQConnectionFactory activeMQConnectionFactory() {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory()
activeMQConnectionFactory.brokerURL = 'tcp://localhost:61616'
activeMQConnectionFactory.userName = 'admin'
activeMQConnectionFactory.password = 'admin'
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy()
redeliveryPolicy.maximumRedeliveries = 3
redeliveryPolicy.redeliveryDelay = 150L
redeliveryPolicy.useExponentialBackOff = true
redeliveryPolicy.backOffMultiplier = 1.5
activeMQConnectionFactory.setRedeliveryPolicy(redeliveryPolicy)
activeMQConnectionFactory
}
#Bean
ActiveMQComponent activeMQComponent(#Qualifier('activeMQConnectionFactory')ActiveMQConnectionFactory activeMQConnectionFactory) {
ActiveMQComponent activeMQComponent = new ActiveMQComponent()
activeMQComponent.connectionFactory = activeMQConnectionFactory
activeMQComponent.transacted = true
activeMQComponent.transactionManager = txManager()
activeMQComponent.cacheLevelName = 'CACHE_CONSUMER'
activeMQComponent.lazyCreateTransactionManager = false
activeMQComponent.deliveryMode = DeliveryMode.PERSISTENT
activeMQComponent
}
#Bean
JmsTransactionManager txManager(#Qualifier('activeMQConnectionFactory') ActiveMQConnectionFactory activeMQConnectionFactory) {
JmsTransactionManager txManager = new JmsTransactionManager()
txManager.connectionFactory = activeMQConnectionFactory
txManager.rollbackOnCommitFailure = true
txManager
}
}

There are two issues here
1. You have two transaction managers
Due to the following two lines in your configuration of the Camel ActiveMQ component, you configure two transaction managers. That is a source of problems.
activeMQComponent.transacted = true // activates local JMS transactions
activeMQComponent.transactionManager = txManager() // additional tx manager
if you just want to consume transactional from ActiveMQ, you don't need to configure a Spring transaction manager.
These two lines of your config are enough to get local transactions with your ActiveMQ broker.
activeMQComponent.transacted = true
activeMQComponent.lazyCreateTransactionManager = false
So you should remove this line as well as the whole txManager bean
activeMQComponent.transactionManager = txManager()
If you currently set the transacted flag in your Camel routes, you have to remove this too. And as I wrote, your routes consuming from ActiveMQ are still transacted even if you remove all this.
2. Redelivery not working
You have not published your Camel routes, but according to the error output, I assume that the broker does not redeliver because the error is handled by Camel.
It is the Camel error handler o.a.camel.processor.DefaultErrorHandler that kicks in when the error occurs and because it handles the error, the message is commited against the broker and therefore no redelivery takes place.
Try to disable Camel error handling to see if the broker redelivers messages on errors.
errorHandler(noErrorHandler());

Not very long ago I had problems with dlq queues - not all parameters set in the code worked. I had to add settings to acitvemq configs. Yes, it is not a good decision to divide configs but I didn't find another.
Below is my config class for jms and an example queue configuration via activemq.xml:
#Configuration
#EnableJms
public class JmsConfig {
private Environment env;
#Autowired
public void setEnv(Environment env) {
this.env = env;
}
#Bean(name = "activemq")
public ActiveMQComponent activemq(#Qualifier("activemqTransactionManager") JmsTransactionManager jmsTransactionManager,
#Qualifier("activemqConnectionFactory") ConnectionFactory connectionFactory) {
ActiveMQComponent activeMQComponent = new ActiveMQComponent();
activeMQComponent.setTransactionManager(jmsTransactionManager);
activeMQComponent.setConnectionFactory(connectionFactory);
return activeMQComponent;
}
#Bean(name = "activemqJmsTemplate")
public JmsTemplate jmsTemplate(#Qualifier("activemqConnectionFactory") ConnectionFactory connectionFactory) {
JmsTemplate template = new JmsTemplate();
template.setConnectionFactory(connectionFactory);
return template;
}
#Bean(name = "activemqTransactionPolicy")
public SpringTransactionPolicy activemqTransactionPolicy(
#Qualifier("activemqTransactionManager") JmsTransactionManager jmsTransactionManager) {
SpringTransactionPolicy springTransactionPolicy = new SpringTransactionPolicy(jmsTransactionManager);
springTransactionPolicy.setPropagationBehaviorName("PROPAGATION_REQUIRED");
return springTransactionPolicy;
}
#Bean(name = "activemqTransactionManager")
public JmsTransactionManager activemqTransactionManager(
#Qualifier("activemqConnectionFactory") ConnectionFactory connectionFactory) {
return new JmsTransactionManager(connectionFactory);
}
#Bean(name = "activemqConnectionFactory")
public ConnectionFactory connectionFactory(#Qualifier("activemqRedeliveryPolicy") RedeliveryPolicy redeliveryPolicy) {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL("tcp://" + env.getProperty("queue.url"));
connectionFactory.setTrustAllPackages(true);
RedeliveryPolicyMap map = connectionFactory.getRedeliveryPolicyMap();
map.put(new ActiveMQQueue("queueName.DLQ"), redeliveryPolicy);
return connectionFactory;
}
#Bean(name = "activemqRedeliveryPolicy")
public RedeliveryPolicy redeliveryPolicy() {
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setMaximumRedeliveries(0);
return redeliveryPolicy;
}
}
Changes in activevq.xml:
<destinationPolicy>
<policyMap>
<policyEntries>
<!--set dead letter queue for our queue. It name will be "myQueueName.DLQ"-->
<policyEntry queue="myQueueName">
<deadLetterStrategy>
<individualDeadLetterStrategy queuePrefix="" queueSuffix=".DLQ"/>
</deadLetterStrategy>
</policyEntry>
<policyEntry topic=">">
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<plugins>
<redeliveryPlugin fallbackToDeadLetter="true" sendToDlqIfMaxRetriesExceeded="true">
<redeliveryPolicyMap>
<redeliveryPolicyMap>
<redeliveryPolicyEntries>
<!--Set the redelivery delay to one hour-->
<redeliveryPolicy queue="myQueueName.DLQ" maximumRedeliveries="-1" redeliveryDelay="3600000"/>
</redeliveryPolicyEntries>
</redeliveryPolicyMap>
</redeliveryPolicyMap>
</redeliveryPlugin>
</plugins>

Looking for a way to parse values in springboot application.yml file before SpringBootApplication is initialized

So my problem is as follows:
I'm using spring AMQP to connect to a rabbitMQ instance that using SSL. Unfortunately, spring AMQP does not currently support full length amqps URIs and adding support is not high on the priority list (see issue: https://github.com/spring-projects/spring-boot/issues/6401 ). They need to be separated.
The following fields are required in my application.yml to connect:
spring:
rabbitmq:
host: hostname
port: portnumber
username: username
password: password
virtual-host: virtualhost
ssl:
enabled: true
My VCAP_Services environment for my rabbitMQ instance only provides the virtualhost and the full length uri in the following format: amqps://username:password#hostname:portnumber/virtualhost
Copy and pasting these values into my application.yml is fine for now, but in the long run is not viable. They will need to come from vcap_services.
My #SpringBootApplication has #Beans that initialize a connection to the rabbitMQ instance on startup, so I am looking for a way to parse out the individual values and set them before the application is started.

If you are just interested in reading the properties before your Spring Boot application is initialized, you can parse the yaml file using Spring's YamlPropertiesFactoryBean before you call SpringApplication.run. For example,
#SpringBootApplication
public class Application {
public static void main(String[] args) {
YamlPropertiesFactoryBean yamlFactory = new YamlPropertiesFactoryBean();
yamlFactory.setResources(new ClassPathResource("application.yml"));
Properties props = yamlFactory.getObject();
String hostname = props.getProperty("spring.rabbitmq.hostname");
...
SpringApplication.run(Application.class, args);
}
}

Simply override Boot's auto configured connection factory...
#SpringBootApplication
public class So46937522Application {
public static void main(String[] args) {
SpringApplication.run(So46937522Application.class, args);
}
#Bean
public CachingConnectionFactory rabbitConnectionFactory(RabbitProperties config)
throws Exception {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.getRabbitConnectionFactory()
.setUri("amqps://guest:guest#10.0.0.3:5671/virtualhost");
return connectionFactory;
}
#RabbitListener(queues = "si.test.queue")
public void listen(Message in) {
System.out.println(in);
}
}

Spring Boot Redis configuration not working

I am developing a Spring Boot [web] REST-style application with a ServletInitializer (since it needs to be deployed to an existing Tomcat server). It has a #RestController with a method that, when invoked, needs to write to a Redis pub-sub channel. I have the Redis server running on localhost (default port, no password). The relevant part of the POM file has the required starter dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
When I deploy the WAR and hit the endpoint http://localhost:8080/springBootApp/health, I get this response:
{
"status": "DOWN",
"diskSpace": {
"status": "UP",
"total": 999324516352,
"free": 691261681664,
"threshold": 10485760
},
"redis": {
"status": "DOWN",
"error": "org.springframework.data.redis.RedisConnectionFailureException: java.net.SocketTimeoutException: Read timed out; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out"
}
}
I added the following to my Spring Boot application class:
#Bean
JedisConnectionFactory jedisConnectionFactory() {
return new JedisConnectionFactory();
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<String, Object>();
template.setConnectionFactory(jedisConnectionFactory());
return template;
}
I also tried adding the following to my #RestController before executing some test Redis code, but I get the same error as above in the stack trace:
#Autowired
private RedisTemplate<String, String> redisTemplate;
Edit (2017-05-09)
My understanding is that Spring Boot Redis starter assumes the default values of spring.redis.host=localhost and spring.redis.port=6379, I still added the two to application.properties, but that did not fill the gap.
Update (2017-05-10)
I added an answer to this thread.

I done a simple example with redis and spring boot
First I installed redis on docker:
$ docker run --name some-redis -d redis redis-server --appendonly yes
Then I Used this code for receiver :
import java.util.concurrent.CountDownLatch;
public class Receiver {
private static final Logger LOGGER = LoggerFactory.getLogger(Receiver.class);
private CountDownLatch latch;
#Autowired
public Receiver(CountDownLatch latch) {
this.latch = latch;
}
public void receiveMessage(String message) {
LOGGER.info("Received <" + message + ">");
latch.countDown();
}
}
And this is my spring boot app and my listener:
#SpringBootApplication
// after add security library then it is need to use security configuration.
#ComponentScan("omid.spring.example.springexample.security")
public class RunSpring {
private static final Logger LOGGER = LoggerFactory.getLogger(RunSpring.class);
public static void main(String[] args) throws InterruptedException {
ConfigurableApplicationContext contex = SpringApplication.run(RunSpring.class, args);
}
#Autowired
private ApplicationContext context;
#RestController
public class SimpleController{
#RequestMapping("/test")
public String getHelloWorld(){
StringRedisTemplate template = context.getBean(StringRedisTemplate.class);
CountDownLatch latch = context.getBean(CountDownLatch.class);
LOGGER.info("Sending message...");
Thread t = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0 ; i < 100 ; i++) {
template.convertAndSend("chat", i + " => Hello from Redis!");
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
});
t.start();
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
return "hello world 1";
}
}
///////////////////////////////////////////////////////////////
#Bean
RedisMessageListenerContainer container(RedisConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter) {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.addMessageListener(listenerAdapter, new PatternTopic("chat"));
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(Receiver receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
#Bean
Receiver receiver(CountDownLatch latch) {
return new Receiver(latch);
}
#Bean
CountDownLatch latch() {
return new CountDownLatch(1);
}
#Bean
StringRedisTemplate template(RedisConnectionFactory connectionFactory) {
return new StringRedisTemplate(connectionFactory);
}
}
The important point is the redis IP. if you installed it on docker like me then
you should set ip address in application.properties like this:
spring.redis.host=172.17.0.4
I put all my spring examples on github here
In addition I used redis stat to monitor redis. it is simple monitoring.

Spring data redis properties are updated, e.g. spring.redis.host is now spring.data.redis.host.

You need to configure your redis server information using the application.properties:
# REDIS (RedisProperties)
spring.redis.cluster.nodes= # Comma-separated list of "host:port"
spring.redis.database=0 # Database index
spring.redis.url= # Connection URL,
spring.redis.host=localhost # Redis server host.
spring.redis.password= # Login password of the redis server.
spring.redis.ssl=false # Enable SSL support.
spring.redis.port=6379 # Redis server port.
Spring data docs: https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html#REDIS

This was a proxy related problem, where even access to localhost was somehow being curtailed. Once I disabled the proxy settings, Redis health was UP! So the problem is solved. I did not have to add any property to application.properties and neither did I have to explicitly configure anything in the Spring Boot application class, because Spring Boot and the Redis Starter auto-configures based on Redis defaults (as applicable in my development environment). I just added the following to the pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
and the following to the #RestController annotated class, and Spring Boot auto-wired as needed (awesome!).
#Autowired
private RedisTemplate<String, String> redisTemplate;
To publish a simple message to a channel, this single line of code was sufficient for validating the setup:
this.redisTemplate.convertAndSend(channelName, "hello world");
I appreciate all the comments, which were helpful in backing up my checks.