I'm currently using Oracle AQ and would like to replace it by a persisted ActiveMQ.
My current setup using Oracle AQ is:
DB server: Oracle DB with a queue Q1
App server 1: Has a producer and multiple listeners on Q1
App server 2: Has a producer and multiple listeners on Q1
The following flow is currently followed:
App server 1:
Incoming message via a webservice
Start DB transaction
Save message in DB with id
Post id and other information of the message on queue Q1
commit transaction
App server 2:
Same setup, horizontal scaled
Requirements
When implementing ActiveMQ I want the data to the DB and the post on the queue in the same transaction. So that if one does a rollback, the other will do it as well.
Because I need to be able to produce messages on the queue with both app-servers at the same time, I need to run the ActiveMQ broker on the DB-server, and not on the app-servers. Otherwise they will act as a 'master slave'.
I an article I read, they explain how you can share transaction resources.
But this is done assuming you put the ActiveMQ broker on the same server as where the transaction is started.
Is there any way, except using JTA to accomplish this?
I'm using Java with:
Spring 2.5.6
Hibernate 3.3
TransactionManager: org.springframework.orm.hibernate3.HibernateTransactionManager
DataSource: oracle.jdbc.pool.OracleDataSource
I didn't test this. But isn't this what you're looking for?
import javax.jms.ConnectionFactory;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageProducer;
import javax.jms.Queue;
import javax.jms.Session;
import javax.jms.TextMessage;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jms.JmsException;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.jms.core.MessageCreator;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
#Service
public class SomeServiceImpl implements SomeService {
#Autowired
private Queue someQueue;
private JmsTemplate jmsTemplate;
private SomeRepository repository;
public SomeServiceImpl() {}
public SomeServiceImpl(ConnectionFactory activeMQConnectionFactory, SomeRepository repository) {
this.jmsTemplate = new JmsTemplate(activeMQConnectionFactory);
this.repository = repository;
}
#Transactional(rollbackFor = {JmsException.class, RepositoryException.class})
public void sendMessage(final SomeObject object) {
repository.save(object);
jmsTemplate.send(
new MessageCreator() {
#Override
public Message createMessage(Session session) throws JMSException {
return session.createObjectMessage(object.getSpecialId());
}
});
}
}
This way if there is a failure while saving the object or an issue with sending the message to the queue (JMSException), the local transaction will be rolled back and the object won't be persisted. I don't think you will be needing a distributed transaction.
Related
I am having trouble getting messages from a locally run ActiveMQ. I can produce them onto the queue and my PC also is registered as producer. However, another Spring App on the machine should be configured as a listener. So far it is not working. ActiveMQ is listening on the default ports.
My JMS config for the sender:
package at.dkepr.queueservice;
import org.apache.activemq.command.ActiveMQQueue;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.jms.Queue;
#Configuration
public class JmsConfig {
#Bean
public Queue queue(){
return new ActiveMQQueue("indexing-queue");
}
}
And this is the consumer:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jms.annotation.EnableJms;
import org.springframework.jms.annotation.JmsListener;
import org.springframework.stereotype.Component;
import at.dkepr.entity.UserSearchEntity;
#Component
#EnableJms
public class JmsConsumer {
private final Logger logger = LoggerFactory.getLogger(JmsConsumer.class);
#JmsListener(destination = "indexing-queue", containerFactory = "jmsListenerContainerFactory")
public void receive(UserSearchEntity user){
logger.info(user.getEmail());
}
}
In the application.propertiers I have added the necessary properties:
spring.activemq.broker-url=tcp://localhost:61616
spring.activemq.user=admin
spring.activemq.password=admin
Also the UserSearchEntity implements Serializable.
To the best of my knowledge for this setup I should not even need a config for the consumer. Never the less, I added one.
import org.apache.activemq.ActiveMQConnectionFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jms.annotation.EnableJms;
import org.springframework.jms.config.DefaultJmsListenerContainerFactory;
#Configuration
#EnableJms
public class ConsumerConfig {
#Value("${spring.activemq.broker-url}")
private String brokerUrl;
#Bean
public ActiveMQConnectionFactory activeMQConnectionFactory() {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(brokerUrl);
return activeMQConnectionFactory;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(activeMQConnectionFactory());
factory.setConcurrency("1-3");
return factory;
}
}
I am not getting any error logs. Unfortunately, it is simply doing nothing.
This is a screenshot from the ActiveMQ web console with the enqueued messages:
My consuming application was running when I took this screenshot, but the broker clearly does not recognize it since the "Number of Consumers" is 0.
Edit:
I just tried adding the Listener to the same Spring Application where the Producer is. Surprinsingly, the Listener connected fine. It seems like the problem lies in the different Spring Applications. However, i used the same application.properties for both Spring Apps. The Config File is the same too.
To everyone having the same problem:
For me it was a simple problem with folder structure. For some reason the Application.java for the consumer service was in a subfolder. After i moved the Application.java one folder up, the connection to the ActiveMQ worked.
I have a SpringBoot application. I have a dedicated server, where I shall read data by the HTTP GET requests. I configured the http-outbound-config.xml file for Spring Integration module.
When I run the following code, everything is fine:
http-outbound-config.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-http="http://www.springframework.org/schema/integration/http"
xsi:schemaLocation="http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration https://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/http https://www.springframework.org/schema/integration/http/spring-integration-http.xsd">
<int:channel id="requestChannel"/>
<int:channel id="replyChannel">
<int:queue capacity='10'/>
</int:channel>
<int-http:outbound-gateway id="outboundGateway"
request-channel="requestChannel"
url="http://server/API.jsp?id=1"
http-method="GET"
expected-response-type="java.lang.String"
charset="UTF-8"
reply-channel="replyChannel"/>
</beans>
Main Application Class:
package test;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.ImportResource;
import ru.eco.products.waste.egr.Integration;
#SpringBootApplication
#ImportResource("/META-INF/spring/integration/http-outbound-config.xml")
public class Application {
public static void main(String[] args) {
Integration integration = new Integration();
integration.start();
SpringApplication.run(WasteWebClientApplication.class,
args
);
}
}
Integration class:
package test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.ImportResource;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.PollableChannel;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;
#Component
#Configuration
public class Integration {
public void start() {
ClassPathXmlApplicationContext
context = new ClassPathXmlApplicationContext("classpath:META-INF/spring/integration/http-outbound-config.xml");
context.start();
MessageChannel requestChannel = context.getBean("requestChannel", MessageChannel.class);
PollableChannel replyChannel = context.getBean("replyChannel", PollableChannel.class);
Message<?> message = MessageBuilder.withPayload("").build();
requestChannel.send(message);
Message<?> receivedMsg = replyChannel.receive();
System.out.println("RESULT IS : " + receivedMsg.getPayload());
}
}
BUT, when I try to Autowire MessageChannel and PollableChannel, I receive a null pointer exception.
Integration class(not working example):
package test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.ImportResource;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.PollableChannel;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;
#Component
#Configuration
public class Integration {
#Autowired
#Qualifier("requestChannel")
MessageChannel requestChannel;
#Autowired
#Qualifier("replyChannel")
PollableChannel replyChannel;
public void start() {
Message<?> message = MessageBuilder.withPayload("").build();
requestChannel.send(message);
Message<?> receivedMsg = replyChannel.receive();
System.out.println("RESULT IS : " + receivedMsg.getPayload());
}
}
Question 1: Why Autowiring is not working?
Question 2: What is the best way to get data from dedicated server and save it into DB? Such config is ok? I will create a model class for the response and after will save it into DB via JPA.
Question 3: I need reading from server works in Async mode. How can I implement it in Spring Boot? So the main idea here is that I will receive a POST method from the UI, and will launch the integration with web-service. After integration will be finished, I need to notify user.
Question 4: Maybe Camel will be the best solution here?
Stack: Java 11, Spring Boot, thymeleaf + bootstrap.
Thank you for the answer in advance.
Since you do new Integration();, you definitely not going to have dependency injection since inversion of control container is not involved. Although it is fully not clear why do you need that new ClassPathXmlApplicationContext("classpath:META-INF/spring/integration/http-outbound-config.xml") if you already do Spring Boot and that proper #ImportResource.
The best way as you already do with the #ImportResource for that Spring Integration XML config. Then you need to get access to the ApplicationContext of the SpringApplication.run() and getBean(Integration.class) to call your start() method. However you fully need to forget about new Integratio(). Spring is going to manage a bean for that Integration and then dependency injection is going to work.
Async mode can be achieved with an ExecutorChannel in Spring Integration. So, when you send a message to this channel, the logic is going to be processed on a different thread. However it is not clear why you state such a requirement since you still going to block via that replyChannel.receive()... Although this should be addressed in a separate SO thread.
Camel VS Spring Integration question is out of StackOverflow policies.
Thymeleaf and Bootstrap are misleading in the context of this question.
Please, consider to learn how to ask properly here on SO: https://stackoverflow.com/help/how-to-ask
Also read some docs about dependency injection and inversion of control: https://docs.spring.io/spring-framework/docs/current/reference/html/core.html#spring-core
I created a simple standalone consumer, trying to consume 4 messages sitting on the ActiveMQ. But when I started the application, it created another Queue with the same name as shown in the image below:
My Project Structure looks like this:
And code inside classes looks like the following:
class FebMessageConsumer
package com.consumer.messages.febMessageConsumer;
import org.springframework.jms.annotation.JmsListener;
import org.springframework.stereotype.Component;
#Component
public class FebMessageConsumer {
#JmsListener(destination = "CDD Feb 21 Queue")
///#JmsListener
public void processFebMessage(String message) {
System.out.println("Message Retrieved is:" +message);
}
}
class FebMessageConsumerApplication
package com.consumer.messages.febMessageConsumer;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.jms.annotation.EnableJms;
#SpringBootApplication
#EnableJms
public class FebMessageConsumerApplication {
public static void main(String[] args) {
SpringApplication.run(FebMessageConsumerApplication.class, args);
}
}
1) What's wrong in the above code?
2) Once I get above thing working,I plan on deploying it as a WAR to Apache Tomcat 8.5. Is it like when I deploy the application or start the application as Java Application, it's going to consume all the messages one by one? OR when I start the application, only one message will be consumed at a time and then I'll have to stop the application and then start again to consume next message?
Here's a Google Drive Link to the zipped project in case needed for reference.
The existing queue name includes quotes.
Use #JmsListener(destination = "\"CDD Feb 21 Queue\"").
It will continually receive messages one by one.
I've been developing a project using spring framework 4. I'm trying to create a simple TCP client via spring-integration-ip library. I've adjusted all configurations:
applicationContext.xml
...
<int:channel id="tcpChannel" />
<int-ip:tcp-outbound-channel-adapter id="outboundClient"
channel="tcpChannel"
connection-factory="tcpConnectionFactory"/>
...
bean configuration:
#Configuration
public class MyConfiguration{
#Bean
public AbstractClientConnectionFactory tcpConnectionFactory() {
return new TcpNetClientConnectionFactory("localhost", 2345);
}
}
I've read all documentations about spring tcp here.
I guess I must use tcp-outbound-channel-adapter or gateway to send messages. but I wonder how to use it; what method should I invoke. I'm not supposed to receive any messages from server.
I found the solution. I didn't need gateway. spring messaging gateways have been designed to implement the request-response scenario. So the only thing I need to do is that I send message vi channel. Perhaps there be some better solutions.
import org.springframework.integration.support.MessageBuilder;
import org.springframework.messaging.MessageChannel;
public class MyOwnService{
#Inject
private MessageChannel channel;
public void someMethod(String message){
Message<String> m = MessageBuilder.withPayload(message).build();
channel.send(m);
}
}
How to catch an error writing to the Kafka topic?
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Processor;
import org.springframework.messaging.Message;
import org.springframework.messaging.handler.annotation.SendTo;
import org.springframework.stereotype.Component;
import javax.xml.bind.JAXBException;
#Component
#EnableBinding(Processor.class)
public class Parser {
#StreamListener(Processor.INPUT)
#SendTo(Processor.OUTPUT)
public String process(Message<?> input) {
// do smth
// how to catch an error if sending message to 'output' topic fails
}
}
Switch producer to synchronous mode.
spring.cloud.stream.kafka.bindings.output.producer.sync=true
What next? Any examples? Extend some binding impl, add some AOP magic?
I think what you need is to register a KafkaProducerListener which handles the error scenario in your case and make it available as a bean in your application.
For more info on KafkaProducerListener is here
Also, note that usually the failed messages would be available on errorChannel with the channel name error. All the error messages are published once you have the configure the destination name for the error channel: spring.cloud.stream.bindings.error.destination.