So far I have implemented Spring XD processors, e.g. like this:
#MessageEndpoint
public class MyTransformer
{
#Transformer( inputChannel = "input", outputChannel = "output" )
public String transform( String payload )
{
...
}
};
However, I am stuck at implementing a custom sink now. The current documentation is not very helpful, since it simply configures something "magically" via XML:
<beans ...>
<int:channel id="input" />
<int-redis:store-outbound-channel-adapter
id="redisListAdapter" collection-type="LIST" channel="input" key="${collection}" auto-startup="false"/>
<beans:bean id="redisConnectionFactory"
class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<beans:property name="hostName" value="${host}" />
<beans:property name="port" value="${port}" />
</beans:bean>
</beans>
This will use the redis store-outbound-channel-adapter as a sink. However, the documentation does not tell me how to create a simple, generic sink that simply has one input channel and consumes a message.
So can anyone provide me with a minimal working example?
A sink is just like a processor but without an output channel; use a #ServiceActivator to invoke your code (which should have a void return).
#MessageEndpoint
public class MyService
{
#ServiceActivator( inputChannel = "input")
public void handle( String payload )
{
...
}
};
EDIT
For sources, there are two types:
Polled (messages are pulled from the source):
#InboundChannelAdapter(value = "output",
poller = #Poller(fixedDelay = "5000", maxMessagesPerPoll = "1"))
public String next() {
return "foo";
}
Message-driven (where the source pushes messages):
#Bean
public MySource source() {
// return my subclass of MessageProducer that has outputChannel injected
// and calls sendMessage
// or use a simple POJO that uses MessagingTemplate.convertAndSend(foo)
}
Related
I am struggling to make a working junit test that rolls back actions that the occurred during the camel routing.
I have a camel route setup that listens on a directory. It is expecting a csv file. When the csv file appears it then creates new SearchAnalytics data. It adds a new row into a table per each line in the csv file.
The default spring transaction methods that I have put do not seem to apply to actions that occur on the camel routing.
The code below works. However it saves the data permanently and does not rollback the insert. This means that the test will only pass once unless I manually delete the data.
Given my example code how do I make it roll back the transaction?
my route looks like this
from("ftp://some__remote__ftp_dir_path")
.routeId("searchAnalyticsImport")
.choice()
.when(simple("${in.header.CamelFileName} contains '.csv'"))
.split().method("csvSplitter", "iterator").streaming() // reads the csv file returns data objects
.processRef("searchAnalyticsProcesser") // this some dao saves
.to(Queues.SOME_REQUEST)
.end();
Junit test
#TransactionConfiguration(defaultRollback = true, transactionManager = "transactionManager")
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = { TestAppConfig.class})
public class searchAnalyticsImportTest {
#EndpointInject(uri = "mock:sippmatcher.requestqueue?preserveMessageQos=true")
private MockEndpoint mockEndpointRequest;
#Before
public void setup() throws Exception {
camelContext.getRouteDefinition("searchAnalyticsImport").adviceWith(camelContext, new AdviceWithRouteBuilder() {
#Override
public void configure() throws Exception {
replaceFromWith("file://"+this.getClass().getResource("path to folder etc...")+"?noop=true");
interceptSendToEndpoint(Queues.SOME_REQUEST)
.skipSendToOriginalEndpoint()
.to(mockEndpointRequest);
}
});
}
#Test
public void simpleTest() throws Exception{
// there are 2 results in the test csv file.. need to poll the results till it completes
PollWithTimeout.run("keep polling until route has been statisfied", 15000, new PollWithTimeout.Attempt() {
#Override
public boolean complete() {
Date dateTime1MinuteAgo = new DateTime().minusMinutes(1).toDate();
Integer newSearchCount = searchAnalysiticDao.findBySearchStartedAfter(dateTime1MinuteAgo).size();
System.out.println("Recently added count: " + newSearchCount);
return (newSearchCount == 2);
}
});
mockEndpointRequest.expectedMessageCount(2);
mockEndpointRequest.assertIsSatisfied();
}
}
Add bean to Context (will add javaconfig option to this)
As mentioned in the comment section by Andreas you can add .transacted to the route and ensure that your transaction manager bean is injected in your context file.
Route
from("ftp://some__remote__ftp_dir_path")
.routeId("searchAnalyticsImport")
.end()
.transacted("PROPAGATION_REQUIRED")
etc....
Context Bean Config
<bean id="jmsTransactionManager"
class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="pooledConnectionFactory" />
<property name="defaultTimeout" value="30"/>
</bean>
<bean id="PROPAGATION_REQUIRED" class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="jmsTransactionManager" />
<property name="propagationBehaviorName" value="PROPAGATION_REQUIRED" />
</bean>
Alternatively add transaction to dao
you can use the below annotation at the dao method that is being called in the searchAnalyticsProcesser. A transaction maanger bean will still be required but you can specify it by name in the annotation.
#Transactional(
propagation = Propagation.REQUIRED,
readOnly = false,
value="transactionManager",
rollbackFor = {
Exception.class
})
public void insertStuff()
I have to asynchronously push some files in from my system A to system B. For that i have created a JMS Consumer. Once Entries are made in queue successfully using an enqueue stored procedure in oracle. My consumer should read the message and send it to system B.
Here is my Listeners Code
public class DMSCustomMessageListener extends DefaultMessageListenerContainer{
protected MessageConsumer createConsumer(Session session, Destination destination)
throws JMSException
{
return ((AQjmsSession)session).createConsumer(destination,
getMessageSelector(),
DMS_Master_Type.getORADataFactory(), null, isPubSubNoLocal());
}
}
public class DMSListener implements FactoryBean{
private ConnectionFactory connectionFactory;
private String queueName;
private String queueUser;
#Required
public void setConnectionFactory(QueueConnectionFactory connectionFactory)
{
System.out.println("set connection");
this.connectionFactory = connectionFactory;
}
#Required
public void setQueueName(String queueName) {
System.out.println("set DMS listener queuename");
this.queueName = queueName;
}
#Required
public void setQueueUser(String queueUser) {
System.out.println("set DMS listener queueuser");
this.queueUser = queueUser;
}
public Object getObject() throws Exception
{
QueueConnectionFactory qconn = (QueueConnectionFactory)this.connectionFactory;
AQjmsSession session = (AQjmsSession)qconn.createQueueConnection("score", "score").createQueueSession(true, 0);
return session.getQueue(this.queueUser, this.queueName);
}
public Class getObjectType()
{
return Queue.class;
}
public boolean isSingleton() {
return false;
}
}
Here is how i configured it.
<bean id="messageDMSListener" class="com.test.DMSTextListener">
</bean>
<bean id="testDMS" class="com.test.DMSListener">
<property name="connectionFactory" ref="aqConnectionFactoryRspm"/>
<property name="queueName" value="RSPM_PEND_REQ_Q_DMS"/>
<property name="queueUser" value="score"/>
</bean>
<bean id="jmsDMSContainer" class="com.test.DMSCustomMessageListener">
<property name="connectionFactory" ref="aqConnectionFactoryRspm"/>
<property name="destination" ref="testDMS"/>
<property name="messageListener" ref="messageDMSListener" />
<property name="sessionTransacted" value="true"/>
<property name="errorHandler" ref="listenerErrorHandler"/>
</bean>
In my queue table/view (AQ$RSPM_PEND_REQ_Q_DMS) i am gettting expiration reason as 'MAX_RETRY_EXCEEDED' . I have configured it to 10.
What can be the possible reason ? Kindly help.
Oracle Database Queue System differs from the common JMS system so does the way talk to it.
I assume you can talk with your queue but the messages do not disappear form the queue but expire instead. If that's the case then I think your queue is configured as "multi user" type. In such occasion it won't disappear until all recipients get the message and the queue owner is also the recipient. As you just want to pass the message to another system reconfigure your queue to single user and the message disappear immediately after reading.
As a matter of fact you don't need your java bean either. You can do the job by configuring queue propagation(and the corresponding job) straight in the database without any external objects (example skeleton below is not complete solution):
BEGIN
DBMS_AQADM.SCHEDULE_PROPAGATION (
queue_name => 'init_queue',
destination => NULL,
start_time => SYSDATE,
duration => NULL,
next_time => NULL,
latency => 60,
destination_queue => 'dest_queue');
END;
In my (limited) experience with rabbit-mq, if you create a new listener for a queue that doesn't exist yet, the queue is automatically created. I'm trying to use the Spring AMQP project with rabbit-mq to set up a listener, and I'm getting an error instead. This is my xml config:
<rabbit:connection-factory id="rabbitConnectionFactory" host="172.16.45.1" username="test" password="password" />
<rabbit:listener-container connection-factory="rabbitConnectionFactory" >
<rabbit:listener ref="testQueueListener" queue-names="test" />
</rabbit:listener-container>
<bean id="testQueueListener" class="com.levelsbeyond.rabbit.TestQueueListener">
</bean>
I get this in my RabbitMq logs:
=ERROR REPORT==== 3-May-2013::23:17:24 ===
connection <0.1652.0>, channel 1 - soft error:
{amqp_error,not_found,"no queue 'test' in vhost '/'",'queue.declare'}
And a similar error from AMQP:
2013-05-03 23:17:24,059 ERROR [org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer] (SimpleAsyncTaskExecutor-1) - Consumer received fatal exception on startup
org.springframework.amqp.rabbit.listener.FatalListenerStartupException: Cannot prepare queue for listener. Either the queue doesn't exist or the broker will not allow us to use it.
It would seem from the stack trace that the queue is getting created in a "passive" mode- Can anyone point out how I would create the queue not using the passive mode so I don't see this error? Or am I missing something else?
Older thread, but this still shows up pretty high on Google, so here's some newer information:
2015-11-23
Since Spring 4.2.x with Spring-Messaging and Spring-Amqp 1.4.5.RELEASE and Spring-Rabbit 1.4.5.RELEASE, declaring exchanges, queues and bindings has become very simple through an #Configuration class some annotations:
#EnableRabbit
#Configuration
#PropertySources({
#PropertySource("classpath:rabbitMq.properties")
})
public class RabbitMqConfig {
private static final Logger logger = LoggerFactory.getLogger(RabbitMqConfig.class);
#Value("${rabbitmq.host}")
private String host;
#Value("${rabbitmq.port:5672}")
private int port;
#Value("${rabbitmq.username}")
private String username;
#Value("${rabbitmq.password}")
private String password;
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(host, port);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
logger.info("Creating connection factory with: " + username + "#" + host + ":" + port);
return connectionFactory;
}
/**
* Required for executing adminstration functions against an AMQP Broker
*/
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory());
}
/**
* This queue will be declared. This means it will be created if it does not exist. Once declared, you can do something
* like the following:
*
* #RabbitListener(queues = "#{#myDurableQueue}")
* #Transactional
* public void handleMyDurableQueueMessage(CustomDurableDto myMessage) {
* // Anything you want! This can also return a non-void which will queue it back in to the queue attached to #RabbitListener
* }
*/
#Bean
public Queue myDurableQueue() {
// This queue has the following properties:
// name: my_durable
// durable: true
// exclusive: false
// auto_delete: false
return new Queue("my_durable", true, false, false);
}
/**
* The following is a complete declaration of an exchange, a queue and a exchange-queue binding
*/
#Bean
public TopicExchange emailExchange() {
return new TopicExchange("email", true, false);
}
#Bean
public Queue inboundEmailQueue() {
return new Queue("email_inbound", true, false, false);
}
#Bean
public Binding inboundEmailExchangeBinding() {
// Important part is the routing key -- this is just an example
return BindingBuilder.bind(inboundEmailQueue()).to(emailExchange()).with("from.*");
}
}
Some sources and documentation to help:
Spring annotations
Declaring/configuration RabbitMQ for queue/binding support
Direct exchange binding (for when routing key doesn't matter)
Note: Looks like I missed a version -- starting with Spring AMQP 1.5, things get even easier as you can declare the full binding right at the listener!
What seemed to resolve my issue was adding an admin. Here is my xml:
<rabbit:listener-container connection-factory="rabbitConnectionFactory" >
<rabbit:listener ref="orderQueueListener" queues="test.order" />
</rabbit:listener-container>
<rabbit:queue name="test.order"></rabbit:queue>
<rabbit:admin id="amqpAdmin" connection-factory="rabbitConnectionFactory"/>
<bean id="orderQueueListener" class="com.levelsbeyond.rabbit.OrderQueueListener">
</bean>
As of Spring Boot 2.1.6 and Spring AMQP 2.1.7 you can create queues during startup if they don't exists with this:
#Component
public class QueueConfig {
private AmqpAdmin amqpAdmin;
public QueueConfig(AmqpAdmin amqpAdmin) {
this.amqpAdmin = amqpAdmin;
}
#PostConstruct
public void createQueues() {
amqpAdmin.declareQueue(new Queue("queue_one", true));
amqpAdmin.declareQueue(new Queue("queue_two", true));
}
}
Can you add this after your connection tag, but before the listener:
<rabbit:queue name="test" auto-delete="true" durable="false" passive="false" />
Unfortunately, according to the XSD schema, the passive attribute (listed above) is not valid. However, in every queue_declare implementation I've seen, passive has been a valid queue_declare parameter. I'm curious to see whether that will work or whether they plan to support it in future.
Here is the full list of options for a queue declaration:
http://www.rabbitmq.com/amqp-0-9-1-reference.html#class.queue
And here is the full XSD for the spring rabbit schema (with comments included):
http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd
If previously you were using spring-rabbit version <1.6 and now you upgrade to that version or after and you find your queues arent getting created then most likely you could be missing a RabbitAdmin bean. Previous versions dont seem to need that in the context but 1.6 and after do
I'm trying to get Spring Data's web pagination working. It's described here:
http://static.springsource.org/spring-data/data-jpa/docs/current/reference/html/repositories.html#web-pagination
Here's my Java (Spring Web MVC #Controller handler method):
#RequestMapping(value = "/list", method = RequestMethod.GET)
public String list(
#PageableDefaults(value = 50, pageNumber = 0) Pageable pageable,
Model model) {
log.debug("Params: pageNumber={}, pageSize={}",
pageable.getPageNumber(), pageable.getPageSize());
...
}
And here's my Spring configuration:
<mvc:annotation-driven>
<mvc:argument-resolvers>
<bean class="org.springframework.data.web.PageableArgumentResolver" />
</mvc:argument-resolvers>
</mvc:annotation-driven>
(It appears that the configuration above is the way to do this now; the configuration approach described in the link is deprecated.)
When I actually try to control the pagination using the page and page.size parameters, the latter works just fine, but the former doesn't. For example, if I hit
http://localhost:8080/myapp/list?page=14&page.size=42
the log output is
Params: pageNumber=0, pageSize=42
So I know that the argument resolver is kicking in, but not sure why it's not resolving the page number. I've tried a bunch of other param names (e.g. page.number, pageNumber, page.num, etc.) and none of them work.
Is this working for anybody else?
the page parameter is actually a bit non-intuitive - page.page and not page, changing to page.page should get things to work.
Looking into the PageableArgumentResolver I found that prefix and separator are configurable, so you can configure the class not to have it.
public class PageableArgumentResolver implements WebArgumentResolver {
private static final Pageable DEFAULT_PAGE_REQUEST = new PageRequest(0, 10);
private static final String DEFAULT_PREFIX = "page";
private static final String DEFAULT_SEPARATOR = ".";
in my #Configuration class I did something different to achieve using page, size,sort and sortDir as default.
#Override
public void addArgumentResolvers(List<HandlerMethodArgumentResolver> argumentResolvers) {
PageableArgumentResolver resolver = new PageableArgumentResolver();
resolver.setPrefix("");
resolver.setSeparator("");
argumentResolvers.add(new ServletWebArgumentResolverAdapter(resolver));
}
Now this works
http://:8080/myapp/list?page=14&size=42
You can override parameters if you want by doing:
<mvc:annotation-driven>
<mvc:argument-resolvers>
<bean class="org.springframework.data.web.PageableHandlerMethodArgumentResolver">
<property name="oneIndexedParameters" value="true"></property>
<property name="pageParameterName" value="page"></property>
<property name="sizeParameterName" value="size"></property>
</bean>
</mvc:argument-resolvers>
</mvc:annotation-driven>
I'd like to convert this SimpleFormController to use the annotation support introduced in Spring MVC 2.5
Java
public class PriceIncreaseFormController extends SimpleFormController {
ProductManager productManager = new ProductManager();
#Override
public ModelAndView onSubmit(Object command)
throws ServletException {
int increase = ((PriceIncrease) command).getPercentage();
productManager.increasePrice(increase);
return new ModelAndView(new RedirectView(getSuccessView()));
}
#Override
protected Object formBackingObject(HttpServletRequest request)
throws ServletException {
PriceIncrease priceIncrease = new PriceIncrease();
priceIncrease.setPercentage(20);
return priceIncrease;
}
}
Spring Config
<!-- Include basic annotation support -->
<context:annotation-config/>
<!-- Comma-separated list of packages to search for annotated controllers. Append '.*' to search all sub-packages -->
<context:component-scan base-package="springapp.web"/>
<!-- Enables use of annotations on controller methods to map URLs to methods and request params to method arguments -->
<bean class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter"/>
<bean name="/priceincrease.htm" class="springapp.web.PriceIncreaseFormController">
<property name="sessionForm" value="true"/>
<property name="commandName" value="priceIncrease"/>
<property name="commandClass" value="springapp.service.PriceIncrease"/>
<property name="validator">
<bean class="springapp.service.PriceIncreaseValidator"/>
</property>
<property name="formView" value="priceincrease"/>
<property name="successView" value="hello.htm"/>
<property name="productManager" ref="productManager"/>
</bean>
Basically, I'd like to replace all the XML configuration for the /priceincrease.htm bean with annotations within the Java class. Is this possible, and if so, what are the corresponding annotations that I should use?
Thanks,
Don
It'll look something like the following, although whether it works or not exactly as is will depend a bit on your configuration (view resolver, etc). I should also note that there are about eight billion valid ways to write this thing. See the Spring documentation, 13.11.4 "Supported handler method arguments and return types" for an overview of the insanity. Also note that you can autowire the validator
#Controller
#RequestMapping("/priceincrease.htm")
public class PriceIncreaseFormController {
ProductManager productManager;
#Autowired
public PriceIncreaseFormController(ProductManager productManager) {
this.productManager = productManager;
}
// note: this method does not have to be called onSubmit
#RequestMapping(method = RequestMethod.POST)
public String onSubmit(#ModelAttribute("priceIncrease") PriceIncrease priceIncrease, BindingResult result, SessionStatus status {
new PriceIncreaseValidator().validate(priceIncrease, result);
if (result.hasErrors()) {
return "priceincrease";
}
else {
int increase = priceIncrease.getPercentage();
productManager.increasePrice(increase);
status.setComplete();
return "redirect:hello.htm";
}
}
// note: this method does not have to be called setupForm
#RequestMapping(method = RequestMethod.GET)
public String setupForm(Model model) {
PriceIncrease priceIncrease = new PriceIncrease();
priceIncrease.setPercentage(20);
model.addAttribute("priceIncrease", priceIncrease);
return "priceincrease";
}
}
Someone completed this project with a recent MVC and it's on github, so you can see how all the classes are changed compared to Spring's tutorial.
Link: PriceIncreaseFormController