This is my spring integration case:
1) ftp adapter to download a pdf file
2) pdf2TextTransformer transform pdf to text by using pdfbox
3) pdfText2CsvTransformer transform text to csv
I add an enricher in the chain, however, seems header doesn't propagate to myErrorChannel, anyone can tell me why?
Here is my debug log, header file_originalFile doesn't propagate to myErrorChannel
][Headers={file_originalFile=D:\projects\DMTP\ftp\local2\6000047256 - Copy - Copy.pdf, file_name=6000047256 - Copy - Copy.pdf, id=a091fe5e-83f3-e48c-4be5-927849dbf31a, timestamp=1501472718172}]
11:45:18.180 DEBUG [task-scheduler-2][org.springframework.integration.channel.PublishSubscribeChannel] preSend on channel 'errorChannel', message: [Payload MessageTransformationException content=org.springframework.integration.transformer.MessageTransformationException: org.springframework.messaging.MessageHandlingException: java.lang.StringIndexOutOfBoundsException: String index out of range: -3][Headers={id=251c77f0-30e8-e18c-6980-ebfb289f79a4, timestamp=1501472718180}]
11:45:18.180 DEBUG [task-scheduler-2][org.springframework.integration.handler.ServiceActivatingHandler] ServiceActivator for [org.springframework.integration.handler.MethodInvokingMessageProcessor#56eacc77] received message: [Payload MessageTransformationException content=org.springframework.integration.transformer.MessageTransformationException: org.springframework.messaging.MessageHandlingException: java.lang.StringIndexOutOfBoundsException: String index out of range: -3][Headers={id=251c77f0-30e8-e18c-6980-ebfb289f79a4, timestamp=1501472718180}]
file_originalFile:null
MessageHeaders: {id=251c77f0-30e8-e18c-6980-ebfb289f79a4, timestamp=1501472718180}
MessagePayload: org.springframework.integration.transformer.MessageTransformationException: org.springframework.messaging.MessageHandlingException: java.lang.StringIndexOutOfBoundsException: String index out of range: -3
11:45:18.182 DEBUG [task-scheduler-2][org.springframework.integration.handler.ServiceActivatingHandler] handler 'ServiceActivator for [org.springframework.integration.handler.MethodInvokingMessageProcessor#56eacc77]' produced no reply for request Message: [Payload MessageTransformationException content=org.springframework.integration.transformer.MessageTransformationException: org.springframework.messaging.MessageHandlingException: java.lang.StringIndexOutOfBoundsException: String index out of range: -3][Headers={id=251c77f0-30e8-e18c-6980-ebfb289f79a4, timestamp=1501472718180}]
here is my configuration:
<bean id="ftpClientFactory" class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host" value="${host}"/>
<property name="port" value="${availableServerPort}"/>
<property name="username" value="${userid}"/>
<property name="password" value="${password}"/>
</bean>
<bean id="cachingSessionFactory" class="org.springframework.integration.file.remote.session.CachingSessionFactory">
<constructor-arg ref="ftpClientFactory"/>
</bean>
<int:channel id="ftpChannelIn">
<int:queue capacity="1"/>
</int:channel>
<int-ftp:inbound-channel-adapter id="ftpInbound"
channel="ftpChannelIn"
session-factory="cachingSessionFactory"
filename-pattern="*.pdf"
auto-create-local-directory="true"
delete-remote-files="true"
remote-directory="/in2"
local-directory="D:/projects/DMTP/ftp/local2">
<!--<int:poller fixed-rate="1000"/>-->
<int:poller fixed-delay="1"/>
</int-ftp:inbound-channel-adapter>
<int:chain input-channel="ftpChannelIn">
<int:header-enricher>
<int:header name="foo" value="bar"/>
</int:header-enricher>
<file:file-to-bytes-transformer/>
<int:transformer ref="pdf2TextTransformer"/>
<int:transformer ref="pdfText2CsvTransformer"/>
<int-ftp:outbound-channel-adapter
remote-directory="/out"
session-factory="cachingSessionFactory"
remote-filename-generator="filenameGenerator"/>
</int:chain>
<int:channel id="ftpChannelOut">
</int:channel>
<bean id="filenameGenerator" class="file.DatedDirectoryFactory "/>
<int:poller default="true" fixed-delay="50"/>
<int:channel id="myErrorChannel"/>
<int:service-activator input-channel="errorChannel" ref="errorLogger"/>
The ErrorMessage is built in the Poller and it is based on the exception:
try {
getMessagingTemplate().send(errorChannel, getErrorMessageStrategy().buildErrorMessage(t, null));
sent = true;
}
There is no any info about original headers.
Your headers, in particular <int:header name="foo" value="bar"/> is presented on the downstream requestMessage. Typically, when exception happens in the Integration Component, the MessagingException is thrown.
In your case it is MessageTransformationException. That kind of exception has failedMessage property. This is exactly your requestMessage what is "guilty" in the error. And, as you understand, already here you can get access to you headers - MessagingException.getFailedMessage().getHeaders()get("foo"). And, of course, file_originalFile will be available here as well.
If your transformers are returning Message<?> then they are responsible for copying the headers; the framework assumes that. If your transformers are POJO, and return just the transformed payload (the recommended programming model), the framework will take care of propagating the headers.
Turn on debug logging to examine the message flow; if you still think it "doesn't work", explain in more detail and edit the question to show the log.
BTW, "doesn't work" is not an adequate description of a problem.
Related
Spring integration tcp gateway can be setup as follows:
<!-- Server side -->
<int-ip:tcp-connection-factory id="crLfServer"
type="server"
port="${availableServerSocket}"/>
<int-ip:tcp-inbound-gateway id="gatewayCrLf"
connection-factory="crLfServer"
request-channel="serverBytes2StringChannel"
error-channel="errorChannel"
reply-timeout="10000" />
<int:channel id="toSA" />
<int:service-activator input-channel="toSA"
ref="echoService"
method="test"/>
<bean id="echoService"
class="org.springframework.integration.samples.tcpclientserver.EchoService" />
<int:object-to-string-transformer id="serverBytes2String"
input-channel="serverBytes2StringChannel"
output-channel="toSA"/>
<int:transformer id="errorHandler"
input-channel="errorChannel"
expression="Error processing payload"/>
Notice the reply-timeout which is set as 10 seconds.
Does it mean that the TCP server will call the service and can wait for a maximum of 10 seconds? If the service does not reply within 10 seconds, Does the TCP server will send the message to errorChannel which in turn sends the client error message "Error processing payload"?
When I tested the TCP Server with a service that takes 20 seconds, client is taking 20 seconds to get the response. I am not seeing error message.
Can you please help in understanding the reply-timeout in TCP inbound-gateway?
Thanks
UPDATE:
Thanks for Artem to help out with this issue.
Best way to solve this problem is with the following config:
<beans>
<int-ip:tcp-connection-factory id="crLfServer" type="server" port="${availableServerSocket}"/>
<int-ip:tcp-inbound-gateway id="gatewayCrLf" connection-factory="crLfServer" request-channel="requestChannel" error-channel="errorChannel" reply-timeout="5000" />
<int:service-activator input-channel="requestChannel" ref="gateway" requires-reply="true"/>
<int:gateway id="gateway" default-request-channel="timeoutChannel" default-reply-timeout="5000" />
<int:object-to-string-transformer id="serverBytes2String" input-channel="timeoutChannel" output-channel="serviceChannel"/>
<int:channel id="timeoutChannel">
<int:dispatcher task-executor="executor"/>
</int:channel>
<bean id="executor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="10" />
<property name="queueCapacity" value="25" />
</bean>
<int:service-activator input-channel="serviceChannel" ref="echoService" method="test"/>
<bean id="echoService" class="org.springframework.integration.samples.tcpclientserver.EchoService" />
<int:transformer id="errorHandler" input-channel="errorChannel" expression="payload.failedMessage.payload + ' errorHandleMsg: may be timeout error'"/>
</beans>
Thanks
Well, actually we should on that attribute a description like we have in other similar places, e.g. HTTP Inbound Gateway:
<xsd:attribute name="reply-timeout" type="xsd:string">
<xsd:annotation>
<xsd:documentation><![CDATA[
Used to set the receiveTimeout on the underlying MessagingTemplate instance
(org.springframework.integration.core.MessagingTemplate) for receiving messages
from the reply channel. If not specified this property will default to "1000"
(1 second).
]]></xsd:documentation>
</xsd:annotation>
</xsd:attribute>
That timeout means how much to wait for reply from downstream flow. But! That is possible if you flow is shifted to another thread somewhere. Otherwise everything is performed in the caller's Thread and therefore the wait time isn't deterministic.
Anyway we return null there after timeout without reply. And it is reflected in the TcpInboundGateway:
Message<?> reply = this.sendAndReceiveMessage(message);
if (reply == null) {
if (logger.isDebugEnabled()) {
logger.debug("null reply received for " + message + " nothing to send");
}
return false;
}
We can reconsider a logic in the TcpInboundGateway for :
if (reply == null && this.errorOnTimeout) {
if (object instanceof Message) {
error = new MessageTimeoutException((Message<?>) object, "No reply received within timeout");
}
else {
error = new MessageTimeoutException("No reply received within timeout");
}
}
But seems for me it really would be better on to rely on the timeout from the client.
UPDATE
I think we can overcome the limitation and meet you requirements with the midflow <gateway>:
<gateway id="gateway" default-request-channel="timeoutChannel" default-reply-timeout="10000"/>
<channel id="timeoutChannel">
<dispatcher task-executor="executor"/>
</channel>
<service-activator input-channel="requestChannel"
ref="gateway"
requires-reply="true"/>
So, the <service-activator> calls <gateway> and waits for reply from there. Requiring the last one, of course, to end up with the ReplyRequiredException, which you can convert into desired MessageTimeoutException in your error flow on the error-channel="errorChannel".
The timeoutChannel is an executor one, making our default-reply-timeout="10000" very useful because we shift a message on the gateway into separate thread immediately and move right from there into reply waiting process wrapped with that timeout on the CountDonwLatch.
Hope that is clear.
Configuration details :
<int:publish-subscribe-channel id="toKafka"/>
<int:publish-subscribe-channel id="sendMessageToKafkaChannel"/>
<int:service-activator input-channel="toKafka" output-channel="sendMessageToKafkaChannel" order="1" ref="conditionalProducerService" method="producerCircuitBreaker">
<int:request-handler-advice-chain>
<ref bean="circuitBreakerAdvice" />
</int:request-handler-advice-chain>
</int:service-activator>
<int-kafka:outbound-channel-adapter id="kafkaOutboundChannelAdapter" kafka-producer-context-ref="producerContext"
auto-startup="true" channel="toKafka" message-key="kafka_messageKey"/>
<bean id="circuitBreakerAdvice" class="org.springframework.integration.handler.advice.RequestHandlerCircuitBreakerAdvice">
<property name="threshold" value="2"/>
<property name="halfOpenAfter" value="15000" />
</bean>
public Message<?> producerCircuitBreaker(Message<?> payload) {
throw new RuntimeException("foo Pro");
}
for(int i=0;i<4;i++){
toKafka.send(MessageBuilder
.withPayload(messageVO.getMessageContentVO())
.setHeader(KafkaHeaders.TOPIC, topic)
.setHeader(KafkaHeaders.PARTITION_ID,Integer.parseInt(messageVO.getPartition())).
build());
APPLOGGER.info("sending message");
}
Expecting to get the process to fail 2 times with exception and then "circuit breaker open" exception but it is simply stopping after throwing the below exception in the console.
Also how can we configure error-channel here.
https://gist.github.com/anonymous/67aae50e548c78470cd0
updated config:
<int:service-activator input-channel="toKafka" ref="gw">
<int:request-handler-advice-chain> <ref bean="circuitBreakerAdvice"/>
</int:request-handler-advice-chain>
</int:service-activator>
<int:channel id="failedChannel1" />
<int:gateway id="gw" default-request-channel="toKafka" default-reply-timeout="0" error-channel="failedChannel1" />
<int:chain input-channel="failedChannel1">
<int:transformer expression="'failed:'+payload.failedMessage.payload+ ' with a' +payload.cause.message" />
<int-stream:stderr-channel-adapter append-newline="true"/>
</int:chain>
getting below exception.
failed:TestVo[data=sample message]] with Cannot process message.
https://gist.github.com/anonymous/921be7691c41d125dc84
however it is working with same message otherwise.(message content changed intentionally)
Also tried putting invalid value for the producer context: eg. broker- list/value-class-type as invalid class type than expected as below.
getting below error but expecting to get CB to come into picture and message should flow to the error channel.
in case of value-class-type : CB not invoked however message flowing to the error channel but there are many message are coming for 1 message published.
failed:TestVo [data={tes message}}] with No converter found capable of converting from type xx.xxx.vo.TestVo to type java.lang.String
these is occuring in the console many times.
in case of broker-list : it is simply throwing exception in the console.
https://gist.github.com/anonymous/6ece517fb5e82ac73492
Expected : CB to get invoked and message flow to the error channel in all cases.
<int-kafka:outbound-channel-adapter id="kafkaOutboundChannelAdapter" kafka-producer-context-ref="producerContext"
auto-startup="true" channel="toKafka" message-key="kafka_messageKey"/>
<int-kafka:producer-context id="producerContext" producer-properties="producerProperties">
<int-kafka:producer-configurations>
<int-kafka:producer-configuration
broker-list="1.2.3:9092" topic="headers['topic']" key-class-type="java.lang.String"
value-class-type="java.lang.String"
value-encoder="kafkaEncoder" key-encoder="kafkaKeyEncoder"
compression-type="none" />
</int-kafka:producer-configurations>
</int-kafka:producer-context>
With that code, you need try {...} around the send().
The first two attempts will catch your RuntimeException; the next will catch the circuit breaker exception.
Use a Messaging Gateway with an error channel instead of sending to the channel directly.
EDIT
This code...
<int:service-activator input-channel="toKafka" ref="gw">
<int:request-handler-advice-chain> <ref bean="circuitBreakerAdvice"/>
</int:request-handler-advice-chain>
</int:service-activator>
<int:gateway id="gw" default-request-channel="toKafka" default-reply-timeout="0" error-channel="failedChannel1" />
When you send a message to toKafka, the gateway will be invoked which will send the message to toKafka in a loop.
It will cause a stack overflow.
I would like to apply an Interceptor on the reply-channel of an http:inbound-gateway to save some event related data to a table. The flow continues in a chain which then goes to a header-value-router. As an example let's take a service-activator at the end of this flow, where the output-channel is not specified. In this case, the replyChannel header holds a TemporaryReplyChannel object (anonymous reply channel) instead of the gateway's reply-channel. This way the Interceptor is never called.
Is there a way to "force" the usage of the specified reply-channel? The Spring document states that
by defining a default-reply-channel you can point to a channel of your choosing, which in this case would be a publish-subscribe-channel. The Gateway would create a bridge from it to the temporary, anonymous reply channel that is stored in the header.
I've tried using a publish-subscribe-channel as reply-channel, but it didn't make any difference. Maybe I misunderstood the article...
Inside my chain I've also experimented with a header-enricher. I wanted to overwrite the value of the replyChannel with the id of the channel I want to intercept (submit.reply.channel). While debugging I was able to see "submit.reply.channel" in the header, but then I got an exception java.lang.NoClassDefFoundError: org/springframework/transaction/interceptor/NoRollbackRuleAttribute and stopped trying ;-)
Code snippets
<int-http:inbound-gateway id="submitHttpGateway"
request-channel="submit.request.channel" reply-channel="submit.reply.channel" path="/submit" supported-methods="GET">
<int-http:header name="requestAttributes" expression="#requestAttributes" />
<int-http:header name="requestParametersMap" expression="#requestParams" />
</int-http:inbound-gateway>
<int:channel id="submit.request.channel" />
<int:publish-subscribe-channel id="submit.reply.channel">
<int:interceptors>
<int:ref bean="replyChannelInterceptor" />
</int:interceptors>
</int:publish-subscribe-channel>
Thanks in advance for your help!
The only "easy" way is to explicitly send the reply via the output-channel on the last endpoint.
In fact, all that happens when you send to a declared channel is the reply channel is simply bridged to the replyChannel header.
You could do it by saving off the replyChannel header in another header, set the replyChannel header to some other channel (which you can intercept); then restore the replyChannel header to the saved-off channel before the reply is returned to the gateway.
EDIT:
Sample config...
<int:channel id="in" />
<int:header-enricher input-channel="in" output-channel="next">
<int:header name="origReplyChannel" expression="headers['replyChannel']"/>
<int:reply-channel ref="myReplies" overwrite="true" />
</int:header-enricher>
<int:router input-channel="next" expression="payload.equals('foo')">
<int:mapping value="true" channel="channel1" />
<int:mapping value="false" channel="channel2" />
</int:router>
<int:transformer input-channel="channel1" expression="payload.toUpperCase()" />
<int:transformer input-channel="channel2" expression="payload + payload" />
<int:channel id="myReplies" />
<!-- restore the reply channel -->
<int:header-enricher input-channel="myReplies" output-channel="tapped">
<int:reply-channel expression="headers['origReplyChannel']" overwrite="true" />
</int:header-enricher>
<int:channel id="tapped">
<int:interceptors>
<int:wire-tap channel="loggingChannel" />
</int:interceptors>
</int:channel>
<int:logging-channel-adapter id="loggingChannel" log-full-message="true" logger-name="tapInbound"
level="INFO" />
<!-- route reply -->
<int:bridge id="bridgeToNowhere" input-channel="tapped" />
Test:
MessageChannel channel = context.getBean("in", MessageChannel.class);
MessagingTemplate template = new MessagingTemplate(channel);
String reply = template.convertSendAndReceive("foo", String.class);
System.out.println(reply);
reply = template.convertSendAndReceive("bar", String.class);
System.out.println(reply); }
Result:
09:36:30.224 INFO [main][tapInbound] GenericMessage [payload=FOO, headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#fba92d3, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#fba92d3, id=326a610f-80c6-5b74-0158-e3644b732aab, origReplyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#fba92d3, timestamp=1442496990223}]
FOO
09:36:30.227 INFO [main][tapInbound] GenericMessage [payload=barbar, headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#662b4c69, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#662b4c69, id=d161917c-ca73-a5a9-d0f1-d7a4346a459e, origReplyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#662b4c69, timestamp=1442496990227}]
barbar
I have a service bundle where I want to compress a response by using CXF GZIPFeature. The bundle is deployed on JBoss Fuse(jboss-fuse-6.1.0.redhat-379). Camel route configuraion is:
<cxf:bus id="cxf" name ="cxf">
<cxf:features>
<bean class="org.apache.cxf.transport.common.gzip.GZIPFeature">
<property name="threshold">
<value>1</value>
</property>
</bean>
</cxf:features>
</cxf:bus>
<camel:camelContext ...>
<camel:route id="test-server">
<camel:to uri="cxfbean:servicebeans?bus=#cxf&providers=#providers" />
</camel:route>
</camel:camelContext>
But this throws classcast exception:
java.lang.ClassCastException: org.apache.cxf.transport.common.gzip.GZIPOutInterceptor$GZipThresholdOutputStream cannot be cast to org.apache.cxf.io.CachedOutputStream
at org.apache.camel.component.cxf.transport.CamelDestination$CamelOutputStream.commitOutputMessage(CamelDestination.java:284)[204:org.apache.camel.camel-cxf-transport:2.12.0.redhat-610379]
at org.apache.camel.component.cxf.transport.CamelDestination$CamelOutputStream.doClose(CamelDestination.java:296)[204:org.apache.camel.camel-cxf-transport:2.12.0.redhat-610379]
at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:220)[164:org.apache.cxf.cxf-api:2.7.0.redhat-610379]
at java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:241)[:1.7.0_67]
at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77)[164:org.apache.cxf.cxf-api:2.7.0.redhat-610379]
at org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:102)[164:org.apache.cxf.cxf-api:2.7.0.redhat-610379]
at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)[164:org.apache.cxf.cxf-api:2.7.0.redhat-610379]
at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)[164:org.apache.cxf.cxf-api:2.7.0.redhat-610379]
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272)[164:org.apache.cxf.cxf-api:2.7.0.redhat-610379]
Is there a legal way to use CXF GZIPFeature in the route? I don't want to use Camel's DataFormat to gzip.
I'm using servicemix-camel-cxf and had nearly the same problem. In the end I decided to simply add this in route:
<marshal>
<gzip/>
</marshal>
Check it out - maybe it will help you too.
But in that case you have to set Content-Type: gzip header.
Try to add
<setHeader headerName="Content-Type">
<constant>gzip</constant></setHeader>
However for me it didn't help, so I added header in java code. =)
I'm using camel and cxf component to get some data from web-service. In some case web-service returns standard soap:fault.
I have the next steps:
<camel:route id="someId">
<camel:onException useOriginalMessage="false">
<camel:exception>java.lang.Exception</camel:exception>
<camel:handled>
<camel:constant>false</camel:constant>
</camel:handled>
<camel:process ref="defaultNaoIntegrationErrorHandler" />
<camel:to uri="ref:naointegration.checkAvailability.jms.error.queue" />
</camel:onException>
<camel:from uri="direct:naoCheckAvailabilityOut" />
<camel:marshal ref="soapjaxbSAP" />
<camel:to id="naoCheckAvailabilityEndpoint" uri="cxf:bean:naoCheckAvailabilityEndpoint" />
<camel:unmarshal ref="soapjaxbSAP" />
</camel:route>
where naoCheckAvailabilityEndpoint is:
<cxf:cxfEndpoint id="naoCheckAvailabilityEndpoint"
address="${naointegration.I011.CheckAvailability.soap.address}"
endpointName="s:checkAvailabilityEndpoint" serviceName="s:SOAPService"
xmlns:s="http://www.example.com/test">
<cxf:properties>
<entry key="dataFormat" value="MESSAGE" />
<entry key="setDefaultBus" value="true" />
</cxf:properties>
<cxf:outInterceptors>
<ref bean="logOutbound" />
</cxf:outInterceptors>
<cxf:inFaultInterceptors>
<ref bean="logOutbound" />
</cxf:inFaultInterceptors>
</cxf:cxfEndpoint>
If I'm getting normal soap message everything is ok. And when I'm getting soap foalt/http 500 I have just string message which contains soap message (xml) with soap fault.
Reading cxf and camel mail lists on like problems I understood cxf endpoint should throw exception if there soap foalt, exception of type org.apache.cxf.binding.soap.SoapFault, but I can not get it.
The goal is onException clause do handling of soap fault exception.
Any suggestions?
You are using the MESSAGE data format, which means the camel-cxf endpoint just pass the stream from the transport, it will not read the under layer message, so the camel-cxf endpoint cannot tell which message is normal soap message or soap fault message.
If you want to let the camel route deal with the soap fault, you need to use the PAYLOAD or POJO data format.