I have integration flow and I want to write the entity in the flow to my Gemfire cache between the steps but I am not figuring out how to do it.
#Bean
public IntegrationFlow myFlow(CacheEntityDAL cacheDAL,Transformer t,
MyFilter f) {
return IntegrationFlows.from("inChannel")
.transform(t) //returns the entity
//I want to put to my cacheDAL.put(entity) here
.filter(f)
.channel("outChannel")
.get();
}
Thanks
To write to the Gemfire cache, you need to use a CacheWritingMessageHandler from the Spring Integration Gemfire support.
However since this one is one-way, it is just for writing, there is no straight forward way to insert it in the middle of the flow. On the other you just would like to store and proceed downstream with the same payload. For this purpose I suggest to use a PublishSubscribeChannel with two subscribers: a mentioned CacheWritingMessageHandler and then rest of the flow. Something like this:
return IntegrationFlows.from("inChannel")
.transform(t) //returns the entity
.publishSubscribeChannel(c -> c
.subscribe(sf -> sf
.handle(new CacheWritingMessageHandler(gemfireRegion()))
.filter(f)
.channel("outChannel")
.get();
So, this way you send to the Gemfire and move to the main flow, where the next filter()
is going to be as a second subscriber which won't work until a success of the first one for Gemfire.
Related
I want to use Spring Integration to expose a simple web service that pushes incoming message into ActiveMQ and responds immediately. My go-to solution was MarshallingWebServiceInboundGateway connected to Jms.outboundAdapter with IntegrationFlow. Below the Gateway and IntegrationFlow snippets. Problem with this is Adapter does not provide response (duh) which Gateway expects. The response I get back from the service is empty 202, with delay of about 1500ms. This is caused by a reply timeout I see in TRACE logs:
"2020-04-14 17:17:50.101 TRACE 26524 --- [nio-8080-exec-6] o.s.integration.core.MessagingTemplate : Failed to receive message from channel 'org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#518ffd27' within timeout: 1000"
No hard exceptions anywhere. The other problem is I cannot generate the response myself. I can't add anything to IntegrationFlow after the .handle with Adapter.
Any other way I can try to fulfill the scenario?
How, if at all possible, can I generate and return response in situation there is no better approach?
Most likely the proper way would be to use Gateways on both ends, but this is not possible. I cannot wait with response until message in the queue gets consumed and processed.
'''
#Bean
public MarshallingWebServiceInboundGateway greetingWebServiceInboundGateway() {
MarshallingWebServiceInboundGateway inboundGateway = new MarshallingWebServiceInboundGateway(
jaxb2Marshaller()
);
inboundGateway.setRequestChannelName("greetingAsync.input");
inboundGateway.setLoggingEnabled(true);
return inboundGateway;
}
#Bean
public IntegrationFlow greetingAsync() {
return f -> f
.log(LoggingHandler.Level.INFO)
.handle(Jms.outboundAdapter(this.jmsConnectionFactory)
.configureJmsTemplate(c -> {
c.jmsMessageConverter(new MarshallingMessageConverter(jaxb2Marshaller()));
})
.destination(JmsConfig.HELLO_WORLD_QUEUE));
}
'''
The logic and assumptions are fully correct: you can't return after one-way handle() and similar to that Jms.outboundAdapter().
But your problem that you fully miss one of the first-class citizens in Spring Integration - a MessageChannel. It is important to understand that even in the flow like yours there are channels between endpoints (DSL methods) - implicit (DirectChannel), like in your case, or explicit: when you use a channel() in between and can place there any possible implementation: https://docs.spring.io/spring-integration/docs/5.3.0.M4/reference/html/dsl.html#java-dsl-channels
One of the crucial channel implementation is a PublishSubscribeChannel (a topic in JMS specification) when you can send the same message to several subscribed endpoints: https://docs.spring.io/spring-integration/docs/5.3.0.M4/reference/html/core.html#channel-implementations-publishsubscribechannel
In your case the fists subscriber should be your existing, one-way Jms.outboundAdapter(). And another something what is going to generate response and reply it into a replyChannel header.
For this purpose Java DSL provides a nice hook via sub-flows configuration: https://docs.spring.io/spring-integration/docs/5.3.0.M4/reference/html/dsl.html#java-dsl-subflows
So, some sample of publish-subscriber could be like this:
.publishSubscribeChannel(c -> c
.subscribe(sf -> sf
.handle(Jms.outboundAdapter(this.jmsConnectionFactory))))
.handle([PRODUCE_RESPONSE])
In a project I use graphql-java and spring boot with a postgreSQL Database. Now I would like to use the subscription feature published in version 3.0.0. Unfortunately, the information about the application of the subsciption function is not very mature.
How is the approach to achieve real-time functionality using graphql-java with subscriptions?
As of recent graphql-java versions, subscriptions are fully supported. The DataFetcher for a subscription must return a org.reactivestreams.Publisher, and graphql-java will take care of mapping the query function over the results.
The feature is nicely documented and there's a complete example using web sockets available in the official repo.
If you have a reactive data source in place (e.g. Mongo with a reactive driver, or probably anything that R2DBC supports), you're all set. Just use #Tailable and Spring Data will already give you a Flux (which implements Publisher) and there's nothing else you need to do.
As for a more manual Spring specific implementation, I can't imagine it being too hard to use Spring's own event mechanism (a nice tutorial here as well) to underlie the Publisher.
Every time there's an incoming subscription, create and register a new listener with the application context: context.addApplicationListener(listener) that will publish to the correct Publisher. E.g. in the DataFetcher:
// Somehow create a publisher, probably using Spring's Reactor project. Or RxJava.
Publisher<ResultObject> publisher = ...;
//The listener reacts on application events and pushes new values through the publisher
ApplicationListener listener = createListener(publisher);
context.addApplicationListener(listener);
return publisher;
When the web socket disconnects or you somehow know the event stream is finished, you must make sure to remove the listener.
I haven't actually tried any of this, mind you, I'm just thinking aloud.
Another option is to use Reactor directly (with or without Spring WebFlux). There's a sample using Reactor and WebSocket (through GraphQL SPQR Spring Boot Starter) here.
You create a Publisher like this:
//This is really just a thread-safe wrapper around Map<String, Set<FluxSink<Task>>>
private final ConcurrentMultiRegistry<String, FluxSink<Task>> subscribers = new ConcurrentMultiRegistry<>();
#GraphQLSubscription
public Publisher<Task> taskStatusChanged(String taskId) {
return Flux.create(subscriber -> subscribers.add(taskId, subscriber.onDispose(() -> subscribers.remove(taskId, subscriber))), FluxSink.OverflowStrategy.LATEST);
}
And then push new values from elsewhere (probably a related mutation or a reactive storage) like this:
subscribers.get(taskId).forEach(subscriber -> subscriber.next(task));
E.g.
#GraphQLMutation
public Task updateTask(#GraphQLNonNull String taskId, #GraphQLNonNull Status status) {
Task task = repo.byId(taskId); //find the task
task.setStatus(status); //update the task
repo.save(task); //persist the task
//Notify all the subscribers following this task
subscribers.get(taskId).forEach(subscriber -> subscriber.next(task));
return task;
}
With SPQR Spring Starter, this is all that's needed to get you an Apollo-compatible subscription implementation.
I got the same issue where I was spiking on the lib to integrate with spring boot. I found graphql-java, however, it seems it only support 'subscription' on schema level, it is not perform any transnational support for this feature. Meaning you might need to implement it your self.
Please refer to https://github.com/graphql-java/graphql-java/blob/master/docs/schema.rst#subscription-support
For the record: here is another very nice, compact example that implements GraphQLs essential features queries, mutations and subscriptions: https://github.com/npalm/blog-graphql-spring-service
I had some legacy code that had SOAP services. Now I am building Rest API for some objects that may call one or more SOAP operation. I was looking into Spring Integration. From the docs
In addition to wiring together fine-grained components, Spring Integration provides a wide selection of channel adapters and gateways to communicate with external systems.
Above statement sounds enticing. I was writing rest microservice controller, Validation service, Rest request to SOAP request mapper and SOAP client. I some cases when there are multiple calls, there is even more code I had to write and I did write the code in many cases.
Spring Integration at high level looked like a framework oriented for Async messages. My problem is that the call need to be more or less a synchronous call and performance is critical. Had anyone used Spring integration for this problem and can you share your experiences.
To complement Artem's answer it's worth to note that if you're going to use one of Spring Integration DSLs (Java, Groovy or Scala) then (the synchronous) DirectChannel will be picked by default by Spring Integration to wire up the endpoints of your integration flow. This means that as long as your endpoints stay synchronous and you rely on default channels between them, the whole integration flow stay synchronous as well.
For instance (in Java DSL):
#Bean
public IntegrationFlow syncFlow() {
return IntegrationFlows
.from(/* get a REST message from microservice */)
// here the DirectChannel is used by default
.filter(/* validate (and filter out) incorrect messages */)
// here the DirectChannel is used by default too
.transform(/* map REST to SOAP */)
// guess what would be here?
.handle(/* send a message with SOAP client */)
.get();
}
This absolutely doesn't mean you tied up with synchronous flow forever. At any step you can go async or parallel. For example, if you decide to send SOAP messages in parallel all you need to do is to specify appropriate channel before SOAP client invocation:
#Bean
public IntegrationFlow syncFlow() {
// ... the same as above ...
.transform(/* map REST to SOAP */)
.channel(c -> c.executor(Executors.newCachedThreadPool())) // see (1)
.handle(/* send a message with SOAP client */)
.get();
}
(1) From this point on the downstream flow will be processed in parallel thanks to use of ExecutorChannel.
Note that message endpoints may also behave asynchronously depending on their logic.
I've used Spring Integration for building synchronous integration flows in my home and work projects and it's proven to be a very powerful yet flexible solution.
One of the first class citizens in Spring Integration is MessageChannel abstraction. The simplest, synchronous, and therefore direct method invocation is DirectChannel.
Not sure what makes you think that everything in Spring Integration is async. Actually it is always direct unless you tell to be async.
I'm currently trying to push some files using SI DSL SFTP features.
I'm not very fluent using this fwk, so I'm wondering if there's a better way to do what I'm trying to achieve.
It's kind of working like this, except when the files are copied, the rest call falls in a timeout state...
Bonus : Is there some good readings (book or online) about SI DSL ?
(except cafe si sample and reference..)
edit:
Does this SI flow follows the SI good practices ?
Why my rest call is ending with a timeout, although the files are correctly copied to the sftp server ?
Java config :
#Configuration
#EnableIntegration
#IntegrationComponentScan
public class IntegrationConfig {
//flow gateway
#MessagingGateway
public interface FlowGateway {
#Gateway(requestChannel = "SftpFlow.input")
Collection<String> flow(Collection<File> name);
}
//sftp flow bean
#Bean
public IntegrationFlow SftpFlow() {
return f -> f
.split()
.handle(Sftp.outboundAdapter(this.sftpSessionFactory(), FileExistsMode.REPLACE)
.useTemporaryFileName(false)
.remoteDirectory(sftpFolder));
}
//sftp session config
#Bean
public DefaultSftpSessionFactory sftpSessionFactory() {
System.out.println("Create session");
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(sftpHost);
factory.setPort(Integer.valueOf(sftpPort));
factory.setUser(sftpUser);
factory.setPassword(sftpPwd);
factory.setAllowUnknownKeys(true);
return factory;
}
}
A RestController class :
#Autowired
private FlowGateway flowGateway;
#RequestMapping("/testSftp")
public String testSftp() {
flowGateway.flow(Arrays.asList(file1, file2, file3);
}
Spring Integration Java DSL is fully based on the Spring Integration Core. So, all concepts, recipes, docs, samples etc. are applied here as well.
What is the question though? Let me guess: "why does it timeout and blocks?" That is obvious for me because I know where to read, but that might not be clear for others. Please, be more specific the next time: there are enough guys on SO who can close your question as "Unclear".
So, let's analyze what you have and why it isn't as you'd like.
Endpoints in Spring Integration can be one-way (Sftp.outboundAdapter()) or request-reply (Sftp.outboundGateway()). It's not surprise when it is one-way there is nothing to return and continue the flow or send reply, like in your case.
I'm sure you are not interested in the reply, otherwise you have used a different endpoint.
The process stops exactly after sending all the item from .split() and there is nothing to send back to the #Gateway as your code implies:
Collection<String> flow(Collection<File> name);
Having a non-void return type requires a reply from the downstream flow on the requestChannel.
See Messaging Gateway for more information.
Also consider to use
.channel(c -> c.executor(taskExecutor()))
after .split() to send your files to SFTP in parallel.
P.S. I'm not sure what you need to read else because so far everything is fine in your code, just only this nasty reply issue.
I'm using Spring Integration to develop my integration scenarios. When I have to write some logs to provide some information, I write this way:
#Bean
IntegrationFlow blacklist(BlacklistService service) {
return m -> m
.wireTap(f -> f.handle(t -> log.info("Adding email source address in blacklist...")))
.<MessageHandlingException, Blacklist>transform(p -> SourceBlacklist.of((Email) p.getFailedMessage().getHeaders().get(IntegrationConstants.MailSender.EMAIL)))
.wireTap(f -> f.handle(t -> log.info("Email source address added to blacklist.")))
.handle(service, "voidSave");
}
I'm using a wiretap with lambda and handle to log my messages. Is there a better way to write log with Spring Integration using Java DSL?
Thanks.
You always can just switch on the logging for the org.springframework.integration category.
From other side Spring Integration suggests logging in the integration flow as an adapter - <logging-channel-adapter>. So, what you need is just send message to the channel of that adapter. From the configuration perspective that looks like:
<wire-tap channel="logging" pattern="*"/>
<logging-channel-adapter id="logging"/>
The same we can configure with Java DSL, but we should rely on the target class - LoggingHandler:
#ServiceActivator(inputChannel = "logging")
#Bean
public loggingHandler() {
return new LoggingHandler();
}
...
.transform()
.wireTap("logging")
.handle();
Although I can see your point and we really could add something convenient to framework directly.
Feel free to raise a GH issue (https://github.com/spring-projects/spring-integration-java-dsl/issues) on the matter and we continue to discuss the feature there.
But right now your solution doesn't look bad, to be honest.
UPDATE
The request for the Framework on the matter: https://github.com/spring-projects/spring-integration-java-dsl/issues/70