We Keep Coding

Spring Cloud GCP is not receiving messages from subscription

I have a trained Chatbot in DialogFlow. I enabled logging for the chatbot and it logs to Google StackDriver. From Google StackDriver, using a log router, I have routed the logs to a Google Cloud Pub-Sub topic. Then, with a subscription, I have subscribed to this topic. The Pub-Sub works fine. I have setup Google Cloud SDK in my Ubuntu PC. Using gcloud commands, I can pull messages from the Pub-Sub subscription and it works.
Now, I implemented a Spring Boot application to subscribe to Pub-Sub. I followed this tutorial. Find the code below. The application does not receive messages from Pub-Sub. It is supposed to log the received message. Can anyone tell me what I may be missing here? How can I test if this actually works? Please comment if further information is needed.
ChatboardApplication.java
#SpringBootApplication
public class ChatboardApplication {
public static void main(String[] args) {
SpringApplication.run(ChatboardApplication.class, args);
}
private static final Log LOGGER = LogFactory.getLog(ChatboardApplication.class);
#Bean
public MessageChannel pubsubInputChannel() {
return new DirectChannel();
}
#Bean
public PubSubInboundChannelAdapter messageChannelAdapter(
#Qualifier("pubsubInputChannel") MessageChannel inputChannel,
PubSubTemplate pubSubTemplate) {
PubSubInboundChannelAdapter adapter =
new PubSubInboundChannelAdapter(pubSubTemplate, "agent-genie-sub");
adapter.setOutputChannel(inputChannel);
return adapter;
}
#ServiceActivator(inputChannel = "pubsubInputChannel")
public void messageReceiver(String payload) {
LOGGER.info("Message arrived! Payload: " + payload);
}
}
ChatboardProducerApplication
#SpringBootApplication
#RestController
public class ChatboardProducerApplication {
public static void main(String[] args) {
SpringApplication.run(ChatboardProducerApplication.class, args);
}
#MessagingGateway(defaultRequestChannel = "pubsubOutputChannel")
public interface PubsubOutboundGateway {
void sendToPubsub(String text);
}
#Bean
#ServiceActivator(inputChannel = "pubsubOutputChannel")
public MessageHandler messageSender(PubSubTemplate pubsubTemplate) {
return new PubSubMessageHandler(pubsubTemplate, "agent-genie-logs");
}
#Autowired
private PubsubOutboundGateway messagingGateway;
#PostMapping("/postMessage")
public RedirectView postMessage(#RequestParam("message") String message) {
this.messagingGateway.sendToPubsub(message);
return new RedirectView("/");
}
}
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.dialog</groupId>
<artifactId>chatboard</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>chatboard</name>
<description>PubSub subscriber and Dashboard backend</description>
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Hoxton.SR3</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-core</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<version>2.9.2</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>2.9.2</version>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
application.properties
spring.cloud.gcp.project-id=#########
spring.cloud.gcp.credentials.location=file:#######################
Curl command to test applications
curl --data "message=Hello world!" localhost:8080/postMessage
Edit - Error after several minutes
After some time from the applications started and running curl command, I get the following error.
com.google.api.gax.rpc.UnavailableException: io.grpc.StatusRuntimeException: UNAVAILABLE: Credentials failed to obtain metadata
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:69) ~[gax-1.54.0.jar:1.54.0]
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72) ~[gax-grpc-1.54.0.jar:1.54.0]
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60) ~[gax-grpc-1.54.0.jar:1.54.0]
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97) [gax-grpc-1.54.0.jar:1.54.0]
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68) [api-common-1.8.1.jar:na]
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1039) [guava-28.2-android.jar:na]
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) [guava-28.2-android.jar:na]
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1165) [guava-28.2-android.jar:na]
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:958) [guava-28.2-android.jar:na]
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:749) [guava-28.2-android.jar:na]
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:522) [grpc-stub-1.27.2.jar:1.27.2]
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:497) [grpc-stub-1.27.2.jar:1.27.2]
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426) [grpc-core-1.27.2.jar:1.27.2]
at io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66) [grpc-core-1.27.2.jar:1.27.2]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:689) [grpc-core-1.27.2.jar:1.27.2]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900(ClientCallImpl.java:577) [grpc-core-1.27.2.jar:1.27.2]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:751) [grpc-core-1.27.2.jar:1.27.2]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:740) [grpc-core-1.27.2.jar:1.27.2]
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) [grpc-core-1.27.2.jar:1.27.2]
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) [grpc-core-1.27.2.jar:1.27.2]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: io.grpc.StatusRuntimeException: UNAVAILABLE: Credentials failed to obtain metadata
at io.grpc.Status.asRuntimeException(Status.java:533) ~[grpc-api-1.27.2.jar:1.27.2]
... 16 common frames omitted
Caused by: java.io.IOException: Error getting access token for service account: 400 Bad Request
{"error":"invalid_grant","error_description":"Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim."}
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:444) ~[google-auth-library-oauth2-http-0.20.0.jar:na]
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:157) ~[google-auth-library-oauth2-http-0.20.0.jar:na]
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:145) ~[google-auth-library-oauth2-http-0.20.0.jar:na]
at com.google.auth.oauth2.ServiceAccountCredentials.getRequestMetadata(ServiceAccountCredentials.java:603) ~[google-auth-library-oauth2-http-0.20.0.jar:na]
at com.google.auth.Credentials.blockingGetToCallback(Credentials.java:112) ~[google-auth-library-credentials-0.20.0.jar:na]
at com.google.auth.Credentials$1.run(Credentials.java:98) ~[google-auth-library-credentials-0.20.0.jar:na]
... 7 common frames omitted
Caused by: com.google.api.client.http.HttpResponseException: 400 Bad Request
{"error":"invalid_grant","error_description":"Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim."}
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1113) ~[google-http-client-1.34.2.jar:1.34.2]
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:441) ~[google-auth-library-oauth2-http-0.20.0.jar:na]
... 12 common frames omitted

#SQSListen results in an exception and not working

I've a very simple Spring cloud aws project. I'm using Java 11.
here is the pom:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.5.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.demo.arf</groupId>
<artifactId>testsqs-boot</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>testsqs-boot</name>
<description>Demo project for Spring Boot</description>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Hoxton.SR3</version>
<type>pom</type>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws</artifactId>
<version>2.2.1.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-aws-messaging</artifactId>
<version>2.2.1.RELEASE</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
Config class:
package com.demo.arf.testsqsboot;
import com.amazonaws.services.sqs.AmazonSQSAsync;
import com.amazonaws.services.sqs.AmazonSQSAsyncClientBuilder;
import org.springframework.beans.factory.annotation.Value;
//import org.springframework.cloud.aws.messaging.core.QueueMessagingTemplate;
import org.springframework.cloud.aws.messaging.config.SimpleMessageListenerContainerFactory;
import org.springframework.cloud.aws.messaging.core.QueueMessagingTemplate;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
//import com.amazonaws.services.sqs.AmazonSQSAsync;
//import com.amazonaws.services.sqs.AmazonSQSAsyncClientBuilder;
#Configuration
public class SQSConfig {
#Value("${cloud.aws.region.static}")
private String region;
#Value("${cloud.aws.credentials.access-key}")
private String awsAccessKey;
#Value("${cloud.aws.credentials.secret-key}")
private String awsSecretKey;
#Bean
public QueueMessagingTemplate queueMessagingTemplate() {
return new QueueMessagingTemplate(amazonSQSAsync());
}
public AmazonSQSAsync amazonSQSAsync() {
return AmazonSQSAsyncClientBuilder.standard().withRegion(Regions.US_EAST_1)
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(awsAccessKey, awsSecretKey)))
.build();
}
#Bean
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSQS){
SimpleMessageListenerContainerFactory factory = new SimpleMessageListenerContainerFactory();
factory.setAmazonSqs(amazonSQS);
factory.setMaxNumberOfMessages(10);
return factory;
}
}
the controller class which send/receive message:
package com.demo.arf.testsqsboot.controller;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.aws.messaging.core.QueueMessagingTemplate;
import org.springframework.cloud.aws.messaging.listener.annotation.SqsListener;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
#RestController
public class SQSController {
#Autowired
private QueueMessagingTemplate queueMessagingTemplate;
private static final Logger LOG = LoggerFactory.getLogger(SQSController.class);
#GetMapping("/send-sqs-message")
public String sendMessage() {
String sqsEndPoint= "https://sqs.us-east-2.amazonaws.com/1234567879/my_queue";
queueMessagingTemplate.convertAndSend(sqsEndPoint, MessageBuilder.withPayload("hello from Spring Boot").build());
return "Hello SQS";
}
#SqsListener("my_queue")
public void getMessage(String message) {
LOG.info(" *********** Message from SQS Queue - "+message);
}
}
application.yml:
server:
port: 9001
cloud:
aws:
region:
static: us-east-1
auto: false
credentials:
access-key: "asdmnasdn"
secret-key: "sfkjsdjksdkj"
end-point:
uri: https://sqs.us-east-2.amazonaws.com/1234567879/my_queue
I can get the send working fine. but when I add the listener, I get the following error during startup and listener does not receive messages:
2020-03-15 01:02:00.677 INFO 15423 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 3853 ms
2020-03-15 01:02:01.109 INFO 15423 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
**WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.amazonaws.util.XpathUtils (file:/Users/arf/.m2/repository/com/amazonaws/aws-java-sdk-core/1.11.415/aws-java-sdk-core-1.11.415.jar) to method com.sun.org.apache.xpath.internal.XPathContext.getDTMManager()
WARNING: Please consider reporting this to the maintainers of com.amazonaws.util.XpathUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2020-03-15 01:02:01.749 WARN 15423 --- [ main]**
s.c.a.m.l.SimpleMessageListenerContainer : Ignoring queue with name 'my_queue': The queue does not exist.; nested exception is com.amazonaws.services.sqs.model.QueueDoesNotExistException: The specified queue does not exist for this wsdl version. (Service: AmazonSQS; Status Code: 400; Error Code: AWS.SimpleQueueService.NonExistentQueue; Request ID: 62821505-3f34-5434-a6ee)
2020-03-15 01:02:01.749 INFO 15423 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService
Also, one basic question. How does the #SQSListener know where to find the aws account info and sqs uri?

I've made it work with the following change in config class. However, I wonder, how most of the sample programs online without this code(which construct AmazonSQSAsync with withEndpointConfiguration) is working.
public QueueMessagingTemplate queueMessagingTemplate(AmazonSQSAsync amazonSQS) {
return new QueueMessagingTemplate(amazonSQS);
}
#Bean
#Primary
public AmazonSQSAsync amazonSQS(AWSCredentialsProvider credentials) {
return AmazonSQSAsyncClientBuilder.standard()
.withCredentials(credentials)
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(localStackSqsUrl, awsRegion))
.build();
}
#Bean
#Primary
public AWSCredentialsProvider awsCredentialsProvider() {
return new AWSCredentialsProviderChain(
new AWSStaticCredentialsProvider(
new BasicAWSCredentials("local", "stack")));
}```

A couple things.
- First NEVER store your AK/SK in a property or yml file like that. I can tell those are fake values, but you'll always want to pull these from ~/.aws/credentials or instance metadata. The AWS clients like AmazonSqSAsyncClientBuilder will do automatically if you just call .standard(). No need for the credential provider.
- Second, same with region
- Third, I believe #SqsListener will use the ContainerFactory bean you defined earlier, at least that's how the #JmsListener works.
- The error message you received was that your queue name was not found in your account in your selected region. You told it us-east-1, but in your send code you specified us-east-2. My guess based on your post is that your queue is in us-east-2 since your question came up about #SqsListener, not the queueMessagingTemplate.

Spring Integration 5.1 - integration flow convertion with #IntegrationConverter doesn't work

I upgrade my Spring boot version from 2.0.5.RELEASE to 2.1.8.RELEASE (so Spring Integration from 5.0 to 5.1) and the automatic type casting inside integration flow doesn't work anymore. I am used to define a set of #IntegrationConverter components and automatic casting with the operation transform(Type.class, p -> p) inside integration flow code but with the new version it seems to be broken.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.8.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>org.grorg</groupId>
<artifactId>grointegration</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>grointegration</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-integration</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-ip</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
And the Main.java file:
package org.grorg.grointegration;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.core.convert.converter.Converter;
import org.springframework.integration.config.IntegrationConverter;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.dsl.Transformers;
import org.springframework.integration.ip.dsl.Tcp;
import org.springframework.stereotype.Component;
class Test {
#Override
public String toString() {
return "test";
}
}
#Component
#IntegrationConverter
class Convert implements Converter<String, Test> {
#Override
public Test convert(String s) {
return new Test();
}
}
#SpringBootApplication
public class Main {
public static void main(String[] args) {
SpringApplication.run(GrointegrationApplication.class, args);
}
#Bean
public IntegrationFlow server() {
return IntegrationFlows
.from(Tcp.inboundGateway(Tcp.netServer(1234)))
.transform(Transformers.objectToString())
.transform(Test.class, id -> id) // in 2.1 I could use .convert(Test.class) but the problem is the same
.log()
.handle((p, h) -> "OK")
.get();
}
}
Use with a shell:
telnet localhost 1234
> test
> OK
[...]
With the previous version (2.0.5.RELEASE) the program work nicely like previously, but with the new version (2.1.8.RELEASE) I get this error (and no "OK" response):
org.springframework.integration.handler.ReplyRequiredException: No reply produced by handler 'server.org.springframework.integration.config.ConsumerEndpointFactoryBean#1', and its 'requiresReply' property is set to true.
[...]
What I have found is that the ConversionService has been remplaced by MessageConverter and now Jackson is used to transform message from one type to another.
Am I wrongly using type casting with integration flow? Do you have a new solution for casting object with the new version? Or is this just a regression?
Thanks in advance!

This is a bug in Spring Integration.
We just don't use a proper ConversionService in the GenericMessageConverter.
Please, raise a GH issue on the matter.
Meanwhile a workaround for you is like this:
#Bean(name = IntegrationContextUtils.ARGUMENT_RESOLVER_MESSAGE_CONVERTER_BEAN_NAME)
public static ConfigurableCompositeMessageConverter configurableCompositeMessageConverter(
#Qualifier(IntegrationUtils.INTEGRATION_CONVERSION_SERVICE_BEAN_NAME) ConversionService conversionService) {
return new ConfigurableCompositeMessageConverter(
Collections.singleton(new GenericMessageConverter(conversionService)));
}
So, we override whatever is configured by the framework and inject a proper IntegrationUtils.INTEGRATION_CONVERSION_SERVICE_BEAN_NAME bean which is supplied with your #IntegrationConverter component.

Dialogflow v2 Java Client Library detectIntent with Spring Boot: No DesignTimeAgent found for project

While trying to detectIntent from text I have received a below exception (project-id was replaced). Cannot find anything in google related to DesignTimeAgent issue. Any help is appreciated.
Exception :
There was an unexpected error (type=Internal Server Error, status=500).
io.grpc.StatusRuntimeException: NOT_FOUND: com.google.apps.framework.request.NotFoundException: No DesignTimeAgent found for project 'MY_PROJECT_ID'.
com.google.api.gax.rpc.NotFoundException: io.grpc.StatusRuntimeException: NOT_FOUND: com.google.apps.framework.request.NotFoundException: No DesignTimeAgent found for project 'MY_PROJECT_ID'.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:45)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:95)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:61)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1015)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1137)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:957)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:748)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:493)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:468)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
...
Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at com.google.cloud.dialogflow.v2.SessionsClient.detectIntent(SessionsClient.java:245)
at com.google.cloud.dialogflow.v2.SessionsClient.detectIntent(SessionsClient.java:184)
at com.my.microservices.controllers.DialogFlowRestController.test(DialogFlowRestController.java:35)
...
Caused by: io.grpc.StatusRuntimeException: NOT_FOUND: com.google.apps.framework.request.NotFoundException: No DesignTimeAgent found for project 'MY_PROJECT_ID'.
at io.grpc.Status.asRuntimeException(Status.java:533)
... 23 more
Rest controller to trigger detectIntent test (as https://localhost:8080/api/test):
package com.my.microservices.controllers;
import com.google.cloud.dialogflow.v2.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.*;
import javax.servlet.http.HttpServletRequest;
import java.util.UUID;
#RestController
#RequestMapping("/api")
public class DialogFlowRestController {
private static final Logger logger = LoggerFactory.getLogger(DialogFlowRestController.class);
private String userText = "Weather forecast for today in Berlin";
private final String LANG_CODE = "en-US";
private final String PROJECT_ID = System.getenv().get("GOOGLE_CLOUD_PROJECT");
private String sessionId = UUID.randomUUID().toString();
#GetMapping(value = "/test", produces = MediaType.APPLICATION_JSON_VALUE)
#ResponseBody
public String test() throws Exception {
logger.info("test was called");
try (SessionsClient sessionsClient = SessionsClient.create()) {
SessionName session = SessionName.of(PROJECT_ID, sessionId);
TextInput.Builder textInput = TextInput.newBuilder().setText(userText).setLanguageCode(LANG_CODE);
QueryInput queryInput = QueryInput.newBuilder().setText(textInput).build();
DetectIntentResponse response = sessionsClient.detectIntent(session, queryInput);
return response.toString();
}
}
}
Provided environment variables:
GOOGLE_CLOUD_PROJECT=MY_PROJECT_ID
GOOGLE_APPLICATION_CREDENTIALS=/Users/me/.my/pk.json
Spring-Boot pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.my.microservices</groupId>
<artifactId>dialog-flow-test</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>dialog-flow-test</name>
<description>Demo project with Dialog Flow</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- Dialogflow API Client Library for Java -->
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-dialogflow</artifactId>
<version>v2-rev81-1.25.0</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-dialogflow</artifactId>
<version>0.103.0-alpha</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>

OK, I have figured it out.
The above misleading error message was caused by the use of a wrong PROJECT_ID. My mistake was, that first I have created a new Project with a Service Account, gave to it dialogflow API rights and later I have tried to use its name and related PK JSON for my detectIntent calls. BUT it was not linked with Dialogflow Project ID.
So, to fix it I have started with a default Project ID, which was given to my new Dialogflow Agent and following instruction https://dialogflow.com/docs/reference/v2-auth-setup I have created a new Project with a Service Account which fit my Dialogflow Agent.

In my case I needed to move my Agent to Global serving from EU-W1

Creating custom connection pool in Spring Boot application

I'm writing a Spring Boot application which connects with Snowflake Data Warehouse and execute SQL queries on it.
I have written a Configuration class for configuring Datasource for connecting to Snowflake Data Warehouse as follows:
#Configuration
#EnableAutoConfiguration(exclude={DataSourceAutoConfiguration.class})
public class DBConfig {
Logger logger = LoggerFactory.getLogger(DBConfig.class);
#Bean
JdbcTemplate jdbcTemplate() throws IllegalAccessException, InvocationTargetException, InstantiationException {
logger.info("-----Configuring JDBCTemplate------");
SnowflakeBasicDataSource dataSource = new SnowflakeBasicDataSource();
dataSource.setServerName("<myserver>.snowflakecomputing.com");
dataSource.setUser("<my_username>");
dataSource.setPassword("<my_password>");
dataSource.setWarehouse("DEMO_WH");
dataSource.setDatabaseName("DEMO_DB");
dataSource.setSchema("PUBLIC");
return new JdbcTemplate(dataSource);
}
}
My pom.xml looks like as follows:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.1.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.vaibhav</groupId>
<artifactId>snowflake-1</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>snowflake-1</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>net.snowflake</groupId>
<artifactId>snowflake-jdbc</artifactId>
<version>3.6.21</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web-services</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
I have to use a connection pool for this data source in my Spring boot application.
How can I use HikariCP connection pool in my application which can work perfectly fine with my Customized DataSource?
------EDIT --- Thanks for providing solution, finally my working code looks like
#Configuration
#EnableAutoConfiguration(exclude = { DataSourceAutoConfiguration.class })
public class DBConfig {
Logger logger = LoggerFactory.getLogger(DBConfig.class);
#Bean
JdbcTemplate jdbcTemplate() throws IllegalAccessException, InvocationTargetException, InstantiationException {
logger.info("-----Configuring JDBCTemplate------");
HikariConfig config = new HikariConfig();
config.setDriverClassName("net.snowflake.client.jdbc.SnowflakeDriver");
// config.setDataSourceProperties(properties);
config.setJdbcUrl("jdbc:snowflake://<myserver>.snowflakecomputing.com/?warehouse=DEMO_WH&db=DEMO_DB&schema=PUBLIC");
config.setUsername("<my_username>");
config.setPassword("<my_password>");
HikariDataSource ds = new HikariDataSource(config);
return new JdbcTemplate(ds);
}
}

See setting SnowflakeDriver with Hikari:
HikariConfig config = new HikariConfig();
config.setDriverClassName("com.snowflake.client.jdbc.SnowflakeDriver");
config.setDataSourceProperties(properties);
config.setJdbcUrl(connectStr);
HikariDataSource ds = new HikariDataSource(config);

If spring-jdbc is being included, Spring will automatically create JdbcTemplate based on the available DataSource. So If the above answers does not satisfy you, you may just try:
#Configuration
#EnableAutoConfiguration(exclude={DataSourceAutoConfiguration.class})
public class DBConfig {
Logger logger = LoggerFactory.getLogger(DBConfig.class);
// TAKE NOTE THAT THIS MIGHT ALREADY BE DONE BY SPRING
#Bean
protected JdbcTemplate jdbcTemplate( DataSource dataSource )
{
return new JdbcTemplate( dataSource );
}
#Bean
protected DataSource makeDataSource() throws IllegalAccessException, InvocationTargetException, InstantiationException {
logger.info("-----Configuring JDBCTemplate------");
SnowflakeBasicDataSource dataSource = new SnowflakeBasicDataSource();
dataSource.setServerName("<myserver>.snowflakecomputing.com");
dataSource.setUser("<my_username>");
dataSource.setPassword("<my_password>");
dataSource.setWarehouse("DEMO_WH");
dataSource.setDatabaseName("DEMO_DB");
dataSource.setSchema("PUBLIC");
return dataSource;
}
}

Hikari is default connection pool in Spring-boot 2+
We have nothing to do if we want to use Hikari in an application based
on Spring Boot 2.x.
You can set different properties of connection pool thru application.yml / application.properties. Below is an example of application.yml:
spring:
datasource
hikari:
maximumPoolSize: 4 # Specify maximum pool size
minimumIdle: 1 # Specify minimum pool size
driver-class-name: com.snowflake.client.jdbc.SnowflakeDriver
This is a useful link for configuring Hikari CP.