We Keep Coding

Micronaut java httpclient creation with the existing server

I have a Micronaut application running with the below configuration:
micronaut:
server:
cors:
enabled: true
port: 8080
Now I have an enhancement where I want to call a 3rd party URL and get the response in my application (one of the module in my application). I used the below code snippet:
EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class);
HttpClient client = server .getApplicationContext() .createBean(HttpClient.class, server.getURL());
HttpRequest req = HttpRequest.GET(urlHost);
HttpResponse<String> response = client.toBlocking().exchange(req, String.class);
But this is not working. I get port already in use. I did not find much help in google because Micronaut's HttpClient is usually used in Micronaut Test which is not in my case. Is this possible to use it in my application? If so how? Thanks in Advance.

It is because you are starting another server by ApplicationContext.run(EmbeddedServer.class).
You don't need it. It is enough to inject HttpClient into your class by constructor:
#Singleton
public class MyClient {
private final RxHttpClient client;
public MyClient(#Client("https://some.third-party.com") RxHttpClient client) {
this.client = client;
}
HttpResponse<String> getSomething(Integer id) {
URI uri = UriBuilder.of("/some-objects").path(id).build();
return client.toBlocking().exchange(HttpRequest.GET(uri), String.class);
}
}
If you have third-party server URL in application configuration under some-service.url path for example, then you can use #Client("${some-service.url}")
Another option is to define declarative client for the third-party server and then inject it in your classes where needed.
First define client interface for your third-party service:
#Client("some-service")
public interface SomeServiceClient {
#Get("/api/some-objects/{id}")
String getSomeObject(#QueryValue("id") Integer id);
}
Add client configuration for that service in application configuration (application.yaml):
micronaut:
http:
services:
some-service:
url: "https://some.third-party.com"
read-timeout: 1m
And then you can inject the SomeServiceClient where you need it:
#Singleton
public class SomeServiceConsumer {
private final SomeServiceClient client;
public SomeServiceConsumer(SomeServiceClient client) {
this.client = client;
}
void doWithSomething(Integer id) {
String object = client.getSomeObject(id);
... // processing of object here
}
}
You can find more information in a Micronaut documentation
https://guides.micronaut.io/latest/micronaut-http-client-gradle-java.html

Springboot external api call request and response capture in database

From my backend application(springboot, java8) i will make multiple external api call's. I have a requirement to log all the requests and response data's (including headers, request and response body) into database(MongoDB).
Below is my sample code, this how i am trying to capture request and responses on each external api calls. On exception i will store status as 'FAILED'.
In my project multiple modules will be added on new 3rd party api integration, so in each module for every different external api calls i have to capture all the requests and reponses like this. I am not satisfied with below approach. Kindly suggest best approach to solve this.
Sample Service layer method
public ResponseDTOFromExternalApi externalApiCallServiceMethod(String clientId, RequestDTO requestDTO) {
ExternalApiCallRequestObj externalApiCallRequestObj = prepareExternalApiRequestObjFromRequestDTO(requestDTO);
ApiCall apiCall = ApiCall.builder()
.clientId(clientId)
.status("SUBMITTED")
.requestPayload(externalApiCallRequestObj)
.build();
apiCall = apiCallRepository.save(apiCall);
ExternalApiCallReponseObj externalApiCallReponseObj = externalApiCallService.callExternalApi1(externalApiCallRequestObj);
apiCall = apiCallRepository.findById(apiCall.getId());
apiCall.setResponsePayload(externalApiCallReponseObj);
apiCall.setStatus("COMPLETED");
apiCallRepository.save(apiCall);
return toDTO(externalApiCallReponseObj);
}
Sample Domain for api calls
#Document("api_calls")
#Builder
#Data
public class ApiCall {
#Id
private String id;
private String clientId;
private String status;
private Object requestPayload;
private Object responsePayload;
}

Spring's WebClient already has the ability to log all request and response data by adding exchange filters.
By using it for your network requests the only thing left to do is to write this information in your mongodb.
Here is tutorial on logging requests and responses:
https://www.baeldung.com/spring-log-webclient-calls
Cheers

You may use Spring AOP to address this cross cutting concern.
Assuming ExternalApiCallService is a spring managed bean , following code will intercept all the callExternalApi1() and can log the same to database.
#Component
#Aspect
public class ExternalCallLoggerAspect {
#Autowired
ApiCallRepository apiCallRepository;
#Pointcut("execution(* *..ExternalApiCallService.callExternalApi1(..))")
public void externalApiCallService(){}
#Around("externalApiCallService() && args(request)")
public ExternalApiCallReponseObj logCalls(ProceedingJoinPoint pjp,ExternalApiCallRequestObj request){
Object result=null;
String status = "COMPLETED";
ExternalApiCallReponseObj response = null;
// Build the apiCall from request
ApiCall apiCall = ApiCall.builder()
.clientId(clientId)
.status("SUBMITTED")
.requestPayload(request)
.build();
//save the same to db
apiCall = apiCallRepository.save(apiCall);
// Proceed to call the external Api and get the result
try {
result = pjp.proceed();
} catch (Throwable e) {
status = "FAILED";
}
//Update the response
apiCall = apiCallRepository.findById(apiCall.getId());
apiCall.setStatus(status);
apiCallRepository.save(apiCall);
if(result != null) {
response = (ExternalApiCallReponseObj)result;
apiCall.setResponsePayload(response);
}
//continue with response
return response;
}
}
Note
1.There is a typo with the name ExternalApiCallReponseObj
2.The aspect code is verified that it works and the logic was included later on untested. Please make the required corrections
Ideally the original method should be stripped down to this
public ResponseDTOFromExternalApi externalApiCallServiceMethod(String clientId, RequestDTO requestDTO) {
return toDTO(externalApiCallService.callExternalApi1(prepareExternalApiRequestObjFromRequestDTO(requestDTO)));
}
More about Spring AOP here
Update : on a second thought , if all the external api calls are through a single method , say ExternalApiCallService.callExternalApi1() , this logging logic can be moved to that common point , isn't it ?

Zuul proxy - how to forward requests to services depending on request path

Problem
How to forward requests in Spring Cloud application? I need to forward requests to other services depending on the part of uri.
For example
HTTP GET http://user-application/api/users, returns users JSON.
HTTP GET http://user-application/api/proxy/jobs-application/api/jobs, returns jobs JSON, but this request should be forwarded to another application:
HTTP GET http://jobs-application/api/jobs.
Any HTTP method is allowed, not only GET.
Context
I have a SpringBoot Application, User application which has REST end-points which return data.
For example GET http://user-application/api/users would return users in the JSON format.
User application also has an HTTP end-point which should forward the request to other applications - let's call one of them Jobs application.
This end-point is HTTP {ANY_METHOD} /api/proxy/{dynamic-service}/{dynamic-path} as an example,
GET http://user-application/api/proxy/jobs-application/api/jobs
Please, note, initial request comes to the User application, while then it is forwarded to the Jobs application.
Approaches
I put some my approaches which I think about. Maybe you have done similar things in the past, so you could share your experience doing so. Or even improve one of my approaches.
ProxyController approach
I would create a ProxyController in User application with mapping /proxy
#Controller
#RequestMaping("/proxy/**")
ProxyController
public void proxy(final HttpServletRequest request, HttpResponse response) {
final String requestUri = request.getRequestUri();
if (!requestUri.startsWith("/api/proxy/")) {
return null; // Do not proxy
}
final int proxyIndex = "/api/proxy/".lenght(); // Can be made a constant
final String proxiedUrl = requestUri.subString(proxyIndex, requestUri.lenght());
final Optional<String> payload = retrievePayload(request);
final Headers headers = retrieveHeaders(request);
final HttpRequest proxyRequest = buildProxyRequest(request, headers);
payload.ifPresent(proxyRequest::setPayload);
final HttpResponse proxyResponse = httpClient.execute(proxyRequest)
pdateResponse(response, proxyResponse);
}
The problem with this approach, I have to write a lot of code t build a proxy request, to check if it has payload and if it has, copy it into proxy request, then copy headers, cookies etc to the proxy request, copy HTTP verb into proxy request. Then when I get proxy response, I have to populate its details into the response.
Zuul approach
I was inspired by ZuulFilters:
https://www.baeldung.com/spring-rest-with-zuul-proxy
https://stackoverflow.com/a/47856576/4587961
#Component
public class ProxyFilter extends ZuulFilter {
private static final String PROXY_PART = "/api/proxy";
private static final int PART_LENGTH = PROXY_PART.length();
#Autowired
public ProxyFilter() {
}
#Override
public boolean shouldFilter() {
final RequestContext context = RequestContext.getCurrentContext();
final String requestURI = retrieveRequestUri(context);
return requestURI.startsWith(PROXY_PART);
}
#Override
public Object run() {
final RequestContext context = RequestContext.getCurrentContext();
final String requestURI = retrieveRequestUri(context);
final String forwardUri = requestURI.substring(PART_LENGTH);
context.setRouteHost(buildUrl(forwardUri));
return null;
}
#Override
public String filterType() {
return "proxy";
}
#Override
public int filterOrder() {
return 0;
}
private String retrieveRequestUri(final RequestContext context) {
final HttpServletRequest request = context.getRequest();
return request.getRequestURI();
}
private URL buildUrl(final String uri) {
try {
return new URL(uri);
} catch (MalformedURLException e) {
throw new RuntimeException(String.format("Failed to forward request uri %s}.", uri), e);
}
}
}
This code allows me to forward requests with less effort. However, we also use client side load balancer Ribbon and circuit breaker Hystrix in Spring Cloud Zuul out of box. How to enable these features? Will they be enabled out of box in context.setRouteHost(forwardUrl);

I would like to add another approach, maybe it can also work.
Static application.yml file to configure Zuul proxy approach
This approach does not requre dynamic Zuul Filters.
application.yml
zuul:
routes:
user-application:
path: /api/users/**
serviceId: user-service
stripPrefix: false
sensitiveHeaders:
# I have to define all other services similarly.
jobs-application:
path: /api/proxy/jobs/**
serviceId: jobs-application
stripPrefix: true
sensitiveHeaders:
It will work only if I know all the services my clients need to call before I deploy the User application. What if a new application is added dynamically? Then I will have to update the configuration.

Mocking 3rd party api for rest api functional testing

I need to test the functionality of rest end point using Rest Template.
Below is the dummy rest api i need to test
#GetMapping(value = "/test")
public String getText(){
RestTemplate restTemplate = new RestTemplate();
String value = restTemplate.getForObject("EXTERNALURL/helloWorld",String.class);
return value +" got the value";
}
I have the below test cases which hits the above rest end point
private void expectFixedData() {
MockServerClient mockServerClient = ClientAndServer.startClientAndServer("127.0.0.1",1080);
try {
mockServerClient.when(HttpRequest.request().withMethod("GET")
.withPath("/helloWorld"), Times.exactly(1))
.respond(HttpResponse.response().withStatusCode(200)
.withBody("{ 'incorrect username and password combination' }")
.withDelay(TimeUnit.SECONDS, 1));
}
finally{
mockServerClient.close();
}
}
#Test
public void callExternalService(){
expectFixedData();
RestTemplate restTemplate = new RestTemplate();
String value = restTemplate.getForObject("http://localhost:8080/test",String.class);
Assert.assertTrue(value.equals("incorrect username and password combination got the value"));
}
But when i run the test cases it still calls the external URL
Any help would be appreciated.

Was able to mock response for 3rd party using WireMock

Spring Integration application with external Web Services monitoring

Currently I have an application with Spring Integration DSL that has AMQP inbound gateway with different service activators, each service activator has kind of logic to decide, transform and call external web services (currently with CXF), but all this logic is in code without Spring Integration components.
These service activators are monitored, in the output channel that returns data from this application is an AMQP adapter that sends headers to a queue (after that, all headers are processed and saved in a database for future analysis). This works well, these service activators even have elapsed time in headers.
Now, the problem is, that I need to monitor external web service calls, like elapsed time in each operation, which service endpoint and operation was called, if an error occurred.
I've been thinking that logic code in each service activator should be converted into a Spring Integration flow, in each service activator, would call a new gateway with the name of the operation of the web service in a header, and monitoring every flow as currently I had been doing.
So, I'm not sure if this manual approach is the better approach, so I wonder if there is a way to get the name of the service operation with some kind of interceptor or something similar with CXF or Spring WS to avoid setting the name of the operation in headers in a manual way? What would be your recommendation?
To have more context here is the Spring Integration configuration:
#Bean
public IntegrationFlow inboundFlow() {
return IntegrationFlows.from(Amqp.inboundGateway(simpleMessageListenerContainer())
.mappedReplyHeaders(AMQPConstants.AMQP_CUSTOM_HEADER_FIELD_NAME_MATCH_PATTERN)
.mappedRequestHeaders(AMQPConstants.AMQP_CUSTOM_HEADER_FIELD_NAME_MATCH_PATTERN)
.errorChannel(gatewayErrorChannel())
.requestChannel(gatewayRequestChannel())
.replyChannel(gatewayResponseChannel())
)
.enrichHeaders(new Consumer<HeaderEnricherSpec>() {
#Override
public void accept(HeaderEnricherSpec t) {
t.headerExpression(AMQPConstants.START_TIMESTAMP, "T(java.lang.System).currentTimeMillis()");
}
})
.transform(getCustomFromJsonTransformer())
.route(new HeaderValueRouter(AMQPConstants.OPERATION_ROUTING_KEY))
.get();
}
#Bean
public MessageChannel gatewayRequestChannel() {
return MessageChannels.publishSubscribe().get();
}
#Bean
public MessageChannel gatewayResponseChannel() {
return MessageChannels.publishSubscribe().get();
}
private IntegrationFlow loggerOutboundFlowTemplate(MessageChannel fromMessageChannel) {
return IntegrationFlows.from(fromMessageChannel)
.handle(Amqp.outboundAdapter(new RabbitTemplate(getConnectionFactory()))
.exchangeName(LOGGER_EXCHANGE_NAME)
.routingKey(LOGGER_EXCHANGE_ROUTING_KEY)
.mappedRequestHeaders("*"))
.get();
}
And here is a typical service activator, as you can see, all this logic could be an integration flow:
#ServiceActivator(inputChannel="myServiceActivator", outputChannel = ConfigurationBase.MAP_RESPONSE_CHANNEL_NAME)
public Message<Map<String, Object>> myServiceActivator(Map<String, Object> input, #Header(AMQPConstants.SESSION) UserSession session) throws MyException {
Message<Map<String, Object>> result = null;
Map<String, Object> mapReturn = null;
ExternalService port = serviceConnection.getExternalService();
try {
if (input.containsKey(MappingConstants.TYPE)) {
Request request = transformer
.transformRequest(input, session);
Response response = port
.getSomething(request);
utils.processBackendCommonErrors(response.getCode(), response.getResponse());
mapReturn = transformer.convertToMap(response);
} else {
Request request = transformer
.transformRequest(input, session);
Response response = port
.getSomethingElse(request);
utils.processBackendCommonErrors(response.getCode(),
response.getResponse());
mapReturn = transformer.convertToMap(response);
}
} catch (RuntimeException e) {
String message = "unexcepted exception from the back-end";
logger.warn(message, e);
throw MyException.generateTechnicalException(message, null, e);
}
result = MessageBuilder.withPayload(mapReturn)
.build();
return result;
}

So far so good. Or I don't understand the problem, or you are not clear where it is.
Anyway you always can proxy any Spring Service with the AOP, since it looks like you are pointing to the code:
Response response = port
.getSomething(request);
When this (or similar) method is called, some MethodInterceptor can perform desired tracing logic and send result to some MessageChannel for further analysis or anything else to do:
public Object invoke(MethodInvocation invocation) throws Throwable {
// Extract required operation name and start_date from the MethodInvocation
Object result = invocation.proceed();
// Extract required data from the response
// Build message and send to the channel
return result;
}