We Keep Coding

WebClient Request and Response body logging

I am trying to make a POJO out of request and response data received when making WebClient calls. But I am not getting the request body in string/JSON readable form instead I am getting a BodyInsertor. I am making use of Exchange Filters.
public ExchangeFilterFunction logWebRequest() {
return (request, next) -> {
log.info("Entered in logWebRequest for WebClient");
long startTime = System.currentTimeMillis();
Mono<ClientResponse> response = next.exchange(request);
long processingTimeInMs = System.currentTimeMillis() - startTime;
// request.body() -> Gives Body Insertor
WebRequestLog webRequestLog = webRequestService.makeWebRequestLog(request, response.block());
webRequestLog.setProcessingTimeInMs(processingTimeInMs);
log.info("WebRequest to be produced to kafka topic: " + webRequestLog);
kafkaService.produceAuditLog(webRequestLog);
return response;
};
}
I followed some articles such as https://andrew-flower.com/blog/webclient-body-logging and https://www.gitmemory.com/issue/spring-projects/spring-framework/24262/570245788 but nothing worked for me.
My end goal is to capture requests and responses with their bodies and produce the data collected for Kafka.

Inside ExchangeFilterFunction, you can access HTTP method, URL, headers, cookies but request or response body can not be accessed directly from this filter.
Refer to the answer here. It provides a way to get access to the request and response body. It also provides a link to This blog post. It explains how to get the body in JSON/String format in Web Client.

You can do tracing of request and response payloads with small manipulations with request and responses:
public class TracingExchangeFilterFunction implements ExchangeFilterFunction {
return next.exchange(buildTraceableRequest(request))
.flatMap(response ->
response.body(BodyExtractors.toDataBuffers())
.next()
.doOnNext(dataBuffer -> traceResponse(response, dataBuffer))
.thenReturn(response)) ;
}
private ClientRequest buildTraceableRequest(
final ClientRequest clientRequest) {
return ClientRequest.from(clientRequest).body(
new BodyInserter<>() {
#Override
public Mono<Void> insert(
final ClientHttpRequest outputMessage,
final Context context) {
return clientRequest.body().insert(
new ClientHttpRequestDecorator(outputMessage) {
#Override
public Mono<Void> writeWith(final Publisher<? extends DataBuffer> body) {
return super.writeWith(
from(body).doOnNext(buffer ->
traceRequest(clientRequest, buffer)));
}
}, context);
}
}).build();
}
private void traceRequest(ClientRequest clientRequest, DataBuffer buffer) {
final ByteBuf byteBuf = NettyDataBufferFactory.toByteBuf(buffer);
final byte[] bytes = ByteBufUtil.getBytes(byteBuf);
// do some tracing e.g. new String(bytes)
}
private void traceResponse(ClientResponse response, DataBuffer dataBuffer) {
final byte[] bytes = new byte[dataBuffer.readableByteCount()];
dataBuffer.read(bytes);
// do some tracing e.g. new String(bytes)
}
}

To add to Vicky Ajmera answer best way to get and log a request is with ExchangeFilterFunction.
private ExchangeFilterFunction logRequest() {
return (clientRequest, next) -> {
logger.info("Request: {} {} {}", clientRequest.method(), clientRequest.url(), clientRequest.body());
clientRequest.headers()
.forEach((name, values) -> values.forEach(value -> logger.info("{}={}", name, value)));
return next.exchange(clientRequest);
};
}
But to log a response body you will have to go to the lower level of ClientHttpResponse which then allows you to intercept the body.
First extend ClientHttpResponseDecorator like this:
public class LoggingClientHttpResponse extends ClientHttpResponseDecorator {
private static final Logger logger = LoggerFactory.getLogger(LoggingClientHttpResponse.class);
private static final DataBufferFactory bufferFactory = new DefaultDataBufferFactory();
private final DataBuffer buffer = bufferFactory.allocateBuffer();
public LoggingClientHttpResponse(ClientHttpResponse delegate) {
super(delegate);
}
#Override
public Flux<DataBuffer> getBody() {
return super.getBody()
.doOnNext(this.buffer::write)
.doOnComplete(() -> logger.info("Response Body: {}", buffer.toString(StandardCharsets.UTF_8)));
}
}
Then create your implementation of ClientHttpConnector like this:
public class LoggingClientHttpConnector implements ClientHttpConnector {
private final ClientHttpConnector delegate;
public LoggingClientHttpConnector(ClientHttpConnector delegate) {
this.delegate = delegate;
}
#Override
public Mono<ClientHttpResponse> connect(HttpMethod method, URI uri, Function<? super ClientHttpRequest, Mono<Void>> requestCallback) {
return this.delegate.connect(method, uri, requestCallback).map(LoggingClientHttpResponse::new);
}
}
And last when building your WebClient add a connector:
HttpClient httpClient = HttpClient.create();
ClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
WebClient.builder()
.baseUrl("http://localhost:8080")
.clientConnector(new LoggingClientHttpConnectorDecorator(connector))
.filter(logRequest())
.build();

MockRestServiceServer test sending a file upload through Multipart-formdata

I have a method that sends a rest request to an api with multipart-formdata, this will upload a file to the external api. However, I am not able to finish the unit test method for this.
The first problem I am finding is that the content-type that I am expecting is always different than the one that method create. For some reason when sending the request the mediatype is multipart-formdata but the header is set as that in addition to charset and boundary. The latter, boundary, is always changing its value therefore I can not set the expected value on the unit tests because it will always be different.
Apart from that, how do I also expect that the content of the request is the same content that I initiated the test with? How do I assert that the payload is the same.
Please check the code:
Service class:
#Service
#Slf4j
public class JiraService {
private HttpHeaders createRequestHeaders(JiraClient jiraClient, MediaType contenType) {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(contenType);
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.setBasicAuth(jiraClient.getUsername(), jiraClient.getPassword());
return headers;
}
private <EC, RC> ResponseEntity<RC> createRequestAndSend(HttpMethod method, String url, HttpHeaders headers,
EC payload, Class<RC> responseType) {
HttpEntity<EC> requestEntity = new HttpEntity<>(payload, headers);
ResponseEntity<RC> responseEntity = restTemplate.exchange(url, method, requestEntity, responseType);
// TODO deal with response
log.error("Loggin something");
return responseEntity;
}
public void addAttachment(JiraClient jiraClient, JiraIssue jiraIssue, JiraAttachment jiraAttachment)
throws MalformedURLException, IOException {
String url = jiraClient.getHost() + "/rest/api/2/issue/" + jiraIssue.getKey() + "/attachments";
HttpHeaders headers = createRequestHeaders(jiraClient, MediaType.MULTIPART_FORM_DATA); // What to do here?
headers.set("X-Atlassian-Token", "no-check");
FileSystemResource file = jiraAttachment.downloadFileFromWeb();
MultiValueMap<String, Object> payload = new LinkedMultiValueMap<>();
payload.add("file", file);
createRequestAndSend(HttpMethod.POST, url, headers, payload, String.class);
jiraAttachment.deleteFileFromSystem();
}
}
ServiceTest.class
#ActiveProfiles("test")
#RestClientTest(JiraService.class)
public class JiraServiceTest {
#Value("classpath:jira/add_attachment/validJiraAttachmentAddition.json")
private Resource validJiraAttachmentAddition;
#Autowired
private MockRestServiceServer server;
#Autowired
private JiraService jiraService;
#Mock
private JiraAttachment mockJiraAttachment;
private FileSystemResource attachmentFileSystemResource;
#BeforeEach
public void setupTests() throws IOException {
// initialize mocks
}
#Test
public void addAttachment_WithValidData_ShouldAddAttachmentToJiraIssue() throws Exception {
String url = host + "/rest/api/2/issue/" + issueKey + "/attachments";
ResponseActions stub = createServiceStub(HttpMethod.POST, url, MediaType.MULTIPART_FORM_DATA_VALUE);
stub = stub.andExpect(header("X-Atlassian-Token", "no-check"));
stub.andRespond(withSuccess());
// How to assert that the content of the request is the same as the resource?
when(mockJiraAttachment.downloadFileFromWeb()).thenReturn(attachmentFileSystemResource);
jiraService.addAttachment(mockJiraClient, mockJiraIssue, mockJiraAttachment);
}
private ResponseActions createServiceStub(HttpMethod method, String url, String contenType) {
String encodedCredentials = Base64.getEncoder().encodeToString((username + ":" + password).getBytes());
ResponseActions stub = server.expect(ExpectedCount.once(), requestTo(url));
stub = stub.andExpect(method(method));
stub = stub.andExpect(header("Content-Type", contenType)); // How to expect the content type here ?
stub = stub.andExpect(header("Authorization", "Basic " + encodedCredentials));
return stub;
}
}

Use ContentRequestMatchers.contentTypeCompatibleWith(MediaType contentType)
import static org.springframework.test.web.client.match.MockRestRequestMatchers.content;
...
stub.andExpect(content().contentTypeCompatibleWith(MediaType.MULTIPART_FORM_DATA))

Feign Client Error Handling - Suppress the Error/Exception and convert to 200 success response

I am using feign client to connect to downstream service.
I got a requirement that when one of the downstream service endpoint returns 400 ( it's partial success scenario ) our service need this to be converted to 200 success with the response value.
I am looking for a best way of doing this.
We are using error decoder to handle the errors and the above conversion is applicable for only one endpoint not for all the downstream endpoints and noticed that decode() method should returns exception back.

You will need to create a customized Client to intercept the Response early enough to change the response status and not invoke the ErrorDecoder. The simplest approach is to create a wrapper on an existing client and create a new Response with a 200 status. Here is an example when using Feign's ApacheHttpClient:
public class ClientWrapper extends ApacheHttpClient {
private ApacheHttpClient delegate;
public ClientWrapper(ApacheHttpClient client) {
this.client = client;
}
#Override
public Response execute(Request request, Request.Options options) throws IOException {
/* execute the request on the delegate */
Response response = this.client.execute(request, options);
/* check the response code and change */
if (response.status() == 400) {
response = Response.builder(response).status(200).build();
}
return response;
}
}
This customized client can be used on any Feign client you need.

Another way of doing is by throwing custom exception at error decoder and convert this custom exception to success at spring global exception handler (using #RestControllerAdvice )
public class CustomErrorDecoder implements ErrorDecoder {
#Override
public Exception decode(String methodKey, Response response) {
if (response.status() == 400 && response.request().url().contains("/wanttocovert400to200/clientendpoints") {
ResponseData responseData;
ObjectMapper mapper = new ObjectMapper();
try {
responseData = mapper.readValue(response.body().asInputStream(), ResponseData.class);
} catch (Exception e) {
responseData = new ResponseData();
}
return new PartialSuccessException(responseData);
}
return FeignException.errorStatus(methodKey, response);
}}
And the Exception handler as below
#RestControllerAdvice
public class GlobalControllerExceptionHandler {
#ResponseStatus(HttpStatus.OK)
#ExceptionHandler(PartialSuccessException.class)
public ResponseData handlePartialSuccessException(
PartialSuccessException ex) {
return ex.getResponseData();
}
}

Change the microservice response:
public class CustomFeignClient extends Client.Default {
public CustomFeignClient(
final SSLSocketFactory sslContextFactory, final HostnameVerifier
hostnameVerifier) {
super(sslContextFactory, hostnameVerifier);
}
#Override
public Response execute(final Request request, final Request.Options
options) throws IOException {
Response response = super.execute(request, options);
if (HttpStatus.SC_OK != response.status()) {
response =
Response.builder()
.status(HttpStatus.SC_OK)
.body(InputStream.nullInputStream(), 0)
.headers(response.headers())
.request(response.request())
.build();
}
return response;
}
}
Add a Feign Client Config:
#Configuration
public class FeignClientConfig {
#Bean
public Client client() {
return new CustomFeignClient(null, null);
}
}

Android Login authentication with Spring boot server

I'm working on an android app where I need to login and register. I've created the server side using the java spring boot framework. Registration works well, but I cannot login. How can I solve? The code I wrote is as follows:
Server code for register:
#PostMapping("/register")
public Person addUser(#RequestBody Person user) {
user.setPassword(encoder.encode(user.getPassword()));
return userService.saveUser(user);
}
Server code for login:
#PostMapping("/login")
public UserDetails authenticate(#RequestBody Person principal) throws Exception {
return authService.authenticate(principal);
}
And this works well. Because if I try with postman, I can log well.
This is postman result
Also, registration with andorid works weel too. If I register an user from android, it'll be save into my db and with postman i can log with that user.
This is my db
This is my andorid code for registration:
class HttpRequestTask extends AsyncTask<Void, Void, Person> {
#Override
protected Person doInBackground(Void... params) {
try {
final String url = "myurl/register";
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new MappingJackson2HttpMessageConverter());
Person person= new Person();
person.setNome(myname);
person.setCognome(myusername);
person.setUsername(mynickname);
person.setPassword(mypassword);
person.setProfileType(myprofileType);
return restTemplate.postForObject(url, person, Person.class);
}
catch(Exception e){
Log.e("MainActivity", e.getMessage(), e);
}
return null;
}
}
And it works well.
This is my android code for login:
class HttpRequestTask extends AsyncTask<Void, Void, Person> {
#Override
protected Person doInBackground(Void... params) {
try {
final String url = "myurl/login";
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new MappingJackson2HttpMessageConverter());
Person person= new Person();
person.setUsername(myusername);
person.setPassword(mypassword);
return restTemplate.postForObject(url, person, Person.class);
}
catch(Exception e){
Log.e("MainActivity", e.getMessage(), e);
}
return null;
}
}
Login does not work.
Should I use Spring boot for android?
Does anyone have an example?
Thanks in Advance!.

I think you must send your auth token in the header. This sould be accomplished with an interceptor like this
public class AuthInterceptor implements ClientHttpRequestInterceptor {
public static final String TAG = "INTERCEPTOR";
#Bean
EgomniaLoginManager loginManager;
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
HttpHeaders headers = request.getHeaders();
headers.set(ACCESS_TOKEN, loginManager.getAccessToken());
// logga request
if (BuildConfig.DEBUG) {
this.logRequest(request, body);
}
ClientHttpResponse response = execution.execute(request, body);
// logga response
if (BuildConfig.DEBUG) {
// this.logResponse(response);
// execute second time because logging consume response body
// response = execution.execute(request, body);
}
return response;
}
}
You must add this interceptor to your RestTemplate, I use Android Annotations to do that, so I don't remember how it could be done

How to create a oAuth request using java?

I need to make a connection with Viagogo website using oAuth. Referring to their documentation I need to create a request similar to the following one
Using the example in step 1A, this means you may generate a signature base string that looks like the following:
GET&http%3A%2F%2Fapi.viagogo.net%2FPublic%2FSimpleOAuthAccessRequest&oauth_consumer_key%3Dtestkey%26oauth_nonce%3Dmyn0nc3%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1292404912%26oauth_version%3D1.0%26scope%3DAPI.Public
I am using the following code but when I comment lines 1,2 it return unauthorized error, and when I use them it shows oauthService.signRequest returns void.
TradeKingAPI.java
import org.scribe.builder.api.DefaultApi10a;
import org.scribe.model.Token;
public class TradeKingAPI extends DefaultApi10a {
#Override
public String getRequestTokenEndpoint() {
return "http://api.viagogo.net/Public/SimpleOAuthAccessRequest";
}
#Override
public String getAccessTokenEndpoint() {
return "http://api.viagogo.net/Public/SimpleOAuthAccessRequest";
}
#Override
public String getAuthorizationUrl(Token requestToken) {
return "http://api.viagogo.net/Public/SimpleOAuthAccessRequest";
}
}
main.java
import org.scribe.builder.ServiceBuilder;
import org.scribe.model.OAuthRequest;
import org.scribe.model.Response;
import org.scribe.model.Token;
import org.scribe.model.Verb;
import org.scribe.oauth.OAuthService;
import api.TradeKingAPI;
import org.scribe.builder.api.DefaultApi10a;
import org.scribe.model.OAuthConstants;
import org.scribe.oauth.OAuthService;
........
OAuthService oauthService = new ServiceBuilder()
.provider(TradeKingAPI.class)
.apiKey("My consumer key")
.apiSecret("My secret")
.scope("API.Public")
.build();
Long seconds = (System.currentTimeMillis() / 1000);
System.out.println(">>>" + seconds);
String stSeconds = seconds.toString();
OAuthRequest request = new OAuthRequest(Verb.GET, "http://api.viagogo.net/Public
/SimpleOAuthAccessRequest");
request.addOAuthParameter(OAuthConstants.CONSUMER_KEY, "My consumer key");
request.addOAuthParameter(OAuthConstants.NONCE, "myn0nc3");
request.addOAuthParameter(OAuthConstants.SIGN_METHOD, "HMAC-SHA1");
request.addOAuthParameter(OAuthConstants.TIMESTAMP, seconds.toString());
request.addOAuthParameter(OAuthConstants.VERSION, "1.0");
request.addOAuthParameter("scope", "API.Public");
1 String signature = oauthService.signRequest(OAuthConstants.EMPTY_TOKEN, request);
2 request.addOAuthParameter(OAuthConstants.SIGNATURE,signature);
Response response = request.send();
System.err.println(">>" + response.isSuccessful());
System.err.println(">>" + response.getMessage());
System.err.println(">>" + response.getBody());

From what I understand from Viagogo public API access documentation, the token you get in the step 1, is the equivalent to a request token in a complete OAuth 1.0a "dance".
So, you should be able to use scribe-java internal classes to get this token without doing it by hand. The only difference is that in scribe, this request sends also a callback url to the OAuth server for the next step of OAuth "dance".
As I can't get a consumer account I can only make assumption here. So let's have 2 scenarios :
Scenario 1 : Viagogo server tolerate extra parameter (i.e. call back URL)
so you can go with this code
import org.scribe.builder.api.DefaultApi10a;
import org.scribe.model.Token;
public class TradeKingAPI extends DefaultApi10a {
#Override
public Verb getRequestTokenVerb()
{
return Verb.GET;
}
#Override
public String getRequestTokenEndpoint() {
return "http://api.viagogo.net/Public/SimpleOAuthAccessRequest";
}
#Override
public String getAccessTokenEndpoint() {
return "none";
}
#Override
public String getAuthorizationUrl(Token requestToken) {
return "none";
}
}
Then your calling code will be :
OAuthService service = new ServiceBuilder()
.provider(TradeKingAPI.class)
.signatureType(QueryString)
.apiKey("My consumer key")
.apiSecret("My secret")
.scope("API.Public")
.build();
Token requestToken = service.getRequestToken();
//make your API calls
OAuthRequest request = new OAuthRequest(Verb.GET,
"http://api.viagogo.net/Public/Event/235");
service.signRequest(requestToken, request);
Response response = request.send();
System.out.println(response.getBody());
But as I said, if Viagogo security is a bit strict and it refuses the useless param oauth_callback, you'll need to switch to scenario 2
Scenario 2 : Build your own OAuthService
In this scenario you have to create a new OAuthService to avoid dealing with OAuthCallback parameter.
import org.scribe.builder.api.DefaultApi10a;
import org.scribe.model.*;
import org.scribe.oauth.OAuth10aServiceImpl;
import java.util.Map;
public class OAuth10aServiceForViagogo extends OAuth10aServiceImpl {
private OAuthConfig config;
private DefaultApi10a api;
public OAuth10aServiceForViagogo(DefaultApi10a api, OAuthConfig config) {
super(api, config);
this.api = api;
this.config = config;
}
private void addOAuthParams(OAuthRequest request, Token token) {
request.addOAuthParameter(OAuthConstants.TIMESTAMP, api.getTimestampService().getTimestampInSeconds());
request.addOAuthParameter(OAuthConstants.NONCE, api.getTimestampService().getNonce());
request.addOAuthParameter(OAuthConstants.CONSUMER_KEY, config.getApiKey());
request.addOAuthParameter(OAuthConstants.SIGN_METHOD, api.getSignatureService().getSignatureMethod());
request.addOAuthParameter(OAuthConstants.VERSION, getVersion());
request.addOAuthParameter(OAuthConstants.SCOPE, config.getScope());
request.addOAuthParameter(OAuthConstants.SIGNATURE, getSignature(request, token));
}
private String getSignature(OAuthRequest request, Token token) {
String baseString = api.getBaseStringExtractor().extract(request);
String signature = api.getSignatureService().getSignature(baseString, config.getApiSecret(), token.getSecret());
return signature;
}
private void appendSignature(OAuthRequest request) {
for (Map.Entry<String, String> entry : request.getOauthParameters().entrySet()) {
request.addQuerystringParameter(entry.getKey(), entry.getValue());
}
}
#Override
public Token getRequestToken(RequestTuner tuner) {
OAuthRequest request = new OAuthRequest(api.getRequestTokenVerb(), api.getRequestTokenEndpoint());
addOAuthParams(request, OAuthConstants.EMPTY_TOKEN);
appendSignature(request);
Response response = request.send(tuner);
String body = response.getBody();
return api.getRequestTokenExtractor().extract(body);
}
}
TrakingApi class will be slightly different to create the an OAuth10aServiceForViagogo when calling createService :
import org.scribe.builder.api.DefaultApi10a;
import org.scribe.model.Token;
public class TradeKingAPI extends DefaultApi10a {
#override
public OAuthService createService(OAuthConfig config)
{
return new OAuth10aServiceForViagogo(this, config);
}
#Override
public Verb getRequestTokenVerb()
{
return Verb.GET;
}
#Override
public String getRequestTokenEndpoint() {
return "http://api.viagogo.net/Public/SimpleOAuthAccessRequest";
}
#Override
public String getAccessTokenEndpoint() {
return "none";
}
#Override
public String getAuthorizationUrl(Token requestToken) {
return "none";
}
}
Then your calling code will be the same :
OAuthService service = new ServiceBuilder()
.provider(TradeKingAPI.class)
.signatureType(QueryString)
.apiKey("My consumer key")
.apiSecret("My secret")
.scope("API.Public")
.build();
Token requestToken = service.getRequestToken();
//make your API calls
OAuthRequest request = new OAuthRequest(Verb.GET,
"http://api.viagogo.net/Public/Event/235");
service.signRequest(requestToken, request);
Response response = request.send();
System.out.println(response.getBody());
I didn't test all this code because I can't access consumer and secret key, but it should be close to what you need.

I'm assuming you're trying to get the access token (e.g you're calling SimpleOAuthAccessRequest). Scribe's OauthService has methods to handle this.
BUT ... if you're going to do it manually, here is what's wrong with your code - at least with what you've listed here. I'm assuming you've configured scribe correctly.
don't pass the consumer secret with your request, that is only for signing the request
you should use addOauthParameter vs addQueryStringParameter
you should use the Scribe constants
you need to sign the request (again, Scribe's OauthService has help method for signing request)
Here's your updated snippet of code.
UPDATE:
Have Scribe provide all the Oauth parameters for you
OAuthRequest request = new OAuthRequest(Verb.GET, ...
//since you're just passing Oauth parameters and nothing else,
//you can use signRequest will create Oauth Parameters for you
service.signRequest(OAuthConstants.EMPTY_TOKEN, request)
Response response = request.send()