We Keep Coding

How to get response body of a post request without escape sequences in Java?

I am trying to convert the HttpResponse's response body to a String using the following piece of code:
HttpGet request = new HttpGet(url);
request.setHeader("Accept", "application/json");
response = client.execute(request);
EntityUtils.toString(response.getEntity(), standardCharsets.UTF_8.displayName());
The expected response after converting it to a String is as follows:
{"errors": ["permission denied"]}
This code was working fine until recently. But now the response body is somehow being modified to this:
{"errors":["1 error occurred:\n\t* permission denied\n\n"]}
I do not understand how the "1 error occurred" message is being inserted into the response body as the response from the same call in Postman gives me the appropriate response i.e.:
{"errors": ["permission denied"]}
And I've checked the code thoroughly and am certain that the "1 error occurred" message is not being inserted manually.
EDIT:
These were the response headers from Postman:
cache-control →no-store
content-length →33
content-type →application/json
date →Thu, 30 May 2019 06:52:33 GMT
status →403
And these were the response headers from the HttpClient call:
Cache-Control: no-store
Content-Type: application/json
Date: Thu, 30 May 2019 09:06:21 GMT
Content-Length: 60
The only difference is that the status code isn't being printed. But checking the status code through the response.getStatusLine().getStatusCode() yields 403 as well.

Looks like you are using Apache HttpClient, which does not modify responses this way. It's definitely the response generated by server. Check your configuration of request in Postman to see if there are any additional headers, that may alter server behavior.

Spring Boot: How to resolve Content-Type when incorrectly received from server

I have the following class in Java. I'm expecting it to issue a GET request to the url, get back a JSON payload, and transform that payload to List<LocationData>.
package ...
import ...
#Repository
public class ProxiedLocationRepo {
public List<LocationData> findAll() throws Exception {
RestTemplate restTemplate = new RestTemplate();
String url = UriComponentsBuilder
.fromUriString("https://my-host/path")
.queryParams("some", "queryParams")
.toUriString();
ResponseEntity<List<LocationData>> res = restTemplate.exchange(
url,
HttpMethod.GET,
null,
new ParameterizedTypeReference<List<LocationData>>(){});
if (res.getStatusCode() == HttpStatus.ACCEPTED) {
return res.getBody();
} else {
throw new ResponseStatusException(res.getStatusCode(), "Did not receive a 200 response from Server.");
}
}
}
However, I'm getting back this error:
org.springframework.http.InvalidMediaTypeException: Invalid mime type "charset=UTF-8": does not contain '/'
Which is expected, because if I do the same request from curl, and check the headers I get this (notice Content-Type line):
$ curl -sfi 'https://my-host/path?some=queryParams'
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 06 Mar 2019 13:58:58 GMT
Content-Type: charset=UTF-8
Content-Length: 1821
Connection: keep-alive
Vary: Accept-Encoding
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: SAMEORIGIN
... # perfectly formatted JSON payload here
I know that the Content-Type returned from this server is going to be application/json, but it is not providing it to me.
Is there anyway to inform RestTemplate#exchange of what the Content-Type of the response will be? If not, is there any other methodology I could use to resolve this issue besides getting the owners of the server to set the Content-Type correctly?
EDIT:
I have also tried adding the "Accept" header but got the same results:
$ curl -sfi 'https://my-host/path?some=queryParams' \
-H 'Accept: application/json'

Unfortunately I don't think there's any way to fix this while leveraging the Spring framework. Even if you were to create a custom JsonbHttpMessageConverter that accepts a MIME type of ANY, Spring would still fail to parse the incorrect Content-Type received from the request (because it can't find "/" in the Content-Type string).
So the resolution here was to do use java.net.HttpURLConnection to do the networking instead, and then use com.fasterxml.jackson.databind.ObjectMapper to map from the JSON to a POJO.
It works, but at the cost of no longer being able to leverage any of Spring's HTTP handling, which is likely much much more robust than anything I can implement alone.

REST Response - logging payload via readEntity(String.class) fails

Consider this code for a REST proxy running under Jersey 2.22.2:
#POST
#Produces(MediaType.APPLICATION_JSON)
#Path("/foo")
public Response foo(String request) {
logger.info("Request to /foo :\n" + request);
WebTarget target = ClientBuilder.newClient().target(esbWsUrl).path("/foo");
Response response = target.request(MediaType.APPLICATION_JSON_TYPE).post(Entity.entity(request, MediaType.APPLICATION_JSON_TYPE));
// problem is in the 2 lines below
logger.info("Buffered: " + response.bufferEntity());
logger.info("Response from /foo :\n" + response.readEntity(String.class));
return response;
}
First I tried this without the response.bufferedEntity() and got IllegalStateException since response.readEntity(String.class) consumes from the underlying stream in the Response.
When I use response.bufferEntity(), I am able to call response.readEntity(String.class) multiple times (the buffering seems to work), yet the response returned by Jersey gives a zero bytes response.
$ curl -m 5 -i -X POST -d '{"input_values": "abc"}' http://localhost:9000/services/rest/foo
HTTP/1.1 200 OK
Date: Thu, 27 Oct 2016 08:18:09 GMT
Keep-Alive: timeout=20
X-Type: default
Server: nginx
Content-Type: application/json; charset=UTF-8
Content-Length: 364
curl: (28) Operation timed out after 5000 milliseconds with 0 out of 364 bytes received
Some relevent documentation is here: Response.readEntity
Can anyone please show me how to log the contents of the Response, before returning a valid Response ?
Maybe there is some stream handling issues I havent thought of, or perhaps another way to turn the Response entity to a string (without consuming the stream).

I ended up making a completely new Response, based on the status code and entity of the previous one.

How to handle Github Webhook in Java?

Simple question.Got registered Payload URL on Github: using ngrok.com (ngrok) link like explained in Github documentation for Webhooks: Creating Webhooks
ngrok definition: “I want to securely expose a local web server to the internet and capture all traffic for detailed inspection and replay.”
When i send POST request with payload from github on correct Payload URL the response code is 200, how can I handle that request/response and get payload (JSON) in java? With servlet or?
I have no idea where to start. Tried to search but nothing for Java :(
If i put ngrok.com/something, Eclipse console throw:[WARN] 404 - POST /pas (127.0.0.1) 1368 bytes
Request headers
Host: ....ngrok.com
X-Real-IP: 192....
X-Forwarded-Proto: http
Connection: close
Content-Length: 5759
Accept: */*
User-Agent: GitHub-Hookshot/e9dfd89
X-GitHub-Event: ping
X-GitHub-Delivery: c5493000-b67e-11e4-8199-8b09d3d66948
Content-Type: application/json
X-Hub-Signature: sha1=b2947ce6a6de23f4274831523bae375d64e20021
Response headers
Connection: close
Content-Type: text/html;charset=ISO-8859-1
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 1368
If i put good URL, status is 200. Response on Github Webhooks / Manage webhook:Accept-Ranges: bytes
Connection: keep-alive
Content-Length: 1521
Content-Type: text/html
Date: Tue, 17 Feb 2015 10:17:46 GMT
Last-Modified: Thu, 12 Feb 2015 09:06:18 GMT
Server: nginx/1.6.2
So the question is actually "How to handle that payload?"
In documentation they use Sinatra and that's a big ? for me.
Sinatra code looks like this:
require "sinatra"
require "json"
post "/payload" do
push = JSON.parse(request.body.read)
puts "I got some JSON: #{push.inspect}"
end
New to this, sorry if its stupid question.

Resolved, i used HttpServlet doPost method to fetch request, then from request i getReader() and read line so i can make JSONObject. My servlet is on page/Payload and Webhook is on http://server.com/page/Payload
public class Payload extends HttpServlet {
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
StringBuilder builder = new StringBuilder();
String aux = "";
while ((aux = req.getReader().readLine()) != null) {
builder.append(aux);
}
String text = builder.toString();
try {
JSONObject json = new JSONObject(text);
String teams_url = json.getJSONObject("repository").getString("teams_url");
System.out.println("Teams URL:: "+teams_url);
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}}

You can make use of gitlab4j-api library. For example usage, have a look at simple review project, exactly here: https://github.com/gitlab4j/simple-cr/blob/master/src/main/java/org/gitlab4j/simplecr/service/GitLabWebHookService.java

Custom Response + HTTP status?

I have a rest interface for my project.
For one class i have a POST method where you can post an xml and i RETURN a custom response like:
<customResponse>Invalid email</customResponse>
if the email from the xml which was posted, was incorrect + other custom messages i have defined for different situations.
For all of these the HTTP STATUS is automatically put on 200 (OK).
Is there any way to change it?
Ps: I know that i can throw a web application like :
throw new WebApplicationException(Response.Status.BAD_REQUEST);
but in this case my custom response is no more included.
So i just want to return my custom error + 400 as http response.
Thanks in advance.
UPDATE after comments:
My method is:
#POST
#Path("{membershipExternalId}")
#Consumes(MediaType.APPLICATION_XML)
#Produces("application/xml")
public CustomResponse invite(){ //code}
You see that i return my CUSTOM RESPONSE. If i would return simple RESPONSE i could set the STATUS but in this case i cannot see any way.

Found the solution:
Put the return type as Response to the method:
#POST
#Path("{membershipExternalId}")
#Consumes(MediaType.APPLICATION_XML)
#Produces("application/xml")
public Response invite(){ //code
if (fail())
return Response.status(400).entity(customResponse).build();
}
Response.status(400).entity(customResponse) will do the trick. When build() it will convert your custom response xml =>
HTTP/1.1 400 Bad Request
Server: Apache-Coyote/1.1
X-Powered-By: Servlet 2.4; JBoss-4.2.3.GA (build: SVNTag=JBoss_4_2_3_GA date=200807181439)/JBossWeb-2.0
Set-Cookie: JSESSIONID=1C72921619A6B32BC1166B3567A39ADA; Path=/
Content-Type: application/xml
Content-Length: 140
Date: Thu, 18 Mar 2010 12:15:15 GMT
Connection: close
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><customResponse><message>Invalid email</message></customResponse>

setStatus or sendError on HttpServletResponse should do the trick.

This is tagged Java but I don't recognize Response.Status.BAD_REQUEST.
For Java just call setStatus on the HttpServletResponse object.
For .NET it looks like this:
HttpContext.Current.Response.StatusCode = xxx;