Github api Problems Parsing JSON error - java

So I am getting the infamous 'Problems parsing JSON' error whenever I try to perform an API call that takes a body. I have tried both the stringified and non-stringified versions of the json:
{"path":"bin/foo","message":"[SPS-CodeDeploy] Added routing file \u0027bin/foo\u0027","content":"","branch":"foobar"}
And:
"{\"path\":\"bin/foo\",\"message\":\"[Blah] Added file \\u0027bin/foo\\u0027\",\"content\":\"\",\"branch\":\"foobar\"}"
I execute this against THIS api and I was able to run this exact call via curl without a problem.... I tried setting the content type and I tried to specify 'json' in the body, but to no avail.
PS, im doing this in Java with the apache OLTU OAuth libraries.
Here is my code:
public String writeFile(String repoPath, String contents, String branchName) throws OAuthSystemException, OAuthProblemException {
String urlPath = String.format("/repos/%s/%s/contents/%s", repoOwner, repoName, repoPath);
String message = String.format("[Blah] Added file '%s'", repoPath);
GitHubCreateFileRequest gitHubCreateFileRequest = new GitHubCreateFileRequest(repoPath, message, contents, branchName);
OAuthClientRequest bearerClientRequest = buildRequest(urlPath);
OAuthResourceResponse resp = performRequest(bearerClientRequest, "PUT", new Gson().toJson(gitHubCreateFileRequest));
if (resp.getResponseCode() >= 200 && resp.getResponseCode() < 299) {
return branchName;
}
return null;
}
private OAuthClientRequest buildRequest(String urlPath) throws OAuthSystemException {
return new OAuthBearerClientRequest(String.format("%s%s", githubUrlPrefix, urlPath))
.setAccessToken(GITHUB_OAUTH_TOKEN).buildHeaderMessage();
}
private OAuthResourceResponse performRequest(OAuthClientRequest bearerClientRequest, String verb, String body) throws OAuthProblemException, OAuthSystemException {
if (body != null) {
bearerClientRequest.setBody(body);
}
bearerClientRequest.addHeader("Content-Type", "application/json");
return oAuthClient.resource(bearerClientRequest, verb, OAuthResourceResponse.class);
}
Any ideas?
PPS
Im setting the Authorization header to bearer abcd1234.... rather than a queryparam or body

Related

OkHttp: A simple GET request: response.body().string() returns unreadable escaped unicode symbols inside json can't convert to gson

When sending a request in Postman, I get this output:
{
"valid": false,
"reason": "taken",
"msg": "Username has already been taken",
"desc": "That username has been taken. Please choose another."
}
However when doing it using okhttp, I get encoding problems and can't convert the resulting json string to a Java object using gson.
I have this code:
public static void main(String[] args) throws Exception {
TwitterChecker checker = new TwitterChecker();
TwitterJson twitterJson = checker.checkUsername("dogster");
System.out.println(twitterJson.getValid()); //NPE
System.out.println(twitterJson.getReason());
System.out.println("Done");
}
public TwitterJson checkUsername(String username) throws Exception {
HttpUrl.Builder urlBuilder = HttpUrl.parse("https://twitter.com/users/username_available").newBuilder();
urlBuilder.addQueryParameter("username", username);
String url = urlBuilder.build().toString();
Request request = new Request.Builder()
.url(url)
.addHeader("Content-Type", "application/json; charset=utf-8")
.build();
OkHttpClient client = new OkHttpClient();
Call call = client.newCall(request);
Response response = call.execute();
System.out.println(response.body().string());
Gson gson = new Gson();
return gson.fromJson(
response.body().string(), new TypeToken<TwitterJson>() {
}.getType());
}
Which prints this:
{"valid":false,"reason":"taken","msg":"\u0414\u0430\u043d\u043d\u043e\u0435 \u0438\u043c\u044f \u0443\u0436\u0435 \u0437\u0430\u043d\u044f\u0442\u043e","desc":"\u0414\u0430\u043d\u043d\u043e\u0435 \u0438\u043c\u044f \u0443\u0436\u0435 \u0437\u0430\u043d\u044f\u0442\u043e. \u041f\u043e\u0436\u0430\u043b\u0443\u0439\u0441\u0442\u0430, \u0432\u044b\u0431\u0435\u0440\u0438\u0442\u0435 \u0434\u0440\u0443\u0433\u043e\u0435."}
and then throws a NullPointerException when trying to access a twitterJson. Debugger shows that object as being null.
TwitterJson:
#Generated("net.hexar.json2pojo")
#SuppressWarnings("unused")
public class TwitterJson {
#Expose
private String desc;
#Expose
private String msg;
#Expose
private String reason;
#Expose
private Boolean valid;
public String getDesc() {
return desc;
}
public String getMsg() {
return msg;
}
public String getReason() {
return reason;
}
public Boolean getValid() {
return valid;
}
...
How can I fix the encoding issues with okhttp?
It is because the response object can be consumed only once. OKHTTP says that in their documentation. After the execute is invoked, you are calling the response object twice. Store the result of response.body().string() to a variable and then do the convert into GSON.
If I were to use a hello world example...
private void testOkHttpClient() {
OkHttpClient httpClient = new OkHttpClient();
try {
Request request = new Request.Builder()
.url("https://www.google.com")
.build();
Call call = httpClient.newCall(request);
Response response = call.execute();
System.out.println("First time " + response.body().string()); // I get the response
System.out.println("Second time " + response.body().string()); // This will be empty
} catch (IOException e) {
e.printStackTrace();
}
}
The reason it is empty the second time is because the response object can be consumed only once. So you either
Return the response as it is. Do not do a sysOut
System.out.println(response.body().string()); // Instead of doing a sysOut return the value.
Or
Store the value of the response to a JSON then convert it to GSON and then return the value.
EDIT: Concerning Unicode characters. It turned out since my location is not an English-speaking country, the json i was accepting was not in English as well. I added this header:
.addHeader("Accept-Language", Locale.US.getLanguage())
to the request to fix that.

Requesting Twitter api with OAuth 1.0

I'm trying to use Twitter's friends list api and was successful to do so without any parameters.
However whenever I add a parameter, I would get the error "Could not authenticate you." and I have no choice but to add a cursor parameter when the friend list is too long.
The fact that I get a list of users of friends when I call the api without any parameters makes me think that authenticating the request works properly.
I have tried to change the request url to https://api.twitter.com/1.1/friends/list.json?cursor=-1 which gives me the authentication error.
I tried using both https://api.twitter.com/1.1/friends/list.json and https://api.twitter.com/1.1/friends/list.json?cursor=-1 to make oauth_signature and they both failed me.
I tried using different parameters such as screen_name or user_id and they all will give me the same error.
I even tried to add cursor: -1 header like a POST request and that didn't work either.
Right now my code looks like this
public String getFriendList() {
String baseUrl = "https://api.twitter.com/1.1/friends/list.json";
// Creates a map with all necessary headers
Map<String, String> headers = createMap();
headers.put("oauth_token", <OAuth token of user>);
String signature = createSignature("GET", baseUrl, headers, <OAuth secret of user>);
// Add oauth_signature to header
headers.put("oauth_signature", signature);
String body = sendGetRequest(baseUrl, headers);
return body;
}
public String sendGetRequest(String baseUrl, Map<String, String> parameters) throws AuthException, IOException {
try (CloseableHttpClient client = CloseableHttpClientFactory.getHttpClient()) {
HttpGet httpGet = new HttpGet(baseUrl);
if (parameters != null) {
httpGet.setHeader("Authorization", createHeader(parameters));
}
CloseableHttpResponse response = client.execute(httpGet);
if (response.getStatusLine().getStatusCode() != 200) {
LOGGER.info("GET Request Failed : " + EntityUtils.toString(response.getEntity()));
throw new Exception();
}
String responseBody = EntityUtils.toString(response.getEntity());
return responseBody;
}
}
which is the working code.
Could anyone tell me where to add parameters and what I have missed to authenticate the request?
EDIT : Added code of sendGetRequest. Making the signature and adding the header was made by following the documentations from twitter

How to send multiple HTTP requests from the same service in Java?

Java noob here. I'm trying to develop a web service as per the following diagram.
When a POST request is sent to the REST server, with certain values, the values (being read from a list, in a loop) get inserted in a table (new row with an id). Server returns HTTP 202 Accepted.
To ensure that the resource(with id from 1) is created, a GET request is issued that returns the POJO as Json.
Finally a PATCH request is sent to update a certain column.
I have written a service class that does all three tasks when each API is called individually. I need to implement something that would automatically execute steps 2 and 3 when a POST request is sent to the server. Here's my code so far.
#Path("attachments")
public class FilesService {
private TiedostoService tiedostoService;
private AttachmentService attachmentService;
#GET
#Path("{id}")
#Produces({MediaType.APPLICATION_JSON})
public Response listAttachmentsAsJson(#PathParam("id") Integer attachmentId) throws Exception {
attachmentService = new AttachmentService();
Gson gson = new GsonBuilder().excludeFieldsWithoutExposeAnnotation().create();
Attachment attachment = attachmentService.getAttachment(attachmentId);
String jsonString = gson.toJson(attachment.toString());
return Response.status(Response.Status.OK).entity(jsonString).build();
}
#PATCH
#Path("{id}")
#Produces({MediaType.APPLICATION_JSON})
public Response patchAttachments(#PathParam("id") Integer attachmentId) throws Exception {
attachmentService = new AttachmentService();
Integer update = attachmentService.update(attachmentId);
String jsonString = new Gson().toJson(update);
return Response.status(Response.Status.ACCEPTED).entity(jsonString).build();
}
#POST
#Produces({MediaType.APPLICATION_JSON})
public Response migrateToMinio(#Context UriInfo uriInfo) throws Exception {
Response response;
List<String> responseList = new ArrayList<>();
tiedostoService = new TiedostoService();
attachmentService = new AttachmentService();
List<Tiedosto> tiedostoList = tiedostoService.getAllFiles();
String responseString = null;
int i = 1;
for (Tiedosto tiedosto : tiedostoList) {
Attachment attachment = new Attachment();
attachment.setCustomerId(tiedosto.getCustomerId());
attachment.setSize(tiedosto.getFileSize());
Integer id = attachmentService.createNew(attachment);
if (id == 1) {
UriBuilder builder = uriInfo.getAbsolutePathBuilder();
builder.path(Integer.toString(i));
response = Response.created(builder.build()).build();
System.out.println(response);
responseString = response.toString();
}
responseList.add(responseString);
i++;
}
String jsonString = new Gson().toJson(responseList);
return Response.status(Response.Status.OK).entity(jsonString).build();
}
}
when I test the individual endpoints with curl or postman, they work as expected, but I got stuck on how to execute GET and PATCH automatically after POST. I'd really appreciate some advice/suggestions/help.

As a body HttpServletRequest always returns CoyoteInputStream and not actual body

I'm trying to write a method that checks user credentials and if these are correct parses sent JSON. It is working fine but I cannot access JSON. In my code there is a command InputStream inputStream = request.getInputStream(); that should read JSON but every time it returns org.apache.catalina.connector.CoyoteInputStream#3f1e9348. Please have a look at my code:
#POST
#Path("auth")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.TEXT_HTML)
public String controller(#Context HttpServletRequest request) {
String authorization = request.getHeader("Authorization");
if (authorization == null) {
authorization = request.getHeader("authorization");
}
String basicHeader = "basic";
if (authorization != null && authorization.toLowerCase().startsWith(basicHeader)) {
String base64Credentials = authorization.substring(basicHeader.length()).trim();
String credentials = new String(Base64.getDecoder().decode(base64Credentials),
Charset.forName("UTF-8"));
String[] values = credentials.split(":", 2);
}
try {
InputStream inputStream = request.getInputStream();
System.out.println(inputStream);
} catch (IOException e) {
e.printStackTrace();
}
}
When I try to use request.getReader() I get the infamous IllegalStateException: getInputStream() has already been called for this request exception. Please see the relevant piece of code:
if ("POST".equalsIgnoreCase(request.getMethod()))
{
try {
String req = request.getReader().lines().collect(Collectors.joining(System.lineSeparator()));
} catch (IOException e) {
e.printStackTrace();
}
}
I use curl to send POST:
curl -u myusername:mypasswor -H "Content-Type: application/json"
-X POST -d '{"username":"xyz","password":"xyz"}' localhost
You can obtain your body content by declaring a parameter on your method:
public String controller(String body, #Context HttpServletRequest request)
But you can also get the JAX-RS implementation to deserialize that JSON to your intended type:
public String controller(MyExpectedType body, #Context HttpServletRequest request)
This should work because you've declared your expected content type, assuming you have a provider that's applicable (such as jackson-jaxrs...).
Regarding the input stream error: this is probably because the container JAXRS implementation has already parsed the request.
But if you were to process it in a normal scenario, such as in a servlet, you'd still have to correct the way you read it:
The documentation for getInputStream state:
Retrieves the body of the request as binary data using a ServletInputStream. Either this method or getReader() may be called to read the body, not both.
This means that to get the content sent by your client in the body, you need to read the stream:
String body = request.getReader().lines()
.collect(Collectors.joining("\n"));
You can also use the Stream-based API:
byte[] bytes = new byte[request.getContentLength()];
request.getInputStream().read(bytes);
String body = new String(bytes); //you may need to specify the character set
The actual input stream class is implementation-based (container-supplied), so you shouldn't need to be concerned with CoyoteInputStream

Java HttpURLConnection status code 302

I'm trying to get this code block to run but I keep getting a 302. I've tried to show the flow of the code. I just don't know what's wrong.
import java.net.HttpURLConnection;
import java.util.HashMap;
import java.util.Map;
import java.util.Base64;
public class AuthenticateLoginLogoutExample {
public static void main(String[] args) throws Exception {
new AuthenticateLoginLogoutExample().authenticateLoginLogoutExample(
"http://" + Constants.HOST + "/qcbin",
Constants.DOMAIN,
Constants.PROJECT,
Constants.USERNAME,
Constants.PASSWORD);
}
public void authenticateLoginLogoutExample(final String serverUrl,
final String domain, final String project, String username,
String password) throws Exception {
RestConnector con =
RestConnector.getInstance().init(
new HashMap<String, String>(),
serverUrl,
domain,
project);
AuthenticateLoginLogoutExample example =
new AuthenticateLoginLogoutExample();
//if we're authenticated we'll get a null, otherwise a URL where we should login at (we're not logged in, so we'll get a URL).
It's this next line when it starts on the isAuthenticated() method.
String authenticationPoint = example.isAuthenticated();
Assert.assertTrue("response from isAuthenticated means we're authenticated. that can't be.", authenticationPoint != null);
//do a bunch of other stuff
}
So we go into the isAuthenticated method:
public String isAuthenticated() throws Exception {
String isAuthenticateUrl = con.buildUrl("rest/is-authenticated");
String ret;
Then here on this next line trying to get the response. con.httpGet
Response response = con.httpGet(isAuthenticateUrl, null, null);
int responseCode = response.getStatusCode();
//if already authenticated
if (responseCode == HttpURLConnection.HTTP_OK) {
ret = null;
}
//if not authenticated - get the address where to authenticate
// via WWW-Authenticate
else if (responseCode == HttpURLConnection.HTTP_UNAUTHORIZED) {
Iterable<String> authenticationHeader =
response.getResponseHeaders().get("WWW-Authenticate");
String newUrl =
authenticationHeader.iterator().next().split("=")[1];
newUrl = newUrl.replace("\"", "");
newUrl += "/authenticate";
ret = newUrl;
}
//Not ok, not unauthorized. An error, such as 404, or 500
else {
throw response.getFailure();
}
return ret;
}
That jumps us to another class and into this method:
public Response httpGet(String url, String queryString, Map<String,
String> headers)throws Exception {
return doHttp("GET", url, queryString, null, headers, cookies);
}
The doHttp takes us here. type = "GET", url = "http://SERVER/qcbin/rest/is-authenticated", the rest are all empty.
private Response doHttp(
String type,
String url,
String queryString,
byte[] data,
Map<String, String> headers,
Map<String, String> cookies) throws Exception {
if ((queryString != null) && !queryString.isEmpty()) {
url += "?" + queryString;
}
HttpURLConnection con = (HttpURLConnection) new URL(url).openConnection();
con.setRequestMethod(type);
String cookieString = getCookieString();
prepareHttpRequest(con, headers, data, cookieString);
This con.connect() on the next line never connects.
con.connect();
Response ret = retrieveHtmlResponse(con);
updateCookies(ret);
return ret;
}
The prepareHttpRequest code:
private void prepareHttpRequest(
HttpURLConnection con,
Map<String, String> headers,
byte[] bytes,
String cookieString) throws IOException {
String contentType = null;
//attach cookie information if such exists
if ((cookieString != null) && !cookieString.isEmpty()) {
con.setRequestProperty("Cookie", cookieString);
}
//send data from headers
if (headers != null) {
//Skip the content-type header - should only be sent
//if you actually have any content to send. see below.
contentType = headers.remove("Content-Type");
Iterator<Entry<String, String>>
headersIterator = headers.entrySet().iterator();
while (headersIterator.hasNext()) {
Entry<String, String> header = headersIterator.next();
con.setRequestProperty(header.getKey(), header.getValue());
}
}
// If there's data to attach to the request, it's handled here.
// Note that if data exists, we take into account previously removed
// content-type.
if ((bytes != null) && (bytes.length > 0)) {
con.setDoOutput(true);
//warning: if you add content-type header then you MUST send
// information or receive error.
//so only do so if you're writing information...
if (contentType != null) {
con.setRequestProperty("Content-Type", contentType);
}
OutputStream out = con.getOutputStream();
out.write(bytes);
out.flush();
out.close();
}
}
And the getCookieString method:
public String getCookieString() {
StringBuilder sb = new StringBuilder();
if (!cookies.isEmpty()) {
Set<Entry<String, String>> cookieEntries =
cookies.entrySet();
for (Entry<String, String> entry : cookieEntries) {
sb.append(entry.getKey()).append("=").append(entry.getValue()).append(";");
}
}
String ret = sb.toString();
return ret;
}
Does anyone have any idea what went wrong? I don't know why it keeps returning a 302.
EDIT: Added chrome developer image as requested.
I haven't followed your entire code, but http 302 means a redirection
https://en.wikipedia.org/wiki/HTTP_302
Depending on the kind of redirection, that could work smoothly or not. For instance the other day I faced a http to https redirection and I have to solve it checking the location header manually.
What I would do is to check first the headers in the browser, in Chrome go to Developer Tools, Network and check the Response Headers (screenshot). You should see there for a 302 response a Location Header, with the new URL you should follow.
302 means there's a page there, but you really want a different page (or you want this page and then that other page). If you look at the headers you get back from the server when it gives you a 302, you'll probably find a "Location:" header telling you where to query next, and you'll have to write yet another transaction.
Browsers interpret the 302 response and automatically redirect to the URL specified in the "Location:" header.

Categories