We Keep Coding

Attempted read from closed stream

I'm attempted to make a request to the ROBLOX API to rank a user, however after about 15 minutes or so I get the error "attempted to read from closed stream. After researching this error, I close the client after each request, and release the connection from the request. However I still get the error. Any help would be greatly appreciated.
public void rankUser(String username, long rank, SlashCommandEvent exceptionCatcher) throws IOException {
try {
URL url = new URL(apiUrl + "v1/groups/" + group + "/users/" + getUserIdFromUsername(username, exceptionCatcher));
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
HttpPatch request = new HttpPatch(url.toString());
try {
request.addHeader("Cookie", ".ROBLOSECURITY=" + token);
request.addHeader("X-CSRF-Token", xcsrfToken);
request.addHeader("Content-Type", "application/json");
StringEntity params = new StringEntity("{\"roleId\":" + rank + "}");
request.setEntity(params);
HttpResponse response = httpClient.execute(request);
if(response.getHeaders("x-csrf-token") != null && xcsrfToken.equalsIgnoreCase("")) {
xcsrfToken = response.getHeaders("x-csrf-token")[0].getValue();
rankUser(username, rank, exceptionCatcher);
return;
}
request.releaseConnection();
statusCode = response.getStatusLine().getStatusCode();
if (statusCode == 200) {
String responseAsString = getResponseAsString(response);
JsonParser parser = new JsonParser();
JsonObject jsonObject = (JsonObject) parser.parse(responseAsString);
} else if(statusCode == 429) {
exceptionCatcher.reply("You are being rate limited. Please try again in a few minutes.").setEphemeral(true).queue();
return;
} else {
System.out.println(getResponseAsString(response));
exceptionCatcher.reply("Could not rank user. Response: " + getResponseAsString(response)).setEphemeral(true).queue();
return;
}
httpClient.close();
} catch (Exception e) {
exceptionCatcher.reply("Could not rank. Ensure that the user is in the group. Error: " + e.getMessage()).setEphemeral(true).queue();
httpClient.close();
e.printStackTrace();
}
} catch (Exception e) {
e.printStackTrace();
}
}

Mocking httpclient v4 in java Mockito

I hava a java apache client written as :
public JSONObject createProject(String accessToken) throws PostEditException {
JSONObject jobj = new JSONObject();
HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost("https://"+HOST_TRANSPORT+"/api/projects/create");
try{
post.addHeader("Content-Type","application/json");
post.addHeader("Accept","application/json");
String authorizationValue = "Bearer "+accessToken;
post.addHeader("Authorization",authorizationValue);
HttpResponse response = client.execute(post);
if (response == null) {
System.out.println("this is null");
}
int code = response.getStatusLine().getStatusCode();
if (code != 200) {
HttpEntity entity = response.getEntity();
ContentType contentType = ContentType.getOrDefault(entity);
Charset charset = contentType.getCharset();
if (charset == null) {
charset = Charset.forName("UTF-8");
}
String errorBody = EntityUtils.toString(entity, charset);
log.info("createProject: Unable to process request. HTTP code: " + code + " responseBody: " + errorBody);
throw new PostEditException("createProject: Unable to process createProject request. HTTP code: " + code + " responseBody: " + errorBody);
} else {
HttpEntity entity = response.getEntity();
ContentType contentType = ContentType.getOrDefault(entity);
Charset charset = contentType.getCharset();
if (charset == null) {
charset = Charset.forName("UTF-8");
}
String taggedResult = EntityUtils.toString(entity, charset);
jobj = new JSONObject(taggedResult) ;
}
}catch (NoRouteToHostException re) {
log.error("createProject: unable to route to host."+re);
throw new PostEditException("createProject: unable to route to host.",re);
}catch (IOException ie) {
log.error("createProject: problem executing HTTP request."+ ie);
throw new PostEditException("createProject: problem executing HTTP request.",ie);
}catch (Exception e) {
log.error("createProject: an error occurred." +e );
throw new PostEditException("createProject: an error occurred",e);
} finally {
post.releaseConnection();
}
return jobj;
} //createProject
so , I am trying to write a test case to mock the HttpClient.
so I Wrote the Mockito test case as :
PowerMockito.mockStatic(HttpClientBuilder.class);
HttpClientBuilder builderMock = Mockito.mock(HttpClientBuilder.class);
PowerMockito.doReturn(builderMock).when(HttpClientBuilder.class, "create");
HttpClient clientMock = Mockito.mock(CloseableHttpClient.class);
CloseableHttpResponse responseMock = Mockito.mock(CloseableHttpResponse.class);
Mockito.doReturn(clientMock).when(builderMock).build();
Mockito.doReturn(responseMock).when(clientMock).execute(Mockito.any());
classname obj= new classname();
Method m = classname.class.getDeclaredMethod("createProject", String.class);
JSONObject result =(JSONObject) m.invoke(obj,"accesstoken");
But when I debug the program, I see the real httpclient is created instead of the mocked one. How Do I create the mock httpclient so that I can mock the status and response as well.

How to check in/out files to SharePoint using httpclient in java?

I wrote a programm that can up-/download documents to sharepoint and check them in/out. It is used for data integration purposes and works quite well.
It was implemented using SOAP, but unfortunately the Server is configured to only be able to handle files with a size lesser than 50MB via SOAP.
The server configuration is fixed, so I have to work around that.
I added some code and I am able to up/download bigger files now, but If I want to check them in via SOAP I get the same error.
Now I wonder If it is possible to checkin/out files using the httpclient.
My code so far...
public class HttpClient {
private static final Logger LOGGER = LogManager.getLogger(HttpClient.class.getName());
HttpClient() {
}
public static void download(final String source, final File resultingFile) {
CloseableHttpClient client = WinHttpClients.createSystem();
HttpGet httpRequest = new HttpGet(source);
CloseableHttpResponse httpResponse = null;
try {
httpResponse = client.execute(httpRequest);
HttpEntity entity = httpResponse.getEntity();
if(httpResponse.getStatusLine() != null && httpResponse.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
LOGGER.warn(httpResponse.getStatusLine());
}else {
LOGGER.debug(httpResponse.getStatusLine());
FileUtils.touch(resultingFile);
InputStream is = entity.getContent();
File outFile = new File(resultingFile.getAbsolutePath());
FileOutputStream fos = new FileOutputStream(outFile);
int inByte;
while ((inByte = is.read()) != -1) {
fos.write(inByte);
}
is.close();
fos.close();
client.close();
}
} catch (ClientProtocolException e) {
LOGGER.warn(e);
} catch (UnsupportedOperationException e) {
LOGGER.warn(e);
} catch (IOException e) {
LOGGER.warn(e);
}
}
public static void upload(final File source, final String destination) {
CloseableHttpClient httpclient = WinHttpClients.createSystem();
HttpPut httpRequest = new HttpPut(destination);
httpRequest.setEntity(new FileEntity(new File(source.getPath())));
CloseableHttpResponse httpResponse = null;
try {
httpResponse = httpclient.execute(httpRequest);
EntityUtils.consume(httpResponse.getEntity());
if (httpResponse.getStatusLine() != null && httpResponse.getStatusLine().getStatusCode() == HttpStatus.SC_CREATED) {
LOGGER.debug(httpResponse.getStatusLine());
LOGGER.info("Upload of " + source.getName() + " via HTTP-Client succeeded.");
} else if (httpResponse.getStatusLine() != null && httpResponse.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
LOGGER.debug(httpResponse.getStatusLine());
}else {
LOGGER.warn("Uploading " + source.getName() + " failed.");
LOGGER.warn(httpResponse.getStatusLine().getStatusCode() + ": " + httpResponse.getStatusLine().getReasonPhrase());
}
} catch (IOException e) {
LOGGER.warn(e);
LOGGER.warn(e.getMessage());
}
return;
}
}

Deal with huge JSON responses

I think I need to rewrite some modules of my app because when the number of entities that are rendered increases, it fails and errors too. At this moment, I'm using Jackson and HttpClient. As much as I trust in Jackson, something tells me that the problem is the second lib. Can HttpClient deal with large responses? (e.g. this one that is about 400 lines)
Besides that, in my app, the way I parse the request goes something like this:
public Object handle(HttpResponse response, String rootName) {
try {
String json = EntityUtils.toString(response.getEntity());
// better "new BasicResponseHandler().handleResponse(response)" ????
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode >= 200 && statusCode < 300 ) {
return createObject(json, rootName);
}
else{
return null;
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public Object createObject (String json, String rootName) {
try {
this.root = this.mapper.readTree(json);
String className = Finder.findClassName(rootName);
Class clazz = this.getObjectClass(className);
return mapper.treeToValue(root.get(rootName), clazz);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
How I can improve this piece of code to be more efficient with large responses?
Thanks in advance!

There's no need to create the String json, as ObjectMapper#readTree can accept an InputStream as well. For example, this will be slightly more efficient:
public Object handle(HttpResponse response, String rootName) {
try {
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode >= 200 && statusCode < 300 ) {
return createObject(response.getEntity().getContent(), rootName);
}
else{
return null;
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public Object createObject (InputStream json, String rootName) {
try {
this.root = this.mapper.readTree(json);
String className = Finder.findClassName(rootName);
Class clazz = this.getObjectClass(className);
return mapper.treeToValue(root.get(rootName), clazz);
} catch (Exception e) {
throw new RuntimeException(e);
}
}

Ive had 1000+ line json responses handled without issue, so that shouldnt be a problem. As for better ways of doing it, Google GSON is amazing, itll map your json to you java object with no special parsing code whatsoever.

I guess you could read data to a good old StringBuffer. Something like
HttpEntity httpEntity = httpResponse.getEntity();
if (httpEntity != null) {
InputStream is = AndroidHttpClient.getUngzippedContent(httpEntity);
br = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder(8192);
String s;
while ((s = br.readLine()) != null) sb.append(s);
}

Is there a way to get the String value of an HttpEntity when EntityUtils.toString() returns an exception?

I keep running into this situation where I get back a bad HTTP response (like a 400) but cannot look at the HttpEntity in the HttpResponse object. When I step through with the debugger, I can see that the entity has content (length > 0) and I can even look at the content, but all I see is an array of numbers (ASCII codes I guess?) which isn't helpful. I'll call EntityUtils.toString() on the entity, but I get back an exception -- either an IOException, or some kind of "object is in an invalid state" exception. This is really frustrating! Is there any way to get at this content in a human-readable form?
Here is my code :
protected JSONObject makeRequest(HttpRequestBase request) throws ClientProtocolException, IOException, JSONException, WebRequestBadStatusException {
HttpClient httpclient = new DefaultHttpClient();
try {
request.addHeader("Content-Type", "application/json");
request.addHeader("Authorization", "OAuth " + accessToken);
request.addHeader("X-PrettyPrint", "1");
HttpResponse response = httpclient.execute(request);
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode < 200 || statusCode >= 300) {
throw new WebRequestBadStatusException(statusCode);
}
HttpEntity entity = response.getEntity();
if (entity != null) {
return new JSONObject(EntityUtils.toString(entity));
} else {
return null;
}
} finally {
httpclient.getConnectionManager().shutdown();
}
}
See where I throw the exception? What I'd like to do is suck out the content of the HttpEntity and put it in the exception.

Appache has already provided a Util class for that called EntityUtils.
String responseXml = EntityUtils.toString(httpResponse.getEntity());
EntityUtils.consume(httpResponse.getEntity());

Here's some code to view the entity as a string (given that your request contentType is html or similar):
String inputLine ;
BufferedReader br = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent()));
try {
while ((inputLine = br.readLine()) != null) {
System.out.println(inputLine);
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}

To enable a human readable form you can convert HttpEntity to string with UTF-8 code
EntityUtils.toString(response.getEntity(), "UTF-8")
This will give you Response parameters in json form like:
{ "error": { "errors": [ { "domain": "global", "reason": "forbidden", "message": "Forbidden" } ], "code": 403, "message": "Forbidden" }}
Hope this solves the issue.

In general if you want to convert your DTO to String format then you can use ObjectMapper. Please find the following example if it helpful.
public static String getObjectAsString(Object object) {
ObjectMapper mapper = new ObjectMapper();
try {
return mapper.writeValueAsString(object);
} catch (Exception e) {
return null;
}
}