We Keep Coding

Send JSON body in HTTP GET request in java/spring boot

I need to send a GET request with a json body in java/spring boot. I'm aware of the advice against it, however I have to do it this was for a couple of reasons:
1. The 3rd party API I'm using only allows GET requests, so POST is not an option.
2. I need to pass an extremely large parameter in the body (a comma separated list of about 8-10k characters) so tacking query params onto the url is not an option either.
I've tried a few different things:
apache HttpClient from here: Send content body with HTTP GET Request in Java. This gave some error straight from the API itself about a bad key.
URIComponentsBuilder from here: Spring RestTemplate GET with parameters. This just tacked the params onto the url, which as I explained before is not an option.
restTemplate.exchange. This seemed the most straightforward, but the object wouldn't pass: https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/web/client/RestTemplate.html#exchange-java.lang.String-org.springframework.http.HttpMethod-org.springframework.http.HttpEntity-java.lang.Class-java.util.Map-
as well as probably another thing or two that I've forgotten about.
Here is what I'm talking about in Postman. I need to be able to pass both of the parameters given here. It works fine if run through Postman, but I can't figure it out in Java/Spring Boot.
Here is a code snippet from the restTemplate.exchange attempt:
public String makeMMSICall(String uri, List<String> MMSIBatchList, HashMap<String, String> headersList) {
ResponseEntity<String> result = null;
try {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
for (String key : headersList.keySet()) {
headers.add(key, headersList.get(key));
}
Map<String, String> params = new HashMap<String, String>();
params.put("mmsi", String.join(",", MMSIBatchList));
params.put("limit", mmsiBatchSize);
HttpEntity<?> entity = new HttpEntity<>(headers);
result = restTemplate.exchange(uri, HttpMethod.GET, entity, String.class, params);
System.out.println(result.getBody());
} catch (RestClientException e) {
LOGGER.error("Exception in makeGetHTTPCall :" + e.getMessage());
throw e;
} catch (Exception e) {
LOGGER.error("Exception in makeGetHTTPCall :" + e.getMessage());
throw e;
}
return result.getBody();
}
Thanks for helping!

You can try java.net.HttpUrlConnection, it works for me but indeed I normally use a POST
HttpURLConnection connection = null;
BufferedReader reader = null;
String payload = "body";
try {
URL url = new URL("url endpoint");
if (url.getProtocol().equalsIgnoreCase("https")) {
connection = (HttpsURLConnection) url.openConnection();
} else {
connection = (HttpURLConnection) url.openConnection();
}
// Set connection properties
connection.setRequestMethod(method); // get or post
connection.setReadTimeout(3 * 1000);
connection.setDoOutput(true);
connection.setUseCaches(false);
if (payload != null) {
OutputStream os = connection.getOutputStream();
os.write(payload.getBytes(StandardCharsets.UTF_8));
os.flush();
os.close();
}
int responseCode = connection.getResponseCode();
}

There's no way of implementing it via RestTemplate, even with .exchange method. It'll simply not send the request body for GET calls even if we pass the entity within the function parameters.(Tested via interceptor logs)
You can use the Apache client to solve this issue/request (whatever you'd like to call it). The code you need is something along following lines.
private static class HttpGetWithBody extends HttpEntityEnclosingRequestBase {
JSONObject requestBody;
public HttpGetWithBody(URI uri, JSONObject requestBody) throws UnsupportedEncodingException {
this.setURI(uri);
StringEntity stringEntity = new StringEntity(requestBody.toString());
super.setEntity(stringEntity);
this.requestBody = requestBody;
}
#Override
public String getMethod() {
return "GET";
}
}
private JSONObject executeGetRequestWithBody(String host, Object entity) throws ClientProtocolException, IOException {
CloseableHttpClient httpClient = HttpClients.createDefault();
try{
JSONObject requestBody = new JSONObject(entity);
URL url = new URL(host);
HttpRequest request = new HttpGetWithBody(url.toURI(), requestBody);
request.addHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
request.addHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE);
HttpResponse response;
if(url.getPort() != 0) response = httpClient.execute(new HttpHost(url.getHost(), url.getPort()), request);
else response = httpClient.execute(new HttpHost(url.getHost()), request);
if(response.getStatusLine().getStatusCode() == HttpStatus.SC_OK){
JSONObject res = new JSONObject(EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8));
httpClient.close();
return res;
}
}catch (Exception e){
log.error("Error occurred in executeGetRequestWithBody. Error: ", e.getStackTrace());
}
httpClient.close();
return null;
}
If you inspect even Apache client library doesn't support passing the body natively(checked via code implementation of HttpGet method), since contextually request body for a GET request is not a good and obvious practice.

Try creating a new custom RequestFactory.
Similar to
get request with body

Retrofit: Making Web Requests to Internal APIs

I want to make a request to my organisation api's. The request contains Headers, UserName, Password, & Cookie for session management.
Below is the actual code (in HttpClient) which I want to rewrite using Retrofit. I have heard that HttpClient libraries have been deprecated or someting so have opted Retrofit. I expect the response with 200 status code.
public static CookieStore cookingStore = new BasicCookieStore();
public static HttpContext context = new BasicHttpContext();
public String getAuth(String login,String password) {
String resp = null;
try {
String url = DOMAIN+"myxyzapi/myanything";
context.setAttribute(HttpClientContext.COOKIE_STORE, cookingStore);
HttpClient client = HttpClientBuilder.create().build();
HttpPost post = new HttpPost(url);
String log = URLEncoder.encode(login, "UTF-8");
String pass = URLEncoder.encode(password, "UTF-8");
String json = "username="+log+"&password="+pass+"&maintain=true&finish=Go";
StringEntity entity = new StringEntity(json);
post.setEntity(entity);
post.addHeader("Content-Type", "application/x-www-form-urlencoded");
HttpResponse response = client.execute(post,context);
resp = EntityUtils.toString(response.getEntity());
accountPoller();
} catch(Exception a) {
log.info("Exception in authentication api:"+a.getMessage().toString());
}
return resp;
}
Below is my code where I can't figure out how to pass the context with request. HttpResponse response = client.execute(post,**context**); using retrofit.
I don't even know if I have made my retrofit request right.
try {
String log = URLEncoder.encode(login, "UTF-8");
String pass = URLEncoder.encode(password, "UTF-8");
RequestBody formBody = new FormBody.Builder()
.add("username=", xyz)
.add("password=", mypass)
.add("&maintain=", "true")
.add("finish=", "Go")
.build();
String url = www.xyz.com+"myxyzapi/myanything";
OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder().url(url).post(formBody).addHeader("Content-Type", "application/x-www-form-urlencoded").build();
client.newCall(request).enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
e.printStackTrace();
}
#Override
public void onResponse(Call call, Response response) throws IOException {
if(response.isSuccessful()){
final String myresp = response.body().string();
}
}
});
} catch(Exception a) {
a.getMessage();
}

You have to catch exception and use this class.
retrofit2.HttpException
retrofit2
Class HttpException
int
code()
HTTP status code.
String
message()
HTTP status message.
Response
response()
The full HTTP response.

Spring boot Multipart file upload using Client Side Java Code

I have written a restful web service in spring boot which receives the file.
#RequestMapping(value = "/upload", method = RequestMethod.POST)
#ResponseBody
public void uploadFile(#RequestParam("file") MultipartFile uploadfile) {
System.out.println("filename: " + uploadfile.getName());
}
How can we upload the file from client side java code to web service. Instead of AJAX call or HTML page form multipart request?
The code below call the web service with JSON object. Like this I want to receive the file in above written web service.
void clientRequest(String server_url, JSONObject fileObj){
try {
URL url = new URL(server_url);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
OutputStream os = conn.getOutputStream();
os.write(fileObj.toString().getBytes());
os.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
String output;
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
logger.info("output :: " + output);
}
conn.disconnect();
} catch (Exception e) {
e.printStackTrace();
}
}

You can use Spring's HttpEntity along with ByteArrayResource to upload the file, here is an example:
MultiValueMap<String, Object> data = new LinkedMultiValueMap<String, Object>();
ByteArrayResource resource = new ByteArrayResource(file.getBytes()) {
#Override
public String getFilename() {
return file.getName();
}
};
data.add("file", resource);
HttpHeaders requestHeaders = new HttpHeaders();
requestHeaders.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<MultiValueMap<String, Object>>(data, requestHeaders);
final ResponseEntity<<SomeClass>> responseEntity = restTemplate.exchange(<url>,
HttpMethod.POST, requestEntity, new ParameterizedTypeReference<SomeClass>(){});
SomeClass result = responseEntity.getBody();

If you want to use a MultipartFile, you must use the multipart/form-data mimetype when requesting. Instead of sending the JSON as request entity, you should construct a specific multipart-entity with a single field file in it.
This is how it's done: How can I make a multipart/form-data POST request using Java?

GlassFish 3 - 400 Bad Request on GET/POST/PUT/DELETE

In a script I have, I've created a small and simple REST client. The script itself is a prototype, and therefore the code is not 'production worthy' - so ignore lazy catch expressions and alike.
There are two types of servers that contain the REST service that I fetch data from; either a WildFly 8.2.0 or a GlassFish 3.1.2.2. And the catch here is: My REST client works fine for fetching data from the Wildfly server, but the GlassFish server returns an HTTP 400 Bad Request, for any request.
I can access the REST service for both servers through a web browser, so I know that they are both working properly. I can even do a raw connection though a socket to both servers and they response with the correct data.
So, what could be the reason for GlassFish to not accept the requests?
Socket connection (for testing)
import java.net.Socket;
Socket s = new Socket("localhost", 8080);
String t = "GET /rest/appointment/appointments/search/?fromDate=2016-11-21&branchId=3 HTTP/1.1\nhost: localhost:8080\nAuthorization: Basic base64encodedUsername:PasswordHere\n\n"
OutputStream out = s.getOutputStream();
out.write(t.getBytes());
InputStream inn = s.getInputStream();
Scanner scan = new Scanner(inn);
String line;
while ((line = scan.nextLine()) != null) {
println line;
}
s.close();
REST client code:
import groovy.json.JsonSlurper;
import javax.xml.bind.DatatypeConverter;
/*
REST-client (a very simple one)
*/
public class RESTclient {
public static Object get(URL url, Map<String, String> headers) {
return http(url, "GET", null, headers);
}
public static Object post(URL url, String data, Map<String, String> headers) {
return http(url, "POST", data, headers);
}
public static Object put(URL url, String data, Map<String, String> headers) {
return http(url, "PUT", data, headers);
}
public static Object delete(URL url, String data, Map<String, String> headers) {
return http(url, "DELETE", data, headers);
}
private static Object http(URL url, String method, String data, Map<String, String> headers) {
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("username", "password".toCharArray());
}
});
connection.setRequestMethod(method);
for (String header : headers.keySet()) {
connection.setRequestProperty(header, headers.get(header));
}
if (data != null) {
connection.setRequestProperty("Content-Type", "application/json");
connection.setDoOutput(true);
OutputStream outputStream =connection.getOutputStream();
outputStream.write(data.getBytes());
}
int responseCode = connection.getResponseCode();
switch (responseCode) {
case HttpURLConnection.HTTP_NO_CONTENT:
// This happens when the server doesn't give back content, but all was ok.
return (new HashMap());
case HttpURLConnection.HTTP_OK:
InputStream inputStream = connection.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
String response = reader.readLine();
JsonSlurper parser = new JsonSlurper();
Object jsonResponse = parser.parseText(response); // This can be either a List or a Map
// Close the connection
try { connection.close(); } catch (Exception e) { /* Already closed */ }
return jsonResponse;
default:
println "response code: " + responseCode;
println connection.getResponseMessage();
println connection.getHeaderFields();
// Close the connection
try { connection.close(); } catch (Exception e) { /* Already closed */ }
return null;
}
}
}
Usage:
URL appointmentSearchURL = new URL("http://localhost:8080/rest/appointment/appointments/search/?fromDate=2016-11-21&branchId=3");
Object response = RESTclient.get(appointmentSearchURL, new HashMap<String, String>());
println response;
All that is printed out:
response code: 400
Bad Request
[null:[HTTP/1.1 400 Bad Request], Server:[GlassFish Server Open Source Edition 3.1.2.2], Connection:[close], Set-Cookie:[rememberMe=deleteMe; Path=/; Max-Age=0; Expires=Tue, 22-Nov-2016 08:43:29 GMT, SSOcookie=2a86cf4b-a772-435a-b92e-f12845dc20a2; Path=/; HttpOnly], Content-Length:[1090], Date:[Wed, 23 Nov 2016 08:43:28 GMT], Content-Type:[text/html], X-Powered-By:[Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.2.2 Java/Oracle Corporation/1.7)]]
null

I found my answer! So, I will leave this here if any other stumble across the same issue in the future:
There was a missing Accept header, I guess the server-side only accept json content. I have not researched further on why the WildFly server does not response with a 400 bad request, but I suppose WildFly tries to guess/deduce the incoming data.
So the whole issue was resolved by adding the following:
connection.setRequestProperty("Accept", "application/json");

Android: NTLM Authentication, ksoap, and persistent connections

After working with iOS and dealing with auth challenges without much of a learning curve, I've found that Windows Authentication is much more complicated of a process in Java/Android.
I tried multiple different approaches, so without getting too much into those, I will get to the one that worked for the most part. I'm now using the class created for NTLM and ksoap called NtlmTransport
I'm now successfully authenticating in the following way:
NtlmTransport httpTransport = new NtlmTransport();
httpTransport.setCredentials(serverURL, Login.username, Login.password, deviceIp, "DOMAINNAME");
httpTransport.call(SOAP_ACTION, envelope);
If you take a look at the NtlmTransport class, you'll see that it's returning the following headers from the setupNtlm():
status Line HTTP/1.1 200 OK
Setup Cache-Control:private, max-age=0
Setup Content-Type:text/html; charset=utf-8
Setup Server:Microsoft-IIS/8.0
Setup X-AspNet-Version:4.0.30319
Setup Persistent-Auth:true
Setup X-Powered-By:ASP.NET
Setup Date:Tue, 17 Sep 2013 20:57:45 GMT
Setup Content-Length:11549
The "Persistent-Auth:true is the main one I'm concerned about at this time. I'm getting the SoapObjects just fine and can get the data I need from that one connection, but as soon as I try to access the web service again, which is presumably able to be hit after the successful authentication, I can't access a different method using HttpTransportSE:
private void setSomething() {
xml = null;
final String SOAP_ACTION = "http://this.ismy.org/AWebServiceMethod";
final String METHOD_NAME = "AWebServiceMethod";
final String URL = protocol + "://" + host + ":" + port + "/WebService.asmx";
SoapObject request = new SoapObject(NAMESPACE, METHOD_NAME);
SoapSerializationEnvelope envelope = new SoapSerializationEnvelope(SoapEnvelope.VER11);
envelope.dotNet = true;
envelope.setOutputSoapObject(request);
envelope.implicitTypes = true;
envelope.setAddAdornments(false);
try
{
HttpTransportSE transport = new HttpTransportSE(URL);
transport.debug = true;
transport.call(SOAP_ACTION, envelope);
xml = transport.responseDump.toString();
Log.d(TAG, xml);
}
catch(SocketException ex)
{
Log.e("SocketException : " , "Error on setSomething() " + ex.getMessage());
}
catch (Exception e)
{
Log.e("Exception : " , "Error on setSomething() " + e.getMessage());
}
}
This all works just fine as the background task of an AsyncTask, which then passes the "xml" to an XMLPullParser method.
The main question here is why am I getting a:
Error on setSomething() No authentication challenges found
??
After IIS successfully validates the user with a 200, why is it asking me to authenticate again? How can I persist that first authenticated challenge to hit whatever method I want inside WebService.asmx? What are the headers that need to be added/changed to create a session if necessary? What am I missing that makes this whole NTLM process work and persist for more than the WS method that needs to pass the authentication challenges?
EDIT : Adding the Library code
Here's the link to the JCIFS from Apache
public static final class JCIFSEngine implements NTLMEngine {
private static final int TYPE_1_FLAGS =
NtlmFlags.NTLMSSP_NEGOTIATE_56 |
NtlmFlags.NTLMSSP_NEGOTIATE_128 |
NtlmFlags.NTLMSSP_NEGOTIATE_NTLM2 |
NtlmFlags.NTLMSSP_NEGOTIATE_ALWAYS_SIGN |
NtlmFlags.NTLMSSP_REQUEST_TARGET;
public String generateType1Msg(final String domain, final String workstation)
throws NTLMEngineException {
final Type1Message type1Message = new Type1Message(TYPE_1_FLAGS, domain, workstation);
return jcifs.util.Base64.encode(type1Message.toByteArray());
}
public String generateType3Msg(final String username, final String password,
final String domain, final String workstation, final String challenge)
throws NTLMEngineException {
Type2Message type2Message;
try {
type2Message = new Type2Message(jcifs.util.Base64.decode(challenge));
} catch (final IOException exception) {
throw new NTLMEngineException("Invalid NTLM type 2 message", exception);
}
final int type2Flags = type2Message.getFlags();
final int type3Flags = type2Flags
& (0xffffffff ^ (NtlmFlags.NTLMSSP_TARGET_TYPE_DOMAIN | NtlmFlags.NTLMSSP_TARGET_TYPE_SERVER));
final Type3Message type3Message = new Type3Message(type2Message, Login.password, "",
Login.username, deviceIp, type3Flags);
System.out.println("type3Message: " + type3Message.toByteArray());
return jcifs.util.Base64.encode(type3Message.toByteArray());
}
}
So is the "NtlmFlags.NTLMSSP_NEGOTIATE_ALWAYS_SIGN" causing this problem? Is there another flag I'm supposed to set for the keep-alive? Also, I found a great resource for a list of NTLM flags and more: http://fossies.org/dox/jcifs-1.3.17/interfacejcifs_1_1ntlmssp_1_1NtlmFlags.html

I was also struggling about windows authentication from Android.
I found android-ntlm-master on https://github.com/masconsult/android-ntlm. Add this class as library in your project.
Change is in NtlmTransport.java class.I made change in call method of NtlmTransport class =>
public List call(String soapAction, SoapEnvelope envelope,
List headers, File outputFile)
throws IOException, XmlPullParserException {
HttpResponse resp = null;
try {
//setupNtlm(urlString, user, password);
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getAuthSchemes().register("ntlm", new NTLMSchemeFactory());
httpclient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
new NTCredentials(user, password, "", "")
);
HttpPost httpget = new HttpPost(urlString);
httpget.addHeader("soapaction", soapAction);
httpget.addHeader("Content-Type", "text/xml; charset=utf-8");
byte[] requestData = null;
try {
requestData = createRequestData(envelope);
} catch (IOException iOException) {
}
ByteArrayEntity byteArrayEntity = new ByteArrayEntity(requestData);
httpget.setEntity(byteArrayEntity);
resp = httpclient.execute(httpget);
if(resp == null) {
System.out.println("Response is null");
}
HttpEntity respEntity = resp.getEntity();
InputStream is = respEntity.getContent();
if(is == null) {
System.out.println("InputStream is null");
}
parseResponse(envelope, is);
} catch (Exception ex) {
// ex.printStackTrace();
}
if (resp != null) {
return Arrays.asList(resp.getAllHeaders());
} else {
return null;
}
}
And below is the code how I make call:
SoapObject request = new SoapObject(NAMESPACE, PRODUCT_DETAILS_METHOD_NAME);
request.addProperty("ListingID", Integer.parseInt(Product_ID));
NtlmTransport httpTransport = new NtlmTransport();
httpTransport.setCredentials(URL, USERNAME, PASSWORD, "","");
SoapSerializationEnvelope envelope = new SoapSerializationEnvelope(SoapEnvelope.VER11);
envelope.dotNet = true;
envelope.implicitTypes = true;
envelope.setOutputSoapObject(request);
httpTransport.call(PRODUCT_DETAILS_SOAP_ACTION, envelope);
SoapObject response = (SoapObject) envelope.getResponse();
It worked for me.
More you can find here: https://suhas1989.wordpress.com/2015/01/28/ntlm-authentication-in-android/