We Keep Coding

Prevent Spring's RestTemplate from adding header for each parameters in multipart/form-data

I have to use Spring's RestTemplate to call an external API that takes a POST request with Content-Type: multipart/form-data. The input data are only key-values, no attachments but the server enforce me the use multipart/form-data.
Following is the raw request that works fine.
POST http://the-api:8080 HTTP/1.1
Content-Type: multipart/form-data; boundary=--Eh0oKOHPOSEIJTzFevDxHhPNKhQl7AP6kQL
Accept: */*
Host: the-api:8080
accept-encoding: gzip, deflate
content-length: 680
Connection: keep-alive
--Eh0oKOHPOSEIJTzFevDxHhPNKhQl7AP6kQL
Content-Disposition: form-data; name="param1"
value1
--Eh0oKOHPOSEIJTzFevDxHhPNKhQl7AP6kQL
Content-Disposition: form-data; name="param2"
value2
--Eh0oKOHPOSEIJTzFevDxHhPNKhQl7AP6kQL--
Following is the raw request that I extracted and rearranged from the log of the RestTemplate, it did not work because the server mistook the header for the value.
POST http://the-api:8080 HTTP/1.1
Content-Type: multipart/form-data; boundary=--Eh0oKOHPOSEIJTzFevDxHhPNKhQl7AP6kQL
Accept: */*
Host: the-api:8080
accept-encoding: gzip, deflate
content-length: 680
Connection: keep-alive
--Eh0oKOHPOSEIJTzFevDxHhPNKhQl7AP6kQL
Content-Disposition: form-data; name="param1"
Content-Type: text/plain;charset=UTF-8
Content-Length: 29
value1
--Eh0oKOHPOSEIJTzFevDxHhPNKhQl7AP6kQL
Content-Disposition: form-data; name="param2"
Content-Type: text/plain;charset=UTF-8
Content-Length: 14
value2
--Eh0oKOHPOSEIJTzFevDxHhPNKhQl7AP6kQL--
Following is the code
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
MultiValueMap<String, String> params = new LinkedMultiValueMap<>();
params.add("param1", "value1);
params.add("param2", "value2);
HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<>(params, headers);
URI uri = UriComponentsBuilder.fromHttpUrl("http://the-api:8080")
.build().encode(Charset.forName("UTF-8")).toUri();
return restTemplate.postForObject(uri, request, KKPMailResponse.class);
Question
How to prevent Spring's RestTemplate from automatically add the header Content-Type: text/plain;charset=UTF-8 and Content-Length: xx for each parameters

I didn't find a way to prevent Spring from generating the entries, but you can use an interceptor to remove them before sending the request. For that you have to manipulate the request body in the interceptor as follows:
public class MultiPartFormDataCleaningInterceptor implements ClientHttpRequestInterceptor {
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
final MediaType contentType = request.getHeaders().getContentType();
if (contentType != null
&& MediaType.MULTIPART_FORM_DATA.getType().equals(contentType.getType())
&& MediaType.MULTIPART_FORM_DATA.getSubtype().equals(contentType.getSubtype())) {
return execution.execute(request, stripContentTypeAndLength(body));
}
return execution.execute(request, body);
}
private byte[] stripContentTypeAndLength(byte[] body) {
final String bodyStr = new String(body);
final StringBuilder builder = new StringBuilder();
try (final Scanner scanner = new Scanner(bodyStr)) {
while (scanner.hasNextLine()) {
final String line = scanner.nextLine();
if (!line.startsWith("Content-Type:")
&& !line.startsWith("Content-Length:")) {
builder.append(line).append("\r\n");
}
}
}
final String newBodyStr = builder.toString();
return newBodyStr.getBytes();
}
}

If think you can use ClientHttpRequestInterceptor to remove headers:
public class SomeHttpRequestInterceptor implements ClientHttpRequestInterceptor
{
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException
{
HttpHeaders headers = request.getHeaders();
headers.remove("your header 1);
headers.remove("your header 2);
return execution.execute(request, body);
}
}
And set it in RestTemplate in this way:
RestTemplate restTemplate = new RestTemplate();
List<ClientHttpRequestInterceptor> interceptors = Arrays.asList(new CustomHttpRequestInterceptor())
restTemplate.setInterceptors(interceptors);

Apache HttpClient 4.5.5 - wrong headers in response after a GET request

When I'm using Apache HttpClient and loading a webpage via GET request after the page is loaded in the response I have the headers that are different from ones I have when loading the same page in browser. Here is the example of the page: http://empoweredfoundation.org/wp-login.php?action=register
In browser I have the following headers:
Status code: 302
Content-Type: text/html; charset=UTF-8
X-Port: port_10210
X-Cacheable: YES:Forced
Location: http://empoweredfoundation.org/register/
Content-Encoding: gzip
Transfer-Encoding: chunked
Date: Thu, 22 Feb 2018 04:04:35 GMT
Age: 0
Vary: User-Agent
X-Cache: uncached
X-Cache-Hit: MISS
X-Backend: all_requests
When I use HttpClient in my application I have these headers in response:
Status code: 200
Content-Type: text/html; charset=UTF-8
X-Port: port_10210
X-Cacheable: YES:Forced
Transfer-Encoding: chunked
Date: Thu, 22 Feb 2018 04:44:58 GMT
Age: 28224
Vary: Accept-Encoding, User-Agent
X-Cache: cached
X-Cache-Hit: HIT
X-Backend: all_requests
Server: nginx/1.12.1
Date: Thu, 22 Feb 2018 04:45:24 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.45
So, I should have a 302 status code but I have 200. And also I see other headers are different than the ones from the browser. I can't figure out why and what should I do to fix this.
Here is the code:
HttpClient httpclient = null;
HttpClientBuilder builder = HttpClients.custom();
Builder requestConfigBuilder = RequestConfig.custom();
// here goes the cookie store creation, ssl configuration etc
builder.setDefaultRequestConfig(requestConfigBuilder.build());
httpclient = builder.build();
HttpResponse response = null;
HttpGet httpget = null;
Escaper escaper = UrlEscapers.urlFragmentEscaper();
httpget = new HttpGet(escaper.escape(url));
httpget.getParams().setParameter("http.socket.timeout", new Integer(socketTimeout));
httpget.getParams().setParameter("http.connection.timeout", new Integer(connectTimeout));
httpget.addHeader("Accept", "text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1");
httpget.addHeader("Accept-Language", "en-US,en;q=0.9");
httpget.addHeader("Accept-Encoding", "identity, *;q=0");
response = httpclient.execute(httpget);
I also tried CloseableHttpClient, had the same result.

I resolved this issue, this solution works: https://memorynotfound.com/apache-httpclient-redirect-handling-example/
I still have 200 status code, not 302. But now I can handle 302 redirects (even when response.getStatusLine() shows 200).
Here is the code from the article:
package com.memorynotfound.httpclient;
import org.apache.http.HttpHost;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.client.utils.URIUtils;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.client.LaxRedirectStrategy;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.List;
/**
* This example demonstrates the use of {#link HttpGet} request method.
* and handling redirect strategy with {#link LaxRedirectStrategy}
*/
public class HttpClientRedirectHandlingExample {
public static void main(String... args) throws IOException, URISyntaxException {
CloseableHttpClient httpclient = HttpClients.custom()
.setRedirectStrategy(new LaxRedirectStrategy())
.build();
try {
HttpClientContext context = HttpClientContext.create();
HttpGet httpGet = new HttpGet("http://httpbin.org/redirect/3");
System.out.println("Executing request " + httpGet.getRequestLine());
System.out.println("----------------------------------------");
httpclient.execute(httpGet, context);
HttpHost target = context.getTargetHost();
List<URI> redirectLocations = context.getRedirectLocations();
URI location = URIUtils.resolve(httpGet.getURI(), target, redirectLocations);
System.out.println("Final HTTP location: " + location.toASCIIString());
} finally {
httpclient.close();
}
}
}
And also I added builder.setRedirectStrategy(new LaxRedirectStrategy()); when created the HttpClient class object.
If you know any solution to get the correct status code (which should be 302), please tell me.

Retrofit can't get data after log in

I need to log in ip/api/login with parameters email, password and then I can retrieve data from ip/api/async. So far i can only log in and retrieve first call. On second app is getting SocketTimeoutException
class talkToWebSite extends AsyncTask<String, Void, String> {
protected String doInBackground(String... strings) {
OkHttpClient client = new OkHttpClient();
HttpLoggingInterceptor interceptor = new HttpLoggingInterceptor();
interceptor.setLevel(HttpLoggingInterceptor.Level.BODY);
client.interceptors().add(interceptor);
Retrofit retrofit = new Retrofit.Builder()
.baseUrl("http://ip/")
.client(client)
.addConverterFactory(GsonConverterFactory.create())
.build();
TraccarApi stackOverflowAPI = retrofit.create(TraccarApi.class);
Call<Login> call1 = stackOverflowAPI.postUser("admin", "admin");
try {
call1.execute();
} catch (IOException e) {
e.printStackTrace();
}
Call<Data> call = stackOverflowAPI.getData();
Response<Data> response = null;
try {
response = call.execute();
} catch (IOException e) {
e.printStackTrace();
}
return response.body().getDataset().getData_latitude() + "";
}
protected void onPostExecute(String result) {
longitiude.setText(result);
}
}
When I am running that code I receive it:
OkHttp: --> GET /api/login?email=admin&password=admin HTTP/1.1
OkHttp: --> END GET
OkHttp: <-- HTTP/1.1 200 OK (73ms)
OkHttp: Date: Sat, 28 Nov 2015 22:19:12 GMT
OkHttp: Content-Type: application/json; charset=UTF-8
OkHttp: Access-Control-Allow-Origin: *
OkHttp: Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept
OkHttp: Access-Control-Allow-Methods: GET, POST
OkHttp: Set-Cookie: JSESSIONID=1fk50j768xc0v1w7tb8inf29nc;Path=/api
OkHttp: Expires: Thu, 01 Jan 1970 00:00:00 GMT
OkHttp: Content-Length: 189
OkHttp: Server: Jetty(9.2.14.v20151106)
OkHttp: OkHttp-Selected-Protocol: http/1.1
OkHttp: OkHttp-Sent-Millis: 1448752725041
OkHttp: OkHttp-Received-Millis: 1448752725091
OkHttp: {"success":true,"data":{"name":"admin","language":"","id":1,"map":"","readonly":false,"distanceUnit":"","speedUnit":"","latitude":0.0,"longitude":0.0,"email":"admin","admin":true,"zoom":0}}
OkHttp: <-- END HTTP (189-byte body)
OkHttp: --> GET /api/async/ HTTP/1.1
OkHttp: --> END GET
Thanks in advice.

To retrieve data from post which need authorization I need add it:
String credentials = "login:password";
final String basic = "Basic " +
Base64.encodeToString(credentials.getBytes(), Base64.NO_WRAP);
client.interceptors().add(new Interceptor() {
#Override
public com.squareup.okhttp.Response intercept(Interceptor.Chain chain) throws IOException {
Request original = chain.request();
Request.Builder requestBuilder = original.newBuilder()
.header("Authorization", basic)
.method(original.method(), original.body());
Request request = requestBuilder.build();
return chain.proceed(request);
}
});

Android Volley with GSON Success but return null values

I have a problem using Volley with GSON. There is no error/exception but the result returned by webservice is always null.
This is my webservice setting :
Status
200 OK Show explanation Loading time: 59
Request headers
CSP: active
Origin: chrome-extension://hgmloofddffdnphfgcellkdfbfbjeloo
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8,id;q=0.6
Response headers
Date: Wed, 24 Jun 2015 02:37:06 GMT
Server: Apache/2.4.10 (Win32) OpenSSL/1.0.1i PHP/5.5.19
X-Powered-By: PHP/5.5.19
Content-Length: 81
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
I'm sure the webservice is working fine, so the problem is in the Java code.
This is my custom request, taken from here :
#Override
protected Response<T> parseNetworkResponse(NetworkResponse response) {
try {
Log.e("gson", "test");
String json = new String(
response.data,
HttpHeaderParser.parseCharset(response.headers));
Response result = Response.success(
gson.fromJson(json, gsonClass),
HttpHeaderParser.parseCacheHeaders(response));
return result;
} catch (UnsupportedEncodingException e) {
Log.e("gson", e.getLocalizedMessage()); //never printed
return Response.error(new ParseError(e));
} catch (JsonSyntaxException e) {
Log.e("gson", e.getLocalizedMessage()); //never printed
return Response.error(new ParseError(e));
}
}
And this is how i call the Volley, the Customer in onResponse is null :
RequestQueue queue = Volley.newRequestQueue(getActivity());
String url = "https://sister-8.tafinance.com/cust_gathering/index.php/customer/get/format/json";
final Map<String,String> header = new HashMap<String, String>();
//header.put("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
header.put("name", "test1");
header.put("born_date", "1970-06-15 00:00:00.000");
GsonRequest request = new GsonRequest(Request.Method.POST, url, Customer.class, header,
new Response.Listener<Customer>()
{
#Override
public void onResponse(Customer customer) {
txtTest.setText(customer.getName());
}
},
new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError volleyError) {
txtTest.setText("" + volleyError.getMessage());
}
}
);
queue.add(request);
Please kindly help me. Thanks a lot for your help.

What kind of web server do you use? Usually underscore is not accepted for http header field name. (born_date (x), Born-Date(o)
Why underscores are forbidden in HTTP header names, https://en.wikipedia.org/wiki/List_of_HTTP_header_fields#Field_names)
You can also change settings of web server to accept underscore for http header field name.
(Refer to http://nginx.org/en/docs/http/ngx_http_core_module.html#underscores_in_headers)

Accept-Encoding in volley doesn't response with Content-Encoding

I am developing a Restful Api. I am using laravel as backend (with apache) and for client I am using Android (with volley library for network communications).
In one of my call I have the following:
JsonArrayRequest jsonObjReq = new JsonArrayRequest(url,
new Response.Listener<JSONArray>() {
#Override
public void onResponse(JSONArray response) {
Log.d("Response", response.toString());
//PARSE JSON RESPONSE
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
}
}){
#Override
protected Response<JSONArray> parseNetworkResponse(NetworkResponse response) {
for (Map.Entry<String, String> e: response.headers.entrySet()){
Log.d(e.getKey(), e.getValue());
}
return super.parseNetworkResponse(response);
}
#Override
public Map<String, String> getHeaders() throws AuthFailureError {
Map<String, String> map = new HashMap<String, String>();
map.put("Accept-Encoding","gzip,deflate");
return map;
}
};
As you can see I set Accept-Encoding :gzip,deflate.
When laravel receive the request, the headers exist:
array (
'accept-encoding' =>
array (
0 => 'gzip,deflate',
),
'host' =>
array (
0 => '192.168.1.104',
),
'connection' =>
array (
0 => 'Keep-Alive',
),
'user-agent' =>
array (
0 => 'Apache-HttpClient/UNAVAILABLE (java 1.5)',
),
'cookie' =>
array (
0 => 'laravel_session=,
),
'cookie2' =>
array (
0 => '$Version=1',
),
)
But when android receive the response it doesn't contains Content-Encoding : gzip header, the headers that it contains are:
Transfer-Encoding﹕ chunked
Date﹕ Sun, 21 Sep 2014 15:15:37 GMT
Keep-Alive﹕ timeout=5, max=99
Set-Cookie﹕ laravel_session=
Content-Type﹕ application/json
Connection﹕ Keep-Alive
X-Powered-By﹕ PHP/5.4.9-4ubuntu2.4
Server﹕ Apache/2.2.22 (Ubuntu)
Cache-Control﹕ no-cache
When I do the same request via curl:
curl -I -H 'Accept-Encoding: gzip' url
it return:
HTTP/1.1 200 OK
Date: Sun, 21 Sep 2014 14:46:12 GMT
Server: Apache/2.2.22 (Ubuntu)
X-Powered-By: PHP/5.4.9-4ubuntu2.4
Cache-Control: no-cache
Set-Cookie: laravel_session=
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=UTF-8
So, summarizing, I set Accept-Encoding:gzip,deflate, the request is receiveb by server with that headers but when response is received by android Content-Encoding doesn't exist. It is not problem of my server because curl works good.
Any suggestion? thanks
EDIT:
I am watching data sent and received using Wireshark between android and my server. I am watching other request a part from that. This other request is made with JsonObjectRequest instead of JsonArrayRequest and with wireshark I can watch the following headers.
Android -> Server
Content-Type: application/json\r\n
Host: 192.168.1.104\r\n
Connection: Keep-Alive\r\n
User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.5)\r\n
[truncated] Cookie: laravel_session=
Cookie2: $Version=1\r\n
Accept-Encoding: gzip,deflate\r\n
Server -> android
Date: Sun, 21 Sep 2014 18:59:15 GMT\r\n
Server: Apache/2.2.22 (Ubuntu)\r\n
X-Powered-By: PHP/5.4.9-4ubuntu2.4\r\n
Cache-Control: no-cache\r\n
[truncated] Set-Cookie: laravel_session=
Vary: Accept-Encoding\r\n
Content-Encoding: gzip\r\n
Connection: Keep-Alive\r\n
Transfer-Encoding: chunked\r\n
Content-Type: text/html; charset=UTF-8\r\n
In this request the response contains Content-Encoding: gzip. The only different between this request and the other is that this request uses JsonObjectRequest instead JsonArrayRequest, so cant ´JsonArrayRequest use Gzip encode?