We Keep Coding

How to avoid URIBuilder to encode comma to "%2C" or other characters in query params

I would like to create a URI using apache class org.apache.http.client.utils.URIBuilder and I need to not encode query params to percent-encoding.
import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpRequestBase;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import java.net.URI;
public class Main {
public static void main(String[] args) throws Exception {
RequestConfig config = RequestConfig.custom().build();
HttpClientBuilder builder = HttpClientBuilder.create().setDefaultRequestConfig(config);
// URI url = new URI("http://some-website.com/?range=10,20");
// If url is created with the line above the comma "," is not encoded when sending the request
// When you use URIBuilder the comma "," is converted to "%2C"
URIBuilder uribuilder = new URIBuilder("http://some-website.com/");
uribuilder.addParameter("range", "10,20");
URI url = uribuilder.build();
System.out.println("URL => " + url.toString());
HttpHost targetHost = new HttpHost(url.getHost(), url.getPort(), url.getScheme());
HttpClient client = builder.build();
HttpRequestBase req = new HttpPost(url);
HttpResponse httpResponse = client.execute(targetHost, req);
HttpEntity entity = httpResponse.getEntity();
String responseString = EntityUtils.toString(entity, "UTF-8");
System.out.println(responseString);
System.out.println("Finished");
}
}
Is there a way to do that using the URIBuilder class like some flag I don't know?
I would appreciate other suggests (maybe better ways than my code) to accomplish this. But I can't send the characters in query string encoded.
Thanks in advance.

Apache HTTP Client 400 Error

I was Trying to Automate a Web Service and I am passing the XML in the form of a String and later converting it to String Entity and Setting the Entity. But I don't know why it is throwing 400 Error. I am new to WebServices Automation could any please help me on this.
Below is my Code:
package com.WebServices.Automation;
import java.io.IOException;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.junit.Assert;
import org.junit.Test;
public class HTTPClientA {
static String url = "http://www.dneonline.com/calculator.asmx?wsdl";
String xml = "\"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"+
"<soap:Envelope xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:tem=\"http://tempuri.org/\">\r\n" +
" <soap:Header/>\r\n" +
" <soap:Body>\r\n" +
" <tem:Add>\r\n" +
" <tem:intA>10</tem:intA>\r\n" +
" <tem:intB>20</tem:intB>\r\n" +
" </tem:Add>\r\n" +
" </soap:Body>\r\n" +
"</soap:Envelope>";
#Test
public void main() throws ClientProtocolException, IOException
{
StringEntity stringEntity = new StringEntity(xml);
HttpPost post = new HttpPost(url);
post.setEntity(stringEntity);
HttpClient client = HttpClientBuilder.create().build();
post.setHeader("Content-Type", "text/xml; charset=utf-8");
post.setHeader("SOAPAction", "http://tempuri.org/Add");
HttpResponse res = client.execute(post);
int actualresponse = res.getStatusLine().getStatusCode();
System.out.println(actualresponse);
try
{
Assert.assertEquals(actualresponse, 200);
}
catch (Exception e) {
// TODO: handle exception
}
HttpEntity entity = res.getEntity();
String strResponse = null;
if (entity != null) {
strResponse = EntityUtils.toString(entity);
System.out.println(strResponse);
}
}
}

Your XML is invalid, it start with double quote instead of <?xml, change your assignment to start with:
String xml = "<?xml version ... –

Post request to login in Booking.com from Java

I need to login in Booking.com and i am trying to do it in Java. I made a post request in different ways but i can not get the html from de index page.
This is the target page:
Admin Booking
I need the following parameters for login:
loginname=
password=
ses=
lang=en
login=Login
I know that these are the parameters because a partner made that login in python and it works.
The ses parameter is on the form login (as a hidden input field) and the loginname and password are provided by myself.
So... to get the ses i make a previous GET request and then i add it as a parameter in my POST request. I get without problems the html from the first request but not the second one (POST).
I know that the POST login resquest should send the html of the logged page because as i said above a partner of mine obtains that result in python. In addition, i also tried this with Postman Chrome Application (Postman) and it works fine (with the difference that i only provide loginname and password).
Here is my code:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;
import org.apache.http.HttpEntity;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicNameValuePair;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
public class Test {
private static final String USER_AGENT = "Mozilla/5.0";
private static final String GET_URL = "https://admin.booking.com/hotel/hoteladmin/login.html";
private static final String POST_URL = "https://admin.booking.com/hotel/hoteladmin/login.html";
public static void main(String[] args) throws IOException {
String ses = sendGET();
System.out.println("GET DONE");
sendPOST(ses);
System.out.println("POST DONE");
}
private static String sendGET() throws IOException {
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(GET_URL);
httpGet.addHeader("User-Agent", USER_AGENT);
CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
System.out.println("GET Response Status:: "
+ httpResponse.getStatusLine().getStatusCode());
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpResponse.getEntity().getContent()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = reader.readLine()) != null) {
response.append(inputLine);
}
reader.close();
// print result
Document doc = Jsoup.parse(response.toString());
String ses = doc.select("#ses").val();
System.out.println(response.toString());
httpClient.close();
return ses;
}
private static void sendPOST(String ses) throws IOException {
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost(POST_URL);
httpPost.addHeader("User-Agent", USER_AGENT);
List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
urlParameters.add(new BasicNameValuePair("loginname", "467933"));
urlParameters.add(new BasicNameValuePair("password", "moncloa1895"));
urlParameters.add(new BasicNameValuePair("ses", ses));
urlParameters.add(new BasicNameValuePair("lang", "en"));
urlParameters.add(new BasicNameValuePair("login", "Login"));
HttpEntity postParams = new UrlEncodedFormEntity(urlParameters);
httpPost.setEntity(postParams);
CloseableHttpResponse httpResponse = httpClient.execute(httpPost);
System.out.println("POST Response Status:: "
+ httpResponse.getStatusLine().getStatusCode());
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpResponse.getEntity().getContent()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = reader.readLine()) != null) {
response.append(inputLine);
}
reader.close();
// print result
System.out.println(response.toString());
httpClient.close();
}
}
Anyone knows how can i solve it??
Thanks!

You should follow redirects. Add something like this:
httpPost.setRedirectStrategy(new LaxRedirectStrategy());
Im not sure what the actual method is on HTTPPost, but HttpClient has this ability.

Unexpected char in the http get json file while using org.apache.http

There is a json file on my websites floder.
Here is the content:
{
"IsUpdateForcibly": "false",
"Version": "1.0",
"ReleaseNote": "OHOHOHOHOHO",
"DownloadLink": "http://192.168.1.37:11604/APK/FrauleinProject.apk"
}
If I use the browser to see,like http://localhost:11604/Content/CheckVersion.json, the result is same as thefile's content.
While I use the Java code. the response content is a little bit different.
?{
"IsUpdateForcibly": "false",
"Version": "1.0",
"ReleaseNote": "OHOHOHOHOHO",
"DownloadLink": "http://192.168.1.37:11604/APK/FrauleinProject.apk"
}
Why there is a question mark in the front of the string?
Here is is my httpclient code.
import net.sf.json.JSONArray;
import net.sf.json.JSONObject;
import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;
import sun.misc.IOUtils;
import sun.net.www.http.HttpClient;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.InputStreamEntity;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.entity.mime.content.StringBody;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.client.WinHttpClients;
import org.apache.http.util.EntityUtils;
public class DesUtil {
public static void main(String[] args) throws Exception {
CloseableHttpClient httpclient = WinHttpClients.createDefault();
// There is no need to provide user credentials
// HttpClient will attempt to access current user security context through
// Windows platform specific methods via JNI.
try {
HttpGet httpget = new HttpGet("http://localhost:11604/Content/CheckVersion.json");
System.out.println("Executing request " + httpget.getRequestLine());
CloseableHttpResponse response = httpclient.execute(httpget);
try {
System.out.println("----------------------------------------");
ResponseHandler<String> responseHandler = new ResponseHandler<String>() {
#Override
public String handleResponse(
final HttpResponse response) throws ClientProtocolException, IOException {
int status = response.getStatusLine().getStatusCode();
if (status >= 200 && status < 300) {
HttpEntity entity = response.getEntity();
return entity != null ? EntityUtils.toString(entity) : null;
} else {
throw new ClientProtocolException("Unexpected response status: " + status);
}
}
};
String json= new String(httpclient.execute(httpget, responseHandler).getBytes("ISO-8859-1"),"UTF-8");
PrintWriter out = new PrintWriter("filename.txt");
out.println(json);
out.close();
System.out.println(json);
JSONObject obj = JSONObject.fromObject(json);
System.out.println(obj==null);
Sb newSB= (Sb)JSONObject.toBean(obj,Sb.class);
System.out.println(newSB==null);
System.out.println(newSB.IsUpdateForcibly);
System.out.println(newSB.Version);
System.out.println(newSB.ReleaseNote);
System.out.println(newSB.DownloadLink);
}
catch(Exception ex){
System.out.println(ex.getMessage());
}
finally {
response.close();
}
}
catch(Exception ex){
System.out.println(ex.getMessage());
}
finally {
httpclient.close();
}
System.out.println("end");
}
}

I had a similar problem. I solved it by adding "UTF-8"
String str= EntityUtils.toString(entity2);
to
String str= EntityUtils.toString(entity2,"UTF-8");
demo:
private static void sendPost() throws ClientProtocolException, IOException
{
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost("http://127.0.0.1:8911/crr");
ArrayList<NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("crawlId", "123"));
nvps.add(new BasicNameValuePair("transType", "0"));
httpPost.setEntity(new UrlEncodedFormEntity(nvps));
CloseableHttpResponse response2 = httpClient.execute(httpPost);
try {
System.out.println(response2.getStatusLine());
HttpEntity entity2 = response2.getEntity();
String str= EntityUtils.toString(entity2,"UTF-8");
System.out.println(str);
} finally {
response2.close();
}
}

This probably stems from a Unicode BOM character, a zero-width space in Unicode that is used in UTF-8, UTF-16LE, UTF-16BE at the beginning of a file to mark it as Unicode: \uFEFF. It is redundant, unneeded, and - as seen here - causes several problems.
It was replaced with a question mark, as the character encoding of the saved text could not represent the BOM character.
As #zhizhi mentioned, better safe the JSON as UTF-8. Still better is to remove the BOM.
PrintWriter out = new PrintWriter("filename.txt", "UTF-8");
json = json.replaceFirst("^\uFEFF", "");
Mind that removing the BOM poses a UTF-8 recognition problem for Notepad.

HTTP 403 Service Error when trying to post XML to HTTPS URL

I am trying to write a small class using the Apache HttpClient library that would do an HTTPS post to a specified URL sending some XML. When I run my code, the HTTP status line I receive back is "403 Service Error". Here's the complete error HTML returned:
$errorDump java.net.SocketTimeoutException:Read timed out
$errorInfo
$errorDump java.net.SocketTimeoutException:Read timed out
$error Read timed out
$localizedError Read timed out
$errorType java.net.SocketTimeoutException
$user
$time 2011-10-25 09:39:29 EDT
$error Read timed out
$errorType java.net.SocketTimeoutException
This is the code I am using:
import java.io.ByteArrayInputStream;
import java.io.InputStreamReader;
import java.io.BufferedReader;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.InputStreamEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
public class HttpXmlPost {
public static void main(String[] args)
{
String url = "https://someurlhere.com";
String xmlStr = "<?xml version=\"1.0\" encoding=\"UTF-8\" ?><xmlTag></xmlTag>";
String content = request(xmlStr, url);
System.out.println(content);
}
private static String request(String xmlStr, String url) {
boolean success = false;
String content = "";
HttpClient httpclient = new DefaultHttpClient();
try {
HttpPost httpPost = new HttpPost(url.trim());
InputStreamEntity reqEntity = new InputStreamEntity(new ByteArrayInputStream(xmlStr.getBytes() ), -1);
reqEntity.setContentType("application/xml");
reqEntity.setChunked(true);
httpPost.setEntity(reqEntity);
System.out.println("Executing request " + httpPost.getRequestLine());
HttpResponse response = httpclient.execute(httpPost);
HttpEntity resEntity = response.getEntity();
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
if(response.getStatusLine().getStatusCode() == 200){
success = true;
}
if (resEntity != null) {
System.out.println("Response content length: " + resEntity.getContentLength());
System.out.println("Chunked?: " + resEntity.isChunked());
}
BufferedReader reader = new BufferedReader(new InputStreamReader(resEntity.getContent()));
StringBuilder buf = new StringBuilder();
char[] cbuf = new char[ 2048 ];
int num;
while ( -1 != (num=reader.read( cbuf ))) {
buf.append( cbuf, 0, num );
}
content = buf.toString();
EntityUtils.consume(resEntity);
}
catch (Exception e) {
System.out.println(e);
}
finally {
httpclient.getConnectionManager().shutdown();
}
return content;
}
}
Whatever XML I pass in doesn't seem to matter, it gives the same error no matter what. Note that this actually works with some URLs. For example, if I put https://www.facebook.com, it goes through. However, it doesn't work for my specified URL. I thought it might be a certificate issue, tried to add some code to trust any certificate, didn't seem to work either, though I may have done it wrong. Any help is appreciated.

Based on the SocketTimeoutException in the first line of the response HTML, I'm guessing that the component which implements the handler for the URL to which you are posting is having some connection problems to a source system it needs to generate the response data.
Basically, it looks like the problem is on the server, not your client.