We Keep Coding

How to encode Post Data JSON in CloseableHttpClient APi

I have used the CloseableHttpClient APi for a Post call and Basic Auth for authorisation
private CloseableHttpClient httpclient = HttpClients.createDefault();
HttpPost httppost = new HttpPost("https://example.com");
MyJson myJson = new MyJson(); //custom java object to be posted as Request Body
Gson gson = new Gson();
String param = gson.toJson(myJson);
StringEntity urlparam = new StringEntity(param);
String credentials = username + ":" + passwprd;
String base64Credentials = new String(Base64.getencoder().encode(credentials.getBytes()));
String authorizartionHeader = "Basic" + base64Credentials;
httppost.setHeader("Content-Type", "application/Json");
httppost.setHeader("Authorization", authorizartionHeader);
urlparam.setContentEncoding("UTF-8");
httppost.setEntity(urlparam);
httpclient.execute(httppost);
I am getting error
"Invalid UTF-8 middle byte"
I have encoded the JSON still the encoding is not working for other locales except English. How to encode the Post data.
I tried using the method
httppost.setEntity(new URLEncodedFormEntity(namevaluePair, "UTF-8")) but I don't have any Namevaluepair and if the add the Username-pswd in that then getting Null pointer response.

You should try to set everything as UTF-8
StringEntity urlparam = new StringEntity(param, StandardCharsets.UTF_8);
And add proper header
httppost.setHeader("Content-Type", "application/json;charset=UTF-8");

Send special characters in JSON from android to PHP

I want to send a JSON to a PHP file that I have on my server, it works fine except when some field contains a special characters (accents, ñ, etc.).
Java file:
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(uri);
JSONObject json = new JSONObject();
try {
// JSON data:
json.put("id_u", viaje.getID_U());
json.put("id_vo", viaje.getID_VO());
json.put("titulo", viaje.getTitulo());
[...]
JSONArray postjson=new JSONArray();
postjson.put(json);
// Post the data:
httppost.setHeader("json",json.toString());
httppost.getParams().setParameter("jsonpost",postjson);
// Execute HTTP Post Request
System.out.print(json);
HttpResponse response = httpclient.execute(httppost);
PHP file:
$json = $_SERVER['HTTP_JSON'];
$data = json_decode($json);
$id_u = $data->id_u;
$id_vo = $data->id_vo;
$titulo = $data->titulo;
[...]
For example, if titulo = "día", $title is empty, but instead whether titulo = "example" works correctly.
I do not know how to convert to utf-8 before sending the items, I tried many things and nothing works for me. Any idea?
EDIT:
I could solve the problem. It was clear that the problem was the encoding. I solved by adding 2 lines to the code:
Java file:
// Post the data:
httppost.setHeader("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
httppost.setHeader("json",json.toString());
httppost.getParams().setParameter("jsonpost",postjson);
PHP file:
$json = $_SERVER['HTTP_JSON'];
$cadena = utf8_encode($json);
$data = json_decode($cadena);
thanks for your help! :)

Sounds like an encoding issue. Try setting the encoding like this:
HttpPost httppost = new HttpPost(builder.getUrl());
httppost.setHeader("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
You can also force the proper encoding on your content like this. But that's probably not needed here:
// Add your data
httppost.setEntity(new UrlEncodedFormEntity(builder
.getNameValuePairs(), "UTF-8"));

how to create issue using bitbucket API and singpost in java

I am trying to create new issue at bibucket, but i don't know how to work with http. I try many things but it stil dosen't work. This is one of my attempts:
URL url = new URL("https://api.bitbucket.org/1.0/repositories/"
+ accountname + "/" + repo_slug + "/issues/"
+ "?title=test&content=testtest");
HttpsURLConnection request = (HttpsURLConnection) url.openConnection();
request.setRequestMethod("POST");
consumer.sign(request);
request.connect();
I don't have a problem with GET requests. But here I don't know how to send parametrs and sign the message.
Here is documentation of API
https://confluence.atlassian.com/display/BITBUCKET/issues+Resource#issuesResource-POSTanewissue
How to do this properly?

In the end I figured this out. The parametrs isn't part of the URL, but if you use stream, You can't sign it.
The solution is use Apache HttpComponents library and add parameters like in code below:
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("https://api.bitbucket.org/1.0/repositories/"
+ accountname + "/" + repo_slug + "/issues/");
List<NameValuePair> nvps = new ArrayList<NameValuePair>();
nvps.add(new BasicNameValuePair("title", "test"));
nvps.add(new BasicNameValuePair("content", "testtest"));
httpPost.setEntity(new UrlEncodedFormEntity(nvps));
consumer.sign(httpPost);
HttpResponse response2 = httpclient.execute(httpPost);
try {
System.out.println(response2.getStatusLine());
HttpEntity entity2 = response2.getEntity();
// do something useful with the response body
// and ensure it is fully consumed
EntityUtils.consume(entity2);
} finally {
httpPost.releaseConnection();
}
}
But you must use CommonsHttpOAuthConsumer which is in special signpost library for commonshttp.

I have seen you've already solved but here it says that you need to authenticate with OAuth and in the page you linked that you need to authenticate in order to create new issues.
It also links to this page for OAuth implementations for many languages. I am going to post it for knowledge.

URL Encoding with httpclient

I have a list of URLs which I need to get the content of.
The URL is with special characters and thus needs to be encoded.
I use Commons HtpClient to get the content.
when I use:
GetMethod get = new GetMethod(url);
I get a " Invalid "illegal escape character" exception.
when I use
GetMethod get = new GetMethod();
get.setURI(new URI(url.toString(), false, "UTF-8"));
I get 404 when trying to get the page, because a space is turned to %2520 instead of just %20.
I've seen many posts about this problem, and most of them advice to build the URI part by part. The problem is that it's a given list of URLs, not a one that I can handle manually.
Any other solution for this problem?
thanks.

What if you create a new URL object from it's string like URL urlObject = new URL(url), then do urlObject.getQuery() and urlObject.getPath() to split it right, parse the Query Params into a List or a Map or something and do something like:
EDIT: I just found out that HttpClient Library has a URLEncodedUtils.parse() method which you can use easily with the code provided below. I'll edit it to fit, however is untested.
With Apache HttpClient it would be something like:
URI urlObject = new URI(url,"UTF-8");
HttpClient httpclient = new DefaultHttpClient();
List<NameValuePair> formparams = URLEncodedUtils.parse(urlObject,"UTF-8");
UrlEncodedFormEntity entity;
entity = new UrlEncodedFormEntity(formparams);
HttpPost httppost = new HttpPost(urlObject.getPath());
httppost.setEntity(entity);
httppost.addHeader("Content-Type","application/x-www-form-urlencoded");
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity2 = response.getEntity();
With Java URLConnection it would be something like:
// Iterate over query params from urlObject.getQuery() like
while(en.hasMoreElements()){
String paramName = (String)en.nextElement(); // Iterator over yourListOfKeys
String paramValue = yourMapOfValues.get(paramName); // replace yourMapOfNameValues
str = str + "&" + paramName + "=" + URLEncoder.encode(paramValue);
}
try{
URL u = new URL(urlObject.getPath()); //here's the url path from your urlObject
URLConnection uc = u.openConnection();
uc.setDoOutput(true);
uc.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
PrintWriter pw = new PrintWriter(uc.getOutputStream());
pw.println(str);
pw.close();
BufferedReader in = new BufferedReader(new
InputStreamReader(uc.getInputStream()));
String res = in.readLine();
in.close();
// ...
}

If you need to manipulate with request URIs it is strongly advisable to use URIBuilder shipped with Apache HttpClient.

try it out
GetMethod get = new GetMethod(url.replace(" ","%20")).toASCIIString());

Please use the URLEncoder class.
I used it in an exact scenario and it worked just fine for me.
What I did is to use the URL class, to get the part that comes after the host
(for example - at www.bla.com/mystuff/bla.jpg this would be "mystuff/bla.jpg" - you should URLEncode only this part, and then consturct the URL again.
For example, if the orignal string is "http://www.bla.com/mystuff/bla foo.jpg" then:
Encode - "mystuff/bla foo.jpg" and get "mystuff/bla%20foo.jpg" and then attach this to the host and protocol parts:
"http://www.bla.com/mystuff/bla%20foo.jpg"
I hope this helps

How to Authenticate Against Website Using Apache HttpClient 4.2.1 or >?

I'm new to http programming and I'm attempting to authenticate against a website that doesn't use an API and I'm having some trouble. I found a few other questions that seemed to be similar to mine, but none had an answer that worked for me.
I've tried several different approaches but haven't found one that works yet. Actually, I had it work for me once, but didn't check in that code (I know - what was I thinking?). I haven't been able to get back to that working version again. Here are a few things that I've tried so far:
HttpClient client = new DefaultHttpClient(); //or any method to get a client instance, with a 'threadsafe' connection manager or otherwise
Credentials credentials = new UsernamePasswordCredentials(userName, passwd);
((DefaultHttpClient) client).getCredentialsProvider()
.setCredentials(AuthScope.ANY, credentials);
// website is defined as my target website & this constitutes the valid login URL
HttpPost post = new HttpPost(https + website + "/login");
HttpEntity entity = new StringEntity("Username="+ userName +"&Password="+ passwd);
post.setEntity(entity);
HttpResponse response = client.execute(post);
EntityUtils.consume(entity);
entity = response.getEntity();
EntityUtils.consume(entity);
// the protected part of the site is over http after authentication succeeds
HttpGet get = new HttpGet(http + website +"/protected");
response = client.execute(get);
entity = response.getEntity();
String content = EntityUtils.toString(entity);
At this point the 'entity' I get back from the website is "error":"Unauthorized Access."
You'll notice that I have a 'Credentials' instance being passed to the HttpClient and I'm also putting the user name & password in the HttpPost entity. I tried each of these approaches separately and they both returned the "error":"Unauthorized Access." result.
I've tried the DefaultHttpClient, which uses a single thread connection manager, as well as 'ThreadSafeClientConnManager' and neither worked.

first try to login using this url (copy the url to textpad or something first - then edit it):
https://www.centraldispatch.com/login?uri=%2Fprotected%2F&Username=XXX&Password=YYY
Just replace the XXX and YYY with your real user/pass - make sure it works.
Then use:
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("Username","Your username"));
nameValuePairs.add(new BasicNameValuePair("Password","Your password"));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = client.execute(post);
BufferedReader rd = new BufferedReader(new InputStreamReader(
response.getEntity().getContent()));
String line = "";
while ((line = rd.readLine()) != null) {
System.out.println(line);
}

Did you set the User-Agent filed?Like that:
get.setHeader("Content-Type", "text/html");
get.setHeader("User-Agent","Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.0; .NET CLR 1.1.4322; .NET CLR 2.0.50215;)");
get.setHeader("Accept-Charset", Chareset+";q=0.7,*;q=0.7");//"utf-8;q=0.7,*;q=0.7");
And maybe you should get the login page firstly,to use the cookie.
If all this don't work,you shloud use some tools like firebug+firefox to track the network process,and emu them step by step.That should work.
Websites check some fields,I think this is definitely the reason.