We Keep Coding

How to fix dialogflow authentication issue

I have a dialogflow project that I'm trying to access from Java with a rest call.
It is giving me an authentication issue.
I have followed all online instructions (and many forum suggestions) to no avail.
I have tried generating the key json, as per the instructions here:
https://dialogflow.com/docs/reference/v2-auth-setup
and setting my environment variable as described, but nothing seems to work.
I have checked my projectID, and am running off the same machine with the environment variable, and have double, triple and quadruple checked it's name and location, but I still get the following error:
java.net.HttpRetryException: cannot retry due to server authentication, in streaming mode
Here is my code (though it's a REST call, so I don't know if it's so relevant):
String url = https://dialogflow.googleapis.com/v2/projects/MYPROJECT/agent/sessions/SESSION_NUM:detectIntent
URL url = new URL(full_url);
String inText = "Hello World";
String outText = "";
HttpURLConnection con = (HttpURLConnection) url.openConnection();
try {
con.setDoOutput(true);
con.setRequestMethod("POST");
// set body of http post
Map<String,String> arguments = new HashMap<>();
JSONObject inTextJsn = new JSONObject();
inTextJsn.append("text",inText);
inTextJsn.append("languageCode","en");
JSONObject fieldJsn = new JSONObject();
fieldJsn.append("text", inTextJsn);
arguments.put("queryInput", fieldJsn.toString());
StringJoiner sj = new StringJoiner("&");
for(Map.Entry<String,String> entry : arguments.entrySet())
sj.add(URLEncoder.encode(entry.getKey(), "UTF-8") + "="
+ URLEncoder.encode(entry.getValue(), "UTF-8"));
// post http post as bytes
byte[] bytes_out = sj.toString().getBytes(StandardCharsets.UTF_8);
con.setFixedLengthStreamingMode(bytes_out.length);
con.connect();
try (OutputStream os = con.getOutputStream()) {
os.write(bytes_out);
}
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream(),
"UTF-8"));
// read all lines to a string
String line;
String response = "";
while ((line = reader.readLine()) != null) {
response += line;
}
JSONObject responseJsn = new JSONObject(response);
outText = responseJsn.get("fulfillmentText").toString();
} catch (Exception e) {
System.err.println(e);
} finally {
con.disconnect();
}
return restResponse;
The gist of the code is to simply send a message ("Hello World!") to my dialogflow, and get back my agent's response (the code may have bugs - it's a bit hard to test when I can't get passed this authentication issue, so please help with the authentication, not code bugs).
Thanks all!

The directions at that page assume you're going to use the gcloud program to generate a currently valid bearer token, which is then sent along with the HTTP headers. That page illustrates
Your code doesn't seem to be generating an Authorization HTTP header at all, which is why you're getting the error you do.
Since you're using Java, you should look at the google-auth-library-java library, which will give you the tools to generate the token you need to provide in the Authorization header.
You may also wish to check out the google-cloud-java library. This contains Java classes to directly perform operations against Dialogflow instead of coding the REST/HTTP calls yourself. (However, it is still at an Alpha level for Dialogflow, so may not be stable or forwards compatible.)

What's the recommended way to get the HTTP response as a String when using Apache's HTTP Client? [duplicate]

This question already has answers here:
How to get HttpClient returning status code and response body?
(5 answers)
Closed 2 years ago.
I've just begun using Apache's HTTP Client library and noticed that there wasn't a built-in method of getting the HTTP response as a String. I'm just looking to get it as as String so that i can pass it to whatever parsing library I'm using.
What's the recommended way of getting the HTTP response as a String? Here's my code to make the request:
public String doGet(String strUrl, List<NameValuePair> lstParams) {
String strResponse = null;
try {
HttpGet htpGet = new HttpGet(strUrl);
htpGet.setEntity(new UrlEncodedFormEntity(lstParams));
DefaultHttpClient dhcClient = new DefaultHttpClient();
PersistentCookieStore pscStore = new PersistentCookieStore(this);
dhcClient.setCookieStore(pscStore);
HttpResponse resResponse = dhcClient.execute(htpGet);
//strResponse = getResponse(resResponse);
} catch (ClientProtocolException e) {
throw e;
} catch (IOException e) {
throw e;
}
return strResponse;
}

You can use EntityUtils#toString() for this.
// ...
HttpResponse response = client.execute(get);
String responseAsString = EntityUtils.toString(response.getEntity());
// ...

You need to consume the response body and get the response:
BufferedReader br = new BufferedReader(new InputStreamReader(httpresponse.getEntity().getContent()));
And then read it:
String readLine;
String responseBody = "";
while (((readLine = br.readLine()) != null)) {
responseBody += "\n" + readLine;
}
The responseBody now contains your response as string.
(Don't forget to close the BufferedReader in the end: br.close())

You can do something like:
Reader in = new BufferedReader(
new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
Using the reader you will be able to build your string. But if you are using SAX you can give the stream to the parser directly. This way you will not have to create the string and your memory footprint will be lower too.

In terms of conciseness of code it might be using the Fluent API like this:
import org.apache.http.client.fluent.Request;
[...]
String result = Request.Get(uri).execute().returnContent().asString();
The documentation warns though that this approach is not ideal in terms of memory consumption.

URL Encoding with httpclient

I have a list of URLs which I need to get the content of.
The URL is with special characters and thus needs to be encoded.
I use Commons HtpClient to get the content.
when I use:
GetMethod get = new GetMethod(url);
I get a " Invalid "illegal escape character" exception.
when I use
GetMethod get = new GetMethod();
get.setURI(new URI(url.toString(), false, "UTF-8"));
I get 404 when trying to get the page, because a space is turned to %2520 instead of just %20.
I've seen many posts about this problem, and most of them advice to build the URI part by part. The problem is that it's a given list of URLs, not a one that I can handle manually.
Any other solution for this problem?
thanks.

What if you create a new URL object from it's string like URL urlObject = new URL(url), then do urlObject.getQuery() and urlObject.getPath() to split it right, parse the Query Params into a List or a Map or something and do something like:
EDIT: I just found out that HttpClient Library has a URLEncodedUtils.parse() method which you can use easily with the code provided below. I'll edit it to fit, however is untested.
With Apache HttpClient it would be something like:
URI urlObject = new URI(url,"UTF-8");
HttpClient httpclient = new DefaultHttpClient();
List<NameValuePair> formparams = URLEncodedUtils.parse(urlObject,"UTF-8");
UrlEncodedFormEntity entity;
entity = new UrlEncodedFormEntity(formparams);
HttpPost httppost = new HttpPost(urlObject.getPath());
httppost.setEntity(entity);
httppost.addHeader("Content-Type","application/x-www-form-urlencoded");
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity2 = response.getEntity();
With Java URLConnection it would be something like:
// Iterate over query params from urlObject.getQuery() like
while(en.hasMoreElements()){
String paramName = (String)en.nextElement(); // Iterator over yourListOfKeys
String paramValue = yourMapOfValues.get(paramName); // replace yourMapOfNameValues
str = str + "&" + paramName + "=" + URLEncoder.encode(paramValue);
}
try{
URL u = new URL(urlObject.getPath()); //here's the url path from your urlObject
URLConnection uc = u.openConnection();
uc.setDoOutput(true);
uc.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
PrintWriter pw = new PrintWriter(uc.getOutputStream());
pw.println(str);
pw.close();
BufferedReader in = new BufferedReader(new
InputStreamReader(uc.getInputStream()));
String res = in.readLine();
in.close();
// ...
}

If you need to manipulate with request URIs it is strongly advisable to use URIBuilder shipped with Apache HttpClient.

try it out
GetMethod get = new GetMethod(url.replace(" ","%20")).toASCIIString());

Please use the URLEncoder class.
I used it in an exact scenario and it worked just fine for me.
What I did is to use the URL class, to get the part that comes after the host
(for example - at www.bla.com/mystuff/bla.jpg this would be "mystuff/bla.jpg" - you should URLEncode only this part, and then consturct the URL again.
For example, if the orignal string is "http://www.bla.com/mystuff/bla foo.jpg" then:
Encode - "mystuff/bla foo.jpg" and get "mystuff/bla%20foo.jpg" and then attach this to the host and protocol parts:
"http://www.bla.com/mystuff/bla%20foo.jpg"
I hope this helps

MySQL PHP JSON null text

I found a problem with some results of a search on a DB. When, some fields have extra characters like "ü", the field return as null so it appear as null on the search. My code is like this:
the php Script
$q=mysql_query("SELECT * FROM PRODFAR WHERE ARTI LIKE '%".$_REQUEST['search']."%'");
while($e=mysql_fetch_assoc($q))
$output[]=$e;
print(json_encode($output));
mysql_close();
the JSON PARSER constructor:
public class JsonParser {
static InputStream is = null;
static JSONObject json_data = null;
static String result = "";
// constructor
public JsonParser() {
}
public JSONArray getJSONFromUrl(ArrayList<NameValuePair> nameValuePairs, String url) {
//http post this will keep the same way as it was (it's important to do not forget to add Internet access to androidmanifest.xml
InputStream is = null;
String result ="";
JSONArray jArray = null;
try{
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
is = entity.getContent();
}
catch(Exception e){
Log.e("log_tag", "Error in http connection "+e.toString());
}
//convert response that we receive from the php file into a String()
try{
BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"),8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
result = sb.toString();
}catch(Exception e){
Log.e("log_tag", "Error converting result "+e.toString());
}
// try parse the string to a Json object
try {
//json_data = new JSONObject(result);
jArray = new JSONArray(result);
} catch (JSONException e) {
Log.e("JSON Parser", "Error parsing data " + e.toString());
}
// return Json String
return jArray;
}
}
any hint of how I can solve this?
update: as long as I cannot pass it with JSON, because it only admits UTF-8 characters. I think that one of the possible solutions is to convert the text through PHP into UTF-8 encoding archive, and the other is to use an alternative to JSON that supports other encodings. so I would like to try the first one. so if any know a good algorithm to covert the encoding of a text to UTF-8 using PHP will help to. Also other hints or tips of possible directions to find a solution are welcome please comment on this post any Idea is welcome
SOLVED
I solved it encoding it to UTF-8, it changed my characters like "ü" to something like u\000f, but the java editor show it as iso-8859-1 like Ü when it's showed on the screen. The edited PHP code have the following lines after the query:
$q=mysql_query("SELECT * FROM PRODFAR WHERE ARTI LIKE '%$search1%'");
while($e=mysql_fetch_assoc($q)){
$e['ARTI'] = utf8_encode ( $e['ARTI'] );
$e['DESC'] = utf8_encode ( $e['DESC'] );
$e['PRESENT'] = utf8_encode ( $e['PRESENT'] );
$output[]=$e;
}
print(json_encode($output));

JSON only support UTF8. So try using utf8_encode() / utf8_decode() for conversion.

At first blush, I'd guess it as an encoding issue. Make sure your tables are UTF-8 and also that your views are the same.
Also: Please don't put $_REQUEST data directly into your SQL. That will come back to haunt you.

Bing Maps REST services returns incorrect characters for French. Is there a way to request them to encode their response or decode it somehow?

In the Android application I have written, there is a portion which allows the user to enter the start and end location of their trip, and a route itinerary is returned. I am using Bing Maps REST services for this. I want the directions returned to be in French.
A sample request: request. This is best seen on a Chrome browser, Safari and Firefox take care of this. You can see that the directions have lots of strange characters where they are not supposed to be. I have tried decoding on the device, by doing:
URLDecoder.decode(obj.optString("text"), HTTP.ISO_8859_1)
which does not work (the response stays the same), which makes sense I think since it has already become the special characters. I cannot use Windows-1252 to decode because Android does not seem to support that.
An example of what I am being sent back: Léger Encombrement. What it should be: Léger Encombrement.
It works perfectly on an iPhone as well, but not on Android. Any suggestions on how I can solve this?
My code in the connection class is:
public static JSONObject getJSONResult(final String url) {
try {
final HttpClient client = new DefaultHttpClient();
final HttpGet get = new HttpGet(url);
final HttpResponse responsePost = client.execute(get);
final HttpEntity resEntity = responsePost.getEntity();
boolean DEBUG = true;
if (DEBUG) {
Log.d("", "[JSON-ENV] url: " + url);
}
final String str = EntityUtils.toString(resEntity);
Log.d("connection", "response str: " + str);
if (resEntity != null) {
final JSONObject obj = new JSONObject(str);
Log.d("connection", "JSON RESPONSE IS " + obj);
return obj;
} else {
return null;
}
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
Is there something I need to add into my connection class?
UPDATE:
I added the JSON parsing code to format as "ISO_8859_1" as seen at this link: http://p-xr.com/android-tutorial-how-to-parse-read-json-data-into-a-android-listview/ but I still get the same results ...

This is JSON. You don't need to use URLDecoder. The error is before that, probably when you create the String for the JSON Parser. JSON is always in UTF-8 (or 16, rarely)
Can you post the code for reading the server response?
edit
EntityUtils uses ISO_8859_1 as a default Charset if it does not find one in the content. Simply change
final String str = EntityUtils.toString(resEntity);
to
final String str = EntityUtils.toString(resEntity, HTTP.UTF_8);