I programmed an Android App, which manages data to be stored and deleted in a MySQL database (on server). Whenever on the smartphone special characters ("ä", "ü", ...) are used the symbol is converted badly.
In the log I can see that the "Umlaut" (e.g. "ä") is transmitted properly. I also use in my php file "SET NAMES 'UTF-8'", see here:
function connect()
{
....
// Connecting to mysql database
$con = mysql_connect(DB_SERVER, DB_USER, DB_PASSWORD) or die(mysql_error());
mysql_query("SET NAMES 'utf8'");
// Selecting database
$db = mysql_select_db(DB_DATABASE) or die(mysql_error()) or die(mysql_error());
return $con;
}
If I start a request via Postman, I can add words with special characters, e.g. "TÄST", the json response looks as follows:
{"catId":"35","category":"T\u00c4ST"}
So the words are well converted to UTF-8. But if I add an Item via smartphone, the response from the server (to retrieve the added item) looks like this:
{"catId":"37","category":"T?ST"}
The position in my code, where I add the parameters for the JSON Object is that (note that "name" is the string content of the edit text field):
List<NameValuePair> params = new ArrayList<NameValuePair>();
...
params.add(new BasicNameValuePair("category", catName));
After that the HTTPRequest is send:
JSONObject json = jsonParser.makeHttpRequest(url_dest, "POST", params);
If I print out the params the word "Täst" is visible... But unfortunately I'm not able to check which json string is arriving on the server (due to my bad php knowledge). So where's the problem? In my android application or in the php files located on the server?
Do I have to encode the outgoing json object in any way?
Thanks a lot for your help.
To debug on Eclipse I suggest you:
http://projects.eclipse.org/projects/tools.pdt
It is quite easy to configure.
Control the encoding in the DB; maybe the VARCHAR default latin1_swedish_ci can't save that string.
Another way to solve it could be looking to the apache commons:
http://commons.apache.org/proper/commons-lang/javadocs/api-2.6/org/apache/commons/lang/StringEscapeUtils.html
in particular the functions provided by: escapeHtml4 and unescapeHtml4
#dafocus, if you set your Android app project to UTF-8 encoding in your IDE, it should ensure that UTF-8 is handled properly in the app. It looks like an encoding change somewhere. You might not need to convert TÄST to T\u00c4ST at all if the encoding is UTF-8 throughout. PHP used to be difficult with charsets. I would suggest you to look at the PHP to figure out if the handler page causes this (possible) and/or the DB - as #madx just said.
Related
I guess this is a stupid question with an obvious solution, but I don't see it yet. So the problem is: I get an IllegalArgumentException on Android, which says my base64 input is not valid. This input took the following way before:
Upload: PDF file -(Java Base64 encoder) > Java Base64 encoded string -(POST)-> PHP -(INSERT as mediumtext via mysqli query)-> MySQL DB
Download: MySQL record -(SELECT via mysqli query and fetch assoc afterwards)-> PHP vars -(JSON)-> Java as JSON -(Jackson library, maps JSON to object containing String)-> Java Base64 String - (Android Base64 decoder)-> Exception
Is there any failure in my workflow? Communication is done with UTF-8 via HttpUrlConnection.
I was able to solve the problem: During transfer to the server + and / got omitted. After manually replacing them, everything is working now.
I have the following code in my JSP
... <%
out.println(request.getAttribute("textFromDB")); %> ...
When the JSP is called it just prints question marks (????..) instead of the actual text stored in a MySQL database which is not in English. What can I do to make it display the text correctly. I tried to change the charset and pageEncoding to UTF-8 but it didn't help.
Isn't encoding nice? Unfortunately it's hard to tell where it gets wrong: Your database might store in another character set than UTF-8. Or your database driver might be configured to work in another encoding. Or your server defaults to another one. Or your HTTP connection assumes different encoding and your charset change comes too late.
You'll have to go through all those layers - and keep in mind that everything might look fine and it's been the long-past write-operations to your database that already messed up the data beyond repair.
System.out.println(this.request.getHeader("Content-Encoding")); //check the content type
String data = new String(this.request.getParameter("data").getBytes("ISO-8859-1"), "UTF-8"); //this way it is encoded to byte and then to a string
if the above method didnt work you might wanna check with the database
if it encode characters to "UTF-8"
or
you can configure URIEncoding="UTF-8" in your tomcat setup and just (request.getAttribute("textFromDB")); do the rest.
I'm writing an Android app that posts some CSV formatted data to a PHP server.
If I only post one row, the server reads it correctly and does what it needs to do. However, I'm unable to post multiple lines. I have the CSV data in a .txt file and I read this data into a string and send that string using HttpURLConnection. The issue is that while it'll look fine looking at the .txt file in my phone, the server does not interpret "\n" or "\r\n" correctly and won't update at all. My guess is that it interprets the entire string as one line or something along those lines.
Each line sent is read like this:
list($value1, $value2, $value3) = explode(",", $datarec);
Is there some sort of newline character that PHP can recognize? Apparently, it doesn't recognize \n or \r\n.
I currently have the content-type set to text/csv but I've also tried text/html and using br/> instead.
conn.setRequestProperty("Content-Type", "text/csv");
So here I prefer is to use NameValuePair. Each line, you combine to a string and create it as a NameValuePair, like
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("line1", "string1string2string3"));
And send it .
Refer to Add parameters to URLconnection and use of namevaluepair for more information.
I have a JTextArea from which I get the text, and send it through a POST request to a PHP page, which uses PDO and a prepare statement to insert the text into a MySQL database. This works for everything that I can type directly into the JTextArea. However, it doesn't work when I copy and paste something from Word, for example, into the JTextArea and then upload it.
For example, if I type into Word: "This is a super—long dash." Then copy and paste that in the JTextArea and have it send to the PHP page, which will upload it to the MySQL server, when I get the information back from the MySQL server (using another PHP page) it comes back as "This is a super—long dash."
This also happens for other various special characters, not just long dashes.
Is there anything I can do about this?
EDIT:
I tried to correct the issue by making sure all my php files are using utf8 to connect to mysql like so:
"mysql:host=$servername;dbname=$dbname;charset=utf8"
I also use
echo $_POST[xxx];
To make sure the text is coming in from java correctly, which it is. When I check what is the mysql database, it shows as "This is a superùlong dash". I tried to correct this by using the fixUTF8 function in the solution here: Detect encoding and make everything UTF-8 by Sebastián Grignoli. However, as a result I get "This is a super?long dash".
I am posting Events to Google Calendar using their REST API from Java (GAE), NOT the client library.
This works very well, except that non-ASCII characters are showing up as question marks.
Things I have done to try to address this (some are clutching at straws):
The content-type is set to UTF-8: "Content-Type", "application/json; charset=UTF-8"
I have tried URL encoding the "summary" (which is the Event title) field within the JSON body
I tried html escaping the "summary" field within the JSON body
Searched StackOverflow, and I found this reference that seems like the same problem, but it's in the context of PHP, not Java, so (I think) String handling is quite different, and therefore it doesn't seem to directly apply. Basically this one says make sure that the summary value itself is UTF-8 encoded, and uses a utf8_encode function for which there isn't a direct Java comparison
I realize that one answer is probably just to use the Java library, but for various reasons I don't want to do that unless I absolutely have to.
Any ideas please? Thank you.
EDIT TO ADD CODE:
I create a Scribe Request like this (and, yes, I know that this is not what Scribe is intended for...):
String url = String.format("https://www.googleapis.com/calendar/v3/calendars/%s/events", inTargetCalendar.getId());
OAuthRequest req = new OAuthRequest(Verb.POST, url);
req.addQuerystringParameter("access_token", <TOKEN>);
JSONObject jBody = new JSONObject();
jBody.put( "start", <START> );
jBody.put( "end", <END> );
jBody.put( "summary", getSummary() );
log.info("*** getSummary() is: " + getSummary());
jBody.put( "colorId", getColorId() );
req.addPayload(jBody.toString());
req.addHeader("Content-Type", "application/json; charset=UTF-8");
Note that in this example, when I look at my log file, the logs show (correctly):
*** getSummary() is: Côte Brasserie
I then post the request like this:
Response response=null;
try {
response = request.send();
} catch (...) {...}
Per notes above, this works perfectly, except that the "ô" doesn't show up correctly. Specifically when I look at the Event through the Google calendar website it looks like this:
C?te Brasserie
I.e. the "ô" shows up as a "?". This is the same for other non-ASCII characters (from what I have seen).
The answer was: give up trying to do this without the SDK even though that's what I wanted, and just use the SDK... and it worked straight away - non-ASCII characters showing up fine now.