We Keep Coding

Object.toString() OutOfMemory - How to split if it contains Base64?

I have a response from a web server which contains the whole Base64 content of a pdf (I CANNOT CHANGE SERVER RESPONSE). Dimension of this file is over 50MB.
So, when i try to process this response with an AsynkTask the application runs into OOM exception.
#Override
protected String doInBackground(Object... params) {
String response = null;
try {
CommandCode = params[0].toString();
App application = (App) params[1];
String data = params[2].toString(); <--- Exception here
if (application != null) {
switch (CommandCode) {
//STORE ATTACHMENT
case Constants.ParserCommand.STORE_ATTACHMENT: {
try {
if (JSONParser.ParseGetAttachment(new JSONObject(data), application)) {
response = "attachment stored";
} else {
response = null;
}
} catch (Exception ex) {
response = null;
}
}
}
}
} catch (Exception ex) {
// some code
}
}
"Data" is a JSONObject which contains various informations including the base64 content. Above code is called by this few lines:
// GET_ATTACHMENT_BY_ATID
JSONObject jo = new JSONObject(_response).getJSONObject("DATA").getJSONObject("Attachment");
task.executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR, Constants.ParserCommand.STORE_ATTACHMENT, appCDA, jo);
All data are needed to create a new record into internal DB to store the document, but the content is stored into a File in filesystem.
My first thought is to isolate the content and work with it. Is there a way to do that? For example:
// All data withouth base64Content
JSONObject mainInfo = new JSONObject(_response).getJSONObject("DATA").getJSONObject("Attachment").without("Base64Content");
// Content isolated
String contentInfo = new JSONObject(_response).getJSONObject("DATA").getJSONObject("Attachment").getString("Base64Content");
Now the OOM exception still raise. Is there a way to split this base64 content in multiple parts and save them into the file above mentioned by appending?

Using s As FileStream = File.Open("C:\FilePath\File", FileMode.Open)
Using sr As New StreamReader(s)
Using reader As New JsonTextReader(sr)
While reader.Read()
//Perform your action here
End While
End Using
End Using
End Using
Here you can use FileStream to read the JSON file, it will only load apart of file in memory at a time and not the whole file. The code is in VB, you can implement it in Java in some way.

Java speech API null response

I am using the java speech recognition API - Jarvis located at https://github.com/lkuza2/java-speech-api
However when I run my application, I get an error : Server returned HTTP response code: 400 for URL: https://www.google.com/speech-api/v1/recognize?xjerr=1&client=chromium&lang=en-US&maxresults=1 (This is the URL that this api uses to get response from Google)
I also created a API key as mentioned in the earlier posts and tried to use the url (this is version 2 API): www.google.com/speech-api/v2/recognize?output=json&lang=en-US&key=MYKey". But in this case I get a null response from Google.
Can anybody please tell me how to get around this?

I change some things from the Recognizer class:
I change the GOOGLE_RECOGNIZER_URL constant to:
private static final String GOOGLE_RECOGNIZER_URL = "https://www.google.com/speech-api/v2/recognize?output=json&lang=en-us&key=YOUR_KEY";
Then I changed this method because the response data have 2 lines
private String rawRequest(File inputFile, int maxResults, int sampleRate) throws IOException
The first line (the one that is read and sent) is null (i don¡t really know why) and the second line has the response of the speech recognized. For this you must read the second line (don't know if there is a nicer way):
String response = br.readLine();
response = br.readLine();
br.close();
return response;
Then I change this method, I think it was using the v1 URL response or something because this method looks for utterance in the json response and there is none utterance key.
private void parseResponse(String rawResponse, GoogleResponse googleResponse)
if (rawResponse == null)
return;
JSONObject jsonObject = new JSONObject(rawResponse);
JSONArray jsonArray= (JSONArray) jsonObject.get("result");
JSONArray jsonArrayResult = (JSONArray) jsonArray.getJSONObject(0).get("alternative");
googleResponse.setResponse(jsonArrayResult.getJSONObject(0).get("transcript").toString());
googleResponse.setConfidence(jsonArrayResult.getJSONObject(0).get("confidence").toString());
I'm new with the json library so it might be a better and shorter way but this worked for me!

Sending JSON as HTTP POST parameter (Android/Java)

I'm trying to send a JSON object which contains a JSON array with JSON objects inside via HTTP POST parameters.
The format of the parameter (what the server expects) is something like:
{""team"":[
{""teamid"":""179228"",""position"":1},
{""teamid"":""218036"",""position"":2},
{""teamid"":""88109"",""position"":3},
{""teamid"":""88111"",""position"":4},
{""teamid"":""165536"",""position"":5},
{""teamid"":""224645"",""position"":6}
]}
nevertheless, what gets sent is:
{"team":"[
\"{\\\"position\\\":0,\\\"teamid\\\":\\\"88107\\\"}\",\"{\\\"position\\\":1,\\\"teamid\\\":\\\"88109\\\"}\",\"{\\\"position\\\":2,\\\"teamid\\\":\\\"156714\\\"}\",\"{\\\"position\\\":3,\\\"teamid\\\":\\\"138877\\\"}\",\"{\\\"position\\\":4,\\\"teamid\\\":\\\"168730\\\"}\",\"{\\\"position\\\":5,\\\"teamid\\\":\\\"88110\\\"}\",\"{\\\"position\\\":6,\\\"teamid\\\":\\\"88111\\\"}\",\"{\\\"position\\\":7,\\\"teamid\\\":\\\"134431\\\"}\",\"{\\\"position\\\":8,\\\"teamid\\\":\\\"88112\\\"}\",\"{\\\"position\\\":9,\\\"teamid\\\":\\\"138507\\\"}\",\"{\\\"position\\\":10,\\\"teamid\\\":\\\"138880\\\"}\",\"{\\\"position\\\":11,\\\"teamid\\\":\\\"138881\\\"}\",\"{\\\"position\\\":12,\\\"teamid\\\":\\\"151465\\\"}\",\"{\\\"position\\\":13,\\\"teamid\\\":\\\"151464\\\"}\
"]"}
The way I build that JSON object is the following:
JSONArray teamArray = new JSONArray();
JSONObject jsonRoot = new JSONObject();
for (int i = 0; i < mTeams.size(); i++) {
String teamId = null;
BaseModel data = mTeams.get(i);
if (data != null && data instanceof TeamModel) {
teamId = ((TeamModel) data).getId();
}
JSONObject teamObject = new JSONObject();
try {
teamObject.put(
getResources().getString(
R.string.sendResortedTeamsPosition), i);
teamObject.put(
getResources().getString(
R.string.sendResortedTeamsTeamId), teamId);
teamArray.put(teamObject);
} catch (NotFoundException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
}
try {
jsonRoot.put("team", teamArray);
mNameValuePairs.put("teams", jsonRoot);
} catch (JSONException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
In the last but one line (jsonRoot.put("team", teamArray);) it has the same format as what gets sent in the last line, but with one less \, so one less times "parsed" apparently.
Part of my HTTP code:
String postBody = json.toString();
Log.d("HTTPHelper", "posting JSON: " + postBody);
((HttpPost) httpRequest).setEntity(new StringEntity(postBody));
Why is this happening? Is it Java?
Any ideas how could I build the correct JSON? or any work around?
Thanks a lot in advance!

I gave this up, and decided to go the dirty-way: replacing chars manually the following way:
json = new JSONObject(nameValuePairs.toString().replace("\"", "'"));
json = new JSONObject(json.toString().replace("\"", ""));
It's ugly, probably dangerous or risky, but it works...

Too much code for this task, checkout this library https://github.com/kodart/Httpzoid
It uses GSON internally and provides API that works with objects. All JSON details are hidden.
Http http = HttpFactory.create(context);
http.get("http://example.com/users")
.handler(new ResponseHandler<User[]>() {
#Override
public void success(User[] users, HttpResponse response) {
}
}).execute();

just a tip... i'm not really into android, just java..
but you could de- and encode your json with Base64 on both sides, and pass the encoded "String".
so you don't have to worry about any "dirty-way" replacing.
hope this helps u or others, too. :)

GSON Library parse JSON UTF-8 not properly, replace accent by "?"

I'm try to develop a Java application with GSON library to parse JSON (from PHP file (encoding in UTF-8) -> json_encode)
My php source :
<?php
$base = mysql_connect ('****', '*****', '*****');
mysql_select_db ('*****', $base) ;
$req = mysql_query("SELECT ***, ****, ***, ****, **** from *****");
function jsonRemoveUnicodeSequences($struct) {
return preg_replace("/\\\\u([a-f0-9]{4})/e", "iconv('UCS-4LE','UTF-8',pack('V', hexdec('U$1')))", json_encode($struct));
}
while ($row = mysql_fetch_array($req)) {
$output[] = $row;
}
print(jsonRemoveUnicodeSequences($output));
mysql_free_result ($req);
?>
JSON string is properly displayed with accent in in my web browser.
My Java source :
BufferedReader reader = null;
try {
URL url = new URL("**************");
URLConnection urlConnection = url.openConnection();
reader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "UTF-8"));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line);
System.out.println("JSON data issu du PHP : "+ line + "\n");
Gson gson = new Gson();
Type type = new TypeToken<List<AlertTable>>(){}.getType();
ArrayList<AlertTable> bddListJson = gson.fromJson(line, type);
bddList = (ArrayList<AlertTable>) bddListJson.clone();
}
} catch (IOException e) {
//
} finally {
if (reader != null) {
//
}
}
System.out.println(bddList.get(1).getTypeAlert());
System.out.println(bddList.get(1).getLigne());
System.out.println(bddList.get(1).getSens());
System.out.println(bddList.get(1).getStation());
System.out.println(bddList.get(1).getTimeAlert());
And in console, character with accent is replaced by "?".
Any idea ?

First, note that your code may fail on some systems because MySQL uses a connection charset to move text between the client and the server. You should issue a raw query like SET NAMES <charset> to set the encoding of the input data available in your script.
Now, assuming there's no transcoding issue (because your columns only store ASCII characters and all charset involved are ASCII-compatible), I wrote the following code to replace \u0000 escape sequences with UTF8-encoded strings:
<?php
while ($row = mysql_fetch_array($req)) {
$out[] = preg_replace_callback("/\\\\u([a-f0-9]{4})/i", "unescape", $row);
}
// On PHP 5.4+ use json_encode($out, JSON_UNESCAPED_UNICODE)
echo json_encode($out);
/* Accept the matcher array
* return the UTF-8 encoded string
*/
function unescape($match) {
return call_user_func_array('pack', get_pack_args(hexdec($match[1])));
}
function get_pack_args($cp) {
if ($cp < 0x80) return array('C1', $cp);
if ($cp < 0x0800) {
$length = 2;
} else if ($cp < 0x010000) {
$length = 3;
} else {
$length = 4;
}
$args[0] = "C{$length}";
// lead byte
$args[1] = (0xFE << (7 - $length)) | ($cp >> (6 * ($length - 1)));
// continuation bytes
for ($l = 0; $l < ($length - 1); $l++) {
$args[$length - $l] = 0x80 | (($cp >> (6 * $l)) & 0x3F);
}
// stupid PHP...
ksort($args);
return $args;
}
You should test the code deeply, however it's a good starting point. For displaying in a browser, you should specify the encoding with a HTTP header:
header('Content-Type: application/json; charset=utf8');
Note that json_encode() escapes unicode sequences on its part by default. This is inefficient, so you may want to use the JSON_UNESCAPED_UNICODE flag, or choose another JSON library for encoding.
At this point the server part should be ok, and you can test it with your browser. If it works, but the Java program can't still show the right characters on the console, it may be a problem with the console itself, or even with the font used by the console application. Without the actual data one can't tell, however follow my advice and print the JSON to a text file UTF8-encoded. Then open it with a text editor (specifying UTF-8 if needed) and tell what you see.
Finally, note that the usage of the mysql PHP extension is discouraged. Use mysqli or PDO instead.

if You set parameter of json_encode Correctly like
json_encode($WhoRank,JSON_HEX_TAG | JSON_HEX_APOS | JSON_HEX_QUOT | JSON_HEX_AMP | JSON_UNESCAPED_UNICODE);
no problem in encoding value and transfer data but if is not work again you can Use urlencode for Encode UTF-8 Value and Decode it on Java
like this
urlencode('aیgfسبd');
output
a%DB%8Cgf%D8%B3%D8%A8d
with this output You haven't problem in json value.

Java: how to check the content of a php script output?

sorry for the stupid question but my knowledge of java net is terrible.
BAsically in my android application a call many php scripts to get data from a mysql db.
These data are returned in json format and i use Google json library to parse them.
Everything works fine but know in each php page i have to add a test. It the test is successfull, then the script continues and returns the json file, but if the test fails, the script return the string "false", or the value false (that's up to me) and my application instead of showing data has to redirect the user to a login page.
The code is the following:
URL url = new URL(Costanti.IP_SERVER+"myApps.php"+"?userId="+this.userId);
try
{
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
int status = conn.getResponseCode();
if (status >= 200 && status <= 299)
{
Reader r = new InputStreamReader(conn.getInputStream());
Applicazioni dati = new Applicazioni();
try
{
dati = gsonReader.fromJson(r, Applicazioni.class);
return dati;
}
catch (Exception e)
{
System.out.println("Ho fallito a prendere i dati");
Log.e("JSON_Parser",e.toString());
}
}
}
catch (IOException e)
{
System.out.println("Ho fallito la connnection");
e.printStackTrace();
}
}
So basically i use this google library to read the json file inside the imputStreamReader and fill the Applicazioni object with my data.
How can i check if the content of the imputStreamReader is the string "false" or the boolean false and if it's different parse it with the json library????
In the php at the end i do
echo json_encode($applicazione);
in one case or
echo "false" in the other case
Tnx

InputStream in = new URL(url).openStream();
Scanner scanner = new Scanner(new InputStreamReader(in));
String result = scanner.useDelimiter("\\Z").next(); // this reads the whole
// script output in a string
if(result.equals("false"))
handle the false value...
else
dati = gsonReader.fromJson(result, Applicazioni.class);

You can json encode the false result also like ["result"=>"false"] from PHP, This way you can always JSON decode in your Java program, and then look for result value.
You can put the result value in both cases in the output.