We Keep Coding

Converting malformed json array string to Java object

I have a malformed json array string which I get from an API call as follows:
[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Processed successfully\"}]\"}]
There is a double quote before open square bracket in the value of Response Msg property.
Is there a way to convert this into Java object ?
What I have tried so far:
I have used Jackson to parse it as follows but it gives error
ObjectMapper mapper = new ObjectMapper();
mapper.setPropertyNamingStrategy(new ResponseNameStrategy());
Response[] response = mapper.readValue(strOutput1, Response[].class);
Error: Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token
I have also tried using Gson to parse it but it also gives error
Gson gson = new GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE)
.create();
Response[] response = gson.fromJson(strOutput1, Response[].class);
Error: Expected BEGIN_ARRAY but was STRING at line 1 column 35 path $[0].ResponseMsg
I have gone through the following links on StackOverflow but none of them has addressed my issue:
How to Convert String Array JSON in a Java Object
Convert a JSON string to object in Java ME?
JSON Array to Java objects
Convert json String to array of Objects
converting 'malformed' java json object to javascript

I think the answer is in the comments, you appear to be trying to solve the issue on the wrong place.
You are receiving json which you wish to parse into java objects, unfortunately the json is malformed so will not parse.
As a general rule you should never be trying to solve the symptom, but should look for the root cause and fix that, it may sound trivial but fixing symptoms leads to messy, unpredictable, and unmaintainable systems.
So the answer is fix the json where it is being broken. If this is something or of your control, while you wait for the fix, you could put a hack in to fix the json before you parse it.
This way you won't compromise your parsing, and only have a small piece of string replacement to remove when the third party has fixed the issue. But do not go live with the hack, it should only be used during development.

As i mentioned in the comment, you should prepare your service response in order to parse it.
I implemented an example:
public class JsonTest {
public static void main(String args[]) throws JsonProcessingException, IOException{
String rawJson =
"[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Processed successfully\"}]\"}]";
String goodJson = "{"+rawJson.split("[{{.}]")[2]+"}";
ObjectMapper mapper = new ObjectMapper();
final ObjectNode node = mapper.readValue(goodJson, ObjectNode.class);
System.out.println("Pretty Print: " + mapper.writerWithDefaultPrettyPrinter().writeValueAsString(node));
System.out.println("Just code: " + node.get("Code"));
}
}
Which returns:

This is how I finally solved my issue:
String inputJsonStr = "[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Claim has been added successfully.\"}"
+ "]\"}]";
int indexOfRes = inputJsonStr.indexOf("ResponseMsg");
if(inputJsonStr.substring(indexOfRes+13,indexOfRes+14).equals("\""))
{
inputJsonStr = inputJsonStr.substring(0,indexOfRes+13) + inputJsonStr.substring(indexOfRes+14);
}
int indexOfFirstClosingSquare = inputJsonStr.indexOf("]");
if(inputJsonStr.substring(indexOfFirstClosingSquare+1, indexOfFirstClosingSquare+2).equals("\"")) {
inputJsonStr = inputJsonStr.substring(0, indexOfFirstClosingSquare+1)+inputJsonStr.substring(indexOfFirstClosingSquare+2);
}
Now inputJsonStr contains a valid json array which can be parsed into Java custom object array easily with gson as given in this SO link:
Convert json String to array of Objects

Android Firebase - Can't receive proper JSON from Firebase snapshot

My android app connects to Firebase and pulls "Alert Objects" that are sent there by my server.
When I export the data from Firebase, I get a beautifully formated JSON representation of the data.
Problem:
When I pull the data to my android device using a DataSnapshot, the data has '=' (equals signs) instead of ':' (semicolons). Also the quotations are not there.
When I try to do something like JSONObject alert = new JSONObject(data.getValue().toString()); I get errors for obvious reasons. I say obvious because if you look at what my code prints to the console you can see that it is no longer in valid JSON format.
A friend mentioned that I need to do something with encoding but we didn't have time to discuss it.
How can I iterate through these (kinda weird) Alert Objects that I have created and turn them into JSON objects within my Java so that I can access their properties like alert.date and alert.message.
I thought screenshots would help you see what I am doing. The firebase is not secured at all so you can feel free to take a look at it. It won't really do much and when I go to production I will be moving it anyways.
I am sure this is a super easy question to answer, I am just not too well versed with JSON and encoding as a whole.
Thanks!

you can it using gson library
public void onChildAdded(DataSnapshot dataSnapshot, String s) {
/*JSONObject jsonObject = null;
try {
jsonObject=new JSONObject();
} catch (JSONException e) {
e.printStackTrace();
}*/
Gson gson = new Gson();
String s1 = gson.toJson(dataSnapshot.getValue());
JSONArray object = null;
try {
object = new JSONArray(s1);
} catch (JSONException e) {
e.printStackTrace();
}
JSONArray jsonArray = object;
Log.e("string", s1);
}

You cannot access JSON natively in Java.
But the Firebase DataSnapshot class provides everything you need.
If you have a DataSnapshot of the data at the fbAlerts in your screenshot, you can print the date+message and recipients for each:
for (DataSnapshot alert: alerts.getChildren()) {
System.out.println(alert.child("date").getValue();
System.out.println(alert.child("message").getValue();
for (DataSnapshot recipient: alert.child("recipients").getChildren()) {
System.out.println(recipient.child("name").getValue();
}
}
Alternatively, you can build a Java class that represents an alert. See the Firebase guide on reading data in Android for examples of that.

Use this way to convert the jsonObject form the dataSnapshot
Map<String, String> value = (Map<String, String>) dataSnapshot.getValue();
Log.i("dataSnapshot", "dataSnapshot" + new JSONObject(value));

From the above code extract, it looks like you have a JSONArray rather than a JSONObject.
In which case, you need to do something like the following:
// Find the right array object
JSONArray jsonArray = response.getJSONArray("fbAlerts");
// Loop through the array
for (int i=0; i< jsonArray.length(); i++) {
JSONObject myObj = jsonArray.getJSONObject(i);
strMsg = myObj.getString("message");
}
In the example - when you have repeating groups, this would seem to indicate an array and therefore needs an iterator to access the object contents.

How to save and load the object's list to SharedPreferences with/without using Gson?

In my Android app, I used Gson in order to save/load the object's Arraylist in SharedPreferences. Follows are my code using Gson.
public static ArrayList<RequestModal> getModalList(Context ctx) {
Gson gson = new Gson();
String json = getSharedPreferences(ctx).getString("ModalList", new Gson().toJson(new ArrayList<>()));
Type type = new TypeToken<ArrayList<RequestModal>>() {}.getType();
return gson.fromJson(json, type);
}
In here "RequestModal" is the simple object include a bit of strings and integers.
It works well in case "online". But if internet is offline, forever works on below code.
Type type = new TypeToken<ArrayList<RequestModal>>() {}.getType();
How can I solve it? What is the way implement the feature like this with/without using Gson? Please help me anyone having a good idea.
Thank you in advance.

You can implement this without Gson:
public static EpisodeDetails parseEpisodeDetails(String content) {
EpisodeDetails episodeDetails = new EpisodeDetails();
try {
JSONObject jsonObject = new JSONObject(content);
episodeDetails.title = jsonObject.getString("title");
episodeDetails.subTitle = jsonObject.getString("subtitle");
episodeDetails.synopsis = jsonObject.getString("synopsis");
episodeDetails.ends_on = jsonObject.getString("ends_on");
JSONArray images = jsonObject.getJSONArray("image_urls");
if (images.length() > 0) {
episodeDetails.image_url = images.getString(0);
}
} catch (JSONException e) {
e.printStackTrace();
return null;
}
return episodeDetails;
}
What I'm doing is just taking the String, in your case the one saved on the shared prefs called ModalList and inserting the values on my structure, on my code the structure is called EpisodeDetails, on your code the correspondent is RequestModal. If you don't want to do it via code and want to try another library I recommend Jackson.
Another thing, on this line:
String json = getSharedPreferences(ctx).getString("ModalList", new Gson().toJson(new ArrayList<>()));
Your second parameter is not necessary. getString takes the key to load as first parameter and a default value as second paramter (in the case of empty result). You could change this to "" or null.

Well, another solution to your problem could be TinyDB. It makes use of Gson to save ArrayLists of objects in sharedPrefs, its usage is so simple as:
Person person = new Person("john", 24);
tinydb.putObject("user1", person);
ArrayList<Person> usersWhoWon = new ArrayList<Person>();
tinydb.putListObject("allWinners", usersWhoWon);
and that's it, check out my link given above to see the usage details.

Parsing JSON using GSON in java and populating a list view

I realize my questions have been asked a lot but I have spent a considerable amount of time scouring both SO and google trying to get a better understanding of this concept with no success. I've seen many different implementations, which is what leads me to get some advice about my specific situation.
MY OBJECTIVE
I need to perform a post request to a php file and the goal is to ultimately populate fields in a list activity with some of the json data.
HTTP POST RESPONSE
Here is the format of the response data I'm getting back from the server, which appears to be a JSON object of arrays(?).
{"expense":[{"cat_id_PK":237,"cat_name":"Name1","cat_amount":"100.00","is_recurring":0},
{"cat_id_PK":238,"cat_name":"Name2","cat_amount":"200.00","is_recurring":0},
{"cat_id_PK":239,"cat_name":"Name3","cat_amount":"300.00","is_recurring":0},
{"cat_id_PK":240,"cat_name":"Name4","cat_amount":"400.00","is_recurring":0}],
"expense_rec": [{"cat_id_PK":207,"cat_name":"Name5","cat_amount":"500.00","is_recurring":1}]}
FIRST QUESTION
The code below is what I'm using to read the response. Is this how I should be handling that? It seems weird to get a json encoded response and then change it to a string, only to try and access elements of a json object again. Am I on the wrong track here?
//This code is in the doInBackground method of my "sendPostRequest" async task.
HttpResponse httpResponse = httpClient.execute(httpPost);
InputStream inputStream = httpResponse.getEntity().getContent();
InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
StringBuilder stringBuilder = new StringBuilder();
String bufferedStrChunk = null;
while ((bufferedStrChunk = bufferedReader.readLine()) != null) {
stringBuilder.append(bufferedStrChunk);
}
//Returns string to onPostExecute()
return stringBuilder.toString();
SECOND QUESTION
I have another file called "PostResponse.java" that holds the following code I modeled after a tutorial online. I'm unsure of how to interact with this class from the onPostExecute method. How can I access say, the first item in the first object (something like in PHP where you could do: expense[0]['cat_name']). I've tried to do this various ways with no success. Here is the PostResponse.java class:
public class PostResponse {
public Integer cat_id_PK;
public String cat_name;
public BigDecimal cat_amount;
public Integer is_recurring;
public int getID() {
return this.cat_id_PK;
}
public void setID(int cat_id_PK){
this.cat_id_PK = cat_id_PK;
}
public String getName() {
return this.cat_name;
}
public void setName(String cat_name) {
this.cat_name = cat_name;
}
public BigDecimal getAmount() {
return this.cat_amount;
}
public void setAmount(BigDecimal cat_amount) {
this.cat_amount = cat_amount;
}
public int getRecurring() {
return this.is_recurring;
}
public void setRecurring(int is_recurring) {
this.is_recurring = is_recurring;
}
#Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("*** Categories ***");
sb.append("cat_id_PK="+getID()+"\n");
sb.append("cat_name="+getName()+"\n");
sb.append("cat_amount="+getAmount()+"\n");
sb.append("is_recurring="+getRecurring()+"\n");
return sb.toString();
}
}
and here is the content of my onPostExecute method:
protected void onPostExecute(String result) {
super.onPostExecute(result);
Gson gson = new Gson();
PostResponse response = gson.fromJson(result, PostResponse.class);
System.out.println(result);
}
Like I said originally, my ultimate goal is to populate these items to a list activity, but at this point I'd settle for just knowing how to get at specific elements. However, if anyone would like to include how to populate a list activity in their response, it would save me a lot more headaches, as nothing with java is coming easy for me!

FIRST QUESTION
The code below is what I'm using to read the response. Is this how I should be handling that? It seems weird to get a json encoded response and then change it to a string, only to try and access elements of a json object again. Am I on the wrong track here?
It's one way of handling the http response. A "json encoded response" is little more than a text-based response, so converting it into a string on the receiving end would make sense. That is, the json you receive isn't an 'object' as far as Java is concerned: it's just textual representation of an object (or a bunch of objects in your case), received as a stream of bytes.
That being said, you can potentially shorten your code by skipping the string(builder) part. Gson provides an alternative constructor that takes a Reader instance, for which you can suppy the BufferedReader in your code snippet.
As a side note: the conversion of textual json into Java objects is a potentially 'heavy' operation. As such, you'd best avoid doing it on the main/ui thread, so just move it into the doInBackground() method of your AsyncTask (and change types appropriately).
SECOND QUESTION
I have another file called "PostResponse.java" that holds the following code I modeled after a tutorial online. I'm unsure of how to interact with this class from the onPostExecute method. How can I access say, the first item in the first object (something like in PHP where you could do: expense[0]['cat_name']). I've tried to do this various ways with no success.
You're close, but if you look more closely to the json sample, you'll see that your PostResponse class is not a good match for it:
{
"expense": [
{
"cat_id_PK": 237,
"cat_name": "Name1",
"cat_amount": "100.00",
"is_recurring": 0
},
{
"cat_id_PK": 238,
"cat_name": "Name2",
"cat_amount": "200.00",
"is_recurring": 0
},
{
"cat_id_PK": 239,
"cat_name": "Name3",
"cat_amount": "300.00",
"is_recurring": 0
},
{
"cat_id_PK": 240,
"cat_name": "Name4",
"cat_amount": "400.00",
"is_recurring": 0
}
],
"expense_rec": [
{
"cat_id_PK": 207,
"cat_name": "Name5",
"cat_amount": "500.00",
"is_recurring": 1
}
]
}
Consider the more hierarchical formatting above. On the first level there are two (json) objects: expense and expense_rec (both contain 0...* elements, as the square brackets indicate). That means that whatever class you're going to be trying to map the json onto, should define these fields too. If you now look at your PostResponse class, it should become obvious that in its current form it in fact models one of the child objects of the aforementioned fields.
Basically, the classes to map the json onto, should look somewhat like this:
PostResponse:
public class PostResponse {
public ExpenseItem[] expense;
public ExpenseItem[] expense_rec;
// List<ExpenseItem> is also supported
// getters & setters
}
ExpenseItem:
public class ExpenseItem {
public Integer cat_id_PK;
public String cat_name;
public BigDecimal cat_amount;
public Integer is_recurring;
// getters & setters
}
With the model classes defined, try to let Gson work its magic again. If all goes well, you should be able to access the data in a way similar to what you're used to in PHP:
// map json to POJOs
PostResponse response = new Gson().fromJson(bufferedReader, PostResponse.class);
// retrieve the cat_name for the first item (assuming there is one)
String catName = response.getExpense()[0].getName();
... or any of the other fields through the getters defined in ExpenseItem.
Once you have this part working, it's going to be fairly straightforward to supply the array or list of expenses to an adapter (have a look at ArrayAdapter in the Android framenwork) and bind that adapter to a ListView.

The answer is yes, you will get the response in InputStream
For your second question check this out - jsonschema2pojo this can be helpful while creating models for your JSON data.
Then to use GSON
Gson gson = new Gson();
YourObj yourObj = (YourObj) gson.fromJson(result, YourObj.class);

The answer is yes.Response is received as InputSteam
protected void onPostExecute(String result)
{
super.onPostExecute(result);
Gson gson = new Gson();
PostResponse response = gson.fromJson(result, PostResponse.class);
System.out.println(result);
}
This code section mostly means that after an AsyncTask that get web Response and get the String format json response,this onPostExecute will be called with that Stringfied json.
Gson gson = new Gson();
Gson is a library supported by Google for android to deserialization into your class OBject.
gson.fromJson(result, PostResponse.class);
This method is the real process of deserialization. result is Stringfied json and the second is the Target class you want to deserialize into.
This will return a PostResponse Object and you can use it now.

For the json data (
{"expense":[{"cat_id_PK":237,"cat_name":"Name1","cat_amount":"100.00","is_recurring":0},
{"cat_id_PK":238,"cat_name":"Name2","cat_amount":"200.00","is_recurring":0},
{"cat_id_PK":239,"cat_name":"Name3","cat_amount":"300.00","is_recurring":0},
{"cat_id_PK":240,"cat_name":"Name4","cat_amount":"400.00","is_recurring":0}],
"expense_rec": [{"cat_id_PK":207,"cat_name":"Name5","cat_amount":"500.00","is_recurring":1}]}),
it contains two different arrays here, one is "expense" and another is "expense_rec". So if you want to populate these items to a list activity, you can try the follow methods.
protected void onPostExecute(String result) {
super.onPostExecute(result);
JSONObject jsonObject = new JSONObject(builder.toString());
Log.i(TAG, "jsonObject is : " + jsonObject.toString());
//this is the first array data
JSONArray jsonArray = jsonObject.getJSONArray("expense");
Log.i(TAG, "Array length is: " + jsonArray.length());
for(int i = 0; i < jsonArray.length(); i++){
JSONObject jsoObj = jsonArray.getJSONObject(i);
String name = jsoObj.getString("cat_name");
Log.i(TAG, "file name is: " + name);
}
//this is the second array data
jsonArray = jsonObject.getJSONArray("expense_rec");
for(int i = 0; i < jsonArray.length(); i++){
JSONObject jsoObj = jsonArray.getJSONObject(i);
String name = jsoObj.getString("cat_name");
Log.i(TAG, "file name is: " + name);
}
}

JSONObject toString and Base64 performance

In my app I am sending base64 encoded files in Json format. My code is:
JSONObject jsonRequest = prepareJsonObject(expense, contentUri);
String jsonString = jsonRequest.toString();
StringEntity se = new StringEntity(jsonString);
The jsonRequest object looks like this:
{
"category":"660",
"user":"458",
"dated_on":"Wed Mar 12 10:38:11 GMT+00:00 2014",
"description":"document",
"gross_value":"-2.0",
"currency":"GBP",
"attachment":{
"data":"<base64>"
"mimetype":"image/jpeg"
}
}
The problem is that jsonRequest.toString() is taking like 2 minutes for a 700Kb file.
Is there a way to make this quicker? Am I doing something wrong?
thanks.
EDIT: I am testing in an actual Nexus 4 running KITKAT.
For completion purpose, this is the code for prepareJsonObject(), which runs in less than 2 secs.
public static JSONObject prepareJsonObject(Expense expense, String contentUri){
JSONObject expenseJson = null;
try{
expenseJson = new JSONObject();
if(expense.getUserId()!=null) expenseJson.put("user", expense.getUserId().toString());
if(expense.getProjectId()!=null) expenseJson.put("project", expense.getProjectId().toString());
if(expense.getCurrency()!=null) expenseJson.put("currency", expense.getCurrency().toString());
if(expense.getGrossValue()!=null) expenseJson.put("gross_value", expense.getGrossValue().toString());
if(expense.getNativeGrossValue()!=null) expenseJson.put("native_gross_value", expense.getNativeGrossValue().toString());
if(expense.getSalesTaxRate()!=null) expenseJson.put("sales_tax_rate", expense.getSalesTaxRate().toString());
if(expense.getDescription()!=null)expenseJson.put("description", expense.getDescription().toString());
if(expense.getDated()!=null)expenseJson.put("dated_on", expense.getDated());
if(expense.getCategoryId()!=null) expenseJson.put("category", expense.getCategoryId().toString());
if(expense.getManualSalesTaxAmount()!=null)expenseJson.put("manual_sales_tax_amount", expense.getManualSalesTaxAmount().toString());
if(contentUri!=null){
JSONObject attachmentJson = new JSONObject();
String base64data = AttachmentsUtils.getBase64ForUri(Uri.parse(contentUri));
attachmentJson.put("data", base64data);
attachmentJson.put("content_type", AttachmentsUtils.getMimetypeFromContentUri(Uri.parse(contentUri)));
expenseJson.put("attachment", attachmentJson);
}
}catch(Exception e){
Log.e("ExpensesUtils", "Couldn't encode attachment: "+e.getLocalizedMessage());
return null;
}
return expenseJson;
}

I think this is because the toString pare the full content of your base64 data to handle unicode characters and escape some specific characters. You can see this in JSONStringer#string, which is called for every string value in your JsonObject when you call toString.
Of course, as your data is base64, you don't actually need this. So I think you will need to implement your own toString implementation, probably based on the JSONStringer without escaping