I'm developing an app for android that communicates with a server and receives JSON responses from it. Recently I changed the responses structure to contain more information about the requests.
Initially my responses where just the response from a mysqli_query encoded to JSON by the php function json_encode, and looked like this:
[
{"id":"31","description":"Hello"},
{"id":"32","description":"World"},
]
And to retrieve the objects, I would just create a new JSONArray object by calling my_json_array=new JSONArray(response) and iterate over it.
But now, my new responses contain extra data, like this:
{
"error":false,
"idCode":0,
"message":[
{
"id":"32",
"description":"Hello"
},
{
"id":"31",
"description":"World"
}
]
}
So now, as far as I understand, I need to convert this response to a JSONObject, and then extract the message as a JSONarray object by calling my_json_object.getJSONArray("message");.
I can extract the data from the "error" and the "idCode" fields normally, but when I try to extract the JSONArray from "message" it returns null, and no exceptions are thrown except for the NullPointerException.
I've validated the JSON string here, escaped the characters that could be problematic for java like the double quotes, removed uppercases, written the response as a single line, removed the double quotes surrounding the "id" value, used single quotes instead of double quotes, used harcoded string instead of the response from the server, checked the unicode representation of the string to see if contains any invalid characters (and it looks like it doesn't) and I don't know what else to do. Something to notice: the method toString() on the object returns the string "null", and based on the toString() method of the JSONObjects, it seems like the problem comes from the JSONStringer library, and going deeper on the classes it looks like the exception that there is a nesting problem, but I can't find any problem in my JSON string.
The important part of the code:
public String getResponse(){
web_helper.sendRequest();
//everything fine over here
return web_helper.getResponse();
}
public void printData(){ throws JSONExpection
JSONObject my_json_object=new JSONObject(getResponse());
System.out.println("error: "+my_json_object.getBoolean("error"));
System.out.println("idCode: "+my_json_object.getInt("idCode"));
JSONArray my_json_array=my_json_object.getJSONArray("message");
//not sure if the next part works because my_json_array is null
for(int i=0; i<my_json_array.length(); i++){
JSONObject obj=my_json_array.getJSONObject(i);
System.out.println("id: "+obj.getInt("id"));
System.out.println("description: "+obj.getString("description"));
}
}
Any help is appreciated, thank you in advance!.
Related
I have the following JSON file format :
{"code":200,
"status":"OK",
"data":[
{"timings":
{"Fajr":"03:11 (EET)",
"Sunrise":"04:54 (EET)",
"Dhuhr":"11:53 (EET)",
"Asr":"15:29 (EET)",
"Sunset":"18:52 (EET)",
"Maghrib":"18:52 (EET)",
"Isha":"20:23 (EET)",
"Imsak":"03:01 (EET)",
"Midnight":"23:53 (EET)"},....
I want to get the value of Asr (for example) & parse it into string variable, I tried the below but nothing works (Please note: response is successfully retrieved, so no issue with reaching the JSON file, but only to get the value of string).
String mfajr = response1.getJSONArray("data").getJSONObject(0)
.get("Fajr").toString();
String mdoher = response1.getJSONArray("data").getJSONObject(0)
.getJSONArray("timings")
.get(Integer.parseInt("Dhuhr")).toString();
Assuming that response1 is a JSONObject.
String mfajr = response1.getJSONArray("data")
.getJSONObject(0)
.get("Fajr").toString();
That will fail because getJSONObject(0) returns an object that has a "timings" attribute ... not a "Fajr" attribute.
String mdoher = response1.getJSONArray("data")
.getJSONObject(0).getJSONArray("timings")
.get(Integer.parseInt("Dhuhr")).toString();
That will fail because the value of "timings" attribute is JSON object, not a JSON array.
Also, the value of the "Dhuhr" attribute is not an integer, so parseInt is not going to work. If you want the value as an integer, you are going to do some string analysis to convert "11:53 (EET)" to an integer.
Basically, you need to make sure that your Java code precisely matches the JSON structure. Close enough is NOT good enough.
OK .... so you want someone to write your code for you.
Try this:
String mfajr = response1.getJSONArray("data")
.getJSONObject(0)
.getJSONObject("timings")
.getString("Fajr");
Now, compare it with the structure of the JSON.
I'm trying to make sure my Jersey request parameters are sanitized.
When processing a Jersey GET request, do I need to filter non String types?
For example, if the parameter submitted is an integer are both option 1 (getIntData) and option 2 (getStringData) hacker safe? What about a JSON PUT request, is my ESAPI implementation enough, or do I need to validate each data parameter after it is mapped? Could it be validated before it is mapped?
Jersey Rest Example Class:
public class RestExample {
//Option 1 Submit data as an Integer
//Jersey throws an internal server error if the type is not Integer
//Is that a valid way to validate the data?
//Integer Data, not filtered
#Path("/data/int/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getIntData(#PathParam("data") Integer data){
return Response.ok("You entered:" + data).build();
}
//Option 2 Submit data as a String, then validate it and cast it to an Integer
//String Data, filtered
#Path("/data/string/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getStringData(#PathParam("data") String data) {
data = ESAPI.encoder().canonicalize(data);
if (ESAPI.validator().isValidInteger("data", data, 0, 999999, false))
{
int intData = Integer.parseInt(data);
return Response.ok("You entered:" + intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
//JSON data, HTML encoded
#Path("/post/{requestid}")
#POST
#Consumes({MediaType.APPLICATION_FORM_URLENCODED, MediaType.APPLICATION_JSON})
#Produces(MediaType.TEXT_HTML)
public Response postData(String json) {
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
//Is there a way to iterate through each JSON KeyValue and filter here?
ObjectMapper mapper = new ObjectMapper();
DataMap dm = new DataMap();
try {
dm = mapper.readValue(json, DataMap.class);
} catch (Exception e) {
e.printStackTrace();
}
//Do we need to validate each DataMap object value and is there a dynamic way to do it?
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
}
Data Map Class:
public class DataMap {
public DataMap(){}
String strData;
Integer intData;
}
The short answer is yes, though by "filter" I interpret it as "validate," because no amount of "filtering" will EVER provide you with SAFE data. You can still run into integer overflows in Java, and while those may not have immediate security concerns, they could still put parts of your application in an unplanned for state, and hacking is all about perturbing the system in ways you can control.
You packed waaaaay too many questions into one "question," but here we go:
First off, the lines
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
Aren't doing what you think they're doing. If your JSON is coming in as a raw String right here, these two calls are going to be applying mass rules across the entire string, when you really need to handle these with more surgical precision, which you seem to at least be subconsciously aware of in the next question.
//Is there a way to iterate through each JSON KeyValue and filter
here?
Partial duplicate of this question.
While you're in the loop discussed here, you can perform any data transformations you want, but what you should really be considering is using the JSONObject class referenced in that first link. Then you'll have JSON parsed into an object where you'll have better access to JSON key/value pairs.
//Do we need to validate each DataMap object value and is there a
dynamic way to do it?
Yes, we validate everything that comes from a user. All users are assumed to be trained hackers, and smarter than you. However if you handled filtering before you do your data mapping transformation, you don't need to do it a second time. Doing it dynamically?
Something like:
JSONObject json = new JSONObject(s);
Iterator iterator = json.keys();
while( iterator.hasNext() ){
String data = iterator.next();
//filter and or business logic
}
^^That syntax is skipping typechecks but it should get you where you need to go.
/Is Integer validation needed or will the thrown exception be good
enough?
I don't see where you're throwing an exception with these lines of code:
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
Firstly, in java we have autoboxing which means this:
int foo = 555555;
String bar = "";
//the code
foo + bar;
Will be cast to a string in any instance. The compiler will promote the int to an Integer and then silently call the Integer.toString() method. Also, in your Response.ok( String ); call, THIS is where you're going to want to encodeForHTML or whatever the output context may be. Encoding methods are ALWAYS For outputting data to user, whereas canonicalize you want to call when receiving data. Finally, in this segment of code we also have an error where you're assuming that you're dealing with an HTTPParameter. NOT at this point in the code. You'll validate http Parameters in instances where you're calling request.getParameter("id"): where id isn't a large blob of data like an entire JSON response or an entire XML response. At this point you should be validating for things like "SafeString"
Usually there are parsing libraries in Java that can at least get you to the level of Java objects, but on the validation side you're always going to be running through every item and punting whatever might be malicious.
As a final note, while coding, keep these principles in mind your code will be cleaner and your thought process much more focused:
user input is NEVER safe. (Yes, even if you've run it through an XSS filter.)
Use validate and canonicalize methods whenever RECEIVING data, and encode methods whenever transferring data to a different context, where context is defined as "Html field. Http attribute. Javascript input, etc...)
Instead of using the method isValidInput() I'd suggest using getValidInput() because it will call canonicalize for you, making you have to provide one less call.
Encode ANY time your data is going to be passed to another dynamic language, like SQL, groovy, Perl, or javascript.
Since the Processing forum is down I hope one of you guys can help me out.
I made a Processing sketch that pulls data from an API (test here: http://www.europeana.eu/portal/api/console.html ) in JSON-format and reads some fields of this data for visualization. Everything works fine so far except when there are fields that are not bound to every JSONObject.
This is the code for the retrieval of the data:
JSONObject json;
json = loadJSONObject("data.json"); // loading my JSON file into the object
JSONArray CHOData = json.getJSONArray("items"); // array of the items I want data from
for (int i = 0; i < CHOData.size(); i++) {
JSONObject CHO = CHOData.getJSONObject(i); // get a specific item
JSONArray previewObj = CHO.getJSONArray("edmPreview"); // this field is an array, so I need to store it in a JSONArray object first
String[] previewArray = previewObj.getStringArray(); // here I store it in my actual string array
String preview = previewArray[0]; // I only need the first element
}
As you can see, I want the first string of the "edmPreview" array in the object. When I run the sketch, I get an error: "JSONObject["edmPreview"] not found."
This is of course because not every item has such an object. But how can I test if there is an object with this name in an item? I tried with if(CHO.getJSONObject("edmPreview") != null), but same error. Is there a way to look into the JSONObject and check the data values for something called "edmPreview"? There is no such function explained in the Processing reference.
The JSON file essentially looks as follows:
{
"items": [
{
"id": "someID",
"edmPreview": [
"http://europeanastatic.eu/api/image?uri=someimage.jpg"
],
// some other fields
}, // some other items
]
}
I'm new to this JSON-stuff, so maybe I miss something important... Thanks for the help!
So I found the answer myself, but not in the Processing reference, but in the actual reference for the JSONObject class in Java (http://www.json.org/javadoc/org/json/JSONObject.html). I tried some of the methods there and found that hasKey() (which is actually something I came up with myself, combining the method "has()" with the term "key", pure coincidence) as a boolean value works very well:
String preview = "";
if (CHO.hasKey("edmPreview")) {
JSONArray previewObj = CHO.getJSONArray("edmPreview");
String[] previewArray = previewObj.getStringArray();
preview = previewArray[0];
}
So now I learned that Processing is essentially just Java and sometimes not every method is written in the Reference. :-)
I am passing a json object from javascript to a java servlet using ajax.
var jsonObj = JSON.stringify(objArray); //Then I pass it to Java using ajax.
In my Java I am getting the json string from the request, then creating a jsonarray, then looping through that array and i'm getting errors when trying to pull one of the json objects from the array.
String dataObj = request.getParameter("obj");
String sql = request.getParameter("sql");
ArrayList<Object> returnArray = new ArrayList<Object>();
int key;
//Get type of object being passed.
JSONArray jsonArray = JSONArray.fromObject(dataObj);
for(int i=0; i<jsonArray.size(); i++) {
String obj = new Gson().toJson(jsonArray.getJSONObject(i)); //This is where i'm getting an error
String className = getClassName(jsonArray.getJSONObject(i));
Class targetClass = null;
try {
targetClass = Class.forName(className);
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
//Create Object
Object data = new Gson().fromJson(obj, targetClass);
I'm posting the relevant code, the for loop isn't closed because the rest of the code is quite long, and this is the part where i'm getting the error.
net.sf.json.JSONException: JSONArray[0] is not a JSONObject.
Here is what the json array looks like when its passed in from javascript. This is a println of the jsonArray object.
[{"number":"(123) 456-7050","type":"Home","contactId":1,"id":16662,"className":"beans.PhoneNumber","position":0}]
With one object in it, this code works. But as soon as I get 2 or more, my error comes up.
[[{"number":"(123) 456-7050","type":"Home","contactId":1,"id":16662,"className":"beans.PhoneNumber","position":1},{"number":"(555) 555-1233","type":"Mobile","contactId":1,"id":16656,"className":"beans.PhoneNumber","position":0},{"number":"(999) 999-9999","type":"Home","contactId":1,"id":16664,"className":"beans.PhoneNumber","position":3},{"number":"(222) 222-2222","type":"Home","contactId":1,"id":16666,"className":"beans.PhoneNumber","position":4}]]
It almost looks like when i'm passing more than one object, it create an array of an array, which could be why its not working. But how do I avoid doing that when i'm passing a jsonarray from javascript? Using just the dataObj I have no access to size or get to loop through it.
[
[
{
"number":"(123) 456-7050","type":"Home",
"contactId":1,
"id":16662,
"className":"beans.PhoneNumber",
"position":1
},
{
"number":"(555) 555-1233",
"type":"Mobile",
"contactId":1,
"id":16656,
"className":"beans.PhoneNumber",
"position":0
},
{
"number":"(999) 999-9999",
"type":"Home",
"contactId":1,
"id":16664,
"className":"beans.PhoneNumber",
"position":3
},
{
"number":"(222) 222-2222",
"type":"Home",
"contactId":1,
"id":16666,
"className":"beans.PhoneNumber",
"position":4
}
]
]
This is not an array of objects. This is an array of arrays of objects. According to your description, you are expecting something like the following to be fed to your Java:
[{"foo":"bar"}, {"bar":"baz"}]
But you are really trying to parse:
[[{"foo":"bar"}, {"bar":"baz"}]]
I am not completely sure, because you have not shared the json that you are trying to parse, but the most probable error you have is just what it says: the first element of the array is not JSONObject. Note that string values, lons and booleans are not JSONObjects. I would suggest you to use the more genereal JSONArray.get and check instance of what class it is. Maybe this can head you to the problem with the json you have. If I got it completely wrong - write back and I will try to help. In such a case it will be still useful to share the results of the proposed experiment.
EDIT:
This is double array -> maybe you using getJSONArray(int index) will help you. as the other answer mentioned - this is array of arrays. Also consider changing the javascript to reduce the level of arrays included.
I'm new to JSON.
I need to receive a response (in form of a String) from a server. That response can be an object like
{"a" : "value", "b" : "value2", ...}
if the request was successful, or a single string like
"ERROR"
on error.
Using org.json.JSONObject, how do I check which one has been returned?
EDIT
I think this could work, but I'm not sure if this is the right way to do it
if(JSONString.equals("\"ERROR\"") {
//handle error
} else {
//parse actual object
}
Where JSONString is a String containing the server response
Could this work?
The string "ERROR" is not valid JSON. Look at the JSONWriter API and you will see there is no way to produce a JSON string like "ERROR".
If you always want to treat the server response as json, you will need to have it return something like { "error" : true } or { error : false }. Your program will then be able to deserialize check the error field.
If you don't have control on the server response, then you will need to test String.equals("ERROR") before deserializing.
Since you can't make the third party service output valid json, before you do json parsing, just do a string comparison to see if the response is "error".
A quoted string is a valid json value in my opinion. The grammar at json.org does not define object or array as special topleve productions, rfc 4627 defines json-text as being an object or array, but a json value can also be a number, string, boolean or null.
From my reading of the org.json javadoc the following should work:
Object value = new JSONTokener(inputString).nextValue();
if (value instanceof String && ((String)value).equals("ERROR")) {
// handle error
} else if (value instanceof JSONObject) {
// handle response data
}
Using the tokener you are not affected by eventual additional whitespace in the response.