I have a text file as input which contains Employees. Now I need to refactor the code and the JSON can either contain an Employee[] array or a Building[] array. I am using Jackson, and there are enums in my Java class.
{
Employee[]
}
{
Building[]
}
I have code to selectively parse each document, but how can the code detect whether it is an Employee doc or a Building doc?
I know it's kind of a bad design, but there are some constraints for which I am doing so.
I need something like this:
boolean isBuildingDoc(String json);
boolean isEmployeeDoc(String json);
How can I do this?
My advice would be to add a top level property to your json called 'info'.
For example:
{
"info": {
"document_type": "employee"
},
"content": {
// rest of your json goes here...
}
}
Put the old json you used to generate in 'content'. Then, when you parse your json file, you can easily check the info.document_type flag to see what document type you are working with.
However, if you do not have control over the json that is being parsed (e.g. if it is being sent from another program), then this approach will not work.
You can use tree modal of jackson to detect Employee or Building node.
E.g.
ObjectMapper mapper = new ObjectMapper();
JsonNode node = mapper.readTree(jsonString);
if(node.get("Employee") != null){
//handle it according to employee
}
else if(node.get("Building ") != null){
//handle it according to building
}
Note : Do remember one thing, get method searches for field in child elements also, So if your Employee array contains Building field as a child element, then this method will not work.
Related
I am using Room and Volley to get data from a generic web service that only returns column and value for each row of the selected table.
The thing is that I don't want to modify the web service to return a specific data tree by table.
So having :
{"columns": {
"0": [{"column":"name","value":"pepe"},{"column":"age","value":20}],
"1": [{"column":"name","value":"paco"},{"column":"age","value":23}]
}}
.. I want to be able to fit that response creating an instance of the following class for example...
public class Person {
String name = "";
int age = 0;
}
So people make the pojo from their responses to consume each object with gson like...
{"persons": {
"person": [{"name":"pepe","age":20},{"name":"paco","age":23}]
}}
Person p = gson.fromJson(jsonString, Person.class);
So i want to be able as im invoking the table person in the web service to return that generic json and convert it to a custom class because the columns are the same in the response and in the class itself.
Is there a possibility to load that column name and ask the object the type of data and parse it properly in a few lines of code...? ...Or ill need to loop de JSONObject to get the specific variable to later build the object in the constructor...?
THX
I have the following programming requirement:
problem:
Given two JSONs A and B, if the fields x,y,z in JSON A match the fields i,o,p in B return true else false.
approach:
I want to stay away from building a matching engine that depends on the json's format. I don't want to format the jsons by using pojos and then do object matching. My approach is to convert all the jsons into a hash map and then specify the location of the fields by using a string:
Example:
money -> a,b,c
{
a :
{
b : {
c: {
money : "100"
}
}
}
}
However this approach seems to be a bit tricky as we have to take into account collections. I have to cover all of the edge cases. Is there any spring library or java tool I can use to fulfill this purpose?.
There are many libraries being used for this purpose.The most popular one is com.google.gson
Usage:
JsonObject jo = (JsonObject)(jsonParser.parse("{somejsonstring}");<br>
jo.has("objectProperty") //Check if property exists
jo.get("objectProperty") // returns JsonElement,
jo.get("objectProperty").isJsonArray() // check if the property is the type that want
jo.getAsJsonArray("objectProperty") get the property
You may simplify this work by using im.wilk.vor:Voritem library gitHub or in Maven repository.
JsonElement je_one = jsonParser.parse("{some_json_string"})
JsonElement je_two = jsonParser.parse("{other_json_string"})
VorItem vi_one = vorItemFactory.from(je_one);
VorItem vi_two = vorItemFactory.from(je_two);
if (vi_one.get("a.b.c").optionalLong().isPresent() ) {
return vi_one.get("a.b.c").optionalLong().equals(vi_one.get("i.o.p").optionalLong())
}
return false;
I'm parsing some output from an api, and I'd like to make the parsing logic as generic as possible. Many of the api calls return an object within a common wrapper. The responses look something like this:
{
"success": true,
"data": {
"object_type": {
"object_id": { /* actual object data */ }
}
}
}
In this response, object_type can be one of a few different strings. I am writing a function to handle these responses that takes in a class reference for the inner object:
private static ObjectMapper MAPPER = new ObjectMapper();
<T> APIResponse<T> parseResponse(String input, Class<T> clazz) {
JsonNode node = MAPPER.readTree(input);
boolean success = node.path("success").asBoolean(true);
// This is what I'd like to be able to do but I can't seem to
// find anything like it
JsonNode objectNode = node.path("data").unwrap().firstValue();
T object = MAPPER.treeToValue(objectNode, clazz);
return APIResponse.of(success, object);
}
Is there a way to skip this string and id fields to get at the object? I know I can call .elements on the JsonNode and grab the first one, but is there an easier way?
examining JsonNode javdoc, I came with the following snipet:
JsonNode data = node.path("data");
if (!data.isMissingNode()) { // checks if data exists
if (data.iterator().hasNext()) { // checks if data is not empty
// get first child ("unwrap")
JsonNode firstNodeInsideData = data.iterator().next();
}
}
you can use the same trick to get deeper inside the tree
note: this code sacrifices performance for brevity by calling iterator() twice. you can of course modify that to save the iterator into a variable first.
I'm trying to make sure my Jersey request parameters are sanitized.
When processing a Jersey GET request, do I need to filter non String types?
For example, if the parameter submitted is an integer are both option 1 (getIntData) and option 2 (getStringData) hacker safe? What about a JSON PUT request, is my ESAPI implementation enough, or do I need to validate each data parameter after it is mapped? Could it be validated before it is mapped?
Jersey Rest Example Class:
public class RestExample {
//Option 1 Submit data as an Integer
//Jersey throws an internal server error if the type is not Integer
//Is that a valid way to validate the data?
//Integer Data, not filtered
#Path("/data/int/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getIntData(#PathParam("data") Integer data){
return Response.ok("You entered:" + data).build();
}
//Option 2 Submit data as a String, then validate it and cast it to an Integer
//String Data, filtered
#Path("/data/string/{data}/")
#GET
#Produces(MediaType.TEXT_HTML)
public Response getStringData(#PathParam("data") String data) {
data = ESAPI.encoder().canonicalize(data);
if (ESAPI.validator().isValidInteger("data", data, 0, 999999, false))
{
int intData = Integer.parseInt(data);
return Response.ok("You entered:" + intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
//JSON data, HTML encoded
#Path("/post/{requestid}")
#POST
#Consumes({MediaType.APPLICATION_FORM_URLENCODED, MediaType.APPLICATION_JSON})
#Produces(MediaType.TEXT_HTML)
public Response postData(String json) {
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
//Is there a way to iterate through each JSON KeyValue and filter here?
ObjectMapper mapper = new ObjectMapper();
DataMap dm = new DataMap();
try {
dm = mapper.readValue(json, DataMap.class);
} catch (Exception e) {
e.printStackTrace();
}
//Do we need to validate each DataMap object value and is there a dynamic way to do it?
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
return Response.status(404).entity("404 Not Found").build();
}
}
Data Map Class:
public class DataMap {
public DataMap(){}
String strData;
Integer intData;
}
The short answer is yes, though by "filter" I interpret it as "validate," because no amount of "filtering" will EVER provide you with SAFE data. You can still run into integer overflows in Java, and while those may not have immediate security concerns, they could still put parts of your application in an unplanned for state, and hacking is all about perturbing the system in ways you can control.
You packed waaaaay too many questions into one "question," but here we go:
First off, the lines
json = ESAPI.encoder().canonicalize(json);
json = ESAPI.encoder().encodeForHTML(json);
Aren't doing what you think they're doing. If your JSON is coming in as a raw String right here, these two calls are going to be applying mass rules across the entire string, when you really need to handle these with more surgical precision, which you seem to at least be subconsciously aware of in the next question.
//Is there a way to iterate through each JSON KeyValue and filter
here?
Partial duplicate of this question.
While you're in the loop discussed here, you can perform any data transformations you want, but what you should really be considering is using the JSONObject class referenced in that first link. Then you'll have JSON parsed into an object where you'll have better access to JSON key/value pairs.
//Do we need to validate each DataMap object value and is there a
dynamic way to do it?
Yes, we validate everything that comes from a user. All users are assumed to be trained hackers, and smarter than you. However if you handled filtering before you do your data mapping transformation, you don't need to do it a second time. Doing it dynamically?
Something like:
JSONObject json = new JSONObject(s);
Iterator iterator = json.keys();
while( iterator.hasNext() ){
String data = iterator.next();
//filter and or business logic
}
^^That syntax is skipping typechecks but it should get you where you need to go.
/Is Integer validation needed or will the thrown exception be good
enough?
I don't see where you're throwing an exception with these lines of code:
if (ESAPI.validator().isValidInput("strData", dm.strData, "HTTPParameterValue", 25, false, true))
{
//Is Integer validation needed or will the thrown exception be good enough?
return Response.ok("You entered:" + dm.strData + " and " + dm.intData).build();
}
Firstly, in java we have autoboxing which means this:
int foo = 555555;
String bar = "";
//the code
foo + bar;
Will be cast to a string in any instance. The compiler will promote the int to an Integer and then silently call the Integer.toString() method. Also, in your Response.ok( String ); call, THIS is where you're going to want to encodeForHTML or whatever the output context may be. Encoding methods are ALWAYS For outputting data to user, whereas canonicalize you want to call when receiving data. Finally, in this segment of code we also have an error where you're assuming that you're dealing with an HTTPParameter. NOT at this point in the code. You'll validate http Parameters in instances where you're calling request.getParameter("id"): where id isn't a large blob of data like an entire JSON response or an entire XML response. At this point you should be validating for things like "SafeString"
Usually there are parsing libraries in Java that can at least get you to the level of Java objects, but on the validation side you're always going to be running through every item and punting whatever might be malicious.
As a final note, while coding, keep these principles in mind your code will be cleaner and your thought process much more focused:
user input is NEVER safe. (Yes, even if you've run it through an XSS filter.)
Use validate and canonicalize methods whenever RECEIVING data, and encode methods whenever transferring data to a different context, where context is defined as "Html field. Http attribute. Javascript input, etc...)
Instead of using the method isValidInput() I'd suggest using getValidInput() because it will call canonicalize for you, making you have to provide one less call.
Encode ANY time your data is going to be passed to another dynamic language, like SQL, groovy, Perl, or javascript.
I am using Jackson 2.2 to read a very large JSON string into a Java object. Using ObjectMapper, I read everything into the memory and transform the JSON string to object. This is all happening in memory, so the entire JSON string is loaded into the memory and transformed into the object.
If there a more memory efficient way of doing it? i.e. without loading the entire JSON string into the memory, I should still be able to load the object.
Yes, you typically use so-called "streaming" API to iterate over JSON tokens; and once positioned over first token of the value you want (START_OBJECT for JSON Objects), use data-binding API, passing reader/parser for it to use. Details of this depend on library. I know that at least following support this mode of operation:
Jackson
Gson
Genson
For Jackson, basic Streaming API usage is talked about here (for example); but one thing that does not show is how to bind objects once you are positioned at the right place.
So assuming JSON like:
{ "comment" : "...",
"values" : [
{ ... value object 1 ... },
{ ... value object 2. ... }
]
}
you could do:
ObjectMapper mapper = new ObjectMapper();
JsonParser jp = mapper.getFactory().createJsonParser(jsonInput);
jp.nextToken(); // will return START_OBJECT, may want to verify
while (jp.nextValue() != null) { // 'nextValue' skips FIELD_NAME token, if any
String fieldName = jp.getCurrentName();
if ("values".equals(fieldName)) {
// yes, should now point to START_ARRAY
while (jp.nextToken() == JsonToken.START_OBJECT) {
ValueObject v = mapper.readValue(jp, ValueObject.class);
// process individual value in whatever way to you want to...
}
} else if ("comment".equals(fieldName)) {
// handle comment?
} // may use another else to catch unknown fields, if any
}
jp.close();
and that should let you only bind one object at a time.