I'm using the following code to parse json
new com.fasterxml.jackson.databind.ObjectMapper().readTree(jsonStr)
But it parses the following string successfully since it looks like it stops processing once it finds a valid tree, even though the string in its entirety is not a valid json.
{
"name": "test",
},
"field": "c"
}
Is there a way to make it consider the entire string or stream passed? I couldn't find an appropriate option in DeserializationFeature.
Note that the solution doesn't have to involve jackson. If there's a simpler way to do that in java or scala, that'll suffice too.
With Jackson you can use Streaming API, JsonParser, to read a json like and validate like follows:
final JsonFactory jsonFactory = new JsonFactory();
jsonFactory.enable(JsonParser.Feature.STRICT_DUPLICATE_DETECTION);
try (JsonParser parser = jsonFactory.createParser(invalidJson)) {
while (!parser.isClosed()) {
parser.nextToken();
}
}
For example, if there is json string of
{
"name": "test"
},
"field": "c"
}
A JsonParseException will be thrown as follows:
Exception in thread "main"
com.fasterxml.jackson.core.JsonParseException: Unexpected character
(',' (code 44)): expected a valid value (number, String, array,
object, 'true', 'false' or 'null') at [Source: {
"name": "test"
},
"field": "c"
}; line: 3, column: 3]
jsonFactory.enable(JsonParser.Feature.STRICT_DUPLICATE_DETECTION) is to explicitly check that no duplicate JSON Object field names are encountered. If enabled, parser will check all names within context and report duplicates by throwing a JsonParseException
According to : http://jsonlint.com/
The JSON you are using is not valid. You might want to correct it to get rid of the Exception :
{
"name": "test",
"field": "c"
}
Related
I am trying to learn how to parse JSON data to CSV format with Apache Beam and Jackson. I am starting with a very simple JSON file:
{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 27
}
I have a corresponding POJO structure:
import java.io.Serializable;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
#JsonIgnoreProperties(ignoreUnknown = true)
public class Person implements Serializable {
private String firstName;
private String lastName;
private int age;
public Person() {}
public String getFirstName() {
return firstName;
}
... getters & setters ...
However, when I try to parse this json, I get a formatting error:
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected close marker '}': expected ']' (for root starting at [Source: }; line: 1, column: 0])
at [Source: }; line: 1, column: 2]
Which I solved by converting the json to this format:
{"firstName": "John", "lastName": "Smith", "isAlive": true, "age": 27}
My eventual need is to deal with plain old json. Is there a way to do so, and if so, how?
The Apache Beam code is this simple pipeline:
public class DataToModel {
public static void main(String[] args) {
PipelineOptions options = PipelineOptionsFactory.create();
options.setRunner(DirectRunner.class);
Pipeline p = Pipeline.create(options);
// read data from json
PCollection<String> json = p.apply(TextIO.read().from("src/main/resources/test.json"));
PCollection<Person> person = json
.apply(ParseJsons.of(Person.class))
.setCoder(SerializableCoder.of(Person.class));
// parse json
PCollection<String> names = person.apply(MapElements
.into(TypeDescriptors.strings())
.via(Person::getFirstName)
);
// write information to file.
names.apply(TextIO.write().to("src/main/resources/test_out"));
p.run().waitUntilFinish();
}
The problem is that you are using TextIO.read() to read from the json files. TextIO reads each line of a text file into a separate element, so a multi-line JSON object gets split into multiple elements. That means your parse function attempts to parse JSON strings such as };. That also explains why it succeeds if you format your object entirely on one line.
There are two approaches you could go with depending on what's available to you.
If possible with your JSON files, you could use the withDelimiter method to use a custom delimeter aside from the default newline. However this is pretty brittle and requires your files to be formatted very specifically.
You could switch from TextIO to FileIO, and read each file into a single string to send to ParseJsons. This is slightly more work but far less brittle, and is what I would recommend.
You can use the org.json library; it is easy to use.
Just remember (while casting or using methods like getJSONObject and getJSONArray) that in JSON notation
[ … ] represents an array, so library will parse it to JSONArray
{ … } represents an object, so library will parse it to JSONObject
You can see more information about.
You can see a simple example below:
JSON file:
{
"pageInfo": {
"pageName": "abc",
"pagePic": "http://example.com/content.jpg"
},
"posts": [
{
"post_id": "123456789012_123456789012",
"actor_id": "1234567890",
"picOfPersonWhoPosted": "http://example.com/photo.jpg",
"nameOfPersonWhoPosted": "Jane Doe",
"message": "Sounds cool. Can't wait to see it!",
"likesCount": "2",
"comments": [],
"timeOfPost": "1234567890"
}
]
}
Code example:
import org.json.*;
String jsonString = ... ; //assign your JSON String here
JSONObject obj = new JSONObject(jsonString);
String pageName = obj.getJSONObject("pageInfo").getString("pageName");
JSONArray arr = obj.getJSONArray("posts"); // notice that `"posts": [...]`
for (int i = 0; i < arr.length(); i++)
{
String post_id = arr.getJSONObject(i).getString("post_id");
......
}
You may find more examples here: Parse JSON in Java.
Downloadable jar.
I was hoping to use Jackson to find JSON diff but it does not give detailed error messages.
So I tried using JSOnAssert to find the diff between two JSON strings.
JSONAssert.assertEquals(expectedJsonResponse, actualJsonResponse, false);
Sadly, it does not appear to match correctly and give the detailed error messages as in the examples. If you have used it, Can you please clarify?
java.lang.AssertionError: data[0] Could not find match for element {"errors":[{"httpStatus":"BAD_REQUEST","personId":null,"details":"User ID [UNKNOWN]. Invalid ID: NONSENSE"}],"successfulIds":["A0","B1","C3"]}
at org.skyscreamer.jsonassert.JSONAssert.assertEquals(JSONAssert.java:222)
Actual JSON:
{"_links":{"self":{"href":"https://myserver.com:1000/api/person/upload? myCsvFile={myCsvFile}","templated":true}},"data":[{"successfulIds":["A0","XYZ","C3"],"errors":[{"personId":null,"httpStatus":"BAD_REQUEST","details":"User ID [UNKNOWN]. Invalid ID: NONSENSE"}]}]}
Expected JSON:
{
"_links": {
"self": {
"href": "https://myserver.com:1000/api/person/upload?myCsvFile={myCsvFile}",
"templated": true
}
},
"data": [
{
"successfulIds": [
"A0",
"B1",
"C3"
],
"errors": [
{
"personId": null,
"httpStatus": "BAD_REQUEST",
"details": "User ID [UNKNOWN]. Invalid ID: NONSENSE"
}
]
}
]
}
I tried to email the address at http://jsonassert.skyscreamer.org/ but got a
The following message to jsonassert-dev#skyscreamer.org was
undeliverable. The reason for the problem:
5.1.0 - Unknown address error 550-"5.1.1 The email account that you tried to reach does not exist
So I tried ZJsonPatch. I like the fact that using Jackson with it, the ordering of the members does not matter. In other words, I first try to check for equality using Jackson. Jackson is ordering independent. Then if it fails, I use ZJsonPatch to tell me what the diff is.
{"op":"replace","path":"/data/0/successfulIds/1","value":"B9"}
which handles nested JSON well.
ObjectMapper mapper = new ObjectMapper();
JsonNode expected = mapper.readTree(expectedJsonResponse);
JsonNode actual = mapper.readTree(actualJsonResponse);
try {
assertEquals(expected, actual);
} catch (AssertionError ae) {
JsonNode patch = JsonDiff.asJson(actual, expected);
throw new Exception(patch.toString(), ae);
}
Sometime client send Json-RPC request with Json value as unicorde symboles.
Example:
{ "jsonrpc": "2.0", "method": "add", "params": { "fields": [ { "id": 1, "val": "\u0414\u0435\u043d\u0438\u0441" }, { "id": 2, "val": "\u041c\u043e\u044f" } ] }, "id": "564b0f7d-868a-4ff0-9703-17e4f768699d" }
How do I processing Json-RPC request:
My server get the request like byte[];
Convert it to io.vertx.core.json.JsonObject;
Make some manipulations;
Save to DB;
And I found in DB records like:
"val": "\u0414\u0435\u043d\u0438\u0441"
And the worst in this story. If client try to search this data, he'll get:
"val": "\\u0414\\u0435\\u043d\\u0438\\u0441"
So I think, that I need to convert request data before deserialization to JsonObject.
I tried and it didn't help:
String json = new String(incomingJsonBytes, StandardCharsets.UTF_8);
return json.getBytes(StandardCharsets.UTF_8);
Also I tried to use StandardCharsets.US_ASCII.
Note: Variant with StringEscapeUtils.unescapeJava() I can not, because it unescape all necessary and unnecessary '\' symbols.
If anyone know how to solve it? Or library that already makes it?
Thank a lot.
io.vertx.core.json.JsonObject depends on Jackson ObjectMapper to perform the actual JSON deserialization (e.g. io.vertx.core.json.Json has a ObjectMapper field). By default Jackson will convert \u0414\u0435\u043d\u0438\u0441 into Денис. You can verify this with a simple code snippet:
String json = "{ \"jsonrpc\": \"2.0\", \"method\": \"add\", \"params\": { \"fields\": [ { \"id\": 1, \"val\": \"\\u0414\\u0435\\u043d\\u0438\\u0441\" }, { \"id\": 2, \"val\": \"\\u041c\\u043e\\u044f\" } ] }, \"id\": \"564b0f7d-868a-4ff0-9703-17e4f768699d\" }";
ObjectMapper mapper = new ObjectMapper();
Map map = mapper.readValue(json, Map.class);
System.out.println(map); // {jsonrpc=2.0, method=add, params={fields=[{id=1, val=Денис}, {id=2, val=Моя}]}, id=564b0f7d-868a-4ff0-9703-17e4f768699d}
Most likely the client is sending something else because your example value is deserialized correctly. Perhaps it's doubly escaped \\u0414\\u0435\\u043d\\u0438\\u0441 value which Jackson will convert to \u0414\u0435\u043d\u0438\u0441 removing one layer of escaping?
There is no magic solution for this. Either write your own Jackson deserialization configuration or make the client stop sending garbage.
I've following JSON structure coming in,
{
"name": "product new",
"brand": {
"id": 1
},
"category": {
"id": 1
}
}
I can extract
jsonObject = Json.createReader(httpServletRequest.getInputStream()).readObject();
jsonObject.getString("name")
Errors:
jsonObject.getInt("brand.id")
jsonObject.getInt("category.id")
I'm using Java API for JSON.
Edit If I access
System.out.println(jsonObject.get("brand"));
// response {"id":1}
System.out.println(jsonObject.get("brand.id"));
// null
http://www.oracle.com/technetwork/articles/java/json-1973242.html
I don't think the API you're using supports nested expressions. You'll need to access the parent object, and then the specific field:
System.out.println(jsonObject.getJsonObject("brand").getInt("id"));
Or you can use an API that accepts a path expression, like Jackson:
JsonNode node = new ObjectMapper().readTree(httpServletRequest.getInputStream());
System.out.println(node.at("/brand/id").asInt());
i have a JSON like example below and i'm trying to get some values, for example value of.
results.shipper.id
{
"results": [
{
"updated": false,
"notification": false,
"some_data": {
"id": 15989,
"pieces": 0,
},
"shipper": {
"updated": false,
"notification": false,
"id": 1587,
"parent": {
"updated": false
},
I'm trying to get value by this way:
String test = shipmentData.getJSONObject("shipper").getString("id");
But it always throws a exception. I think, that exception is caused because of the i am not accessing to the values via "results" array.
How can i easy access to the value what i need.
I tried find some helpers (Gson, fast-json, etc..) but it seems to be a quite complicated for using (i would like to work with JSON tree for direct access to values "on-the-fly" or access to values like to a object, it means.. Object.InnerObject.value ).
So question is how can i do it right?
Thanks for any advice.
JSON needs to be traversed in order to access id:
JSONArray results = shipmentData.getJSONArray("results");
JSONObject first = results.getJSONObject(0);
JSONObject shipper = first.getJSONObject("shipper");
Integer id = shipper.getInt("id");
Parse int to string:
String id = String.valueOf(shipper.getInt("id"));