We Keep Coding

Java Spring - How to get one specific key value from a very nested json without mapping to a specific pojo?

Small java and spring question regarding how to get a specific key value from a very nested json, without having to map back to java pojos please.
I am consuming an api, where the json response is gigantic. The raw response does not fit in a screen.
The response is also very nested. Meaning, it has fields inside fields inside fields... etc
I have no access, no way to modify this API.
Nonetheless, it is a very interesting API, and in this very gigantic payload, very nested, there is always exactly one "the-key-i-need": "the-value-i-need",
Furthermore, there is no way to know in advanced how nested (which layer, which child) and no way to know where will "the-key-i-need": "the-value-i-need", be.
Also, there is no way to map the response back to any POJO, it changes always, the only information, "the-key-i-need": "the-value-i-need", exists, and it is always there.
I am well aware of GSON or fasterxml libraries, that can help map the json string back to existing pojos.
However, in my case, such will not help, since the existing pojo does not exists. the response is always different, the structure, the level of nesting is always different.
My question is, instead of trying to map back to pojos that will always change, and very nested, is there a simpler way, to just use some kind of regex, or something else, to extract the key value, "the-key-i-need": "the-value-i-need", and only this please?
I already tried mapping to all kinds of pojos, and unfortunately the response structure is too dynamic.
Thank you

Since you're using Spring, you very likely already have Jackson FasterXML in you classpath.
Normally, Spring uses the Databind module, which relies on the Streaming module, aka the Core module.
In this case, you want to use Streaming directly, so get the JSON text as a String, and start the parser.
static String getFieldValue(String json, String field) throws JsonParseException, IOException {
JsonFactory factory = new JsonFactory();
try (JsonParser parser = factory.createParser(json)) {
for (JsonToken token; (token = parser.nextToken()) != null; ) {
if (token == JsonToken.FIELD_NAME && parser.getCurrentName().equals(field)) {
token = parser.nextToken();
// check token here if object or array should throw exception instead of returning null
return parser.getValueAsString();
}
}
}
return null; // or throw exception if "not found" shouldn't return null
}
Test
String json = "{ \"A\": { \"B\": [ 5, { \"C\": \"D\" }, true ], \"E\": null, \"F\": 42, \"G\": false }}";
System.out.println("C: " + getFieldValue(json, "C")); // "D"
System.out.println("E: " + getFieldValue(json, "E")); // null (null-value)
System.out.println("F: " + getFieldValue(json, "F")); // "42"
System.out.println("G: " + getFieldValue(json, "G")); // "false"
System.out.println("H: " + getFieldValue(json, "H")); // null (not found)
System.out.println("B: " + getFieldValue(json, "B")); // null (not a value)

JsonPath
If using an external library is an option, then JsonPath might help.
Maven
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>2.5.0</version>
</dependency>
StackOverflow Tag for JsonPath
Example usage as shared in the README
{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century"
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}
To access all the author names
List<String> authors = JsonPath.read(json, "$..author");
System.out.println(authors);
Output
["Nigel Rees","Evelyn Waugh"]
To access all the prices (across both book and bicycle)
List<Double> prices = JsonPath.read(json, "$..price");
System.out.println(prices);
Output
[12.99, 19.95]
Note
Some missing keys can cause remapping extracted data across two fields difficult
Say fetching category and price from the above example will make it difficult to summarize a category to price mapping
List<String> categories = JsonPath.read(json, "$..category");
System.out.println(categories);
Output
["reference", "fiction"]
Based on the above example, price and category does not have correct 1-1 mapping

Removing escape characters in json using Java

I am new in Java coding, I am trying to post a JSOn file, here is my JSON file object
{
"report": "[{\"patientId\":\"abcd-efg\",\"campaignId\":\"2\",\"phoneCallsMade\":\"[]\",\"message\":\"[]\"}, {\"patientId\":\"abcd-efg\",\"campaignId\":\"2\",\"phoneCallsMade\":\"[]\",\"message\":\"[]\"}]"
}
I am trying to remove the backslash, I tried below methods :
myJsonString.replaceAll("\\","");
myJsonString=myJsonString.replaceAll("\\\\","");
but after doing that the json format is not valid. Can someone help me please.

This is just a string not a json array. And the reason of getting invalid json post removing slashes is due to in correct mapping of double quotes. Just remove double quotes from "[ & ]" from beginning and from end and perform the operation. Your problem will be resolved.
{
"report": [{
"patientId": "abcd-efg",
"campaignId": "2",
"phoneCallsMade": "[]",
"message": "[]"
}, {
"patientId": "abcd-efg",
"campaignId": "2",
"phoneCallsMade": "[]",
"message": "[]"
}]
}
This will be the output.

Valid Json throws "Expected BEGIN_OBJECT but was STRING at line 1 column 4"

I know lot of people asked similars questions but, hell I'm stuck and I don't understand why. That's weird because the JSON is valid on jsonlint :
{
"Name": "Geography",
"Questions": [{
"_question": "Where is Max?",
"_answers": ["France", "USA", "Spain", "Tunisia"],
"_correctOne": 2,
"Asked": false,
"ID": 0
}, {
"_question": "Where is the Eiffel Tower?",
"_answers": ["Marseilles", "Le Mans", "Paris", "Lyon"],
"_correctOne": 3,
"Asked": false,
"ID": 1
}, {
"_question": "Where is Barcelona?",
"_answers": ["Italy", "Germany", "Portugual", "Spain"],
"_correctOne": 4,
"Asked": false,
"ID": 2
}, {
"_question": "Where is Malibu point?",
"_answers": ["San Francisco", "San Diego", "Los Angeles", "It\u0027s just in a movie"],
"_correctOne": 3,
"Asked": false,
"ID": 3
}, {
"_question": "Where takes place the famous 24h of Le Mans?",
"_answers": ["France", "Belgium", "Canada", "Martinique"],
"_correctOne": 1,
"Asked": false,
"ID": 4
}, {
"_question": "Which one of the following countries is the biggest one?",
"_answers": ["Mexico", "USA", "Russia", "India"],
"_correctOne": 3,
"Asked": false,
"ID": 5
}, {
"_question": "Where can you find a Camel?",
"_answers": ["Siberia", "Antartic", "Artic", "Sahara"],
"_correctOne": 4,
"Asked": false,
"ID": 6
}, {
"_question": "Where can\u0027t you find the statue of the liberty?",
"_answers": ["New York", "Paris", "Las Vegas", "Strasbourg"],
"_correctOne": 2,
"Asked": false,
"ID": 7
}, {
"_question": "What did Christophe Colomb has discovered?",
"_answers": ["Europe", "America", "Africa", "Asia"],
"_correctOne": 2,
"Asked": false,
"ID": 8
}, {
"_question": "Where can\u0027t you practice sky?",
"_answers": ["Maroco", "Canada", "Norway", "Dubaï"],
"_correctOne": 1,
"Asked": false,
"ID": 9
}, {
"_question": "Which one of the following countries isn\u0027t a neighboor of the France?",
"_answers": ["Germany", "Italy", "Spain", "Portugual"],
"_correctOne": 4,
"Asked": false,
"ID": 10
}]
}
So, it's just an ArrayList<Category>. Category is the following class:
public class Category {
public String Name;
public ArrayList<Question> Questions;
}
Also, the Question class below:
public class Question {
private String _question;
private String[] _answers;
private int _correctOne;
public boolean Asked;
public int ID;
}
Everything seems alright to me, I checked again and again, still have this error. However, something seems weird, each ' is replaced by \u0027, but it doesn't seems to be the problem..
I'm parsing the Gson from the following function:
public static boolean ReadCategoryFileTxt(Category category) {
try {
File file = new File(category.Name + ".txt");
Scanner sc = new Scanner(file);
String JSONString = "";
while (sc.hasNextLine())
JSONString += sc.nextLine();
Gson gson = new Gson();
Category _category = gson.fromJson(JSONString, Category.class);
category.Name = _category.Name;
category.Questions = _category.Questions;
//Debug.Println(gson.toJson(category, Category.class));
sc.close();
}
catch (Exception e)
{
Debug.PrintException(e);
return (false);
}
return (true);
}
Any idea for this "Expected BEGIN_OBJECT but was STRING at line 1 column 4"?
Thank in advance !

Looks the start of the JSON read from file is not open flower braces {
Point 1 - See if the file has any incorrect start characters
Point 2 - Set your text file encoding to UTF -8
Point 3 - Use String Builder instead of using + append assignment
Point 4 - If you have notepad++, you shall remove non-ascii characters using - Notepad++, How to remove all non ascii characters with regex?
public static boolean ReadCategoryFileTxt(Category category) {
try {
File file = new File(category.Name + ".txt");
Scanner sc = new Scanner(file);
StringBuilder JSONString = new StringBuilder;
while (sc.hasNextLine())
JSONString = JSONString.append(sc.nextLine());
Gson gson = new Gson();
Category _category = gson.fromJson(JSONString.toString().trim(), Category.class);
category.Name = _category.Name;
category.Questions = _category.Questions;
//Debug.Println(gson.toJson(category, Category.class));
sc.close();
}
catch (Exception e)
{
Debug.PrintException(e);
return (false);
}
return (true);
}

I'd like to leave a few words based on what you clarified in the comments. To be honest, I could not reproduce the exact exception with ... line 1 column 4 having ... line 1 column 1 instead, regardless the default JVM file.encoding property, but you're faced with a classic file encoding issue.
Your JSON is really well-formed.
Java promotes some naming conventions so _question and Asked appearing in the JSON can be expressed as #SerializedName("_question") String question; and #SerializedName("Asked") boolean isAsked; respectively.
\u0027 is an escaped character of '. Characters can be unescaped if they are a part of the enclosing document encoding. The apostrophe fits ASCII just perfect, however it may be escaped in order not to break JSON documents syntactically, say, string literals in JSON strings.
The real problem was that your JSON file was not invalid characters per se, but an UTF-8 encoded file with an explicit Byte Order Mark that should be processed before parsing begins to make sure that file content encoding is detected and decoded fine ( literally stands for UTF-8). UTF-aware editors just do not show BOMs since the latter are not considered bad or illegal characters, but use them to deal with UTF-encoded files (file encoding names are usually shown in statusbars, etc). That's why copying/pasting worked for you: BOMs were not included to copies (they "live" in files only, and in clipboard metadata I guess). By default, Java classes do not make any assumptions on the incoming file encoding leaving the decision on it on your own or even on user's own. It is described here. Therefore Gson should not process it and Gson really does not do it. All Gson can only consume is JSON tokens streams (and that's perfect design), so you have either to detect the input encoding, or let your user specify it.
You don't need to accumulate an intermediate String to parse a JSON: this is just memory wasting especially for big JSON documents. Gson can work in streaming manner, so it accepts Reader instances in order not to accumulate intermediate JSON strings reading a given input stream token by token trying to consume as less memory as possible. Also note that reading line by line is cheaper with BufferedReader rather than Scanner, however it's still discouraged for you case.
If you're using Java 7 or later, use try-with-resources to make sure your I/O resources are not leaked (otherwise use finally to make sure there are no resource leaks). Also it's worth noting that instantiating a Gson instance may be considered a little expensive operation, and since Gson instances are thread-safe and immutable, they can be instantiated once per application and shared globally (not necessarily a public static final Gson ..., of course, but something encapsulated elsewhere in a good design solution).
try( final Reader reader = new InputStreamReader(new FileInputStream(new File(category.Name + ".txt")), StandardCharsets.UTF_8)) {
Category _category = gson.fromJson(reader, Category.class);
category.Name = _category.Name;
category.Questions = _category.Questions;
//Debug.Println(gson.toJson(category, Category.class));
} catch ( Exception e ) {
Debug.PrintException(e);
return (false);
}
return (true);

Why do I always get JSON Exception?

This Is my first time with parsing JSON data. I am using the Google knowledge graph api. I got the api working and I can get the JSON result. This is Google 's sample return data for a sample query which I'm using now for testing.
{
"#context": {
"#vocab": "http://schema.org/",
"goog": "http://schema.googleapis.com/",
"resultScore": "goog:resultScore",
"detailedDescription": "goog:detailedDescription",
"EntitySearchResult": "goog:EntitySearchResult",
"kg": "http://g.co/kg"
},
"#type": "ItemList",
"itemListElement": [
{
"#type": "EntitySearchResult",
"result": {
"#id": "kg:/m/0dl567",
"name": "Taylor Swift",
"#type": [
"Thing",
"Person"
],
"description": "Singer-songwriter",
"image": {
"contentUrl": "https://t1.gstatic.com/images?q=tbn:ANd9GcQmVDAhjhWnN2OWys2ZMO3PGAhupp5tN2LwF_BJmiHgi19hf8Ku",
"url": "https://en.wikipedia.org/wiki/Taylor_Swift",
"license": "http://creativecommons.org/licenses/by-sa/2.0"
},
"detailedDescription": {
"articleBody": "Taylor Alison Swift is an American singer-songwriter and actress. Raised in Wyomissing, Pennsylvania, she moved to Nashville, Tennessee, at the age of 14 to pursue a career in country music. ",
"url": "http://en.wikipedia.org/wiki/Taylor_Swift",
"license": "https://en.wikipedia.org/wiki/Wikipedia:Text_of_Creative_Commons_Attribution-ShareAlike_3.0_Unported_License"
},
"url": "http://taylorswift.com/"
},
"resultScore": 896.576599
}
]
}
So I want to parse it so that I can get the name, description, detailed description. This is my code but I always seem to get the exception. Any ideas why?
try {
JSONObject object=new JSONObject(gggg);
JSONArray itemListElement = object.getJSONArray("itemListElement");
for(int i=0; i < itemListElement.length();i++){
JSONObject c = itemListElement.getJSONObject(i);
JSONObject results = c.getJSONObject("result");
String name = results.getString("name").toString();
String description = results.getString("description").toString();
String detailedDescription = results.getString("articleBody").toString();
gggg = "Name: "+name+"\n Description: "+description+"\n "+detailedDescription;
}
responseView.append(gggg);
} catch (JSONException e) {
Toast.makeText(MainActivity.this,gggg,Toast.LENGTH_LONG).show();
}
Also the string gggg contains the JSON data. I don't know why but I am always getting the exception. Please tell me what is the error in my code and how to repair it.
Thanks.

"Name: Taylor Swift Description: Singer-songwriter Taylor Alison
Swift is an American singer-songwriter and actress. Raised in
Wyomissing, Pennsylvania, she moved to Nashville, Tennessee, at the
age of 14 to pursue a career in country music. "
The problem is your String detailedDescription line.
You need to get the detailedDescription object before you retrieve the articleBody.
for(int i=0; i < itemListElement.length();i++){
JSONObject c = itemListElement.getJSONObject(i);
JSONObject results = c.getJSONObject("result");
String name = results.getString("name");
String description = results.getString("description");
JSONObject detailedDescription = results.getJSONObject("detailedDescription");
String articleBody = detailedDescription.getString("articleBody");
String x = "Name: "+name+"\n Description: "+description+"\n "+articleBody;
}
Also your .toString() method calls are redundant as you are calling .getString() on the JSON object.

With in the android json library it has a method called has element, which returns true or false. After successfully checking then access the element. The expection be caused by tring to access an element that isn't there.
Might be worth printing out after each time you create a new object to ensure that the objects are being created. It will also piont to where the expection is happening.

How do I read multiple JSON objects in Java?

So this is my sample Json Text,
{
"Date": [
1545,
"20 January 2014"
],
"docText": "ABCD",
"docSource": "",
"docID": 99,
"docDate": "",
"Date": [
1930,
"1995/11",
"past decade",
"today",
"Today"
],
"docText": "JJJJJJJ\"",
"Organization": [
"x",
"y",
"z"
],
"docSource": "",
"docID": 98,
"docDate": "",
"Person": "xxxxxx"
}
Now I need a Java code to Read from this file and Display all the docText, docID Entities. I'm able to retrieve only one Entity.
This is the part of the code im using.
JSONParser jsonParser = new JSONParser();
try {
FileReader fileReader = new FileReader(jsonFilePath);
JSONObject jsonObject = (JSONObject) jsonParser.parse(fileReader);
String docid = (String) jsonObject.get("docText");
System.out.println("DocText: " +docid);
long members = (long) jsonObject.get("docID");
System.out.println("docID: " + members);
JSONArray names = (JSONArray) jsonObject.get("Organization");
Iterator i = names.iterator();
while (i.hasNext()) {
System.out.println("Organization: "+i.next());
I really need this working soon! Thank you

The JSON file has duplicate keys so you probably can't read them all using a standard parser.
From http://www.ietf.org/rfc/rfc4627.txt:
An object structure is represented as a pair of curly brackets surrounding zero or more name/value pairs (or members). A name is a string. A single colon comes after each name, separating the name from the value. A single comma separates a value from a following name. The names within an object SHOULD be unique.
Instead, I'd expect each entity to be a separate object within an array.

AFAIK there is no option to fetch multiple values from a JSON object in a single get...