We Keep Coding

Java Spring - How to get one specific key value from a very nested json without mapping to a specific pojo?

Small java and spring question regarding how to get a specific key value from a very nested json, without having to map back to java pojos please.
I am consuming an api, where the json response is gigantic. The raw response does not fit in a screen.
The response is also very nested. Meaning, it has fields inside fields inside fields... etc
I have no access, no way to modify this API.
Nonetheless, it is a very interesting API, and in this very gigantic payload, very nested, there is always exactly one "the-key-i-need": "the-value-i-need",
Furthermore, there is no way to know in advanced how nested (which layer, which child) and no way to know where will "the-key-i-need": "the-value-i-need", be.
Also, there is no way to map the response back to any POJO, it changes always, the only information, "the-key-i-need": "the-value-i-need", exists, and it is always there.
I am well aware of GSON or fasterxml libraries, that can help map the json string back to existing pojos.
However, in my case, such will not help, since the existing pojo does not exists. the response is always different, the structure, the level of nesting is always different.
My question is, instead of trying to map back to pojos that will always change, and very nested, is there a simpler way, to just use some kind of regex, or something else, to extract the key value, "the-key-i-need": "the-value-i-need", and only this please?
I already tried mapping to all kinds of pojos, and unfortunately the response structure is too dynamic.
Thank you

Since you're using Spring, you very likely already have Jackson FasterXML in you classpath.
Normally, Spring uses the Databind module, which relies on the Streaming module, aka the Core module.
In this case, you want to use Streaming directly, so get the JSON text as a String, and start the parser.
static String getFieldValue(String json, String field) throws JsonParseException, IOException {
JsonFactory factory = new JsonFactory();
try (JsonParser parser = factory.createParser(json)) {
for (JsonToken token; (token = parser.nextToken()) != null; ) {
if (token == JsonToken.FIELD_NAME && parser.getCurrentName().equals(field)) {
token = parser.nextToken();
// check token here if object or array should throw exception instead of returning null
return parser.getValueAsString();
}
}
}
return null; // or throw exception if "not found" shouldn't return null
}
Test
String json = "{ \"A\": { \"B\": [ 5, { \"C\": \"D\" }, true ], \"E\": null, \"F\": 42, \"G\": false }}";
System.out.println("C: " + getFieldValue(json, "C")); // "D"
System.out.println("E: " + getFieldValue(json, "E")); // null (null-value)
System.out.println("F: " + getFieldValue(json, "F")); // "42"
System.out.println("G: " + getFieldValue(json, "G")); // "false"
System.out.println("H: " + getFieldValue(json, "H")); // null (not found)
System.out.println("B: " + getFieldValue(json, "B")); // null (not a value)

JsonPath
If using an external library is an option, then JsonPath might help.
Maven
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>2.5.0</version>
</dependency>
StackOverflow Tag for JsonPath
Example usage as shared in the README
{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century"
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}
To access all the author names
List<String> authors = JsonPath.read(json, "$..author");
System.out.println(authors);
Output
["Nigel Rees","Evelyn Waugh"]
To access all the prices (across both book and bicycle)
List<Double> prices = JsonPath.read(json, "$..price");
System.out.println(prices);
Output
[12.99, 19.95]
Note
Some missing keys can cause remapping extracted data across two fields difficult
Say fetching category and price from the above example will make it difficult to summarize a category to price mapping
List<String> categories = JsonPath.read(json, "$..category");
System.out.println(categories);
Output
["reference", "fiction"]
Based on the above example, price and category does not have correct 1-1 mapping

Cloudera Navigator API fail to fetch nested data

I am working in Cloudera Manager Navigator REST API where extracting result is working fine, but unable to get any nested value.
The type of data is extracting as below.
{
"parentPath": "String",
"customProperties": "Map[string,string]",
"sourceType": "String",
"entityType": "String"
}
And data should be like
{
"parentPath": "abcd",
"customProperties": {
"nameservice" : "xyz"
},
"sourceType": "rcs",
"entityType": "ufo"
}
But I am getting key-value result as follows.
parentPath :abcd
customProperties : null
sourceType : rcs
entityType : ufo
In above response data, "customProperties" is coming with a null value where it should return a map object contains ["nameservice" : "xyz"]. This is the problem with following code snippet.
MetadataResultSet metadataResultSet = extractor.extractMetadata(null, null,"sourceType:HDFS", "identity:*");
Iterator<Map<String, Object>> entitiesIt = metadataResultSet.getEntities().iterator();
while(entitiesIt.hasNext()){
Map<String, Object> result = entitiesIt.next();
for(String data : result.keySet()){
System.out.println(" key:"+data+" value:"+result.get(data));
}
}
Can you suggest me how to get the nested value where datatype is complex.

have u checked how the data looks on navigator ui? You can first verify that once, and also try cloudera /entities/entity-id rest API in browser to check how json response is coming

java 8 - javax.json - How to get a specific value from pre-made JSON

I have this sample JSON object
{
"Elements" : [
{
"name" : "Hydrogen",
"Symbol" : "H",
"atomicNumber" : "1",
"electronegativity" : "2.2",
"group" : "Hydrogen",
"ionCharge1" : "1+",
"ionCharge2" : "1-",
"molarMass" : "1.01",
"naturalState" : "Gas",
"synthetic" : "false",
"diatomic" : "true",
"columnNumber" : "1",
"row" : "1",
"columnCode" : "IA",
"nobleGasConfiguration" : [
{
"term:" : "No Noble Gas Configuration",
"superScript" : "-"
}
],
"electronConfiguration" : [
{
"term" : "1s",
"superScript" : "1"
}
]
}
}
Through the following code I've gotten the Json database into a JsonStructure.
import javax.json.Json;
import javax.json.JsonReader;
import javax.json.JsonStructure;
import java.io.*;
public class DataTest
{
public static void main(String[]args) throws IOException
{
String strName;
JsonReader reader = Json.createReader(new FileReader("Elements.JSON"));
JsonStructure jsonst = reader.read();
/*strName = jsonst.
* get.JsonObject(String name)
* get.JsonArray(String name)
* get.JsonString(String name).getString()
*/
}
}
What I want to do is simply get the value of "name", that value being "Hydrogen", and this value would be placed in the variable strName.
I've been trying to get basic things like this done for a couple of days, and anything that got somewhere I was derailing myself from my real intentions. Everything else just failed or never worked.
My latest attempt was using the methods commented out at the bottom of the code, I believe I have the methods I need to get this done. (These methods were obtained from the following link: https://docs.oracle.com/javaee/7/api/javax/json/JsonObject.html )
What I had tried to do was:
jsonst.getJsonArray("Elements").getJsonObject(0).getJsonString("name").getString();
This gave me a "cannot find symbol" compile error with a ^ at the period in "jsonst.getJsonArray("Elements")"
So, what am I doing wrong? How can I get this simple task done?
What programs, text-editors, versions, etc am I using?
Command prompt
Notepad
Java 8
javax.json-1.0.jar
Please keep the answers to java and javax.json, one of the tough roads I've slowly ventured was getting a library to use json. I'd rather not go through the trouble of finding another.

So this line is okay to get the reader.
JsonReader reader = Json.createReader(new FileReader("Elements.JSON"));
Your problem is that JsonStructure is an interface that doesn't have the method getJsonArray, but JsonObject does.
Therefore, you need to be doing
JsonObject jsonst = reader.readObject();
And then you can use the methods of JsonObject like so
JsonArray elements = jsonst.getJsonArray("Elements");
for (int i = 0; i < elements.size(); i++) {
JsonObject element = elements.getJsonObject(i);
String name = element.getString("name");
}
Finally, when you are done with the reader, you need to close it.
reader.close();

try:
jsonst.getJsonArray("Elements")[0].getJsonString("name");
and also modify the line which says read instead of readObject:
JsonStructure jsonst = reader.readObject();

reader.readObject().getJsonArray("Elements").getJsonObject(0).getString("name")
Adding maven dependency details
<dependency>
<groupId>javax.json</groupId>
<artifactId>javax.json-api</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.glassfish</groupId>
<artifactId>javax.json</artifactId>
<version>1.0.4</version>
</dependency>

Update nested field in an index of ElasticSearch with Java API

I am using Java API for CRUD operation on elasticsearch.
I have an typewith a nested field and I want to update this field.
Here is my mapping for the type:
"enduser": {
"properties": {
"location": {
"type": "nested",
"properties":{
"point":{"type":"geo_point"}
}
}
}
}
Of course my enduser type will have other parameters.
Now I want to add this document in my nested field:
"location":{
"name": "London",
"point": "44.5, 5.2"
}
I was searching in documentation on how to update nested document but I couldn't find anything. For example I have in a string the previous JSON obect (let's call this string json). I tried the following code but seems to not working:
params.put("location", json);
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location = location").setScriptParams(params).execute().actionGet();
I have got a parsing error from elasticsearch. Anyone knows what I am doing wrong ?

You don't need the script, just update it.
UpdateRequestBuilder br = client.prepareUpdate("index", "enduser", "1");
br.setDoc("{\"location\":{ \"name\": \"london\", \"point\": \"44.5,5.2\" }}".getBytes());
br.execute();

I tried to recreate your situation and i solved it by using an other way the .setScript method.
Your updating request now would looks like :
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location =" + json).execute().actionGet()
Hope it will help you.

I am not sure which ES version you were using, but the below solution worked perfectly for me on 2.2.0. I had to store information about named entities for news articles. I guess if you wish to have multiple locations in your case, it would also suit you.
This is the nested object I wanted to update:
"entities" : [
{
"disambiguated" : {
"entitySubTypes" : [],
"disambiguatedName" : "NameX"
},
"frequency" : 1,
"entityType" : "Organization",
"quotations" : ["...", "..."],
"name" : "entityX"
},
{
"disambiguated" : {
"entitySubType" : ["a", "b" ],
"disambiguatedName" : "NameQ"
},
"frequency" : 5,
"entityType" : "secondTypeTest",
"quotations" : [ "...", "..."],
"name" : "entityY"
}
],
and this is the code:
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.index(indexName);
updateRequest.type(mappingName);
updateRequest.id(url); // docID is a url
XContentBuilder jb = XContentFactory.jsonBuilder();
jb.startObject(); // article
jb.startArray("entities"); // multiple entities
for ( /*each namedEntity*/) {
jb.startObject() // entity
.field("name", name)
.field("frequency",n)
.field("entityType", entityType)
.startObject("disambiguated") // disambiguation
.field("disambiguatedName", disambiguatedNameStr)
.field("entitySubTypes", entitySubTypeArray) // multi value field
.endObject() // disambiguation
.field("quotations", quotationsArray) // multi value field
.endObject(); // entity
}
jb.endArray(); // array of nested objects
b.endObject(); // article
updateRequest.doc(jb);
Blblblblblblbl's answer couldn't work for me atm, because scripts are not enabled in our server. I didn't try Bask's answer yet - Alcanzar's gave me a hard time, because I supposedly couldn't formulate the json string correctly that setDoc receives. I was constantly getting errors that either I am using objects instead of fields or vice versa. I also tried wrapping the json string with doc{} as indicated here, but I didn't manage to make it work. As you mentioned it is difficult to understand how to formulate a curl statement at ES's java API.

A simple way to update the arraylist and object value using Java API.
UpdateResponse update = client.prepareUpdate("indexname","type",""+id)
.addScriptParam("param1", arrayvalue)
.addScriptParam("param2", objectvalue)
.setScript("ctx._source.field1=param1;ctx._source.field2=param2").execute()
.actionGet();
arrayvalue-[
{
"text": "stackoverflow",
"datetime": "2010-07-27T05:41:52.763Z",
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
},
"id": 1,
}
object value -
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
}

Best way to get a list of countries in Java

Other than Locale.getISOCountries() that is, because I'm already getting some strange errors with that. What else is the best way to get the 2-letter country codes as well as the full country name?

See code snippets :
String[] countryCodes = Locale.getISOCountries();
for (String countryCode : countryCodes) {
Locale obj = new Locale("", countryCode);
System.out.println("Country Code = " + obj.getCountry()
+ ", Country Name = " + obj.getDisplayCountry());
}
Refer to this country list in Java for more examples.

For a separate project, I took the country code data from the ISO site.
Beware of the following:
The names are in all caps. You will probably want to tweak it so it's not.
The names are not all in simple ASCII.
The names are not entirely political neutral (it is probably impossible for any purported list of countries to be). E.g., "Taiwan, Province of China" is a name. A good starting point to learn about the issues is this blog post.

Create a Map out of this page http://www.theodora.com/country_digraphs.html
Save it to a file (I suggest XMLEncoder/XMLDecoder class)
Create a wrapping class that loads this Map from the file (I'd use a lazily initialized singleton) and allows access to the get(...) methods.
Repeat (or use a bi-directional map) these steps for each column of the table on the afore mentioned webpage.
Fancy-Time: Throw in some code to wrap the entries in a Reference object (SoftReference?) so that the Map won't throw MemoryErrors

You can use from json bellow like
Json parsing..
String jsonString =JSON_DATA;
ObjectMapper mapper = new ObjectMapper();
try {
JsonNode rootNode = mapper.readTree(jsonString);
for (JsonNode node : rootNode) {
String countrycode = node.path("code").asText();
String dialnumber = node.path("dial_code").asText();
String countryname = node.path("name").asText();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Json String here
public static String JSON_DATA="
[
{
"name": "Afghanistan",
"dial_code": "+93",
"code": "AF"
},
{
"name": "Aland Islands",
"dial_code": "+358",
"code": "AX"
},
{
"name": "Albania",
"dial_code": "+355",
"code": "AL"
},
{
"name": "Algeria",
"dial_code": "+213",
"code": "DZ"
},
{
"name": "AmericanSamoa",
"dial_code": "+1684",
"code": "AS"
}]";
Or You can download full json from link : https://gist.github.com/Goles/3196253