We Keep Coding

Best way to compare JSON array with another JSON array with Regex Values?

I have to validate the values of output json file. First validation is all the key value pairs needed are in the file and the second one, values are in the correct format as expected ( say, timestamp, string, integer ...). I cannot compare the values directly with another file having same content as some of the values like timestamp, id are random.
JSON File:
[
{
"metadata": {
"RecordEnd": 19,
"Type": null,
"RecordOffset": 0,
"Character_Set": "UTF-8",
"MsgType": 8,
"Expire": 0,
"Name": "y1",
"delivered": false,
"Timestamp": 1664189426609,
"UserID": "1jnj2232",
"Encoding": 273,
"id": "ID:414d51205120202020210040",
"DType": "type1"
}
]
I have two approches to do this.
Create a hashmap with keys as same in the JSON file and have values as regex patterns and compare each key value pairs with the regex matching the keys.
HashMap<String,String> metaData = new HashMap();
metaData.put("RecordEnd","\\d+");metaData.put("Type","\\w+");
metaData.put("RecordOffset","\\d+");metaData.put("Character_Set","UTF-8");
metaData.put("MsgType","\\d+");metaData.put("Expire","\\w+");
metaData.put("Name","\\w+");metaData.put("delivered","\\w+");
metaData.put("Timestamp","\\d+");metaData.put("UserID","\\w+");
metaData.put("Encoding","\\d+");metaData.put("id","ID\\:\\w+");
metaData.put("DType","type1");
ObjectMapper mapper = new ObjectMapper();
String json = FileUtils.readFileToString(new File(outputJSONFile),"UTF-8");
json = json.substring(1, json.length() - 1);
Map<?, ?> map = mapper.readValue(json, Map.class);
HashMap<String,Object> metaMap = (HashMap<String, Object>) map.get("metadata");
metaMap.entrySet().forEach(e-> {
if (!(e.getValue() == null)) {
if (e.getValue().toString().matches(metaData.get(e.getKey()))) {
log.info(e + "- Matched");
} else {
throw new RuntimeException(
"MetaData key " + e.getKey() + " data is invalid");
}
}
});
Here, if the fields size become larger ( say 40+ fields) , I have to add hashmap values for all the fields , it becomes a tedious process and reduces readability.
Create a static json file with the all the keys and the values in the regex format. Then, compare the two files based on regex of one file to match with another file. I haven't tried this yet.
Is the second approach more efficient or is there a more suitable approach to do this?

How to get json value with jackson?

String url = "https://ko.wikipedia.org/w/api.php?action=query&format=json&list=search&srprop=sectiontitle&srlimit=1&srsearch=grand-theft-auto-v";
String test = restTemplate.getForObject(url, String.class);
Map<String, String> testToJson = objectMapper.readValue(test, Map.class);
testToJson is:
{
batchcomplete: "",
continue: {
sroffset: 1,
continue: "-||",
},
query: {
searchinfo: {
totalhits: 12
},
search: [
{
ns: 0,
title: "그랜드 테프트 오토 V",
pageid: 797633,
}
],
},
}
I want to get title value.
I try
testToJson.get("title")
but it returns null.
How to get title value with jackson?

You can deserialise it to a JsonNode and use JSON Pointer to get required field:
JsonNode node = mapper.readValue(jsonFile, JsonNode.class);
String title = node.at("/query/search/0/title").asText();

you could build a class for this json result then read from it.
public class Result {
private JsonNode searchinfo;
private JsonNode[] searches;
}
// then read:
Result testToJson = objectMapper.readValue(test, Result.class);
System.out.println(testToJson.getSearches(0).get("title"));
refer

It is impossible to read JSON into an instance of a generic class like that because the info about generics are used in compile time and already lost when program is running.
Jackson captures the data about generics using a sub-classed instance of TypeReference<T>.
Map<String, String> testToJson = objectMapper.readValue(test, new TypeReference<Map<String, String>>(){});
The problem with this approach is that Map<String, String> almost never describes complex data (like in the example) correctly. The example contains not only string values, there are numbers and even nested objects.
In situations like that, when you don't want or cannot write a class that describes the structure of the JSON, the better choice is parsing the JSON into a tree structure and traverse it. For example:
JsonNode node = objectMapper.readTree(test);
String title = node.get("query").get("search").get(0).get("title").asText();
Integer offset = node.get("continue").get("strOffset").asInt()

Parse a URI String into a JSON object

I've got the URI like this:
http://localhost:8080/profile/55cbd?id=123&type=product&productCategories.id=ICTLicense&productCategories.name.firstName=Jack&productCategories.name.lastName=Sparrow&groups=a&groups=b
I need a JSON object like this:
{
"id": "123",
"type": "product",
"productCategories": {
"id": "ICTlicense",
"name": {
"firstName": "Jack",
"lastName": "Sparrow"
}
},
"groups":["a", "b"]
}
Query parameters nesting can be dynamic, like for example abc.def.ghi.jkl.mno=value1&abc.xyz=value2 will result in
{
"abc": {
"def": {
"ghi": {
"jkl": {
"mno": "value1"
}
}
},
"xyz": "value2"
}
}
I have tried this but it can not handle the nesting.
final Map<String, String> map = Splitter.on('&').trimResults().withKeyValueSeparator('=').split(request.getQuery());
How to do this in Java?

With the way that your URI string is structured it wouldn't be possible to nest it the way you'd like, here's why.
id=123 This is simple enough since id would just be an int
productCategories.id=ICTLicense This would also be simple enough since we can assume that productCategories is an object and id is a key inside of the object
However, it gets more complex when you start using arrays, for instance:
&groups=a&groups=b
How do you know that groups is an array, and not simply a key called groups with a value of a or b
Also, you store all your data to Map<String, String>,
This wouldn't support arrays as it stores objects to key-value, so you wouldn't be able to have multiple keys of groups with different values.
I'd also suggest you use a library like Gson and parse your data to a JsonObject
https://github.com/google/gson
If you were to use Gson, you could do something similar to this:
public JsonObject convertToJson(String urlString) {
//Create a JsonObject to store all our data to
JsonObject json = new JsonObject();
//Split the data part of the url by the props
String[] props = urlString.split("&");
//Loop through every prop in the url
for (String prop : props) {
//Create a list of all the props and nested props
String[] nestedProps = prop.split("=")[0].split("\\.");
//Get the actual key for our prop
String key = nestedProps[nestedProps.length - 1];
//Get the value
String value = prop.split("=")[1];
//Loop through our props array
for (String nestedProp : nestedProps) {
//If the property already exists, then skip
if (json.has(nestedProp)) continue;
//If the prop is the key, add it to the json object
if(nestedProp.equalsIgnoreCase(key)) {
json.addProperty(nestedProp, value);
continue;
}
//If the above checks fail, then create an object in the json
json.add(nestedProp, new JsonObject());
}
}
return json;
}

How to construct this JSON array of JSONObjects

I have a Problem which I have to solve in Java.I have a data in YAML where the data is in this structure
600450:
STATE:STATE1
CITY:CITY1
ID:1
CONTACT:1234
600453:
STATE:STATE1
CITY:CITY1
ID:2
CONTACT:3456
600451:
STATE:STATE2
CITY:CITY2
ID:3
CONTACT:2234
.....
I converted this into JSONObject but I am strugling to change this into this JSONObject object where the structure should be of this form
{
STATE1:
{[
CITY1:{
[{ID:1,CODE:600450,CONTACT:1234},
{ID:2,CODE:600453,CONTACT:3456}
]
},
CITY2:{
[
{ID:3,CODE:600451,CONTACT:1234}
]
}
]}
}
I have almost lost a hour by doing different Things with JSONObject and JSONArray and then switched to HashMap and ArrayList of HashMap but I am not able to get it !
This was my try I am sure this is absurd I know that How to achieve this in Java .

Assuming that your converted initial JSON looks like this:
{
"600450": {
"STATE": "STATE1",
"CITY": "CITY1",
"ID": 1,
"CONTACT": 1234
},
"600451": {
"STATE": "STATE2",
"CITY": "CITY2",
"ID": 3,
"CONTACT": 2234
},
"600453": {
"STATE": "STATE1",
"CITY": "CITY1",
"ID": 2,
"CONTACT": 3456
}
}
Here is a static method that does the full conversion to your desired format:
static JSONObject convert(JSONObject initial) {
// STATE -> CITY -> Address[]
Map<String, Map<String, List<Map<String, Object>>>> stateToCityToAddresses = new HashMap<>();
// Get list of codes
String[] codes = JSONObject.getNames(initial);
// Loop over codes - "600450", "600451", "600453", ...
for (String code : codes) {
// Get the JSONObject containing state data
JSONObject state = initial.getJSONObject(code);
// Extract information from state JSONObject
String stateName = state.getString("STATE");
String cityName = state.getString("CITY");
long id = state.getLong("ID");
long contact = state.getLong("CONTACT");
// Some Java 8 awesomeness!
List<Map<String, Object>> addresses = stateToCityToAddresses
.computeIfAbsent(stateName, sn -> new HashMap<>()) // This makes sure that there is a Map available to hold cities for a given state
.computeIfAbsent(cityName, cn -> new ArrayList<>()); // This makes sure that there is a List available to hold addresses for a given city
// Save data in a map representing a json object like: {"CONTACT":1234,"CODE":600450,"ID":1}
Map<String, Object> address = new HashMap<>();
address.put("ID", id);
address.put("CONTACT", contact);
address.put("CODE", Long.parseLong(code));
// Add the address under city
addresses.add(address);
}
// Just use the JSONObject.JSONObject(Map<?, ?>) constructor to get the final result
JSONObject result = new JSONObject(stateToCityToAddresses);
// You can sysout the result to see the data
// System.out.println(result);
return result;
}

Allow me to play Devil's Advocate.
I think you may risk deviating away from the relationships that have been described in the .yaml. You should try to avoid embedding any application-specific logic inside of your data models, since your assumptions may land you in trickier places in the future.
Generally speaking, you should respect the initial form of the data and interpret the relationships or associated logic with the flexibility of runtime processing. Otherwise, you end up serializing data structures which do not correlate directly with the source, and your assumptions may land you in a hot spot.
The "real" JSON equivalent, I suspect; would look something like this:
{
"600450": {
"STATE": "STATE1",
"CITY": "CITY1",
"ID": 1,
"CONTACT": 1234
},
"600453": { ...etc }
}
It would still be possible to pair these relationships. If you want to relate all Objects by city, you could first separate them into bins. You could do this by using a Map which relates a String city to a List of JSONObjects:
// This will be a Map of List of JSONObjects separated by the City they belong to.
final Map<String, List<JSONObject>> mCityBins = new ArrayList();
// Iterate the List of JSONObjects.
for(final JSONObject lJSONObject : lSomeListOfJSONObjects) {
// Fetch the appropriate bin for this kind of JSONObject's city.
List<JSONObject> lBin = mCityBins.get(lJSONObject.get("city"));
// Does the right bin not exist yet?
if (lBin == null) {
// Create it!
lBin = new ArrayList();
// Make sure it is in the Map for next time!
mCityBins.add(lJSONObject.get("city"), lBin);
}
// Add the JSONObject to the selected bin.
lBin.add(lJSONObject);
}
Whilst you process the JSONObjects, whenever you come across a city whose key does not exist in the Map, you can allocate a new List<JSONObject>, add the current item to that List and add it into the Map. For the next JSONObject you process, if it belongs to the same city, you'd find the existing List to add it to.
Once they're separated into bins, generating the corresponding JSON will be easy!

As I see your git repo you just try to convert first Json structure to new Structure you want. I must to tell you probably you could create this structure directly from YAML file.
As I don't see your first Json structure so I guess that must be something like this :
{600450:{STATE:STATE1 , CITY:CITY2 , ...} , ...}
If this is true so this way can help you :
public static JSONObject convert(JSONObject first) throws JSONException {
HashMap<String , HashMap<String , JSONArray>> hashMap = new HashMap<>();
Iterator<String> keys = first.keys();
while (keys.hasNext())
{
String key = keys.next();
JSONObject inner = first.getJSONObject(key);
String state = inner.getString("STATE");
HashMap<String , JSONArray> stateMap =
hashMap.computeIfAbsent(state , s -> new HashMap<>());
String city = inner.getString("CITY");
JSONArray array = stateMap.computeIfAbsent(city , s->new JSONArray());
JSONObject o = new JSONObject();
o.put("ID" , inner.getInt("ID"));
//in this section you could create int key by calling Integer.parse(String s);
o.put("CODE" , Integer.valueOf(key));
o.put("CONTACT" , inner.getInt("CONTACT"));
array.put(o);
}
JSONObject newStructureObject = new JSONObject();
for(String stateKey:hashMap.keySet())
{
JSONArray array = new JSONArray();
JSONObject cityObject = new JSONObject();
HashMap<String , JSONArray> cityMap = hashMap.get(stateKey);
for(String cityKey : cityMap.keySet())
{
cityObject.put(cityKey , cityMap.get(cityKey));
}
array.put(cityObject);
newStructureObject.put(stateKey , array);
}
return newStructureObject;
}

How to rename key in JSONObject using java?

I want to rename the keys of a JSON object using Java.
My input JSON is:
{
"serviceCentreLon":73.003742,
"type":"servicecentre",
"serviceCentreLat":19.121737,
"clientId":"NMMC01"
}
I want to change it to:
{
"longitude":73.003742,
"type":"servicecentre",
"latitude":19.121737,
"clientId":"NMMC01"
}
i.e. I want to rename "serviceCentreLon" to "longitude" and "serviceCentreLat" to "latitude". I am using the JSONObject type in my code.

Assuming you're using the json.org library: once you have a JSONObject, why not just do this?
obj.put("longitude", obj.get("serviceCentreLon"));
obj.remove("serviceCentreLon");
obj.put("latitude", obj.get("serviceCentreLat"));
obj.remove("serviceCentreLat");
You could create a rename method that does this (then call it twice), but that's probably overkill if these are the only fields you're renaming.

String data= json.toString();
data=data.replace("serviceCentreLon","longitude");
data=data.replace("serviceCentreLat","latitude");
convert back to json object

I'm not sure whether I get your question right, but shouldn't the following work?
You could use a regular expression to replace the keys, for example:
String str = myJsonObject.toString();
str = str.replace(/"serviceCentreLon":/g, '"longitude":');
str = str.replace(/"serviceCentreLat":/g, '"latitude":');
It's not as "clean", but it might get the job done fast.

To build on Danyal Sandeelo's approach, instead of:
data=data.replace("serviceCentreLon","longitude");
use
data=data.replace("\"serviceCentreLon\":","\"longitude\":");
This method explicitly matches the json key syntax, and avoids obscure errors where the key value is present as valid data elsewhere in the json string.

The best way to approach the problem is to parse the JSON data and then replace the key. A number of parsers are available - google gson, Jackson serializer de-serializers, org.json.me are a few such java libraries to handle JSON data.
http://www.mkyong.com/java/how-do-convert-java-object-to-from-json-format-gson-api/
is a good way to deal with it if you have a pretty generic and relatively huge JSON data. Of course, you have to spend time in learning the library and how to use it well.
http://www.baeldung.com/jackson-map is another such parser.
https://stleary.github.io/JSON-java/ is the simplest one especially if you don't want any serious serialization or deserialization

Have an object that maps to this object data structure.
Use GSON parser or Jackson parser to convert this json into POJO.
Then map this object to another Java Object with required configuration
Convert that POJO back to json using the same GSON parsers.
refer this for further reference
http://www.mkyong.com/java/how-do-convert-java-object-to-from-json-format-gson-api/

I faced this problem during my work so I've made a useful Utils class and I want to share it with you.
package net.so.json;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.json.JSONObject;
public class Utils {
/**
* replace json object keys with the new one
* #param jsonString represents json object as string
* #param oldJsonKeyNewJsonKeyMap map its key old json key & its value new key name if nested json key you have traverse
* through it using . example (root.coutry, count) "root.country" means country is a key inside root object
* and "count" is the new name for country key
* Also, if the value for the map key is null, this key will be removed from json
*/
public static void replaceJsonKeys(final JSONObject jsonObject, final Map<String, String> oldJsonKeyNewJsonKeyMap) {
if (null == jsonObject || null == oldJsonKeyNewJsonKeyMap) {
return;
}
// sort the old json keys descending because we want to replace the name of the inner most key first, then
// the outer one
final List<String> oldJsonKeys = oldJsonKeyNewJsonKeyMap.keySet().stream().sorted((k2, k1) -> k1.compareTo(k2)).collect(Collectors.toList());
oldJsonKeys.forEach(k -> {
// split old key, remember old key is something like than root.country
final String[] oldJsonKeyArr = k.split("\\.");
final int N = oldJsonKeyArr.length;
// get the object hold that old key
JSONObject tempJsonObject = jsonObject;
for (int i = 0; i < N - 1; i++)
tempJsonObject = tempJsonObject.getJSONObject(oldJsonKeyArr[i]);
final String newJsonKey = oldJsonKeyNewJsonKeyMap.get(k);
// if value of the map for a give old json key is null, we just remove that key from json object
if (!"null".equalsIgnoreCase(newJsonKey))
tempJsonObject.put(newJsonKey, tempJsonObject.get(oldJsonKeyArr[N - 1]));
// remove the old json key
tempJsonObject.remove(oldJsonKeyArr[N - 1]);
});
}
}
you can test this class by running App
package net.so.json;
import java.util.HashMap;
import java.util.Map;
import org.json.JSONObject;
public class App {
public static void main(String[] args) {
final String jsonString = "{\"root\":{\"country\": \"test-country\", \"city\": \"test-city\"}}";
final JSONObject jsonObject = new JSONObject(jsonString);
System.out.println("json before replacement: " + jsonObject);
/* will get >>
{
"root": {
"country": "test-country",
"city": "test-city"
}
}
*/
// construct map of key replacements
final Map<String, String> map = new HashMap<>();
map.put("root", "root2");
map.put("root.country", "count");
map.put("root.city", "null"); // null as a value means we want to remove this key
Utils.replaceJsonKeys(jsonObject, map);
System.out.println("json after replacement: " + jsonObject);
/* will get >>
{
"root2": {
"count": "test-country"
}
}
*/
}
}

I ran into a scenario where I wanted to remove a hyphen from an unknown number of keys in a nested object.
So this:
{
"-frame": {
"-shape": {
"-rectangle": {
"-version": "1"
}
},
"-path": {
"-geometry": {
"-start": {
"-x": "26.883513064453602",
"-y": "31.986310940359715"
}
},
"-id": 1,
"-type": "dribble",
"-name": "MultiSegmentStencil",
"-arrowhead": "0"
}
}
}
Would be this:
{
"frame": {
"shape": {
"rectangle": {
"version": "1"
}
},
"path": {
"geometry": {
"start": {
"x": "26.883513064453602",
"y": "31.986310940359715"
}
},
"id": 1,
"type": "dribble",
"name": "MultiSegmentStencil",
"arrowhead": "0"
}
}
}
A recursive method(kotlin).. with a list did the trick via Jackson
fun normalizeKeys(tree: JsonNode, fieldsToBeRemoved: MutableList<String>) {
val node = tree as ContainerNode<*>
val firstClassFields = node.fields()
while(firstClassFields.hasNext()) {
val field = firstClassFields.next()
if(field.key.substring(0,1) == "-") {
fieldsToBeRemoved.add(field.key)
}
if(field.value.isContainerNode) {
normalizeKeys(field.value, fieldsToBeRemoved)
}
}
fieldsToBeRemoved.forEach {
val fieldByKey: MutableMap.MutableEntry<String, JsonNode>? = getFieldByKey(tree, it)
if(fieldByKey != null) {
(tree as ObjectNode)[fieldByKey!!.key.replaceFirst("-","")] = fieldByKey.value
(tree as ObjectNode).remove(fieldByKey!!.key)
}
}
}