We Keep Coding

How can I read two JSON files with different data at the same time?

My part of the task is:
I need to collect productId-s into list from two JSON files with different data and pass it into the custom method with directory of JSON files.
I have two JSON files with different data as the following:
JSON №1
JSON №2
And my code is in the next format:
public class Main {
public static void main(String[] args) throws IOException, ParseException {
JSONParser parser = new JSONParser();
InputStream isOne = JSONParser.class.getResourceAsStream("/test/java/resources/json/file/product_0001690510.json");
InputStream isTwo = JSONParser.class.getResourceAsStream("/test/java/resources/json/file/product_0001694109.json");
// ... need to use somehow InpuStream for reading two JSON files with different structure inside
JSONArray arr = obj.getJSONArray("products"); // notice that `"products": [...]`
String productId = null;
for (int i = 0; i < arr.length(); i++) {
productId = arr.getJSONObject(i).getString("productID");
}
List<String> productIds = new ArrayList<>(Collections.singleton(productId));
for (var file : obj.keySet()) {
System.out.println(getAllExportsWithProductIds(file, productIds));
}
}
public static List<String> getAllExportsWithProductIds(String directory, List<String> productIds) throws IOException {
var matchingObjects = new ArrayList<String>();
try (var fileStream = Files.walk(Path.of(directory))) {
for (var file : fileStream.toList()) {
var json = Json.readString(Files.readString(file));
var objects = JsonDecoder.array(json);
for (var object : objects) {
var objectProductIDs = JsonDecoder.field(
object, "products",
JsonDecoder.array(JsonDecoder.field("productID", JsonDecoder::string))
);
for (var productId : objectProductIDs) {
if (productIds.contains(productId)) {
matchingObjects.add(Json.writeString(object));
break;
}
}
}
}
}
return matchingObjects;
}
}
Full code
Based on it my question is:
Can I read all provided JSON files at the same time to collect productId-s into list? If yes, how can I do that if I have the different data of two JSON files.
To clarify a bit:
"By different data, I mean, in general, the number of these fields
(for example, productId fields in both of JSON-s: №1 and №2
accordingly), their order. That is, how an array object differs from
another.
If these JSON-s have the same data, it'd be easier to parse, but
in this specific usecase I need to find the way how to handle both of
them in different way as their data is not in the same format of representation."
My detailed explanation how I see it in abstract way:
How I see it abstractly in the lifecycle of developing.
I've already read different articles and checked libraries as:
java-jq
Jackson object mapper tutorial
and so on,
but I'm still struggling with understanding how to apply it correctly, for this reason, I'm looking for a concrete solution.
Thank you in advance for any smart and helpful ideas. If you need some additional details, I'm ready to provide without any problem.
UPD:
There's I have another version of code, which is collecting productID-s, I need just to find the way how to combine with the first version of code and that's it:
public class ProductIdImporter {
public ProductIdImporter() {
// TODO Auto-generated constructor stub
}
public void importJson() {
List<Path> paths = Arrays.asList(Paths.get("C:\\Users\\pc\\IdeaProjects\\jsonapi\\src\\test\\java\\resources\\json\\product_0001690510.json"),
Paths.get("C:\\Users\\pc\\IdeaProjects\\jsonapi\\src\\test\\java\\resources\\json\\product_0001694109.json"));
ObjectMapper jsonMapper = new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
Set<String> productIds = paths.stream().map(path -> {
try {
return jsonMapper.readValue(Files.newInputStream(path), ExportList[].class);
} catch (Exception e) {
throw new RuntimeException(e);
}
}).map(Arrays::asList)
.flatMap(List::stream)
.map(ExportList::productList)
.flatMap(List::stream)
.map(Product::getId)
.collect(Collectors.toSet());
productIds.forEach(System.out::println);
}
public static void main(String[] args) {
ProductIdImporter importer = new ProductIdImporter();
importer.importJson();
}
static class ExportList {
public List<Product> products;
public ExportList() {
}
public List<Product> productList() {
return products;
}
}
static class Product {
public String productID;
public Product() {
}
public String getId() {
return productID;
}
}
}

Taking the JSON №1 you have included in your post basically it is an array of json objects with the same format like below (I have included just the first one element and deleted all the properties you are not interested to parse):
[{
"products": [
{
"colorWayID": "IMP5002620012114",
"productID": "0001755256"
},
{
"colorWayID": "IMP8473190012114",
"productID": "0001690510"
},
{
"colorWayID": "IMP9100570012114",
"productID": "0001700877"
}
],
...other properties excluded from parsing
}]
To iterate over this array you can read your json file into a ArrayNode and iterate over every element:
ArrayNode arrayNode = (ArrayNode) mapper.readTree(json);
for (int i = 0; i < arrayNode.size(); ++i) {/*inspecting products property*/}
The products property is an array of JsonNode containing the productID property you want to extract, you can use the JsonNode#get method to extract both the products and productID properties:
List<String> productIds = new ArrayList<>();
ArrayNode arrayNode = (ArrayNode) mapper.readTree(json);
for (int i = 0; i < arrayNode.size(); ++i) {
//products property is also an array
ArrayNode products =(ArrayNode) arrayNode.get(i).get("products");
for (int j = 0; j < products.size(); ++j) {
//adding productId to your list
productIds.add(products.get(j).get("productID").asText());
}
}
This process can be repeated for other json files with variations depending from the json structure contained in them, but substantially it remains the same.

Simplest way to convert this array in to the specified one using Java

I have this String Json Payload
[
"key1":{
"atr1":"key1",
"atr2":"value1",
"atr3":"value2",
"atr4":"value3,
"atr5":"value4"
},
"key2":{
"atr1":"key2",
"atr2":"value5",
"atr3":"value6",
"atr4":value7,
"atr5":"value8"
}
]
and I want it to be converted in to the following format using Java
[
{
"atr2":"value1",
"atr3":"value2",
"atr4":"value3,
"atr5":"value4"
},
{
"atr2":"value5",
"atr3":"value6",
"atr4": "value7",
"atr5":"value8"
}
]
What would be the simplest way of transforming this ?

You cannot, because the example below is not valid json.
Check it out using this JSON validator.
If you paste this in (I've fixed some basic errors with lack of quotes)
{
{
"atr2":"value1",
"atr3":"value2",
"atr4":"value3",
"atr5":"value4"
},
{
"atr2":"value5",
"atr3":"value6",
"atr4":"value7",
"atr5":"value8"
}
}
You will get these errors ...
It can work if you change the target schema to something like this by using a json-array to contain your data.
[
{
"atr2":"value1",
"atr3":"value2",
"atr4":"value3",
"atr5":"value4"
},
{
"atr2":"value5",
"atr3":"value6",
"atr4":"value7",
"atr5":"value8"
}
]
If this works for you, then this problem can easily be solved by using the ObjectMapper class.
You use it to deserealize the original JSON into a class, which has two fields "key1" and "key2"
Extract the values of these fields and then just store them in an array ...
Serialize the array using the ObjectMapper.
Here a link, which explains how to use the ObjectMapper class to achieve the goals above.
EDIT:
So you'll need the following classes to solve the problem ...
Stores the object data
class MyClass {
String atr2;
String art3;
}
Then you have a container class, which is used to store the initial json.
class MyClassContainer {
MyClass key1;
MyClass key2;
}
Here's how you do the parse from the original json to MyClassContainer
var mapper = new ObjectMapper()
var json = //Get the json String somehow
var myClassContainer = mapper.readValue(json,MyClassContainer.class)
var mc1 = myClassContainer.getKey1();
var mc2 = myClassContainer.getKey2();
var myArray = {key1, key2}
var resultJson = mapper.writeValueAsString(myArray)

Assuming that you will correct the JSON into a valid one (which involves replacing the surrounding square braces with curly ones, and correct enclosure of attribute values within quotes), here's a simpler way which involves only a few lines of core logic.
try{
ObjectMapper mapper = new ObjectMapper();
mapper.configure( DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false );
HashMap<String, Data> map = mapper.readValue( jsonString, new TypeReference<HashMap<String, Data>>(){} );
String json = mapper.writeValueAsString( map.values() );
System.out.println( json );
}
catch( JsonProcessingException e ){
e.printStackTrace();
}
jsonString above is your original JSON corrected and valid JSON input.
Also notice the setting of FAIL_ON_UNKNOWN_PROPERTIES to false to allow atr1 to be ignored while deserializing into Data.
Since we are completely throwing away attr1 and its value, the Data class will represent all fields apart from that.
private static class Data{
private String atr2;
private String atr3;
private String atr4;
private String atr5;
}

How to rename key in JSONObject using java?

I want to rename the keys of a JSON object using Java.
My input JSON is:
{
"serviceCentreLon":73.003742,
"type":"servicecentre",
"serviceCentreLat":19.121737,
"clientId":"NMMC01"
}
I want to change it to:
{
"longitude":73.003742,
"type":"servicecentre",
"latitude":19.121737,
"clientId":"NMMC01"
}
i.e. I want to rename "serviceCentreLon" to "longitude" and "serviceCentreLat" to "latitude". I am using the JSONObject type in my code.

Assuming you're using the json.org library: once you have a JSONObject, why not just do this?
obj.put("longitude", obj.get("serviceCentreLon"));
obj.remove("serviceCentreLon");
obj.put("latitude", obj.get("serviceCentreLat"));
obj.remove("serviceCentreLat");
You could create a rename method that does this (then call it twice), but that's probably overkill if these are the only fields you're renaming.

String data= json.toString();
data=data.replace("serviceCentreLon","longitude");
data=data.replace("serviceCentreLat","latitude");
convert back to json object

I'm not sure whether I get your question right, but shouldn't the following work?
You could use a regular expression to replace the keys, for example:
String str = myJsonObject.toString();
str = str.replace(/"serviceCentreLon":/g, '"longitude":');
str = str.replace(/"serviceCentreLat":/g, '"latitude":');
It's not as "clean", but it might get the job done fast.

To build on Danyal Sandeelo's approach, instead of:
data=data.replace("serviceCentreLon","longitude");
use
data=data.replace("\"serviceCentreLon\":","\"longitude\":");
This method explicitly matches the json key syntax, and avoids obscure errors where the key value is present as valid data elsewhere in the json string.

The best way to approach the problem is to parse the JSON data and then replace the key. A number of parsers are available - google gson, Jackson serializer de-serializers, org.json.me are a few such java libraries to handle JSON data.
http://www.mkyong.com/java/how-do-convert-java-object-to-from-json-format-gson-api/
is a good way to deal with it if you have a pretty generic and relatively huge JSON data. Of course, you have to spend time in learning the library and how to use it well.
http://www.baeldung.com/jackson-map is another such parser.
https://stleary.github.io/JSON-java/ is the simplest one especially if you don't want any serious serialization or deserialization

Have an object that maps to this object data structure.
Use GSON parser or Jackson parser to convert this json into POJO.
Then map this object to another Java Object with required configuration
Convert that POJO back to json using the same GSON parsers.
refer this for further reference
http://www.mkyong.com/java/how-do-convert-java-object-to-from-json-format-gson-api/

I faced this problem during my work so I've made a useful Utils class and I want to share it with you.
package net.so.json;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.json.JSONObject;
public class Utils {
/**
* replace json object keys with the new one
* #param jsonString represents json object as string
* #param oldJsonKeyNewJsonKeyMap map its key old json key & its value new key name if nested json key you have traverse
* through it using . example (root.coutry, count) "root.country" means country is a key inside root object
* and "count" is the new name for country key
* Also, if the value for the map key is null, this key will be removed from json
*/
public static void replaceJsonKeys(final JSONObject jsonObject, final Map<String, String> oldJsonKeyNewJsonKeyMap) {
if (null == jsonObject || null == oldJsonKeyNewJsonKeyMap) {
return;
}
// sort the old json keys descending because we want to replace the name of the inner most key first, then
// the outer one
final List<String> oldJsonKeys = oldJsonKeyNewJsonKeyMap.keySet().stream().sorted((k2, k1) -> k1.compareTo(k2)).collect(Collectors.toList());
oldJsonKeys.forEach(k -> {
// split old key, remember old key is something like than root.country
final String[] oldJsonKeyArr = k.split("\\.");
final int N = oldJsonKeyArr.length;
// get the object hold that old key
JSONObject tempJsonObject = jsonObject;
for (int i = 0; i < N - 1; i++)
tempJsonObject = tempJsonObject.getJSONObject(oldJsonKeyArr[i]);
final String newJsonKey = oldJsonKeyNewJsonKeyMap.get(k);
// if value of the map for a give old json key is null, we just remove that key from json object
if (!"null".equalsIgnoreCase(newJsonKey))
tempJsonObject.put(newJsonKey, tempJsonObject.get(oldJsonKeyArr[N - 1]));
// remove the old json key
tempJsonObject.remove(oldJsonKeyArr[N - 1]);
});
}
}
you can test this class by running App
package net.so.json;
import java.util.HashMap;
import java.util.Map;
import org.json.JSONObject;
public class App {
public static void main(String[] args) {
final String jsonString = "{\"root\":{\"country\": \"test-country\", \"city\": \"test-city\"}}";
final JSONObject jsonObject = new JSONObject(jsonString);
System.out.println("json before replacement: " + jsonObject);
/* will get >>
{
"root": {
"country": "test-country",
"city": "test-city"
}
}
*/
// construct map of key replacements
final Map<String, String> map = new HashMap<>();
map.put("root", "root2");
map.put("root.country", "count");
map.put("root.city", "null"); // null as a value means we want to remove this key
Utils.replaceJsonKeys(jsonObject, map);
System.out.println("json after replacement: " + jsonObject);
/* will get >>
{
"root2": {
"count": "test-country"
}
}
*/
}
}

I ran into a scenario where I wanted to remove a hyphen from an unknown number of keys in a nested object.
So this:
{
"-frame": {
"-shape": {
"-rectangle": {
"-version": "1"
}
},
"-path": {
"-geometry": {
"-start": {
"-x": "26.883513064453602",
"-y": "31.986310940359715"
}
},
"-id": 1,
"-type": "dribble",
"-name": "MultiSegmentStencil",
"-arrowhead": "0"
}
}
}
Would be this:
{
"frame": {
"shape": {
"rectangle": {
"version": "1"
}
},
"path": {
"geometry": {
"start": {
"x": "26.883513064453602",
"y": "31.986310940359715"
}
},
"id": 1,
"type": "dribble",
"name": "MultiSegmentStencil",
"arrowhead": "0"
}
}
}
A recursive method(kotlin).. with a list did the trick via Jackson
fun normalizeKeys(tree: JsonNode, fieldsToBeRemoved: MutableList<String>) {
val node = tree as ContainerNode<*>
val firstClassFields = node.fields()
while(firstClassFields.hasNext()) {
val field = firstClassFields.next()
if(field.key.substring(0,1) == "-") {
fieldsToBeRemoved.add(field.key)
}
if(field.value.isContainerNode) {
normalizeKeys(field.value, fieldsToBeRemoved)
}
}
fieldsToBeRemoved.forEach {
val fieldByKey: MutableMap.MutableEntry<String, JsonNode>? = getFieldByKey(tree, it)
if(fieldByKey != null) {
(tree as ObjectNode)[fieldByKey!!.key.replaceFirst("-","")] = fieldByKey.value
(tree as ObjectNode).remove(fieldByKey!!.key)
}
}
}

Mapping between JSON formats in Java

I'm coming to Java from JavaScript/Ruby. Let's say I've got the following JSON object for an animal:
{
name: {
common: "Tiger",
latin: "Panthera tigris"
}
legs: 4
}
I'm dealing with lots of animal APIs, and I want to normalize them all into my own common format, like:
{
common_name: "Tiger",
latin_name: "Panthera tigris",
limbs: {
legs: 4,
arms: 0
}
}
In, say, JavaScript, this would be straightforward:
normalizeAnimal = function(original){
return {
common_name: original.name.common,
latin_name: original.name.latin,
limbs: {
legs: original.legs || 0,
arms: original.arms || 0
}
}
}
But what about in Java? Using the JSONObject class from org.json, I could go down the road of doing something like this:
public JSONObject normalizeAnimal(JSONObject original) throws JSONException{
JSONObject name = original.getJSONObject("name");
JSONObject limbs = new JSONObject();
JSONObject normalized = new JSONObject();
normalized.put("name_name", name.get("common"));
normalized.put("latin_name", name.get("latin"));
try{
limbs.put("legs", original.get("legs");
}catch(e){
limbs.put("legs", 0);
};
try{
limbs.put("arms", original.get("arms");
}catch(e){
limbs.put("arms", 0);
};
normalized.put("limbs", limbs);
return normalized;
}
This gets worse as the JSON objects I'm dealing with get longer and deeper. In addition to all of this, I'm dealing with many providers for animal objects and I'll eventually be looking to have some succinct configuration format for describing the transformations (like, maybe, "common_name": "name.common", "limbs.legs": "legs").
How would I go about making this suck less in Java?

Use a library like Gson or Jackson and map the JSON to a Java Object.
So you're going to have a bean like
public class JsonAnima {
private JsonName name;
private int legs;
}
public class JsonName {
private String commonName;
private String latinName;
}
which can be easily converted with any library with something like (with Jackson)
ObjectMapper mapper = new ObjectMapper();
JsonAnimal animal = mapper.readValue(jsonString, JsonAnimal.class);
then you can create a "converter" to map the JsonAnimal to you Animal class.
This can be a way of doing it. : )
Some links:
Gson: http://code.google.com/p/google-gson/
Jackson: http://wiki.fasterxml.com/JacksonHome

The pure Java solutions all are challenged to deal with unreliable structure of your source data. If you're running in a JVM, I recommend that you consider using Groovy to do the Parse and the Build of your source JSON. The result ends up looking a lot like the Javascript solution you outlined above:
import groovy.json.JsonBuilder
import groovy.json.JsonSlurper
def originals = [
'{ "name": { "common": "Tiger", "latin": "Panthera tigris" }, "legs": 4 }',
'{ "name": { "common": "Gecko", "latin": "Gek-onero" }, "legs": 4, "arms": 0 }',
'{ "name": { "common": "Liger" }, "legs": 4, "wings": 2 }',
'{ "name": { "common": "Human", "latin": "Homo Sapien" }, "legs": 2, "arms": 2 }'
]
originals.each { orig ->
def slurper = new JsonSlurper()
def parsed = slurper.parseText( orig )
def builder = new JsonBuilder()
// This builder looks a lot like the Javascript solution, no?
builder {
common_name parsed.name.common
latin_name parsed.name.latin
limbs {
legs parsed.legs ?: 0
arms parsed.arms ?: 0
}
}
def normalized = builder.toString()
println "$normalized"
}
Running the script above deals with "jagged" JSON (not all elements have the same attributes) and outputs like...
{"common_name":"Tiger","latin_name":"Panthera tigris","limbs":{"legs":4,"arms":0}}
{"common_name":"Gecko","latin_name":"Gek-onero","limbs":{"legs":4,"arms":0}}
{"common_name":"Liger","latin_name":null,"limbs":{"legs":4,"arms":0}}
{"common_name":"Human","latin_name":"Homo Sapien","limbs":{"legs":2,"arms":2}}

If you'll be using this for many different types of objects, I would suggest to use reflection instead of serializing each object manually. By using reflection you will not need to create methods like normalizeAnimal, you just create one method or one class to do the serialization to json format.
If you search for "mapping json java" you'll find some useful references. Like gson. Here is an example that is on their website:
class BagOfPrimitives {
private int value1 = 1;
private String value2 = "abc";
private transient int value3 = 3;
BagOfPrimitives() {
// no-args constructor
}
}
//(Serialization)
BagOfPrimitives obj = new BagOfPrimitives();
Gson gson = new Gson();
String json = gson.toJson(obj);
///==> json is {"value1":1,"value2":"abc"}
///Note that you can not serialize objects with circular references since that will result in infinite recursion.
//(Deserialization)
BagOfPrimitives obj2 = gson.fromJson(json, BagOfPrimitives.class);
//==> obj2 is just like obj

You can try little jmom java library
JsonValue json = JsonParser.parse(stringvariablewithjsoninside);
Jmom mom = Jmom.instance()
.copy("/name/common", "/common_name", true)
.copy("/name/latin", "/latin_name", true)
.copy("/arms", "/limbs/arms", true)
.copy("/legs", "/limbs/legs", true)
.remove("/name")
;
mom.apply(json);
String str = json.toPrettyString(" ");

How to properly format JSON string in java?

I have a jersey client that is getting JSON from a source that I need to get into properly formatted JSON:
My JSON String looks like the folllowing when grabbing it via http request:
{
"properties": [
{
someproperty: "aproperty",
set of data: {
keyA: "SomeValueA",
keyB: "SomeValueB",
keyC: "SomeValueC"
}
}
]
}
I am having problems because the json has to be properly formatted and keyA, keB, and keyC are not surrounded in quotes. Is there some library that helps add quotes or some best way to go about turning this string to properly formatted json? Or if there is some easy way to convert this to a json object without writing a bunch of classes with variables and lists that match the incoming structure?

you can use json-lib. it's very convenient! you can construct your json string like this:
JSONObject dataSet = new JSONObject();
dataSet.put("keyA", "SomeValueA") ;
dataSet.put("keyB", "SomeValueB") ;
dataSet.put("keyC", "SomeValueC") ;
JSONObject someProperty = new JSONObject();
dataSet.put("someproperty", "aproperty") ;
JSONArray properties = new JSONArray();
properties.add(dataSet);
properties.add(someProperty);
and of course you can get your JSON String simply by calling properties.toString()

I like Flexjson, and using lots of initilizers:
public static void main(String[] args) {
Map<String, Object> object = new HashMap<String, Object>() {
{
put("properties", new Object[] { new HashMap<String, Object>() {
{
put("someproperty", "aproperty");
put("set of dada", new HashMap<String, Object>() {
{
put("keyA", "SomeValueA");
put("keyB", "SomeValueB");
put("keyC", "SomeValueC");
}
});
}
} });
}
};
JSONSerializer json = new JSONSerializer();
json.prettyPrint(true);
System.out.println(json.deepSerialize(object));
}
results in:
{
"properties": [
{
"someproperty": "aproperty",
"set of dada": {
"keyA": "SomeValueA",
"keyB": "SomeValueB",
"keyC": "SomeValueC"
}
}
]
}

Your string isn't JSON. It's something that bears a resemblance to JSON. There is no form of JSON that makes those quotes optional. AFAIK, there is no library that will reads your string and cope with the missing quotes and then spit it back out correctly. You need to find the code that produced this and repair it to produce actual JSON.

You can use argo, a simple JSON parser and generator in Java