Deserialize JSON with unknown Keys - java

I'm trying to deserialize a JSON object (from JIRA REST API createMeta) with unknown keys.
{
"expand": "projects",
"projects": [
{
"self": "http://www.example.com/jira/rest/api/2/project/EX",
"id": "10000",
"key": "EX",
"name": "Example Project",
"avatarUrls": {
"24x24": "http://www.example.com/jira/secure/projectavatar?size=small&pid=10000&avatarId=10011",
"16x16": "http://www.example.com/jira/secure/projectavatar?size=xsmall&pid=10000&avatarId=10011",
"32x32": "http://www.example.com/jira/secure/projectavatar?size=medium&pid=10000&avatarId=10011",
"48x48": "http://www.example.com/jira/secure/projectavatar?pid=10000&avatarId=10011"
},
"issuetypes": [
{
"self": "http://www.example.com/jira/rest/api/2/issueType/1",
"id": "1",
"description": "An error in the code",
"iconUrl": "http://www.example.com/jira/images/icons/issuetypes/bug.png",
"name": "Bug",
"subtask": false,
"fields": {
"issuetype": {
"required": true,
"name": "Issue Type",
"hasDefaultValue": false,
"operations": [
"set"
]
}
}
}
]
}
]
}
My problem is: I don't know the keys into "fields" (in the example below "issuetype", "summary", "description", "customfield_12345").
"fields": {
"issuetype": { ... },
"summary": { ... },
"description": { ... },
"customfield_12345": { ... }
}
It would be awesome if I could deserialize it as an array with the key as "id" in my POJO so the above example will looke like the following:
class IssueType {
...
public List<Field> fields;
...
}
class Field {
public String id; // the key from the JSON object e.g. "issuetype"
public boolean required;
public String name;
...
}
Is there a way I can achieve this and wrap in my model? I hope my problem is somehow understandable :)


If you don't know the keys beforehand, you can't define the appropriate fields. The best you can do is use a Map<String,Object>.
If there are in fact a handful of types, for which you can identify a collection of fields, you could write a custom deserializer to inspect the fields and return an object of the appropriate type.


I know it's old question but I also had problem with this and there are results..
Meybe will help someone in future : )
My Response with unknow keys:
in Model Class
private JsonElement attributes;
"attributes": {
"16": [],
"24": {
"165": "50000 H",
"166": "900 lm",
"167": "b.neutr.",
"168": "SMD 3528",
"169": "G 13",
"170": "10 W",
"171": "230V AC / 50Hz"
}
},
So I also checked if jsonElement is jsonArray its empty.
If is jsonObject we have data.
ProductModel productModel = productModels.get(position);
TreeMap<String, String> attrsHashMap = new TreeMap<>();
if (productModel.getAttributes().isJsonObject())
{
for (Map.Entry<String,JsonElement> entry : productModel.getAttributes().getAsJsonObject().entrySet())
{
Log.e("KEYS", "KEYS: " + entry.getKey() + " is empty: " + entry.getValue().isJsonArray());
if (entry.getValue() != null && entry.getValue().isJsonObject())
{
for (Map.Entry<String, JsonElement> entry1 : entry.getValue().getAsJsonObject().entrySet())
{
Log.e("KEYS", "KEYS INSIDE: " + entry1.getKey() + " VALUE: " + entry1.getValue().getAsString());
// and there is my keys and values.. in your case You can get it in upper for loop..
}
}
}


There is a perfectly adequate JSON library for Java that will convert any valid JSON into Java POJOs. http://www.json.org/java/

Related

Null values being returned in json while using gson in java

I have the following Json file which I am trying to read:
{
"billingInformation": {
"taxes": {
"gst": 2.5,
"hst": 7.8
},
"billTo": {
"name" : "Mike",
"address" : "123, Lake Shore Drive, California",
"phoneNumber" : "601 855 1249"
},
"salesAgent": {
"name" : "Charlotte Thompson",
"agentCode" : 44551
},
"items": {
"item": [
{
"hsnCode": "5112",
"description": "TV Set",
"originCountry": "US",
"quantity": 1,
"unitPrice": 150.00
}
],
"currency": "USD"
}
}
}
I used direct Object Mapping provided by Gson:
result = new String(Files.readAllBytes(Paths.get(path)));
BillingInformation billingInformation = gson.fromJson(result, BillingInformation.class);
But it always resulted in:
class BillingInformation {
taxes: null
billTo: null
salesAgent: null
items: null
}
How can I get the data inside the other objects?
Edit:
Here is the BillingInformation Class:
public class BillingInformation {
#SerializedName("taxes")
private Taxes taxes;
#SerializedName("billTo")
private BillTo billTo;
#SerializedName("salesAgent")
private SalesAgent salesAgent;
#SerializedName("items")
private Items items;
}
and I have the usual getters and setters for the above fields.

You should create a class that looks like this:
class Data {
private BillingInformation billingInformation;
}
And then do the deserialization:
Data fromFile = gson.fromJson(result, Data.class);

How to customize error messages for JSONSchema?

Is there a way to provide a custom error message depending on the given condition?
I'm using https://github.com/networknt/json-schema-validator, version 1.0.43
This is my JSON Schema:
{
"$id": "https://configurations/provider.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Configuration",
"type": "object",
"properties": {
"provider": {
"description": "Name of the provider.",
"enum": [
"Provider #1",
"Provider #2"
]
},
"configuration": {
"$schema": "json-schema/configurations/configuration.json"
}
},
"if": {
"properties": {
"parcsProvider": {
"const": "Provider #1"
}
}
},
"then": {
"required": [
"configuration"
]
},
"else": {
"not": {
"required": [
"configuration"
]
}
}
}
If the value for the provider is "Provider #1" then the configuration object is required, and if it's "Provider #2" and configuration is passed an error will occur. I want to customize that error so that the response is the same as it is now but with a custom message like "Provider 2 can't have a configuration."
Current error message/response:
{
"timestamp": "2020-11-23T12:50:56.20658+01:00",
"message": "invalid.json.input",
"validationErrors": [
{
"field": "$",
"message": "$: should not be valid to the schema \"not\" : {\"required\":[\"configuration\"]}"
}
]
}

I had a similar requirement to implement in one of my projects. For validation, I was using https://github.com/everit-org/json-schema.
Here is what I did
Categorized all kind of errors[there must be some specific keyword] thrown by the validator
Now once you have all the keys, you can easily manipulate the errors and send the custom error/response.
Below are the keys I have collected for different cases, this might help you -
MIN_LENGTH_VIOLATION = "expected minLength"
MAX_LENGTH_VIOLATION = "expected maxLength"
PATTERN_VIOLATION = "does not match pattern"
DATA_TYPE_VIOLATION = "expected type"
DEPENDENCY_VIOLATION = "is required"
FORMAT_VIOLATION_OR_ENUM_VALIDATION_VIOLATION = "is not a valid"
MANDATORY_FIELD_VIOLATION_OR_CONDITIONAL_VIOLATION = "required key"
NUMBER_IS_LESS_THAN_VIOLATION = "is not greater or equal to"
NUMBER_IS_GREATER_THAN_VIOLATION = "is not less or equal"
EXCLUSIVE_NUMBER_IS_GREATER_THAN_VIOLATION = "is not less than"
EXCLUSIVE_NUMBER_IS_LESS_THAN_VIOLATION = "is not greater than"
MULTIPLE_OF_VIOLATION = "is not a multiple"
Sample Code -
private static void validate(JSONObject request) {
try {
// Schema, that might be fetched dynamically from some data source
JSONObject schema = new JSONObject();
Schema loadedSchema = SchemaLoader.load(schema);
loadedSchema.validate(request);
} catch (ValidationException ve) {
List<String> allErrorMessages = ve.getAllMessages();
List<String> mandatoryFields = parseMandatoryField(allErrorMessages);
if (CollectionUtils.isNotEmpty(mandatoryFields)) {
throw new MandFieldMissingExp(mandatoryFields);
} else {
List<String> invalidFields = parseInvalids(allErrorMessages);
throw new InvalidFieldExp(invalidFields);
}
}
}
private static List<String> parseMandatoryField(List<String> validationExceptionMessages) {
Set<String> mandatoryListSet = new HashSet<>();
validationExceptionMessages.forEach(errorMessage -> {
if (StringUtils.containsAny(errorMessage, MANDATORY_FIELD_VIOLATION_OR_CONDITIONAL_VIOLATION, DEPENDENCY_VIOLATION)) {
mandatoryListSet.add(StringUtils.substring(errorMessage, errorMessage.indexOf('[') + 1, errorMessage.indexOf(']')));
}
});
return new ArrayList<>(mandatoryListSet);
}
private static List<String> parseInvalids(List<String> validationExceptionMessages) {
Set<String> invalidParamsSet = new HashSet<>();
validationExceptionMessages.forEach(errorMessage -> {
if (StringUtils.containsAny(errorMessage, MIN_LENGTH_VIOLATION, MAX_LENGTH_VIOLATION, PATTERN_VIOLATION,
FORMAT_VIOLATION_OR_ENUM_VALIDATION_VIOLATION, DATA_TYPE_VIOLATION, NUMBER_IS_LESS_THAN_VIOLATION,
MULTIPLE_OF_VIOLATION, EXCLUSIVE_NUMBER_IS_GREATER_THAN_VIOLATION, NUMBER_IS_GREATER_THAN_VIOLATION
, EXCLUSIVE_NUMBER_IS_LESS_THAN_VIOLATION)) {
invalidParamsSet.add(StringUtils.substring(errorMessage, errorMessage.indexOf('/') + 1, errorMessage.indexOf(':')));
}
});
return new ArrayList<>(invalidParamsSet);
}
Hope it helps

How to add the name before the array in the array object itself in a JSON response?

I am consuming an external web service and receiving a JSON response. In this response, there is an object "entities" containing multiple arrays in it, with a name before each array.
I want to add the name before the array in the array object itself.
For example this is the original response:
{
"entities": {
"entity": [
{
"confidence": 1,
"value": "user",
"type": "value"
},
{
"confidence": 1,
"value": "insurance form",
"type": "value"
}
],
"ui_page_step": [
{
"confidence": 1,
"value": "step 1",
"type": "value"
}
],
"userrole_ano": [
{
"confidence": 0.96535832252792,
"value": "anonymous user"
}
]
}
}
I need to convert it to:
{
"entities": {
"entity": [
{
"name": "entity",
"confidence": 1,
"value": "user",
"type": "value"
},
{
"name": "entity",
"confidence": 1,
"value": "insurance form",
"type": "value"
}
],
"ui_page_step": [
{
"name": "ui_page_step",
"confidence": 1,
"value": "step 1",
"type": "value"
}
],
"userrole_ano": [
{
"name": "userrole_ano",
"confidence": 0.96535832252792,
"value": "anonymous user"
}
]
}
}
How can I convert the original response to the desired one in Java?

Here is a (one of several possible) solutions:
It uses Jackson library to parse the Json into a java Map that is (relatively) easier to navigate and modify than JSONObject.
the method putCollectionNamesInsideEntries() assumes one root "entities" entry that has several collections as values. it iterates over all of them, adding "name" entry with name of collection.
the map is serialized back to Json (and sent to System.out)
import java.io.*;
import java.nio.file.*;
import java.util.*;
import com.fasterxml.jackson.databind.ObjectMapper;
public class JacksonTest
{
public static void main(String[] args) {
try (InputStream is = Files.newInputStream(Paths.get("C:/temp/test.json"))) {
ObjectMapper mapper = new ObjectMapper();
// deserialize json into map
Map<String, Object> map = (Map<String, Object>)mapper.readValue(is, Map.class);
putCollectionNamesInsideEntries(map);
// serialize map into json
mapper.writeValue(System.out, map);
} catch (Exception e) {
e.printStackTrace();
}
}
private static void putCollectionNamesInsideEntries(Map<String, Object> map) {
// get root "entities" entry
Map<String, Object> entitiesMap = (Map<String, Object>)map.get("entities");
for (Map.Entry<String, Object> entitiesEntry : entitiesMap.entrySet()) {
// iterate over collection entries
if (entitiesEntry.getValue() instanceof Collection) {
Collection coll = (Collection)entitiesEntry.getValue();
// iterate over entries in collection
for (Object collEntry : coll) {
if (collEntry instanceof Map) {
// add "name" with ame of collection (key entry under "entries")
((Map<String, Object>)collEntry).put("name", entitiesEntry.getKey());
}
}
}
}
}
}

JSONObject is not getting converted to String properly

I am converting JSONObject to String. I am using below code:
String decresponse=obj.getFileWithUtil("Files/v3user22.txt");
System.out.println("Decrypted string is "+decresponse);
JSONObject js = JSONObject(decresponse);
System.out.println("JSON Object is "+js.toString());
Here, i am getting the value of decresponse from a file since the json is very large. Value of decresponse is:
{
"userid":123456,
"status":"SUCCESS",
"name":{
"firstName":"firstname",
"lastName":"lastname"
},
"dob":"03/02/1993",
"gender":"M",
"kycType":"Manual",
"address":{
"permanentAddress":{
"country":"INDIA",
"street_1":"K-26",
"street_2":"",
"city":"North",
"state":"Delhi",
"postal_code":"110052",
"locality":"abc"
},
"correspondenceAddress":{
"country":"INDIA",
"street_1":"abc",
"street_2":"abc",
"city":"ABC",
"state":"Punjab",
"postal_code":"111000",
"locality":"def"
}
},
"docs":[
{
"nameOnDoc":"name",
"verificationStatus":"FAILED",
"kycNameMatch":"SUCCESS",
"docCode":"aadhar",
"docValue":"1898989",
"submittedAs":"AdditionalDoc"
},
{
"nameOnDoc":"abc",
"verificationStatus":"NOT_ATTEMPTED",
"kycNameMatch":"NOT_ATTEMPTED",
"docCode":"pan",
"docValue":"KSKA1234F",
"submittedAs":"AdditionalDoc",
"expiryDate":"03/02/2018"
},
{
"docCode":"voter",
"docValue":"CIBPS2107P",
"submittedAs":"Poi_Poa"
}
],
"agents":[
{
"bankAgentType":"BF",
"agentBranch":"nodia",
"agentDesignation":"agent manager",
"agentEmpcode":"1010111",
"custId":"119990",
"agentId":"",
"agencyType":"CFA",
"agencyName":"internal"
},
{
"bankAgentType":"BC",
"agentBranch":"nodia",
"agentDesignation":"agent manager",
"agentEmpcode":"",
"custId":"119999",
"agentId":"MORPHO-1782",
"agencyType":"VA",
"agencyName":"morpho"
}
],
"relatives":[
{
"relationShip":"FATHER",
"firstName":"firstname",
"lastName":"lastname"
},
{
"relationShip":"MOTHER",
"firstName":"firstname",
"lastName":"lastname"
}
],
"useKycDetails":"UNDER_REVIEW",
"amlflags":{
"sanction":"N",
"pep":"N"
},
"walletflags":{
"upgraded":"1",
"updated":"1",
"blocked":"0"
},
"suspended":"false",
"aadhar_type1_check":"FAILED",
"aadhar_kyc_name_check":"SUCCESS",
"aadharSubmittedAs":"AdditionalDoc",
"aadharVerified":"false",
"panSubmittedAs":"AdditionalDoc",
"panVerified":"false",
"maritalStatus":"MARRIED",
"profession":"PRIVATE_SECTOR_JOB",
"nationality":"INDIAN",
"kycVerificationDate":"04/01/2017",
"declarationPlace":"Delhi",
"dmsInfos":[
{
"type":"",
"dmsid":""
}
],
"aadharAuthCode":"56bd65db0dbc4b2a848841a44eabb54e",
"agriculturalIncome":"100000",
"nonAgriculturalIncome":"50000",
"seedingStatus":"consent_given"
}
But, on converting the json object to string the value comes as below:
{
"panVerified":"false",
"gender":"M",
"userid":123456,
"panSubmittedAs":"AdditionalDoc",
"aadharAuthCode":"56bd65db0dbc4b2a848841a44eabb54e",
"docs":[
{
"kycNameMatch":"SUCCESS",
"verificationStatus":"FAILED",
"nameOnDoc":"name",
"docCode":"aadhar",
"docValue":"1898989",
"submittedAs":"AdditionalDoc"
},
{
"expiryDate":"03/02/2018",
"kycNameMatch":"NOT_ATTEMPTED",
"verificationStatus":"NOT_ATTEMPTED",
"nameOnDoc":"abc",
"docCode":"pan",
"docValue":"KSKA1234F",
"submittedAs":"AdditionalDoc"
},
{
"docCode":"voter",
"docValue":"CIBPS2107P",
"submittedAs":"Poi_Poa"
}
],
"aadhar_type1_check":"FAILED",
"aadharSubmittedAs":"AdditionalDoc",
"useKycDetails":"UNDER_REVIEW",
"kycVerificationDate":"04/01/2017",
"kycType":"Manual",
"profession":"PRIVATE_SECTOR_JOB",
"address":{
"permanentAddress":{
"country":"INDIA",
"street_1":"K-26",
"city":"North",
"street_2":"",
"locality":"abc",
"state":"Delhi",
"postal_code":"110052"
},
"correspondenceAddress":{
"country":"INDIA",
"street_1":"abc",
"city":"ABC",
"street_2":"abc",
"locality":"def",
"state":"Punjab",
"postal_code":"111000"
}
},
"nonAgriculturalIncome":"50000",
"seedingStatus":"consent_given",
"dmsInfos":[
{
"dmsid":"",
"type":""
}
],
"relatives":[
{
"firstName":"firstname",
"lastName":"lastname",
"relationShip":"FATHER"
},
{
"firstName":"firstname",
"lastName":"lastname",
"relationShip":"MOTHER"
}
],
"suspended":"false",
"agents":[
{
"agentId":"",
"agentEmpcode":"1010111",
"custId":"119990",
"agentBranch":"nodia",
"agentDesignation":"agent manager",
"bankAgentType":"BF",
"agencyType":"CFA",
"agencyName":"internal"
},
{
"agentId":"MORPHO-1782",
"agentEmpcode":"",
"custId":"119999",
"agentBranch":"nodia",
"agentDesignation":"agent manager",
"bankAgentType":"BC",
"agencyType":"VA",
"agencyName":"morpho"
}
],
"amlflags":{
"sanction":"N",
"pep":"N"
},
"aadhar_kyc_name_check":"SUCCESS",
"nationality":"INDIAN",
"dob":"03/02/1993",
"walletflags":{
"upgraded":"1",
"blocked":"0",
"updated":"1"
},
"name":{
"firstName":"firstname",
"lastName":"lastname"
},
"aadharVerified":"false",
"maritalStatus":"MARRIED",
"status":"SUCCESS",
"declarationPlace":"Delhi",
"agriculturalIncome":"100000"
}
Why am I getting different values?

Why am I getting different values
Those values are not that different. They simply have key:value pairs in different order.
JSON structure holds key:value pairs where keys are unique. In most cases order of keys is not important so classes like org.json.JSONObject are storing them in internal HashMap which doesn't preserve insertion order (but allows quick access to values).
When toString() is invoked internally it builds String using that HashMap iterator, so order depends on amount of keys and their hashes, not insertion order.
If you want to preserve order consider using other libraries like gson. Your parsing could look like:
JsonParser jsonParser = new JsonParser();
JsonObject js = jsonParser.parse(decresponse).getAsJsonObject();
and js.toString() would result in
{"userid":123456,"status":"SUCCESS","name":{"firstName":"firstname", ... which seems to be what you ware after.

Spring RestTemplate parse JSON object with variable keyname

I have a REST API call that returns the following JSON object. I need to parse this with Spring's RestTemplate. The problem is that the first key ISBN:0132856204 is variable (the numbers change depending on the book). How would I go about doing this?
{
"ISBN:0132856204": {
"publishers": [
{
"name": "Pearson"
}
],
"pagination": "xxiv, 862p",
"identifiers": {
"isbn_13": [
"978-0-13-285620-1"
],
"openlibrary": [
"OL25617855M"
]
},
"weight": "1340 grams",
"title": "Computer networking",
"url": "https://openlibrary.org/books/OL25617855M/Computer_networking",
"number_of_pages": 862,
"cover": {
"small": "https://covers.openlibrary.org/b/id/7290810-S.jpg",
"large": "https://covers.openlibrary.org/b/id/7290810-L.jpg",
"medium": "https://covers.openlibrary.org/b/id/7290810-M.jpg"
},
"publish_date": "2013",
"key": "/books/OL25617855M",
"authors": [
{
"url": "https://openlibrary.org/authors/OL31244A/James_F._Kurose",
"name": "James F. Kurose"
},
{
"url": "https://openlibrary.org/authors/OL658909A/Keith_W._Ross",
"name": "Keith W. Ross"
}
],
"subtitle": "A Top-Down Approach"
}
}

In here "ISBN:0132856204" is a value and also a key for your business.
To get ISBN first, what about wrapping json content with 1 more closure?
{
"yourAwesomePlaceHolderKey" :
{
"ISBN:0132856204": {
......
}
}
}
First get the ISBN key as a value, then your ISBN value can be used as a key to get related content.
First goal will be extracting -String1,Object1- pair where String1 is "yourAwesomePlaceholderKey" and second goal will be again extracting -String2,Object2- from Object1 where String2 is your ISBN key.

This is the way I solved it, using JsonPath for getting the book out of the JSON object and Jackson for mapping it to a Book object:
RestTemplate restTemplate = new RestTemplate();
String isbn = "0132856204";
String endpoint = "https://openlibrary.org/api/books?jscmd=data&format=json&bibkeys=ISBN:{isbn}";
//Get JSON as String
String jsonString = restTemplate.getForObject(endpoint, String.class, isbn);
//Configure JsonPath to use Jackson for mapping
Configuration.setDefaults(new Configuration.Defaults() {
private final JsonProvider jsonProvider = new JacksonJsonProvider();
private final MappingProvider mappingProvider = new JacksonMappingProvider();
#Override
public JsonProvider jsonProvider() {
return jsonProvider;
}
#Override
public MappingProvider mappingProvider() {
return mappingProvider;
}
#Override
public Set<Option> options() {
return EnumSet.noneOf(Option.class);
}
});
//Parse the JSON as a book
Book book = JsonPath.parse(jsonString).read("$.ISBN:" + isbn, Book.class);

You can use JsonProperty to solve
#JsonProperty("ISBN:0132856204")

Categories