We Keep Coding

How to insert null values for an Avro map

I have a usecase where I need to have null values allowed for an Avro Map, but it seems like Avro doesn't allow unions for Map values. Basically, I need to implement the functionality of a POJO defined as Map<String,<Optional<String>>>.
How can I achieve this?
The following avro schema throws no type found error:
Error:
org.apache.avro:avro-maven-plugin:1.10.0: schema failed:
No type: {"type":["null","string"]}
{
"namespace": "com.testclass.avro",
"name": "test",
"type": "record",
"fields": [
{
"name": "user",
"type": {
"name": "userdetails",
"type": "record",
"fields": [
{
"name": "isPresent",
"type": "boolean"
},
{
"name": "address",
"type": {
"type": "map",
"name": "address",
"values": {
"type": ["null","string"]
}
}
}
]
}
}
]
}

Specifying the string as a string within the json definition helped solved the problem.
"address":{"test":{"string":"a"}, "test2":{"string":"a"}}

Get available attributes (possibly recursive) from JSON Schema in Java

Let's say I've got the following JSON Schema:
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://example.com/product.schema.json",
"title": "Draft JSON Schema",
"type": "object",
"properties": {
"person": {
"type": "object",
"properties": {
"details": {
"type": "object",
"properties": {
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
},
"groups": {
"type": "array",
"items": { "$ref": "#/$defs/existing_groups"
}
}
}
}
},
"$defs": {
"existing_groups": [ "Teachers", "Students" ]
},
"book": {
"type": "object",
"properties": {
"title": {
"type": "string"
},
"author": {
"type": "string"
}
}
}
}
}
From this schema, I would like to retrieve the available attributes and values at a defined depth:
So what's given is e.g. person.details and I want first_name, last_name, groups to be returned.
If person.details.groups is given, the possible values Student, Teacher should be returned.
If book.title is given, an empty Array or Set should be returned.
Apparently you can get attribute values from JSON objects with JsonPath, but I rather want to get possible attributes (and their possible values, if any are given) from a com.networknt.schema.JsonSchema.
What is the easiest way to do this in Java?

JSON Schema is for validating data. It has nothing to do with data manipulation or extraction. It's not comparable to JSONPath in any way.

Empty object validation in json schema

I'm trying to validate a json using a JSON schema.
In the below json "industry" is of type "object" and it is "not required".
however i need to find out if "industry" is provided in the json or not.
here is my json schema
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"properties": {
"id": {
"enum": ["Russia", "Canada"]
},
"name": {
"type": "string"
},
"industry": {
"$ref": "#/definitions/industry"
}
},
"required": [
"id",
"name"
],
"definitions": {
"industry": {
"type": "object",
"additionalProperties": false,
"properties": {
"type": {
"type": "string"
},
"codes": {
"type": "array",
"items": {
"type": "integer"
}
}
},
"required": [
"codes",
"type"
],
"title": "industry"
}
}
}
here is my json
{
"id": "Russia",
"price": 10.50
}
I want to know if "industry" object is present or not in the given json bcos if the "industry" object is present in the json. I need to do something else. currently if i send the json as above and try this if statement like below code. It is passing as true even though "industry" object is not present in the json. I believe it is considering "industry" object like this {} and not as null.
"if":{
"properties": {"industry" : { "type": "object" }}
},
Any solution to validate if the "industry" object is present in the json object or not will be helpful. Thank you.

A schema containing "properties" will evaluate to true if the property is not present. What you want to put as the conditional of your "if" is "required":
"if": {"required":["industry"]}, "then": { ... }

Can declare your JSON Schema by a reference to type?

I am trying to validate a small bit of JSON like:
{
"success": true,
"message": "all's good!"
}
which works with the schema:
{
"type": "object",
"properties": {
"success": { "type": "boolean" },
"message": { "type": "string" }
}
}
however it fails with the schema
{
"definitions": {
"response": {
"type": "object",
"properties": {
"success": { "type": "boolean" },
"message": { "type": "string" }
}
}
},
"type": { "$ref": "#/definitions/response" }
}
with the error
java.lang.AssertionError: schema resource:/json-schema/sample.schema.json was > invalid: fatal: invalid JSON Schema, cannot continue
Syntax errors:
[ {
"level" : "error",
"message" : "value has incorrect type (found object, expected one of [array, string])",
"domain" : "syntax",
"schema" : {
"loadingURI" : "resource:/json-schema/sample.schema.json#",
"pointer" : ""
},
"keyword" : "type",
"found" : "object",
"expected" : [ "array", "string" ]
} ]
level: "fatal"
are you not allowed to use a reference for a type outside the definitions section? My motivation is that this is a response to a singular case, but there are cases where this structure is nested in others as well.
If it matters I'm using json-schema-validator version 2.2.6.
PS - this is a simplified example, the actual schema is more complicated as to justify why reuse and not copying and pasting is desirable.

You can use "id" and "$ref".
id for identifying, e.g.:
{
"type": "object",
"id": "#response",
"properties": {
"success": { "type": "boolean" },
"message": { "type": "string" }
}
}
}
And then you use $ref, e.g.:
"some": { "$ref": "#response" }
or external ref:
"ext": { "$ref": "http://url.com#response" }
See
http://json-schema.org/latest/json-schema-core.html#anchor27

The value of the type keyword must be a string of the name of one of the JSON primitave types (e.g. "string", "array", etc.), or an array of these strings. That is what the error message is saying. Keyword type must be a string or an array. The closest thing to what I think you are trying to do is this ...
{
"definitions": {
"response": {
"type": "object",
"properties": {
"success": { "type": "boolean" },
"message": { "type": "string" }
}
}
},
"allOf": [{ "$ref": "#/definitions/response" }]
}

You should declare your definition in it's own file, and they have your types refer to that file reference. See How to manage multiple JSON schema files? for details.

Jackson Parser for recursively parsing unknown input structure

I'm trying to parse recursively json input structure in java like the format below and trying to rewrite the same structure in another json.
Meanwhile I need to validate each & every json key/values while parsing.
{"Verbs":[{
"aaaa":"30d", "type":"ed", "rel":1.0, "id":"80", "spoken":"en", "ct":"on", "sps":null
},{
"aaaa":"31", "type":"cc", "rel":3.0, "id":"10", "spoken":"en", "ct":"off", "sps":null
},{
"aaaa":"81", "type":"nn", "rel":3.0, "id":"60", "spoken":"en", "ct":"on", "sps":null
}]}
Please advice how I can use Jackson parser JsonToken enums for reading and writing unknown json content.

You can use JSON Schema to validate your inputs.
Find the documentation for the data format, but from what I can read here, the schema would be something like this:
{
"$schema": "http://json-schema.org/schema#",
"type": "object",
"required": [ "Verbs" ],
"properties": {
"Verbs": { "type": "array", "items": { "$ref": "#/definitions/verb" } }
},
"definitions": {
"verb": {
"type": "object",
"required": [ "aaaa", "type", "rel", "id", "spoken", "ct", "sps" ],
"additionalProperties": false,
"properties": {
"aaaa": { "type": "string" },
"type": { "type": "string" },
"rel": { "type": "number" },
"id": { "type": "string", "pattern": "^[0-9]+$" },
"spoken": { "type": "string" },
"ct": { "enum": [ "on", "off" ] },
"sps": { "enum": [ null ] }
}
}
}
}
As you use Jackson, you can use this library which can validate your data for you.
Transforming your JSON after that can be done by creating a new JsonNode, for instance.