We Keep Coding

Write custom document to Cosmos DB with Java API

I have a Cosmos DB and want to write different kind of documents to it. The structure of the documents is dynamic and can change.
I tried the following. Let's say I have the following class:
class CosmosDbItem implements Serializable {
private final String _id;
private final String _payload;
public CosmosDbItem(String id, String payload) {
_id = id;
_payload = payload;
}
public String getId() {
return _id;
}
public String getPayload() {
return _payload;
}
}
I can create then the document with some JSON as follows:
CosmosContainer _container = ...
CosmosDbItem dataToWrite = new CosmosDbItem("what-ever-id-18357", "{\"name\":\"Jane Doe\", \"age\":42}")
item = _cosmosContainer.createItem(dataToWrite, partitionKey, cosmosItemRequestOptions);
This results in a document like that:
{
"id": "what-ever-id-18357",
"payload": "{\"name\":\"Jane Doe\", \"age\":42}",
"_rid": "aaaaaaDaaAAAAAAAAAA==",
"_self": "dbs/aaaaAA==/colls/aaaaAaaaDI=/docs/aaaaapaaaaaAAAAAAAAAA==/",
"_etag": "\"6e00c443-0000-0700-0000-5f8499a70000\"",
"_attachments": "attachments/",
"_ts": 1602525607
}
Is there a way in generating the payload as real JSON object in that document? What do I need to change in my CosmosDbItem class? Like this:
{
"id": "what-ever-id-18357",
"payload": {
"name":"Jane Doe",
"age":42
},
"_rid": "aaaaaaDaaAAAAAAAAAA==",
"_self": "dbs/aaaaAA==/colls/aaaaAaaaDI=/docs/aaaaapaaaaaAAAAAAAAAA==/",
"_etag": "\"6e00c443-0000-0700-0000-5f8499a70000\"",
"_attachments": "attachments/",
"_ts": 1602525607
}

Here is my solution that I ended up. Actually it is pretty simple once I got behind it. Instead of using CosmosDbItem I use a simple HashMap<String, Object>.
public void writeData() {
...
Map<String, Object> stringObjectMap = buildDocumentMap("the-id-", "{\"key\":\"vale\"}");
_cosmosContainer.createItem(stringObjectMap, partitionKey, cosmosItemRequestOptions);
...
}
public Map<String, Object> buildDocumentMap(String id, String jsonToUse) {
JSONObject jsonObject = new JSONObject(jsonToUse);
jsonObject.put("id", id);
return jsonObject.toMap();
}
This can produce the following document:
{
"key": "value",
"id": "the-id-",
"_rid": "eaaaaaaaaaaaAAAAAAAAAA==",
"_self": "dbs/eaaaAA==/colls/eaaaaaaaaaM=/docs/eaaaaaaaaaaaaaAAAAAAAA==/",
"_etag": "\"3b0063ea-0000-0700-0000-5f804b3d0000\"",
"_attachments": "attachments/",
"_ts": 1602243389
}
One remark: it is important to set the id key in the HashMap. Otherwise one will get the error
"The input content is invalid because the required properties - 'id; ' - are missing"

Reformat JSON arrays/object in java

I am trying to reformat this json file to a different format. I never used jackson or gson before. I get the idea of it but I don't know how to implement it.
So what I have is a json file: file.json that contains:
{
"Fruits": [
{
"name": "avocado",
"organic": true
},
{
"name": "mango",
"organic": true
}
]
}
What I want is to get in this format:
{
"List Fruits":{
"Fruits": [
{
"name": "avocado",
"organic": true
},
{
"name": "mango",
"organic": true
}
]
}
}
Somehow add the "List Fruits" in the json file.
I am trying to use the jackson api but I don't know how.

Assign the JSON to String variable, for example assign the above JSON to variable called json:
String json = "..." // here put your JSON text;
Prepare classes for your objects:
class Fruit {
private String name;
private boolean organic;
}
class Fruits {
private List<Fruit> fruits;
}
then use Gson to convert JSON to your objects:
Gson gson = new Gson();
Fruits fruits = gson.fromJson(json, Fruits.class);
Next prepare wrapper class ListOfFruits for your fruits object:
class ListOfFruits {
private Fruits listOfFruits;
public ListOfFruits(Fruits fruits) {
listOfFruits = fruits;
}
}
Next pack your fruits object into another one:
ListOfFruits lof = new ListOfFruits(fruits);
And finally generate back the output JSON:
String newJson = gson.toJson(lof);

You do not need to create POJO model for reading and updating JSON. Using Jackson, you can read whole JSON payload to JsonNode, create a Map with required key and serialising to JSON back. See below example:
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import java.io.File;
import java.util.Collections;
import java.util.Map;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./resource/test.json").getAbsoluteFile();
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
JsonNode root = mapper.readTree(jsonFile);
Map<String, JsonNode> output = Collections.singletonMap("List Fruits", root);
System.out.println(mapper.writeValueAsString(output));
}
}
Above code prints:
{
"List Fruits" : {
"Fruits" : [ {
"name" : "avocado",
"organic" : true
}, {
"name" : "mango",
"organic" : true
} ]
}
}

I would highly recommend going through the documentations of Jackson or Gson libraries as you mentioned you are new.
I have created a sample git repo for this item. This sample uses Jackson API.
Visit https://github.com/rajramo61/jsonwrapper
final InputStream fileData = ClassLoader.getSystemResourceAsStream("file.json");
ObjectMapper mapper = new ObjectMapper();
InitialJson initialJson = mapper.readValue(fileData, InitialJson.class);
System.out.println(mapper.writeValueAsString(initialJson));
final FinalJson finalJson = new FinalJson();
finalJson.setListOfFruits(initialJson);
System.out.println(mapper.writeValueAsString(finalJson));
This is the Fruit class.
public class Fruit {
private String name;
private boolean organic;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public boolean getOrganic() {
return organic;
}
public void setOrganic(boolean organic) {
this.organic = organic;
}
#Override
public String toString() {
return "Fruit{" +
"name='" + name + '\'' +
", organic=" + organic +
'}';
}
}
Here is FinalJson class detail. This is the class will wrap the initial json loaded from jsn file.
public class FinalJson {
private InitialJson listOfFruits;
#JsonProperty("List Fruits")
public InitialJson getListOfFruits() {
return listOfFruits;
}
public void setListOfFruits(InitialJson listOfFruits) {
this.listOfFruits = listOfFruits;
}
}
Here is InitialJson class detail. This is the class pulls data from json file.
public class InitialJson {
private List<Fruit> fruits;
#JsonProperty("Fruits")
public List<Fruit> getFruits() {
return fruits;
}
public void setFruits(List<Fruit> fruits) {
this.fruits = fruits;
}
#Override
public String toString() {
return "InitialJson{" +
"fruits=" + fruits +
'}';
}
}
You can fork the repo and close this in local and it should work fine.

Java Jackson, Marshalling class with Map<String, Object> without access to the class code base

I'm trying to marshall (serialize) class using Jackson mapper.
Class has Map property. Property has to have some kind of serialization... everything i got was a serialized byte stream or badly serialized map.toString().
I've tried using mixins or setup Jackson mapper... without any help.
com.fasterxml.jackson 2.8.11
com.rabbitmq.client 5.4.3
My Code:
private RawMessage parseMetadata(RawMessage rawMessage, Envelope envelope, AMQP.BasicProperties properties) throws JsonProcessingException {
ObjectMapper mapper = new ObjectMapper();
mapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
JsonNode rootNode = mapper.createObjectNode();
JsonNode message = mapper.valueToTree(new String(rawMessage.getPayload()));
((ObjectNode) rootNode).set("message", message);
JsonNode envelopeNode = mapper.valueToTree(envelope);
((ObjectNode) rootNode).set("envelope", envelopeNode);
JsonNode propertiesNode = mapper.valueToTree(properties);
((ObjectNode) rootNode).set("properties", propertiesNode);
return new RawMessage(mapper.writerWithDefaultPrettyPrinter().writeValueAsBytes(rootNode));
}
Result:
{
"properties": {
"bodySize": 0,
"headers": {
"connection_name": {
"bytes": "MTcyLjE5LjAuMTo0NTgzNiAtPiAxNzIuMTkuMC40OjU2NzI=",
"stream": {
"in": { "buf": "MTcyLjE5LjAuMTo0NTgzNiAtPiAxNzIuMTkuMC40OjU2NzI=", "pos": 0, "mark": 0, "count": 35 },
"bytearr": "A...",
"chararr": "\u0000...",
"readBuffer": "AAAAAAAAAAA="
}
},
"timestamp_in_ms": 1565957758662,
"protocol": {
"bytes": "ezAsOSwxfQ==",
"stream": {
"in": { "buf": "ezAsOSwxfQ==", "pos": 0, "mark": 0, "count": 7 },
"bytearr": "AAA...",
"chararr": "\u0000\u0000\...",
"readBuffer": "AAAAAAAAAAA="
}
},...
},
"ssl": false
},
"deliveryMode": 2,
"timestamp": 1565957758000,
"classId": 60,
"className": "basic"
}
}
class BasicProperties extends com.rabbitmq.client.impl.AMQBasicProperties {
private String contentType;
private String contentEncoding;
private Map<String,Object> headers; <---
private Integer deliveryMode;
private Integer priority;
private String correlationId;
private String replyTo;
private String expiration;
private String messageId;
private Date timestamp;
private String type;
private String userId;
private String appId;
private String clusterId;
...}
Reproduction:
import com.rabbitmq.client.AMQP;
Map<String, Object> map = new HashMap<>();
byte[] test = "test".getBytes();
map.put("test", test);
AMQP.BasicProperties prop = new AMQP.BasicProperties(null, null, map,null,null,null,null,null,null,null,null,null,null,null);
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
JsonNode root = objectMapper.createObjectNode();
JsonNode propertiesN = objectMapper.valueToTree(prop);
((ObjectNode) root).set("properties", propertiesN);
LOG.info(root.toString());
Result:
{
"properties": {
"bodySize": 0,
"headers": { "test": "dGVzdA==" },
"classId": 60,
"className": "basic"
}
}
three dots means there's more of it. It's too long. Too many byte streams.
I expect the output of map would be -> key: String
I want to output look like his -> properties.headers.test: "test"
Without any annotation of AMQP.BasicProperties class.
I think that Jacksons Mixins should be the way to do it.

Answer to this question:
- make you own Jackson Mixin that annotates your class with custom JsonSerialize Class
- create you own Serialize class that extends JsonSerializer class
- register Mixin to mapper with addMixIn(DataClass.class, Mixin.class) method
Example:
Create Mixin
abstract class MixIn {
#JsonSerialize(using = MyPairSerializer.class)
abstract Map<String, Object> get_map();
}
Create MapSerializer
class MyPairSerializer extends JsonSerializer<Map<String, Object>> {
public void serialize(Map<String, Object> map, JsonGenerator gen, SerializerProvider serializerProvider) throws IOException {
for (Map.Entry<String, Object> entry : map.entrySet()) {
gen.writeStringField(entry.getKey(), entry.getValue().toString());
}
}
}
Register Serializer and map object
// API call
Data data = getData();
ObjectMapper objectMapper = new ObjectMapper();
JsonNode node = objectMapper.valueToTree(data);
System.out.println("node = " + node);
before:
{"data":"test","map":{"test":"dGVzdA=="}}
after:
{"data":"test","map":{"test":"test"}}
TLDR Answer is Mixins with Custom Serializers...

jackson deserialize object with list of spring's interface

I need to save and load objects from redis.
The object contains list of GrantedAuthority (among other things) which is an interface:
public class UserAccountAuthentication implements Authentication {
private List<GrantedAuthority> authorities;
private boolean authenticated = true;
...
}
Jackson successfully serializes the object but fails to deserialize it with the following exception:
abstract types can only be instantiated with additional type information
I know that I can specify the type by adding:
#JsonTypeInfo(
But I can't do it in this case because the GrantedAuthority is an interface of Spring and I cannot change it.
the serialized json is:
{
"authorities": [
{
"authority": "ROLE_NORMAL_USER"
}
],
"authenticated": true,
"securityToken": {
"expiration": 1458635906853,
"token": "sxKi3Pddfewl2rgpatVE7KiSR5qGmhpGl0spiHUTLAAW8zuoLFE0VLFYcfk72VLnli66fcVmb8aK9qFavyix3bOwgp1DRGtGacPI",
"roles": [
"ROLE_NORMAL_USER"
],
"expired": false,
"expirationDateFormatted": "2016-03-22 08:38:26.853 UTC"
},
"name": "admin",
"expired": false
}
the abstract GrantedAuthority is only filled with SimpleGrantedAuthority.
so i tried:
objectMapper.registerSubtypes(SimpleGrantedAuthority.class);
and still no luck.

I think you need to add a custom deserializer
public class UserAccountAuthenticationSerializer extends JsonDeserializer<UserAccountAuthentication> {
#Override
public UserAccountAuthentication deserialize(JsonParser jsonParser, DeserializationContext deserializationContext)
throws IOException {
UserAccountAuthentication userAccountAuthentication = new UserAccountAuthentication();
ObjectCodec oc = jsonParser.getCodec();
JsonNode node = oc.readTree(jsonParser);
userAccountAuthentication.setAuthenticated(node.get("authenticated").booleanValue());
Iterator<JsonNode> elements = node.get("authorities").elements();
while (elements.hasNext()) {
JsonNode next = elements.next();
JsonNode authority = next.get("authority");
userAccountAuthentication.getAuthorities().add(new SimpleGrantedAuthority(authority.asText()));
}
return userAccountAuthentication;
}
}
This is my json
{"authenticated":true,"authorities":[{"authority":"role1"},{"authority":"role2"}],"details":null,"principal":null,"credentials":null,"name":null}
Then at the top of your POJO
#JsonDeserialize(using = UserAccountAuthenticationSerializer.class)
public class UserAccountAuthentication implements Authentication {
Here's the test
#Test
public void test1() throws IOException {
UserAccountAuthentication userAccountAuthentication = new UserAccountAuthentication();
userAccountAuthentication.setAuthenticated(true);
userAccountAuthentication.getAuthorities().add(new SimpleGrantedAuthority("role1"));
userAccountAuthentication.getAuthorities().add(new SimpleGrantedAuthority("role2"));
String json1 = new ObjectMapper().writeValueAsString(userAccountAuthentication);
UserAccountAuthentication readValue = new ObjectMapper().readValue(json1, UserAccountAuthentication.class);
String json2 = new ObjectMapper().writeValueAsString(readValue);
assertEquals(json1, json2);
}

How do I make nested JSON Objects using Gson?

I have written a program that does some probability calculations and gives its results in the form of arrays. I want to convert these results to JSON format, but I am having issues.
I want my json object to look like this:
{
"totalSuggestions": 6,
"routes": {
"rank_2": {
"Source": "ABC",
"Weight": "0.719010390625",
"Destination": "XYZ"
},
"rank_1": {
"Source": "XYZ",
"Weight": "0.7411458281249999",
"Destination": "ABC"
},
"rank_0": {
"Source": "LMN",
"Weight": "0.994583325",
"Destination": "PQR"
}
}
}
What I understood is that I need to have an object class with the structure of my objects. For now I am experimenting with the rank object only but failing to form the required JSON.
My code for the object structure:
public class Object {
int rank_;
public class Inner{
String Source;
String Destination;
String Weightage;
}
}
I can pass either an instance of Object or an instance of Inner to toJson() method so I either get {"rank_":1} or {"Source":"ABC","Destination":"XYZ","Weightage":"123"}.
I cant seem to put each of the inner object to the corresponding rank object.
I did it with relative ease with org.json but that library has some issues with Android studio so I had to switch to Gson. What I did earlier (which worked as well) was:
public JSONObject convertToJson(int mkr, String[][] result){
JSONObject outerObj = new JSONObject();
JSONObject innerObj = new JSONObject();
JSONObject[] temp = new JSONObject[mkr];
outerObj.put("totalSuggestions", marker);
outerObj.put("routes",innerObj);
for (int i=0;i<marker;i++){
String[] useless = result[i][0].split("-");
temp[i]= new JSONObject();
temp[i].put("Source",useless[0] );
temp[i].put("Destination", useless[1]);
temp[i].put("Weight", result[i][1]);
innerObj.put("rank_"+i, temp[i]);
}
System.out.println(outerObj.toString());
return outerObj;
}

Well, first: related objects should probably be in a class together. So lets start with a simple class:
public class Results {
int mkr;
String[][] result;
}
Now we want to serialize it. We could construct a different data structure, or we could just write our own custom serializer. We want to have our custom class to allow us to use Gson's type inference for doing so, plus the code is just easier to understand. I will show you how to serialize the data structure, and I'll leave the deserialization as an exercise for you.
We create a TypeAdapter<Results>:
public class ResultsAdapter extends TypeAdapter<Results> {
public Results read(JsonReader reader) throws IOException {
if (reader.peek() == JsonToken.NULL) {
reader.nextNull();
return null;
}
// exercise for you
return results;
}
public void write(JsonWriter writer, Results value) throws IOException {
if (value == null) {
writer.nullValue();
return;
}
writer.beginObject();
writer.name("totalSuggestions").value(value.mkr);
writer.name("routes");
writer.beginObject();
for(int i = 0; i < value.mkr; i++) {
writer.name("rank_"+i);
writer.beginObject();
String[] sourceDestSplit = result[i][0].split("-");
writer.name("Source").value(sourceDestSplit[0]);
writer.name("Destination").value(sourceDestSplit[1]);
writer.name("Weight").value(result[i][1]);
writer.endObject();
}
writer.endObject();
writer.endObject();
}
}
You can then call this method by doing (note: should only create the Gson object once, but I did it this way to keep the code short):
public String convertToJson(Results results) {
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(new ResultsAdapter()):
Gson gson = builder.build();
return gson.toJson(results);
}
This will work you the way you've asked, but I strongly recommend using JSON's array syntax instead (using []). Try this instead:
public void write(JsonWriter writer, Results value) throws IOException {
if (value == null) {
writer.nullValue();
return;
}
writer.beginObject();
writer.name("totalSuggestions").value(value.mkr);
writer.name("routes");
writer.beginArray();
for(int i = 0; i < value.mkr; i++) {
writer.beginObject();
String[] sourceDestSplit = result[i][0].split("-");
writer.name("Source").value(sourceDestSplit[0]);
writer.name("Destination").value(sourceDestSplit[1]);
writer.name("Weight").value(result[i][1]);
writer.endObject();
}
writer.endArray();
writer.endObject();
}
Doing it this will will result in JSON that looks like this, which will be easier to deserialize on the other side and iterate through, because you won't have to dynamically generate maps for the keys.:
{
"totalSuggestions": 6,
"routes": [
{
"Source": "ABC",
"Weight": "0.719010390625",
"Destination": "XYZ"
},
{
"Source": "XYZ",
"Weight": "0.7411458281249999",
"Destination": "ABC"
},
{
"Source": "LMN",
"Weight": "0.994583325",
"Destination": "PQR"
}
]
}

I landed here while searching for a similar solution for the com.google.gson.JsonObject library. Now, I've found it:
JsonObject mainJson = new JsonObject();
JsonObject innerJson = new JsonObject();
innerJson.addProperty("#iot.id", "31");
mainJson.add("Datastream", innerJson); // <-- here the nesting happens
mainJson.addProperty("result", 12.3);
// fetch inner variable like this
System.out.println(mainJson.get("Datastream").getAsJsonObject().get("#iot.id").getAsString());
This works fine for me using the com.google.gson.JsonObject library.

For the record, this is what i did.
import java.util.*;
public class DataObject {
public int Suggestions;
HashMap<String, route> routes = new HashMap<>();
//constructor
public DataObject(int mkr, String[][] routesArr){
Suggestions = mkr;
{
for (int i=0;i<Suggestions;i++){
routes.put("rank_"+(i+1),new route(routesArr[i]));
}
}
}
//class to populate the hashmap
public class route{
public String Origin;
public String Destination;
public String Weight;
public route(String arr[]){
String[] splitter = arr[0].split("-");
this.Origin = splitter[0];
this.Destination = splitter[1];
this.Weight = arr[1];
}
}
}