We Keep Coding

JSON contains unicode characters

My Java - Jersey framework REST API makes a call to another service which returns the following JSON response. I have logged the response from the child service in my logs, and I can see that the value of ErrorMessage contains a Unicode value like \u2019 instead of a single quote (').
{
"id": "SAMPLE",
"version": 1,
"status": {
"lastReceivedError": {
"ErrorDateTime": 1576588715,
"ErrorCode": "TEST3200",
"ErrorMessage": "We\u2019re sorry, the content is not available."
}
}
}
I have to map these values into my model and return as a JSON as well. I used GSON to convert the above JSON string into an object. And mapped the values from that object into my response object. My final outgoing JSON response is like below, wherein the single quote is appearing as question mark (?).
{
"MyResponse": {
"success": {
"lastReceivedError": {
"errorDateTime": "2019-12-17T13:18:35Z",
"errorCode": "TEST3200",
"errorMessage": "We?re sorry, the content is not available."
}
}
}
}
I believe there is something around encoding characters, but I am unable to fix the issue.

TL;DR
Seeing is not believing. It depends on the encoding in your environment.
Code snippet
Following code snippet shows to deserialize the JSON string (part of original response).
If the encoding of your environment is UTF-8, then Gson will convert it correctly without specifying encoding.
And if you already knew the original string was encoded with UTF-8, you will get different results if you view it with UTF-8 and ISO-8859-1.
String jsonStr = "{\"ErrorMessage\": \"We\u2019re sorry, the content is not available.\"}";
Gson gson = new Gson();
JsonObject data = gson.fromJson(jsonStr, JsonObject.class);
System.out.println(data.toString());
System.out.println(new String(jsonStr.getBytes("UTF-8"), "UTF-8"));
System.out.println(new String(jsonStr.getBytes("ISO-8859-1"), "UTF-8"));
Console output
{"ErrorMessage":"We’re sorry, the content is not available."}
{"ErrorMessage": "We’re sorry, the content is not available."}
{"ErrorMessage": "We?re sorry, the content is not available."}

Java: Update value of nested array node in JSON file

Because of the project requirement, I have to use com.fasterxml.jackson.databind library to parse JSON data cannot use other JSON libraries available.
I am new to JSON parsing, so not sure if there are better options here?
I would like to know how can I update a string value in an Array node in the JSON file.
Following is a sample JSON. Please note this is not the entire file content, it's a simplified version.
{
"call": "SimpleAnswer",
"environment": "prod",
"question": {
"assertions": [
{
"assertionType": "regex",
"expectedString": "(.*)world cup(.*)"
}
],
"questionVariations": [
{
"questionList": [
"when is the next world cup"
]
}
]
}
}
Following is the code to read JSON into java object.
byte[] jsonData = Files.readAllBytes(Paths.get(PATH_TO_JSON));
JsonNode jsonNodeFromFile = mapper.readValue(jsonData, JsonNode.class);
To update a root level node value e.g. environment in the JSON file , I found following approach on some SO threads.
ObjectNode objectNode = (ObjectNode)jsonNodeFromFile;
objectNode.remove("environment");
objectNode.put("environment", "test");
jsonNodeFromFile = (JsonNode)objectNode;
FileWriter file = new FileWriter(PATH_TO_JSON);
file.write(jsonNodeFromFile.toString());
file.flush();
file.close();
QUESTION 1: Is this the only way to update a value in JSON file and is it the best way possible? I'm concerned on double casting and file I/O here.
QUESTION 2: I could not find a way to update the value for a nested Array node e.g. questionList. Update the question from when is the next world cup to when is the next soccer world cup

You can use ObjectMapper to parse that JSON, it is very easy to parse and update JSON using pojo class.
use link to convert your json to java class, just paste your json here n download class structure.
You can access or update nested json field by using . (dot) operator
ObjectMapper mapper = new ObjectMapper();
String jsonString="{\"call\":\"SimpleAnswer\",\"environment\":\"prod\",\"question\":{\"assertions\":[{\"assertionType\":\"regex\",\"expectedString\":\"(.*)world cup(.*)\"}],\"questionVariations\":[{\"questionList\":[\"when is the next world cup\"]}]}}";
TestClass sc=mapper.readValue(jsonString,TestClass.class);
// to update environment
sc.setEnvironment("new Environment");
System.out.println(sc);
//to update assertionType
Question que=sc.getQuestion();
List assertions=que.getAssertions();
for (int i = 0; i < assertions.size(); i++) {
Assertion ass= (Assertion) assertions.get(i);
ass.setAssertionType("New Type");
}

Convert AWS APIGateway's Model to JSON in Java

I am using Amazon's APIGateway service client side. When you make a request the data returned is stored in a Model data type that the schema is set up beforehand. the calls look like this:
MyModel myModel = client.settingsPost();
String volume = myModel.getVolume();
the schema for this simple object would look like this:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "MyModel",
"type": "object",
"properties" : {
"volume" : { "type" : "string" }
}
}
I would like to convert the Model returned directly to JSON instead of having to go manually reconstruct a new JSONObject from each value of this Model. The Models seem to be very simple and I cant even iterate through them. But I wonder if there is a way to convert them using the GSON library somehow?
EDIT: I am using the APIGateway SDK generated to Java.

Using Jackson:
ObjectMapper mapper = new ObjectMapper();
String jsonInString = mapper.writeValueAsString(myModel);
Using Gson:
Gson gson = new Gson();
String json = gson.toJson(myModel);

creating a web API for JSON GET Request, how can i create JSON Objects/array for this Pattern

suggest me how to work on this, i am having all the data in linkedlinkHashmap like key pair values. i want the output in this format i am trying this but couldn't get exact format
{
RequestorId:123
UserId:111
FirstName:john
LastName:peter
Phone_Number:xxx
Email_Address:#com
Address:yyy
Picture:eeee
Work_Location:rrrr
CurrentRole:bca
LanguageSkills:english
Groups: [
{
GroupID: 1
GroupName:1
ContentGroup:f&r
Owner:[
{
UserId:111
FirstName:eee
LastName:rrr
}
]
}
{
GroupID: 2
GroupName:2
ContentGroup:bca
Owner:[
{
UserId:121
FirstName:www
LastName:qqq
},
{
UserId:123
FirstName:ttt
LastName:uuu
}
]
}
}

You can create a Map <String,Object> valueMap and should have the following classes
public Owner{
int UserId;
String FirstName;
String LastName;
}
public Groups{
List<Owner> owner;
int GroupId;
int GroupName;
String ContentGroup;
}
and assign values to HashMap
valueMap.put("PhoneNumber","xxx");
valueMap.put("Email_Address","#com");
.
.
.
valueMap.put("Groups",<Group>Object);
Populate these classes with value and use Google GSON library to convert the hashmap to JSON using
Gson gson=new Gson();
String json=gson.toJson(valueMap);

You could look for a java json library to convert your java data object to json such as gson library eg http://www.mkyong.com/java/how-do-convert-java-object-to-from-json-format-gson-api/
Your example here to converting list of objects to json is similar to the solution offered in this stackoverflow post
Parse List of JSON objects using GSON
Where mention importantly match java model to json model via TypeAdapter

In XStream is there a better way to marshall/unmarshall List<Object>'s in JSON and Java

I'm using XStream and JETTISON's Stax JSON serializer to send/receive messages to/from JSON javascripts clients and Java web applications.
I want to be able to create a list of objects to send to the server and be properly marshalled into Java but the format that XStream and JSON expect it in is very non-intuitive and requires our javascript libraries to jump through hoops.
[EDIT Update issues using GSON library]
I attempted to use the GSON library but it cannot deserialize concrete objects when I only have it expect generic super classes (XStream and Jettison handles this because type information is baked into the serialization).
GSON FAQ states Collection Limitation:
Collections Limitations
Can serialize collection of arbitrary objects but can not deserialize from it
Because there is no way for the user to indicate the type of the resulting object
While deserializing, Collection must be of a specific generic type
Maybe I'm using bad java practices but how would I go about building a JSON to Java messaging framework that sent/received various concrete Message objects in JSON format?
For example this fails:
public static void main(String[] args) {
Gson gson = new Gson();
MockMessage mock1 = new MockMessage();
MockMessage mock2 = new MockMessage();
MockMessageOther mock3 = new MockMessageOther();
List<MockMessage> messages = new ArrayList<MockMessage>();
messages.add(mock1);
messages.add(mock2);
messages.add(mock3);
String jsonString = gson.toJson(messages);
//JSON list format is non-intuitive single element array with class name fields
System.out.println(jsonString);
List gsonJSONUnmarshalledMessages = (List)gson.fromJson(jsonString, List.class);
//This will print 3 messages unmarshalled
System.out.println("XStream format JSON Number of messages unmarshalled: " + gsonJSONUnmarshalledMessages.size());
}
[{"val":1},{"val":1},{"otherVal":1,"val":1}]
Exception in thread "main" com.google.gson.JsonParseException: The JsonDeserializer com.google.gson.DefaultTypeAdapters$CollectionTypeAdapter#638bd7f1 failed to deserialized json object [{"val":1},{"val":1},{"otherVal":1,"val":1}] given the type interface java.util.List
Here's an example, I want to send a list of 3 Message objects, 2 are of the same type and the 3rd is a different type.
import java.util.ArrayList;
import java.util.List;
import com.thoughtworks.xstream.XStream;
import com.thoughtworks.xstream.io.json.JettisonMappedXmlDriver;
class MockMessage {
int val = 1;
}
class MockMessageOther {
int otherVal = 1;
}
public class TestJSONXStream {
public static void main(String[] args) {
JettisonMappedXmlDriver xmlDriver = new JettisonMappedXmlDriver();
XStream xstream = new XStream(xmlDriver);
MockMessage mock1 = new MockMessage();
MockMessage mock2 = new MockMessage();
MockMessageOther mock3 = new MockMessageOther();
List messages = new ArrayList();
messages.add(mock1);
messages.add(mock2);
messages.add(mock3);
String jsonString = xstream.toXML(messages);
//JSON list format is non-intuitive single element array with class name fields
System.out.println(jsonString);
List xstreamJSONUnmarshalledMessages = (List)xstream.fromXML(jsonString);
//This will print 3 messages unmarshalled
System.out.println("XStream format JSON Number of messages unmarshalled: " + xstreamJSONUnmarshalledMessages.size());
//Attempt to deserialize a reasonable looking JSON string
String jsonTest =
"{"+
"\"list\" : ["+
"{"+
"\"MockMessage\" : {"+
"\"val\" : 1"+
"}"+
"}, {"+
"\"MockMessage\" : {"+
"\"val\" : 1"+
"}"+
"}, {"+
"\"MockMessageOther\" : {"+
"\"otherVal\" : 1"+
"}"+
"} ]"+
"};";
List unmarshalledMessages = (List)xstream.fromXML(jsonTest);
//We expect 3 messages but XStream only deserializes one
System.out.println("Normal format JSON Number of messages unmarshalled: " + unmarshalledMessages.size());
}
}
Intuitively I expect the XStream JSON to be serialized (and able to deserialize correctly) from the following format:
{
"list" : [
{
"MockMessage" : {
"val" : 1
}
}, {
"MockMessage" : {
"val" : 1
}
}, {
"MockMessageOther" : {
"otherVal" : 1
}
} ]
}
Instead XStream creates a single element list with fields that are named the classnames and nested arrays of Objects of the same type.
{
"list" : [ {
"MockMessage" : [ {
"val" : 1
}, {
"val" : 1
} ],
"MockMessageOther" : {
"otherVal" : 1
}
} ]
}
The trouble may be caused by it using the XStream XML CollectionConverter?
Does anyone have a suggestion for a good JSON Java object serialization that allows you to read/write arbitrary Java objects. I looked at the Jackson Java JSON Processor but when you were reading in objects from a stream you had to specify what type of object it was unlike XStream where it will read in any object (because the serialized XStream JSON contains class name information).

I agree with other poster in that XStream is not a good fit -- it's an OXM (Object/Xml Mapper), and JSON is handled as a secondary output format using XML processing path. This is why a "convention" (of how to convert hierarchich xml model into object-graph model of json and vice versa) is needed; and your choice boils down to using whatever is least intrusive of sub-optimal choices.
That works ok if XML is your primary data format, and you just need some rudimentary JSON(-like) support.
To get good JSON-support, I would consider using a JSON processing library that does real OJM mapping (I assume Svenson does too, but additionally), such as:
Jackson
Google-gson
Also: even if you do need to support both XML and JSON, you are IMO better off using separate libraries for these tasks -- objects (beans) to use on server-side need not be different, just serialization libs that convert to/from xml and json.

I realize this is off-topic, but I'd like to present a solution in svenson JSON.
Do you really need public fields in your domain classes? Apart from having to use properties, svenson can handle cases like this with a more simple JSON output with a discriminator property
class Message
{
// .. your properties with getters and setters ..
// special property "type" acts a signal for conversion
}
class MessageOther
{
...
}
List list = new ArrayList();
list.add(new Message());
list.add(new MessageOther());
list.add(new Message());
String jsonDataSet = JSON.defaultJSON().forValue(list);
would output JSON like
[
{"type":"message", ... },
{"type":"message_other", ... },
{"type":"message", ... }
]
which could be parsed again with code like this
// configure reusable parse instance
JSONParser parser = new JSONParser();
// type mapper to map to your types
PropertyValueBasedTypeMapper mapper = new PropertyValueBasedTypeMapper();
mapper.setParsePathInfo("[]");
mapper.addFieldValueMapping("message", Message.class);
mapper.addFieldValueMapping("message_other", MessageOther.class);
parser.setTypeMapper(mapper);
List list = parser.parse(List.class, jsonDataset);

A svenson type mapper based on the full class name would look something like this
public class ClassNameBasedTypeMapper extends PropertyValueBasedTypeMapper
{
protected Class getTypeHintFromTypeProperty(String value) throws IllegalStateException
{
try
{
return Class.forName(value);
}
catch (ClassNotFoundException e)
{
throw new IllegalStateException(value + " is no valid class", e);
}
}
}
which is not an ideal implementation as it inherits the configuration of PropertyValueBasedTypeMapper without really needing. (should include a cleaner version in svenson)
The setup is very much like above
JSONParser parser = new JSONParser();
ClassNameBasedTypeMapper mapper = new ClassNameBasedTypeMapper();
mapper.setParsePathInfo("[]");
parser.setTypeMapper(mapper);
List foos = parser
.parse( List.class, "[{\"type\":\"package.Foo\"},{\"type\":\"package.Bar\"}]");