We Keep Coding

Importing json that was serialized by JSON is failing

So I have an object with some fields...
protected String name;
protected String relativePathAndFileName;
protected DateTime next_Run;
protected ArrayList<String> hosts;
Which gets serialized to JSON like this:
public void serialize(){
Gson gson = Converters.registerDateTime(new GsonBuilder()).setPrettyPrinting().create();
String json = gson.toJson(this);
try {
FileWriter writer = new FileWriter(this.relativePathAndFileName);
writer.write (json);
writer.close();
} catch (IOException e) {
logger.error("Error while trying to write myAlert to json: ", e);
}
}
Later when I need to read in this json file, I try to do so like this:
try {
for (File f : alertConfigFiles) {
Gson gson = new Gson();
JsonReader reader = new JsonReader(new FileReader(f));
Type type = new TypeToken<Map<String, String>>(){}.getType();
Map<String, String> myMap = gson.fromJson(reader, type);
Alert tempAlert = new Alert(myMap);
myAlerts.add(tempAlert);
logger.debug("Imported: " + f.toString());
}
The error that I'm getting is:
Unhandled exception when trying to import config files:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a string but was BEGIN_ARRAY at line 28 column 13 path $.
The JSON inside the file is something to the effect of:
{
"name": "Logs Per Host - past 24 hours",
"relativePathAndFileName": "./Elk-Reporting/Alerts/logs_per_host24h.json",
"next_Run": "2017-06-07T22:24:56.682-04:00",
"hosts": [
"app-12c",
"app1-18",
"wp-01",
"app-02",
"wp-02",
"cent-04",
"app-06",
"app-05"
]
}
It seems to be choking when it tries to import the ArrayList of hosts, but it was able to write them out without issues.
Can anyone offer some advice on how to get my import working without issues?

try to keep it simple. Using maps and so on, is a way to have issues.
Here is a working code to deserialise / serialise :
package com.rizze.beans.labs.sof;
import static org.junit.Assert.assertTrue;
import org.junit.Test;
import com.google.gson.Gson;
public class SOFGson {
public String json = "{ \"name\": \"Logs Per Host - past 24 hours\", \"relativePathAndFileName\": \"./Elk-Reporting/Alerts/logs_per_host24h.json\", \"next_Run\": \"2017-06-07T22:24:56.682-04:00\", \"hosts\": [ \"bos-qa-app-12c\", \"bos-qa-app1-18\", \"bos-qa-wp-01\", \"bos-lt-app-02\", \"bos-qa-wp-02\", \"bos-dev-cent-04.americanwell.com\", \"bos-qa-app-06\", \"bos-qa-app-05\" ]}";
public class MyObj{
protected String name;
protected String relativePathAndFileName;
protected String next_Run;
protected String[] hosts;
}
#Test
public void test() {
Gson gson = new Gson();
MyObj obj = gson.fromJson(json, MyObj.class);
assertTrue(obj!=null);
assertTrue(obj.hosts.length==8);
System.out.println(gson.toJson(obj));
}
}
here is the class in gist : https://gist.github.com/jeorfevre/7b32a96d4ddc4af68e40bf95f63f2c26

Those two lines seem to be the problem:
Type type = new TypeToken<Map<String, String>>(){}.getType();
Map<String, String> myMap = gson.fromJson(reader, type);
You serialize your object of some specific class. You then deserialize it to type. But your JSON does not fit into a Map. Better do it like this, so you can use your own class.
YourClass myMap = gson.fromJson(reader, YourClass.class);
If you want to use this approach, you might want to change your Java class to hold an array of strings instead of an ArrayList of strings.
Maybe this page helps you a bit. Especially the first case fits your situation.
Another option is a custom Deserialzer as described here.

JSON validation using JSONAssert with regular expressions

I am working in a open-source project which uses REST interface. To validate (match actual response with expected) our rest interfaces in the JUnit, we would like to use the JSONAssert. (https://github.com/skyscreamer/JSONassert). But I have a problem with the usage.Kindly help to resolve it.
Expected JSON:
{
"objectId": "[^\\s]",
"protocol": "[^\\s]",
"hostName": "[^\\s]",
"port": "[^\\s]",
"commParams": "[^\\s]"
}
Remarks: objectId/protocol/hostName/port/commParams can be anything but should not be empty
Actual JSON:
{
"objectId": "controller2",
"protocol": "ftp",
"hostName": "sdnorchestrator",
"port": "21",
"commParams": "username:tomcat, password:tomdog"
}
Problem1: Which interface of JSON Assert, i need to use to solve the above issue:Below one?
JSONAssert.assertEquals("Expected JSON", "Actual JSON" new CustomComparator(
JSONCompareMode.LENIENT_ORDER, new Customization(PREFIX, new RegularExpressionValueMatcher<Object>())));
Problem 2: What should be the PREFIX here?(I tried with "", "., "." but had no success)
Any other recommendation (other than JSONAssert) for the above problem is also welcome.

If you want to globally apply regular expression customizations with JSONAssert, you can construct a single Customization with path = "***", and use the RegularExpressionValueMatcher constructor with no arguments.
Example:
final String expectedJson = "{objectId: \"[^\\s]+\", protocol: \"[^\\s]+\"}";
final String actualJson = "{\"objectId\": \"controller2\", \"protocol\": \"ftp\"}";
JSONAssert.assertEquals(
expectedJson,
actualJson,
new CustomComparator(
JSONCompareMode.LENIENT,
new Customization("***", new RegularExpressionValueMatcher<>())
)
);
This assertion passes successfully (tested with JSONassert version 1.5.0).

I have found better alternative.Usage of JsonSchema validator would solve most of the problem (http://json-schema.org/). Write the json schema in expected response and validate it using json validator jar. (json-schema/json-schema-validator-2.0.1.jar.zip( 166 k))

I could not get the path to except an RE either. Eventually I just over-rode the compareValues of DefaultComparator to force it to apply the RegularExpressionValueMatcher to all paths:
import org.json.JSONException;
import org.skyscreamer.jsonassert.JSONAssert;
import org.skyscreamer.jsonassert.JSONCompareResult;
import org.skyscreamer.jsonassert.RegularExpressionValueMatcher;
import org.skyscreamer.jsonassert.ValueMatcherException;
import org.skyscreamer.jsonassert.comparator.DefaultComparator;
import static org.skyscreamer.jsonassert.JSONCompareMode.STRICT;
...
final RegularExpressionValueMatcher reMatcher = new RegularExpressionValueMatcher();
JSONAssert.assertEquals( "{\"key\":\"v.*\"}", "{\"key\":\"value\"}",
new DefaultComparator( STRICT ) {
#Override
public void compareValues(String prefix, Object expectedValue, Object actualValue, JSONCompareResult result) throws JSONException {
try {
if( !reMatcher.equal(actualValue, expectedValue) ) result.fail(prefix, expectedValue, actualValue);
} catch( ValueMatcherException e ) { result.fail(prefix, e); }
}
}

Serialize ArrayList of custom class to JSON using Gson and send data over HTTP

I would like to convert the data in an ArrayList to JSON and then send it to my webserver. The list mTeamDataList is of type ArrayList<TeamData>.
The TeamData class is:
public class TeamData
{
String mFullName;
String mShortName;
String mLeague;
//constructor, getters and setters are here
}
I have a addTeamsToDB() method that is responsible for writing the data in the array to the webserver. Here is what I have so far:
public static void addTeamsToDB()
{
if(mTeamDataList.size() == 0)
return;
HttpURLConnection urlConnection = null;
String addTeamURL = "http://api.somewebsite.com/add_team.php";
try
{
Gson gson = new Gson();
URL urlObj = new URL(addTeamURL);
urlConnection = (HttpURLConnection) urlObj.openConnection();
urlConnection.setDoOutput(true);
urlConnection.setRequestMethod("POST");
urlConnection.connect();
//I believe converting to json goes here
OutputStream out = new BufferedOutputStream(urlConnection.getOutputStream());
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
assert urlConnection != null;
urlConnection.disconnect();
}
}
I found several answers on SO but they only showed basic examples of inserting one object or hard-coded data. I have not found one to convert an array of custom data-type objects to JSON using Gson.
Is there a method that does this or do I have to manually convert each item in the array through a loop?
I was following this tutorial but he's using the NameValuePairs class to achieve this. But it's deprecated so I'm not sure what to use instead.
Also, the tutorial is using the built-in JSON java library so if someone can show the Gson way instead, that would be very helpful.

If your class has nothing more than what you posted, then there shouldn't be a problem using Gson's toJson(Object src, Type typeOfSrc) directly on your ArrayList :
Type listType = new TypeToken<ArrayList<TeamData>>() {}.getType();
String json = gson.toJson(mTeamDataList, listType);
The reason for that, is that your TeamData class has only generic fields.
Of course, if you want the "keys" to appear in your JSON, you should add the #SerializedName annotation on your class members.
Then, to send the data using your HttpUrlConnection, replace the line where you declare your out variable by :
OutputStreamWriter wr= new OutputStreamWriter(urlConnection.getOutputStream());
wr.write(json);

Use the class some think like this,
import com.google.gson.annotations.SerializedName;
import java.util.ArrayList;
/**
* #author Krish
*/
public class TeamDataList {
#SerializedName("mTeamDataList")
private ArrayList<TeamData> mTeamDataList;
public TeamDataList(ArrayList<TeamData> teamDataList) {
this.mTeamDataList = teamDataList;
}
public class TeamData {
#SerializedName("mFullName")
String mFullName;
#SerializedName("mShortName")
String mShortName;
#SerializedName("mLeague")
String mLeague;
//constructor, getters and setters are here
}
}
And use this this method for serializing,
public static String toJson(Object object) {
Gson gson = new Gson();
return gson.toJson(object);
}

Gson - Automatic quote (") escaping?

I'm using GSON on my Java EE server to provide some json to views.
In some object, I have long text, that can contains anything (like 'What a "great" news!').
I'm supprised that by default GSON doesn't escape the double quote, so it doesn't generate a valid JSON.
Is there a good way of doing this ?

Maybe I'm not understanding your question, but I was able to get GSON to handle Strings with quotes without any settings or changes.
import com.google.gson.Gson;
public class GSONTest {
public String value;
public static void main(String[] args) {
Gson g = new Gson();
GSONTest gt = new GSONTest();
gt.value = "This is a \"test\" of quoted strings";
System.out.println("String: " + gt.value);
System.out.println("JSON: " + g.toJson(gt));
}
}
Output:
String: This is a "test" of quoted strings
JSON: {"value":"This is a \"test\" of quoted strings"}
Maybe I don't understand what you're asking?

Try using setPrettyPrinting with DisableHtml escaping.
import com.google.gson.Gson;
Gson gson = new GsonBuilder().setPrettyPrinting().disableHtmlEscaping().create();
JsonParser jp = new JsonParser();
JsonElement je = jp.parse(jsonArray.toString());
System.out.println( gson.toJson(je));

Here's some sample GSON code:
final JsonObject obj = new JsonObject();
obj.addProperty("foo", "b\"a\"r");
System.out.println(obj.toString());
The Output is:
{"foo":"b\"a\"r"}
(as it should be)
So either you are doing something wrong, or you are using an ancient version of GSON. Perhaps you should show some of your code?

That is what I did to solve that
private final Gson mGson;
{
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(String.class, new EscapeStringSerializer());
mGson = builder.create();
}
private static class EscapeStringSerializer implements JsonSerializer<String> {
#Override
public JsonElement serialize(String s, Type type, JsonSerializationContext jsonSerializationContext) {
return new JsonPrimitive(escapeJS(s));
}
public static String escapeJS(String string) {
String escapes[][] = new String[][]{
{"\\", "\\\\"},
{"\"", "\\\""},
{"\n", "\\n"},
{"\r", "\\r"},
{"\b", "\\b"},
{"\f", "\\f"},
{"\t", "\\t"}
};
for (String[] esc : escapes) {
string = string.replace(esc[0], esc[1]);
}
return string;
}
}

Convert XML to JSON format

I have to convert docx file format (which is in openXML format) into JSON format. I need some guidelines to do it. Thanks in advance.

You may take a look at the Json-lib Java library, that provides XML-to-JSON conversion.
String xml = "<hello><test>1.2</test><test2>123</test2></hello>";
XMLSerializer xmlSerializer = new XMLSerializer();
JSON json = xmlSerializer.read( xml );
If you need the root tag too, simply add an outer dummy tag:
String xml = "<hello><test>1.2</test><test2>123</test2></hello>";
XMLSerializer xmlSerializer = new XMLSerializer();
JSON json = xmlSerializer.read("<x>" + xml + "</x>");

There is no direct mapping between XML and JSON; XML carries with it type information (each element has a name) as well as namespacing. Therefore, unless each JSON object has type information embedded, the conversion is going to be lossy.
But that doesn't necessarily matter. What does matter is that the consumer of the JSON knows the data contract. For example, given this XML:
<books>
<book author="Jimbo Jones" title="Bar Baz">
<summary>Foo</summary>
</book>
<book title="Don't Care" author="Fake Person">
<summary>Dummy Data</summary>
</book>
</books>
You could convert it to this:
{
"books": [
{ "author": "Jimbo Jones", "title": "Bar Baz", "summary": "Foo" },
{ "author": "Fake Person", "title": "Don't Care", "summary": "Dummy Data" },
]
}
And the consumer wouldn't need to know that each object in the books collection was a book object.
Edit:
If you have an XML Schema for the XML and are using .NET, you can generate classes from the schema using xsd.exe. Then, you could parse the source XML into objects of these classes, then use a DataContractJsonSerializer to serialize the classes as JSON.
If you don't have a schema, it will be hard getting around manually defining your JSON format yourself.

The XML class in the org.json namespace provides you with this functionality.
You have to call the static toJSONObject method
Converts a well-formed (but not necessarily valid) XML string into a JSONObject. Some information may be lost in this transformation because JSON is a data format and XML is a document format. XML uses elements, attributes, and content text, while JSON uses unordered collections of name/value pairs and arrays of values. JSON does not does not like to distinguish between elements and attributes. Sequences of similar elements are represented as JSONArrays. Content text may be placed in a "content" member. Comments, prologs, DTDs, and <[ [ ]]> are ignored.

If you are dissatisfied with the various implementations, try rolling your own. Here is some code I wrote this afternoon to get you started. It works with net.sf.json and apache common-lang:
static public JSONObject readToJSON(InputStream stream) throws Exception {
SAXParserFactory factory = SAXParserFactory.newInstance();
factory.setNamespaceAware(true);
SAXParser parser = factory.newSAXParser();
SAXJsonParser handler = new SAXJsonParser();
parser.parse(stream, handler);
return handler.getJson();
}
And the SAXJsonParser implementation:
package xml2json;
import net.sf.json.*;
import org.apache.commons.lang.StringUtils;
import org.xml.sax.*;
import org.xml.sax.helpers.DefaultHandler;
import java.util.ArrayList;
import java.util.List;
public class SAXJsonParser extends DefaultHandler {
static final String TEXTKEY = "_text";
JSONObject result;
List<JSONObject> stack;
public SAXJsonParser(){}
public JSONObject getJson(){return result;}
public String attributeName(String name){return "#"+name;}
public void startDocument () throws SAXException {
stack = new ArrayList<JSONObject>();
stack.add(0,new JSONObject());
}
public void endDocument () throws SAXException {result = stack.remove(0);}
public void startElement (String uri, String localName,String qName, Attributes attributes) throws SAXException {
JSONObject work = new JSONObject();
for (int ix=0;ix<attributes.getLength();ix++)
work.put( attributeName( attributes.getLocalName(ix) ), attributes.getValue(ix) );
stack.add(0,work);
}
public void endElement (String uri, String localName, String qName) throws SAXException {
JSONObject pop = stack.remove(0); // examine stack
Object stashable = pop;
if (pop.containsKey(TEXTKEY)) {
String value = pop.getString(TEXTKEY).trim();
if (pop.keySet().size()==1) stashable = value; // single value
else if (StringUtils.isBlank(value)) pop.remove(TEXTKEY);
}
JSONObject parent = stack.get(0);
if (!parent.containsKey(localName)) { // add new object
parent.put( localName, stashable );
}
else { // aggregate into arrays
Object work = parent.get(localName);
if (work instanceof JSONArray) {
((JSONArray)work).add(stashable);
}
else {
parent.put(localName,new JSONArray());
parent.getJSONArray(localName).add(work);
parent.getJSONArray(localName).add(stashable);
}
}
}
public void characters (char ch[], int start, int length) throws SAXException {
JSONObject work = stack.get(0); // aggregate characters
String value = (work.containsKey(TEXTKEY) ? work.getString(TEXTKEY) : "" );
work.put(TEXTKEY, value+new String(ch,start,length) );
}
public void warning (SAXParseException e) throws SAXException {
System.out.println("warning e=" + e.getMessage());
}
public void error (SAXParseException e) throws SAXException {
System.err.println("error e=" + e.getMessage());
}
public void fatalError (SAXParseException e) throws SAXException {
System.err.println("fatalError e=" + e.getMessage());
throw e;
}
}

Converting complete docx files into JSON does not look like a good idea, because docx is a document centric XML format and JSON is a data centric format. XML in general is designed to be both, document and data centric. Though it is technical possible to convert document centric XML into JSON, handling the generated data might be overly complex. Try to focus on the actual needed data and convert only that part.

If you need to be able to manipulate your XML before it gets converted to JSON, or want fine-grained control of your representation, go with XStream. It's really easy to convert between: xml-to-object, json-to-object, object-to-xml, and object-to-json. Here's an example from XStream's docs:
XML
<person>
<firstname>Joe</firstname>
<lastname>Walnes</lastname>
<phone>
<code>123</code>
<number>1234-456</number>
</phone>
<fax>
<code>123</code>
<number>9999-999</number>
</fax>
</person>
POJO (DTO)
public class Person {
private String firstname;
private String lastname;
private PhoneNumber phone;
private PhoneNumber fax;
// ... constructors and methods
}
Convert from XML to POJO:
String xml = "<person>...</person>";
XStream xstream = new XStream();
Person person = (Person)xstream.fromXML(xml);
And then from POJO to JSON:
XStream xstream = new XStream(new JettisonMappedXmlDriver());
String json = xstream.toXML(person);
Note: although the method reads toXML() XStream will produce JSON, since the Jettison driver is used.

If you have a valid dtd file for the xml snippet, then you can easily convert xml to json and json to xml using the open source eclipse link jar. Detailed sample JAVA project can be found here: http://www.cubicrace.com/2015/06/How-to-convert-XML-to-JSON-format.html

I have come across a tutorial, hope it helps you.
http://www.techrecite.com/xml-to-json-data-parser-converter

Use
xmlSerializer.setForceTopLevelObject(true)
to include root element in resulting JSON.
Your code would be like this
String xml = "<hello><test>1.2</test><test2>123</test2></hello>";
XMLSerializer xmlSerializer = new XMLSerializer();
xmlSerializer.setForceTopLevelObject(true);
JSON json = xmlSerializer.read(xml);

Docx4j
I've used docx4j before, and it's worth taking a look at.
unXml
You could also check out my open source unXml-library that is available on Maven Central.
It is lightweight, and has a simple syntax to pick out XPaths from your xml, and get them returned as Json attributes in a Jackson ObjectNode.