We Keep Coding

How to create and append to a JSON file iteratively?

My test class calls the addEachEmployeeDetailsToJSONFile method multiple times during the execution with different employee details.
The expected end result is a JSON file named mapping.json that looks like below:
EXPECTED mapping.json
[
{
"employee": {
"ID": "123",
"Name": "Gupta",
"Department": "Accounts"
}
},
{
"employee": {
"ID": "456",
"Name": "Mark",
"Department": "Marketing"
}
}
]
Below is my Java class.
import org.json.simple.JSONObject;
import org.json.simple.JSONArray;
import java.io.*;
public class Test {
private String filePath = new File("").getAbsolutePath() + File.separator + "target" + File.separator + "mapping.json";
protected JSONObject employeeDetails = new JSONObject();
protected JSONObject employeeObject = new JSONObject();
protected JSONArray employeeList = new JSONArray();
private FileWriter fileWriter;
{
try {
fileWriter = new FileWriter(getMappingFile(), true);
} catch (IOException e) {
e.printStackTrace();
}
}
//Write to JSON file
private File getMappingFile (){
File mappingFile = new File(getMappingFilePath);
mappingFile.setWritable(true);
mappingFile.setReadable(true);
return mappingFile;
}
public void addEachEmployeeDetailsToJSONFile(ITestContext iTestContext) {
try {
//Write to first JSONObject
employeeDetails.put("ID", iTestContext.getId());
employeeDetails.put("Name", iTestContext.getName());
employeeDetails.put("Department", iTestContext.getDepartment());
//Put the above JSON Object in another JSON object.
employeeObject.put("employees", employeeDetails);
//Write above object to JSONArray
employeeList.add(employeeObject);
//We can write any JSONArray or JSONObject instance to the file
fileWriter.append(beforeTestClassesList.toJSONString());
}
catch (Exception e)
{}
}
public static void main( String[] args )
{
addEachEmployeeDetailsToJSONFile(<pass employee Gupta instance>);
addEachEmployeeDetailsToJSONFile(<pass employee Mark instance>);
fileWriter.flush();
}
}
The JSON file is being created successfully and an entry is also being created like below, but, only the first entry of one employee details is being created and not a list of multiple employees and their details. What am I missing?
Current mapping.json file:
[
{
"employee": {
"ID": "123",
"Name": "Gupta",
"Department": "Accounting"
}
}
]

Your code as shown only adds 1 item. So this is as expected. Also, you would create a new JSON object for each employee, not reuse the same one repeatedly. That will cause problems on subsequent saves. Further, your code is really a bit poorly architected. Due to the nature of JSON, appending to a file isn't really done. You write the entire file at once. This means you don't write until you have all the data, you don't write after adding each item to the array. The reason for this is that there's data after the element in the array to make it valid JSON, and you can't stick stuff in the middle of a file. Appending to a JSONArray in memory is fine, but when writing to disk you want to write the entire file.

Gson throws IllegalStateException when attempting to parse List

I am learning data persistence and this is my first attempt at JSON. I have read a few guides and from what little I can tell the code has been correct in both attempts at storing the objects. I get the file written using Gson but Gson throws exceptions when attempted to parse the objects using the fromJson() method. My question is as follows:
If I am using the same type to convert to/from JSON what am I missing that would tell Gson how to properly parse my object(s)?
I have tried three different approaches, two of which are included below. First I tried storing the wrapper class for the list of objects which a guide suggested I should be able to do:
public class JSONConverter {
private static Path path = Paths.get("src\\json\\JSONList.json");
private static Type stockType = new TypeToken<StocksList>(){}.getType();
public static void convertToJSON(StocksList stocks, Path path) {
Gson json = new Gson();
String storedStocks = json.toJson(stocks, stockType);// I also tried "StocksList.class" here
checkForFile(path);
try (BufferedWriter writer = Files.newBufferedWriter(path)) {
writer.write(storedStocks);
} catch (IOException e) {
e.printStackTrace();
//handle later
}
}
static void checkForFile(Path path) {
if (Files.notExists(path)) {
try {
Files.createFile(path);
} catch (IOException e) {
e.printStackTrace();
//handle later
}
}
}
public static StocksList convertFromJSON(Path path) {
StocksList stocksList = new StocksList();
Gson json = new Gson();
String fromJson;
try {
fromJson = Files.readAllBytes(path).toString();
stocksList = json.fromJson(fromJson, stockType);
return stocksList;
} catch (IOException e) {
return stocksList;
}
}
}
My second approach was to get the list out of the wrapper class and try to convert that to JSON:
public class JSONConverter {
private static Path path = Paths.get("src\\json\\JSONList.json");
private static Type listType = new TypeToken<List<Stock>>(){}.getType();
public static void convertToJSON(StocksList stocks, Path path) {
Gson json = new Gson();
List<Stock> temp = stocks.getStocks();
String storedStocks = json.toJson(temp, listType);
checkForFile(path);
try (BufferedWriter writer = Files.newBufferedWriter(path)) {
writer.write(storedStocks);
} catch (IOException e) {
e.printStackTrace();
//handle later
}
}
static void checkForFile(Path path) {
if (Files.notExists(path)) {
try {
Files.createFile(path);
} catch (IOException e) {
e.printStackTrace();
//handle later
}
}
}
public static StocksList convertFromJSON(Path path) {
StocksList stocksList = new StocksList();
List<Stock> stocks = new ArrayList<>();
Gson json = new Gson();
String fromJson;
try {
fromJson = Files.readAllBytes(path).toString();
stocks = json.fromJson(fromJson, listType);
//wraps the list in the stockslist class
stocksList.setStocks(stocks);
return stocksList;
} catch (IOException e) {
e.printStackTrace();
return stocksList;
}
}
}
Here is a sample of the JSON written by the first method using the second approach. The first looks like it except adds "{ "stocks" :" (what you see below) "}":
[
{
"ticker": "INTC",
"currentPrice": "45.94",
"marginOfSafety": 0.25,
"lastDate": "2019-12-28",
"cashYield": "7.4",
"MCap": "196485365760",
"enterpriseValue": "281213850000",
"sharesOut": "4417000000",
"oddPercentGrowth": false,
"newCompany": false,
"safeValue": "51.35",
"fairValue": "68.47",
"evEbitda": "8.56",
"fcf": [
"16932000000",
"14611750000"
],
"rOnAssets": "21",
"rOnCapital": "20",
"croic": "16.47",
"equityToDebt": "3.0",
"cashOnHand": "4194000000",
"cashToDebt": "0.17",
"changeInDebt": "210000000",
"capEfficiency": [
"18",
"7",
"-26",
"-21",
"1"
],
"fcfChange": [
"18.81",
"11.71"
],
"profitMargin": [
"46",
"38"
]
},
{
"ticker": "HCC",
"currentPrice": "12.99",
"marginOfSafety": 0.5,
"lastDate": "2018-12-31",
"cashYield": "46.1",
"MCap": "664587904",
"enterpriseValue": "1572623480",
"sharesOut": "52812000",
"oddPercentGrowth": true,
"newCompany": true,
"safeValue": "236.94",
"fairValue": "473.87",
"evEbitda": "2.59",
"fcf": [
"457776000",
"306126750"
],
"rOnAssets": "49",
"rOnCapital": "59",
"croic": "38.77",
"equityToDebt": "1.0",
"cashOnHand": "205577000",
"cashToDebt": "0.44",
"changeInDebt": "125283000",
"capEfficiency": [
"292",
"798",
"-365",
"-397",
"-1"
],
"fcfChange": [
"33.9",
"33.9"
],
"profitMargin": [
"40",
"8"
]
}
]
Both throw:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException:
Expected BEGIN_OBJECT but was STRING at line 1 column 12
(this line changes to "Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 2" when using the first approach).
at
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
...
I was going to try adding each object individually to a JSONArray but when I started encountering trouble there I thought I should just ask. The guide mentioned that reflection was important and I am guessing that my trouble lies therein due to the second line of the stack trace but again, this is my first time trying to use JSON. If I forgot to include anything let me know and I'll post it in a comment.
Thanks for the help.
ADDENDUM: the objects only throw these exceptions when being written to and pulled from a file. They do not throw when being converted to a JSON String and then back again. It occurs whether I use Files.write() or Files.newBufferedWriter().

Thanks to all those who viewed my question. I reached out to Gson's github page and they responded with the following corrections to my class:
All the code you've provided can be greatly fixed, improved and refactored.
No need to create multiple Gson instances: they are relatively
expensive to instantiate, but are designed to be thread-safe and
immutable therefore can be reused. No need to serialize to and
deserialize from java.lang.String -- this is just expensive as long as
it has to create multiple strings in the heap merely wasting the heap
and time decreasing the performance. Why it does not work in your case
is that Files.readAllBytes(...) returns byte[] you're trying to
convert to a string. In Java, no arrays have an intuitive toString
implementation (you can check it by simply printing any byte array to
System.out). In order to convert it to a string (that might be a
memory-consuming instance), new String(byte[]) (or even new
String(byte[], Charset)) is an appropriate way. I don't really
remember how Files works, but there's probably no need to check the
file to exist: they can be overwritten without any additional checks.
No type tokens are necessary in this case: StockList.class is a Type
too. Essentially, all is you need is just as follows:
private static final Gson gson = new GsonBuilder()
.disableHtmlEscaping()
.disableInnerClassSerialization()
.create();
public static void main(final String... args)
throws IOException {
final StocksList before = new StocksList(ImmutableList.of(new Stock("INTC"), new
Stock("HCC")));
final Path path = Paths.get("doc.json");
write(path, before);
final StocksList after = read(path);
System.out.println(after.equals(before));
}
private static void write(final Path path, final StocksList stocks)
throws IOException {
try ( final Writer writer = new OutputStreamWriter(new
FileOutputStream(path.toFile())) ) {
gson.toJson(stocks, writer);
}
}
private static StocksList read(final Path path)
throws IOException {
try ( final Reader reader = new InputStreamReader(new
FileInputStream(path.toFile())) ) {
return gson.fromJson(reader, StocksList.class);
}
}
Thanks to lyubomyr-shaydariv (Gson contributor) for the answer.

How do I get data from JSON string?

Here is my code:
try {
JSONObject json = (JSONObject) new JSONTokener(result).nextValue();
System.out.println(json);
JSONObject json2 = json.getJSONObject("data");
String test = json2.getString("headline");
System.out.println(test);
} catch (JSONException e) {
e.printStackTrace();
}
My String values start with the object data. So I am trying to get that object first and then capture the the object headline inside that.
My problem is, it is not taking the object data from the string.
Once I reach the line JSONObject json2 = json.getJSONObject("data");, it throws the exception. Please shed some light on this.
"data": [
{
"headline": "Close Update"
"docSource": "MIDNIGHTTRADER",
"source": "MTClosing",
"dateTime": "2015-10-23T16:42:46-05:00",
"link": "Markets/News",
"docKey": "1413-A1067083-1B14K77PVTUM1O7PCAFMI3SJO4",
},

The value for the key data is a JSON array containing one object, and not an object itself.
To get that object inside data, replace your line that throws an exception with the following:
JSONObject json2 = json.getJSONArray("data").get(0);
This gets the data array as a JSONArray object and then gets the 0th element, which is the object you want.

Your data "object", isn't actually an object, it's an array, notice the opening square bracket... I'm assuming in your actual code, it closes with one too.
"data": [{
"headline": "Close Update"
"docSource": "MIDNIGHTTRADER",
"source": "MTClosing",
"dateTime": "2015-10-23T16:42:46-05:00",
"link": "Markets/News",
"docKey": "1413-A1067083-1B14K77PVTUM1O7PCAFMI3SJO4",
}]
Try json.getJSONArray("data")[0] instead... or whatever index you need
try {
JSONObject json = (JSONObject) new JSONTokener(result).nextValue();
System.out.println(json);
JSONObject json2 = json.getJSONArray("data")[0];
String test = json2.getString("headline");
System.out.println(test);
}
catch (JSONException e) {
e.printStackTrace();

Your problem is based on the fact that your service returns and array instead of a single json object, so from here you can follow this suggestions to process directly from the JSONArray Can't access getJSONArray in java, or, at server side you can encapsulate your response array into another object like this (java example):
public class Data<T> {
private List<T> elements;
public ObjectSugetionsDTO(){
And build the response like this:
return new ResponseEntity<Data<YourInternalRepresentation>>(
new Data<YourInternalRepresentation>(yourMethodCallForTheArray()),
HttpStatus.OK);
I have found the second way to be better at keeping my API cleaner and more readable
EDIT: Better way
I whould also suggest the use of retrofit (http://square.github.io/retrofit/), by doing so, your service calls is resumed to (Example of calling and API that retrieves a list of users):
public class UserService {
public static IUserService getUserService() {
return RestAdapterManager.createService(IUserService.class );
}
public interface IUserService{
#GET("/api/users")
public void getAllUsers(Callback<List<User>> callback);
}
}
and the service call itself
UserService.getUserService().getAllUsers(new Callback<List<User>>() {
#Override
public void success(List<User> users, Response response) {
Log.d("Exito! " , "" + users.size());
}
#Override
public void failure(RetrofitError error) {
Log.d("Fail!", error.getUrl());
}
});
The simple inicialization of the connection object
public static <S> S createService(Class<S> serviceClass, String username, String password) {
RestAdapter.Builder builder = new RestAdapter.Builder()
.setEndpoint(API_BASE_URL);//Your api base url
RestAdapter adapter = builder.setLogLevel(RestAdapter.LogLevel.FULL).build(); //change the logging level if you need to, full is TOO verbose
return adapter.create(serviceClass);
}

How do I make nested JSON Objects using Gson?

I have written a program that does some probability calculations and gives its results in the form of arrays. I want to convert these results to JSON format, but I am having issues.
I want my json object to look like this:
{
"totalSuggestions": 6,
"routes": {
"rank_2": {
"Source": "ABC",
"Weight": "0.719010390625",
"Destination": "XYZ"
},
"rank_1": {
"Source": "XYZ",
"Weight": "0.7411458281249999",
"Destination": "ABC"
},
"rank_0": {
"Source": "LMN",
"Weight": "0.994583325",
"Destination": "PQR"
}
}
}
What I understood is that I need to have an object class with the structure of my objects. For now I am experimenting with the rank object only but failing to form the required JSON.
My code for the object structure:
public class Object {
int rank_;
public class Inner{
String Source;
String Destination;
String Weightage;
}
}
I can pass either an instance of Object or an instance of Inner to toJson() method so I either get {"rank_":1} or {"Source":"ABC","Destination":"XYZ","Weightage":"123"}.
I cant seem to put each of the inner object to the corresponding rank object.
I did it with relative ease with org.json but that library has some issues with Android studio so I had to switch to Gson. What I did earlier (which worked as well) was:
public JSONObject convertToJson(int mkr, String[][] result){
JSONObject outerObj = new JSONObject();
JSONObject innerObj = new JSONObject();
JSONObject[] temp = new JSONObject[mkr];
outerObj.put("totalSuggestions", marker);
outerObj.put("routes",innerObj);
for (int i=0;i<marker;i++){
String[] useless = result[i][0].split("-");
temp[i]= new JSONObject();
temp[i].put("Source",useless[0] );
temp[i].put("Destination", useless[1]);
temp[i].put("Weight", result[i][1]);
innerObj.put("rank_"+i, temp[i]);
}
System.out.println(outerObj.toString());
return outerObj;
}

Well, first: related objects should probably be in a class together. So lets start with a simple class:
public class Results {
int mkr;
String[][] result;
}
Now we want to serialize it. We could construct a different data structure, or we could just write our own custom serializer. We want to have our custom class to allow us to use Gson's type inference for doing so, plus the code is just easier to understand. I will show you how to serialize the data structure, and I'll leave the deserialization as an exercise for you.
We create a TypeAdapter<Results>:
public class ResultsAdapter extends TypeAdapter<Results> {
public Results read(JsonReader reader) throws IOException {
if (reader.peek() == JsonToken.NULL) {
reader.nextNull();
return null;
}
// exercise for you
return results;
}
public void write(JsonWriter writer, Results value) throws IOException {
if (value == null) {
writer.nullValue();
return;
}
writer.beginObject();
writer.name("totalSuggestions").value(value.mkr);
writer.name("routes");
writer.beginObject();
for(int i = 0; i < value.mkr; i++) {
writer.name("rank_"+i);
writer.beginObject();
String[] sourceDestSplit = result[i][0].split("-");
writer.name("Source").value(sourceDestSplit[0]);
writer.name("Destination").value(sourceDestSplit[1]);
writer.name("Weight").value(result[i][1]);
writer.endObject();
}
writer.endObject();
writer.endObject();
}
}
You can then call this method by doing (note: should only create the Gson object once, but I did it this way to keep the code short):
public String convertToJson(Results results) {
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(new ResultsAdapter()):
Gson gson = builder.build();
return gson.toJson(results);
}
This will work you the way you've asked, but I strongly recommend using JSON's array syntax instead (using []). Try this instead:
public void write(JsonWriter writer, Results value) throws IOException {
if (value == null) {
writer.nullValue();
return;
}
writer.beginObject();
writer.name("totalSuggestions").value(value.mkr);
writer.name("routes");
writer.beginArray();
for(int i = 0; i < value.mkr; i++) {
writer.beginObject();
String[] sourceDestSplit = result[i][0].split("-");
writer.name("Source").value(sourceDestSplit[0]);
writer.name("Destination").value(sourceDestSplit[1]);
writer.name("Weight").value(result[i][1]);
writer.endObject();
}
writer.endArray();
writer.endObject();
}
Doing it this will will result in JSON that looks like this, which will be easier to deserialize on the other side and iterate through, because you won't have to dynamically generate maps for the keys.:
{
"totalSuggestions": 6,
"routes": [
{
"Source": "ABC",
"Weight": "0.719010390625",
"Destination": "XYZ"
},
{
"Source": "XYZ",
"Weight": "0.7411458281249999",
"Destination": "ABC"
},
{
"Source": "LMN",
"Weight": "0.994583325",
"Destination": "PQR"
}
]
}

I landed here while searching for a similar solution for the com.google.gson.JsonObject library. Now, I've found it:
JsonObject mainJson = new JsonObject();
JsonObject innerJson = new JsonObject();
innerJson.addProperty("#iot.id", "31");
mainJson.add("Datastream", innerJson); // <-- here the nesting happens
mainJson.addProperty("result", 12.3);
// fetch inner variable like this
System.out.println(mainJson.get("Datastream").getAsJsonObject().get("#iot.id").getAsString());
This works fine for me using the com.google.gson.JsonObject library.

For the record, this is what i did.
import java.util.*;
public class DataObject {
public int Suggestions;
HashMap<String, route> routes = new HashMap<>();
//constructor
public DataObject(int mkr, String[][] routesArr){
Suggestions = mkr;
{
for (int i=0;i<Suggestions;i++){
routes.put("rank_"+(i+1),new route(routesArr[i]));
}
}
}
//class to populate the hashmap
public class route{
public String Origin;
public String Destination;
public String Weight;
public route(String arr[]){
String[] splitter = arr[0].split("-");
this.Origin = splitter[0];
this.Destination = splitter[1];
this.Weight = arr[1];
}
}
}

Nested JSON Request with Volley -> Escaped chars

I want to make a post request with volley to a REST API.
Therefore, I create a JSONObject and put a JSON String generated from a class in it.
JSONObject jsonObject = new JSONObject();
String json = gson.toJson(MyClazz);
try {
jsonObject.put(PARAM, json);
}
catch (JSONException e1) {
e1.printStackTrace();
}
The problem is that the correct calculated JSON String gets escaped and can't be recognized on the back end.
So toJson() gives something like:
{
"device_identifier":"324234234",
"name":"NameMe",
"list":[
{"prop":"A","prop2":-10},
{"prop":"B","prop2":-12}
]
}
The jsonObject's output is like
{
"PARAM":{
\"device_identifier\":\"324234234\",
\"name\":\"NameMe\",
\"list\":[
{\"prop\":\"A\",\"prop2\":-10},
{\"prop\":\"B\","\prop2\":-12}
]
}
}
I need the PARAM for the JSON structure so I can't give it directly to the REST-API. Any ideas how I can avoid the additional escaping?

You could wrap your MyClazz object with a simple wrapper object, and then pass that wrapped object to Gson's toJson method.
Given this class based on your example JSON,
public class MyClazz {
public String device_identifier;
public String name;
public List<Prop> list;
public class Prop {
public String prop;
public Integer prop2;
}
}
here's a possible wrapper implementation. Note the use of com.google.gson.annotations.SerializedName which tells Gson to use the PARAM key in the JSON representation.
public class MyClazzWrapper {
public MyClazzWrapper(MyClazz myClazz) {
this.myClazz = myClazz;
}
#SerializedName("PARAM")
private MyClazz myClazz;
}
And here's an example using it:
Gson gson = new GsonBuilder().setPrettyPrinting().create();
MyClazz myClazz = gson.fromJson("{\"device_identifier\":\"324234234\",\"name\":\"NameMe\",\"list\":[{\"prop\":\"A\",\"prop2\":-10},{\"prop\":\"B\",\"prop2\":-12}]}", MyClazz.class);
MyClazzWrapper wrapped = new MyClazzWrapper(myClazz);
System.out.println(gson.toJson(wrapped));
The above will print:
{
"PARAM": {
"device_identifier": "324234234",
"name": "NameMe",
"list": [
{
"prop": "A",
"prop2": -10
},
{
"prop": "B",
"prop2": -12
}
]
}
}