I am learning data persistence and this is my first attempt at JSON. I have read a few guides and from what little I can tell the code has been correct in both attempts at storing the objects. I get the file written using Gson but Gson throws exceptions when attempted to parse the objects using the fromJson() method. My question is as follows:
If I am using the same type to convert to/from JSON what am I missing that would tell Gson how to properly parse my object(s)?
I have tried three different approaches, two of which are included below. First I tried storing the wrapper class for the list of objects which a guide suggested I should be able to do:
public class JSONConverter {
private static Path path = Paths.get("src\\json\\JSONList.json");
private static Type stockType = new TypeToken<StocksList>(){}.getType();
public static void convertToJSON(StocksList stocks, Path path) {
Gson json = new Gson();
String storedStocks = json.toJson(stocks, stockType);// I also tried "StocksList.class" here
checkForFile(path);
try (BufferedWriter writer = Files.newBufferedWriter(path)) {
writer.write(storedStocks);
} catch (IOException e) {
e.printStackTrace();
//handle later
}
}
static void checkForFile(Path path) {
if (Files.notExists(path)) {
try {
Files.createFile(path);
} catch (IOException e) {
e.printStackTrace();
//handle later
}
}
}
public static StocksList convertFromJSON(Path path) {
StocksList stocksList = new StocksList();
Gson json = new Gson();
String fromJson;
try {
fromJson = Files.readAllBytes(path).toString();
stocksList = json.fromJson(fromJson, stockType);
return stocksList;
} catch (IOException e) {
return stocksList;
}
}
}
My second approach was to get the list out of the wrapper class and try to convert that to JSON:
public class JSONConverter {
private static Path path = Paths.get("src\\json\\JSONList.json");
private static Type listType = new TypeToken<List<Stock>>(){}.getType();
public static void convertToJSON(StocksList stocks, Path path) {
Gson json = new Gson();
List<Stock> temp = stocks.getStocks();
String storedStocks = json.toJson(temp, listType);
checkForFile(path);
try (BufferedWriter writer = Files.newBufferedWriter(path)) {
writer.write(storedStocks);
} catch (IOException e) {
e.printStackTrace();
//handle later
}
}
static void checkForFile(Path path) {
if (Files.notExists(path)) {
try {
Files.createFile(path);
} catch (IOException e) {
e.printStackTrace();
//handle later
}
}
}
public static StocksList convertFromJSON(Path path) {
StocksList stocksList = new StocksList();
List<Stock> stocks = new ArrayList<>();
Gson json = new Gson();
String fromJson;
try {
fromJson = Files.readAllBytes(path).toString();
stocks = json.fromJson(fromJson, listType);
//wraps the list in the stockslist class
stocksList.setStocks(stocks);
return stocksList;
} catch (IOException e) {
e.printStackTrace();
return stocksList;
}
}
}
Here is a sample of the JSON written by the first method using the second approach. The first looks like it except adds "{ "stocks" :" (what you see below) "}":
[
{
"ticker": "INTC",
"currentPrice": "45.94",
"marginOfSafety": 0.25,
"lastDate": "2019-12-28",
"cashYield": "7.4",
"MCap": "196485365760",
"enterpriseValue": "281213850000",
"sharesOut": "4417000000",
"oddPercentGrowth": false,
"newCompany": false,
"safeValue": "51.35",
"fairValue": "68.47",
"evEbitda": "8.56",
"fcf": [
"16932000000",
"14611750000"
],
"rOnAssets": "21",
"rOnCapital": "20",
"croic": "16.47",
"equityToDebt": "3.0",
"cashOnHand": "4194000000",
"cashToDebt": "0.17",
"changeInDebt": "210000000",
"capEfficiency": [
"18",
"7",
"-26",
"-21",
"1"
],
"fcfChange": [
"18.81",
"11.71"
],
"profitMargin": [
"46",
"38"
]
},
{
"ticker": "HCC",
"currentPrice": "12.99",
"marginOfSafety": 0.5,
"lastDate": "2018-12-31",
"cashYield": "46.1",
"MCap": "664587904",
"enterpriseValue": "1572623480",
"sharesOut": "52812000",
"oddPercentGrowth": true,
"newCompany": true,
"safeValue": "236.94",
"fairValue": "473.87",
"evEbitda": "2.59",
"fcf": [
"457776000",
"306126750"
],
"rOnAssets": "49",
"rOnCapital": "59",
"croic": "38.77",
"equityToDebt": "1.0",
"cashOnHand": "205577000",
"cashToDebt": "0.44",
"changeInDebt": "125283000",
"capEfficiency": [
"292",
"798",
"-365",
"-397",
"-1"
],
"fcfChange": [
"33.9",
"33.9"
],
"profitMargin": [
"40",
"8"
]
}
]
Both throw:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException:
Expected BEGIN_OBJECT but was STRING at line 1 column 12
(this line changes to "Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 2" when using the first approach).
at
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
...
I was going to try adding each object individually to a JSONArray but when I started encountering trouble there I thought I should just ask. The guide mentioned that reflection was important and I am guessing that my trouble lies therein due to the second line of the stack trace but again, this is my first time trying to use JSON. If I forgot to include anything let me know and I'll post it in a comment.
Thanks for the help.
ADDENDUM: the objects only throw these exceptions when being written to and pulled from a file. They do not throw when being converted to a JSON String and then back again. It occurs whether I use Files.write() or Files.newBufferedWriter().
Thanks to all those who viewed my question. I reached out to Gson's github page and they responded with the following corrections to my class:
All the code you've provided can be greatly fixed, improved and refactored.
No need to create multiple Gson instances: they are relatively
expensive to instantiate, but are designed to be thread-safe and
immutable therefore can be reused. No need to serialize to and
deserialize from java.lang.String -- this is just expensive as long as
it has to create multiple strings in the heap merely wasting the heap
and time decreasing the performance. Why it does not work in your case
is that Files.readAllBytes(...) returns byte[] you're trying to
convert to a string. In Java, no arrays have an intuitive toString
implementation (you can check it by simply printing any byte array to
System.out). In order to convert it to a string (that might be a
memory-consuming instance), new String(byte[]) (or even new
String(byte[], Charset)) is an appropriate way. I don't really
remember how Files works, but there's probably no need to check the
file to exist: they can be overwritten without any additional checks.
No type tokens are necessary in this case: StockList.class is a Type
too. Essentially, all is you need is just as follows:
private static final Gson gson = new GsonBuilder()
.disableHtmlEscaping()
.disableInnerClassSerialization()
.create();
public static void main(final String... args)
throws IOException {
final StocksList before = new StocksList(ImmutableList.of(new Stock("INTC"), new
Stock("HCC")));
final Path path = Paths.get("doc.json");
write(path, before);
final StocksList after = read(path);
System.out.println(after.equals(before));
}
private static void write(final Path path, final StocksList stocks)
throws IOException {
try ( final Writer writer = new OutputStreamWriter(new
FileOutputStream(path.toFile())) ) {
gson.toJson(stocks, writer);
}
}
private static StocksList read(final Path path)
throws IOException {
try ( final Reader reader = new InputStreamReader(new
FileInputStream(path.toFile())) ) {
return gson.fromJson(reader, StocksList.class);
}
}
Thanks to lyubomyr-shaydariv (Gson contributor) for the answer.
Related
My test class calls the addEachEmployeeDetailsToJSONFile method multiple times during the execution with different employee details.
The expected end result is a JSON file named mapping.json that looks like below:
EXPECTED mapping.json
[
{
"employee": {
"ID": "123",
"Name": "Gupta",
"Department": "Accounts"
}
},
{
"employee": {
"ID": "456",
"Name": "Mark",
"Department": "Marketing"
}
}
]
Below is my Java class.
import org.json.simple.JSONObject;
import org.json.simple.JSONArray;
import java.io.*;
public class Test {
private String filePath = new File("").getAbsolutePath() + File.separator + "target" + File.separator + "mapping.json";
protected JSONObject employeeDetails = new JSONObject();
protected JSONObject employeeObject = new JSONObject();
protected JSONArray employeeList = new JSONArray();
private FileWriter fileWriter;
{
try {
fileWriter = new FileWriter(getMappingFile(), true);
} catch (IOException e) {
e.printStackTrace();
}
}
//Write to JSON file
private File getMappingFile (){
File mappingFile = new File(getMappingFilePath);
mappingFile.setWritable(true);
mappingFile.setReadable(true);
return mappingFile;
}
public void addEachEmployeeDetailsToJSONFile(ITestContext iTestContext) {
try {
//Write to first JSONObject
employeeDetails.put("ID", iTestContext.getId());
employeeDetails.put("Name", iTestContext.getName());
employeeDetails.put("Department", iTestContext.getDepartment());
//Put the above JSON Object in another JSON object.
employeeObject.put("employees", employeeDetails);
//Write above object to JSONArray
employeeList.add(employeeObject);
//We can write any JSONArray or JSONObject instance to the file
fileWriter.append(beforeTestClassesList.toJSONString());
}
catch (Exception e)
{}
}
public static void main( String[] args )
{
addEachEmployeeDetailsToJSONFile(<pass employee Gupta instance>);
addEachEmployeeDetailsToJSONFile(<pass employee Mark instance>);
fileWriter.flush();
}
}
The JSON file is being created successfully and an entry is also being created like below, but, only the first entry of one employee details is being created and not a list of multiple employees and their details. What am I missing?
Current mapping.json file:
[
{
"employee": {
"ID": "123",
"Name": "Gupta",
"Department": "Accounting"
}
}
]
Your code as shown only adds 1 item. So this is as expected. Also, you would create a new JSON object for each employee, not reuse the same one repeatedly. That will cause problems on subsequent saves. Further, your code is really a bit poorly architected. Due to the nature of JSON, appending to a file isn't really done. You write the entire file at once. This means you don't write until you have all the data, you don't write after adding each item to the array. The reason for this is that there's data after the element in the array to make it valid JSON, and you can't stick stuff in the middle of a file. Appending to a JSONArray in memory is fine, but when writing to disk you want to write the entire file.
I'm trying to build a Springboot app that allows to insert Json object from Postman and save it to existing json file that has other data. I'm new to Jackson so perhaps I missed something?
this is how my json file looks:
[
{
"Name":"After Dark",
"Author":"Haruki Murakami"
},
{
"Name":"It",
"Author":"Stephen King"
}
]
This is what I have tried:
#PostMapping("/insertBook")
public void insertBook(#RequestBody Book book) {
File booksJsonFile = Paths.get(this.getClass().getResource("/books.json").toURI()).toFile();
objectMapper.writeValue(booksJsonFile, book);
}
It's inserts to an empty file but it's doesn't append to existing json file.
I also have tried this:
#PostMapping("/insertBook")
public void insertBook(#RequestBody Book book) throws URISyntaxException {
try {
File file = Paths.get(this.getClass().getResource("/books.json").toURI()).toFile();
FileWriter fileWriter = new FileWriter(file, true);
SequenceWriter seqWriter = objectMapper.writer().writeValuesAsArray(fileWriter);
seqWriter.write(book);
seqWriter.close();
} catch (IOException e) {
e.printStackTrace();
}
}
This is what I'm sending from Postman:
Do I need to use something else to achieve the result that I want?
I will be thankfull for your help.
I have tried to reproduce your problem according to your code and I come to following conclusions:
You can not modify file under resources directly. Here is explanation why.
I managed to append new JSON to the file (using your approach but saving file locally) but it's probably not what you expect (json structure is corrupted):
[
{
"Name":"After Dark",
"Author":"Haruki Murakami"
},
{
"Name":"It",
"Author":"Stephen King"
}
][{"Name":"new name","Author":"new author"}]
I am afraid that it is not possible to update current JSON structure directly in the file.
I managed to solve your problem using org.json library. However, the disadvantage of my solution is necessity of rewriting entire file each time. In addition I used synchronized keyword in order to avoid simultaneous file modification.
public synchronized void updateJsonFile(Book book) throws IOException {
ObjectMapper objectMapper = new ObjectMapper();
Path path = Paths.get("./books.json");
final String currentJsonArrayAsString = Files.readString(path);
try (FileWriter fileWriter = new FileWriter(path.toFile(), false)) {
JSONObject jsonObject = new JSONObject(objectMapper.writeValueAsString(book));
JSONArray jsonArray = new JSONArray(currentJsonArrayAsString);
jsonArray.put(jsonObject);
fileWriter.write(jsonArray.toString());
}
}
And now the books.json has following content:
[
{
"Author":"Haruki Murakami",
"Name":"After Dark"
},
{
"Author":"Stephen King",
"Name":"It"
},
{
"Author":"new author",
"Name":"new name"
}
]
I create a java URL class which contain my Json data and have some function to obtain back my json data for doing some data comparison, I found out it's might not support by JSONObject for passing the data into the JSONObject. Do I need to use JSONArray in my case because my JSON data have array structure as well?
try
{
JSONObject obj = new JSONObject ();
obj.readJsonFromUrl(theUrl);
System.out.println(obj.toString());
}
catch(MalformedURLException e)
{
System.out.print("your problem here ...1");
}
}
else
{
System.out.print("Can't Connect");
}
I am sure that this is the place give me the error message because it return me this error in my compiler
Exception in thread "main" java.lang.Error: Unresolved compilation problem:
The method readJsonFromUrl(URL) is undefined for the type JSONObject
there are also some warning message for that the JSONObject readJsonFromUrl method
private static JSONObject readJsonFromUrl(URL theUrl) throws IOException, JSONException {
Anyone can provide me the explaination of how the JSON data work in java? I saw quite number of Java class for JSON which make me confuse for it such as JSONObject, JSONArray , JSONValue. I search some information online but I also not very clear about it since I am very new to JSON data processing This is my sample json data and the data I need is scan_result only
{
"data_id":"a71a3c2588c6472bb4daea41a0b58835",
"file_info":{
"display_name":"",
"file_size":242,
"file_type":"Not available",
"file_type_description":"Not available",
"md5":"aa69ba384f22d0dc0551ace2fbb9ad55",
"sha1":"09ceb54e65df3d3086b222e8643acffe451a6e8a",
"sha256":"dcb46d6ae2a187f789c12f19c44bbe4b9a43bd200a3b306d5e9c1fcf811dc430",
"upload_timestamp":"2016-11-18T09:09:08.390Z"
},
"process_info":{
"blocked_reason":"",
"file_type_skipped_scan":false,
"post_processing":{
"actions_failed":"",
"actions_ran":"",
"converted_destination":"",
"converted_to":"",
"copy_move_destination":""
},
"profile":"File scan",
"progress_percentage":100,
"result":"Allowed",
"user_agent":""
},
"scan_results":{
"data_id":"a71a3c2588c6472bb4daea41a0b58835",
"progress_percentage":100,
"scan_all_result_a":"No Threat Detected",
"scan_all_result_i":0,
"scan_details":{
"Ahnlab":{
"def_time":"2016-11-08T15:00:00.000Z",
"location":"local",
"scan_result_i":0,
"scan_time":1,
"threat_found":""
},
"Avira":{
"def_time":"2016-11-08T00:00:00.000Z",
"location":"local",
"scan_result_i":0,
"scan_time":133,
"threat_found":""
},
"ClamAV":{
"def_time":"2016-11-08T10:28:00.000Z",
"location":"local",
"scan_result_i":0,
"scan_time":94,
"threat_found":""
},
"ESET":{
"def_time":"2016-11-08T00:00:00.000Z",
"location":"local",
"scan_result_i":0,
"scan_time":38,
"threat_found":""
}
},
"start_time":"2016-11-18T09:09:08.405Z",
"total_avs":4,
"total_time":250
},
"vulnerability_info":{
}
}
As mentioned here, there are many ways to solve this. Either you have to implement the read, parse operations yourself (#Roland Illig 's answer)
//you have to implement the readJSON method
InputStream is = new URL(url).openStream();
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));
String jsonText = readAll(rd);
JSONObject json = new JSONObject(jsonText);
return json;
} finally {
is.close();
}
Or you could use a library. The most well-known and widely used libraries are jackson and gson.
The big picture is that you try to "map" your json Object to a class.
You have your json file:
{
"id":1,
"name":"eirini",
"hobbies":["music","philosophy","football"]
}
and a class that represents this file and will store the values (depending on the library that you use there might be different requirements, for example getters, setters etc..)
public class Person {
public int id;
public String name;
public List<String> hobbies = new ArrayList<String>();
public String toString() {
return name +" has the id: " + id + " the following hobbies" + hobbies.get(0) + " " + hobbies.get(2);
}
}
Finally in your main method:
public static void main(String[] args) throws IOException, ParseException {
ObjectMapper mapper = new ObjectMapper();
InputStream input = this.getClass().getResourceAsStream(FILE); //read your file. There are many ways to achieve this.
ObjectMapper mapper = new ObjectMapper(); // just need one
Person eirini = mapper.readValue(input, Person.class);
System.out.println(eirini.toString());
You cannot pass json in url, you can pass it in body. Writing Json to stream body and post it using regular java method.
Here is oracle community url of explanation of your problem.
Required Jar can be downloaded from here.
Test Code Follows:
URL url = new URL("https://graph.facebook.com/search?q=java&type=post");
try (InputStream is = url.openStream();
JsonReader rdr = Json.createReader(is)) {
JsonObject obj = rdr.readObject();
JsonArray results = obj.getJsonArray("data");
for (JsonObject result : results.getValuesAs(JsonObject.class)){
System.out.print(result.getJsonObject("from").getString("name"));
System.out.print(": ");
System.out.println(result.getString("message", ""));
System.out.println("-----------");
}
}
I want to make a post request with volley to a REST API.
Therefore, I create a JSONObject and put a JSON String generated from a class in it.
JSONObject jsonObject = new JSONObject();
String json = gson.toJson(MyClazz);
try {
jsonObject.put(PARAM, json);
}
catch (JSONException e1) {
e1.printStackTrace();
}
The problem is that the correct calculated JSON String gets escaped and can't be recognized on the back end.
So toJson() gives something like:
{
"device_identifier":"324234234",
"name":"NameMe",
"list":[
{"prop":"A","prop2":-10},
{"prop":"B","prop2":-12}
]
}
The jsonObject's output is like
{
"PARAM":{
\"device_identifier\":\"324234234\",
\"name\":\"NameMe\",
\"list\":[
{\"prop\":\"A\",\"prop2\":-10},
{\"prop\":\"B\","\prop2\":-12}
]
}
}
I need the PARAM for the JSON structure so I can't give it directly to the REST-API. Any ideas how I can avoid the additional escaping?
You could wrap your MyClazz object with a simple wrapper object, and then pass that wrapped object to Gson's toJson method.
Given this class based on your example JSON,
public class MyClazz {
public String device_identifier;
public String name;
public List<Prop> list;
public class Prop {
public String prop;
public Integer prop2;
}
}
here's a possible wrapper implementation. Note the use of com.google.gson.annotations.SerializedName which tells Gson to use the PARAM key in the JSON representation.
public class MyClazzWrapper {
public MyClazzWrapper(MyClazz myClazz) {
this.myClazz = myClazz;
}
#SerializedName("PARAM")
private MyClazz myClazz;
}
And here's an example using it:
Gson gson = new GsonBuilder().setPrettyPrinting().create();
MyClazz myClazz = gson.fromJson("{\"device_identifier\":\"324234234\",\"name\":\"NameMe\",\"list\":[{\"prop\":\"A\",\"prop2\":-10},{\"prop\":\"B\",\"prop2\":-12}]}", MyClazz.class);
MyClazzWrapper wrapped = new MyClazzWrapper(myClazz);
System.out.println(gson.toJson(wrapped));
The above will print:
{
"PARAM": {
"device_identifier": "324234234",
"name": "NameMe",
"list": [
{
"prop": "A",
"prop2": -10
},
{
"prop": "B",
"prop2": -12
}
]
}
}
I am writing a relatively simple messaging app that saves its logs in the JSON format, and I am using the GSON library to parse these. I load a JSON file from a server, and put it trough Gson.toJsonTree() function. I'm not sure this is expected, but when I test the result from the previous function with the isJsonSomething() functions (isJsonObject,isJsonAray,isJsonNull,isJsonPrimitive), isJsonPrimitive returns true, and I can't parse it into a object. This is my JSON file's contents:
{
"users": [
{
"picture": "",
"type": "user",
"name": "kroltan"
}
],
"description": "No description",
"messages": [
{
"content": "something",
"time": "2013-08-30 00:38:17.212000",
"type": "message",
"author": "someone"
}
],
"type": "channel",
"name": "default"
}
And here is the class used to parse it into POJOs: (CLEANUP comments is where I've removed irrelevant code from the post)
package com.example.testapp;
//CLEANUP: All needed imports
import com.example.testapp.data.*;
import com.google.gson.*;
public class JSONConverter {
public interface JsonTypeLoadedListener {
public void onSucess(JSONType jsonType);
public void onFailure(Exception e);
}
public static final String DATE_FORMAT = "dd-MM-yyyy HH:mm:ss.SSS";
public static final HashMap<String, Class<?>> JSON_TYPES = new HashMap<String, Class<?>>();
public JSONConverter() {
JSON_TYPES.clear();
JSON_TYPES.put("channel", Channel.class);
JSON_TYPES.put("user", User.class);
JSON_TYPES.put("message", Message.class);
}
public void loadFromURL(final URL url, final JsonTypeLoadedListener listener) {
new Thread(new Runnable() {
#Override
public void run() {
JsonObject result = null;
Gson gson = new GsonBuilder().setDateFormat(DATE_FORMAT).create();
if (url.getProtocol().equals("http")) {
try {
String content = //Loads from a server, omitted for clarity
result = gson.toJsonTree(content).getAsJsonObject();
conn.disconnect();
} catch (Exception e) {
e.printStackTrace();
listener.onFailure(e);
return;
}
} else if (url.getProtocol().equals("file")) {
try {
String content = //Loads from a file, omitted for clarity
result = gson.toJsonTree(content).getAsJsonObject();
br.close();
} catch (Exception e) {
e.printStackTrace();
listener.onFailure(e);
return;
}
}
listener.onSucess((JSONType) gson.fromJson(result, JSON_TYPES.get(result.get("type").getAsString())));
}
}, "URLLoader").start();
}
public JSONType loadFromString(String s) {
Gson gson = new Gson();
JsonObject result = gson.toJsonTree(s).getAsJsonObject();
return (JSONType) gson.fromJson(result, JSON_TYPES.get(result.get("type").getAsString()));
}
}
The classes Message, User and Channel all inherit from JSONType (a custom class with a field called type and some utility methods) and contain all values present in the above mentioned JSON file.
When it reaches gson.toJsonTree(content).getAsJsonObject(), I get this error in Logcat (string omitted for clarity, it's just the full file):
java.lang.IllegalStateException: Not a JSON Object: "String containing all the file with tabs represented as \t"
I'm guessing that the tabs are causing your issue. Try to remove them with:
content = content.replaceAll("\\s","")
this will simply clean your json string from any whitespace.
Btw I suggests you to get rid of Gson library and use directly the JSONObject provided in the android sdk. You can initialize it directly with the json string, as new JSONObject(content). :)