We Keep Coding

When streaming a Json file with GSON's JsonReader, can you dump an object to string?

I am working in Java using the GSON library to process a huge JSON document.
I absolutely cannot load this into memory, it is upwards of 8G and will crash the JVM with an OOM if I try to.
So I use JsonReader to process the stream of JSON.
There are certain times when I reach a BEGIN_OBJECT that want to just dump the contents of the object to string instead of processing each individual element.
For example, I am parsing something like this:
[ { "SchemaVersion":"15.0.0.0","LibraryVersion":"16.0.7324.1200","ErrorInfo":{ "ErrorMessage":"Access denied. You do not have permission to perform this action or access this resource.","ErrorValue":null,"TraceCorrelationId":"03ab459e-7076-5000-c1a7-196f1fc54384","ErrorCode":-2147024891,"ErrorTypeName":"System.UnauthorizedAccessException" },"TraceCorrelationId":"03ab459e-7076-5000-c1a7-196f1fc54384" } ]
When I get to ErrorInfo, I just want that as a string instead of parsed out. I don't want to have to parse each piece.
So here is the code:
try (InputStream is = (InputStream)response.getEntity();
InputStreamReader inputStreamReader = new InputStreamReader(is);
JsonReader jsonReader = new JsonReader(inputStreamReader)) {
if (jsonReader.peek() != JsonToken.BEGIN_ARRAY) {
return;
}
jsonReader.beginArray();
while (jsonReader.hasNext()) {
jsonReader.beginObject(); // Start of the default object on every request
jsonReader.skipValue(); // name SchemaVersion
jsonReader.skipValue(); // value
jsonReader.skipValue(); // name LibraryVersion
jsonReader.skipValue(); // value
jsonReader.skipValue(); // name ErrorInfo
if (jsonReader.peek() == JsonToken.BEGIN_OBJECT) {
// I want to dump the upcoming object to string here
...
Is it possible to dump the contents of the json reader for a given begin object until it ends to string?
Similar question, but not quite the same: Get a dump of a section (object) of JSON

Special thanks to https://stackoverflow.com/users/3389828/nikhil for the answer in the comments. You can use Gson.fromJson(jsonReader, Map.class) for this exact situation.
Gson gson = new Gson();
try (InputStream is = (InputStream)response.getEntity();
InputStreamReader inputStreamReader = new InputStreamReader(is);
JsonReader jsonReader = new JsonReader(inputStreamReader)) {
if (jsonReader.peek() != JsonToken.BEGIN_ARRAY) {
return;
}
jsonReader.beginArray();
while (jsonReader.hasNext()) {
Map header = gson.fromJson(jsonReader, Map.class);
if (header.get("ErrorInfo") != null) {
String errorDump = ((Map)header.get("ErrorInfo")).toString();
// Now I have the error dump I need but I also
// didn't load the entire thing into memory!
...
more info: https://javadeveloperzone.com/java-8/java-parse-large-json-file-gson-example/

Appending jsonobject to an existing jsonobject

I'm having a problem with Json file reading and writing. I want to append something into a json file but it doesn't work properly: it just put in a new jsonobject without the ',' to divide it from the previous one. I searched everywhere, on every site, but nothing that gave me an input on how to do it properly.
For example, I have a json file like this:
{
"Example":{
"Ok":"Ok1",
"Nice":"Nice1",
"Hi":"Hi1",
"Hello":"Hello1",
"Right":"Right1",
"Wow":"Wow1"
}
}
And I want to make it appear like this:
{
"Example":{
"Ok":"Ok1",
"Nice":"Nice1",
"Hi":"Hi1",
"Hello":"Hello1",
"Right":"Right1",
"Wow":"Wow1"
},
"Example1":{
"Ok":"Ok2",
"Nice":"Nice2",
"Hi":"Hi2",
"Hello":"Hello2",
"Right":"Right2",
"Wow":"Wow2"
}
}
So, I tried using this code:
Gson gson = new GsonBuilder().setPrettyPrinting().create();
JsonObject jsonObject = new JsonObject();
JsonObject dati = new JsonObject();
dati.addProperty("Cognome", StringUtils.capitalize((fields[0].getText())));
dati.addProperty("Nome", StringUtils.capitalize((fields[1].getText())));
dati.addProperty("Sesso", lblSesso.getText());
dati.addProperty("Luogo di nascita", StringUtils.capitalize((fields[2].getText())));
dati.addProperty("Provincia", lblProvincia.getText());
dati.addProperty("Data di nascita", fieldDDN.getText());
jsonObject.add(codfis, dati);
String json = gson.toJson(jsonObject);
try (BufferedReader br = new BufferedReader(new FileReader("CodFisCalcolati.json"));
BufferedWriter bw = new BufferedWriter(new FileWriter("CodFisCalcolati.json", true))) {
String jsonString = gson.fromJson(br, JsonElement.class).toString();
JsonElement jelement = new JsonParser().parse(jsonString);
JsonObject jobject = jelement.getAsJsonObject();
jobject.add(codfis, dati);
String resultingJson = gson.toJson(jelement);
bw.write(resultingJson);
bw.close();
} catch (IOException e1) { e1.printStackTrace(); }
But when I use it, it give me this output :
{
"Example":{
"Ok":"Ok1",
"Nice":"Nice1",
"Hi":"Hi1",
"Hello":"Hello1",
"Right":"Right1",
"Wow":"Wow1"
}
}{
"Example":{
"Ok":"Ok1",
"Nice":"Nice1",
"Hi":"Hi1",
"Hello":"Hello1",
"Right":"Right1",
"Wow":"Wow1"
},
"Example1":{
"Ok":"Ok2",
"Nice":"Nice2",
"Hi":"Hi2",
"Hello":"Hello2",
"Right":"Right2",
"Wow":"Wow2"
}
}
That's output, you see, it'wrong and i don't know how to make the code to give me a different output.
I'm using Gson 2.8.5 and I would rather not change to another library.

You change the question but now the answer to your new question is you use the same file to read and write. That's why you add the data inside ot the file. Change the name of the file that you write and see if you have problems

Please check if "br" is not null.
According to the specification of the method fromJson it returns:
an object of type T from the string. Returns null if json is null.
If this is the case than you call on the null toString() method and you get null pointer exception

Unexpected token END OF FILE at position 0 while parsing JSON

In order to find out weather the JSON element is JSONArray or JSONObject type, I am getting Unexpected token END OF FILE at position 0 error.
My JSON is:
{"colors":[{"color":"red","value":"#00"},{"color":"white","value":"#000g"}]}
My code is:
java.io.FileReader reader = new java.io.FileReader("jsonpath");
org.json.simple.parser.JSONParser parser = new JSONParser();
System.Out.Println("aaaaaa JSON Class: "+parser.parse(reader).getClass());
if(parser.parse(reader) instanceof org.json.simple.JSONArray)
System.Out.Println("JSONArray");
else if(parser.parse(reader) instanceof org.json.simple.JSONObject)
System.Out.Println("JSONObject");
When I run above code it shows this output
aaaaaa JSON Class: class org.json.simple.JSONObject Unexpected token END OF FILE at popsition 0
at org.json.simple.parser.JSONParser(Unknown Source)
.
.
.
<rest of the exception>
I don't understand why this exception is occurring.
Please help me out.
Some more details after edit:
My this code is working fine with the given json file:
java.io.FileReader reader = new java.io.FileReader("jsonpath");
org.json.simple.parser.JSONParser parser = new JSONParser();
org.json.simple.JSONObject object = (JSONObject)parser.parse(reader);
System.Out.Println("JSONObject: "+object);
org.json.simple.JSONArray array = (JSONArray)object.get("colors");
System.Out.Println("JSONArray: "+array);
Output of above code:
JSONObject: {"colors":[{"color":"red","value":"#00"},{"color":"white","value":"#000g"}]}
JSONArray: [{"color":"red","value":"#00"},{"color":"white","value":"#000g"}]
But I want to dynamically parse the JSON without knowing the JSON structure.
I want to do something like this:
if(json is object)
JSONObject object = (JSONObject)parser.parse(reader);
else if (json is array)
JSONArray array = (JSONArray)parser.parse(reader);
Thanks.

You're repeatedly parsing the same Reader. The first call exhausts it and then each subsequent call sees an empty stream.

Parse the reader only once. Here is the working code:
java.io.FileReader reader = new java.io.FileReader("jsonpath");
org.json.simple.parser.JSONParser parser = new JSONParser();
Object p = parser.parse(reader);
if(p instanceof org.json.simple.JSONArray){
System.Out.Println("JSONArray");
org.json.simple.JSONArray object = (JSONArray)p;
}
else if(p instanceof org.json.simple.JSONObject){
System.Out.Println("JSONObject");
org.json.simple.JSONObject object = (JSONObject)p;
}
Output of above code
JSONObject

Well, error may occur when you try to pass wrong path.
So check your path to json file properly. Try to use absolute path at first.
Here is my procedure:
private static void ReadWithEncoding(String filePath, String encoding) {
StringBuilder json = new StringBuilder();
File f = new File(filePath);
if (f.exists() && f.isFile()) {
try {
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(f), encoding));
String line;
while ((line = br.readLine()) != null) {
json.append(line);
}
br.close();
} catch (Exception e) {
e.printStackTrace();
}
System.out.println(json);
}
}
You may run it like this for UTF8:
ReadWithEncoding("D:/file.json", "UTF8");
For Cyrillic symbols:
ReadWithEncoding("D:/file.json", "Cp1251");

I too was getting this error ("Unexpected token END OF FILE at position 0.").
I was using same instance of com.google.gson.Gson and org.json.simple.parser.JSONParser on multiple threads.
Now I changed the code and created new instance of these on each thread, and that solved the issue.
gson = new GsonBuilder().create();
parser = new JSONParser();

Parse CSV file to a generic class - Java generics

I'm trying to build up a generic method that parses a CSV file into an Object.
I think I'm quite close to reach the objective but I'm a bit stuck with java generics, I'm still learning it.
Now I'm stuck on my while cycle where I create the objects. I'm using jCSV to do the parsing for me. I'm following their documentation tutorial here.
I can't figure out how to set the beanClass bc = it.next(); because beanClass does not exist as a class on my project, compilation error: cannot find symbol - class beanClass
How can I fix this?
I know I could simply do a List<?> beanClassList = csvFileReader.readAll();
but the problem is that on the first line of each CSV file I've the class name to where that data belongs to. I get this exception, which makes sense:
Exception in thread "AWT-EventQueue-0" java.lang.NumberFormatException: For input string: "Car"
My CSV files are something like this:
ClassName
value,value,value,value,value
value,value,value,value,value
...
Here's my code:
public String importFromCsvFile(File f) throws IOException, ClassNotFoundException, InstantiationException, IllegalAccessException
{
FileReader fr = new FileReader(f);
BufferedReader buffReader = new BufferedReader(fr);
String className = buffReader.readLine();
buffReader.close();
//Java reflection to get the Class Object.
Class beanClass = Class.forName("model." + className);
Object beanObject = beanClass.newInstance();
Reader reader = new FileReader(f);
ValueProcessorProvider provider = new ValueProcessorProvider();
CSVEntryParser<?> entryParser = new AnnotationEntryParser<>(beanClass, provider);
CSVReader<?> csvFileReader= new CSVReaderBuilder<>(reader).entryParser((CSVEntryParser<Object>) entryParser).build();
Iterator<?> it = csvFileReader.iterator();
while (it.hasNext()) {
beanClass bc = it.next(); // here is the compilation error
}
}
Here's a CSV file example:
Car
1,BMW,Z3,2000,20-AC-57
2,Mercedes,C4,2010,23-32-VJ
3,Alfa Romeo,A3,1992,XX-XX-XX

you are nearly there.
public String importFromCsvFile(File f) throws IOException, ClassNotFoundException, InstantiationException, IllegalAccessException
{
FileReader fr = new FileReader(f);
BufferedReader buffReader = new BufferedReader(fr);
String className = buffReader.readLine();
buffReader.close(); // you can also not close it and use buffReader as your reader for the CSV
//Java reflection to get the Class Object.
Class beanClass = Class.forName("model." + className);
Object beanObject = beanClass.newInstance(); // this is never needed
Reader reader = new FileReader(f); // use buffReader instead of creating a new one
ValueProcessorProvider provider = new ValueProcessorProvider();
CSVEntryParser<?> entryParser = new AnnotationEntryParser<>(beanClass, provider);
CSVReader<?> csvFileReader= new CSVReaderBuilder<>(reader).entryParser((CSVEntryParser<Object>) entryParser).build();
Iterator<?> it = csvFileReader.iterator();
while (it.hasNext()) {
Object obj = it.next(); // obj is an instance of Car with your data
boolean isCar = obj instanceof Car; // will be true
}
}
Because you are using , as separator you should consider using UK_DEFAULT as Strategy for the Reader or defining your own (the default separator is ;).
You should also either continue using the BufferedReader or specify skipHeader in the Strategy - else you Car will be treated as entry which is probably not what you want.

As per your comments Please have a look at sample code that reads data from CSV file and store in a map as key-value pair.
List<Map<String, String>> list = new ArrayList<Map<String, String>>();
BufferedReader reader = new BufferedReader(new FileReader(new File("resources/abc.csv")));
String header = reader.readLine();
String[] keys = header.split(",");
String line = null;
while ((line = reader.readLine()) != null) {
Map<String, String> map = new HashMap<String, String>();
String[] values = line.split(",");
for (int i = 0; i < keys.length; i++) {
map.put(keys[i], values[i]);
}
list.add(map);
}
reader.close();
for(Map<String, String> map:list){
for(String key:map.keySet()){
System.out.println(key+":"+map.get(key));
}
System.out.println();
}
CSV:
ID,NAME,MODEL,YEAR,NUMBER
1,BMW,Z3,2000,20-AC-57
2,Mercedes,C4,2010,23-32-VJ
3,Alfa Romeo,A3,1992,XX-XX-XX

With generics, types are specified at compile time (and checked by the compiler). In your case, types are only specified in the CSV file and thus unknown at compile time. So generics is not a tool you can use in this case.
What exactly would you want to accomplish with generics? What would you like the compiler to check?
What you can do is create an instance of the class (you will need a full name including package) using Class.forName(name).newInstance() and use reflection to set some properties. But at compile time you'll only know the result is an Object.

Modify json with GSON without using a POJO

I want to modify a json content without converting it into a POJO. I am using GSON Library.
Following are the use case:
String jsonString = "[{\"key1\":\"Hello\",\"key2\":\"World\"},{\"key1\":\"Nice\",\"key2\":\"Town\"}]";
JsonElement jsonElement = gson.fromJson(jsonString, JsonElement.class);
Is there any way where I can set value of key1 to some value (let say "Test") in each array, without converting things into POJO

Here's the shortest I came up with.
JsonElement je = new Gson().fromJson(jsonString, JsonElement.class);
JsonObject jo = je.getAsJsonObject();
jo.add("key", value);
Once you have the JsonObject, gson has many methods to manipulate it.

You can always get a different type than JsonElement, or use JsonElement.getAsJsonObject to cast to an Object (if possible).
String jsonString = "[{\"key1\":\"Hello\",\"key2\":\"World\"}, ...]";
JsonArray jsonArray = gson.fromJson(jsonString, JsonElement.class).getAsJsonArray();
JsonObject firstObject = jsonArray.get(i).getAsJsonObject();
firstObject.addProperty("key1", "Test");
I was wrong earlier; there seems to be no JsonArray adapter; you'll have to get a JsonElement and use the casting tool.

One approach would be to just convert the JSON to a java.util.Map, modify the Map, and go from there (which may mean serializing the Map back to JSON).
This approach meets my preference to work with the right API for the right job, minimizing the use of tools like Gson to just handle serialization/deserialization (which is what I understand it was designed for). That is, to not use the Gson API as a replacement data structure.

GSON has two separate APIs (that can be combined): one is used for serialization and deserialization, and the other for streaming. If you want to process streams of JSON without memory overhead or using dynamic structures (rather than static POJOs) you can do something like:
create a JsonWriter (in my example I use StringWriter);
create a JsonReader;
make a loop that consumes events from the reader and feeds them to the writer, possibly making changes, additions, omissions etc.
The loop will consist of a single switch statement that must have a case all the possible events (10 of them). Even the simplest example must have all of them, so the code below looks rather verbose. But it is very easy to extend and further extensions will not make it much longer.
An example that appends "test": 1 pair to each object looks something like:
public class Whatever {
static void streamandmodify(JsonReader reader, JsonWriter writer) throws IOException {
while (true) {
JsonToken token = reader.peek();
switch (token) {
// most cases are just consume the event
// and pass an identical one to the writer
case BEGIN_ARRAY:
reader.beginArray();
writer.beginArray();
break;
case END_ARRAY:
reader.endArray();
writer.endArray();
break;
case BEGIN_OBJECT:
reader.beginObject();
writer.beginObject();
// this is where the change happens:
writer.name("test");
writer.value(1);
break;
case END_OBJECT:
reader.endObject();
writer.endObject();
break;
case NAME:
String name = reader.nextName();
writer.name(name);
break;
case STRING:
String s = reader.nextString();
writer.value(s);
break;
case NUMBER:
String n = reader.nextString();
writer.value(new BigDecimal(n));
break;
case BOOLEAN:
boolean b = reader.nextBoolean();
writer.value(b);
break;
case NULL:
reader.nextNull();
writer.nullValue();
break;
case END_DOCUMENT:
return;
}
}
}
public static void main(String[] args) throws IOException {
// just for test:
JsonReader jr = new JsonReader(new StringReader("{\"a\":1, \"b\":{\"c\":[1,2,3,{},{}]}}"));
StringWriter sw = new StringWriter();
JsonWriter jw = new JsonWriter(sw);
streamandmodify(jr, jw);
System.out.println(sw.getBuffer().toString());
}
}

The jsonString is a plain, ordinary Java String; so you can modify it whatever you like using the standards String functions of Java and replace the substring key1 with Test1:
jsonString = "[{\"key1\":\"Test\",\"key2\":\"World\"},{\"key1\":\"Nice\",\"key2\":\"Town\"}]";
Of course, String in Java are immutable so converting it first to a StringBuilder will possibly give you a better performance in term of memory usage.

Modify json with GSON JsonArray Java 8
Example of how to use GSON to modify a value within a JSON
import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
public class ModifyJson {
public static void main(String[] args) {
String data = "[{\"ct_pk\":24,\"ct_name\":\"SISTEMA DE PRUEBAS\"},"
+ "{\"ct_pk\":1,\"ct_name\":\"CAPITAL FEDERAL\"}," +
"{\"ct_pk\":5,\"ct_name\":\"SISTEMA DE PRUEBAS DOS\"}]";
System.out.println("before................." + data);
JsonArray jsonArray = new Gson().fromJson(data, JsonElement.class).getAsJsonArray();
JsonArray jsonArray2 = new JsonArray();
for (JsonElement pa : jsonArray) {
JsonObject jsonObject2 = pa.getAsJsonObject();
String ct_name = jsonObject2.get("ct_name").getAsString();
if (ct_name.equals("SISTEMA DE PRUEBAS")) {
jsonObject2.addProperty("ct_name", "TODOS");
}
jsonArray2.add(jsonObject2);
}
System.out.println("after.................." +jsonArray2);
}
}