How to Read Large Json File in chunks - java

I am using below JSON format:
[{object1},{object2},{object3}...]
I am reading a JSON file object wise means one by one (object by object) and my below code working fine.
I want to read it in chunks(at a time 10 objects). I have tried a lot but I am not getting any solution (without using Spring Batch). Can anybody please help me how to read in chunks
#Component
public class PdpiRunner3 implements CommandLineRunner {
#Override
public void run(String... args) throws Exception {
try {
JsonReader jsonReader = new JsonReader(new InputStreamReader(new ClassPathResource("/json/trades2.json").getInputStream(),StandardCharsets.UTF_8));
Gson gson = new GsonBuilder().create();
jsonReader.beginArray();
while (jsonReader.hasNext()) { // next json array element
PDPIfiles json = gson.fromJson(jsonReader, PDPIfiles.class);
if (json != null) {
System.out.println(json);
}
}
jsonReader.endArray();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Is it possible to read json element In Chunk without spring Batch pleas help ?

Related

How can i add a new object to my json file in java instead of overwriting it?

Im currently building a GSONFileWriter class.
public class GSONFileWriter {
private File jsonFile;
private final String json;
public GSONFileWriter(String json) {
this.json = json;
}
public void generateJsonFileIfNotExists(String pathname) {
try {
jsonFile = new File(pathname);
if (!jsonFile.exists()) {
if (jsonFile.createNewFile()) {
System.out.println("File successful created.");
} else {
System.out.println("Error: Building the file went wrong!");
System.exit(1);
}
}
fillJsonFile();
} catch (Exception e) {
System.err.println("Error: Building the file went wrong!");
}
}
private void fillJsonFile() {
try (PrintWriter writer = new PrintWriter(jsonFile, StandardCharsets.UTF_8)) {
writer.append(json);
writer.println();
} catch (Exception e) {
e.printStackTrace();
}
}
}
it is called inside my CLI class
Gson gson = new Gson();
String json = gson.toJson(target);
GSONFileWriter gsonFileWriter = new GSONFileWriter(json);
gsonFileWriter.generateJsonFileIfNotExists("EmployeeData.json");
It creates and builds a new JSON File with an object inside it.
{"salary":34000.0,"name":"Hans","age":30,"id":"d40507a7-a802-4494-9a0c-5a97a0a4d0bf"}
However the Problem is, that whenever i run the code again, the old file gets overwritten and a new one created. I tried to change the code, so that it adds a new object to the file, instead of overwrite it. Any tips?
I would recommend:
At first: Read your json file as String if it exists.
String file = "<File>";
String json = new String(Files.readAllBytes(Paths.get(file)));
Second you transform this string into a JsonObject using JsonParser:
JsonObject object = JsonParser.parseString(json).getAsJsonObject();
now you obtained your json-file as JsonObject tree and can manipulate this as you like. Using:
JsonObject.get(<MemberName>);
JsonObject.add(String <propertyName>, JsonElement <value>);
JsonObject.addProperty(String <propertyName>, Number/String/Boolean/Char <value>);
, etc.
Whenever you are finished you can write this to your json-file, either through a OutputStream or whatever you like.

Converting a JSON payload into useful data in Java

Just to start off, I'm new to Java. I'm trying to make a notification system for Twitch using their API and an Arduino, and I'm currently stuck at the start pretty much. I am able to access the API and get information, but I don't know how to convert it into something useful. This is my code currently.
public class commandRun {
public static void main(String[] args) {
Process p;
try {
p = Runtime.getRuntime().exec("cmd /c curl -H \"Client-ID: clientID\" -X GET \"https://api.twitch.tv/helix/users/follows?first=1&to_id=channelID\n");
p.waitFor();
BufferedReader reader=new BufferedReader(new InputStreamReader(p.getInputStream()));
String twitchFollower;
while((twitchFollower = reader.readLine()) != null) {
System.out.println(twitchFollower);
}
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
This returns
{
"total": 106,
"data": [
{
"from_id": "followerID",
"from_name": "follower",
"to_id": "channelid",
"to_name": "channel",
"followed_at": "2019-10-06T21:03:03Z"
}
],
"pagination": {
"cursor": "dontknowifthisissensitiveinfo"
}
}
I want to be able to take specific info out of string such as the followers' name.
There are many JSON libraries can achieve this as jrook's comment.
And following sample code shows how the most 2 popular libraries - Jackson and Gson - get the value of specific field. (I assumed that the key of follower's name is from_name.)
// Jackson
ObjectMapper mapper = new ObjectMapper();
String followerName = mapper.readTree(jsonStr)
.get("data")
.get(0)
.get("from_name")
.asText();
System.out.println(followerName);
// Gson
followerName = new JsonParser().parse(jsonStr)
.getAsJsonObject()
.get("data")
.getAsJsonArray()
.get(0)
.getAsJsonObject()
.get("from_name")
.getAsString();
System.out.println(followerName);
Console output:
follower
follower

Unable to write Json output into file in java

I am trying to write simple JSON data into a file but seems my file is blank. It does not write anything. When I print output in java console then it shows me correct.
import org.json.simple.JSONObject;
JSONObject obj = new JSONObject();
obj.put("Phone Number:", "XXXXXXXXX");
obj.put("Fname:", "Mike");
obj.put("Lname:", "Miller");
obj.put("Street:", "101");
try {
FileWriter file = new FileWriter("D:\\file1.json");
file.write(obj.toJSONString());
}
catch (Exception e) {
e.printStackTrace();
}
I saw similar code on the internet and SO as well. I am following same but still not sure why it is not writing output into a file.
This is updated working version
public static void main(String...strings) throws IOException{
FileWriter file = new FileWriter("C:\\file1.json");
try {
JSONObject obj = new JSONObject();
obj.put("Phone Number:","XXXXXXXXX");
obj.put("Fname:","Mike");
obj.put("Lname:","Miller");
obj.put("Street:","101");
file.write(obj.toString());
}catch (Exception E)
{
System.out.println(E);
E.printStackTrace();
}finally{
file.close();
}
}
The OS may not write the data into hardware until you call flush:
public static void main(String[] args) throws Exception {
JSONObject obj = new JSONObject();
obj.put("Phone Number:","XXXXXXXXX");
obj.put("Fname:","Mike");
obj.put("Lname:","Miller");
obj.put("Street:","101");
FileWriter file = new FileWriter("D:\\file1.json");
try {
file.write(obj.toJSONString());
}catch (Exception E) {
E.printStackTrace();
} finally {
file.flush();
file.close();
}
}
Here you are not flushing the data into files. That's why data is not written into file. If you also want to keep the existing data into the file, try appending. Here is the updated code that will append new data to that existing file.
import org.json.simple.JSONObject;
JSONObject obj = new JSONObject();
obj.put("Phone Number:","XXXXXXXXX");
obj.put("Fname:","Mike");
obj.put("Lname:","Miller");
obj.put("Street:","101");
try {
FileWriter file = new FileWriter("D:\\file1.json", true); // true to append at the end of file.
file.write(obj.toJSONString());
file.flush()
}catch (Exception E)
{
E.printStackTrace();
}finally{
file.close();
}
}
When you are writing code by using any Files classes, I mean either you writing the content into file or reading from file, closing the stream is always best practice.
The code should be like this,
FileWriter file = new FileWriter("D:\\file1.json");
file.write(obj.toJSONString());
file.close();

try catch code reading JSON file Android

having some serious problem when trying to read a local JSON file. I've looked everywhere for many days now and the best and farthest I could get was copying from Faizan's answer.
Reading a json file in Android
How come that Android Studio doesn't let me generate the second try-catch code block here?
Help and advice are very much appreciated!!
This is My code
public String loadJSONFromAsset() {
String json = null;
try {
InputStream is = getAssets().open("names.json");
int size = is.available();
byte[] buffer = new byte[size];
is.read(buffer);
is.close();
json = new String(buffer, "UTF-8");
} catch (IOException ex) {
ex.printStackTrace();
return null;
}
return json;
}
String jsonString = loadJSONFromAsset();
try {
JSONObject json = new JSONObject(jsonString);
JSONObject jObject = json.getJSONObject("female");
JSONObject jObject2 = jObject.getJSONObject("adult");
String name = jObject2.toString();
}
catch (Exception e) {
e.printStackTrace();
}
}
How come that Android Studio doesn't let me generate the second
try-catch code block here?
Simply, because your code is not inside a method.
Doing something like below should solve the error.
public void someMethodIdentifier(){ // doesn't have to be void return type, you know better than me what type you want to return.
String jsonString = loadJSONFromAsset();
try {
JSONObject json = new JSONObject(jsonString);
JSONObject jObject = json.getJSONObject("female");
JSONObject jObject2 = jObject.getJSONObject("adult");
String name = jObject2.toString();
}
catch (Exception e) {
e.printStackTrace();
}
}
Note - from the looks of the statements that's contained within the try block I think you intended to return some data? if that's the case just replace the void return type with the appropriate return type and return that data.

Writing pretty print json output to fileoutput stream with GSON JsonWriter [duplicate]

This question already has an answer here:
Why will Gson pretty-print to the console, but not the file?
(1 answer)
Closed 4 years ago.
I am having hard time in writing pretty print json string on file using Gson library. Code I am using is here:
Gson gs = new GsonBuilder().setPrettyPrinting().disableHtmlEscaping().create();
JsonWriter jsonWriter = null;
try {
jsonWriter = new JsonWriter(new OutputStreamWriter(stream, "UTF-8"));
jsonWriter.beginArray();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
for (Feed feed : feeds) {
gs.toJson(feed, Feed.class, jsonWriter);
}
try {
jsonWriter.endArray();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
jsonWriter.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Here stream is nothing but a file output stream. Even though I have enabled the pretty printing but still I am getting unformatted json string on file.
Any help is appreciated.
You can simply enable the pretty print by setting the indent.
jsonWriter.setIndent(" ");

Categories