We Keep Coding

Java - How can I parse multiple JSON objects in Jackson?

I'm trying to parse multiple objects from a REST response. They initially get returned as JSON Lines (jsonl file format). i.e.:
{"AccountId":"123","Name":"Jeff"}
{"AccountId":"456","Name":"Joe"}
{"AccountId":"789","Name":"John"}
The first thing I do is convert it to a string:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy.com", 8080));
HttpsURLConnection httpsCon = (HttpsURLConnection) (new URL(url)).openConnection(proxy);
InputStream inputStream = httpsCon.getInputStream();
Reader reader = new InputStreamReader(inputStream);
char[] arr = new char[8 * 1024];
StringBuilder buffer = new StringBuilder();
int numCharsRead;
while ((numCharsRead = reader.read(arr, 0, arr.length)) != -1) {
buffer.append(arr, 0, numCharsRead);
}
reader.close();
return buffer.toString();
But I get errors trying to parse the String it as it's not JSON structure. So I've used a regex to format the objects into JSON structure.
{"AccountId":"123","Name":"Jeff"},{"AccountId":"456","Name":"Joe"},{"AccountId":"789","Name":"John"}
What I've tried so far:
JSONArray dataArr = new JSONArray(newTargetString);
Results in: type org.json.JSONObject cannot be converted to JSONArray
JSONObject dataObj = new JSONObject(newTargetString);
Results in a single object getting parsed instead of all 3.

Google Dataflow: how to parse big file with valid JSON array from FileIO.ReadableFile

In my pipeline FileIO.readMatches() transform reads big JSON file(around 300-400MB) with a valid JSON array and returns FileIO.ReadableFile object to the next transform. My task is to read each JSON object from that JSON array, add new properties and output to the next transform.
At the moment my code to parse the JSON file looks like this:
// file is a FileIO.ReadableFile object
InputStream bis = new ByteArrayInputStream(file.readFullyAsBytes());
// Im using gson library to parse JSON
JsonReader reader = new JsonReader(new InputStreamReader(bis, "UTF-8"));
JsonParser jsonParser = new JsonParser();
reader.beginArray();
while (reader.hasNext()) {
JsonObject jsonObject = jsonParser.parse(reader).getAsJsonObject();
jsonObject.addProperty("Somename", "Somedata");
// processContext is a ProcessContext object
processContext.output(jsonObject.toString());
}
reader.close();
In this case the whole content of the file will be in my memory which brings options to get java.lang.OutOfMemoryError. Im searching for solution to read one by one all JSON objects without keeping the whole file in my memory.
Possible solution is to use method open() from object FileIO.ReadableFile which returns ReadableByteChannel channel but Im not sure how to use that channel to read specifically one JSON object from that channel.
Updated solution
This is my updated solution which reads the file line by line
ReadableByteChannel readableByteChannel = null;
InputStream inputStream = null;
BufferedReader bufferedReader = null;
try {
// file is a FileIO.ReadableFile
readableByteChannel = file.open();
inputStream = Channels.newInputStream(readableByteChannel);
bufferedReader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"));
String line;
while ((line = bufferedReader.readLine()) != null) {
if (line.length() > 1) {
// my final output should contain both filename and line
processContext.output(fileName + file);
}
}
} catch (IOException ex) {
logger.error("Exception during reading the file: {}", ex);
} finally {
IOUtils.closeQuietly(bufferedReader);
IOUtils.closeQuietly(inputStream);
}
I see that this solution doesnt work with Dataflow running on n1-standard-1 machine and throws java.lang.OutOfMemoryError: GC overhead limit exceeded exception and works correctly on n1-standard-2 machine.

ReadableByteChannel is a java NIO API, introduced in Java 7. Java provides a way to convert it to an InputStream: InputStream bis = Channels.newInputStream(file.open()); - I believe this is the only change you need to make.

How can i read data from a json file saved in the sdcard?

I would like to know using Android Studio how to load & read a json file saved in the sdcard?
The json file contains simple json objects. Also after getting the data how can i parse it & set it as pojo.
Json example:
{
"name":"Abcd",
"id":"xyz"
}
Thanks.

Reading file:
InputStream iStream = new FileInputStream("filename.json");
BufferedReader bReader = new BufferedReader(new InputStreamReader(iStream));
String line;
String content = "";
while((line = bReader.readLine()) != null) {
content += line;
}
the easiest way to convert to POJO is to use one of the popular libraries (Gson, Jackson, etc). Using Gson:
Gson gson = new GsonBuilder().create();
gson.fromJson(content, Pojo.class);

JAVA IO : Issue with reading a stream using BufferedReader

am trying to read a JSON response using buffered reader as shown below. I'm using Apache Commons Http client. Response comes as a single line JSON and no of characters are around 1060000 and size is approximately 1 MB. Problem am facing is only part of stream is read by reader and other part is missing. How can i read the full JSON without losing any data.? Is this related to 'CharBufferSize' of BufferedReader or no of characters in the stream ?
InputStream stream = method.getResponseBodyAsStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
StringBuilder builder = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
builder.append(line);
}

try using a json parser.
import org.codehaus.jackson.*;
JsonFactory fac = new JsonFactory();
JsonParser parser = fac .createJsonParser(stream);

If you just want to copy the complete stream into the StringBuilder, you should use the InputStreamReader and a char-array buffer.
InputStream stream = method.getResponseBodyAsStream();
InputStreamReader reader = new InputStreamReader(stream, "UTF-8");
StringBuilder builder = new StringBuilder();
char[] buffer = new char[4096];
int read;
while ((read = reader.read(buffer)) != -1) {
builder.append(buffer, 0, read);
}

Finally i was able to solve using the IOUtils in Apache Commons library. Here is the code.
BoundedInputStream boundedInputStream= new BoundedInputStream(stream);
BufferedReader reader = new BufferedReader(new InputStreamReader(boundedInputStream,"UTF-8"));
StringBuilder builder= new StringBuilder();
StringBuilderWriter writer = new StringBuilderWriter(builder);
IOUtils.copy(reader, writer);
Although it is been a while, it may be helpful for someone.
Here is the original source,
Most Robust way of reading a file or stream using Java (To prevent DoS attacks)

Decoding HttpEntity into android string - encoding issue

I know that should be basics but i had no formation :( and I don't understand it, everywhere it seems obvious to people. I get that one side encode data with his set and android is probably expecting another one, but what can I do to translate?
My app perform a get request on google maps api to retrieve an address from a Lat/lng. But I failed to decode properly the result as a French è is displayed as è
I have not enough xp in Java to understand what to do. It is linked with UTF-8, right?
What should I do?
response = client.execute(httpGet);
HttpEntity entity = response.getEntity();
InputStream stream = entity.getContent();
int b;
while ((b = stream.read()) != -1) {
stringBuilder.append((char) b);
}
JSONObject jsonObject = new JSONObject();
jsonObject = new JSONObject(stringBuilder.toString());
retList = new ArrayList<Address>();
if("OK".equalsIgnoreCase(jsonObject.getString("status"))){
JSONArray results = jsonObject.getJSONArray("results");
for (int i=0;i<results.length();i++ ) {
JSONObject result = results.getJSONObject(i);
String indiStr = result.getString("formatted_address");
Address addr = new Address(Locale.ITALY);
addr.setAddressLine(0, indiStr);
Dbg.d(TAG, "adresse :"+addr.toString());
retList.add(addr);
}
}
Thanks for help !

Try using UTF-8,
instead of using InputStream try something like,
String responseBody = EntityUtils.toString(response.getEntity(), HTTP.UTF_8);

you can use BufferReader
your code will be like this:
InputStream stream = entity.getContent();
BufferedReader br = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
int b;
while ((b = br.read()) != -1) {
stringBuilder.append(b);
}