I have the below piece of code where I write a string to an output stream with Jackson's ObjectMapper.
OutputStream outputStream = new PrintStream(System.out);
ObjectMapper objectMapper = new ObjectMapper();
String output = "{\"test\":\"mytest/123\"}";
objectMapper.writeValue(outputStream, output);
When I run this code, the output stream prints the following string, where there are additional escape characters.
"{\"vfi\":\"test/mytest/123\"}"
How can I avoid the escape characters here? I am not using ObjectMapper.writeValueAsString but still they get printed. Appreciate any help.
You are serializing a Java string as JSON string. JSON strings must start and end with " and any embedded quotes must be escaped. (ref: https://json.org)
You have two options:
Print the string directly, without serializing via ObjectMapper
OutputStream outputStream = new PrintStream(System.out);
String output = "{\"test\":\"mytest/123\"}";
outputStream.println(output);
Serialize an object, e.g. Map, to a JSON string
OutputStream outputStream = new PrintStream(System.out);
ObjectMapper objectMapper = new ObjectMapper();
Map<String, String> output = Map.of("test", "mytest/123");
objectMapper.writeValue(outputStream, output);
It's also possible to define a custom class to represent your JSON structure. Depending on the complexity of your structure and other requirements, this could be the preferrable option.
class MyOutput {
private final String test;
public MyOutput(final String test) { this.test = test; }
public String getTest() { return test; }
}
OutputStream outputStream = new PrintStream(System.out);
ObjectMapper objectMapper = new ObjectMapper();
MyOutput output = new MyOutput("mytest/123");
objectMapper.writeValue(outputStream, output);
Related
String input = "Vish,Path,123456789";
Expected output as Json string, and thread safe = {"name":"Vish","surname":"Path","mobile":"123456789"}
I tried by using
GsonBuilder gsonBuilder = new GsonBuilder();
Gson gson = gsonBuilder.create();
But every time I'm creating new Object -
MappingObject[] studentArray = new MappingObject[1];
studentArray[0] = new MappingObject("Vish","Path","123456789");
I separated this comma separated string by using split()
System.out.println("JSON "+gson.toJson(studentArray));
You will have to create a Map:
Map<String,String> jsonMap = new HashMap<String,String>();
jsonMap.put("name","Vish");
jsonMap.put("surname","Path");
jsonMap.put("mobile","123456789");
Then use com.google.gson JSONObject:
JSONObject jsonObj = new JSONObject(jsonMap);
If you don't want to use any library then you have to split string by comma and make a new String.
String input = "Vish,Path,123456789";
String[] values=input.split("[,]");
StringBuffer json = new StringBuffer();// StringBuffer is Thread Safe
json.append("{")
.append("\"name\": \"").append(values[0]).append("\",")
.append("\"surname\": \"").append(values[1]).append("\",")
.append("\"mobile\": \"").append(values[2]).append("\"")
.append("}");
System.out.println(json.toString());
Output :
{"name": "Vish","surname": "Path","mobile": "123456789"}
If you want to use library then you will achive this by Jackson. Simple make a class and make json by it.
public class Person {
private String name;
private String surname;
private String mobile;
// ... getters and Setters
}
String input = "Vish,Path,123456789";
String[] values=input.split("[,]");
Person person = new Person(values[0],values[1],values[2]);// Assume you have All Argumets Constructor in specified order
ObjectMapper mapper = new ObjectMapper(); //com.fasterxml.jackson.databind.ObjectMapper;
String json = mapper.writeValueAsString(person);
I would like to find the most clever way to create an InputStream from a JsonNode, of the Java library Jackson.
Until now I have done:
IOUtils.toInputStream(jsonNode.toString());
But this way converts the JsonNode into a String before creating the InputStream.
ex of need:
org.apache.http.entity.InputStreamEntity entity = new InputStreamEntity(IOUtils.toInputStream(jsonNode.toString()));
In most cases JSON is written as UTF-8 and you can save some memory, if you directly generate byte array using ObjectMapper.
ObjectMapper objectMapper = new ObjectMapper();
JsonNode json = ...;
byte[] bytes = objectMapper.writeValueAsBytes(json);
Specifically, Apache HTTP client provides ByteArrayEntity for use with byte array. For other uses, there is a ByteArrayInputStream.
Of course, ObjectMapper should be created only once and reused.
If you really want JSON to be written in a streaming manner, it is possible to use a pair of PipedInputStream and PipedOutputStream, however, as JavaDoc states
Typically, data is read from a PipedInputStream object by one thread and data is written to the corresponding PipedOutputStream by some other thread. Attempting to use both objects from a single thread is not recommended, as it may deadlock the thread.
Example:
ObjectMapper objectMapper = new ObjectMapper();
JsonNode json = ...;
PipedInputStream in = new PipedInputStream();
new Thread(() -> {
try {
IOUtils.copy(in, System.out);
} catch (IOException e) {
...
}
}).start();
try (
PipedOutputStream out = new PipedOutputStream(in);
JsonGenerator gen = objectMapper.getFactory().createGenerator(out);
) {
gen.writeTree(json);
} catch (IOException e) {
...
}
Previously my code was just this:
objJson = gson.toJson(objList);
return objJson;
And I got the string JSON with that return value.
But then I started getting famous Out Of Memory errors when the JSON becomes too large.
Then I followed the following post that converts the list of objects into JSON String in an efficient way that OOM errors will be no more:
https://sites.google.com/site/gson/streaming
So, I have followed the above approach here:
public String writeJsonStream(List<MyObj> objList) throws IOException {
ByteArrayOutputStream baos=new ByteArrayOutputStream();
OutputStreamWriter outputStreamWriter=new OutputStreamWriter(baos,"UTF-8");
JsonWriter writer = new JsonWriter(outputStreamWriter);
writer.setIndent(" ");
writer.beginArray();
Gson gson = new GsonBuilder().excludeFieldsWithoutExposeAnnotation().setPrettyPrinting().create();
for (MyObj myobj : objList) {
gson.toJson(myobj, MyObj.class, writer);
}
String objJson = writer.toString();
writer.endArray();
writer.close();
return objJson;
}
But this returning object com.google.gson.stream.JsonWriter#6a9454cd.
The method gson.toJson(myobj, MyObj.class, writer); is of type void and doesn't returns JSON string. So how can I get the JSON string in this case?
That's because you are getting the JSON String from String objJson = writer.toString();. This is not how you should retrieve it: this code will just call the toString() method on the writer instance. Since toString() is not overriden for this object, the result is the default com.google.gson.stream.JsonWriter#6a9454cd.
What you want to do instead is get the bytes that the JsonWriter wrote to the output stream. In your case, you are using a ByteArrayOutputStream so you can call toString(charsetName) to get the content as a String:
public String writeJsonStream(List<MyObj> objList) throws IOException {
ByteArrayOutputStream baos=new ByteArrayOutputStream();
OutputStreamWriter outputStreamWriter=new OutputStreamWriter(baos,"UTF-8");
JsonWriter writer = new JsonWriter(outputStreamWriter);
writer.setIndent(" ");
writer.beginArray();
Gson gson = new GsonBuilder().excludeFieldsWithoutExposeAnnotation().setPrettyPrinting().create();
for (MyObj myobj : objList) {
gson.toJson(myobj, MyObj.class, writer);
}
writer.endArray();
writer.close();
return baos.toString("UTF-8");
}
As a side note, you could just use a StringWriter instead and get the content with toString():
StringWriter stringWriter = new StringWriter();
JsonWriter writer = new JsonWriter(stringWriter);
// rest of code
writer.close();
String json = stringWriter.toString();
Well, converting stream to a static object unfortunately does not avoid OOM problem. In either way, you are trying to allocate/request memory to construct/write that string. You must pipe the resulting stream to another one, or simply consume all the data in that stream by using BufferedWriter in some loop or else.
Try this. Change ByteArrayOutputStream to OutputStream and get object from System.out
public String writeJsonStream(List<MyObj> objList) throws IOException {
OutputStream baos = System.out;
OutputStreamWriter outputStreamWriter=new OutputStreamWriter(baos,"UTF-8");
JsonWriter writer = new JsonWriter(outputStreamWriter);
writer.setIndent(" ");
writer.beginArray();
Gson gson = new GsonBuilder().excludeFieldsWithoutExposeAnnotation().setPrettyPrinting().create();
for (MyObj myobj : objList) {
gson.toJson(myobj, MyObj.class, writer);
}
String objJson = writer.toString();
writer.endArray();
writer.close();
return objJson;
My Question is similar to what has been asked here .
few points :
I can not change the format. (No commas to be added etc)
This is basically a huge .txt file containing 1000's of Json objects.
My Json objects are HUGE.
This is what I am doing right now :
FileReader fileReader = new FileReader(fileName);
BufferedReader reader = new BufferedReader(fileReader);
String data = "";
while((data = reader.readLine()) != null){
ObjectMapper mapper = new ObjectMapper();
Map<String,String> map = mapper.readValue(data, Map.class);
}
Currently I am using Jackson and Ideally I would like to read one Json Object from the file at a time, Parse it and then move on to the next one. I need to count let say unique number of id's from these Json object and do more operations. It will be best to read them one by one.
Is jackson would be the best way going forward ?
This is a good example of parsing huge Json, But it deals with only one object per file. My file has huge Jsons (1000s of them).
Here is a Jackson example that works for me. I have thousands json objects (tokens) in a single json file. This code will iterate through the file read each token and print it's serial.
Required imports:
import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.ObjectMapper;
Using Jackson to read multiple json objects from FileInputStream:
try (FileInputStream fis = new FileInputStream("D:/temp/tokens.json")) {
JsonFactory jf = new JsonFactory();
JsonParser jp = jf.createParser(fis);
jp.setCodec(new ObjectMapper());
jp.nextToken();
while (jp.hasCurrentToken()) {
Token token = jp.readValueAs(Token.class);
jp.nextToken();
System.out.println("Token serial "+token.getSerialNumber());
}
}
Here is a more JAVA 8ish solution for your query, I always lean toward BufferedReader over InputStreams for any place where parsing is going to be done a lot of time.
ObjectMapper mapper = new ObjectMapper();
JsonFactory jsonFactory = new JsonFactory();
try(BufferedReader br = new BufferedReader(new FileReader("luser.txt"))) {
Iterator<luser> value = mapper.readValues( jsonFactory.createParser(br), luser.class);
value.forEachRemaining((u)->{System.out.println(u);});
}
The deserialization for each object happens as part of next(), in each iteration.
Here is how I used Gson's JSONReader API to handle similar requirement as above
public static List<YOURPOJO> readTraceLog(String filepath) throws IOException {
Gson gson = new Gson();
JsonReader jsonReader = new JsonReader(new FileReader(filepath));
// important as handles unwanted formatting stuffs such empty spaces
jsonReader.setLenient(true);
boolean start = true; // start of read
jsonReader.beginObject(); // first object begins
//List to hold object
List<YOURPOJO> completeList = new ArrayList<YOURPOJO>();
//YOURPOJO has two attributes one is ID and other is list of ANOTHERPOJO
while (jsonReader.hasNext()) {
if (!start) {
//to stop end of Document
if (jsonReader.peek().toString().matches("END_DOCUMENT")) {
break;
}
//continue reading object as the come -{
jsonReader.beginObject();
}
start = false;
YOURPOJO pojo = new YOURPOJO();
//read attribute id
String name = jsonReader.nextName();
pojo.setId(name);
//list to store ANOTHERPOJO objects
List<ANOTHERPOJO> tempList = new ArrayList<ANOTHERPOJO>();
//begin reading list - [
jsonReader.beginArray();
while (jsonReader.hasNext()) {
ANOTHERPOJO t = gson.fromJson(jsonReader, ANOTHERPOJO.class);
tempList.add(t);
}
//end reading list - ]
jsonReader.endArray();
//store data
pojo.setTraceDetails(tempList);
completeList.add(YOURPOJO);
//end of object - }
jsonReader.endObject();
}
jsonReader.close();
return completeList;
}
i like to encode a java map of strings as a single base 64 encoded string. The encoded string will be transmitted to a remote endpoint and maybe manipulated by a not nice person. So the worst thing that should happen are invaild key,value-tuples, but should not bring any other security risks aside.
Example:
Map<String,String> map = ...
String encoded = Base64.encode(map);
// somewhere else
Map<String,String> map = Base64.decode(encoded);
Yes, must be Base64. Not like that or that or any other of these. Is there an existing lightweight solution (Single Utils-Class prefered) out there? Or do i have to create my own?
Anything better than this?
// marshalling
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(map);
oos.close();
String encoded = new String(Base64.encodeBase64(baos.toByteArray()));
// unmarshalling
byte[] decoded = Base64.decodeBase64(encoded.getBytes());
ByteArrayInputStream bais = new ByteArrayInputStream(decoded);
ObjectInputStream ois = new ObjectInputStream(bais);
map = (Map<String,String>) ois.readObject();
ois.close();
Thanks,
my primary requirements are: encoded string should be as short as possible and contain only latin characters or characters from the base64 alphabet (not my call). there are no other reqs.
Use Google Gson to convert Map to JSON. Use GZIPOutputStream to compress the JSON string. Use Apache Commons Codec Base64 or Base64OutputStream to encode the compressed bytes to a Base64 string.
Kickoff example:
public static void main(String[] args) throws IOException {
Map<String, String> map = new HashMap<String, String>();
map.put("key1", "value1");
map.put("key2", "value2");
map.put("key3", "value3");
String serialized = serialize(map);
Map<String, String> deserialized = deserialize(serialized, new TypeToken<Map<String, String>>() {}.getType());
System.out.println(deserialized);
}
public static String serialize(Object object) throws IOException {
ByteArrayOutputStream byteaOut = new ByteArrayOutputStream();
GZIPOutputStream gzipOut = null;
try {
gzipOut = new GZIPOutputStream(new Base64OutputStream(byteaOut));
gzipOut.write(new Gson().toJson(object).getBytes("UTF-8"));
} finally {
if (gzipOut != null) try { gzipOut.close(); } catch (IOException logOrIgnore) {}
}
return new String(byteaOut.toByteArray());
}
public static <T> T deserialize(String string, Type type) throws IOException {
ByteArrayOutputStream byteaOut = new ByteArrayOutputStream();
GZIPInputStream gzipIn = null;
try {
gzipIn = new GZIPInputStream(new Base64InputStream(new ByteArrayInputStream(string.getBytes("UTF-8"))));
for (int data; (data = gzipIn.read()) > -1;) {
byteaOut.write(data);
}
} finally {
if (gzipIn != null) try { gzipIn.close(); } catch (IOException logOrIgnore) {}
}
return new Gson().fromJson(new String(byteaOut.toByteArray()), type);
}
Another possible way would be using JSON which is a very ligthweight lib.
The the encoding then would look like this:
JSONObject jso = new JSONObject( map );
String encoded = new String(Base64.encodeBase64( jso.toString( 4 ).toByteArray()));
Your solution works. The only other approach would be to serialize the map yourself (iterate over the keys and values). That would mean you'd have to make sure you handle all the cases correctly (for example, if you transmit the values as key=value, you must find a way to allow = in the key/value and you must separate the pairs somehow which means you must also allow this separation character in the name, etc).
All in all, it's hard to get right, easy to get wrong and would take a whole lot more code and headache. Plus don't forget that you'd have to write a lot of error handling code in the parser (receiver side).