how to read a csv to a nested json with jackson java - java

i have this type of csv :
metric,value,date
temp_a,622.0,1477895624866
temp_a,-3.0,1477916224866
temp_a,365.0,1477917224866
temp_b,861.0,1477895624866
temp_b,767.0,1477917224866
and i want to use java jackson to convert it to json but not any json; it needs to be like this:
[
{
"metric":"temp_a",
"datapoints":[
[622, 1477895624866],
[-3, 1477916224866],
[365, 1477917224866]
]
},
{
"metric":"temp_b",
"datapoints":[
[861, 1477895624866],
[767, 1477917224866]
]
}
]
where dataponits is an array containing the value and the date in the csv .
i have managed to use the jackson to get this result :
{metric=temp_a, value=622.0, date=1477895624866}
{metric=temp_a, value=-3.0, date=1477916224866}
{metric=temp_a, value=365.0, date=1477917224866}
{metric=temp_b, value=861.0, date=1477895624866}
{metric=temp_b, value=767.0, date=1477917224866}
but it is not what i want and the jackson doc is a bit hard for me to understand and play with , may be this is possible with Pojos or annotations but i can't understand them, i couldn't find how to do a nested json.
if i can do this better this something else then jackson please tell me .
thank you for helping.

You do not have to always deserialise CSV to a POJO structure and implement custom serialisers. In this case, you can also:
Deserialise CSV to a Map
Group by elements in a Map to a form metric -> [[...], [...]]
Convert above Map to another form of Map
Serialise Map to a JSON
Example code could look like below:
import com.fasterxml.jackson.core.util.DefaultIndenter;
import com.fasterxml.jackson.core.util.DefaultPrettyPrinter;
import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;
import java.io.File;
import java.math.BigDecimal;
import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Spliterator;
import java.util.Spliterators;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;
public class CsvApp {
public static void main(String[] args) throws Exception {
File csvFile = new File("./resource/test.csv").getAbsoluteFile();
CsvMapper csvMapper = CsvMapper.builder().build();
MappingIterator<Map> rows = csvMapper
.readerWithSchemaFor(Map.class)
.with(CsvSchema.emptySchema().withHeader())
.readValues(csvFile);
DataConverter converter = new DataConverter();
List<Map<String, Object>> result = converter.toMetricDataPoints(rows);
ObjectMapper jsonMapper = JsonMapper.builder()
.enable(SerializationFeature.INDENT_OUTPUT)
.build();
jsonMapper.writeValue(System.out, result);
}
}
class DataConverter {
public List<Map<String, Object>> toMetricDataPoints(MappingIterator<Map> rows) {
return toStream(rows)
//group by metric -> [value, date]
.collect(Collectors.groupingBy(map -> map.get("metric"),
Collectors.mapping(map -> Arrays.asList(toNumber(map.get("value")), toNumber(map.get("date"))),
Collectors.toList())))
.entrySet().stream()
// convert to Map: metric + datapoints
.map(entry -> {
Map<String, Object> res = new LinkedHashMap<>(4);
res.put("metric", entry.getKey());
res.put("datapoints", entry.getValue());
return res;
}).collect(Collectors.toList());
}
private Stream<Map> toStream(MappingIterator<Map> rowIterator) {
return StreamSupport.stream(Spliterators.spliteratorUnknownSize(rowIterator, Spliterator.ORDERED), false);
}
private long toNumber(Object value) {
return new BigDecimal(Objects.toString(value, "0")).longValue();
}
}
Above code prints:
[ {
"metric" : "temp_a",
"datapoints" : [ [ 622, 1477895624866 ], [ -3, 1477916224866 ], [ 365, 1477917224866 ] ]
}, {
"metric" : "temp_b",
"datapoints" : [ [ 861, 1477895624866 ], [ 767, 1477917224866 ] ]
} ]
As you can see, we used only basic Jackson functionality, rest of manipulation on data we implemented using Java 8 API.
See also:
Directly convert CSV file to JSON file using the Jackson library
How to convert an iterator to a stream?
Jackson JSON Deserialization: array elements in each line

Related

Map JSON string or JSON array to String in Java object [duplicate]

This question already has answers here:
Make Jackson interpret single JSON object as array with one element
(3 answers)
Deserializing json to pojo where json field has different data types
(2 answers)
Closed 1 year ago.
I have a JSON created by Elixir class which has a field which can be string or array:
field :logs, {:array, :string}
If anyone doesn't get this it will be like
{"logs" : "some log 1"}
or
{
"logs": ["some log 1", "some log 2"]
}
I have a Java field mapped for this:
#JsonProperty("logs")
private String logs;
This mapping works only when the logs comes as a String, but fails if the logs comes as array with error saying it will not be able to convert START_ARRAY to string.
How to serialize the field if it comes as array and store it as a comma separated string?
I see in tags that you use Jackson for parsing. This means you need to write and register with Jackson a custom deserializer for your logs field.
An example of such solution:
package tmp;
import java.io.IOException;
import java.util.Spliterator;
import java.util.Spliterators;
import java.util.stream.Collectors;
import java.util.stream.StreamSupport;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.exc.MismatchedInputException;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ValueNode;
public class JacksonDemo {
public static class LogHolder {
#JsonProperty("logs")
#JsonDeserialize(using = ArrayOrStringJsonDeserializer.class)
private String logs;
#Override
public String toString() {
return "LogHolder(logs=" + logs + ")";
}
}
public static class ArrayOrStringJsonDeserializer extends JsonDeserializer<String> {
#Override
public String deserialize(JsonParser jsonParser, DeserializationContext ctxt) throws IOException, JsonProcessingException {
JsonNode node = (JsonNode) jsonParser.readValueAsTree();
if (node.isValueNode()) {
ValueNode valueNode = (ValueNode) node;
if (valueNode.isTextual()) {
return valueNode.textValue();
}
} else if (node.isArray()) {
ArrayNode arrayNode = (ArrayNode) node;
return StreamSupport.stream(Spliterators.spliteratorUnknownSize(arrayNode.iterator(), Spliterator.ORDERED), false)
.map(JsonNode::textValue)
.collect(Collectors.joining(", "));
}
throw MismatchedInputException.from(jsonParser, String.class,
"Expected node to be of type String or array, but got " + node.getNodeType().toString());
}
}
public static void main(String args[]) throws Exception {
String[] docs = { "{\"logs\" : \"some log 1\"}", "{\"logs\": [\"some log 1\", \"some log 2\"]}" };
ObjectMapper om = new ObjectMapper();
for (String doc : docs) {
System.out.println(om.readValue(doc, LogHolder.class));
}
}
}
Result of executing this code:
LogHolder(logs=some log 1)
LogHolder(logs=some log 1, some log 2)

Deserialize json with duplicate keys using Jackson

I am trying to parse a json payload with duplicate keys to a Map<String, List<String>> using Jackson
Consider the following payload
{
"foo" : "val1",
"foo" : "val2",
"bar" : "val3"
}
I want to convert this to a Map<String, List<String>> type.
ex:
+--------------+----------------------+
| Key (String) | Value (List<String>) |
+--------------+----------------------+
| foo | [val1, val2] |
| bar | [val3] |
+--------------+----------------------+
What would be the best way to handle this with Jackson? I'm hoping that there's a way in Jackson where I can register a custom deserialization logic for Map<String, List<String>> type. (Note that I have no control over the json payload sent in the request)
Appreciate any help on this.
Thanks!
Check if it works for you.
Using #JsonAnySetter to deserialize unmapped JSON properties. #JsonAnySetter annotation can be used to define "any setter" mutator.
Using #JsonAnyGetter Annotation to serialize any arbitrary properties. #JsonAnyGetter can be used on a method which returns a Map.
POJO Definition
Test.java
package oct2020.json;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.fasterxml.jackson.annotation.JsonAnyGetter;
import com.fasterxml.jackson.annotation.JsonAnySetter;
public class Test {
private Map<String, List<String>> keyValuesMap = new HashMap<String, List<String>>();
/**
* #return the keyValuesMap
*/
#JsonAnyGetter
public Map<String, List<String>> getKeyValuesMap() {
return keyValuesMap;
}
/**
* #param keyValuesMap
* the keyValuesMap to set
*/
public void setKeyValuesMap(Map<String, List<String>> keyValuesMap) {
this.keyValuesMap = keyValuesMap;
}
#JsonAnySetter
public void duplicateKeyValues(String key, String value) {
List<String> values = null;
if (!keyValuesMap.containsKey(key)) {
values = new ArrayList<String>();
} else {
values = keyValuesMap.get(key);
}
values.add(value);
keyValuesMap.put(key, values);
}
}
Converting the json to desired format.
JSONConverter .java
package oct2020.json;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import com.fasterxml.jackson.databind.ObjectMapper;
public class JSONConverter {
public static void main(String[] args) throws IOException {
ObjectMapper mapper = new ObjectMapper();
String json = "{\n\"foo\" : \"val1\",\n\"foo\" : \"val2\",\n\"bar\" : \"val3\"\n,"
+ "\n\"bar\" : \"val3\"\n,\n\"bar\" : \"val3\"\n,\n\"bar\" : \"val3\"\n,"
+ "\n\"bar\" : \"val3\"\n,\n\"bar\" : \"val3\"\n,\n\"bar\" : \"val3\"\n,"
+ "\n\"bar\" : \"val3\"\n,\n\"bar\" : \"val3\"\n,\n\"bar\" : \"val3\"}";
Test test = mapper.readValue(json, Test.class);
Map<String, List<String>> keyValuesMap = test.getKeyValuesMap();
System.out.println(mapper.writerWithDefaultPrettyPrinter()
.writeValueAsString(keyValuesMap));
}
}
Output:
{
"bar" : [ "val3", "val3", "val3", "val3", "val3", "val3", "val3", "val3", "val3", "val3" ],
"foo" : [ "val1", "val2" ]
}
please put this property JsonTypeInfo.As.EXISTING_PROPERTY in the #JsonTypeInfo annotation. It might work.

Escape String backslash using GSON

I am trying to convert my hashmap to a JSON pretty print output.
I tried GSON and believe there is some issue with it handling string as inputs inside a map, is there any other way to do this?
Map<String, String> map = new LinkedHashMap();
Gson gson = new GsonBuilder().enableComplexMapKeySerialization().setPrettyPrinting().create();
map.put("Intro", Map_to_String);
map.put("Output", String_Val);
System.out.println(gson.toJson(map));
Output:
{
"Intro": {\"No\":0,\"Cast\":2},
"Output": "123"
}
Required Output:
{
"Intro": {"No":0,"Cast":2},
"Output": "123"
}
You need to deserialise Map_to_String back to object - Map in this case and after that serialise again.
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.reflect.TypeToken;
import java.lang.reflect.Type;
import java.util.LinkedHashMap;
import java.util.Map;
public class GsonApp {
public static void main(String[] args) {
Map<String, Object> map = new LinkedHashMap<>();
Gson gson = new GsonBuilder()
.enableComplexMapKeySerialization()
.setPrettyPrinting()
.create();
String jsonString = "{\"No\":0,\"Cast\":2}";
Type mapType = new TypeToken<Map<String, String>>() {}.getType();
map.put("Intro", gson.fromJson(jsonString, mapType));
map.put("Output", "123");
System.out.println(gson.toJson(map));
}
}
Prints:
{
"Intro": {
"No": "0",
"Cast": "2"
},
"Output": "123"
}

How do I modify a spring service to produce jsonl

I have a service consumer who wants my service to produce line delimited JSONL. How can I modify the Jackson parser or provide a custom serializer so that a retuned array of objects is serialized as JSONL and not JSON.
For example the following code
import java.util.Arrays;
import org.apache.commons.lang3.tuple.Pair;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
#SpringBootApplication
#RestController
public class JsonlServiceApplication {
public static void main(String[] args) {
SpringApplication.run(JsonlServiceApplication.class, args);
}
#GetMapping("jsonl")
private ResponseEntity<?> getJsonl(){
Pair<String, String> p1 = Pair.of("foo", "baa");
Pair<String, Integer> p2 = Pair.of("key", 10);
return new ResponseEntity(Arrays.asList(p1, p2), HttpStatus.OK);
}
}
Will produce this JSON:
[
{
"foo": "baa"
},
{
"key": 10
}
]
But the consumer would like:
{"foo": "baa"}
{"key": 10}
Maybe you can parse your json into an Object[] and iterate on each elem ? like that :
public static void main(String[] args) {
String json = "[{\"foo\":\"baa\"},{\"key\":10}]";
Gson gson = new Gson();
Object yourObj[] = gson.fromJson(json, Object[].class);
Arrays.stream(yourObj).forEach(e -> {
System.out.println(e);
});
}

how to apply Json serialization or deserialization in request and response to HTTP method of rest api

I am very new to Rest api in java .My Question is how to directly convert json string request to java class object before post or get function ,like
json string : '{"id":3,"name":name}'
rest api post method :
#Post
public Something postData(Something obj) throws Exception {
}
so how to apply json serialization before request to this method.
right now i am converting it inside postData method.
You can use Jackson API to play with JSON.
For the Following JSON data the Java object mapping can be done as follows.
{
"id": 123,
"name": "Pankaj",
"permanent": true,
"address": {
"street": "Albany Dr",
"city": "San Jose",
"zipcode": 95129
},
"phoneNumbers": [
123456,
987654
],
"role": "Manager",
"cities": [
"Los Angeles",
"New York"
],
"properties": {
"age": "29 years",
"salary": "1000 USD"
}
}
import java.io.File;
import java.io.IOException;
import java.io.StringWriter;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.journaldev.jackson.model.Address;
import com.journaldev.jackson.model.Employee;
public class JacksonObjectMapperExample {
public static void main(String[] args) throws IOException {
//read json file data to String
byte[] jsonData = Files.readAllBytes(Paths.get("employee.txt"));
//create ObjectMapper instance
ObjectMapper objectMapper = new ObjectMapper();
//convert json string to object
Employee emp = objectMapper.readValue(jsonData, Employee.class);
System.out.println("Employee Object\n"+emp);
//convert Object to json string
Employee emp1 = createEmployee();
//configure Object mapper for pretty print
objectMapper.configure(SerializationFeature.INDENT_OUTPUT, true);
//writing to console, can write to any output stream such as file
StringWriter stringEmp = new StringWriter();
objectMapper.writeValue(stringEmp, emp1);
System.out.println("Employee JSON is\n"+stringEmp);
}
public static Employee createEmployee() {
Employee emp = new Employee();
emp.setId(100);
emp.setName("David");
emp.setPermanent(false);
emp.setPhoneNumbers(new long[] { 123456, 987654 });
emp.setRole("Manager");
Address add = new Address();
add.setCity("Bangalore");
add.setStreet("BTM 1st Stage");
add.setZipcode(560100);
emp.setAddress(add);
List<String> cities = new ArrayList<String>();
cities.add("Los Angeles");
cities.add("New York");
emp.setCities(cities);
Map<String, String> props = new HashMap<String, String>();
props.put("salary", "1000 Rs");
props.put("age", "28 years");
emp.setProperties(props);
return emp;
}
}
Source : http://www.journaldev.com/2324/jackson-json-processing-api-in-java-example-tutorial
You can use Gson or do a manually serialization/deserialization using JSONObject/JSONArray classes (example here). There are many other ways/libs to do this.

Categories