I have a Java class and i want to serialize it into CSV using jackson.
In addition i want to exclude from the csv a single field base on a external property.
I have tried using all features provides by jackson like Feature.IGNORE_UNKNOWN=true or #JsonIgnoreProperties(ignoreUnknown = true) on my data class, csvMapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) but none of them works. I still receive the exception column not found in schema when not all columns are declared in the schema. Code works fine in the other case.
Here i upload my implementation
public class MyClass{
#JsonProperty("A")
private string a;
#JsonProperty("B")
private string b;
#JsonProperty("C")
private string c;
}
CsvMapper mapper = new CsvMapper();
mapper.configure(Feature.IGNORE_UNKNOWN, true);
CsvSchema csvSchema;
if (disable = true) {
schema = CsvSchema.builder()
.addColumn("A")
.addColumn("C")
.build()
} else {
schema = CsvSchema.builder()
.addColumn("A")
.addColumn("B")
.addColumn("C")
.build()
}
ObjectWriter ow = mapper.writer(schema);
String csv = ow.writeValueAsString(list);
use #JsonIgnore on top of property.
Ex:
import com.fasterxml.jackson.annotation.JsonIgnore;
#Validated
#Data
#NoArgsConstructor
#AllArgsConstructor
#Getter
#Setter
#Builder
public class MyResponseModel {
...
#JsonIgnore
private String createBy;
...
}
Using Feature.IGNORE_UNKNOWN=true or #JsonIgnoreProperties(ignoreUnknown = true) will help when you want to deserialize JSON into your Java class.
For example, this will be useful when you're implementing the REST API and in the POST method dedicated to creating new objects, you want to ignore all invalid/incorrect fields sent to you by the user and process only the valid ones instead of returning the 400/500 error.
In your case, you just need to put the #JsonIgnore annotation on the field in your Java class, which you want to exclude during the serialization. This is an example of excluding the «a» property:
public class MyClass {
#JsonIgnore
#JsonProperty("A")
private String a;
#JsonProperty("B")
private String b;
#JsonProperty("C")
private String c;
}
This should also be useful in cases when you want to exclude some private and sensitive information from application logs, like passwords, PII, PHI, etc.
Updated:
This is the working solution for your case:
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
#Data
#Builder
#NoArgsConstructor
#AllArgsConstructor
public class MyClass {
#JsonProperty("A")
private String a;
#JsonProperty("B")
private String b;
#JsonProperty("C")
private String c;
}
public String test() throws JsonProcessingException {
boolean disable = false;
CsvMapper mapper = new CsvMapper();
mapper.configure(JsonGenerator.Feature.IGNORE_UNKNOWN, true);
CsvSchema csvSchema;
List<MyClass> list = List.of(MyClass.builder()
.a("a1")
.b("b1")
.c("c1")
.build()
);
if (disable) {
csvSchema = CsvSchema.builder()
.addColumn("A")
.addColumn("C")
.build();
} else {
csvSchema = CsvSchema.builder()
.addColumn("A")
.addColumn("B")
.addColumn("C")
.build();
}
ObjectWriter ow = mapper.writer(csvSchema);
String csv = ow.writeValueAsString(list);
return csv;
}
Dependency, which I've been using for tests and verification of the above code:
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-csv</artifactId>
<version>2.13.3</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.24</version>
</dependency>
Related
Before going to actual issue let me brief what i am looking for.
I am looking for encrypt and decrypt the fields inside entity. in JPA, we can use Attribute converter and achieve this. but in spring data jdbc its not supported it seems.
So, i am trying to use customconverstions feature of spring data jdbc. here i am creating one type like below
import lombok.AllArgsConstructor;
import lombok.EqualsAndHashCode;
import lombok.NoArgsConstructor;
#AllArgsConstructor
#NoArgsConstructor
#EqualsAndHashCode
public class EncryptionDataType {
private String value;
#Override public String toString() {
return value ;
}
}
in Pojo i will use this type as field.
#Table("EMPLOYEE")
#Builder
#AllArgsConstructor
#NoArgsConstructor
#EqualsAndHashCode(callSuper = true)
#Data
public class Employee
{
#Column("name")
private EncryptionDataType name;
#Id
private Integer id;
#Version
private Long version;
}
so from this i am expecting to save the 'EncryptionDataType' as normal string column in mysql for this i have created converter for read and write
#WritingConverter
public class EncryptionDataTypeWriteConverter implements Converter<EncryptionDataType, String> {
#Override public String convert(EncryptionDataType source) {
return source.toString()+"add";
}
}
#ReadingConverter
public class EncryptionDataTypeReadConverter implements Converter<String, EncryptionDataType> {
#Override public EncryptionDataType convert(String source) {
return new EncryptionDataType(source);
}
}
configuring these converts in configuration file.
#Configuration
public class MyConfig {
#Bean
protected JdbcCustomConversions JdbcConversion(Dialect dialect) {
return new JdbcCustomConversions(
Arrays.asList(new EncryptionDataTypeReadConverter(),new EncryptionDataTypeWriteConverter()));
}
}
This configurations seems not working. i am getting below error.
PreparedStatementCallback; bad SQL grammar [INSERT INTO `encryption_data_type` (`name`, `value`) VALUES (?, ?)]; nested exception is java.sql.SQLSyntaxErrorException: Table 'testschema.encryption_data_type' doesn't exist
seems instead of converting my encryptionDataType to string its trying to insert into new table. Please help me. am i missing anything ?
Updated configuration code:
#Configuration
#EnableJdbcRepositories(transactionManagerRef = "CustomJdbcTranasactionManager", jdbcOperationsRef = "CustomJdbcOperationsReference", repositoryFactoryBeanClass = CustomRepositoryFactoryBean.class, basePackages = {
"com.java.testy"
})
#EnableAutoConfiguration(exclude = {
org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration.class,
org.springframework.boot.autoconfigure.data.jdbc.JdbcRepositoriesAutoConfiguration.class
})
public class MyConfig {
#Bean
protected JdbcCustomConversions JdbcConversion(Dialect dialect) {
return new JdbcCustomConversions(
Arrays.asList(new EncryptionDataTypeReadConverter(),new EncryptionDataTypeWriteConverter()));
}
// creating beans for datasource,JdbcOperationsReference,JdbcTranasactionManager,JdbcConverter,JdbcMappingContext,DataAccessStrategy,JdbcAggregateTemplate
}
Make your configuration extend AbstractJdcbcConfiguration and overwrite jdbcConfiguration().
Just updating Jens Schauder's answer (I think it's just a typo - I would comment but don't have the rep):
Make your configuration extend AbstractJdcbcConfiguration and overwrite jdbcCustomConversions() (or possibly userConverters(), if that suits the purpose).
I have the the following JSON and the following code I am getting object the object but how to fetch one by one and how to set in POJO class?it is basically google direction map api response i want to store only legs and waypoints
{
"legs":[
{"end_address":"Uttam Nagar, Delhi, 110059, India",
"end_location":{
"lat":28.6195607,
"lng":77.0550097
},
"start_address":"Delhi, India",
"start_location":{
"lat":28.7040873,
"lng":77.1024072
}
}
]}
#PostMapping("/create")
public ResponseEntity<ResponseWrapper> savedRoutedata(#RequestBody MapRequest data) throws JsonProcessingException {
ObjectMapper mapper=new ObjectMapper();
String jsonContent = mapper.writeValueAsString(data);
JsonObject o=new JsonParser().parse(jsonContent).getAsJsonObject();
System.out.println("My Legs:-"+o.get("legs"));
}
#Data
#NoArgsConstructor
#AllArgsConstructor
#JsonIgnoreProperties(ignoreUnknown = true)
public class MapRequest {
#JsonProperty("legs")
private List<Legs> legs;
#JsonIgnoreProperties(ignoreUnknown = true)
#JsonProperty("overview_path")
private List<WaypointPath>wayPoint;
}
In case your POJOs are correct, your MapRequest should already have everything set and you can use them as any Java object, without the need to convert them however.
import java.util.List;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestBody;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
#Controller
public class MyTestController {
#GetMapping("/create")
public String savedRoutedata(#RequestBody MapRequest data) {
for(Legs leg : data.getLegs()) {
System.out.println(leg);
}
return null;
}
}
#Data
#NoArgsConstructor
#AllArgsConstructor
#JsonIgnoreProperties(ignoreUnknown = true)
class MapRequest {
#JsonProperty("legs")
private List<Legs> legs;
}
#Data
#NoArgsConstructor
#AllArgsConstructor
#JsonIgnoreProperties(ignoreUnknown = true)
class Legs {
#JsonProperty("end_address")
private String endAddress;
#JsonProperty("start_address")
private String startAddress;
#JsonProperty("end_location")
private Location endLocation;
#JsonProperty("start_location")
private Location startLocation;
}
#Data
#NoArgsConstructor
#AllArgsConstructor
#JsonIgnoreProperties(ignoreUnknown = true)
class Location {
#JsonProperty("lat")
private Double lat;
#JsonProperty("lng")
private Double lng;
}
I would like that my ParentClass has final fields, 'brokenChildList' list is wrapped xml element and list items have different tag than the list (<brokenChildList><brokenChild/></brokenChildList>).
Here is a snippet of code to reproduce my issues (imports are partially truncated, setters and getters omitted)
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.xml.XmlMapper;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlElementWrapper;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlProperty;
public class Main {
public static void main(String... args) throws IOException {
ObjectMapper xmlMapper = new XmlMapper();
String xmlString = "<ParentClass><childClass name=\"name1\" value=\"val1\"/><brokenChildList><brokenChild name=\"bc1\" reason=\"bc-val1\"/><brokenChild name=\"bc2\" reason=\"bc-val2\"/></brokenChildList></ParentClass>";
ParentClass parentClass = xmlMapper.readValue(xmlString, ParentClass.class);
StringWriter stringWriter = new StringWriter();
xmlMapper.writeValue(stringWriter, parentClass);
String serialised = stringWriter.toString();
System.out.println(serialised);
System.out.println(xmlString.equals(serialised));
}
public static class ChildClass {
#JacksonXmlProperty(isAttribute = true)
private String name;
#JacksonXmlProperty(isAttribute = true)
private String value;
//getters & setters
}
public static class BrokenChild {
#JacksonXmlProperty(isAttribute = true)
private String name;
#JacksonXmlProperty(isAttribute = true)
private String reason;
//getters & setters
}
public static class ParentClass {
private final ChildClass childClass;
private final List<BrokenChild> brokenChildList;
#JsonCreator
public ParentClass(
#JsonProperty("childClass") ChildClass childClass,
#JsonProperty("brokenChildList") List<BrokenChild> brokenChildList
) {
this.childClass = childClass;
this.brokenChildList = brokenChildList;
}
#JacksonXmlProperty(localName = "childClass")
public ChildClass getChildClass() {
return childClass;
}
#JacksonXmlElementWrapper(localName = "brokenChildList")
#JacksonXmlProperty(localName = "brokenChild")
public List<BrokenChild> getBrokenChildList() {
return brokenChildList;
}
}
}
The above code gives output with Jackson version 2.8.10:
<ParentClass><childClass name="name1" value="val1"/><brokenChildList><brokenChild name="bc1" reason="bc-val1"/><brokenChild name="bc2" reason="bc-val2"/></brokenChildList></ParentClass>
true
With Jackson version 2.9.0 it gives:
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Duplicate property 'brokenChildList' for [simple type, class org.test.Main$ParentClass]
at [Source: (StringReader); line: 1, column: 1]
I would like to find a solution (and any version after 2.9.0) that will give same output with the attached code.
My failed attempts include:
Replacing #JacksonXmlElementWrapper(localName = "brokenChildList") with #JacksonXmlElementWrapper will rename wrapper element as 'brokenChild' which is undesirable.
Removing #JacksonXmlElementWrapper(localName = "brokenChildList") will rename wrapper element as 'brokenChild' which is undesirable.
This problem is really tricky because Jackson collects metadata from different places: fields, getters, setters, constructor parameters. Also, you can use MixIn but in your case it does not appear.
#JacksonXmlElementWrapper annotation can be attached to FIELD and METHOD type elements and this forces you to declare it on getter. Because ParentClass is immutable and you want to build it with constructor we need to annotate constructor parameters as well. And this is where collision appears: you have a constructor parameter with #JsonProperty("brokenChildList") annotation and getter with #JacksonXmlElementWrapper(localName = "brokenChildList") which reuses the same name. If you would changed localName to #JacksonXmlElementWrapper(localName = "brokenChildListXYZ") (added XYZ) everything would be deserialised and serialised but output would be different then input.
To solve this problem, we can use com.fasterxml.jackson.databind.deser.BeanDeserializerModifier class which allows to filter out fields we do not want to use for deserialisation and which creates collision. Example usage:
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.BeanDescription;
import com.fasterxml.jackson.databind.DeserializationConfig;
import com.fasterxml.jackson.databind.deser.BeanDeserializerModifier;
import com.fasterxml.jackson.databind.introspect.BeanPropertyDefinition;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.dataformat.xml.XmlMapper;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlElementWrapper;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlProperty;
import java.io.IOException;
import java.io.StringWriter;
import java.util.List;
import java.util.stream.Collectors;
public class XmlMapperApp {
public static void main(String... args) throws IOException {
SimpleModule module = new SimpleModule();
module.setDeserializerModifier(new BeanDeserializerModifier() {
#Override
public List<BeanPropertyDefinition> updateProperties(DeserializationConfig config, BeanDescription beanDesc, List<BeanPropertyDefinition> propDefs) {
if (beanDesc.getBeanClass() == ParentClass.class) {
return propDefs.stream().filter(p -> p.getConstructorParameter() != null).collect(Collectors.toList());
}
return super.updateProperties(config, beanDesc, propDefs);
}
});
XmlMapper xmlMapper = XmlMapper.xmlBuilder()
.addModule(module)
.build();
//yours code
}
}
To create this example I used version 2.10.0.
See also:
Jackson 2.10 features
Jackson Release 2.10
I have two classes. RequestDTO and Entity. I want to map RequestDTO to the Entity. In that case, I want to insert one of the Entity property manually which means that property is not in the Request DTO. How to achieve this using modelmapper.
public class RequestDTO {
private String priceType;
private String batchType;
}
public class Entity {
private long id;
private String priceType;
private String batchType;
}
Entity newEntity = modelMapper.map(requestDto, Entity.class);
But this does not work, it says it can't convert string to long. I request a solution to this or a better approach to this.
If you want to perform the mapping manually (ideally for dissimilar objects)
You can check the documentation for dissimilar object mapping Property Mapping,
You can define a property mapping by using method references to match
a source getter and destination setter.
typeMap.addMapping(Source::getFirstName, Destination::setName);
The source and destination types do not need to match.
typeMap.addMapping(Source::getAge, Destination::setAgeString);
If you don't want to do the mapping field by field to avoid boilerplate code
you can configure a skip mapper, to avoid mapping certain fields to your destination model:
modelMapper.addMappings(mapper -> mapper.skip(Entity::setId));
I've created a test for your case and the mapping works for both side without configuring anything :
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.junit.Before;
import org.junit.Test;
import org.modelmapper.ModelMapper;
import static junit.framework.TestCase.assertEquals;
import static junit.framework.TestCase.assertNotNull;
public class ModelMapperTest {
private ModelMapper modelMapper;
#Before
public void beforeTest() {
this.modelMapper = new ModelMapper();
}
#Test
public void fromSourceToDestination() {
Source source = new Source(1L, "Hello");
Destination destination = modelMapper.map(source, Destination.class);
assertNotNull(destination);
assertEquals("Hello", destination.getName());
}
#Test
public void fromDestinationToSource() {
Destination destination = new Destination("olleH");
Source source = modelMapper.map(destination, Source.class);
assertNotNull(source);
assertEquals("olleH", destination.getName());
}
}
#Data
#NoArgsConstructor
#AllArgsConstructor
class Source {
private Long id;
private String name;
}
#Data
#NoArgsConstructor
#AllArgsConstructor
class Destination {
private String name;
}
I am trying to let Jackson ignore some properties of a DTO, however I can't seem to be able to.
I have a fairly big project with many dependencies (Lombok, Spring, GWT, Gemfire and many more),
and I can't change these dependencies (maybe I can change versions, but it's not my call).
I have prepared a test case, here it is:
this is my test dto class, it has a map that is only useful
server side. A copy of this dto is serialized to be sent to gwt
to display (the implementation is not complete, only the relevant parts
are shown).
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonIgnoreType;
import lombok.*;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
#Getter
#Setter
#Builder
#NoArgsConstructor
#AllArgsConstructor
#EqualsAndHashCode(of = "id", callSuper = true)
public class MyClass extends MyAbstractClass {
#Getter
#Setter
#Builder
public static class AValueClass {
int someInt;
String SomeString;
}
#Data
#AllArgsConstructor
#NoArgsConstructor
#JsonIgnoreType
public static class MyJsonIgnoreKeyClass {
protected Integer anInt;
protected String aString;
}
#JsonIgnore
#Getter(AccessLevel.NONE) #Setter(AccessLevel.NONE)
private transient Map<MyJsonIgnoreKeyClass, List<AValueClass>> aMapThatJacksonShouldIgnore = new HashMap<>();
public void addToMap(MyJsonIgnoreKeyClass key, AValueClass value) {
List<AValueClass> valueList = aMapThatJacksonShouldIgnore.get(key);
if(valueList == null) {
valueList = new ArrayList<>();
}
valueList.add(value);
aMapThatJacksonShouldIgnore.put(key,valueList);
}
public boolean noMap() {
return aMapThatJacksonShouldIgnore == null || aMapThatJacksonShouldIgnore.keySet().isEmpty();
}
public void nullifyMap() {
aMapThatJacksonShouldIgnore = null;
}
// other methods operating on maps omitted
}
the test model inherits some fields from a superclass
import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.Setter;
import java.util.Date;
#Setter
#Getter
#EqualsAndHashCode(of = "id")
public class MyAbstractClass {
protected String id;
protected Date aDay;
}
here are the unit tests I've prepared
public class MyClassJacksonTest {
ObjectMapper om;
#Before
public void setUp() throws Exception {
om = new ObjectMapper().registerModule(new Jdk8Module());
om.setSerializationInclusion(JsonInclude.Include.NON_NULL);
om.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
}
#Test
public void testWithMapValues() throws Exception {
MyClass testClass = new MyClass();
testClass.setADay(new Date());
testClass.setId(UUID.randomUUID().toString());
testClass.addToMap(
new MyClass.MyJsonIgnoreKeyClass(1,"test"),
new MyClass.AValueClass(1,"test"));
StringWriter writer = new StringWriter();
om.writeValue(writer,testClass);
writer.flush();
String there = writer.toString();
MyClass andBackAgain = om.readValue(there, MyClass.class);
assertTrue(andBackAgain.noMap());
}
#Test
public void testWithEmptyMaps() throws Exception {
MyClass testClass = new MyClass();
testClass.setADay(new Date());
testClass.setId(UUID.randomUUID().toString());
StringWriter writer = new StringWriter();
om.writeValue(writer,testClass);
writer.flush();
String there = writer.toString();
MyClass andBackAgain = om.readValue(there, MyClass.class);
assertTrue(andBackAgain.noMap());
}
#Test
public void testWithNullMaps() throws Exception {
MyClass testClass = new MyClass();
testClass.setADay(new Date());
testClass.setId(UUID.randomUUID().toString());
testClass.nullifyMap();
StringWriter writer = new StringWriter();
om.writeValue(writer,testClass);
writer.flush();
String there = writer.toString();
MyClass andBackAgain = om.readValue(there, MyClass.class);
assertTrue(andBackAgain.noMap());
}
}
All of the tests are failing with
com.fasterxml.jackson.databind.JsonMappingException: Can not find a (Map) Key deserializer for type [simple type, class MyClass$MyJsonIgnoreKeyClass]
So the questions are :
Why Jackson tries to find a deserializer for the keys of a map that can't be accessed (since there are no getter and setter) and that is annotated with #JsonIgnore?
More importantly, how can I tell it not to search for the deserializers?
These are the relevant dependencies on my pom, if it can be of any help :
<properties>
<!-- ... -->
<jackson.version>2.7.4</jackson.version>
</properties>
<dependencies>
<!-- other dependencies omitted -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-jsonSchema</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr353</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
<version>${jackson.version}</version>
<exclusions>
<exclusion>
<groupId>org.codehaus.woodstox</groupId>
<artifactId>stax2-api</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
It turns out that this is a case of bad interaction between Lombok and Jackson.
The Lombok annotation #AllArgsConstructor generates a constructor that is annotated with #ConstructorProperties, which in turn lists all the properties that are declared in the class.
This is then used by Jackson when the default deserializer is to be used.
In this case, the absence of setters and getters and the presence of #JsonIgnore annotations is not taken into account.
The solution is simply to specify the #AllArgsConstructor with the attribute suppressConstructorProperties set to true :
#Getter
#Setter
#Builder
#NoArgsConstructor
#AllArgsConstructor(suppressConstructorProperties = true)
#EqualsAndHashCode(of = "id", callSuper = true)
public class MyClass extends MyAbstractClass {
// everything else is unchanged
Tricky one, indeed. What I think is happening is that you are generating a public all arguments constructor with Lombok. When deserializing, that is the one that Jackson will try to use. If you change your annotation for MyClass to
#AllArgsConstructor(access = AccessLevel.PRIVATE)
... it should work fine. Good luck!