We Keep Coding

With default typing enabled, is there a way to override the type Jackson outputs?

I am serializing a class that includes an unmodifiable list with default typing enabled. The problem is that the type that Jackson uses is
java.util.Collections$UnmodifiableRandomAccessList
which, for some reason, the deserializer does not know how to handle.
Is there a way to tell Jackson to set the type as
java.util.ArrayList
which the deserializer does know how to handle, instead? If possible, I'd like to do it using mixins.
Something like
public abstract class ObjectMixin {
#JsonCreator
public ObjectMixin(
#JsonProperty("id") String id,
#JsonProperty("list") #JsonSerialize(as = ArrayList.class) List<String> list;
) {}
}
which, unfortunately, does not work.

I would like to start from security risk warning which comes from ObjectMapper documentation:
Notes on security: use "default typing" feature (see
enableDefaultTyping()) is a potential security risk, if used with
untrusted content (content generated by untrusted external parties).
If so, you may want to construct a custom TypeResolverBuilder
implementation to limit possible types to instantiate, (using
setDefaultTyping(com.fasterxml.jackson.databind.jsontype.TypeResolverBuilder<?)).
Lets implement custom resolver:
class CollectionsDefaultTypeResolverBuilder extends ObjectMapper.DefaultTypeResolverBuilder {
private final Map<String, String> notValid2ValidIds = new HashMap<>();
public CollectionsDefaultTypeResolverBuilder() {
super(ObjectMapper.DefaultTyping.OBJECT_AND_NON_CONCRETE);
this._idType = JsonTypeInfo.Id.CLASS;
this._includeAs = JsonTypeInfo.As.PROPERTY;
notValid2ValidIds.put("java.util.Collections$UnmodifiableRandomAccessList", ArrayList.class.getName());
// add more here...
}
#Override
protected TypeIdResolver idResolver(MapperConfig<?> config, JavaType baseType, Collection<NamedType> subtypes,
boolean forSer, boolean forDeser) {
return new ClassNameIdResolver(baseType, config.getTypeFactory()) {
#Override
protected String _idFrom(Object value, Class<?> cls, TypeFactory typeFactory) {
String id = notValid2ValidIds.get(cls.getName());
if (id != null) {
return id;
}
return super._idFrom(value, cls, typeFactory);
}
};
}
}
Now, we can use it as below:
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.cfg.MapperConfig;
import com.fasterxml.jackson.databind.jsontype.NamedType;
import com.fasterxml.jackson.databind.jsontype.TypeIdResolver;
import com.fasterxml.jackson.databind.jsontype.impl.ClassNameIdResolver;
import com.fasterxml.jackson.databind.type.TypeFactory;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class JsonApp {
public static void main(String[] args) throws Exception {
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
mapper.setDefaultTyping(new CollectionsDefaultTypeResolverBuilder());
Root root = new Root();
root.setData(Collections.unmodifiableList(Arrays.asList("1", "b")));
String json = mapper.writeValueAsString(root);
System.out.println(json);
System.out.println(mapper.readValue(json, Root.class));
}
}
class Root {
private List<String> data;
public List<String> getData() {
return data;
}
public void setData(List<String> data) {
this.data = data;
}
#Override
public String toString() {
return "Root{" +
"data=" + data +
'}';
}
}
Above code prints:
{
"data" : [ "java.util.ArrayList", [ "1", "b" ] ]
}
Root{data=[1, b]}
You can even map it to List interface:
notValid2ValidIds.put("java.util.Collections$UnmodifiableRandomAccessList", List.class.getName());
And output would be:
{
"data" : [ "java.util.List", [ "1", "b" ] ]
}

Mask json fields using jackson

I am trying to mask sensitive data while serializing using jackson.
I have tried using #JsonSerialize and a custom annotation #Mask .
Mask.java
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.FIELD)
public #interface Mask {
String value() default "XXX-DEFAULT MASK FORMAT-XXX";
}
Employee.java
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import java.util.Map;
public class Employee {
#Mask(value = "*** The value of this attribute is masked for security reason ***")
#JsonSerialize(using = MaskStringValueSerializer.class)
protected String name;
#Mask
#JsonSerialize(using = MaskStringValueSerializer.class)
protected String empId;
#JsonSerialize(using = MaskMapStringValueSerializer.class)
protected Map<Category, String> categoryMap;
public Employee() {
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getEmpId() {
return empId;
}
public void setEmpId(String empId) {
this.empId = empId;
}
public Map<Category, String> getCategoryMap() {
return categoryMap;
}
public void setCategoryMap(Map<Category, String> categoryMap) {
this.categoryMap = categoryMap;
}
}
Category.java
public enum Category {
#Mask
CATEGORY1,
#Mask(value = "*** This value of this attribute is masked for security reason ***")
CATEGORY2,
CATEGORY3;
}
MaskMapStringValueSerializer.java
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import java.io.IOException;
import java.util.Map;
public class MaskMapStringValueSerializer extends JsonSerializer<Map<Category, String>> {
#Override
public void serialize(Map<Category, String> map, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
jsonGenerator.writeStartObject();
for (Category key : map.keySet()) {
Mask annot = null;
try {
annot = key.getClass().getField(key.name()).getAnnotation(Mask.class);
} catch (NoSuchFieldException e) {
e.printStackTrace();
}
if (annot != null) {
jsonGenerator.writeStringField(((Category) key).name(), annot.value());
} else {
jsonGenerator.writeObjectField(((Category) key).name(), map.get(key));
}
}
jsonGenerator.writeEndObject();
}
}
MaskStringValueSerializer.java
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.BeanProperty;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.ser.ContextualSerializer;
import com.fasterxml.jackson.databind.ser.std.StdSerializer;
import java.io.IOException;
public class MaskStringValueSerializer extends StdSerializer<String> implements ContextualSerializer {
private Mask annot;
public MaskStringValueSerializer() {
super(String.class);
}
public MaskStringValueSerializer(Mask logMaskAnnotation) {
super(String.class);
this.annot = logMaskAnnotation;
}
public void serialize(String s, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
if (annot != null && s != null && !s.isEmpty()) {
jsonGenerator.writeString(annot.value());
} else {
jsonGenerator.writeString(s);
}
}
public JsonSerializer<?> createContextual(SerializerProvider serializerProvider, BeanProperty beanProperty) throws JsonMappingException {
Mask annot = null;
if (beanProperty != null) {
annot = beanProperty.getAnnotation(Mask.class);
}
return new MaskStringValueSerializer(annot);
}
}
MaskValueTest.java
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.HashMap;
import java.util.Map;
public class MaskValueTest {
public static void main(String args[]) throws Exception{
Employee employee = new Employee();
employee.setName("John Doe");
employee.setEmpId("1234567890");
Map<Category, String> catMap = new HashMap<>();
catMap.put(Category.CATEGORY1, "CATEGORY1");
catMap.put(Category.CATEGORY2, "CATEGORY2");
catMap.put(Category.CATEGORY3, "CATEGORY3");
employee.setCategoryMap(catMap);
ObjectMapper mapper = new ObjectMapper();
System.out.println(mapper.writerWithDefaultPrettyPrinter().writeValueAsString(employee));
}
}
Output -
{
"name" : "*** The value of this attribute is masked for security reason ***",
"empId" : "XXX-DEFAULT MASK FORMAT-XXX",
"categoryMap" : {
"CATEGORY1" : "XXX-DEFAULT MASK FORMAT-XXX",
"CATEGORY2" : "*** The value of this attribute is masked for security reason ***",
"CATEGORY3" : "CATEGORY3"
}
}
The result is as per expectation, however, this seems to be static masking.
The intention was to mask only when needed, e.g. while printing in the logs where the all these sensitive data should be masked.
If I have to send this json for document indexing where the values should be as it is, this implementation fails.
I am looking for an Annotation based solution, where I can use 2 different instance of ObjectMapper initialized with JsonSerializers.

This can be an implementation for what Andreas suggested:
create a class MaskAnnotationIntrospector which extend from JacksonAnnotationIntrospector and override its findSerializer method, like this:
public class MaskAnnotationIntrospector extends JacksonAnnotationIntrospector {
#Override
public Object findSerializer(Annotated am) {
Mask annotation = am.getAnnotation(Mask.class);
if (annotation != null)
return MaskingSerializer.class;
return super.findSerializer(am);
}
}
Therefore, you can have two instance of ObjectMapper. Add MaskAnnotationIntrospector to the one in which you want to Mask (e.g. for logging purpose):
mapper.setAnnotationIntrospector(new MaskAnnotationIntrospector());
The other instance which MaskAnnotationIntrospector has not set into it, do not mask any during serialization.
P.S. MaskAnnotationIntrospector can be extended from both JacksonAnnotationIntrospector & NopAnnotationIntrospector, but the latter does not provide any implementation for findSerializer method and calling super.findSerializer(am) simply return null and as a direct result, other Jackson annotation (such as #JsonIgnore) discarded, but by using the former, this problem solved

Remove the #JsonSerialize annotations, and put the logic of how to handle the #Mask annotation in a Module, e.g. have it add an AnnotationIntrospector.
You can now choose whether or not to call registerModule(Module module).
As for writing the module, I'll leave that up to you. If you have any questions about that, ask another Question.

Instead of having MaskStringValueSerializer.java you can create module to bundle the serializer and register the module with objectmapper whenever you want , which will eventually allow you to have two different instances of objectmapper.
Create a module to bundle the serializer
public class MaskingModule extends SimpleModule {
private static final String NAME = "CustomIntervalModule";
private static final VersionUtil VERSION_UTIL = new VersionUtil() {};
public MaskingModule() {
super(NAME, VERSION_UTIL.version());
addSerializer(MyBean.class, new MaskMapStringValueSerializer());
}
}
Register the module with ObjectMapper and use it
ObjectMapper objectMapper = new ObjectMapper().registerModule(new MaskingModule());
System.out.println(objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(employee));
Also you can extend the Object Mapper , register the module and use it
public class CustomObjectMapper extends ObjectMapper {
public CustomObjectMapper() {
registerModule(new MaskingModule());
}
}
CustomObjectMapper customObjectMapper = new CustomObjectMapper ();
System.out.println(customObjectMapper .writerWithDefaultPrettyPrinter().writeValueAsString(employee));

why don't you use two parameters one for original value and one for masked value. For example in this case you can use String name and String maskedName. then for logging you can use masked value.

fastjson can't correctly transfer complex object or the cast I use is wrong?

fastjson can't correctly transfer complex object or the cast I use is wrong.
I find a question in my project.so write a test like this:
import com.alibaba.fastjson.JSONObject;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/**
*/
public class Test {
public static void main(String[] args) {
Map<Integer,List<String>> map1 = new HashMap<>();
List<String> list1 = new ArrayList<>();
list1.add("555");
map1.put(5,list1);
System.out.println(map1.get("5"));
Pack pack1 = new Pack();
pack1.setT(1);
pack1.setInfo(map1);
String jsonStr1 = JSONObject.toJSONString(pack1);
Pack pack2 = JSONObject.parseObject(jsonStr1,Pack.class);
Object ob1 = pack2.getInfo();
Map<Integer,List<String>> map2 = (Map<Integer, List<String>>)ob1;
List<String> list2 = map2.get(5);
System.out.println(list2 == null);
}
static class Pack{
Integer t;
Object info;
public Object getInfo() {
return info;
}
public void setInfo(Object info) {
this.info = info;
}
public Integer getT() {
return t;
}
public void setT(Integer t) {
this.t = t;
}
}
}
I can't understand why the list2 is null when the debug shows like the picture
the debug info is :

Getting the value of parameters passed in #Factory annotated class using TestNG

Is it possible to get the value of parameters that are used to initialize a #Factory annotated test class from any of the ITestListener, ISuiteListener or any other listener methods?
Below is a sample test class. My intention is to get the value of the class init parameter 'value' using any of the listener methods, possibly.
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import org.testng.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
import org.testng.reporters.EmailableReporter2;
#Listeners({ TestExecutionListener.class, EmailableReporter2.class })
public class TestClass {
private int value;
#Factory(dataProvider = "data", dataProviderClass = TestClass.class)
public TestClass(final int value) {
this.value = value;
}
#Test(alwaysRun = true)
public void testOdd() {
Assert.assertTrue(value % 2 != 0);
}
#Test(alwaysRun = true)
public void testEven() {
Assert.assertTrue(value % 2 == 0);
}
#DataProvider(name = "data")
public static Iterator<Object[]> data() {
List<Object[]> list = new ArrayList<>();
for (int i = 0; i < 2; i++) {
list.add(new Object[] { i });
}
return list.iterator();
}
}
Thanks in advance.

You can have access to your object from ITestResult#getInstance().
You'll just have to cast to object in the appropriate type (TestClass) and add a getter for value (or change its visibility).
ITestResult is available in many listeners.

How do you serialize Guava's immutable collections using Protostuff?

I use protostuff-runtime to serialize object graphs. Some of these objects have reference to Guava immutable collections, such as ImmutableList and ImmutableSet. Protostuff is unable to deserialize these collections out of the box, because it tries to construct an instance and then "add" elements to it from the inputStream (which fails, since the collections are immutable).
Do you know of any library / protostuff plugin that does that out of the box? If not, is there a best practice to do this myself?
I've investigated, and found that protostuff has a concept of "delegate", that lets you take control of the serialization for specific types. It seems to be the answer to my problem, but I can't seem to get it working.
Here is what I have right now:
import com.dyuproject.protostuff.Schema;
import com.dyuproject.protostuff.runtime.RuntimeSchema;
import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
import javax.annotation.Nonnull;
import javax.annotation.concurrent.Immutable;
/**
* This is the POJO I want to serialize. Note that the {#code strings} field refers to an {#link ImmutableList}.
*/
#Immutable
public class Foo {
public static final Schema<Foo> SCHEMA = RuntimeSchema.getSchema(Foo.class);
#Nonnull
private final ImmutableList<String> strings;
public Foo(ImmutableList<String> strings) {
this.strings = Preconditions.checkNotNull(strings);
}
#Nonnull
public ImmutableList<String> getStrings() {
return strings;
}
#Override
public boolean equals(Object obj) {
if (obj instanceof Foo) {
Foo that = (Foo) obj;
return this.strings.equals(that.strings);
}
return false;
}
#Override
public int hashCode() {
return strings.hashCode();
}
}
import com.dyuproject.protostuff.*;
import com.dyuproject.protostuff.runtime.Delegate;
import com.dyuproject.protostuff.runtime.RuntimeSchema;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Lists;
import java.io.IOException;
import java.util.ArrayList;
public class ImmutableListDelegate implements Delegate<ImmutableList<?>> {
private static final Schema<ArrayList> LIST_SCHEMA = RuntimeSchema.getSchema(ArrayList.class);
#Override
public WireFormat.FieldType getFieldType() {
return WireFormat.FieldType.MESSAGE;
}
#Override
public ImmutableList<?> readFrom(Input input) throws IOException {
ArrayList<?> list = LIST_SCHEMA.newMessage();
input.mergeObject(list, LIST_SCHEMA);
return ImmutableList.copyOf(list);
}
#Override
public void writeTo(Output output, int number, ImmutableList<?> value, boolean repeated) throws IOException {
ArrayList<?> list = Lists.newArrayList(value);
output.writeObject(number, list, LIST_SCHEMA, repeated);
LIST_SCHEMA.writeTo(output, list);
}
#Override
public void transfer(Pipe pipe, Input input, Output output, int number, boolean repeated) throws IOException {
throw new UnsupportedOperationException("TODO");
}
#Override
public Class<?> typeClass() {
return ImmutableList.class;
}
}
import com.dyuproject.protostuff.LinkedBuffer;
import com.dyuproject.protostuff.ProtostuffIOUtil;
import com.dyuproject.protostuff.runtime.DefaultIdStrategy;
import com.dyuproject.protostuff.runtime.RuntimeEnv;
import com.google.common.collect.ImmutableList;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
public class ImmutableListDelegateTest {
#Before
public void before() {
// registers the delegate
if (RuntimeEnv.ID_STRATEGY instanceof DefaultIdStrategy) {
((DefaultIdStrategy) RuntimeEnv.ID_STRATEGY).registerDelegate(new ImmutableListDelegate());
}
}
#Test
public void testDelegate() throws IOException {
Foo foo = new Foo(ImmutableList.of("foo"));
Assert.assertEquals(foo, serializeThenDeserialize(foo));
}
private Foo serializeThenDeserialize(Foo fooToSerialize) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
ProtostuffIOUtil.writeDelimitedTo(out, fooToSerialize, Foo.SCHEMA, buffer());
ByteArrayInputStream in = new ByteArrayInputStream(out.toByteArray());
Foo fooDeserialized = Foo.SCHEMA.newMessage();
ProtostuffIOUtil.mergeDelimitedFrom(in, fooDeserialized, Foo.SCHEMA, buffer());
return fooDeserialized;
}
private LinkedBuffer buffer() {
return LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
}
}
The test fails with the following exception, which seems to mean that my delegate only deserializes null values:
java.lang.NullPointerException
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
at com.google.common.collect.SingletonImmutableList.<init>(SingletonImmutableList.java:40)
at com.google.common.collect.ImmutableList.asImmutableList(ImmutableList.java:305)
at com.google.common.collect.ImmutableList.copyFromCollection(ImmutableList.java:314)
at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:253)
at test.ImmutableListDelegate.readFrom(ImmutableListDelegate.java:25)
at test.ImmutableListDelegate.readFrom(ImmutableListDelegate.java:12)
at com.dyuproject.protostuff.runtime.RuntimeUnsafeFieldFactory$19$1.mergeFrom(RuntimeUnsafeFieldFactory.java:1111)
at com.dyuproject.protostuff.runtime.MappedSchema.mergeFrom(MappedSchema.java:188)
at com.dyuproject.protostuff.IOUtil.mergeDelimitedFrom(IOUtil.java:109)
at com.dyuproject.protostuff.ProtostuffIOUtil.mergeDelimitedFrom(ProtostuffIOUtil.java:151)
at test.ImmutableListDelegateTest.serializeThenDeserialize(ImmutableListDelegateTest.java:38)
at test.ImmutableListDelegateTest.testDelegate(ImmutableListDelegateTest.java:30)
Is this the right approach? What am I missing?
This is not a duplicate of the What is a Null Pointer Exception, and how do I fix it? question, which makes no sense. The fact that I mentioned that an NPE is thrown when trying to use a Protostuff delegate to de-serialize immutable collections, doesn't mean that this duplicates the "What is a NPE?" question in any way, shape, or form.

Everything looks fine and the
java.lang.NullPointerException
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
at com.google.common.collect.SingletonImmutableList.<init>(SingletonImmutableList.java:40)
says that you're trying to put null into an ImmutableList, which is forbidden. To be sure, inspect you list just before the failing line. Make sure your input json doesn't look like [null].