Trying to create a generic Kafka consumer factory - java

I'm trying to implement a generic Kafka consumer factory. By generic I mean that the factory will receive generics K,V that will indicate the types of the key and the value of the consumed message.
public class KafkaConsumerFactory<K, V> {
private Properties properties;
In my constructor I'm calling a method that responsible for validating that the generics K,V are compatible with the deseralization classes :
public KafkaConsumerFactory(Properties p) {
....
....
validateGenericsWithDeserializationProperties();
}
The relevant function :
private void validateGenericsWithDeserializationProperties() throws Exception {
try {
String keyDeserializerClass = (String) (this.properties.get(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG));
String valueDeserializerClass = (String) (this.properties.get(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG));
Deserializer<K> keyDeserializer = (Deserializer<K>) Class.forName(keyDeserializerClass).getConstructor().newInstance();
Deserializer<V> valueDeserializer = (Deserializer<V>) Class.forName(valueDeserializerClass ).getConstructor().newInstance();
System.out.println(keyDeserializer.toString());
System.out.println(valueDeserializer.toString());
} catch (Exception e) {
throw new Exception("There is an incompatibility with the deserialization parameters in the properties with the generics <K,V>");
}
}
But it seems that when I pass generics K,V that aren't compatible with the deserializer class the code doesn't throw exception , for example:
K=String,
V=Long,
key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
value.deserializer=org.apache.kafka.common.serialization.StringDeserializer
I was expecting an exception on the downcasting on the following lines :
Deserializer<K> keyDeserializer = (Deserializer<K>)(Class.forName(keyDeserializerClass).getConstructor().newInstance());
Deserializer<V> valueDeserializer = (Deserializer<V>)Class.forName(valueDeserializerClass).getConstructor().newInstance();
I understand that those lines won't throw exception because of erasure of generic types.
Is there any other way to check for incompatibility between the deserialization class to the type of generic before trying to consume data?

Deserializer<K> keyDeserializer = (Deserializer<K>) Class.forName(keyDeserializerClass).getConstructor().newInstance();
Deserializer<V> valueDeserializer = (Deserializer<V>) Class.forName(valueDeserializerClass ).getConstructor().newInstance();
Why do you want to explicity create Deserializer object?
It will automatically be created as per the one specified in the properties.
If you want to verify (make sure) the appropriate deserializer is put for the appropriate key/value type, you can take the keyClass, valueClass in the constructor of your KafkaConsumerFactory
public KafkaConsumerFactory(Properties p, Class<K> keyClass, Class<V> valueClass) {
p.putIfAbsent("key.deserializer", getDeserializer(keyClass).getName());
p.putIfAbsent("value.deserializer", getDeserializer(valueClass).getName());
}
and then you can automatically map the keyClass, valueClass to appropriate deserializer and override them in the properties.
Class<Deserializer> getDeserializer(Class clazz) {
if(Long.class.equals(clazz)) { return LongDeserializer.class; }
else if (...)
...
}
If you want to support new custom deserializers, you can use an interface like CustomDeserializer for example that has a method called canBeProcessed(Class clazz) which determines if the object can be deserialized and ensure that this interface is implemented by all your custom deserializers and that the method is overriden.
If you are looking to support already existing deserializers, you can write the code to check the compatability of the deserializer with a certain class in your KafkaConsumerFactory itself.
Usually, serializers/deserializers are mapped to certain class(es), which means that you don't need an object to verify if it can be serialized/deserialized because the data in the object has pretty less (if not nothing) to do with the actual serialization/deserialization.
In case, if the class type information isn't enough for finding out whether it's object can be deserialized or not, the option left is to create a dummy object and then pass it to the deserialize() method. If it raises an exception, then it is invalid, not otherwise.

Related

What is the Java idiomatic way to deduce generic object class based on string/enum or class property?

I'm using an API from an external Maven dependency with the following signature:
public <T> T get(String key, Class<T> responseType)
the API returns an object of type T for a given key from a key value store.
In my application there're several object types which can be returned, for example: Product, Customer etc.
I'd like to wrap the external API into my service which will receive a key and will return the object found. I can't figure out though how to return the object type from the service. Below is my attempt:
public class MyService {
public <T> T get(String key, String objType) {
Class clazz = null;
if (objType == "Customer") {
clazz = Customer.class;
} else if (objType == "Product") {
clazz = Product.class;
}
return externalApi.get(key, clazz); // doesn't compile
}
}
This code doesn't compile because of Incompatible types: Object is not convertible to T error.
How can I properly pass responseType to externalApi.get and return the correct type without reflection?
As the OP may have guessed, this is inherently impossible.
If the call site for get could do anything useful to preserve the returned type, T, then it would know the type anyway and could supply the correct class (providing this is transitively propagated through call sites).
(Also note, the code uses == for String instead of equals or switch.)

How to add a parameterized super interface in JavaPoet?

I am writing an annotation processor that generates JSON serialization code. Here's my annotation that I use to identify the POJOs that need a serializer
#Target(ElementType.TYPE)
#Retention(RetentionPolicy.SOURCE)
public #interface JsonSerialize {
}
And here's the base interface of my serializer
public interface JsonSerializer<T> {
String serialize(T t);
}
Here's the annotation processor code that looks for that annotation and generates the serializer code
#Override
public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
for (Element element : roundEnv.getElementsAnnotatedWith(JsonSerialize.class)) {
if (element.getKind() == ElementKind.CLASS) {
MethodSpec serializeMethod = MethodSpec
.methodBuilder("serialize")
.addModifiers(Modifier.PUBLIC)
.addParameter(ParameterSpec.builder(TypeName.get(element.asType()), "obj", Modifier.FINAL).build())
.returns(String.class)
.addStatement("return \"dummy string\"")
.build();
TypeSpec serializer = TypeSpec
.classBuilder(element.getSimpleName().toString() + "JsonSerializer")
.addSuperinterface(JsonSerializer.class) // THIS LINE IS WRONG
.addModifiers(Modifier.PUBLIC)
.addMethod(serializeMethod)
.build();
try {
JavaFile.builder(processingEnv.getElementUtils().getPackageOf(element).toString(), serializer)
.build()
.writeTo(processingEnv.getFiler());
} catch (IOException e) {
e.printStackTrace();
}
}
}
return true;
}
But I get a compile error, because my generated class is not specifying the generic parameter in it's inheritance. How can I specify that?
Instead of passing a java.lang.Class to the addSuperinterface method, you'll need to pass something with the specific type details you have in mind. This method has two overloads - one which takes java.lang.reflect.Type (and Class is a subtype of this), and another which one which takes com.squareup.javapoet.TypeName). Technically either works, though since you are already using JavaPoet, I'd encourage trying to create the TypeName instance.
TypeName has a number of subclasses, ClassName, ParameterizedTypeName are probably the main ones to focus on here. In an annotation processor, they have some big advantages over using a Class instance - mostly that you don't need to actually be able to load or reference the class you are talking about - kind of like how you are using element.getSimpleName().toString() elsewhere in your code.
These classes have static methods to create them, which can be based on a variety of things. The one we're interested in here is this:
/** Returns a parameterized type, applying {#code typeArguments} to {#code rawType}. */
public static ParameterizedTypeName get(ClassName rawType, TypeName... typeArguments)
In you code, you would use it roughly like this:
...
.addSuperinterface(ParameterizedTypeName.get(
ClassName.get(JsonSerializer.class),//rawType
ClassName.get(whateverTShouldBe) //the value for T
))
...
Chance are excellent that T could eventually be generic here too, like List<String>, so you should take care to properly build the type which is passed in there - it might itself be a ParameterizedTypeName. Keep an eye on the various methods in TypeName for this too - the TypeName.get(TypeMirror) overload for example will take an already-parameterized declared type mirror and give you the expected ParameterizedTypeName back again.
With that said, according to your other code, T today cannot be generic - you look for the #JsonSerialize annotation on an Element, which means it would be the equivelent of List<T> rather than the usage of it, List<String>. Then, in this line, you make the Element into a TypeMirror to build the type name as I've described above:
.addParameter(ParameterSpec.builder(TypeName.get(element.asType()), "obj", Modifier.FINAL).build())
This means the final code would probably be
...
.addSuperinterface(ParameterizedTypeName.get(
ClassName.get(JsonSerializer.class),//rawType
TypeName.get(element.asType()) //the value for T
))
...

Proper way to cast object to generic type

I have an implementation of something like chain of responsibility patterns where common context has method
Object getVariable(String name)
When I get an object from the context I want to validate it with an appropriate validator. Validator interface:
public interface PageValidator<DTO> {
ValidationResult validate(DTO dto);
}
So, when I try to do validation
Object dtoForValidation = getDtoForValidation(delegateExecution);
ValidationResult validationResult = getPageValidator(delegateExecution).validate(dtoForValidation);
it fails with a compilation error
incompatible types: java.lang.Object cannot be converted to capture#1 of ?
So, I wanted to ask what is the proper way to design it?
(I don't really want to let validators accept Object as input argument as it looks ugly)
My suggestion is to change method getValiable(String name). You can pass Class<T> varClass to this method. And your signature will be something like that:
<T> T getVariable(String name, Class<T> varClass);
If this method is placed in 3rd party library, I would recommend to you to create some wrapper for this class.
class Wrapper {
private OriginalClass originalObject;
<T> T getVariable(String name, Class<T> varClass) {
return (T) originalObject.getVariable(name);
}
}
Off the top of my head, you can do one of the following.
1) Modify your getDtoForValidation method to return a DTO
2) Explicitly cast your object using ValidationResult validationResult = getPageValidator(delegateExecution).validate((DTO)dtoForValidation);
3) You mentioned you don't want your validators to accept Object as input argument, but you can do that and in the validator, start by checking Object type.

Resolving generic type with java classmate

After failing miserably trying to use TypeTools Resolving generic type information with TypeTools I am attempting to use https://github.com/cowtowncoder/java-classmate instead.
Can someone help me fix this code?
public T fromMap(S map) {
TypeResolver typeResolver = new TypeResolver();
ResolvedType type = typeResolver.resolve((new MapperImpl<T, S>() {}).getClass());
List<ResolvedType> params = type.typeParametersFor(MapperImpl.class);
ResolvedType typeT = params.get(0);
ObjectMapper objectMapper = new ObjectMapper();
T obj = objectMapper.convertValue(map, (Class<T>) typeT.getErasedType());
return obj;
}
I am getting this error:
java.util.LinkedHashMap cannot be cast to LoginInputMapTest$Foo
java.lang.ClassCastException at
shouldMapToFoo(LoginInputMapTest.java:83)
with this minimal test case:
public static class Foo {
private String a;
public String getA() {
return a;
}
public void setA(String a) {
this.a = a;
}
}
#Test
public void shouldMapToFoo() {
Map<String, Object> map = new HashMap<>();
map.put("a", "aaa");
Mapper<Foo, Map<String, Object>> mapper = new MapperImpl<>();
Foo foo = mapper.fromMap(map);
Assert.assertEquals(foo.getA(), map.get("a"));
}
There's nothing you can do within your fromMap method to get the type argument provided that was bound to your type variable T.
I suggest you create a Mapper implementation specifically for Foo.
class FooMapperImpl<S> implements Mapper<Foo, S> {
public Foo fromMap(S map) {
ObjectMapper objectMapper = new ObjectMapper();
Foo obj = objectMapper
.convertValue(map, Foo.class);
return obj;
}
}
(Though I don't see why you need a source type S if it's always going to be a Map.)
It seems to me that you do not fully understand the way Java generic types work, with respect to type variables (T, S). A good place to learn more about this is:
http://www.angelikalanger.com/GenericsFAQ/FAQSections/ParameterizedTypes.html
but basically type variables do not carry any run time generic type information. So while you are nominally calling a method with certain parameterization, nothing happens unless you pass actual Class instance suitable parameterized. So your method compiled to bytecode is little more than:
public Object fromMap(Object map) { ... }
Now, if you pass a Map as map, runtime type will be simply Map.class and there are no type parameters specified: Java values do not have any runtime type parameterization information. Underlying class is the same between, say, Map<String,Number> and Map<UUID,byte[]>. Declarations of parameters only affect Java compiler, which adds necessary casts to ensure that value types get cast properly.
No library can find information that is there, unfortunately. So usage as you suggest is not possible to implement as-is.
This does not mean that you could not pass typing, but it means that it must be passed from outside. With basic Jackson, you have TypeReference you can use:
new TypeReference<Map<KeyType, ValueType>>() { };
would construct reference to type Map<KeyType,ValueType>.
Or you can construct these programmatically using TypeFactory; something like:
mapper.getTypeFactory().constructMapType(Map.class, KeyType.class, ValueType.class);
// or with recursively constructing nested generic types
Now: ClassMate can, conversely, extract type information out of class definitions. If you have class with fields, methods that use generic type declaration, it is difficult to easily find out declared parameterization. But it does not sound like this is what you actually want or need here. Rather you should be able to build it using Jackson's type handling functionality.

Avoiding an unchecked cast for cast to a collection of a generic interface in Java for an event publisher

I'm trying to create a lightweight, thread-safe in-app publish/subscribe mechanism for an Android app that I'm building. My basic approach is to keep track of a list of IEventSubscriber<T> for each event type T and then be able to publish events to subscribing objects by passing along a payload of type T.
I use generic method parameters to (I think) ensure that subscriptions are created in a type safe way. Thus, I'm pretty sure that when I obtain the list of subscribers from my subscription map when it comes time to publish an event that I'm OK casting it to a list of IEventSubscriber<T>, however, this generates the unchecked cast warning.
My questions:
Is the unchecked cast actually safe here?
How can I actually check to see if the items in the subscriber list implement IEventSubscriber<T>?
Presuming that (2) involves some nasty reflection, what would you do here?
Code (Java 1.6):
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.CopyOnWriteArraySet;
public class EventManager {
private ConcurrentMap<Class, CopyOnWriteArraySet<IEventSubscriber>> subscriptions =
new ConcurrentHashMap<Class, CopyOnWriteArraySet<IEventSubscriber>>();
public <T> boolean subscribe(IEventSubscriber<T> subscriber,
Class<T> eventClass) {
CopyOnWriteArraySet<IEventSubscriber> existingSubscribers = subscriptions.
putIfAbsent(eventClass, new CopyOnWriteArraySet<IEventSubscriber>());
return existingSubscribers.add(subscriber);
}
public <T> boolean removeSubscription(IEventSubscriber<T> subscriber,
Class<T> eventClass) {
CopyOnWriteArraySet<IEventSubscriber> existingSubscribers =
subscriptions.get(eventClass);
return existingSubscribers == null || !existingSubscribers.remove(subscriber);
}
public <T> void publish(T message, Class<T> eventClass) {
#SuppressWarnings("unchecked")
CopyOnWriteArraySet<IEventSubscriber<T>> existingSubscribers =
(CopyOnWriteArraySet<IEventSubscriber<T>>) subscriptions.get(eventClass);
if (existingSubscribers != null) {
for (IEventSubscriber<T> subscriber: existingSubscribers) {
subscriber.trigger(message);
}
}
}
}
Is the unchecked cast actually safe here?
Quite. Your code will not cause heap pollution because the signature of subcribe ensures that you only put IEventSubscribers of the proper compile time type into the map. It might propagate heap pollution caused by an unsafe unchecked cast elsewhere, but there is little you can do about that.
How can I actually check to see if the items in the subscriber list implement IEventSubscriber?
By casting each item to IEventSubscriber. Your code already does this in the following line:
for (IEventSubscriber<T> subscriber: existingSubscribers) {
If existingSubscribers contained an object not assignable to IEventSubscriber, this line would throw a ClassCastException. Standard practice to avoid a warning when iterating over a list of unknown type parameter is to explicitly cast each item:
List<?> list = ...
for (Object item : list) {
IEventSubscriber<T> subscriber = (IEventSubscriber<T>) item;
}
That code explicitly checks that each item is an IEventSubscriber, but can not check that it is an IEventSubscriber<T>.
To actually check the type parameter of IEventSubscriber, the IEventSubscriber needs to help you out. That is due to erasure, specifically, given the declaration
class MyEventSubscriber<T> implements IEventSubscriber<T> { ... }
the following expression will always be true:
new MyEventSubscriber<String>.getClass() == new MyEventSubscriber<Integer>.getClass()
Presuming that (2) involves some nasty reflection, what would you do here?
I'd leave the code as it is. It is quite easy to reason that the cast is correct, and I would not find it worth my time to rewrite it to compile without warnings. If you do wish to rewrite it, the following idea may be of use:
class SubscriberList<E> extends CopyOnWriteArrayList<E> {
final Class<E> eventClass;
public void trigger(Object event) {
E event = eventClass.cast(event);
for (IEventSubscriber<E> subscriber : this) {
subscriber.trigger(event);
}
}
}
and
SubscriberList<?> subscribers = (SubscriberList<?>) subscriptions.get(eventClass);
subscribers.trigger(message);
Not exactly. It will be safe if all clients of the EventManager class always use generics and never rawtypes; i.e., if your client code compiles without generics-related warnings.
However, it is not difficult for client code to ignore those and insert an IEventSubscriber that's expecting the wrong type:
EventManager manager = ...;
IEventSubscriber<Integer> integerSubscriber = ...; // subscriber expecting integers
// casting to a rawtype generates a warning, but will compile:
manager.subscribe((IEventSubscriber) integerSubscriber, String.class);
// the integer subscriber is now subscribed to string messages
// this will cause a ClassCastException when the integer subscriber tries to use "test" as an Integer:
manager.publish("test", String.class);
I don't know of a compile-time way to prevent this scenario, but you can check the generic parameter types of instances of IEventSubscriber<T> at runtime if the generic type T was bound to the class at compile time. Consider:
public class ClassA implements IEventSubscriber<String> { ... }
public class ClassB<T> implements IEventSubscriber<T> { ... }
IEventSubscriber<String> a = new ClassA();
IEventSubscriber<String> b = new ClassB<String>();
In the above example, for ClassA, String is bound to the parameter T at compile time. All instances of ClassA will have String for the T in IEventSubscriber<T>. But in ClassB, String is bound to T at runtime. Instances of ClassB could have any value for T. If your implementations of IEventSubscriber<T> bind the parameter T at compile time as with ClassA above, then you can obtain that type at runtime the following way:
public <T> boolean subscribe(IEventSubscriber<T> subscriber, Class<T> eventClass) {
Class<? extends IEventSubscriber<T>> subscriberClass = subscriber.getClass();
// get generic interfaces implemented by subscriber class
for (Type type: subscriberClass.getGenericInterfaces()) {
ParameterizedType ptype = (ParameterizedType) type;
// is this interface IEventSubscriber?
if (IEventSubscriber.class.equals(ptype.getRawType())) {
// make sure T matches eventClass
if (!ptype.getActualTypeArguments()[0].equals(eventClass)) {
throw new ClassCastException("subscriber class does not match eventClass parameter");
}
}
}
CopyOnWriteArraySet<IEventSubscriber> existingSubscribers = subscriptions.putIfAbsent(eventClass, new CopyOnWriteArraySet<IEventSubscriber>());
return existingSubscribers.add(subscriber);
}
This will cause the types to be checked when the subscriber is registered with the EventManager, allowing you to track down bad code more easily, rather than if the types were to just get checked much later when publishing an event. However, it does do some hokey reflection and can only check the types if T is bound at compile time. If you can trust the code that will be passing subscribers to EventManager, I'd just leave the code as it is, because it's much simpler. However checking the type using reflection as above will make you a little safer IMO.
One other note, you may want to refactor the way you initialize your CopyOnWriteArraySets, because the subscribe method is currently creating a new set on every invocation, whether it needs to or not. Try this:
CopyOnWriteArraySet<IEventSubscriber> existingSubscribers = subscriptions.get(eventClass);
if (existingSubscribers == null) {
existingSubscribers = subscriptions.putIfAbsent(eventClass, new CopyOnWriteArraySet<IEventSubscriber>());
}
This avoids creating a new CopyOnWriteArraySet on every method call, but if you have a race condition and two threads try to put in a set at once, putIfAbsent will still return the first set created to the second thread, so there is no danger of overwriting it.
Since your subscribe implementation ensures that every Class<?> key in the ConcurrentMap maps to the correct IEventSubscriber<?>, it is safe to use #SuppressWarnings("unchecked") when retrieving from the map in publish.
Just make sure to properly document the reason why the warning is being suppressed so that any future developer making changes to the class is aware of what's going on.
See also these related posts:
Generic Map of Generic key/values with related types
Java map with values limited by key's type parameter

Categories