We Keep Coding

How to implement Hazelcast PortableFactory for Portable enum

I have a Java enum A that I want to serialize with Hazelcast Portable interface.
To implement the PortableFactory associated with A I need to call an empty constructor to create an empty instance of A, but that is not possible when it comes to enum in Java.
What's the best way to implement a PortableFactory to serialize enum?
Here an example of what I'd like to achive:
public class MyPortableFactory implements PortableFactory {
#Override
public Portable create( int classId ) {
if (Foo.ID == classId) {
return new Foo(); //This is how you return normal class
} else if(MyEnum.ID == classId) {
return ???; //What should I return for the enum?
} else {
return null;
}
}
}
I have tried a workoround using a map to change the ids:
public enum MyEnum implements Portable {
Value1("1", "1", PortableIds.one),
Value2("2", "2", PortableIds.two),
Invalid("0", "0", PortableIds.InvalidID);
private String first;
private String second;
private Integer portableId;
public static final Map<Integer, MyEnum> mapPortableIdValues = new HashMap<>() {{
for (MyEnum myEnum : MyEnum.values()) {
put(myEnum.getPortableId(), myEnum);
}
}};
#Override
public int getFactoryId() {
return MyPortableFactory.FACTORY_ID;
}
#Override
public int getClassId() {
return portableId;
}
public class MyPortableFactory implements PortableFactory {
public static final int FACTORY_ID = 2;
#Override
public Portable create(int classId) {
if(mapPortableIdValues.containsKey(classId)){
return mapPortableIdValues.get(classId);
}
return null;
}
}
public interface PortableIds {
Integer one = 100;
Integer two = 101;
Integer InvalidID = 106;
}
But when I run my application I get the following exception:
com.hazelcast.nio.serialization.HazelcastSerializationException: Wrong Portable type! Generic portable types are not supported! Expected class-id: 100, Actual class-id: 106
at com.hazelcast.internal.serialization.impl.portable.DefaultPortableWriter.checkPortableAttributes(DefaultPortableWriter.java:174)
at com.hazelcast.internal.serialization.impl.portable.DefaultPortableWriter.writePortable(DefaultPortableWriter.java:147)

I think this use case is not a good fit for the design of the Portable serialization. The mechanism of Portable serialization instantiates an instance of a class and fills the fields with readPortable method at runtime. This is not really possible with enums.
I would suggest using a wrapper class to carry simply an integer enum tag or the name of the enum member as a string with Portable serialization.

Generic parameter determining instance field type at execution

Explanation of the situation:
I want to instanciate an object that can that have basically 2 parameters. A set of 'meta-parameters' and a value. The type of the value is determined by the 'meta-parameters'.
Example:
public class Meta
{
public static final Meta META_FIRST = new Meta(String.class, new byte[] {0x00, 0x01});
public static final Meta META_SECOND = new Meta(Float.class, new byte[] {0x00, 0x02});
public static final Meta META_THIRD = new Meta(Double.class, new byte[] {0x00, 0x03});
private Class<?> type;
private byte[] prelude;
private Meta(Class<?> type, byte[] prelude)
{
this.type = type;
this.prelude = prelude;
}
public Class<?> getType()
{
return this.type;
}
public byte[] getPrelude()
{
return this.prelude;
}
}
public class Record
{
private # value;
private byte[] prelude;
public Record(Meta meta, # value)
{
this.prelude = meta.getPrelude();
}
public void doSomeWork()
{
//Do some work with prelude and value
}
}
Expected usage:
Record recordString = new Record(Meta.META_FIRST, "hello");
Record recordDouble = new Record(Meta.META_THIRD, 12.8);
My doubt yet is how to determine the type of 'value' (actually symbolized by '#').
I think generics or reflexion could solve my problem but I can't figure out how a parameter in the constructor can influence the type of another parameter.
I would like to avoid using the generic notation when instanciating a Record (that's the reason why I putted this 'generic' information in the Meta-class).
Can anyone has an idea how to solve that ? (feel free to suggest an other approach)
Note: it is also acceptable for me to initialize the record value later with a setter.

In order to have it compiling, you have to make the Record class generic (parameterized by the type of the value):
public class Record<T> {
private T value;
public Record(Meta meta, T value) {
//Initialization
}
}
However, I don't see a reason you have a Meta class, since it does nothing but holding the Class type of the value. In order to simplify the hierarchy and to make sure the Meta is compatible with the value type, I would remove the Meta class and keep a Class<T> in Record, which will represent the meta about the value.
public class Record<T> {
private T value;
private Class<T> meta;
public Record(T value, Class<T> meta) {
//Initialization
}
public Class<T> getMeta() {
return meta;
}
}
and will use it like this:
Record recordString = new Record("hello", String.class);
Class<String> recordStringMeta = recordString.getMeta();
Record recordDouble = new Record(12.8, Double.class);
Class<Double> recordDoubleMeta = recordDouble.getMeta();
Update:
Since you don't want to have the Record class generic (which I don't advice you, but ...), you can introduce three constructors there and copy the passed value to an Object member. Unfortunately, this will force you to do casts when extracting the value back:
public class Record {
private Object value;
public Record(Meta meta, String value) { ... }
public Record(Meta meta, Double value) { ... }
public Record(Meta meta, Float value) { ... }
}

How to model a medium-sized data set with multiple types as a class

Problem
I don't know the best way to model my data. I'm worried my current approach has gotten overly complex, and I want to correct it now before I base any more code off it.
Data to be Modeled
I have data sets that consist of 50+ different data items. Each item consists of:
a unique identifier int
a label String.
validation criteria (min, max, legal characters, etc...).
a value Float, Long, Integer, String, or Date.
The label and validation criteria for each item is the same in every data set. Only the values are dynamic. Order is not important.
Needed Usage Examples
Add data to the data set
dataSet.put(itemIdentifier, value);
Traverse and validate all non-null values in the data set
for (DataItem item : dataSet.values()) {
boolean valid = item.validate();
if (valid) {...}
}
Show the specified items in the given data sets
public void displayData(List<DataSet> dataSets, int... itemsIdentifiers) {...}
Implementation Attempt
My current implementation has an abstract Key class as the "key" to a map. Each type subclasses for its own validation needs. Then, inside the DataSet class, I have public static keys for each item.
abstract public class Key {
public int mId;
public String mLabel;
public Key(int id, String label) {...}
abstract public boolean validate(Object Value);
}
public class FloatKey extends Key {
private int mMin, mMax;
public Key(int id, String label, int min, int max) {...}
public boolean validate(Object Value) {...}
}
// one for each type
...
public class DataSet {
public static Key ITEM_A = new FloatKey(1, "item A", 0, 100);
public static Key ITEM_B = new DateKey(2, "item B", "January 1, 1990");
// ~50 more of these
private Map<Key, Object> mMap;
public void put(int itemId, Object value) {...}
public Set<Object> values() {...};
...
}
I don't like that when I pull values out of DataSet, I need to hold onto the value AND the key so I can do things like DataSet.ITEM_A.validate(someFloat). I also find myself using instanceof and casting frequently when I traverse objects in a set because I need to call subclass-only methods in some situations.
Edits for further clarification
Data items and their validation criteria will require occasional changes and so maintenance should be relatively easy / painless.
Although I could use the Key objects themselves as keys into the map, I will sometimes need to put these keys in a Bundle (part of the android API). I would rather use the label or id (in case labels are the same) to avoid making my Key class Parcelable.

What about this approach:
Create this interface:
interface Validable {
boolean isValid();
}
Then, all data items inherit the following class and implicitly the interface ::
abstract class DataItem implements Validable {
public DataItem(int id, String label, int min, int max) {
}
}
Configure each specific instance of DataItem via constructor parameters, passing the common and the distinct values:
class FloatItem extends DataItem {
public FloatItem(int id, String label, int min, int max, Float value) {
super(id, label, min, max);
// set the Float value here
}
#Override
public boolean isValid() {
// validate here
return true;
}
}
class DateItem extends DataItem {
public DateItem(int id, String label, int min, int max, Date value) {
super(id, label, min, max);
}
#Override
public boolean isValid() {
// validate here
return true;
}
}
The client code would assemble the objects like this::
List<Validable> items = Lists.<Validable>newArrayList(new FloatItem(0, "", 0, 0, Float.NaN),
new DateItem(0, "", 0, 0, new Date()));
(note the usage of Google Guava)
Calling code only needs to do this::
for (Validable validable : items) {
System.out.println(validable.isValid());
}
Please note that this approach requires you to first create 'target' objects, and then ask the question if they are valid. In other words, you are passing the valid-able parameters via constructor and then, you ask the object if it is valid. The object itself will answer the question using the validation criteria inside it...
I hope I understood your problem correctly.

I don't quite understand your goals with the design, so maybe not all of this is correct or directly useful to you, but it's some ideas to play with.
First I'd point out that there are lots of fields in the code you've shown that should be marked final. For example, Key.mId, Key.mLabel, FloatKey.mMin, FloatKey.mMax, all the DataSet.ITEM_X, and DataSet.mMap. Marking them final (1) conveys the intended behavior better, (2) prevents accidents where something like a Key's mId changes, and (3) might have marginal performance benefits.
I wonder why you need the numeric ID for each key/field? If they're required for interfacing with some external application or storage format which already defines those IDs, that makes sense, but if it's only for internal things like this method:
public void displayData(List<DataSet> dataSets, int... itemsIdentifiers) {...}
then that could be more meaningfully implemented using a list of String labels or Key objects, instead of the numeric IDs. Likewise, DataSet.put could possibly use the Key or label instead of the ID.
I find myself using instanceof and casting frequently when I traverse objects in a set
Making Key generic can eliminate some casts. (Well, they will still be present in the bytecode, but not in the source because the compiler will take care of it.) E.g.,
abstract public class Key<T> {
...
abstract public boolean validate(T Value);
}
public class FloatKey extends Key<Float> {
...
public boolean validate(Float value) { ... }
}
In the validate method, you thus avoid the need to cast value.
Also, I'm guessing you currently have a method on class DataSet like this:
public Object get(int itemId) { ... }
If you use the Key instead of numeric ID to retrieve values, and make the method generic, you'll often be able to avoid the need for callers to cast the return value (though the cast is still present inside the get method):
public <T> T get(Key<T> key) { ... }
I don't like that when I pull values out of DataSet, I need to hold onto the value AND the key so I can do things like DataSet.ITEM_A.validate(someFloat).
You could make a class for the value instead of the key. E.g.,
abstract public class Value<T> {
public final int id;
public final String label;
protected Value(int id, String label) {
this.id = id;
this.label = label;
}
abstract public T get();
abstract public void set(T value);
}
public class FloatValue extends Value<Float> {
private final float min, max;
private float value;
public FloatValue(int id, String label, float min, float max, float value) {
super(id, label);
this.min = min;
this.max = max;
set(value);
}
public Float get() { return value; }
public void set(Float value) {
if (value < min | value > max) throw new IllegalArgumentException();
this.value = value;
}
}
public class DataSet {
public final FloatValue itemA = new FloatValue(1, "item A", 0, 100, 0);
...
}
That solves the stated problem, and also eliminates the map lookup previously required on every get/set of a value. However it has the side effect of duplicating the storage for the labels and numeric IDs, as the Value classes are not static fields any more.
In this scenario, to access DataSet values by label (or ID?), you can use reflection to build a map. In class DataSet:
private final Map<String, Value<?>> labelMap = new HashMap<>();
{
for (Field f : DataSet.class.getFields()) {
if (Value.class.isAssignableFrom(f.getType())) {
Value<?> v;
try {
v = (Value<?>)f.get(this);
} catch (IllegalAccessException | IllegalArgumentException e) {
throw new AssertionError(e); // shouldn't happen
}
labelMap.put(v.label, v);
}
}
}
There's a subtlety here: if you subclass DataSet to represent different types of data, then the Value fields of the subclasses will not have been initialized yet at the time DataSet's initializer builds the map. So if you create subclasses of DataSet, you might need a protected init() method to be called from subclass constructors, to tell it to (re)build the map, which is a bit ugly but it would work.
You can re-use this map to provide convenient iteration of a DataSet's values:
public Collection<Value<?>> values() {
return Collections.unmodifiableCollection(labelMap.values());
}
A final idea: if you're using reflection anyway, it might be possible to use ordinary fields for the values, with annotation interfaces to implement their behavior.
import java.lang.annotation.*;
import java.lang.reflect.*;
public class DataSet {
#Label("item A") #ValidateFloat(min=0, max=100) public float itemA;
#Label("item B") public String itemB;
#Retention(RetentionPolicy.RUNTIME)
public static #interface Label {
String value();
}
#Retention(RetentionPolicy.RUNTIME)
public static #interface ValidateFloat {
float min();
float max();
}
public final class Value {
public final String label;
private final Field field;
protected Value(String label, Field field) {
this.label = label;
this.field = field;
}
public Object get() {
try {
return field.get(DataSet.this);
} catch (IllegalArgumentException | IllegalAccessException e) {
throw new AssertionError(e); // shouldn't happen
}
}
public void set(Object value) {
try {
field.set(DataSet.this, value);
} catch (IllegalArgumentException | IllegalAccessException e) {
throw new AssertionError(e); // shouldn't happen
}
}
public void validate() {
Object value = get();
// Test for presence of each validation rule and implement its logic.
// Ugly but not sure how best to improve this...
if (field.isAnnotationPresent(ValidateFloat.class)) {
float floatValue = (float)value;
ValidateFloat rule = field.getAnnotation(ValidateFloat.class);
if (floatValue < rule.min() || floatValue > rule.max()) {
//throw new Whatever();
}
}
//if (field.isAnnotationPresent(...)) {
// ...
//}
}
}
private final Map<String, Value> labelMap = new HashMap<>();
{
for (Field f : DataSet.class.getFields()) {
if (f.isAnnotationPresent(Label.class)) {
Value value = new Value(f.getAnnotation(Label.class).value(), f);
labelMap.put(value.label, value);
}
}
}
public Collection<Value> values() {
return Collections.unmodifiableCollection(labelMap.values());
}
}
This approach has different tradeoffs. Code that knows exactly what field it wants can access it directly. E.g., dataSet.itemA instead of dataSet.get(DataSet.ITEM_A). Code that needs to iterate multiple fields does so via the Value wrapper (would Property be a better class name? Or Item?), which encapsulates the ugliness of the field reflection code.
I also put the validation logic into the annotations. If there are lots of fields with very simple numeric limits, that works well. If it's too complex for that you'd be better off with a DataSet.validate method that accesses the fields directly. E.g,
public void validate() {
if (itemC < 10 || itemC > itemD) ...
}
Okay, one more idea:
public class DataSet {
public float itemA;
public String itemB;
public static abstract class Value<T> {
public final String label;
protected Value(String label) {
this.label = label;
}
public abstract T get();
public abstract void set(T value);
}
public Value<?>[] values() {
return new Value[] {
new Value<Float>("itemA") {
public Float get() {
return itemA;
}
public void set(Float value) {
itemA = value;
}
},
new Value<String>("itemB") {
public String get() {
return itemB;
}
public void set(String value) {
itemB = value;
}
},
};
}
}
This is simple (no annotations or reflection) but it's repetitive. Since you have "50+" fields, the repetitiveness is probably not ideal as it's easy when copy-pasting to slip up at some point, forgetting to replace itemX = value with itemY = value, but if you only need to write it once it might be acceptable. Validation code could go either on the Value class or the DataSet class.

Efficiently access fields by name

A little background as to what I'm trying to achieve:
I'm parsing JSON (over 15GB) and I must store it in memory so any wrappers and extra data is not welcomed, due to the framework and interfaces used within it I must provide functionality to access fields by name. By replacing some String with Enum, Integer with int, Double with double, etc. I'm able to shave about 90% of memory footprint (in comparison with Jackson).
I'm looking to efficiently access the fields at runtime in Java by their name. I'm aware of reflection, but for my case its performance is simply unacceptable, so I don't want to use it.
If it makes the problem easier to solve I'm not too bothered about setting the fields values. I also know at compile time the names of supported fields.
I don't want to store everything in a map i.e. Map<String,Object> due to the memory footprint of boxed object, but I don't mind returning them in a boxed form.
I'm sure this problem was encountered by others and I'm interested in any clever solutions - cleverer than tons of if ... else ... statements.
Let's say the interface to implement is:
public interface Accessor {
Object get(String fieldName);
}
The Object returned by get can be of any type including enum. A naive implementation would be:
public class TestObject implements Accessor {
public enum MyEnum {ONE, TWO, THREE};
private final MyEnum myEnum;
private final int myInt;
private final double myDouble;
private final String myString;
public TestObject(MyEnum myEnum, int myInt, double myDouble, String myString) {
this.myEnum = myEnum;
this.myInt = myInt;
this.myDouble = myDouble;
this.myString = myString;
}
#Override
public Object get(String fieldName) {
if ("myEnum".equals(fieldName)) {
return myEnum;
} else if ("myInt".equals(fieldName)) {
return myInt;
} else if ("myDouble".equals(fieldName)) {
return myDouble;
} else if ("myString".equals(fieldName)) {
return myString;
} else {
throw new UnsupportedOperationException(); // Or could simply return null
}
}
}

What you want is a mapping from a fieldName to a value, the type of which is determined by the fieldName. You know the set of field names up-front, so this is an ideal task for an Enum.
If you don't like the idea of hard-coding each field as an enum, then the variation would be an enum-per-type (MY_FIELD1 becomes MY_ENUM), with a mapping from fieldName to this EnumType.
In the code below I'm making assumptions about the relationship between fieldName and TestObject. Specifically it looks like TestObject is presenting various types of the same value (surely where reasonable), as opposed to a separate value for each field name?
So, to the code:
Rewrite:
#Override
public Object get(String fieldName) {
MyField field = MyField.mapNameToField(fieldName);
if (field == null)
throw new UnsupportedOperationException(); // Or could simply return null
return field.getValue(this);
}
Given (something like):
enum MyField {
MY_FIELD1("myField1") {
public Object getValue(TestObject obj) { return obj.myEnum; }
},
MY_FIELD2("myField2") {
public Object getValue(TestObject obj) { return obj.myInt; }
},
...
;
public abstract Object getValue(TestObject obj);
public String getName() { return name; }
public static MyField mapNameToField(String name) { return map.get(name); }
static {
map = new HashMap<String,MyField>();
for(MyField value: values()) {
map.put(value.getName(), value);
}
}
private MyField(String fieldName) { name = fieldName; }
private String name;
private static Map<String, MyField> map;
}

I've never used this, but looks promising:
http://labs.carrotsearch.com/download/hppc/0.4.1/api/
"High Performance Primitive Collections (HPPC) library provides typical data structures (lists, stacks, maps) template-generated for all Java primitive types (byte, int, etc.) to conserve memory and boost performance."
In particular, the Object{Type}OpenHashMap classes might be what you're looking for:
ObjectByteOpenHashMap
ObjectCharOpenHashMap
ObjectDoubleOpenHashMap
ObjectFloatOpenHashMap
ObjectIntOpenHashMap
ObjectLongOpenHashMap
ObjectShortOpenHashMap
I imagine you would have all 7 of these defined as fields (or whatever subset of them you like), and you would probe each one in turn to see if the key was present for that type of primitive value. E.g.,
if (byteMap.containsKey(key)) {
return byteMap.lget(); // last value saved in a call to containsKey()
} else if (charMap.containsKey(key)) {
return charMap.lget();
} else if {
// and so on...
}
Notice they have their own special lget() method call to optimize the containsKey() / get() usage pattern so typical with maps.

Java Generics Selective Return Value

I've been struggling with this for a while and have yet to find an answer. As a result, my brain is somewhat muddled, so pardon me if I make a dumb mistake.
I'm trying to implement a typed INI parser, that will parse this kind of file:
[section1]
<int>intkey=0
<float>floatkey=0.0
<str>stringkey=test
[section2]
<float>x=1.0
<float>y=1.0
<float>z=0.0
In doing so, I have a central class named Config, which handles the basic reading and writing operations. One of the methods of Config is called get(String section, String key), which ideally would return a value appropriate for the requested section-key pair, like so:
Config cfg = new Config("test.ini");
cfg.get("section2", "x"); // 1.0, not "1.0" or some Object that technically represents the float
cfg.get("section1", "intkey"); // 0
cfg.get("section1", "strkey"); // "test"
I'm currently using an enum to handle the conversion of the String to various types, with an abstract method overridden by the different types:
enum Type
{
INTEGER ("int") {
public Object parse(String value) {
try
{
return Integer.parseInt(value);
} catch (NumberFormatException e)
{
return null;
}
}
},
FLOAT ("float") {
public Object parse(String value) {
try
{
return Float.parseFloat(value);
} catch (NumberFormatException e)
{
return null;
}
}
},
STRING ("str") {
public Object parse(String value) {
return value;
}
};
public final String name;
Type(String name)
{
this.name = name;
}
private static HashMap<String, Type> configMap = generateConfigMap();
private static HashMap<String, Type> generateConfigMap()
{
HashMap<String, Type> map = new HashMap<String, Type>();
for (Type type : Type.values())
map.put(type.name, type);
return map;
}
public static Type get(String name)
{
return configMap.get(name);
}
abstract public Object parse(String value);
}
Unfortunately, parse(String value) returns an Object, and when passed out of Config, requires a cast or similar, and ideally this would be self-contained.
If I'm going about this completely wrong and there's a more flexible or simple way to code it, please let me know. I'm open to suggestions. Though I would like to know if there's a way to do this. Maybe with generics...?
Note: I know I'm missing imports and the like. That's not why I'm posting here.

Here's the thing. If the code that calls config.get() doesn't know what type to expect, you can't possibly return anything other than Object since the calling code doesn't know what to expect. Of course you'll have to cast.
Now, if you wanted to design Config in a way that the caller did know what type it was asking for, than that becomes a bit easier. The easiest approach then is to do something like this:
public class Config {
public int getInt(String a, String b) {
return ((Integer)get(a, b)).intValue();
}
}
But until the caller knows what to expect, you really gain nothing from avoiding casts.

If you want to return a a type of object depending on what you get you can do this:
public <T extends MyObject> T myMethod(Class<T> type) {
return type.cast(myObj);
}