We Keep Coding

Optimize insertion from ArrayList to HashMap

I'm trying to insert data from ArrayList to HashMap<String, Language> optimally.
Many items may have the same languge_name (code below), so I need to group items having the same language in Language class and store languages in a HashMap with the name of the language as a Key.
Item
String name;
String language_name;
Language
String language_name;
int numberItems;
LinkedList<String> Items;
I solved this as follows:
ArrayList<Item> items; // given array of items
HashMap<String, Language> languages = new HashMap<String, Language>();
items.forEach(item -> {
/** case 1: language isn't specified */
if (item.getLanguageName() == null) {
item.setLanguageName("unknown");
}
/** case 2: language already added */
if (languages.containsKey(item.getLanguageName())) {
languages.get(item.getLanguageName()).getItems().add(item.getName());
languages.get(item.getLanguageName())
.setNumberItems(languages.get(item.getLanguageName()).getNumberItems() + 1);
} else {
/** case 3: language isn't added yet */
LinkedList<String> languageItems = new LinkedList<String>();
languageItems.add(item.getName());
Language language = new Language(item.getLanguageName(), 1, languageItems);
languages.put(item.getLanguageName(), language);
}
});
Any help would be appreciated!

Assuming you're using Java 8 or later, this can be accomplished nicely with built-in stream functions.
HashMap<String, List<Items>> itemsGroupedByLanguage =
items.stream().collect(Collectors.groupingBy(Items::getLanguage));

tl;dr
It's not possible to achieve what you desire using Java (8+) inbuilt collector, but you can write your own custom collector and write code like below to collect into a map as -
Map<String, Language> languages = items.stream().collect(LanguageCollector.toLanguage());
Let's first look at Collector<T, A, R> interface
public interface Collector<T, A, R> {
/**
* A function that creates and returns a new mutable result container.
*/
Supplier<A> supplier();
/**
* A function that folds a value into a mutable result container.
*/
BiConsumer<A, T> accumulator();
/**
* A function that accepts two partial results and merges them. The
* combiner function may fold state from one argument into the other and
* return that, or may return a new result container.
*/
BinaryOperator<A> combiner();
/**
* Perform the final transformation from the intermediate accumulation type
*/
Function<A, R> finisher();
/**
* Returns a Set of Collector.Characteristics indicating
* the characteristics of this Collector. This set should be immutable.
*/
Set<Characteristics> characteristics();
}
Where T is the generic type of the items in the stream to be collected.
A is the type of the accumulator, the object on which the partial result will be accumulated during the collection process.
R is the type of the object (typically, but not always, the collection) resulting
from the collect operation
Now let's look at the custom LanguageCollector
public class LanguageCollector
implements Collector<Item, Map<String, Language>, Map<String, Language>> {
/**
* The supplier method has to return a Supplier of an empty accumulator - a parameterless
* function that when invoked creates an instance of an empty accumulator used during the
* collection process.
*/
#Override
public Supplier<Map<String, Language>> supplier() {
return HashMap::new;
}
/**
* The accumulator method returns the function that performs the reduction operation. When
* traversing the nth element in the stream, this function is applied with two arguments, the
* accumulator being the result of the reduction (after having collected the first n–1 items of
* the stream) and the nth element itself. The function returns void because the accumulator is
* modified in place, meaning that its internal state is changed by the function application to
* reflect the effect of the traversed element
*/
#Override
public BiConsumer<Map<String, Language>, Item> accumulator() {
return (map, item) -> {
if (item.getLanguageName() == null) {
item.setLanguageName("unknown");
} else if (map.containsKey(item.getLanguageName())) {
map.get(item.getLanguageName()).getItems().add(item.getName());
map.get(item.getLanguageName())
.setNumberItems(map.get(item.getLanguageName()).getNumberItems() + 1);
} else {
Language language = new Language(item.getLanguageName(), 1);
language.add(item.getName());
map.put(item.getLanguageName(), language);
}
};
}
/**
* The combiner method, return a function used by the reduction operation, defines how the
* accumulators resulting from the reduction of different subparts of the stream are combined
* when the subparts are processed in parallel
*/
#Override
public BinaryOperator<Map<String, Language>> combiner() {
return (map1, map2) -> {
map1.putAll(map2);
return map1;
};
}
/**
* The finisher() method needs to return a function which transforms the accumulator to the
* final result. In this case, the accumulator is the final result as well. Therefore it is
* possible to return the identity function
*/
#Override
public Function<Map<String, Language>, Map<String, Language>> finisher() {
return Function.identity();
}
/**
* The characteristics, returns an immutable set of Characteristics, defining the behavior of
* the collector—in particular providing hints about whether the stream can be reduced in
* parallel and which optimizations are valid when doing so
*/
#Override
public Set<Characteristics> characteristics() {
return Collections.unmodifiableSet(
EnumSet.of(Characteristics.IDENTITY_FINISH));
}
/**
* Static method to create LanguageCollector
*/
public static LanguageCollector toLanguage() {
return new LanguageCollector();
}
}
I have modified your classes at little bit to (to follow the naming convention and more for readable accumulator operation).
Class Item
public class Item {
private String name;
private String languageName;
public Item(String name, String languageName) {
this.name = name;
this.languageName = languageName;
}
//Getter and Setter
}
Class Language
public class Language {
private String languageName;
private int numberItems;
private LinkedList<String> items;
public Language(String languageName, int numberItems) {
this.languageName = languageName;
this.numberItems = numberItems;
items = new LinkedList<>();
}
public void add(String item) {
items.add(item);
}
// Getter and Setter
public String toString() {
return "Language(languageName=" + this.getLanguageName() + ", numberItems=" + this.getNumberItems() + ", items=" + this.getItems() + ")";
}
}
Running code
public static void main(String[] args) {
List<Item> items =
Arrays.asList(
new Item("ItemA", "Java"),
new Item("ItemB", "Python"),
new Item("ItemC", "Java"),
new Item("ItemD", "Ruby"),
new Item("ItemE", "Python"));
Map<String, Language> languages = items.stream().collect(LanguageCollector.toLanguage());
System.out.println(languages);
}
prints
{Java=Language(languageName=Java, numberItems=2, items=[ItemA, ItemC]), Ruby=Language(languageName=Ruby, numberItems=1, items=[ItemD]), Python=Language(languageName=Python, numberItems=2, items=[ItemB, ItemE])}
For more information please read book 'Modern Java in Action: Lambdas, streams, functional and reactive programming' chapter 6.5 or check this link

Is there any way to generate an Avro schema from a hashmap of values?

We are putting all our data points for one row into a hashmap. We don't want to use a pojo because the values are a different set each time. For example, we might get "place" on some records and we might get "hometown" on others. Actually we have thousands of different column names to choose from. Our code looks like this:
Map<String, Object> aMap = new HashMap<>();
aMap.put("id", Integer.valueOf(1));
aMap.put("age", Integer.valueOf(45));
aMap.put("name", "mark");
aMap.put("place", "home");
final GenericRecord record = new GenericData.Record(avroSchema);
aMap.forEach((k, v) -> {
record.put(k, v);
});
writer.write(record);
We would like to put all the values in a map and then generate a schema. Since using the Reflect api, it can be done for a pojo, I was wondering if it could be done from a hashmap as well?
As a side question, Is there any way to eliminate the forEach above and just write the map?

Here is what we came up with. We also had nested columns.
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.apache.avro.Schema;
import org.apache.avro.Schema.Parser;
import org.apache.avro.Schema.Type;
/**
* This does NOT do all types. So far just the types we think we need. See
* https://docs.oracle.com/database/nosql-12.1.3.0/GettingStartedGuide/avroschemas.html
* <p>
* We need some error handlling here and when we don't have the correct type, call it out!
* <p>
* This runs in 1-2ms even with a large payload.
*/
public class AvroSchemaBuilder {
/**
* Construct!
*/
private AvroSchemaBuilder() {
//private constructor. All methods are static.
}
/**
* Build the Avro schema and return it.
*
* #param name Name of object.
* #param nameTypeConsumer The nameTypeConsumer of objects being saved.
* #return the Avro schema.
*/
public static Schema getAvroSchema(String name, NameTypeConsumer nameTypeConsumer) {
String json = Lson.toJson(getAvroSchemaAsMap(name, nameTypeConsumer, true));
Parser parser = new Parser().setValidate(true);
return parser.parse(json);
}
/**
* Returns the map with all the attributes to build a schema. This would be recursive if we need
* to build a complex schema. For example for Trends this would build a complex schema where some
* of the types are maps that are themselves described as another nested schema.
*/
private static Map<String, Object> getAvroSchemaAsMap(String name,
NameTypeConsumer nameTypeConsumer,
boolean addNameSpace) {
Map<String, Object> schemaMap = new LinkedHashMap<>();
schemaMap.put("type", "record");
schemaMap.put("name", name);
if (addNameSpace) {
schemaMap.put("namespace", "com.blah.blah");
}
List<Field> fields = new ArrayList();
nameTypeConsumer.consumeNestedNameType((columnName, nestedNameType) -> {
Object avroType;
if (nestedNameType.getNameTypeConsumer() != null) {
avroType = getAvroSchemaAsMap(columnName, nestedNameType.getNameTypeConsumer(), false);
} else {
avroType = getAvroType(nestedNameType.getType()).getName();
}
Object[] types = {"null", avroType}; //adding null first always.
fields.add(new Field(columnName, types));
});
schemaMap.put("fields", fields);
return schemaMap;
}
/**
* Finds the avro type by class.
*
* #param type the Type (this is an avro type).
* #return avro constant.
*/
private static Type getAvroType(Class<?> type) {
if (type.equals(Integer.class)) {
return Type.INT;
}
if (type.equals(Long.class)) {
return Type.LONG;
}
if (type.equals(Float.class)) {
return Type.FLOAT;
}
if (type.equals(Double.class)) {
return Type.DOUBLE;
}
if (type.equals(String.class)) {
return Type.STRING;
}
if (type.equals(Boolean.class)) {
return Type.BOOLEAN;
}
throw new GenericRuntimeException("Cannot get Avro type for type " + type.getName());
}
/**
* Nested class to make our field.
*/
private static class Field {
public final String name;
public final Object[] type;
public Field(String name, Object[] type) {
this.name = name;
this.type = type;
}
}
}

Key-value map of each value of an enum in Java

Task
In a Java back-end project, I have some (100+) external enums that I can't edit and I need to output them to our front-end. I wanted to output them in a JSON-object like manner. Each enum has different properties name.
e.g. for the following enum
public enum Colors {
RED(1, "RED", "ff0000", Boolean.TRUE),
GREEN(2, "GREEN", "00ff00", Boolean.FALSE),
BLUE(3, "BLUE", "0000ff", Boolean.TRUE);
private int code;
private String label;
private String hexCode;
private boolean isAwesome;
// ... getters and other methods
}
i want to output
[
{
label: "RED"
hexCode: "ff0000"
isAwesome: true
},
{
label: "GREEN"
hexCode: "00ff00"
isAwesome: false
},
...
]
My attempt
I am new to Java, this is the first time I used reflection and I didn't really study anything before going into this. Probably there are some major problems with this code (like performance or some other weird stuff that I don't know), but it compiles and does the job. I don't know if this is safe, so I ask if there are some better ways to do this.
private <T> List<HashMap<String, Object>> enumInserter(Class<T> clazz, List<String> properties) {
return valuesToMap(clazz.getEnumConstants(), parserFactory(clazz, properties));
}
/**
*
* #param <T> type of the enum class
* #param values enumConstants of the enum
* #param parser a function that take a single enumValue of type <T> and returns
* an property-value map
* #return the array of the property-value maps of each value
*/
private <T> List<HashMap<String, Object>> valuesToMap(T[] values, Function<T, HashMap<String, Object>> parser) {
List<HashMap<String, Object>> enumValues = new ArrayList<>();
for (T enumValue : values) {
HashMap<String, Object> processedValue = parser.apply(enumValue);
enumValues.add(processedValue);
}
return enumValues;
}
/**
*
* #param <T> the type of the enum class
* #param clazz the enum class
* #param properties the properties to be added in the map
* #return a parser function that take a single enumValue of type <T> as input and
* returns a property-value map of the given enumValue
*/
private <T> Function<T, HashMap<String, Object>> parserFactory(Class<T> clazz, List<String> properties) {
return ((T enumValue) -> {
HashMap<String, Object> map = new HashMap<>();
properties.stream().forEach(propertyName -> {
String methodName = getterFromProperty(propertyName);
try {
Method method = clazz.getMethod(methodName);
Object methodResult = method.invoke(enumValue);
map.put(propertyName, methodResult);
} catch (Exception e) {
// ... error logging
}
});
return map;
});
}
/**
* Return the "standard" property getter of a property. e.g. "example" will
* return "getExample"
*
* #param property
* #return property getter method name
*/
private String getterFromProperty(String property) {
return "get" + property.substring(0, 1).toUpperCase() + property.substring(1);
}

The usual approach to this is either through use of annotation #JsonFormat(shape = JsonFormat.Shape.OBJECT) or through usage of a custom tailored serializer.
Lets say that you are referencing the enum from class A
class A {
#JsonFormat(shape = JsonFormat.Shape.OBJECT)
private Colors color;
}
this will make the color to be serialized the way you want.
Alternative aproach would to register a custom serializer for your Enum this you can do the following way:
public class ColorSerializer extends StdSerializer {
public ColorSerializer() {
super(Color.class);
}
public ColorSerializer(Class t) {
super(t);
}
public void serialize(Color color, JsonGenerator generator,
SerializerProvider provider)
throws IOException, JsonProcessingException {
generator.writeStartObject();
generator.writeFieldName("code");
generator.writeString(color.getCode());
generator.writeFieldName("hexCode");
generator.writeString(color.getHexcode());
generator.writeFieldName("isAwsome");
generator.writeNumber(color.isAwsome());
generator.writeEndObject();
}
}
Since your enums are external you can always place them in wrappers which are internal to your project and this way control their serialization process.
If you want to serialize them using the same strategy you can place your reflection code in the serializer. This way you will get a single generic serializer, instead of writing for each enum.
This is how you can register the custom serializer:
ObjectMapper mapper = new ObjectMapper();
SimpleModule module = new SimpleModule();
module.addSerializer(Color.class, new ColorSerializer ());
mapper.registerModule(module);

Storing specific data types in hashmap

I have to use a map which stores keys of type Integer, String and Long only.
One solution: To store type Object and in put method check with instanceof operator. Is there any better solution, maybe with enum

You can use a map and storing Long as String into it
or you can use two different hashmap and duplicate put/get methods. If you have two types, it is probably for two different things, and having two different map should probably be the correct answer

Create a class that has a map as a member and add methods that will store and retrieve int and long as Strings.
class MyMap {
private Map mabObject = Map<String, Object>;
public void add(long key, Object value) {
mapObject.put(Long.toString(key),value);
}
public void add(String key, Object value) {
mapObject.put(key, value);
}
public Object get(long key) {
return mapObject.get(Long.toString(key));
}
public Object get(String key) {
return mapObject.get(key);
}
}

I agree with Paul Boddington's comment, and the need of such trick shows that code smells.
Just for a funny excercise (not for production code) I've made an example that shows what we can do in compile time for limiting types of keys in a map.
For example we can create a wrapper allowing only values of specific classes.
common/map/Wrap.java
package common.map;
import java.util.Arrays;
import java.util.List;
public class Wrap<T> {
private T value;
private Wrap(T value){
this.value = value;
}
public T get() {
return this.value;
}
/*
* it's important to implement this method
* if we intend to use Wrap instances as map's key
*
* and it's needed to see that hash codes are computing differently in different classes,
* and depending on `allowedClasses` contents we can face some unexpected collisions
* so if you care of performance - test your maps usage accurately
*/
public int hashCode() {
return this.value.hashCode();
}
/*
* static
*/
private static List<Class> allowedClasses = Arrays.asList(Long.class, String.class);
public static <T> Wrap<T> create(Class<? extends T> clazz, T value) {
if (!allowedClasses.contains(clazz)) {
throw new IllegalArgumentException("Unexpected class " + clazz);
}
return new Wrap<>(value);
}
public static <T> Wrap<T> create(AllowedClasses allowedClass, T value) {
return create(allowedClass.clazz, value);
}
public enum AllowedClasses {
LONG(Long.class),
STRING(String.class);
private Class clazz;
AllowedClasses(Class clazz) {
this.clazz = clazz;
}
}
}
And let's run it
common/map/Example.java
package common.map;
import common.map.Wrap.AllowedClasses;
import java.util.HashMap;
import java.util.Map;
public class Example {
public static void main(String... args) {
Map<Wrap, Object> map = new HashMap<>();
// next two lines create wrappers for values of types we added to enum AllowedClasses
// but since enums cannot have type parameters, we are not able to check
// if the second parameter type is compatible with a type associated with given enum value
// so I think usage of enum is useless for your purpose
Wrap<?> valLong0 = Wrap.create(AllowedClasses.LONG, "the string in place of Long is OK");
Wrap<?> valString0 = Wrap.create(AllowedClasses.STRING, 12345);
// from the next lines you can see how we can use the Wrap class to keep
// only allowed types to be associated with the map keys
Wrap<Long> valLong = Wrap.create(Long.class, 1L); // legal
Wrap<String> valString = Wrap.create(String.class, "abc"); // legal
Wrap<String> valWrong = Wrap.create(String.class, 123); // doesn't compile
Wrap<Object> valWrong2 = Wrap.create(Object.class, 123); // compiles but throws exception in runtime
Object obj = ThirdParty.getObjectOfUnknownClass();
Wrap<?> valDynamic = Wrap.create(obj.getClass(), obj); // compiles but MAYBE throws exception in runtime
// so we get to this point only if all the wrappers are legal,
// and we can add them as keys to the map
map.put(valLong, new Object());
map.put(valString, new Object());
map.put(valDynamic, new Object());
}
}

HashMap<DataType1,DataType2>hm = new HashMap<DataType1,DataType2>();
or
Map<DataType1,DataType2> m = new HashMap<DataType1,DataType2>();
m.put(key, value);
Instead of DataType1 & DataType2 you can add Integer,String,Long ,etc. and use the put(key,value) method to enter key and values into the HashMap.

GSON converts to LinkedHashMap instead of my object

I have this snippet of code:
public abstract class Repository<Entity extends BaseObject> {
...
public void readFromJson(){
String content = "JSON content here";
Gson gson = new Gson();
Type entityType = new TypeToken<JSONObject<Entity>>(){}.getType();
jsonObject = gson.fromJson(content, entityType);
for (Entity ent : jsonObject.getEntities()) ;
}
}
When I try to do the foreach my entities object is no longer of type Entity but LinkedHashMap and I get this exception: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to com.tranca.bookstore.domain.shared.BaseObject
Here is the JSONObject class(created by me)
public class JSONObject<Entity> {
private List<Entity> entities = new ArrayList<Entity>();
private long lastId = -1;
public List<Entity> getEntities() {
return entities;
}
public void setEntities(List<Entity> entities) {
this.entities = entities;
}
public long getLastId() {
return lastId;
}
public void setLastId(long lastId) {
this.lastId = lastId;
}
public void incrementLastId() {
this.lastId++;
}
}
maybe the base object is relevant so I will put the code here:
public abstract class BaseObject implements Serializable {
protected long id = (long) -1;
protected int version = 0;
protected BaseObject(){}
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
public int getVersion() {
return version;
}
public void setVersion(int version) {
this.version = version;
}
}

I had the same / a similar problem. To give a more clear answer in a slightly different context:
I had following Method which produced the error "com.google.gson.internal.LinkedTreeMap cannot be cast to MyType":
/**
* Reads a LinkedHashMap from the specified parcel.
*
* #param <TKey>
* The type of the key.
* #param <TValue>
* The type of the value.
* #param in
* The in parcel.
* #return Returns an instance of linked hash map or null.
*/
public static <TKey, TValue> LinkedHashMap<TKey, TValue> readLinkedHashMap(Parcel in) {
Gson gson = JsonHelper.getGsonInstance();
String content = in.readString();
LinkedHashMap<TKey, TValue> result = gson.fromJson(content, new TypeToken<LinkedHashMap<TKey, TValue>>(){}.getType());
return result;
}
I wanted an easy generic way to read/write linked hashmap. The above solution does not work because the type information of the TypeToken with TKey an TValue will be lost after compilation as far as i understand. And this is the problem. If you change you're code to following example, then it works, because now we explicitly define the type token. I am not so much into java that i understand why in this case it is possible to read the type information at runtime.
/**
* Reads a LinkedHashMap from the specified parcel.
*
* #param <TKey>
* The type of the key.
* #param <TValue>
* The type of the value.
* #param in
* The in parcel.
* #return Returns an instance of linked hash map or null.
*/
public static <TKey, TValue> LinkedHashMap<TKey, TValue> readLinkedHashMap(Parcel in, TypeToken<LinkedHashMap<TKey, TValue>> typeToken) {
Gson gson = JsonHelper.getGsonInstance();
Type type = typeToken.getType();
String content = in.readString();
LinkedHashMap<TKey, TValue> result = gson.fromJson(content, type);
return result;
}
And now you would call the above function like:
readLinkedHashMap(in, new TypeToken<LinkedHashMap<UUID, MyObject>>(){});
A sidenote 1: When writign the linked hash map, you do not need to specify any type token at all. toJson(map) is sufficient.
A sidenote 2 (to a problem which I had): By default gson uses toString() to serialize the key. If you register a type adapter for the key type which is maybe a more complex type, then this type adapter is not applied when serializing, but when deserializing. This leads to a non consistent and therefore failing process. Following options activates complex map key serialization.
gsonBuilder.enableComplexMapKeySerialization()

Finally got it!
The problem was that:
new TypeToken< JSONObject< Entity>>(){}.getType();
returns the type of JSONObject< T> not the specific entity of the subclass that was extending Repository(eg UserRepository extends Repository< User>).
The trick was to create an abstract method to force the subclasses to set the Type for deserialization.
In conclusion if you get this error be sure you have the right type of class (in case you use subclasses be sure it returns the type of you subclass not superclass).