We Keep Coding

Creating POJO/beans dynamically and set values using CGLib

I have a requirement to parse a text file and generate JSON document. The text file has a pattern of text which contains a key which is a name and the value is a huge text of TSV with headers.
I could parse the text file and generate bean classes using the headers and now i want to set the data to this generated bean class. I am using reflection to do this.
Class<?> beanClass = BeanClassGenerator.beanGenerator(k, mapForBeanGeneration);
try {
Object beanClassObject = beanClass.newInstance();
lines.forEach(line -> {
if (line != null && !line.isEmpty() && !line.equals("null")) {
String[] lineData = line.split("\t");
System.out.println("LineData length :: " + lineData.length);
Method[] methods = beanClass.getMethods();
System.out.println("Methods length :: " + methods.length);
int index = 0;
for (Method m : methods) {
m.setAccessible(true);
if (m.getName().startsWith("set")) {
try {
if ((lineData.length <= index) && lineData[index] != null) {
m.invoke(beanClassObject, lineData[index]);
index++;
} else {
m.invoke(beanClassObject, " ");
}
} catch (IllegalAccessException | InvocationTargetException e) {
e.printStackTrace();
}
}
}
}
});
ObjectMapper om = new ObjectMapper();
System.out.println(om.writeValueAsString(beanClassObject));
} catch (InstantiationException | IllegalAccessException | JsonProcessingException e) {
e.printStackTrace();
}});
The problem with the approach is that most of the times all the column values may not have data it can be nulled.
I am wondering if there is an easier way of doing this. Any help is appreciated.
Here is the bean generation method.
public static Class<?> beanGenerator(final String className, final Map<String, Class<?>> properties) {
BeanGenerator beanGenerator = new BeanGenerator();
beanGenerator.setNamingPolicy(new NamingPolicy() {
#Override
public String getClassName(String prefix, String source, Object key, Predicate names) {
return className;
}
});
BeanGenerator.addProperties(beanGenerator, properties);
return (Class<?>) beanGenerator.createClass();
}
Here is the sample text file which needs to be converted to the JSON output.
<Data1>
Col1 col2 col3 col4 col5
even sense met has
root greatest spin mostly
gentle held introduced palace
cold equator remember grandmother
slightly butter depth like
distant second coast everyone
<Data2>
Col1 col2 col3 col4 col5 col6 col7 col8
greatest rope operation flies brown continent combination read
slightly diagram he grandfather where party fifty pour
well put plastic anyway refer careful correct furniture
how since army tongue birthday been clock official
table command specific distant cutting hill movie experience
national though stopped youth army underline five know
<Data3>
Col1 col2 col3 col4 col5 col6 col7 col8 col9 col9 col10
vessels characteristic ship joy than tomorrow high seven future trade
try gray fourth advice week stream motion musical whom tin
limited daughter large rice came home chicken wheat engine box
easy city pair strange stage visitor coach announced allow simple
jet therefore single during construction flag bigger muscle complex pleasure
income several coat range dull cattle damage jump present shake
JSON output:
[{
"<Data1>": [{
"col1": "",
"col2": "",
"col3": "",
"col4": ""
},
{
"col1": "",
"col2": "",
"col3": "",
"col4": ""
},
{
"col1": "",
"col2": "",
"col3": "",
"col4": ""
}
]
}, {
"<Data2>": [{
"col1": "",
"col2": "",
"col3": "",
"col4": "",
"col5": "",
"col6": "",
"col7": "",
"col8": ""
},
{
"col1": "",
"col2": "",
"col3": "",
"col4": "",
"col5": "",
"col6": "",
"col7": "",
"col8": ""
},
{
"col1": "",
"col2": "",
"col3": "",
"col4": "",
"col5": "",
"col6": "",
"col7": "",
"col8": ""
}
]
}]
I came up with a solution using the Maps.
Map<String, List<Map<String, String>>> finalMap = new HashMap<>();
metadataMap.forEach((k, v) -> {
List<Map<String, String>> datamap = new ArrayList<>();
String key = k;
String[] fields = v.getFields();
List<String> lines = v.getLines();
lines.forEach(line -> {
if (line != null && !line.isEmpty() && !line.equals("null")) {
String[] fieldData = line.split("\t");
Map<String, String> eachLineMap = new HashMap<>();
for (int index = 0; index < fields.length; index++) {
if (index < fieldData.length && (fieldData[index] != null && !fieldData[index].isEmpty())) {
eachLineMap.put(fields[index], fieldData[index]);
} else {
eachLineMap.put(fields[index], " ");
}
datamap.add(eachLineMap);
}
}
});
finalMap.put(key, datamap);
});
try {
output = new ObjectMapper().writeValueAsString(finalMap);
}catch(JsonProcessingException e){
e.printStackTrace();
}

You are going way overboard with your solution.
Your data is organized as an array of variable length arrays;
and does not require some crazy on-the-fly class generation solution.
As a side note,
on-the-fly class generation is not inherently crazy;
it is crazy to use on-the-fly class generation in this situation.
Do this:
Look at your data;
it is organized as follows:
first: outer key
second: exactly one line containing a variable number of space separated array of inner keys.
third: some number of lines containing values.
Design a solution to fix your problem
Read the outer key.
Use that value to create the outer key portion of your JSON.
Read the inner keys.
Store these in an array;
use LinkedList,
not ClownList (ArrayList).
Do this until the next empty line:
Read a line of values.
Write the inner JSON; use the inner keys as the keys for this.
Skip empty lines until one of the following:
If at end of file, write the ending portion of the JSON.
If you read the next outer key, goto to line 2 (Read the inner keys) above.
Write the code.

You don't need to write all that logic, you can just use Apache Commons BeanUtils; which provides a utility method (among MANY other utilities), that takes a Map of field names versus field values and populate a given bean with it:
BeanUtils.populate(target, fieldNameValueMap);
Then the only thing you need to implement is the logic to create the fieldNameValueMap Map; which you can do with this simple method:
Map<String, String> createFieldNameValueMap(String headerLine, String valuesLine) {
String[] fieldNames = headerLine.split("\t");
String[] fieldValues = valuesLine.split("\t");
return IntStream.range(0, fieldNames.length)
.mapToObj(Integer::new)
.collect(Collectors.toMap(idx -> fieldNames[idx], idx -> fieldValues[idx]));
}
You can test this solution with the following working demo:
import java.lang.reflect.InvocationTargetException;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import org.apache.commons.beanutils.BeanUtils;
import lombok.Data;
public class DynamicBeanUtils {
static Map<String, String> createFieldNameValueMap(String headerLine, String valuesLine) {
String[] fieldNames = headerLine.split("\t");
String[] fieldValues = valuesLine.split("\t");
return IntStream.range(0, fieldNames.length)
.mapToObj(Integer::new)
.collect(Collectors.toMap(idx -> fieldNames[idx], idx -> fieldValues[idx]));
}
public static void main(String[] args) {
String headerLine = "booleanValue\tintValue\tstringValue\tdoubleValue\totherValue";
String valuesLine = "true\t12\tthis bean will be populated\t22.44\ttest string!!!";
Object target = new MyBean();
try {
BeanUtils.populate(target, createFieldNameValueMap(headerLine, valuesLine));
} catch (IllegalAccessException | InvocationTargetException e) {
// HANDLE EXCEPTIONS!
}
System.out.println(target);
}
#Data
public static class MyBean {
private String stringValue;
private double doubleValue;
private int intValue;
private boolean booleanValue;
private String otherValue;
}
}
This is the maven repository page for this dependency, so you can include it in your build: https://mvnrepository.com/artifact/commons-beanutils/commons-beanutils/1.9.3
I used Lombok in this solution as well, only to save me the pain of writing getter/setters/toString to test this solution; but it is not required for your solution.
Complete code on GitHub
Hope this helps.

I realized that instead of creating the POJOs with a complex approach. It is better to use the Maps and convert them to JSON using Jackson ObjectMapper. Posting for others who think this might be a useful approach.
public String convert(Map<String, ? extends Metadata> metadataMap) {
String output = "";
Map<String, List<Map<String, String>>> finalMap = new HashMap<>();
metadataMap.forEach((k, v) -> {
List<Map<String, String>> datamap = new LinkedList<>();
String key = k;
String[] fields = v.getFields();
List<String> lines = v.getLines();
lines.forEach(line -> {
if (line != null && !line.isEmpty() && !line.equals("null")) {
String[] fieldData = line.split("\t",-1);
Map<String, String> eachLineMap = new HashMap<>();
for (int index = 0; index < fields.length; index++) {
if (index < fieldData.length && (fieldData[index] != null && !fieldData[index].isEmpty())) {
eachLineMap.put(fields[index], fieldData[index]);
} else {
eachLineMap.put(fields[index], " ");
}
datamap.add(eachLineMap);
}
}
});
finalMap.put(key, datamap);
});
try {
output = new ObjectMapper().writeValueAsString(finalMap);
}catch(JsonProcessingException e){
e.printStackTrace();
}
return output;
}

I have to determine from a string the starting and arrival cities

I have the following code which, by means of a keyboard input, gives me the start and arrival .. the start is determined according to the "da" proposition, while the arrival determines it according to the preposition "a" so I'm fighting now is: I want to get the start and the arrival even if I change the order of the propositions .. you know how I could proceed ..
this is the OUTPUT I get :
I want to go from ostuni to trapani
Partenza :ostuni
Arrivo :trapani
but if I wrote like this:
I want to go to ostuni by trapani
I would like to print the same start and finish correctly ..that is
Patenza :trapani
Arrivo :ostuni
Is this processing possible?
thanks a lot for the attention! Good day
package eubot.controller;
import eubot.intent.Intent;
public class EubotEngine {
public Intent getIntent(String stringInput) {
String str1 = "";
String str2 = "";
Intent dictionary = null;
for (String str3 : Intent.keyWord) {
if (stringInput.contains(str3)) {
//System.out.println("La stringa contiene : " + str3);
int indice1 = stringInput.indexOf(str3) + str3.length();
String splittable =
stringInput.substring(indice1,stringInput.length()).trim();
String splittable2[] = splittable.split(" ");
int index = 0;
for (String str : splittable2) {
str = splittable2[index +1];
str1 = str;
System.out.println("Partenza :" + str1);
break;
}
String splittable3[] = splittable.split(" ");
for(String str : splittable3) {
str = splittable3[index + 3];
str2 = str;
System.out.println("Arrivo :" + str2);
break;
}
index++;
dictionary = new Intent();
dictionary.setTesto(stringInput);
}
}
return dictionary;
}
}
package eustema.eubot.intent;
public class Intent {
public String testo;
public String getTesto() {
return testo;
}
public void setTesto(String testo) {
this.testo = testo;
}
public static String[] keyWord = { "devo andare", "voglio andare", "vorrei andare", "devo recarmi"};
public static String[] parameter = { "bari", "roma", "milano","pisa","firenze","napoli","como","torino" };
}
package eustema.eubot.main;
import java.util.Scanner;
import eustema.eubot.controller.*;
import eustema.eubot.intent.*;
public class Test {
public static void main(String[] args) {
System.out.println("<<-|-|-|-|-|-|-|-|-|<<<BENVENUTO IN EuBoT>>>|-|-|-|-|-|-|-|-|->>");
EubotEngine controller = new EubotEngine();
Scanner input = new Scanner(System.in);
String string;
while (true) {
string = input.nextLine();
Intent intent = controller.getIntent(string);
}
}
}

I know this will not be considered a good answer:)
This is non-trivial to solve by means of imperative programming. The reason is there are many forms in which one can express the same intent. Things like filler words, synonyms, inversions and in general things you did not think about could disrupt your algorithm.
Of course it depends on the level of accuracy you want to achieve. If you are happy that this will not work for all cases, you could always put in conditions like:
if (arr[index-1] == "from") setStart(arr[index]);
if (arr[index-1] == "to") setDestination(arr[index]);
Google, Amazon and Apple are battling to improve this sort of human-computer interaction, but they are using a more mathematical/statistical approach through machine learning.
So, if you're looking for state of the art:
Main search terms: context-free grammars.
Other key words: Markov models, Information extraction, vector space models, tf-idf

Take out single record of each type on basis from arraylist

I have a ArrayList of String array. String array looks something like below
["Type-A","Date","Expert"]
["Type-A","Date","07 Expert"]
["Type-A","Date","10 Expert"]
["Type-B","Date","Expert"]
["Type-B","Date","10 Expert"]
["Type-C","Date","07 Expert"]
["Type-C","Date","10 Expert"]
Consider arraylist have above string arrays. I want to take out unique type from each type from arraylist on thee basis of expertise level. If we cosider above list then I want Type-A with level as Expert and Type-B with level as Expert and Type-C with level as Expert10 as in hierarchy Expert is at top and Expert10 and Expert07 are lower to it.
The structure is like I have arraylist which contains string array. Each String array record has Type and expertise level. Arraylist can have multiple records of same Type with different expertise level. I want record of each Type but with highest expertise level. I have a list if different expertise level. Now my confusion is how to use that expertise level list to take out or make another arraylist with single record of each type with highest expertise level.
List of expertise level.
Expert
10 Expert
07 Expert
Professional
Systems
10 System
07 System

Just loop it and find out, using a map to store current max expert item for each type during looping:
private boolean isMoreExpert(String expertPicked, String expertToCompare){
//TODO: return true is expertPicked is lower expert than expertToCompare, false otherwise
return false;
}
private List<String[]> mapToList<Map<String[]> map>{
//TODO: iterate the map and store items to a list
return null;
}
private List<String[]> getMostExpertListOfEachType(List<String[]> items){
Map<String, String[]> tempRecord = new HashMap<>();
for(String[] item in items){
//assume first item is the type
String[] current = tempRecord.get(item[0]);
//assume the third item is the expert
if(current != null ){
if(isMoreExpert(current[2], item[2])){tempRecord.put(item[0], item);}
}else{
tempRecord.put(item[0], item);
}
}
return mapToList(tempRecord);
}
Have not tested the code yet:)

Here I used Pattern-Matcher to extract expert level. Iterate the given array and then created array, If Same type is found - compare expert level - if expert level if created is smaller - then replace with iterated expert.
List<String[]> givenExp = Arrays.asList(
new String[]{"Type-A", "Date", "Expert"},
new String[]{"Type-A", "Date", "Expert07"},
new String[]{"Type-A", "Date", "Expert10"},
new String[]{"Type-B", "Date", "Expert"},
new String[]{"Type-B", "Date", "Expert10"},
new String[]{"Type-C", "Date", "Expert07"},
new String[]{"Type-C", "Date", "Expert10"});
List<String[]> filteredExp = new ArrayList<>();
Pattern pat = Pattern.compile("(?<=Expert)\\d*");
Matcher gmat, fmat;
String gexplvl, fexplvl;
int giexplvl, fiexplvl;
main:
for (String[] gexp : givenExp) {
for (String[] fexp : filteredExp) {
if (fexp[0].equals(gexp[0])) {
gmat = pat.matcher(gexp[2]);
fmat = pat.matcher(fexp[2]);
gmat.find();
fmat.find();
gexplvl = gmat.group();
fexplvl = fmat.group();
if (gexplvl.length() == 0) {
filteredExp.remove(fexp);
filteredExp.add(gexp);
} else {
if (fexplvl.length() != 0 && Integer.parseInt(fexplvl) < Integer.parseInt(gexplvl)) {
filteredExp.remove(fexp);
filteredExp.add(gexp);
}
}
continue main;
}
}
filteredExp.add(gexp);
}
for (String[] fexp : filteredExp) {
for (String val : fexp) {
System.out.printf("%-10s", val);
}
System.out.println();
}
Output :
Type-A Date Expert
Type-B Date Expert
Type-C Date Expert10

With Java 8 Streams, it is as simple as:
list.stream().sorted((s1, s2) -> s2[2].compareTo(s1[2])).filter(isMaxExpert())
.collect(Collectors.toList());
private static Predicate<String[]> isMaxExpert() {
final Map<String, String[]> map = new HashMap<>();
return p -> {
if (map.get(p[0]) != null) {
return false;
} else {
map.put(p[0], p);
return true;
}
};
};
Usage:
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Predicate;
import java.util.stream.Collectors;
public class SelectOne {
public static void main(String[] args) {
List<String[]> list = new ArrayList<>();
list.add(new String[] { "Type-A", "Date", "Expert" });
list.add(new String[] { "Type-A", "Date", "07 Expert" });
list.add(new String[] { "Type-A", "Date", "10 Expert" });
list.add(new String[] { "Type-B", "Date", "Expert" });
list.add(new String[] { "Type-B", "Date", "10 Expert" });
list.add(new String[] { "Type-C", "Date", "07 Expert" });
list.add(new String[] { "Type-C", "Date", "10 Expert" });
List<String[]> modified = list.stream()
.sorted((s1, s2) -> s2[2].compareTo(s1[2])).filter(isMaxExpert())
.collect(Collectors.toList());
for (String[] strArray : modified) {
System.out.println(" " + strArray[0] + " " + strArray[1] + " "
+ strArray[2]);
}
}
private static Predicate<String[]> isMaxExpert() {
final Map<String, String[]> map = new HashMap<>();
return p -> {
if (map.get(p[0]) != null) {
return false;
} else {
map.put(p[0], p);
return true;
}
};
};
}

Array is Returning NULL

My piece of code..
package com.xchanging.selenium.utility;
import java.io.IOException;
import java.util.LinkedHashMap;
import org.apache.commons.lang3.ArrayUtils;
public class DataProviderConvertor extends ReadExcel {
public static Object[][] convertData(String sheetName, String testCaseName)
throws IOException {
LinkedHashMap<String, String> table = ReadExcel.getData(sheetName,
testCaseName);
String[] myStringArray = new String[table.size()];
for (String key : table.values()) {
System.out.println("Keyvalues " + key.toString());
String value = key.toString();
ArrayUtils.add(myStringArray, value);
}
System.out.println("1st Index: " myStringArray[0]);
}
}
It is returning
Keyvalues Y
Keyvalues ie
Keyvalues QC
Keyvalues Yes
Keyvalues Rework Checklist Comments
Keyvalues Yes
Keyvalues MRI Updated Comments
1st Index: null
I am expecting 6 elements in this array but all are returning NULL.. Why it is not returning the expected values..??

How about much simpler way.
public static Object[][] convertData(String sheetName, String testCaseName)
throws IOException {
LinkedHashMap<String, String> table = ReadExcel.getData(sheetName,
testCaseName);
String[] myStringArray = table.values().toArray( new String[ 0 ] );
System.out.println("1st Index: " + myStringArray[0]);
}

Try
for (String key : table.values()) {
System.out.println("Keyvalues " + key.toString());
String value = key.toString();
myStringArray =ArrayUtils.addAll(myStringArray, value);
}
Or
int cnt=0;
for (String key : table.values()) {
System.out.println("Keyvalues " + key.toString());
String value = key.toString();
myStringArray[cnt++] =value;
}

ArrayUtil.add method copies the array and then adds the new element at the end of the new copied array.
So, i think that's where the problem lies in your code.
That is why myStringArray is showing the size as 0.As the myStringArray is copied and a new string array is formed and then the element is added to it.

You can try this -:
int index =0;
for (String key : table.values()) {
System.out.println("Keyvalues " + key.toString());
String value = key.toString();
myStringArray[index++] = value;
}

ArrayUtils.add(myStringArray, value);
This method creates and returns a new array with the value added to it.
You need to assign the result:
myStringArray = ArrayUtils.add(myStringArray, value);
It has to be done that way because arrays cannot be resized.

Just two changes required, code is perfectly fine otherwise.
1.) String[] myStringArray = new String[table.size()];, change to
String[] myStringArray = null;
2.) ArrayUtils.add(myStringArray, value); change to
myStringArray = ArrayUtils.add(myStringArray, value);
Rest you can read and debug about this method in API.

Array detects only a part from an input word instead of the entire word

I have a problem in my Android app, it's an app using voice recognition, and Google TTS. It's like a SIRI client. As you can see here when the user says a word given in the array:
String[] g = { "hallo", "heey", "hoi", "hey", "he", "hee", "hay" };
for (String strings : g) {
if (mostLikelyThingHeard.contains(strings)) {
String[] array = { "leuk dat je er bent", "heeyy" };
String randomStr = array[new Random()
.nextInt(array.length)];
tts.speak(randomStr, TextToSpeech.QUEUE_FLUSH, null);
return;
}
}
String[] f = { "haha", "hah", "ha" };
for (String strings : f) {
if (mostLikelyThingHeard.contains(strings)) {
String[] array = { "haha leuk grapje", "hiehaho" };
String randomStr = array[new Random()
.nextInt(array.length)];
tts.speak(randomStr, TextToSpeech.QUEUE_FLUSH, null);
return;
}
}
Everything is working fine but when the user says "hallo" it detects the first two characters "ha" first. Which is mentioned in String array 'f'. So this is annoying, the whole word isn't detected but only a part from it.
When I swap the two string arrays like this:
String[] f = { "haha", "hah", "ha" };
for (String strings : f) {
if (mostLikelyThingHeard.contains(strings)) {
String[] array = { "haha leuk grapje", "hiehaho" };
String randomStr = array[new Random()
.nextInt(array.length)];
tts.speak(randomStr, TextToSpeech.QUEUE_FLUSH, null);
return;
}
}
String[] g = { "hallo", "heey", "hoi", "hey", "he", "hee",
"hay" };
for (String strings : g) {
if (mostLikelyThingHeard.contains(strings)) {
String[] array = { "leuk dat je er bent", "heeyy" };
String randomStr = array[new Random()
.nextInt(array.length)];
tts.speak(randomStr, TextToSpeech.QUEUE_FLUSH, null);
return;
}
}
Then it does detect "hallo" first instead of "ha"
But if I make more than 100 arrays this would be annoying, so how can I let java use the best matching word from a array instead of just a part?
I know it's hard to understand but if you guys don't understand it see my source here:
http://github.com/gi097/PWS
EDIT:
When I change
contains
to
equals
I fix this problem but now I got a new one:
If I create an array like:
"I am"
When the user says: "I am Giovanni" we have the problem that "I am" isn't detected anymore due the equals....
EDIT2
It can be fixed I think with splitting mostLikelythingHeard "I am Giovanni" to "I" "am" "Giovanni" but how?

Instead of
mostLikelyThingHeard.contains(strings)
Use:
Arrays.sort(g);
int index = Arrays.binarySearch(g, mostLikelyThingHeard);
if (index >= 0) {
System.out.println("I heard it properly and its ", g[index]);
}
Contains check for subsequence of String as mentioned here. So if your string was "hallo", then "Hallo".contains("ha") searches for ha within hallo and yes ha is within hallo and hence your issue.