We Keep Coding

Visualizating incorrect dada using the stream filter in Java

I have a csv file of the stock exchange I am taking data from, reading it and printing it under certain conditions. One of the conditions under which I must display them is to display the lowest value of a share of each share. My code only shows the first record of each of the three stocks, and does not show the lowest value of these stocks. Where am I going wrong in my filtering using the filter?
My code
private static void smallestPriceByStock(List<String> stocks) throws ParseException {
List<StockPrice> stockPrices = new ArrayList<>();
HashSet<String> stock = new HashSet<>();
SimpleDateFormat sdfOriginal = new SimpleDateFormat("yyyy-MM-dd");
SimpleDateFormat sdfNovaData = new SimpleDateFormat("dd/MM/yyyy");
try {
for (String s : stocks) {
String[] array = a.split(",");
String name = array[0];
String date = array[1];
Date dataOriginal = sdfOriginal.parse(date);
float fechamento = Float.parseFloat(array[2]);
long volume = Long.parseLong(array[3]);
StockPrice stockPrice = new StockPrice(nome, dataOriginal, price, volume);
stockPrices.add(stockPrice);
Stock st = new Stock(name);
st.addStockPrice(stockPrice);
stock.add(st.getNome());
}
System.out.println("Menores fechamentos");
List<Cotacao> cotas = new ArrayList<>();
double smallestValue = 0;
for(String a : acao) {
for (Cotacao c : cotacoes) {
if (c.getNome().contains(a)) {
smallestValue = cotacoes.stream()
.filter(x -> x.getNome() == a)
.mapToDouble(x -> x.getValue()).summaryStatistics().getMax();
System.out.println("Stock: " + c.getNome() + ", " + "Price: " + smallestValue + ", " + "Date: " + sdfNovaData.format(c.getData()));
break;
}
}
}
for(Cotacao cotacao : cotas){
System.out.println(cotacao.getNome());
}
System.out.println();
} catch (UnsupportedOperationException | ParseException e) {
throw new UnsupportedOperationException("calculaMenorFechamentoPorAcao não implementado", e);
}
}
A little sampe of the csv file
Acao,Data,Fechamento,Volume
OGXP3,2013-01-01,4.38,0
OGXP3,2013-01-02,4.76,45904000
OGXP3,2013-01-03,4.90,38143400
PETR4,2013-01-02,19.69,30182600
PETR4,2013-01-03,20.40,30552600
PETR4,2013-01-04,20.43,36141000
PETR4,2013-01-07,20.08,28069600
VALE5,2013-01-01,40.87,0
VALE5,2013-01-02,42.60,18515700
VALE5,2013-01-03,42.09,15001800
VALE5,2013-01-04,41.36,26351900

You are making this more complicated that it needs to be. The common pseudo code for finding lowest value is:
set lowestValue to null
for each value
if lowestValue == null || value < lowestValue
lowestValue = value
As you have multiple values to look for, each identified by the stock name, you define lowest_value as Map<String, BigDecimal> where the key is the stock identifier. Once you have processed each line in the input data, the map contains the lowest value for each stock. There is no need for nested loops, as everyting is done during the single pass. Once the you have populated the map, you can do the printing in a separate single loop over the map data.

How do I filter a list of java objects at runtime where the variable I am filtering on is not known in advance?

I have a list of java objects, and I want to use stream to filter them at runtime. However, the variable I want to filter on is only known at runtime.
For eg, the user says I want a list of all cats whose fur length is longer than 3cm, I should be able to do
cats.stream().filter(cat -> cat.getFurLength() > 3).collect(Collectors.toList());
The getFurLength() getter however should be dynamically invoked - if the user instead wants to filter by eye colour then I should be able to call
cats.stream().filter(cat -> cat.getEyeColour() == Colour.BLUE).collect(Collectors.toList());
How do I achieve this without writing all possible filters beforehand?
Ideally the user should send something like:
{
eyeColour:{
operator: "equal_to",
value: "BLUE"
},
furLength: {
operator: "greater_than",
value: 3
}
}
and the code should be able to generate the filters dynamically based on these criteria.

Assuming your Cat class follows JavaBean convention you could use java.beans.PropertyDescriptor to access getter Method based on property name.
This allows us to learn what type of value we are dealing with. If it is numeric we can handle greater_than and other operators, but if it is non-numeric we should handle only equals_to operator.
"Simplified" and very limited solution could look like:
NOTE:
- Solution doesn't support primitive numeric types like int. Use Integer, Double etc. instead.
- I am converting all numbers to BigDecimal and use compareTo to simplify numerical type comparison, if you get any bugs for big numbers or very precise ones feel free to replace it with proper type comparison).
- for equality check it compares string representation of objects (result of toString()), so for Color you can't use BLUE but your JSON would need to hold java.awt.Color[r=0,g=0,b=255])
class PredicatesUtil {
static <T> Predicate<T> filters(Class<T> clazz, String filtersJson) {
JSONObject jsonObject = new JSONObject(filtersJson);
List<Predicate<T>> predicateList = new ArrayList<>();
for (String property : jsonObject.keySet()) {
JSONObject filterSettings = jsonObject.getJSONObject(property);
try {
String operator = filterSettings.getString("operator");
String value = filterSettings.getString("value");
predicateList.add(propertyPredicate(clazz, property, operator, value));
} catch (IntrospectionException e) {
throw new RuntimeException(e);
}
}
return combinePredicatesUsingAND(predicateList);
}
static <T> Predicate<T> combinePredicatesUsingAND(List<Predicate<T>> predicateList) {
return t -> {
for (Predicate<T> pr : predicateList) {
if (!pr.test(t))
return false;
}
return true;
};
}
static <T> Predicate<T> propertyPredicate(Class<T> clazz, String property,
String operator, String value)
throws IntrospectionException {
final Method m = new PropertyDescriptor(property, clazz).getReadMethod();
final Class<?> returnType = m.getReturnType();
return obj -> {
try {
Object getterValue = m.invoke(obj);
if (Number.class.isAssignableFrom(returnType)) {
BigDecimal getValue = new BigDecimal(getterValue.toString());
BigDecimal numValue = new BigDecimal(value);
int compared = getValue.compareTo(numValue);
if (operator.equalsIgnoreCase("equal_to")) {
return compared == 0;
} else if (operator.equalsIgnoreCase("lesser_than")) {
return compared < 0;
} else if (operator.equalsIgnoreCase("greater_than")) {
return compared > 0;
} else {
throw new RuntimeException("not recognized operator for numeric type: " + operator);
}
} else {
//System.out.println("testing non-numeric, only euals_to");
if (operator.equalsIgnoreCase("equal_to")) {
return value.equalsIgnoreCase(getterValue.toString());
}
throw new RuntimeException("not recognized operator: " + operator);
}
} catch (IllegalAccessException | InvocationTargetException e) {
throw new RuntimeException(e);
}
};
}
}
which can be used like:
class Cat {
private Color eyeColour;
private Integer furLength;
Cat(Color eyeColor, Integer furLength) {
this.eyeColour = eyeColor;
this.furLength = furLength;
}
public Color getEyeColour() {
return eyeColour;
}
public Integer getFurLength() {
return furLength;
}
public void setEyeColour(Color eyeColour) {
this.eyeColour = eyeColour;
}
public void setFurLength(Integer furLength) {
this.furLength = furLength;
}
#Override
public String toString() {
return "Cat{" +
"eyeColor=" + eyeColour +
", furLength=" + furLength +
'}';
}
}
class CatsDemo {
public static void main(String[] args) {
String json =
"{\n" +
" eyeColour:{\n" +
" operator: \"equal_to\",\n" +
" value: \"java.awt.Color[r=0,g=0,b=255]\"\n" +
" },\n" +
" furLength: {\n" +
" operator: \"greater_than\",\n" +
" value: \"3\"\n" +
" }\n" +
"}";
List<Cat> cats = List.of(
new Cat(Color.blue, 1),
new Cat(Color.blue, 2),
new Cat(Color.blue, 3),
new Cat(Color.blue, 4),
new Cat(Color.blue, 5),
new Cat(Color.yellow, 1),
new Cat(Color.yellow, 2),
new Cat(Color.yellow, 3),
new Cat(Color.yellow, 4),
new Cat(Color.yellow, 5)
);
cats.stream()
.filter(PredicatesUtil.filters(Cat.class, json))
.forEach(System.out::println);
}
}
Output:
Cat{eyeColor=java.awt.Color[r=0,g=0,b=255], furLength=4}
Cat{eyeColor=java.awt.Color[r=0,g=0,b=255], furLength=5}

Make it reusable with a function.
List<Cat> filterCats(cats, Predicate<Cat> filter) {
return cats.stream().filter(filter).collect(Collectors.toList());
}
And then use it with:
filterCats(cats, cat -> cat.getEyeColour() == Colour.BLUE)
Or,
filterCats(cats, cat -> cat.getFurLength() > 3)

For what it's worth: Apache Commons BeanUtils library is specialized in accessing bean properties in a dynamic way.
See BeanPropertyValueEqualsPredicate for an understanding. This is only a solution for equality matches.

parse JSON Tree in java

I could not find out how to traverse a JSON-tree (nested structure) and decide on the keys of the elements what to expect next and traverse from node to node. Like this (pseudocode):
int traverse(node) {
if (node.key == ADD) {
int res = 0;
for (child:node.children)
res = res + traverse(child);
}
if (node.key == "ADD") {
int res = 0;
for (child:node.children)
res = res + traverse(children);
}
if (node.key == "MULT") {
int res = 0;
for (child:node.children)
res = res * traverse(children);
}
if (node.key == "INT")
return node.value;
}
The json-string to parse could be like this:
{"ADD":[{"INT":"1"},{"INT":"3"},{"INT":"4"}]}
or this:
{"ADD":[{"INT":"1"},{"INT":"3"},{"ADD":[{"INT":"5"},{"INT":"6"}]},
{"INT":"4"}]}
How could I use JSON-Object or JSON-Arrays and
inside the objects access the key and value variables to traverse through this tree recursively?
EDITED:
After all the comments I try to put this as first running example
(still looks a little uneasy to me, but it works):
public static int evaluate(javax.json.JsonObject node) {
Set<?> keySet = node.keySet();
Iterator<?> i = keySet.iterator();
if (i.hasNext()) {
String key = i.next().toString();
System.out.println("key: " + key);
if (key.equals("ADD")) {
JsonArray ja = node.getJsonArray("ADD");
int res = 0;
for (JsonValue jvx: ja) {
if (jvx.getValueType().toString().equals("OBJECT")) {
res = res + evaluate((JsonObject)jvx);
} else{
System.err.println("jvx should not be a " + jvx.getValueType().toString() + " here");
}
}
return res;
}
if (key.equals("MULT")) {
JsonArray ja = node.getJsonArray("MULT");
int res = 1;
for (JsonValue jvx: ja) {
if (jvx.getValueType().toString().equals("OBJECT")) {
res = res * evaluate((JsonObject)jvx);
} else{
System.err.println("jvx should not be a " + jvx.getValueType().toString() + " here");
}
}
return res;
}
if (key.equals("INT")) {
String intStr = node.getString("INT");
System.out.println ("found int = " + intStr);
return Integer.parseInt(intStr);
}
}
return 0;
}
public static void readJSON() {
String jsonText = "{\"ADD\":[{\"INT\":\"1\"},{\"INT\":\"3\"},{\"ADD\":[{\"INT\":\"5\"},{\"INT\":\"6\"}]},{\"INT\":\"4\"}]}";
JsonReader reader = Json.createReader(new StringReader(jsonText));
JsonObject obj = reader.readObject();
reader.close();
int res = evaluate(obj);
System.out.println("res: " + res);
}

Your evaluating pseudocode is OK (just pay attention to the initial value when multiplying!). To adapt it to the javax.json hierarchy, you should code your evaluating method like this:
int evaluate(javax.json.JsonObject node):
Get each on of the admitted keys (ADD, MULT, INT, etc) through node.getJsonObject(key): In case it returns null, check the next admitted key, and stop at the first you find.
On each operation, a proper logic must be coded:
In case the key is a constant value (INT), return its value immediately.
In case the key is an operation, check the value's type (through node.getValueType()): If it is a single value, return it as is. If it is an array, iterate through its elements and call evaluate for each one of them, and perform the proper operation with the returned value (adding, multiplying, etc). Last, return the computation's result.
After your first edit
Your first real approach looks OK; I'd just suggest you some improvements to make the code more readable:
Use an enhanced for.
Replace the if-else chanin by a switch.
Replace each case by a call to a private method.
int result;
Set<String> keySet = node.keySet();
for (String key : keySet)
{
switch (key)
{
case "ADD":
result=evaluateAdd(node.getJsonArray("ADD"));
break;
case "MULT":
result=evaluateMult(node.getJsonArray("ADD"));
break;
case "INT":
result=node.getInt("INT");
break;
...
}
}

JSON [{
"name": "com",
"children": [
{
"name": "project",
"children": [
{
"name": "server"
},
{
"name": "client",
"children": [
{
"name": "util"
}
]
}
]
}
]
}]
Try this:
public static JSONArray getNames(JSONArray inputArray, JSONArray outputArray) {
for (int i = 0; i < inputArray.length(); i++) {
JSONObject inputObject = inputArray.getJSONObject(i);
JSONObject outputObject = new JSONObject();
outputObject.put("name", inputObject.getString("name"));
outputArray.put(outputObject);
if (inputObject.has("children")) {
JSONArray children = inputObject.getJSONArray("children");
getNames(children, outputArray);
}
}
return outputArray;
}

find the most and least used string in an ArrayList

I am having trouble finding the most and least used String in an ArrayList. The program should go through a file of Strings and count how many multiple strings there are in the list. Then print the least and most used name in the list. The ArrayList Part is finished. It is just finding the most and least common name I am having trouble with. I have no idea how to even start with it. This is what I have found online but it is not working.
Map<String, Integer> dogNames = new HashMap<>();
for (Dog dog : dogs) {
Integer value = dogNames.get(dog);
if (value == null) {
value = 0;
}
value++;
dogNames.put(dog.getName(), value);
}
int leastCommon = Integer.MAX_VALUE;
String leastCommonName = null;
for (String name : dogNames.keySet()) {
int value = dogNames.get(name);
if (value < leastCommon) {
leastCommon = value;
leastCommonName = name;
}
}
System.out.println("Least common (" + leastCommon + ") is " + leastCommonName);

The problem with your code seems to be in this line:
Integer value = dogNames.get(dog);
Your map holds dog names (String), but you are getting the entry for the Dog, which does not exist! Thus, value stays 0 even if you've seen that name before. If you fix this, you code should work.
Instead of your loop for searching the least common name, you could also define a custom Comparator based on the counts in the map and then use Collections.min and Collections.max:
Comparator<Dog> comp = new Comparator<Dog>() {
#Override
public int compare(Dog o1, Dog o2) {
return Integer.compare(dogNames.get(o1.getName()), dogNames.get(o2.getName()));
}
};
System.out.println("least " + Collections.min(dogs, comp));
System.out.println("most " + Collections.max(dogs, comp));
With Java 8, you can make it even shorter, using Comparator.comparing:
List<Dog> dogs = ...
Map<String, Integer> dogNames = new HashMap<>();
dogs.forEach(dog -> dogNames.put(dog.getName(), dogNames.getOrDefault(dog.getName(), 0) + 1));
Comparator<Dog> comp = Comparator.comparing(d -> dogNames.get(d.getName()));
System.out.println("least " + Collections.min(dogs, comp));
System.out.println("most " + Collections.max(dogs, comp));
Or even shorter, using Collections.frequency instead of building your own map, and using that to compare. Note, however, that this will be wasteful if the list is very long, since this will search the list each time anew instead of caching the counts in the map.
List<Dog> dogs = ...
Comparator<Dog> comp = Comparator.comparing(d -> Collections.frequency(dogs, d.getName()));
System.out.println("least " + Collections.min(dogs, comp));
System.out.println("most " + Collections.max(dogs, comp));

Your code should look something like this...
Map<String,int> frequencyOfDogNames = new HashMap<String,int>();
for(String dogName:dogNames) {
if(frequencyOfDogNames.contains(dogName)) {
continue;
}
frequencyOfDogNames.put(dogName, Collections.frequency(dogs, "dogName"));
}
This will give you the map of all the names with the occurrences.
Now we should loop thought the map to see which one are the max and min...
int leastCommon = Integer.MAX_VALUE;
int mostCommon = 0;
String leastCommonName, mostCommonName;
int occurrence;
for(String dogName: frequencyOfDogNames.keySet()) {
occurrence = frequencyOfDogNames.get(dogName);
if(leastCommon > occurrence){
leastCommon = occurrence;
leastCommonName = dogName;
}
if(mostCommon < occurrence){
mostCommon = occurrence;
mostCommonName = dogName;
}
}

How can I flag duplicates in an array?

I would like to know a method for flagging values in an array, removing the duplicates and combining some of the data in Java.
I am keeping a record of geo locations using lat, long and description this is encoded in a JSON array as follows:
[{"lon": 0.001, "lat": 0.001, "desc": test}, {"lon": 0.001, "lat": 0.001, "desc": test2}]
I would like to be able to remove the duplicate geo locations while keeping the "desc" part of the array, e.g.
[{"lon": 0.001, "lat": 0.001, "desc": test, test2}]
Edit:
This is what I am currently doing:
//Store locPoints from server in JSONArray
JSONArray jPointsArray = new JSONArray(serverData);
List<JSONObject> jObjects = new ArrayList<JSONObject>();
List<JSONObject> seenObjects = new ArrayList<JSONObject>();
for(int i = 0; i < jPointsArray.length(); ++i)
{
jObjects.add(jPointsArray.getJSONObject(i));
}
for (JSONObject obj : jObjects)
{
//This always returns true
if (!seenObjects.contains(obj))// && !seenObjects.contains(obj.get("lon")))
{
Log.i("Sucess", "Huzzah!");
seenObjects.add(obj);
}
else
{
//merge the 'desc' field in 'obj' with the 'desc' field in
JSONObject original = (JSONObject)seenObjects.get(seenObjects.indexOf(obj));
JSONObject update = obj;
original.put("desc", original.get("desc") + ", " + update.get("desc"));
seenObjects.get(seenObjects.indexOf(obj)).get("desc"));
}
}

You could do something like:
//assuming that the array you are filtering is called 'myArray'
List<Object> seenObjects = new ArrayList<Object>();
for (Object obj : myArray) {
if (! seenObjects.contains(obj)) {
seenObjects.add(obj);
}
else {
//merge the 'desc' field in 'obj' with the 'desc' field in
//'seenObjects.get(seenObjects.indexOf(obj))'
}
}
Note that this will only work if the objects you are comparing have implementations of equals() and hashCode() that do what you want (in your case, they should only take into consideration the 'lat' and 'lon' fields).
Update:
Here is some complete example code:
import java.util.ArrayList;
import java.util.List;
import org.json.simple.JSONObject;
import org.json.simple.JSONValue;
public class JsonMergeTest {
#SuppressWarnings({ "rawtypes", "unchecked" })
public static void main(String[] args) {
List<Object> myArray = new ArrayList<Object>();
myArray.add(MyJsonObject.parse("{\"lon\": 0.001, \"lat\": 0.001, \"desc\": \"test\"}"));
myArray.add(MyJsonObject.parse("{\"lon\": 0.001, \"lat\": 0.001, \"desc\": \"test2\"}"));
List seenObjects = new ArrayList<Object>();
for (Object obj : myArray) {
if (! seenObjects.contains(obj)) {
seenObjects.add(obj);
}
else {
//merge the 'desc' field in 'obj' with the 'desc' field in the list
MyJsonObject original = (MyJsonObject)seenObjects.get(seenObjects.indexOf(obj));
MyJsonObject update = (MyJsonObject)obj;
original.put("desc", original.get("desc") + ", " + update.get("desc"));
}
}
for (MyJsonObject obj : (List<MyJsonObject>)seenObjects) {
System.out.println(obj.toJSONString());
}
}
private static class MyJsonObject extends JSONObject {
#Override
public boolean equals(Object obj) {
if (obj == null || ! (obj instanceof MyJsonObject) || ! this.containsKey("lat") || ! this.containsKey("lon")) {
return super.equals(obj);
}
MyJsonObject jsonObj = (MyJsonObject)obj;
return this.get("lat").equals(jsonObj.get("lat")) && this.get("lon").equals(jsonObj.get("lon"));
}
#Override
public int hashCode() {
if (! this.containsKey("lat") || ! this.containsKey("lon")) {
return super.hashCode();
}
return this.get("lat").hashCode() ^ this.get("lon").hashCode();
}
#SuppressWarnings("unchecked")
public static Object parse(String json) {
Object parsedJson = JSONValue.parse(json);
if (! (parsedJson instanceof JSONObject)) {
return parsedJson;
}
MyJsonObject result = new MyJsonObject();
result.putAll((JSONObject)parsedJson);
return result;
}
}
}

You can use GSon. And follow the steps:
1. Define an equivalent POJO in Java, to map the JSON String
public class Location implements Comparable<Location> {
public String lon;
public String lat;
public String desc;
#Override
public String toString() {
return "<lon: " + lon +", lat: "+ lat +", desc: " + desc +">";
}
#Override
public boolean equals(Object obj) {
return ((Location)obj).lon.equals(lon) && ((Location)obj).lat.equals(lat);
}
public int compareTo(Location obj) {
return ((Location)obj).lon.compareTo(lon) + ((Location)obj).lat.compareTo(lat);
}
}
2. Write the code that merges similar location. OK, it's Sunday, lets do it :)
public static void main(String[] args){
//Some test data
String s = "[" +
" {\"lon\": 0.001, \"lat\": 0.001, \"desc\": \"test\"}," +
" {\"lon\": 0.002, \"lat\": 0.001, \"desc\": \"test3\"}," +
" {\"lon\": 0.002, \"lat\": 0.005, \"desc\": \"test4\"}," +
" {\"lon\": 0.002, \"lat\": 0.001, \"desc\": \"test5\"}," +
" {\"lon\": 0.001, \"lat\": 0.001, \"desc\": \"test2\"}]";
Gson gson = new Gson();
Location[] al = gson.fromJson(s, Location[].class);
List<Location> tl = Arrays.asList(al);
//lets sort so that similar locations are grouped
Collections.sort(tl);
List<Location> fl = new ArrayList<Location>();
Location current = null;
//merge!
for(Iterator<Location> it = tl.iterator(); it.hasNext();){
current = current==null?it.next():current;
Location ltmp = null;
while(it.hasNext() && (ltmp = it.next()).equals(current))
current.desc = current.desc + "," + ltmp.desc;
fl.add(current);
current = ltmp;
}
//convert back to JSON?
System.out.println(gson.toJson(fl));
}
3. output
[{"lon":"0.002","lat":"0.005","desc":"test4"},
{"lon":"0.002","lat":"0.001","desc":"test3,test5"},
{"lon":"0.001","lat":"0.001","desc":"test,test2"}]