We Keep Coding

How to find duplicate elements in a Stream in Java

I'm trying to find duplicate entries in map values. But the thing is the list of values have multiple attributes/properties. Basically, if a title shows up more than once in a database, I would mark one entry as unique and mark the rest as duplicates.
Here's my current code:
// I have a Map that looks like...
host1 : id | title | host1 | url | state | duplicate
id | title | host1 | url | state | duplicate
host2 : id | title | host2 | url | state | duplicate
id | title | host2 | url | state | duplicate
for (Map.Entry<String, List<Record>> e : recordsByHost.entrySet()) {
boolean executed = false;
for (Record r : e.getValue()) {
int frequency = Collections.frequency(
e
.getValue()
.stream()
.map(Record::getTitle)
.collect(Collectors.toList()),
r.getTitle()
);
if ((frequency > 1) && (!executed)) {
markDuplicates(r.getId(), r.getTitle());
executed = true;
} else {
executed = false;
}
The issue is when frequency is more than 2 (three records with the same title), the line evaluates to false and treats the third record / second duplicate as "unique".
I've been trying to rework my logic but I'm afraid I'm stuck. Any help / suggestions to get me unstuck would be greatly appreciated.

Set.add (and in fact, Collection.add) returns true if and only if the value was actually added to the Set. Since a Set always enforces uniqueness, you can use this to find duplicates:
void markDuplicates(Iterable<? extends Record> records) {
Set<String> foundTitles = new HashSet<>();
for (Record r : records) {
String title = r.getTitle();
if (title != null && !foundTitles.add(title)) {
// title was not added, because it's already been found.
markAsDuplicate(r);
}
}
}

How to correctly handle this Date field mapping using JdbcTemplate into a Spring Boot project?

it is pretty long time that I am not using Spring and I am finding some difficulties with this JdbcTemplate row mapper mapping a date field.
I try to explain my situation in details.
On my database (MariaDB) I have this table named TREND002:
+-----------------+----------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+----------+------+-----+---------+-------+
| Time_Stamp | datetime | NO | PRI | NULL | |
| Time_Stamp_ms | int(11) | NO | PRI | NULL | |
| Sensor001DV | double | YES | | NULL | |
| Sensor002DV | double | YES | | NULL | |
| Sensor003DV | double | YES | | NULL | |
..............................................................
..............................................................
..............................................................
| Sensor00NDV | double | YES | | NULL | |
NOTE-1: the Time_Stamp field contains value as 2010-10-22 16:50:12
Then I have this DTO class mapping the field of the previous table:
public class TrendDTO {
private Date dateTime;
private int timeStampMs;
private String sensorValue;
public TrendDTO() {
super();
}
public TrendDTO(Date dateTime, int timeStampMs, String sensorValue) {
super();
this.dateTime = dateTime;
this.timeStampMs = timeStampMs;
this.sensorValue = sensorValue;
}
public Date getDateTime() {
return dateTime;
}
public void setDateTime(Date dateTime) {
this.dateTime = dateTime;
}
public int getTimeStampMs() {
return timeStampMs;
}
public void setTimeStampMs(int timeStampMs) {
this.timeStampMs = timeStampMs;
}
public String getSensorValue() {
return sensorValue;
}
public void setSensorValue(String sensorValue) {
this.sensorValue = sensorValue;
}
#Override
public String toString() {
return "TrendDTO [dateTime=" + dateTime + ", timeStampMs=" + timeStampMs + ", sensorValue=" + sensorValue + "]";
}
}
Basially in this DTO class I have:
A Date field mapping the datetime field of my table.
An int field mapping the Time_Stamp_ms field of my table.
A String field mapping the value of a specific SensorXXXDV field of the table (I know that from a relational database point of view this is ugly as hell...but I inherited the project and at the moment I can't change).
Finnally I have a repository class in which there is this method:
#Override
public List<TrendDTO> findTrendListBySensorName(String tableName, String columnName) {
List<TrendDTO> result = null;
String RETRIEVE_TREND_BY_SENSOR_NAME = "SELECT Time_Stamp, Time_Stamp_ms, Sensor240DV FROM Delphys.TREND003";
//result = jdbcTemplate.query(RETRIEVE_TREND_BY_SENSOR_NAME, BeanPropertyRowMapper.newInstance(TrendDTO.class));
result = jdbcTemplate.query(
RETRIEVE_TREND_BY_SENSOR_NAME,
(rs, rowNum) ->
new TrendDTO(
new java.util.Date(rs.getTimestamp("Time_Stamp").getTime()),
rs.getInt("Time_Stamp_ms"),
String.valueOf(rs.getDouble(columnName))
)
);
return result;
}
It works fine but doing in this way I obtain a list of TrendDTO instances containing dateTime field values like this:
dateTime=Fri Oct 22 16:50:12 PDT 2010
The date is perfectly correct but I think that this is the wrong format. I have to return this DTO object in JSON format to a front end and I need a date in the format:
2010-10-22 16:50:12
I also tried to change my mapper code using this:
result = jdbcTemplate.query(
RETRIEVE_TREND_BY_SENSOR_NAME,
(rs, rowNum) ->
new TrendDTO(
//new java.util.Date(rs.getTimestamp("Time_Stamp").getTime()),
rs.getTimestamp("Time_Stamp"),
rs.getInt("Time_Stamp_ms"),
String.valueOf(rs.getDouble(columnName))
)
);
As you can see I am simply using rs.getTimestamp("Time_Stamp") but doing in this way I am obtaining a dateTime inizialization like this: 2010-10-22 16:50:12.0
As you can see it end with a .0 that should represents milliseconds that I don't want. Can I specify the format to avoid to put also this ending millisecond section?
Another possible approach to solve my problem (maybe the best solution) is: in this pretty horrible database table the millisecond information is contained into the Time_Stamp_ms column that, at the moment, is mapped with my timeStampMs DTO field.
Can I modify my previous code to encapsulate this information directly into the dateTime field of my DTO object?
I am absolutly not sure that with this syntax (is it a lambda expression?) this is possible. Can I do it using this syntax or have I to implement a row mapper class to implement this behavior?

I would do this:
Don't use the old "Date" API anymore, use the newer (Java8+) LocalDateTime instead. Assuming you have a relatively recent JDBC driver, you can do this to get a LocalDateTime, it saves you a few strokes trying to convert your timestamp to a date.
LocalDateTime yourLocalDateTime = rs.getObject("Time_Stamp", LocalDateTime.class) ;
You can format that LocalDateTime any way you want. To do that manually, you would use a DateTimeFormatter.
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
String formatDateTime = yourLocalDateTime.format(formatter);
Though if you are in a Spring Boot project and serializing to JSON, all you need to do is annotate your DTO with the #JsonFormat annotation:
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd HH:mm:ss")
private LocalDateTime myLocalDateTime;

Better practice for custom rows and cells in a tableview JavaFX

I made a table-view that looks like the following:
For doing this I made the following:
1.- Create and observable list of the POJO that represents the table "Modulo" in MySQL database, with this list I created the columns of the table-view with this code:
public ObservableList<Modulo> cargaTablaHabilitadosMasHoras(Connection conex){
ObservableList<Modulo> listaMod = FXCollections.observableArrayList();
String sql = "SELECT id_Mod, nombre_Mod, \n" +
" Turno_Mod, capacidad_Mod, id_Us, \n" +
" status_Mod \n"+
"FROM Modulo;";
//Static first column
listaMod.add(new Modulo("123", "Horario", "Rhut", 10, "123", true));
try(PreparedStatement stta = conex.prepareStatement(sql);
ResultSet res = stta.executeQuery()) {
while (res.next()) {
if (res.getBoolean("status_Mod")) {
listaMod.add(new Modulo( res.getString ("id_Mod"),
res.getString ("nombre_Mod"),
res.getString ("Turno_Mod"),
res.getInt ("capacidad_Mod"),
res.getString ("id_Us"),
res.getBoolean("status_Mod")));
}
}
}catch (SQLException ex) {
ex.printStackTrace();
}
return listaMod;
}
2.- Create a table with the custom data with this code:
public void otraTabla(Connection conex){
//loads the observable list of the POJO that represents the table Modulo
columns = modu.cargaTablaHabilitadosMasHoras(conex);
/*
creates and observable list that is going to be the base
of the tableview creating a grid of 8 x number of colums
obtained of the first list + 1 column that represents the hours
*/
ObservableList<String> row = FXCollections.observableArrayList();
row.addAll("1","2","3","4","5","6","7","8");
//for loop that iterates the tableview columns
for(Modulo columName : columns) {
//creates and column object to be integrated and manipulated
//whit the name of the column in the first list
TableColumn<String, String> col = new TableColumn(columName.getNombre_Mod());
//verify if is the first column with contains the hours
if (columName.getNombre_Mod().equals("Horario")) {
//if is the one creates the rows with the hours staring at 6 am
col.setCellValueFactory(cellData -> {
//star at 6 am
LocalTime lol = LocalTime.of(6, 0);
//get the value of ObservableList<String> row for for adding to LocalTime
Integer p = Integer.valueOf(cellData.getValue());
//adds the value to localtime
lol = lol.plusHours(p);
//Gives a format for the hour
DateTimeFormatter kk = DateTimeFormatter.ofPattern("hh:mm");
//returns the new String
return new SimpleStringProperty(lol.format(kk));
});
}else{
//if is a column load dinamically then gets
//the next date where there is space in the column at that time
col.setCellValueFactory(cellData -> {
String regresaFecha = "";
//Conection to the database it conection to the database it
//have to be inside of the loop or else the conection is lost
try(Connection localConnection = dbConn.conectarBD()) {
//get the level of the row in this case the hour
LocalTime lol = LocalTime.of(6, 0);
Integer p = Integer.valueOf(cellData.getValue());
lol = lol.plusHours(p);
//calls the method that calculed the next date where there is space in the table of the database
LocalDate fechaApunter = rehab.compararDiaADia(localConnection, Date.valueOf(LocalDate.now()),
Time.valueOf(lol), columName.getId_Mod(), columName.getCapacidad_Mod(), 30);
//date sent to the row of the tableview
regresaFecha = fechaApunter.toString();
} catch (SQLException e) {
e.printStackTrace();
}
return new SimpleStringProperty(regresaFecha);
});
}
//change color of the date depending of the
//distant relevant to the day is making the query to the database
if (!columName.getNombre_Mod().equals("Horario")) {
col.setCellFactory (coli -> {
TableCell<String, String> cell = new TableCell<String, String>() {
#Override
public void updateItem(String item, boolean empty) {
super.updateItem(item, empty);
if (item != null) {
LocalDate lol = LocalDate.parse(item);
Text text = new Text(item);
if (lol.isAfter(LocalDate.now())) {
if (lol.isAfter(LocalDate.now().plusDays(5))) {
text.setStyle(" -fx-fill: #990000;" +
" -fx-text-alignment:center;");
}else
text.setStyle(" -fx-fill: #cccc00;" +
" -fx-text-alignment:center;");
}
this.setGraphic(text);
}
}
};
return cell;
});
}
//add the column to the tableview
tvDisponivilidad.getColumns().addAll(col);
}
//add the Observable list place holder
tvDisponivilidad.setItems(row);
}
For loading the data I used this method:
public LocalDate compararDiaADia(Connection conex, Date fecha, Time hora,
String id_Mod, int capacidad, int dias){
LocalDate contador = fecha.toLocalDate();
LocalDate disDeHoy = LocalDate.now();
for (int i = 0; i < dias; i++) {
contador = fecha.toLocalDate();
contador = contador.plusDays(i);
String sttm = "SELECT COUNT(id_Reab) AS Resultado\n" +
"FROM Rehabilitacion\n" +
"WHERE '"+contador+"' BETWEEN inicio_Reab AND fin_Reab\n" +
"AND horario_Reab = '"+hora+"'\n" +
"AND id_Modulo = '"+id_Mod+"';";
try(PreparedStatement stta = conex.prepareStatement(sttm);
ResultSet res = stta.executeQuery(); ) {
if (res.next()) {
if (res.getInt("Resultado") < capacidad || res.getInt("Resultado") == 0) {
disDeHoy = contador;
break;
}else
disDeHoy = contador;
}
} catch (SQLException ex) {
ex.printStackTrace();
}
}
return disDeHoy;
}
What this method does is that for each column it checks where is the next day where there is less of the capacity of the module (each module has different capacity) at certain hour and returns that day, in the calling of the method the hour changes to populate all the rows in the table.
There are several problems with my approach, first is the time, it cost to load the table, it takes like one minute to make the query and populate the table this is a combination of factors but the principal factor is that for every day I made a query to the database and example of this:.
Here is my table where I made the queries:
mysql> SELECT * FROM imssRehab.Rehabilitacion;
+---------+-------------+------------+-----------------+---------+-----------+
| id_Reab | inicio_Reab | fin_Reab | horario_Reab | id_Prog | id_Modulo |
+---------+-------------+------------+-----------------+---------+-----------+
| 1 | 2016-06-01 | 2016-06-10 | 07:00:00.000000 | 1 | 215A3 |
| 2 | 2016-06-01 | 2016-06-10 | 07:00:00.000000 | 1 | 215A3 |
| 3 | 2016-06-01 | 2016-06-10 | 07:00:00.000000 | 1 | 215A3 |
| 4 | 2016-06-01 | 2016-06-10 | 07:00:00.000000 | 1 | 215A3 |
| 5 | 2016-06-01 | 2016-06-10 | 07:00:00.000000 | 1 | 215A3 |
| 6 | 2016-06-01 | 2016-06-10 | 07:00:00.000000 | 1 | 215A3 |
+---------+-------------+------------+-----------------+---------+-----------+
here is my query:
SELECT COUNT(id_Reab) AS Resultado
FROM Rehabilitacion
WHERE '2016-06-01' BETWEEN inicio_Reab AND fin_Reab
AND horario_Reab = '07:00'
AND id_Modulo = '215A3';
The result is 6 in this module my capacity is 5 so I have to advance a day and ask again until it finds a day where are less than 5 in this example until 2016-06-11. To get here I have to make 10 queries and open 10 connections. I use a connection pool and it's very efficient, but it gets overwhelmed by these 10 queries are only for the first row in the first column, normally there are between 15 to 20 columns assuming there is only one query for a row, it still is around 120-160 connections.
I try to reuse a connection every time I can, my first instinct was to use the connection that get pass to the method for loading the Observable List of modules but when I do this the method that makes the query of dates receives the connection closed with out and apparent reason. After many tests I came to the conclusion that has something to do with the lambda of the setCellValueFactory method, and if I want to make a connection it has to be inside creating more connections. I would like i try to alleviate this by loading the table in a different thread with a Task but the results where similar.
A solution to this would be to make a POJO especially for table but I don't think it's possible to create a class dynamically. I could have a POJO whit 20 possible columns and only load the columns that I would use, but what happens when there is more than 20 columns or the name of the modules changes?
So my question is this: how do I make the creation of the table more rapidly? And is there a better way to achieve this table? I don't like my solution with code it's more complex than I would like I'm hoping for a better and cleaner way.

Getting specific data from a cursor

After running a query I have a data like below in a cursor
ID| TOPIC | TITLE | TYPE | NAME |
---------------------------------
1 | AB | BCD | ref | Ferari|
----------------------------------
1 | AB | BCD | ref | TOYOTA|
----------------------------------
2 | BC | ABC | notref| AUDI |
----------------------------------
2 | BC | ABC |notref| BMW |
How can I get the NAME

you can get the NAME.........and you can store all the datas into an arraylist......and retrieve the datas based upon the id value.......
Try this
ArrayList datas=new ArrayList();
ArrayList list=new ArrayList();
String StoreTitle = "", StoreName="";
cursor.moveToFirst();
do{
int getID = cursor.getInt(cursor.getColumnIndexOrThrow("_id"));
String Title = cursor.getString(cursor.getColumnIndexOrThrow("TITLE"));
String StoreName= cursor.getString(cursor.getColumnIndexOrThrow("NAME"));
if(StoreTitle.equalsIgnoreCase(Title)){
list=new ArrayList();
String getTopic = cursor.getString(cursor.getColumnIndexOrThrow("TOPIC"));
String getTitle = cursor.getString(cursor.getColumnIndexOrThrow("TITLE"));
String getType = cursor.getString(cursor.getColumnIndexOrThrow("TYPE"));
String getName = cursor.getString(cursor.getColumnIndexOrThrow("NAME"));
String name=StoreName+" "+getName;
list.add(getTopic);
list.add(getTitle );
list.add(getType );
list.add(name);
datas.remove(getID);
datas.add(getID ,list);
}
else
{
list=new ArrayList();
String getTopic = cursor.getString(cursor.getColumnIndexOrThrow("TOPIC"));
String getTitle = cursor.getString(cursor.getColumnIndexOrThrow("TITLE"));
String getType = cursor.getString(cursor.getColumnIndexOrThrow("TYPE"));
String getName = cursor.getString(cursor.getColumnIndexOrThrow("NAME"));
String name=StoreName+" "+getName;
list.add(getTopic);
list.add(getTitle );
list.add(getType );
list.add(name);
datas.add(getID ,list);
}
StoreTitle = Title;
}while(cursor.moveToNext());

Just modify else part, Keep track of position variable
else
{
String name=arraylist[position].getName;
name=name+" "+cursor.getString(cursor.getColumnIndexOrThrow("Name"));
arraylist.setname(name);
}

Create abstract tree problem from parser

I need big help, I have two simple classes Tree and Node ( I put just interface to use less space on forum, I can easy modify those classes ), I also have flex file and parser file and need to create AST ( abstract syntax tree - to put tokens in Node objects and fill Tree in right way ).
public class Tree {
Node root;
public void AddNode(Node n){}
public void Evaluate(){}
}
public class Node {
public String value;
public int type;
Node left, right;
}
This is parser file
import java_cup.runtime.*;
parser code {:
public boolean result = true;
public void report_fatal_error(String message, Object info) throws java.lang.Exception {
done_parsing();
System.out.println("report_fatal_error");
report_error();
}
public void syntax_error(Symbol cur_token) {
System.out.println("syntax_error");
report_error();
}
public void unrecovered_syntax_error(Symbol cur_token) throws java.lang.Exception {
System.out.println("unrecovered_syntax_error");
report_fatal_error("Fatalna greska, parsiranje se ne moze nastaviti", cur_token);
}
public void report_error(){
System.out.println("report_error");
result = false;
}
:}
init with {: result = true; :};
/* Terminals (tokens returned by the scanner). */
terminal AND, OR, NOT;
terminal LPAREN, RPAREN;
terminal ITEM;
terminal OPEN, CLOSE, MON, MOFF, TIMEOUT, ESERR, BAE, I, O, BUS, EXT, PUSHB;
terminal VAL, OK, BUS_BR_L, BUS_BR_R, SH_CRT_L, SH_CRT_R, BUS_ALL, EXT_ALL, NO_TIMEOUT, NO_ES_ERR, IBUS_OK, CFG_OK, SYNTAX;
terminal OUT;
/* Non-terminals */
non terminal extension;
non terminal Integer expr;
/* Precedences */
precedence left AND, OR;
/* The grammar */
expr ::=
|
expr:e1 AND expr:e2
{:
//System.out.println("AND");
RESULT = 1;
:}
|
expr:e1 OR expr:e2
{:
//System.out.println("OR");
RESULT = 2;
:}
|
NOT expr:e1
{:
//System.out.println("NOT");
RESULT = 3;
:}
|
LPAREN expr:e RPAREN
{:
//System.out.println("()");
RESULT = 4;
:}
|
ITEM extension:e1
{:
//System.out.println("ITEM.");
RESULT = 5;
:}
|
error
{:
System.out.println("error");
parser.report_error();
RESULT = 0;
:}
;
extension ::=
OPEN
|
MON
|
CLOSE
|
MOFF
|
TIMEOUT
|
ESERR
|
BAE
|
I
|
O
|
BUS
|
EXT
|
PUSHB
|
VAL
|
OK
|
BUS_BR_L
|
BUS_BR_R
|
SH_CRT_L
|
SH_CRT_R
|
BUS_ALL
|
EXT_ALL
|
NO_TIMEOUT
|
NO_ES_ERR
|
IBUS_OK
|
CFG_OK
|
SYNTAX
|
OUT
;
This is grammar
%%
%{
public boolean result = true;
//Puni expression sa tokenima radi reimenovanja
public Expression expression=new Expression();
//
public ArrayList<String> items = new ArrayList<String>();
public ArrayList<Integer> extensions = new ArrayList<Integer>();
// ukljucivanje informacije o poziciji tokena
private Symbol new_symbol(int type) {
return new Symbol(type, yyline+1, yycolumn);
}
// ukljucivanje informacije o poziciji tokena
private Symbol new_symbol(int type, Object value) {
return new Symbol(type, yyline+1, yycolumn, value);
}
%}
%cup
%xstate COMMENT
%eofval{
return new_symbol(sym.EOF);
%eofval}
%line
%column
%%
" " {}
"\b" {}
"\t" {}
"\r\n" {}
"\f" {}
"open" {extensions.add(sym.OPEN); return new_symbol(sym.OPEN);}
"close" {extensions.add(sym.CLOSE); return new_symbol(sym.CLOSE);}
"m_on" {extensions.add(sym.MON); return new_symbol(sym.MON);}
"m_off" {extensions.add(sym.MOFF); return new_symbol(sym.MOFF);}
"timeout" {extensions.add(sym.TIMEOUT); return new_symbol(sym.TIMEOUT);}
"es_err" {extensions.add(sym.ESERR); return new_symbol(sym.ESERR);}
"bae" {extensions.add(sym.BAE); return new_symbol(sym.BAE);}
"i" {extensions.add(sym.I); return new_symbol(sym.I);}
"o" {extensions.add(sym.O); return new_symbol(sym.O);}
"bus" {extensions.add(sym.BUS); return new_symbol(sym.BUS);}
"ext" {extensions.add(sym.EXT); return new_symbol(sym.EXT);}
"pushb" {extensions.add(sym.PUSHB); return new_symbol(sym.PUSHB);}
"val" {extensions.add(sym.VAL); return new_symbol(sym.VAL);}
"ok" {extensions.add(sym.OK); return new_symbol(sym.OK);}
"bus_br_l" {extensions.add(sym.BUS_BR_L); return new_symbol(sym.BUS_BR_L);}
"bus_br_r" {extensions.add(sym.BUS_BR_R); return new_symbol(sym.BUS_BR_R);}
"sh_crt_l" {extensions.add(sym.SH_CRT_L); return new_symbol(sym.SH_CRT_L);}
"sh_crt_r" {extensions.add(sym.SH_CRT_R); return new_symbol(sym.SH_CRT_R);}
"bus_all" {extensions.add(sym.BUS_ALL); return new_symbol(sym.BUS_ALL);}
"ext_all" {extensions.add(sym.EXT_ALL); return new_symbol(sym.EXT_ALL);}
"no_timeout" {extensions.add(sym.NO_TIMEOUT); return new_symbol(sym.NO_TIMEOUT);}
"no_es_err" {extensions.add(sym.NO_ES_ERR); return new_symbol(sym.NO_ES_ERR);}
"ibus_ok" {extensions.add(sym.IBUS_OK); return new_symbol(sym.IBUS_OK);}
"cfg_ok" {extensions.add(sym.CFG_OK); return new_symbol(sym.CFG_OK);}
"syntax" {extensions.add(sym.SYNTAX); return new_symbol(sym.SYNTAX);}
"out" {extensions.add(sym.OUT); return new_symbol(sym.OUT);}
"!" { return new_symbol(sym.NOT);}
"&" { return new_symbol(sym.AND);}
"|" { return new_symbol(sym.OR);}
"(" { return new_symbol(sym.LPAREN);}
")" { return new_symbol(sym.RPAREN);}
([[:jletter:]])[[:jletterdigit:]]* \. {items.add(yytext().substring(0, yytext().length()-1)); return new_symbol (sym.ITEM);}
. {result = false;}
Probem is how to create AST from here, I got on input expression something like
A.open && b.i
? Can anybody help ?

The lines in your Parser where you have commented out print statements like:
//System.out.println("OR");
is where you'll want to maintain your AST using the Tree data structure you have. Find out which token will create the tree, add something somewhere in the tree, etc based on your grammar.