I want to encrypt some data stored in a MySQL database using the JPA #Convert options with an AES algorithm. In general, is working fine with all fields, but I am having issues with one of them that is a Timestamp. My version of Hibernate is 4.3.8.Final.
As it is my first time with converters, I am following this GiT example. For this test, the AES encryption is disabled, I will enable it later and is the reason which I want to convert some fields to String. Therefore the issue must be in the converter.
The entity stores a user with several typical information (name, lastname,...) and a birthdate that is stored as a Timestamp. Also, as I want to perform some search by birthdate, I remove all hours, seconds and milliseconds for birthdate in the entity in the setter.
public class User {
#Column(length = 100, nullable = false)
#Convert(converter = StringCryptoConverter.class)
private String firstname;
....
#Column(nullable = false)
#Convert(converter = TimestampCryptoConverter.class)
private Timestamp birthdate;
public void setBirthdate(Timestamp birthdate) {
// Remove time from birthdate. Is useless and can cause troubles when
// using in SQL queries.
if (birthdate != null) {
Calendar calendar = Calendar.getInstance();
calendar.setTimeInMillis(birthdate.getTime());
calendar.set(Calendar.HOUR_OF_DAY, 0); // set hour to midnight
calendar.set(Calendar.MINUTE, 0); // set minute in hour
calendar.set(Calendar.SECOND, 0); // set second in minute
calendar.set(Calendar.MILLISECOND, 0); // set millis in second
this.birthdate = new Timestamp(calendar.getTime().getTime());
} else {
this.birthdate = null;
}
}
....
}
The TimestampCryptoConverter.class has few methods that are very simple. In general the scope is to convert the Timestamp to string to apply later an AES algorithm, I take the time of the timestamp as a long with getTime(), and convert them to String:
#Override
protected Timestamp stringToEntityAttribute(String dbData) {
try {
return (dbData == null || dbData.isEmpty()) ? null : new Timestamp(Long.parseLong(dbData));
} catch (NumberFormatException nfe) {
Logger.errorMessage("Invalid long value in database.");
return null;
}
}
#Override
protected String entityAttributeToString(Timestamp attribute) {
return attribute == null ? null : attribute.getTime() + "";
}
This is a very simple code. And I can store the entity into the database correctly, and retrieve it from database correctly, for example, if I get the user by ID. Therefore, the converter must be correct.
The stored data into MySQL is something like:
# id, birthdate, firstname, ...
'1', '1525384800000', 'TEST', ...
If I search the user by any field, I retrieve the entity with all data correctly converted. The issue appears when I want to perform a search from the birthdate. For example, in my DAO, I have a method is:
public List<User> get(String email, Timestamp birthdate) {
// Get the criteria builder instance from entity manager
CriteriaBuilder criteriaBuilder = getEntityManager().getCriteriaBuilder();
CriteriaQuery<User> criteriaQuery = criteriaBuilder.createQuery(getEntityClass());
// Tell to criteria query which tables/entities you want to fetch
Root<User> typesRoot = criteriaQuery.from(getEntityClass());
List<Predicate> predicates = new ArrayList<Predicate>();
predicates.add(criteriaBuilder.equal(typesRoot.get("email"), email));
if (birthdate != null) {
// Remove hours and seconds.
Calendar calendar = Calendar.getInstance();
calendar.setTimeInMillis(birthdate.getTime());
calendar.set(Calendar.HOUR_OF_DAY, 0); // set hour to midnight
calendar.set(Calendar.MINUTE, 0); // set minute in hour
calendar.set(Calendar.SECOND, 0); // set second in minute
calendar.set(Calendar.MILLISECOND, 0); // set millis in second
birthdate = new Timestamp(calendar.getTime().getTime());
predicates.add(criteriaBuilder.equal(typesRoot.<Timestamp> get("birthdate"), birthdate));
}
criteriaQuery.where(criteriaBuilder.and(predicates.toArray(new Predicate[] {})));
return getEntityManager().createQuery(criteriaQuery).getResultList();
}
As you can see, I also remove the hours, seconds and milliseconds from the search query to match the value on the database.
If I call this method with only the email get('test#email.com', null), it works fine, as before the user is retrieved and the birthdate is correct in the user.
But if I call this method with the birthdate get('test#email.com', 2018-05-04 12:09:05.862) then the result obtained is null. In some unitary tests, the timestamp used in the call is exactly the same parameter used for the creation of the user, and therefore must match the value on the database. For example, I have this unitary tests:
#Test(dependsOnMethods = { "storeUser" })
#Rollback(value = false)
#Transactional(value = TxType.NEVER)
public void searchByMailUser() {
Assert.assertEquals(userDao.getRowCount(), 1);
List<User> dbUsers = userDao.get(EMAIL, null);
Assert.assertTrue(!dbUsers.isEmpty());
User dbUser = dbUsers.iterator().next();
Assert.assertEquals(dbUser.getFirstname(), FIRSTNAME);
Assert.assertEquals(dbUser.getEmail(), EMAIL);
....
Calendar calendar = Calendar.getInstance();
calendar.setTimeInMillis(BIRTHDATE.getTime());
calendar.set(Calendar.HOUR_OF_DAY, 0); // set hour to midnight
calendar.set(Calendar.MINUTE, 0); // set minute in hour
calendar.set(Calendar.SECOND, 0); // set second in minute
calendar.set(Calendar.MILLISECOND, 0); // set millis in second
Timestamp birthdate = new Timestamp(calendar.getTime().getTime());
Assert.assertEquals(dbUser.getBirthdate(), birthdate);
}
That is executed fine, and the last assert tell me that the birthdate is stored and retrieved correctly. But in this test:
#Test(dependsOnMethods = { "storeUser" })
public void searchByMailAndBirthdateUser() {
Assert.assertEquals(userDao.getRowCount(), 1);
Assert.assertTrue(!userDao.get(EMAIL, BIRTHDATE).isEmpty());
}
This test is failed due to no users are found, but passed if changed to:
#Test(dependsOnMethods = { "storeUser" })
public void searchByMailAndBirthdateUser() {
Assert.assertEquals(userDao.getRowCount(), 1);
Assert.assertTrue(!userDao.get(EMAIL, null).isEmpty());
}
BUT, if I disable the converter, both tests are passed.
If the birthdate is correctly retrieved from the database. Why I am having a null value when using birthdate as a criteria?
EDIT
Seems that the method protected String entityAttributeToString(Timestamp attribute); is not used when calling get('test#email.com', 2018-05-04 12:09:05.862).
After some research. It is possible that still Hibernate has some bugs when filtering criteria with an attribute with #Convert. I have make several tests with different options with no success at all.
As a workaround, I have changed the birthdate attribute as a Long.
#Column(nullable = false)
#Convert(converter = LongCryptoConverter.class)
private Long birthdate;
Updating the setters and getters to convert the Timestamp to Long with getTime()
And creating a Long CryptoConverter with very simple methods for converting a Long to String and the opposite:
#Override
protected Long stringToEntityAttribute(String dbData) {
try {
return (dbData == null || dbData.isEmpty()) ? null : Long.parseLong(dbData);
} catch (NumberFormatException nfe) {
UsmoLogger.errorMessage(this.getClass().getName(), "Invalid long value in database.");
return null;
}
}
#Override
protected String entityAttributeToString(Long attribute) {
return attribute == null ? null : attribute.toString();
}
This is working as expected and the criteria filter now is working fine. Still I am not sure why the version with a Timestamp is not working fine.
After some effort, I have changed the version of Hibernate to 5.2.17.Final and the Timestamps are handled correctly with the #Convert annotation. That means that is really an Hibernate bug with version 4.X and this feature is better implemented in Hibernate 5.X
Related
What I tried:-
Select new fully.qualified.classname.ParkingEntry(p.arrivalDate, p.departureDate,
(case when p.chargedAmount is NULL then 0 else p.chargedAmount end) as chargedAmount)
from fully.qualified.classname.ParkingEntry p
Entity ParkingEntry:-
#Entity
class ParkingEntry {
Date arrivalDate;
Date departureDate;
BigDecimal chargedAmount;
ParkingEntry(Date arrivalDate, Date departureDate, BigDecimal chargedAmount) {
this.arrivalDate = arrivalDate;
this.departureDate = departureDate;
this.chargedAmount = chargedAmount;
}
...
}
I am trying to get arrivalDate, departureDate, and chargedAmount from the entity ParkingEntry as an Object. I want to ensure the if chargedAmount is null in the table then it should return the value as 0.
The above query has some syntax errors and hence not working. Any suggestion on how can this be achieved will be highly appreciated.
Use this:
select new fully.qualified.classname.ParkingEntry(p.arrivalDate, p.departureDate, coalesce(p.chargedAmount, 0))
from fully.qualified.classname.ParkingEntry p
I am using a SQLite database with tables that include DATETIME columns. jOOQ default binds the DATETIME columns to java.sql.Timestamp. Querying tables with DATETIME columns causes a NumberFormatException (handled) for each column.
I am using jOOQ 3.11.9.
The exception is thrown in the org.jooq.impl.DefaultBinding.DefaultTimestampBinding parse method when it first tries to convert the timestamp string value as a number.
private static final long parse(Class < ? extends java.util.Date > type, String date) throws SQLException {
// Try reading a plain number first
try {
return Long.valueOf(date);
}
// If that fails, try reading a formatted date
catch (NumberFormatException e) {
// [#7325] In SQLite dates could be stored in both ISO standard formats:
// With T (default standard), or without T (optional standard, JDBC standard)
date = StringUtils.replace(date, "T", " ");
if (type == Timestamp.class)
return Timestamp.valueOf(date).getTime();
// Dates may come with " 00:00:00". This is safely trimming time information
else if (type == Date.class)
return Date.valueOf(date.split(" ")[0]).getTime();
else if (type == Time.class)
return Time.valueOf(date).getTime();
throw new SQLException("Could not parse date " + date, e);
}
}
Looking at the the get0 and set0 DefaultTimestampBinding methods the Timestamp is always get/set as a String. Is there a reason why for SQLite this is not passed to the JDBC statement/result as a Timestamp? Is there any way to override this behavior or avoid the exception?
Override
final void set0(BindingSetStatementContext < U > ctx, Timestamp value) throws SQLException {
if (ctx.family() == SQLITE)
ctx.statement().setString(ctx.index(), value.toString());
else
ctx.statement().setTimestamp(ctx.index(), value);
}
#Override
final Timestamp get0(BindingGetResultSetContext < U > ctx) throws SQLException {
// SQLite's type affinity needs special care...
if (ctx.family() == SQLDialect.SQLITE) {
String timestamp = ctx.resultSet().getString(ctx.index());
return timestamp == null ? null : new Timestamp(parse(Timestamp.class, timestamp));
} else {
return ctx.resultSet().getTimestamp(ctx.index());
}
}
While you could register a custom binding with the code generator, note that this issue will be addressed in the upcoming jOOQ 3.12 release as well as in the next 3.11 service release. See https://github.com/jOOQ/jOOQ/issues/8736 for details.
I need to perfom a select by date from my DB in my spring boot webapp. What I have so far is a list of sport competitions and there respective informations.
Problem : I can not figure out how my select query convert my String type (dateFrom = '2017-05-02' and dateTo = '2017-05-06') to date like '2017-02-12' in the ?
Alos how to fill my RowMapper with more then one date in some competition which have more then one date.
My data base schema:
CREATE TABLE competition (
competition_id integer PRIMARY KEY,
nom varchar(128) NOT NULL,
);
CREATE TABLE date (
id integer PRIMARY KEY,
date_time timestamptz,
competition_id integer REFERENCES competition (competition_id)
);
Json data:
{
"id": "420",
"name": "SOCCER",
"dates": [
"2016-05-12T03:00:00.000Z"
"2016-05-12T04:00:00.000Z"
"2016-05-12T05:00:00.000Z"
]
},
{
"id": "220",
"name": "BASKETBALL",
"dates": [
"2016-05-12T03:00:00.000Z"
"2016-05-12T04:00:00.000Z"
]
}
My competition Class:
public class Competition{
private int id;
private String name;
private String[] dates;
// setters ... getters
}
My RowMapper Class:
public class RowMapper implements RowMapper
{
public Object mapRow(ResultSet rs, int rowNum) throws SQLException {
Competition competition = new Competition();
competition.setId(rs.getInt("id"));
competition.setName(rs.getString("name"));
competition. // How to fill dates
return competition;
}
}
Function to select data :
private static final String SELECT_STMT =
" select * from competition INNER JOIN date ON
+ " competition.competition_id = date.competition_id"
+ " WHERE date(date.date_time) BETWEEN ? AND ?"
;
public List<Competition> findByOptionsAll(String dateFrom, String dateTo ){
List<Competition> competitions = jdbcTemplate.query(SELECT_STMT, new
RowMapper(), dateFrom, dateTo);
return competitions ;
}
Date converting
Right now you have all dates as a String both in your DB and domain model. To convert strings to date you need a date formatter:
private static final String DATE_FORMAT = "dd-MM-yy";
// parsing date; Note you should handle ParseException
java.util.Date date = new SimpleDateFormat(DATE_FORMAT).parse(dateAsString);
// converting date to string
String dateAsString = new SimpleDateFormat(DATE_FORMAT).format(date);
Note that SimpleDateFormat is not thread-safe so it's a good practice to have static final String DATE_FORMAT instead of static final DateFormatter
Converting date and time is tricky in some cases (what about time zone? java.util.Date vs joda.time vs LocalDate from Java 8) but out of scope. I suggest use LocalDate if possible just because it's a new way without old issues.
Mapping
You have two entities in your DB (Competition and Date-of-competition) and only one class Competition in your domain model. Most probably, later you'll want to add additional info to the Date-of-competition (boolean finished, cancelled, Score etc) so it's a good idea to create CompetitionInstance class right now.
Since you have One-to-Many relationship you have to write some additional stuff to map objects. Normally that's what an ORM like Hibernate do istead of you. First, add a 'GROUP BY competition_id' in your sql statement.
Then use RowSetExtractor instead of RowMapper as described here:
private static final class CompetitionMapExtractor implements ResultSetExtractor<List<Competition>> {
#Override
public List<Competition> extractData(ResultSet rs) throws SQLException {
List<Competition> result = new ArrayList<>(rs.getCount());
int previousCompetitionId = NEVER_EXIST; // normally -1 is good enough
while (rs.next()) {
// we have some dates with the same competition_id
// dates are grouped thanks to GROUP BY clause
if ( rs.getInt("id") != previousCompetitionId) {
Competition currentCompetition = new Competition(rs.getInt("id"),
rs.getString("name");
/* I prefer constructor initializers "o = new O(propertyValue)"
instead of snippet "o = new O(); o.setProperty(value)"
*/
result.add(currentCompetition);
previousCompetitionId = currentCompetition.getid();
} else {
currentCompetition.addDate(new CompetitionInstance(rs.getString("date")));
}
}
return result;
}
I suppose Competition has method public void addDate(String date) which simply add a new CompetitionInstance to a list.
Update:
1.
column name in DB and in MapExtractor is different. I prefer to change the query:
SELECT c.id, c.name, d.date_time as date
from competition c
INNER JOIN date d ON c.competition_id = d.competition_id
WHERE date(d.date_time) BETWEEN ? AND ?"
2. I can't reproduce issues you have with date. Most probably you mixed up java.util.Date, java.sql.Date and java.sql.Timestamp - this is a common mistake. There are many answers already, probably you could find one of them useful.
I have a table timestamptest with a single column timestamp of type timestamp without time zone.
I inserted a value to this table :
insert into timestamptest values('2015-09-08 13:11:11')
The timestamp does not contain any millisecond value.
On selecting this data in pgAdmin, it is displayed same as above.
But when I fetch this data using jdbc connection, the value displayed is with milliseconds.
Class.forName("org.postgresql.Driver");
Connection lConnection = null;
lConnection = DriverManager.getConnection(
"jdbc:postgresql://localhost:5432/postgres","postgres", "Password#123");
String lQuery = "select * from timestamptest";
Statement lStatement = lConnection.createStatement();
ResultSet lResultSet = lStatement.executeQuery(lQuery);
while(lResultSet.next()) {
System.out.println(lResultSet.getTimestamp(1));
}
Output : 2015-09-08 13:11:11.0
The desired output is 2015-09-08 13:11:11
It can be achieved by using SimpleDateFormat :
new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(lResultSet.getTimestamp(1).getTime())
Can it be possible without using SimpleDateFormat? Is there any other way by which the result set itself gives me in the desired format?
What I need is that the statement
lResultSet.getTimestamp(1)
directly gives me the output 2015-09-08 13:11:11.
Its not possible. Since ResultSet.getTimestamp(1) return class that extends java.sql.TimeStamp. Returning class based on Database driver. And also we cant change the toString implementation of that.
Yes you can - but you're not going to like it.
class MyTimestamp extends Timestamp {
public MyTimestamp(long time) {
super(time);
}
public MyTimestamp(Timestamp ts) {
this(ts.getTime());
}
#Override
public String toString() {
String s = super.toString();
return s.substring(0, s.lastIndexOf("."));
}
}
public void test() {
System.out.println("Hello");
Timestamp t = new Timestamp(System.currentTimeMillis());
System.out.println(t);
System.out.println(new MyTimestamp(t));
}
In neo4j, how can I index by date and search in a date range. Also for times, I would like to search between 8am and 9am in a date range as well.
Index the dates and times as integer timestamps. Then you can easily search in an index for dates between other timestamps. You can also index the time part of the timestamp separately as another integer, allowing you to query for specific times between given dates.
Example:
The date and time to store is "2012-02-05 8:15 AM"
So in your index, store "timestamp=1328447700" and "time=815"
Now you want to query the index for all events between 2012-02-01 and 2012-02-10 that occurred from 8:00 am to 9:00 am. You do that by querying the index for
"timestamp>=1328072400 and timestamp<=1328936399 and time>=800 and time<=900"
The exact syntax for doing this depends on how you are connecting to Neo4j (REST or embedded) and which programming language you are using. But the idea is the same in any case.
There's a convenient org.neo4j.index.lucene.LuceneTimeline which does this (using an integrated lucene index in neo4j).
This is an extension to Josh Adell's answer. For readability, I suggest having two date and time integer fields like
date:19970716 (YYYYMMDD)
time:203045000 (HHmmssuuu): last three digits for microseconds.
The int datatype can store upto 2147483647. If you are feeling adventurous, the long datatype can store upto 9223372036854775807.
http://docs.neo4j.org/chunked/stable/graphdb-neo4j-properties.html
Inspired from ISO 8601 timestamps like 1997-07-16T19:20:30.45Z.
Disclaimer: I have only minimal experience with Neo4J.
with Spring data neo4j
public List<Email> getAllEmailData(Date startDate, Date endDate) {
List<Email> list = new ArrayList<Email>();
if (startDate == null || endDate == null) {
return null;
}
long first = ConversionsUtils.convertDateToLong(startDate);
long second = ConversionsUtils.convertDateToLong(endDate);
try {
list = emailRepository.searchAllData(first, second);
// System.out.println("List size " +list.size());
} catch (Exception e) {
e.printStackTrace();
}
return list;
}
#Query(
"START email=node:__types__(className='com.backend.core.neo.entities.Email') "
+ "WHERE email.searchDate > {0} and email.searchDate < {1}"
+ "RETURN email")
List<Email> searchAllData(long startDate, long endDate);
email entity
#NodeEntity
public class Email implements Serializable {
private static final long serialVersionUID = 1L;
public static final String CC = "CC";
public static final String TO = "TO";
#GraphId
private Long id;
#GraphProperty
private Long senderId;
#GraphProperty
private String subject;
#Indexed
// #GraphProperty(propertyType = java.util.Date.class)
private String dateSent;
#Indexed
private long searchDate;
#GraphProperty
private String emailTxt;
#GraphProperty
private String emailHtml;
#GraphProperty
private String emailId;
//mail to
#Fetch
#RelatedTo(elementClass = User.class, type = TO, direction = Direction.OUTGOING)
private Set<User> intoUsers;
//mail shared
#Fetch
#RelatedTo(elementClass = User.class, type = CC, direction = Direction.OUTGOING)
private Set<User> sharedUsers;