We Keep Coding

Spring Data JPA ignore null parameter

Say i have the below JPA method :
public List<FrequencyCode> findAllByNameContainingAndAllowExplicitDosingTimesEqualsOrderByName(String name, Boolean allowExplicitDosingTimes);
This method is called by a user filtering a list of these objects with an input field and a select field :
The boolean value can be true, false or null in this case if the user is not making a search with that field. It looks like JPA is ACTUALLY searching for a null value when i would like it to ignore any null values. I have been able to make this combined search work with the below code :
#Override
public List<FrequencyCode> findAllWithFilters(String name, Boolean allowExplicitDosingTimes)
{
if (allowExplicitDosingTimes == null)
{
return ((FrequencyCodeRepository) baseRepository).findAllByNameContainingOrderByName(name);
}
else if (allowExplicitDosingTimes == true)
{
return ((FrequencyCodeRepository) baseRepository).findAllByNameContainingAndAllowExplicitDosingTimesTrueOrderByName(name);
}
else if (allowExplicitDosingTimes == false)
{
return ((FrequencyCodeRepository) baseRepository).findAllByNameContainingAndAllowExplicitDosingTimesFalseOrderByName(name);
}
return null;
}
This works but, obviously, on a page with 8 search options this would become a nightmare. The String parameters do not have this problem because they are actually an empty String when the user doesn't choose a filter. This paired with the Containing keyword, any value contains "" so it behaves as if that parameter is ignored which is exactly what I want for other types. Is there a way for a JPA findAll...() method to simply ignore null parameters?
******SOLUTION******
Here is how i made this work with the help of the accepted answer :
FrequencyCode fc = new FrequencyCode();
fc.setName(name);
fc.setAllowExplicitDosingTimes(allowExplicitDosingTimes);
ExampleMatcher matcher = ExampleMatcher.matching()
.withMatcher("name", match -> match.contains())
.withMatcher("allowExplicitDosingTimes", match -> match.exact())
.withIgnorePaths("id", "uuid")
.withIgnoreNullValues();
Example<FrequencyCode> example = Example.of(fc, matcher);
List<FrequencyCode> frequencyCodes = ((FrequencyCodeRepository) baseRepository).findAll(example);
You HAVE to tell it to ignore any ID fields or really any other fields you do not intend to search with but this is INCREDIBLY powerful!
Thanks!

You can use Example like this
#Override
public List<FrequencyCode> findAllWithFilters(String name, Boolean allowExplicitDosingTimes) {
FrequencyCode fc = new FrequencyCode();
//I assume that you have setters like bellow
fc.setName(name);
fc.setAllowExplicitDosingTimes(allowExplicitDosingTimes);
ExampleMatcher matcher = ExampleMatcher.matching().withIgnoreNullValues();
Example<FrequencyCode> example = Example.of(fc, matcher);
return ((FrequencyCodeRepository) baseRepository).findAll(example);
}

Querying on Java Object's without Database

Note: Little long question. I'm going to give a bounty for best answer.
What I'm trying to do is querying on Object. Here is the details. I have a file called employee.txt. So I parsed it and kept in the list
public static List<Employee> employeeList = new LinkedList<>();
Then here is my logic to query.
Take the query from user, and then parse it. The below is the logic to query through the list.
For ex: here is the query
select * from Employee where id > 10
My codes for that
String valueToCompare = split[5]; //10
EmployeeCriteria criteria = new EmployeeCriteria(
isId, isName, isSalary, expression,
valueToCompare);
result = EmployeeData.findAll(
EmployeeData.employeeList, criteria);
Here is the findAll method
public static List<Employee> findAll(List<Employee> coll,
ISearch<Employee> chk) {
List<Employee> l = new LinkedList<Employee>();
for (Employee obj : coll) {
if (chk.search(new Employee(obj)))
l.add(obj);
}
return l;
}
And here is my search method
/**
* Based on the type provided and for given expression it check against the
* given value
*/
#Override
public boolean search(Employee obj) {
if (expression.equals(EQUAL)) {
if (isId()) {
if (obj.getId() == Long.parseLong(valueToCompare)) {
return true;
}
} else if (isName()) {
if (obj.getName().equals(valueToCompare)) {
return true;
}
} else if (isSalary()) {
if (obj.getSalary() == Long.parseLong(valueToCompare)) {
return true;
}
} else {
System.err.println(UserMessage.INVALIDCOLUMN_NAME);
}
} else if (expression.equals(NOT_EQUAL)) {
if (isId()) {
if (!(obj.getId() == Long.parseLong(valueToCompare))) {
return true;
}
} else if (isName()) {
if (!(obj.getName().equals(valueToCompare))) {
return true;
}
} else if (isSalary()) {
if (!(obj.getSalary() == Long.parseLong(valueToCompare))) {
return true;
}
} else {
System.err.println(UserMessage.INVALIDCOLUMN_NAME);
}
} else if (expression.equals(GREATER)) {
if (isId()) {
if ((obj.getId() > Long.parseLong(valueToCompare))) {
return true;
}
} else if (isSalary()) {
if ((obj.getSalary() > Long.parseLong(valueToCompare))) {
return true;
}
} else {
System.err.println(UserMessage.INVALIDCOLUMN_NAME);
}
} else if (expression.equals(LESSER)) {
if (isId()) {
if ((obj.getId() < Long.parseLong(valueToCompare))) {
return true;
}
} else if (isSalary()) {
if ((obj.getSalary() < Long.parseLong(valueToCompare))) {
return true;
}
} else {
System.err.println(UserMessage.INVALID_IDENTIFIER);
}
}
return false;
}
Let me know if you want to see any other codes.
I just want to know,
In the first place LinkedList is correct data structure to use ? Is this performs well ?? Any enhancements to perform well ?
Any better way to achieve this ?
here are few example queries:
select * where ID > 100
select * where Name != Ramesh
select * where Salary < 500000
select Name order by Name
select ID
Thanks for any help. Bounty will be offered after 2 days. I can't do that now.
Note2: This is a test to check my data manage skills and I cannot use any database.

No, this does not perform well at all. You are searching through N employees every time. So if you have 1 million employees, you will search all 1 million employees before returning the correct employee. Even worst, if it doesn't exist, you will have to search exhaustibly before you can know if it exists.
Is this for use in production? If so then just use SQLite or some other simple database. You want to write once and read multiple times with indexes. I cannot stress enough that what you are writing will have bugs and instead you should use something that was already tested.
Assuming this is not for production and you are just having fun, then you want to emulate what databases do in real life. They create indexes. Indexes are usually best described as Map<String, List<Employee>>.
The idea is that initially reading the data from disk is expensive. But you read it once. For each dimension, Name, Salary, ID, etc... you want create separate indexes.
So let's say you were creating an index of all employees by ID. You would want to do something like:
Map<String, Employee> employeesById = new HashMap<>();
for(Employee e : employees) {
employeesById.put(e.getId(), e);
}
The above assumes that employee ids are unique. If they are not then you would need create a List<Employee>. For example for index by name:
Map<String,List<Employee>> employeesByName = new HashMap<>();
for(Employee e : employees) {
employeesByName.get(e.getName()).add(e); // Make sure to create the array if it doesn't exist
}
So now, for reading, let's say you get SELECT * FROM employees where id = 123; you can simply just return employeesById.get("123").
This solution is O(1). As the file gets bigger, you will not have any performance loss. This is probably the fastest solution.

To add onto Amir's answer, you can also use a TreeMap<> instead of a HashMap<>. Databases normally don't create indexes as hash maps but as balanced binary trees, which is what Java's TreeMap<> class is.
http://docs.oracle.com/javase/8/docs/api/java/util/TreeMap.html
HashMap<> has O(1) lookup on keys, but is exact match on key only. TreeMap<> has O(log n) lookup on keys, but allows ranged lookups higherKey, lowerKey) and map partitioning on key range (subMap).
The problem of duplicates can either be solved by not allowing them (unique key) or storing values in a list and linear search on the subset (non-unique key).

Assuming you are not interested in any of the in-memory database solution and using Java 8,
You should convert all your conditions to predicate and apply them on the stream . This will take advantage of Java 8's parallelism feature http://blog.takipi.com/new-parallelism-apis-in-java-8-behind-the-glitz-and-glamour/
So in short your code can be much cleaner and faster
public static Predicate<Employee> isSalaryGreaterThan(Integer salary) {
return p -> p.getSalary() > salary;
}
Predicate predicate1 = isSalaryGreaterThan(50000)
employeeList.stream().filter(predicate1).filter(predicate2)...collect(Collectors.<Employee>toList())

Hashcode deceives me sometimes

I need clarification whether my approach is right or wrong any modifications required.
Let me explain clearly. I will have a excel file in which there will be country code country name years(mm/yyyy) as extra 10 columns
countrycode country Name 12/2000 11/2000 10/2000 09/2000 08/2000 07/2000 06/2000 05/2000 04/2000 03/2000 02/2000 01/2000
IND India 10.1 10.2 10.3 10.4 10.5 10.6 10.7 10.8 10.9 11.1 11.2 11.3
USA Uinted States 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 9.1 9.2 9.3
In a row if anyof the price is repeated for that particular year and country , i need to show message as Duplicate present in Excel file.
For the above , i implemented by this way. For a VO i override the hashCode() with the hashcode of (coutrycode + year + price) and equals method too and
while inserting in database i pass this VO to HashSet and I eliminate duplicate and compare the size of original list size with HashSet size.
But sometime if there is unique price also I am getting message as duplicate.
Please suggest me my approach is right or wrong or another way I can implement.

Buddy you have taken the right thought and approach to solve the problem but just missing a little edge (information) to solve the problem correctly.
I would like to provide a little hint, that I believe can help and rectify the problem and understand the basics really very well.
If you look at the documentation (or the source code) of hashCode for the String and Double variables, it states
STRING
Returns a hash code for this string. The hash code for a String object is computed as
s[0]*31^(n-1) + s[1]*31^(n-2) + ... + s[n-1]
using int arithmetic, where s[i] is the ith character of the string, n is the length of the string, and ^ indicates exponentiation. (The hash value of the empty string is zero.)
Returns: a hash code value for this object.
DOUBLE
Returns a hash code for this Double object. The result is the exclusive OR of the two halves of the long integer bit representation, exactly as produced by the method doubleToLongBits(double), of the primitive double value represented by this Double object. That is, the hash code is the value of the expression:
(int)(v^(v>>>32))
where v is defined by:
long v = Double.doubleToLongBits(this.doubleValue());
Returns: a hash code value for this object.
So the hashCode() function returns a unique value in most case, but there are so many cases when the it returns the same int value for the two objects.
I think you are also getting caught in the same scenario.
A little more hint, you can use the HashMap<Integer,List<String>> where the Integer value is hashCode as you calculated and the List<String> is the collection of actual value got by forming the String from coutrycode + year + price .
And the last part is comparison, you can get the List<String> at the calculated hashCode() of new value and check if the same String value do exists in the List.

Hashbased collections depends on the hashcode() and equals() methods to correctly identify duplicates. If you modify these to fit exactly one usecase you are probably likely to have all sorts of side-effects in other use cases.
To say it more explicitly. If you change the methods of your VO to use only a subset of the data, you are likely to encounter unforeseen problems some where else where you might store VOs in hashbased collections.
You should keep hashcode() and equals() consistent with data equality, i.e. using all attributes for tests, as suggested in many sources (Source generators in eclipse, #EqualsAndHashcode annotations from Lombok, 'Effective Java' by Joshua Bloch, etc.).
In your explicit case you could create a specific wrapper to calculate your hashcodes and equality based on the subset.
As an example:
public void doit(List<VO> vos) {
Set<VOWrapper> dups = new HashSet<>();
for (VO vo : vos) {
if (dups.contains(new VOWrapper(vo))) {
System.out.println("Found a duplicate");
} else {
dups.add(new VOWrapper(vo));
// Process vo
}
}
}
Based on this VO
#Data // Lombok generates getters/setters/equals/hashcode (using all fields)
public class VO {
private String countrycode;
private String country;
private int month;
private int year;
private double price;
}
And this wrapper
public class VOWrapper {
private final VO vo;
public VOWrapper(VO vo) { this.vo = vo; }
// Equals method with only 3 fields used
#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
VO other = ((VOWrapper) obj).vo;
if (vo.getCountry() == null) {
if (other.getCountry() != null)
return false;
} else if (!vo.getCountry().equals(other.getCountry()))
return false;
if (vo.getCountrycode() == null) {
if (other.getCountrycode() != null)
return false;
} else if (!vo.getCountrycode().equals(other.getCountrycode()))
return false;
if (Double.doubleToLongBits(vo.getPrice()) != Double.doubleToLongBits(other.getPrice()))
return false;
return true;
}
//Hashcode method with only 3 fields used
#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((vo.getCountry() == null) ? 0 : vo.getCountry().hashCode());
result = prime * result + ((vo.getCountrycode() == null) ? 0 : vo.getCountrycode().hashCode());
long temp;
temp = Double.doubleToLongBits(vo.getPrice());
result = prime * result + (int) (temp ^ (temp >>> 32));
return result;
}
}

It is perfectly valid to write code like:
List<CountryInstance> list = ...;
Set<CountryInstance> set = new HashSet<CountryInstance>(list);
if(set.size() < list.size()){
/* There are duplicates */
For it to work you need value class instances. To create one you need to override equals and hashcode. Before you do that read What issues should be considered when overriding equals and hashCode in Java?

If you are just parsing all the values into Strings then your approach sounds logical to me.
I read your description. You seem to say that a unexpected duplicates are detected. So this really means that 'equals' method is not behaving as you expect I think. If 'hashCode' was incorrect, I think you would get the opposite problem (duplicate NOT detected).
If you are still experiencing issues then attach the implementation of 'hashCode' and 'equals' and it might help to quickly answer the problem.
One more thing. I assume that all sample countries are unique in the file? I mean no countries are duplicated later on in the file?

Testing whether a Lucene Query matches all the documents

Given an instance of Query is it possible to somehow check whether that instance happens to represent a query that always matches all the documents in the index?
For example, a MatchAllDocsQuery or a BooleanQuery that contains a MatchAllDocs clause are such queries that always return all the documents. Another example is a BooleanQuery that has a SHOULD-match clause that has a nested SHOULD-match clause that has a MatchAllDocs inside it.
Note that a query that happens to return everything because it has all the possible terms in it or because the index is empty doesn't count as a query that always return all the documents. In other words, I would like to check whether a given query always returns everything no matter what the index contains.
Is it possible or at least approximately possible? I'll accept an answer with a solution that doesn't work for any conceivable case if it works for any query that can be returned from Solr's Extended Dismax Query Parser.

A BooleanQuery that contains a MatchAllDocsQuery as one of it's clauses doesn't necessarily return all documents, as the BooleanQuery may also contain other MUST or MUST_NOT clauses which would restrict the result set. I don't believe there is anything the does this out of the box, and trying to handle any sort of query that Solr might split out would be difficult. You would need to move through the queries recursively to ensure that everything effectively reduces to a MatchAllDocsQuery, ignoring scores.
Something like (this is entirely untested at this point):
boolean willMatchAll(Query query) {
if (query instanceof MatchAllDocsQuery)
return true;
}
else if (query instanceof BooleanQuery) {
boolean foundMatchAll = false;
for (BooleanClause clause : ((BooleanQuery)query).getClauses()) {
if (clause.isProhibited()) {
return false; //A reasonable assumption, that the MUST_NOT clauses won't be empty
}
else if (clause.isRequired()) {
if (willMatchAll(clause.getQuery())) {
foundMatchAll = true;
} else {
return false; //any MUST clause that is not a matchall means the boolean query will not match all
}
}
else {
if (willMatchAll(clause.getQuery())) {
foundMatchAll = true;
}
}
}
//If a matchall has been found, and we haven't return false yet, this boolean query matches all documents
return foundMatchAll;
}
else if (query instanceof DisjunctionMaxQuery) {
boolean isMatchAll = false
//If any disjunct is a matchall, the query will match all documents
for (Query subquery : ((DisjunctuionMaxQuery)query).getDisjuncts()) {
isMatchAll = isMatchAll || willMatchAll(subquery);
}
return isMatchAll;
}
else if (query instanceof ConstantScoreQuery) {
//Traverse right through ConstantScoreQuery. The wrapper isn't of interest here.
Query subquery = ((ConstantScoreQuery)query).getQuery()
if (subquery == null) {
return false; //It wraps a filter, not a query, and I don't believe a filter can be a matchall
}
return willMatchAll(subquery);
}
else {
//No other standard queries may be or contain MatchAllDocsQueries, I don't believe.
//Even a double open-ended range query restricts the results to those with a value in the specified field.
return false;
}
}
And if you also wanted to handle the stuff in org.apache.lucene.queries, there would be more query types to handle, like BoostingQuery and CustomScoreQuery, among others. But hopefully that gives some sort of idea on it.

Good question, i am wondering if you can do : search and get numFound and compare that to see if your actual Query returns same numFound value. Am i missing something?

Lucene: Boolean OR in MultiFieldQueryParser

I have a database with 10 fields, and I need to construct a query that looks something like the following pseudo code:
theQuery = ((field1 == A) &&
(field2 == B) &&
(field3 == C) &&
(field4 == D) &&
(field5 == E) &&
(field6 == F) &&
(field7 == G) &&
((field8 == H) || (field9 == H) || (field10 == H)))
That is to say that I need fields 1-7 to definitely contain the corresponding supplied variable, and I need the variable H to definitely appear in at least one of fields 8-10.
I have been trying to use the MultiFieldQueryParser, but the problem that I have is that the BooleanClauses supplied are MUST, MUST_NOT and SHOULD, and we can set the default operator of the MultiFieldQueryParser to be either AND or OR.
When I try using AND and setting fields 1-7 with MUST and fields 8-10 with SHOULD, the query parser basically ignores fields 8-10 and gives me back anything that contains the specified data in fields 1-7.
I haven't yet tried setting the default operator to OR, because I'm guessing that the query will return results that contain one or more of the supplied variables in fields 1-10.
For those that wish to see code, my code is as follows:
ArrayList queries = new ArrayList();
ArrayList fields = new ArrayList();
ArrayList flags = new ArrayList();
if(varA != null && !varA.equals(""))
{
queries.Add(varA);
fields.Add("field1");
flags.Add(BooleanClause.Occur.Must);
}
//... The same for 2-7
if(varH != null && !varH.equals(""))
{
queries.Add(varA);
queries.Add(varA);
queries.Add(varA);
fields.Add("field8");
fields.Add("field9");
fields.Add("field10");
flags.Add(BooleanClause.Occur.Should);
flags.Add(BooleanClause.Occur.Should);
flags.Add(BooleanClause.Occur.Should);
}
Query q = MultiFieldQueryParser.parse(VERSION.LUCENE_34,
queries.toArray(),
fields.toArray(),
flags.toArray(),
theAnalyzer);
Obviously this is somewhat simplified as the ArrayLists don't neatly return me arrays of Strings and BooleanClause.Occurs, but you get the idea.
Does anyone know of a way of forming a multifield query, including both boolean ANDs and boolean ORs?
Thanks,
Rik

I don't really understand your notation, so it's hard to figure out what the problem is. But just use standard queries:
BooleanQuery topQuery = new BooleanQuery();
topQuery.add(new TermQuery(...), BooleanClause.Occur.Must);
etc.
Or just do it in text and let the parser parse it for you: +field1:A +field2:B ...