In unit tests I verify the result from a Db query that contains an "ORDER BY DESC" part and return String values.
To be clear: I check that the query returns the expected result in the expected order
SQL query:
SELECT member.name FROM member ORDER BY member.name DESC
Test result:
MTH_TESTER_TAXER_W_HLPERS_AFcCmhecUo
MTH_TESTER_TAXER_wCfRUAQuzT
When verifying the result with java, I do:
foreach(String value : values) {
if (previous != null) {
assertTrue(value.compareToIgnoreCase(previous) <= 0);
}
previous = value;
}
Which fails using the above DB result from the ORDER BY (descending order in above example).
So basically the following in java fails but not in DB:
assertTrue("MTH_TESTER_TAXER_wCfRUAQuzT".compareToIgnoreCase("MTH_TESTER_TAXER_W_HLPERS_AFcCmhecUo") <= 0)
Why is the ORDER BY not compatible with the Java compareTo natural String ordering?
And how to make them compatible?
I use Postgresql 9.X on Windowsd 7, default installation.
Try using string's CASE_INSENSITIVE_ORDER:
String[] str = {"abc", "pqr", "zxt", "Zxy", "xyz"};
Arrays.sort(str, String.CASE_INSENSITIVE_ORDER.reversed());
System.out.println(Arrays.toString(str));
If you change your sql to
SELECT member.name FROM member ORDER BY UPPER(member.name) DESC
your Java code will likely verify this order.
Alternatively, if you change your Java code to
foreach(String value : values) {
if (previous != null) {
assertTrue(value.compareTo(previous) <= 0);
}
previous = value;
}
it should verify the case-sensitive order being done by your current sql.
Are you trying to verify that your database query works correctly? If so, then i would first ensure I know exactly what is coming back from the database. There's no information in your code, but I would hope you're seeding the data before the test runs either through direct SQL or something like dbUnit.
Once you have this then you can do an exact comparison on the returned results in the test. I would avoid the test you currently have as it gives you no diagnostics about what is wrong when the test fails. When you have this in place your test can be something like:
List<String> expected = Arrays.asList("MTH_TESTER_TAXER_W_HLPERS_AFcCmhecUo", "MTH_TESTER_TAXER_wCfRUAQuzT");
List<String> actual = myCode.performQuery(); // <-- your method under test goes here
assertThat(expected, contains(actual));
This will ensure the collection you return has the same items in the same order as you specify and if something differs it will tell you exactly what.
There are two possible solutions here:
You don't rely on ORDER BY at all. You can use it but before you compare, you sort the actual output again in Java to make sure the order is always the same.
Use this approach if you only need to make sure that the actual sort order of the query doesn't really matter, only the number and content of the elements.
Instead of comparing each string individually, join the results into one multi-line string and then do a single assertEquals() with a constant string.
The second solution looks like this:
String actual = Joiner.on('\n').join(Ordering.natural().sortedCopy(values));
assertEquals(
"MTH_TESTER_TAXER_W_HLPERS_AFcCmhecUo\n" +
"MTH_TESTER_TAXER_wCfRUAQuzT",
actual
);
The second test checks all values at once, allowing you to see with a single glance which values have changed, which are unexpected and which are missing.
It also allows you to safely test the ordering of results as returned by the database.
Related
I need to find all the record with create date > X. X is a sql.Timestamp and might be null, in which case I want to just return all the records. So I tried: (createdAfter is Timestamp)
SELECT *
FROM sample AS s
WHERE s.isActive
AND (:createdAfter ISNULL OR s.insert_time > :createdAfter)
But all I'm getting is
org.postgresql.util.PSQLException: ERROR: could not determine data type of parameter $1
However, if I'll do the same query where I'm checking for an arbitrary int to be null:
SELECT *
FROM trades
WHERE (:sInt ISNULL OR trades.insert_time > :createdAfter )
Then it works. What's wrong?
There is no simple solution if you want to stick with native queries like that. The null value is converted to a bytea value. See for example this and this.
That value is quite hard to be casted or compared to a timestamp value.
The problem is not so much with the first comparison it would be handled by using coalesce, like:
COALESCE(:createdAfter) ISNULL
because there is no comparison beteween actual values and the data type does not matter. But the comparison
sometimestamp::timestamp > null::bytea (casts just to show the actual types so not working)
would need more logic behind maybe procedure & exception handling or so, not sure.
So if JPQL or CriteriaQueries are not possible for you have only bad options:
contruct the query by setting params by string concatenation or so (NOT! and not sure if realy works)
use PreparedStatement queries, more code & effort
also if using Hibernate, using session api like in this answer
You can try using the pg_typeof function, which returns a text string, and using a CASE statement to force which comparisons are made (otherwise there's no guarantee that postgres will short-circuit the OR in the correct order). You can then force the correct conversion by converting to text and then back to timestamp, which is inelegant but should be effective.
SELECT *
FROM sample AS s
WHERE s.isActive
AND
CASE WHEN pg_typeof( :createdAfter ) = 'bytea' THEN TRUE
WHEN s.insert_time > ( ( :createdAfter )::text)::timestamp THEN TRUE
ELSE FALSE
END
Using the InfluxDB Java client to communicate w/ my InfluxDB instance from a Java app.
I'm trying to accomplish something that doesn't appear to be documented anywhere inside the project. For a particular measurement, I need to list its latest (timestamp-wise) field key(s) and their values without knowing what they are called!
Hence let's say I have a measurement called FizzBuzz:
> SHOW measurements
name: measurements
name
----
FizzBuzz
I believe ORDER BY ASC is InfluxDb's default meaning the last/latest measurement data is always the last to come back in the query by default. So I think I'm looking to run something like this from Java:
SELECT * FROM FizzBuzz ORDER BY DESC LIMIT 1;
However my app will not know what the field key(s) on that returned result will be, and so I need a way (via the API) to inspect the record that is return and obtain the name of any field keys (so I can save them as strings) and obtain their respective values (which I can safely assume can be cast to BigDecimals).
My best attempt thus far:
Query query = new Query("SELECT * FROM FizzBuzz ORDER BY DESC LIMIT 1", "my-app-db");
QueryResult queryResult = connection.query(query);
for(Result result : queryResult.getResults()) {
for(Series series : result.getSeries()) {
List<String> cols = series.getColumns();
// But how to tell whats a field vs whats a tag?!
}
}
However this doesn't allow me to discern which columns are fields vs. which ones are just tags...any ideas?
How to unit generated strings where the end order is fairly flexible. Lets say I'm trying to test some code that prints out out generated SQL that comes from key-value pairs. However, the exact order of many of the fragments does not matter.
For example
SELECT
*
FROM
Cats
WHERE
fur = 'fluffy'
OR
colour = 'white'
is functionally identical to
SELECT
*
FROM
Cats
WHERE
colour = 'white'
OR
fur = 'fluffy'
It doesn't matter in which order the condition clauses get generated, but it does matter that they follow the where clause. Also, it is hard to predict since the ordering of pairs when looping through the entrySet() of a HashMap is not predictable. Sorting the keys would solve this, but introduces a runtime penalty for no (or negative) business value.
How do I unit test the generation of such strings without over-specifying the order?
I thought about using a regexp but* I could not think of how to write one that said:
A regex is what I was thinking of but I can think of a regex that says something like "SELECT * FROM Cats WHERE" followed by one of {"fur = 'fluffy', colour = 'white'} followed by "OR"followed by one of one of {"fur = 'fluffy',colour = 'white'} ... and not the one used last time.
NB: I'm not actually doing this with SQL, it just made for an easier way to frame the problem.
I see a few different options:
If you can live with a modest runtime penalty, LinkedHashMap keeps insertion order.
If you want to solve this completely without changing your implementation, in your example I don't see why you should have to do something more complicated than checking that every fragment appears in the code, and that they appear after the WHERE. Pseudo-code:
Map<String, String> parametersAndValues = { "fur": "fluffy", "colour", "white" };
String generatedSql = generateSql(parametersToValues);
int whereIndex = generatedSql.indexOf("WHERE");
for (String key, value : parametersAndValues) {
String fragment = String.format("%s = '%s'", key, value);
assertThat(generatedSql, containsString(fragment));
assertThat(whereIndex, is(lessThan(generatedSql.indexOf(fragment))));
}
But we can do it even simpler than that. Since you don't actually have to test this with a large set of parameters - for most implementations there are only three important quantities, "none, one, or many" - it's actually feasible to test it against all possible values:
String variation1 = "SELECT ... WHERE fur = 'fluffy' OR colour = 'white'";
String variation2 = "SELECT ... WHERE colour = 'white' OR fur = 'fluffy'";
assertThat(generatedSql, is(anyOf(variation1, variation2)));
Edit: To avoid writing all possible variations by hand (which gets rather tedious if you have more than two or three items as there are n! ways to combine n items), you could have a look at the algorithm for generating all possible permutations of a sequence and do something like this:
List<List<String>> permutations = allPermutationsOf("fur = 'fluffy'",
"colour = 'white'", "scars = 'numerous'", "disposition = 'malignant'");
List<String> allSqlVariations = new ArrayList<>(permutations.size());
for (List<String> permutation : permutations) {
allSqlVariations.add("SELECT ... WHERE " + join(permutation, " OR "));
}
assertThat(generatedSql, is(anyOf(allSqlVariations)));
Well, one option would be to somehow parse the SQL, extract the list of fields and check that everything is ok, disregarding order of the fields. However, this is going to be rather ugly: If done right, you have to implement a complete SQL parser (obviously overkill), if you do it quick-and-dirty using regex or similar, you risk that the test will break for minor changes to the generated SQL.
Instead, I'd propose to use a combination of unit and integration testing:
Have a unit test that tests the code which supplies the list of fields for building the SQL. I.e., have a method Map getRestrictions() which you can easily unit-test.
Have an integration test for the SQL generation as a whole, which runs against a real database (maybe some embedded DB like the H2 database, which you can a start just for the test).
That way, you unit-test the actual values supplied to the SQL, and you integration-test that you are really creating the right SQL.
Note: I my opinion this is an example of "integration code", which cannot be usefully unit-tested. The problem is that the code does not produce a real, testable result by itself. Rather, its purpose is to interface with a database (by sending it SQL), which produces the result. In other words, the code does the right thing not if it produces some specific SQL string, but if it drives the database to do the right thing. Therefore, this code can be meaningfully tested only with the database, i.e. in an integration test.
First, use a LinkedHashMap instead of a regular HashMap. It shouldn't introduce any noticeable performance degradation. Rather than sorting, it retains insertion ordering.
Second, insert the pairs into the map in a well understood manner. Perhaps you are getting the data from a table, and adding an ordering index is unacceptable. But perhaps the database can be ordered by primary key or something.
Combined, those two changes should give you predictable results.
Alternatively, compare actual vs. expected using something smarter than string equals. Perhaps a regex to scrape out all the pairs that have been injected into the actual SQL query?
The best I have come-up so far is to use some library during testing (suck as PowerMockito) to replace the HashMap with a SortedMap like TreeMap. That way for the tests the order will be fixed. However, this only works if the map isn't built in the same code that generates the string.
I would like to know if there is any smart way of making a SQL statement for a search engine where there are 5 optional parameters. All parameters can be used or only one of them, or a mix of any of them.. This makes up to 3000+ different combinations.
The statement needs to be prepared to avoid SQL injections.
I've looked at this post, but it dosent quite cut.
What I'm looking for is something like,
String sql =SELECT * FROM table WHERE (optional1)=? AND (optional2)=? AND (optional3)=? AND (optional4)=? AND (optional5)=?
prepared.setString(1, optional1)
and so on...
Use your java code to add the options to the where clause based on the presence of your arguments (their length or existence, whichever). That way if the optional parameter is not needed, it won't even be part of your SQL expression. Simple.
#a1ex07 has given the answer for doing this as a single query. Using NULLs and checking for them in each condition.
WHERE
table.x = CASE WHEN #x IS NULL THEN table.x ELSE #x END
or...
WHERE
(#x IS NULL OR table.x = #x)
or...
WHERE
table.x = COALESCE(#x, table.x)
etc, etc.
There is one warning, however; As convenient as it is to make one query to do all of this, All of these answers are sub-optimal. Often they're horednous.
When you write ONE query, only ONE execution plan is created. And that ONE execution plan must be suitable for ALL possible combinations of values. But that fixes which indexes are searched, what order they're searched, etc. It yields the least worst plan for a one-size-fits-all query.
Instead, you're better adding the conditions as necessary. You still parameterise them, but you don't include a condition if you know the parameter is NULL.
This is a good link explaining it further, it's for MS SQL Server specifically but it's generally applicatble to any RDBMS that caches the plans after it compiles the SQL.
http://www.sommarskog.se/dyn-search.html
I believe it should work (haven't tested though)
SELECT * FROM table
WHERE
field1 = CASE
WHEN ? IS NULL THEN field1
ELSE ?
END AND
field2 = CASE
WHEN ? IS NULL THEN field2
ELSE ?
END AND .... etc
//java code
if ([optional1 is required])
{
prepared.setString(1, optional1) ;
prepared.setString(2, optional1) ;
}
else
{
prepared.setNull(1, java.sql.Types.VARCHAR) ;
prepared.setNull(2, java.sql.Types.VARCHAR) ;
}
etc.
I don't know if this is a problem that is specific to Google App Engine for Java, but if the value set as the keywords parameter is a null String, then nothing is returned from the query, even if a minPrice is set.
How do I change this query to make it return records that meet the minPrice condition even if the keywords value is null? Ideally I would somehow use the same query for both conditions without creating separate queries based on a null String condition.
Query qry = entityManager.createQuery("SELECT p FROM Test p
WHERE keywords = :keywords and price >= :minPrice");
qry.setParameter("keywords", keywords);
qry.setParameter("minPrice", Integer.parseInt(minPrice));
It's the way the GAE datastore works (most relational databases work that way too, btw!): nulls are not equal to anything, so the keywords = :keywords part of your query is false on records with null keywords -- since that part is false, so is the and, of course.
You'll need two queries, one for keywords = :keywords and one for the "is null" check, and use their two disjoint result sets (Python GAE simulates an "IN" operator in app-level code, which I believe Java GAE doesn't, but since the sets are disjoint in this case there's really no mystery or difficulty to it anyway;-).
Edit: it's a simulated IN (which would be usable here) in Python, not OR; the Java equivalent of that app-level-simulated IN is actually contains.