It is quite usual to find myself writing unit tests against database calls and I always hit the same issue: How to validate if a good query is being sent to the database?
Example, I have this class that will send a final update to the database in the following form:
update credential set password_hash = ?, password_crypt = ?, password_plain = ? where id = ?
(this is a password migration tool, please dont mind the security issues with the password_plain field)
Writing the test class for this class, I have mocked the database access class (in this case I am using Spring JDBCTemplate) and captured the issued sql. After I have the sql, I do the following checks:
String space = "\\s+";
String optSpace = "\\s*";
String something = ".+";
String optSomething = ".*";
sql = sql.toLowerCase();
assertTrue(sql.matches(optSpace + "update" + space + "credential" + space + "set" + space + something));
assertTrue(sql.matches(something + space + "set" + space + optSomething + "password_hash" + optSpace + "=" + optSpace + "\\?" + something + "where" + something));
assertTrue(sql.matches(something + space + "set" + space + optSomething + "password_crypt" + optSpace + "=" + optSpace + "\\?" + something + "where" + something));
assertTrue(sql.matches(something + space + "set" + space + optSomething + "password_plain" + optSpace + "=" + optSpace + "\\?" + something + "where" + something));
assertTrue(sql.matches(something + space + "where" + space + optSomething + "id" + optSpace + "=" + optSpace + "\\?" + optSomething));
With those checks I am indeed validating if the issued SQL contains the most important parts of the update like:
correct table is being updated
all the 3 fields are being updated to values passed as parameters
the id is being used in the where statement, with its value as a parameter
I could simply validate if the issued query is exactly the expected query above, but that would make the test too restrict for future changes and would force a failure if any part of the query was changed, even if the update stays correct. As I think that tests are written to be used mostly in the future (when you are changing software and need more reassurance for that) and not in the present, this option would make the test kinda useless.
Well, finally, I declare my question: Which better options do we have for validating the issued SQL?
I see a lot of projects that create small embedded databases with a small amount of data for testing classes that deal with the database, but I wanted to write a more pure unit test alternative (if I can call that)
I don't think there's a good alternative to testing against a real database (even if it's embedded etc). At the moment you're testing that your SQL is syntactically valid, but will it actually work. e.g. do you know if it would violate constraints etc...
Mocking etc is all well and good but at some stage you have to test against the database. I would ensure that where possible you don't test against the database, and then bite the bullet and construct tests around a small database (with suitable rollback/rebuilds etc) to actually confirm correct db functionality.
I'd implore you to reconsider the benefits of these tests because
it is verifying implementation not behavior
when you change the SQL query in the future (or someone just adds a innocuous space by mistake), you'd have a failing test even if you have preserved behavior.
For the final DataAccessLayer, I'd recommend writing an integration test. One that runs against a real but minimal DB. Sure these tests would be slow but the confidence that they offer is worth it.
So write tests against GetCustomers() and verify the returned DTO contains the right data vs verifying that the SQL query you issued is X.
don't assert your sql. it's pointless. you will end up comparing passed sql string to another string (also created by you so there is no validation) or you'll have to implement your own database. instead just use existing one. check is query returned correct data or correctly changed data in database. use dbunit or sth similar.
Related
Is there any fluent "create trigger" builder? It's weird but I can't find any trigger builder example (also searched in the jOOQ manual but without success).
I would like to transform hard-coded statement:
"CREATE TRIGGER " + TRIGGER_DELETE_TAB + " " +
"BEFORE DELETE ON " + TABLE_TABS + " " +
"BEGIN " +
"DELETE FROM " + TABLE_CHORDNAMES + " " +
"WHERE " + CHORDNAME_TAB + " = " + "OLD."+TAB_ID +"; " +
"END;"
in something like that:
SQLiteDSL.createTrigger(TRIGGER_DELETE_TAB)
.beforeDeleteOn(TABLE_TABS)
.begin()
.deleteFrom(TABLE_CHORDNAMES)
.where(column(CHORDNAME_TAB).eq("OLD."+TAB_ID))
.end()
.getSQL();
jOOQ could contain an API to implement a really trivial trigger like yours seems to be, and chances are, it will in some future version (#6956).
But in order to fully support triggers, jOOQ needs a runtime model abstracting over all sorts of procedural languages first, before going into the details of vendor specific trigger features. There's a feature request "Add procedural language abstraction API", in fact: #6475
This is being worked on for the upcoming version jOOQ 3.11, which will definitely support BEGIN .. END style blocks: #6474.
For jOOQ 3.10 and less, you will need to build your own jOOQ extension API based on the plain SQL templating mechanism documented here:
https://www.jooq.org/doc/latest/manual/sql-building/plain-sql-templating
You don't need too much plumbing to get that functionality working...
I don't think you can do that, but you can use instead ExecuteListeners, which can be considered as triggers and you can do something similar that what you build with your SQL.
You can check out their documentation regarding ExecuteListeners, they also provide a few example, like query statistics listener, logging listener and so forth.
They recommend extending DefaultExecuteListener and start from their with Javadoc, they have quite a variety of methods that you can override. I am pretty sure you will find what you need.
I am getting quite angry with this, so I seek help from the crowd ;)
What I want to do: We have a Unity learning game which shall implement a login window. The entered credentials are then hashed (the pw is) and sent to the server, who then should check this against a database.
I have the following table:
xy.users_confirms with the following colums:
id username email password hashcode created
Why does my code
String sql = "SELECT " + "xy.users_confirms.password as pwhash, "
+"FROM xy.users_confirms " +"WHERE xy.users_confirms.username = " +"\"userNameToGetHashFor\"";
lead me to the SQLException "Parameter index out of range (1 > number of parameters, which is 0)"
?
Thanks, any input is much appreciated!
Try this:
String parameter = "'"+ strNameToGetHashFor + "'";
String sql = "SELECT " + "xy.users_confirms.password as pwhash, "
+"FROM xy.users_confirms "
+"WHERE xy.users_confirms.username ="+ parameter;
You are using varchar value as a parameter so it's need to be quot like this.'username'. or you can use Stored Procedure.
Personally, I would try getting a working query using the custom query box directly in phpmyadmin. Once you have a working query you can re-write it in java.
And I would try writing the syntax like this into the phpmyadmin query box:
SELECT password as pwhash
FROM xy.users_confirms
WHERE username ='userNameToGetHashFor'
Using the above syntax I don't see anyway your error could persist.
Phpmyadmin screen cap showing custom query box: http://screencast.com/t/9h8anH0Aj
(the 2 empty text boxes in screen cap are just me hiding my database info)
The comma after pwhash is one potential cause:
+ "xy.users_confirms.password as pwhash*!*,*!* "
Depending on the DBMS, you may also need to use single quotes instead of double quotes like this:
+ "'userNameToGetHashFor'";
Also this code is potentially vulnerable to a SQL Injection attack so you may want to make the userNameToGetHashFor a parameter rather than concatenating the string into the SQL statement.
I currently have a small database updater that'll insert/update a users IP address every time they enter the server. Since this is ran on an interval which empties a cache every 2 seconds, i really want to find a good quick method to update the table.
Here is what i currently use (MySQL only sadly :( )
INSERT INTO `iban_ipcache` VALUES (
'" + PlayerName + "',
'" + UpdatedTime + "', //This is just System.getCurrentMills
'" + PlayerIP + "')
ON DUPLICATE KEY UPDATE VALUES (
ip='" + PlayerIP + "',
time='" + System.currentTimeMills() + "';"
-- This was modifed so it can be read more easily, the query also uses ` instead of '--
The query works perfectly but.. Well i have to change the IP AND the time no matter what. So if the IP doesn't actually change, it'll still attempt to alter it. The time is used for both GUI reasons, and cross server updating reasons. So basically every time an old user logs in, the database will treat them like they just got a new IP.
My question is, is there any way to possibly check if BOTH the key and IP are the same without having to use Select?
I'm trying to insert values into a SQL database from within Java. This works fine, except for some of the values. Eg, when I insert "foo" it appends null at the start so it becomes "nullfoo". If I insert the same statement in SQL Server Management Studio this doesn't happen.
To be sure: I print the string before inserting it and it reads "foo".
My insert code:
statement.execute("INSERT INTO " + settings.getProperty("table") + " VALUES ('" + value1+ "', '" + value2 + "', '" + value3 + "')");
You're concatenating values into the SQL statement. If any of those references (value1, value2 etc) are null, then those will be converted into the string "null" as part of concatenation.
The correct fix for this is not to change the way you're doing validation - it's to stop putting the values into the SQL statement itself. Use PreparedStatement with parameterized SQL and set parameter values instead.
Benefits:
You won't get "null" inserted any more
You won't be vulnerable to SQL injection attacks any more (you are now)
When inserting non-text data you won't need to worry about problematic conversions (this is particularly relevant for date/time fields)
Your code will be clearer, as you'll be separating the code (SQL) from the data (parameter values)
Your prepared statement query plan can be cached by the server, so it may perform faster
You should use variable binding in your SQL
http://decipherinfosys.wordpress.com/2007/08/29/bind-variables-usage-parameterized-queries-in-sql-server/
It's easier to check for errors.
In your case you are probably adding null+"foo" so you get nullfoo.
I am trying to execute the following JPA query:
public static final String UPDATE_INVENTORY_CUSTOMER_FOR_AMS_MAPPING = "UPDATE Inventory inventory SET"
+ " inventory.customer.id = :" + DataAccessConstants.PARAM_CUSTOMER_ID
+ " ,inventory.lastUpdateUserId = :" + DataAccessConstants.PARAM_USER_ID
+ " where inventory.amsConsignorName = :" + DataAccessConstants.PARAM_AMS_CONSIGNOR_NAME
+ " and inventory.amsConsignorOrgCd = :" + DataAccessConstants.PARAM_AMS_CONSIGNOR_ORG_CD
+ " and inventory.amsConsignorTypeName = :" + DataAccessConstants.PARAM_AMS_CONSIGNOR_TYPE
+ " and inventory.status.code in (:" + DataAccessConstants.PARAM_STATUS + ")";
but it is seeing the following:
update ATL_INVENTORY, set CONSIGNOR_ID=?, LAST_UPDATE_USER_ID=? where AMS_CONSIGNOR_NAME=? and AMS_CONSIGNOR_ORG_CD=? and AMS_CONSIGNOR_TYPE_NAME=? and (CODE in (? , ? , ? , ?))
Any ideal as to why there is a comma after the table name?
Solution
I had to change the original query to the following:
update Inventory inv set "
+ "inv.customer.id = :" + DataAccessConstants.PARAM_CUSTOMER_ID + " "
+ "where inv.amsConsignorName =:" + DataAccessConstants.PARAM_AMS_CONSIGNOR_NAME + " "
+ "and inv.amsConsignorOrgCd =:" + DataAccessConstants.PARAM_AMS_CONSIGNOR_ORG_CD + " "
+ "and inv.amsConsignorTypeName =:" + DataAccessConstants.PARAM_AMS_CONSIGNOR_TYPE + " "
+ "and exists(select 1 from Code code where inv.status = code and code.code in (:" + DataAccessConstants.PARAM_STATUS + "))
Which then produced this:
update ATL_INVENTORY set CONSIGNOR_ID=? where AMS_CONSIGNOR_NAME=? and AMS_CONSIGNOR_ORG_CD=? and AMS_CONSIGNOR_TYPE_NAME=? and (exists (select 1 from ATL_CODE code1_ where ATL_INVENTORY.STATUS=CODE_ID and (code1_.CODE in (? , ? , ? , ?))))
Based on a clarification located here: Incorrect SQL generated for JPA QL Update statement involving multiple entities
Your query is code as UPDATE Inventory inventory SET, but the generated SQL says update ATL_INVENTORY, set. Why is the literal SQL string not what you coded? When I encounter mysteries like this, they're usually caused by assuming that one thing is being done when in fact another is in play.
This suggests that the SQL you coded isn't being used to generate that SQL the way you're assuming. See where else this query might be coming from. I'd bet that the real source has a misplaced comma in it.
Which JPA implementation are you using? If I'm incorrect about a bad assumption, it says that there's a bug in the implementation. Have you used it before? Have you had success with UPDATE? If yes, it's definitely buried somewhere in your code base.
You have an interface with a bunch of constants in it. Personally, I don't care for a design like that. It's an anti-pattern with a name.
A bug in the JPA provider is very unlikely in my opinion so, as #duffymo said, are you sure you're using the right constant, that the code or maybe dependencies are up-to-date? I'd dig in that direction.
That being said, I really wonder why you're not using named queries (that are most of time pre-compiled by the persistence implementation at deployment time), either in the Java code or in meta-data mapping files (the fun part is that people didn't find having EJB-QL queries externalized in XML very manageable in EJB 2.x, hence the #NamedQuery annotation of JPA).