I am using jOOQ for working with a relational database. I have a SELECT query for which I need to write unit tests with mocking. Based on this doc and this post, I need to define my own data provider, which should look something like this:
class MyProvider implements MockDataProvider {
DSLContext create = DSL.using(SQLDialect.MYSQL);
#Override
public MockResult[] execute(MockExecuteContext mockExecuteContext) throws SQLException {
MockResult[] mock = new MockResult[1];
String sql = mockExecuteContext.sql();
if (sql.startsWith("select")) {
Result<Record2<String, String>> result = create.newResult(COL_1, COL_2);
result.add(create.newRecord(COL_1, COL_2)
.values("val1", "val2"));
mock[0] = new MockResult(1, result);
}
return mock;
}
}
where COL_1 and COL_2 are defined as follows:
Field<String> COL_1 = field("Column1", String.class);
Field<String> COL_2 = field("Column2", String.class);
It's quite simple and straightforward when SELECT is a small one (as in the above example, just 2 columns). I am wondering how it should be done in case of complex and large selects. For instance I have a SELECT statement which selects 30+ columns from multiple table joins. Seems the same approach of
Result<Record_X<String, ...>> result = create.newResult(COL_1, ...);
result.add(create.newRecord(COL_1, ...)
.values("val1", ...));
does not work in case of more than 22 columns.
Any help is appreciated.
Answering your question
There is no such limitation as a maximum of 22 columns. As documented here:
Higher-degree records
jOOQ chose to explicitly support degrees up to 22 to match Scala's typesafe tuple, function and product support. Unlike Scala, however, jOOQ also supports higher degrees without the additional typesafety.
You can still construct a record with more than 22 fields using DSLContext.newRecord(Field...). Now, there is no values(Object...) method on the Record type, because the Record type is the super type of all the Record1 - Record22 types. If such an overload were present, then the type safety on the sub types would be lost, because the values(Object...) method is applicable for all types of arguments. This might be fixed in the future by introducing a new RecordN subtype.
But you can load data into your record with other means, e.g. by calling Record.fromArray(Object...):
Record record = create.newRecord(COL_1, ...);
record.fromArray("val1", ...);
result.add(record);
The values() method being mere convenience (adding type safety) for fromArray().
Disclaimer:
I'm assuming you read the disclaimer on the documentation page you've linked. I'm posting it here anyway for other readers of this question, who might not have read the disclaimer:
Disclaimer: The general idea of mocking a JDBC connection with this jOOQ API is to provide quick workarounds, injection points, etc. using a very simple JDBC abstraction. It is NOT RECOMMENDED to emulate an entire database (including complex state transitions, transactions, locking, etc.) using this mock API. Once you have this requirement, please consider using an actual database instead for integration testing, rather than implementing your test database inside of a MockDataProvider.
It seems you're about to re-implement a database which can "run" any type of query, including a query with 23+ columns, and every time you change the query under test, you will also change this test here. I still recommend you do integration testing instead, using testcontainers or even with H2, which will help cover many more queries than any such unit test approach. Here's a quick example showing how to do that: https://github.com/jOOQ/jOOQ/tree/main/jOOQ-examples/jOOQ-testcontainers-example
Also, integration tests will help test query correctness. Unit tests like these will only provide dummy results, irrespective of the actual query. It is likely that such mocks can be implemented much more easily on a higher level than the SQL level, i.e. by mocking the DAO, or repository, or whatever methods, instead.
Related
I use jOOQ to query/insert/update data from/into a table.
Is there a way to see the SQL statements that JOOQ executes at Compile Time instead of Run Time Logging?
The following answer shows them at run time. How can one see the SQL statements that jOOQ executes?
This tool only converts various SQL dialects. https://www.jooq.org/translate/
Statically evaluating a jOOQ query
While it might be possible to build some IDE plugins that are capable of evaluating some static-ish jOOQ statements, remember that in principle and by design, every jOOQ query is a dynamic SQL query. When you write something as simple as:
Result<?> r = ctx.select(T.A, T.B).from(T).fetch();
What the JVM sees (roughly) is:
Field<?> a = T.A;
Field<?> b = T.B;
Field<?>[] select = { a, b };
SelectFromStep<?> s1 = ctx.select(select);
Table<?> t = T;
SelectWhereStep<?> s2 = s1.from(t);
Result<?> r = s2.fetch();
Of course, no one is using jOOQ this way. The DSL was designed to produce call chains that look almost like SQL through its fluent API design. So, your query looks like it's static SQL (which could be evaluated in an IDE), but it is not. And you will often use the dynamic SQL capabilities, e.g.
Result<?> r = ctx
.select(T.A, T.B)
.from(T)
// Dynamic where clause
.where(someCondition ? T.A.eq(1) : T.B.gt(2))
.fetch();
There's no way an IDE could evaluate all this, including all of your SPI implementations, such as the ExecuteListener or the VisitListener, so again, even if it worked for some cases, it would work poorly for many others.
You'll have to execute your query to see the actual SQL (for that specific execution). Or, you put a breakpoint on your fetch() call, and evaluate the query object upon which fetch() is called in the debugger.
The underlying, actual problem
Whenever I see this question, I think there's an underlying actual problem that manifests in this desire of running the jOOQ query outside of your Java code. The problem is that your code seems to be hard to integration test.
This can't be fixed easily, but it is a good reminder that when you start from scratch, you make all of your SQL (jOOQ or not) easily integration testable using:
Something like testcontainers
By separating concerns and moving your SQL logic in an appropriate layer that can be easily integration tested independently of any other logic (UI, etc.)
With such an approach, you will be able to test your jOOQ queries in a much better feedback cycle, in case of which you probably won't even think of running the jOOQ query outside of your Java code again, at least most of the time.
How can i use jpa for query over an object (not an entity)?
For example this simple code:
String [] theList = {a,b,c,d}.
Query q = new Query("Select tl from theList tl")
Reason behind: the queries are dynamically created and executed, but the objects in the from clause of the jpql query aren't necessarily mapped tables. In some cases there are just an Object, So the actual behavior needed is modify the query during execution of the program to meet the criteria, but i don't know how to modify the query.
Edit: I Don't use native queries because of portability of code. It will be the last option.
What you're looking for is called LINQ, and unfortunately (?) it is available only in C#.
However, you can partially emulate it with Stream(s).
A Stream offers basically all the operators you need
.filter() where
.max() max
.sorted() orderby
.limit() limit
.skip() offset
.collect(groupingBy()) group by
And so on. Just give a look at the Javadoc!
I think 'JdbcTemplate' would suffice your requirement.
JdbcTemplate gives you the flexibility to run native queries and map them to a Java class.
However, you'll have to explicitly map your Java class with the column names in the database.
I have solved using joSQL. Is a powerfull opensource tool that allows you to query over java objects using "sql". It is not jpa but satisfied my needs.
Another tool i have seen that do that is called querydsl.
I am looking for a way to compare two MySQL queries in a unit test. Do you know any library that allows that (all of these asserts should pass):
SQLAssert.assertEquals("select id, name from users", "select id, name from users")
SQLAssert.assertEquals("select id, name from users", "select `id`,`name` from `users`")
While running the queries against an in-memory database and comparing results is the best answer, I think there are less-comprehensive and more brittle options that are nevertheless useful.
In practice it's likely that you can place additional constraints on the syntax of the queries. In your example, there are only select statements, a single table, no where clause, and the only query differences are backticks and spaces, so writing a method that normalizes queries with those constraints would probably be doable. Something like:
private String normalize(String str) {
return str.replaceAll(" +", " ").replaceAll("`", "");
}
These normalized strings can then be compared. This way of doing things is very brittle (and therefore not future proof), but that doesn't mean it can't provide value in certain circumstances. Sure, there are quite a few valid sql statements that would cause this to break, but you don't have to deal with the full set of strings that valid sql entails. You just have to deal with whatever subset of sql your queries use.
If your queries are different enough to make this code unreasonable, it might be easier to use a parser library like JSqlParser to parse out the pieces and then navigate the structure to do your comparison. Again, you don't have to support all of SQL, just whatever subset your queries use. Also, the tests don't have to test full logical equivalence to be useful. A test might just make sure that all the tables mentioned in two queries are the same regardless of joins and ordering. That doesn't make them equivalent, but it does guard against a particular kind of error and is more useful than nothing.
An example of a situation where this could be useful is if you are doing large groups of refactorings on your query-builders and you want to make sure the end queries are equivalent. In this case you aren't testing the queries themselves but the query-building.
I wouldn't suggest doing this as a regular practice in unit tests, but I think it can be useful in very particular circumstances.
You could use JSqlParser to parse your queries. Then you could use the so called Deparser of JSqlParser to get version of your SQL without additional spaces, tabs, linefeeds. From this on, you could use a simple String equality check. Sure you have to process all kinds of quotations like " or [] but it works, like the example code shows.
This does not work, of quotation comes into play or different orders of columns or expression within you SQL. The quotation problem is simple to solve through an extention to the expression deparser.
Statement stmt1 = CCJSqlParserUtil.parse("select id, name from users");
Statement stmt2 = CCJSqlParserUtil.parse("select id, name from users");
Statement stmt3 = CCJSqlParserUtil.parse("select `id`,`name` from `users`");
//Equality
System.out.println(stmt1.toString().equals(stmt2.toString()));
ExpressionDeParser exprDep = new ExpressionDeParser() {
#Override
public void visit(Column tableColumn) {
tableColumn.setColumnName(tableColumn.getColumnName().replace("`", ""));
super.visit(tableColumn);
}
};
SelectDeParser stmtDep = new SelectDeParser() {
#Override
public void visit(Table tableName) {
tableName.setName(tableName.getName().replace("`", ""));
super.visit(tableName);
}
};
exprDep.setBuffer(stmtDep.getBuffer());
stmtDep.setExpressionVisitor(exprDep);
((Select)stmt3).getSelectBody().accept(stmtDep);
String stmt3Txt = stmtDep.getBuffer().toString();
System.out.println(stmt1.toString().equals(stmt3Txt));
I would suggest that the only way to assert that 2 queries return the same result is to actually run them. Of course, what you don't want to do is have unit tests connect to a real database. There are a number of reasons for this:
The tests can affect what is in the database, and you don't want to introduce a load of test data into a production database
Each test should be self contained, and work the same way each time it is run, which requires the database be in the same known state at the start of each run. This requires a reset for each test - not something to do with a production (or dev environment) database.
With these constraints in mind, I suggest you look into DBUnit, which is designed for database-driven JUnit tests. I also suggest, instead of using MySQL for unit tests, use an in-memory database (the examples use HSQLDB), that way, you can test queries without having test data actually persisted.
I'm currently evaluating JOOQ because I believe I started reinventing the wheel which looks very close to part of JOOQ :)
Now, while digging in great JOOQ documentation I've found that my use case lies somewhere between Using JOOQ as SQL Builder and Using JOOQ as SQL Builder with Code generation i.e. I would like to:
Create plain SQL strings like it is shown in Using JOOQ as SQL Builder part
Instead of using hard-coded DSL.fieldByName("BOOK","TITLE") constructs, I prefer storing name of a table along with it's column names and types like it's shown in Using JOOQ as SQL Builder with Code generation part
I prefer not to use code generation (at least not on regular basis), but rather creating TableImpl myself when new table is needed.
While digging in manual, I've found how table implementation should look like in chapter Generated tables. However, TableImpl class as well as Table interface should be parameterized with record type and the same goes for TableField class. I believe this is done for easier type inference when directly querying database and retrieving results, though I may be mistaken.
So my questions are:
Is there a guide in manual on how to create Table and TableField implementations? Or I can simply generate them once for my database schema and use generated code as a guideline?
How can I gracefully "discard" record type parameters in implemented classes? First, I thought about using java.lang.Void class as type parameter but then I noticed that only subclasses of Record are allowed... The reason is that I don't need record types at all because I plan to use generated by JOOQ SQL queries in something like Spring JdbcTemplate so mapping is done by myself.
Thanks in advance for any help!
Given your use-case, I'm not sure why you'd like to roll your own Table and TableField implementations rather than using the ones generated by jOOQ. As you stated yourself, you don't have to regenerate that code every time the DB schema changes. Many users will just generate the schema once in a while and then put the generated artefacts under version control. This will help you keep track of newly added changes.
To answer your questions:
Yes, there are some examples around the use of CustomTable. You may also find some people sharing similar experiences on the user group
Yes you can just use Record. Your minimal custom table type would then be:
class X extends TableImpl<Record> {
public X() {
super("x");
}
}
Note that you will be using jOOQ's internal API (TableImpl), which is not officially supported. While I'm positive that it'll work, it might break in the future, e.g. as the super constructor signature might change.
I can't believe I'm asking this, but...
Is there any way, in Java, to execute a SQL statement (not JPQL) and map the results to a List of Plain Old Java Objects?
I want to be able to create small lightweight POJO objects and then have them populated by raw SQL queries. I'm expressly NOT looking to create complex objects: just primitives, with no relationships.
Everything seems to be centered around JPA/JPQL, but the problem with that is that I do not want to bind my objects to a specific table.
I feel like I'm either:
(a) on crazy pills, or
(b) missing something fundamental
A lightweight mapper is not available as part of the JDK itself. You could either roll-your-own simple mapper using Java's standard JDBC API (in fact JPA implementations build on top of that) or you could have a look at external libraries that provide simple SQL-to-Object mappers. I know MyBatis (formerly known as iBatis).
A) No, I think you're not on crazy pills and B) is it possible that you just missed JDBC?
Sormula may be able to do what you want. You would need to extend Table and override getTableName() and/or getQualifiedTableName() to supply the desired table name since sormula normally associates one POJO to one table. See example 2a and example 3a.
jOOQ has a couple of Record -> POJO mapping capabilities that will probably do the job for you (although jOOQ can do much more). Here's an example:
// A "mutable" POJO class
public class MyBook1 {
public int id;
public String title;
}
// The various "into()" methods allow for fetching records into your POJOs:
List<MyBook1> myBooks = create.select().from(BOOK).fetchInto(MyBook1.class);
Taken from the manual here:
http://www.jooq.org/doc/latest/manual/sql-execution/fetching/pojos/
The mapping algorithm is described in the Javadoc:
http://www.jooq.org/javadoc/latest/org/jooq/impl/DefaultRecordMapper.html
While the above example makes use of jOOQ's DSL API, you can do with plain SQL as well:
List<MyBook1> myBooks = create.resultQuery("SELECT * FROM BOOK")
.fetchInto(MyBook1.class);
You can even operate on a JDBC ResultSet, using jOOQ only for mapping:
ResultSet rs = stmt.executeQuery();
List<MyBook1> myBooks = create.fetch(rs).into(MyBook1.class);