I am busy developing a SQL query builder util which will allow me to easily and quickly write SQL queries. I have the fundamentals sorted for the actual query builder but I am stuck when it comes to using some form on String literals/constants for the database entities (i.e. the tables and columns).
What I would like is to do something like this:
String sqlQuery = queryBuilder.select(Tables.Users.FirstName)
.where(Tables.Users.Age >= 10)
.build()
.toString();
I've read up quite a bit on enums and how to nest them but it seems like I will have to create an enum for every table (each containing specific columns). If possible I would like to have one class, which contains all the tables and for each table all the columns.
This should allow me to "build" a string literal by simply calling something like:
Tables.Address (returns the string value for 'address' table)
Tables.Address.Country (returns the string value for 'country' column)
This is what I've tried so far using enums but not quite what I wanted.
public enum Table {
Users("users"),
Addresses("addresses"),
private String tableName;
Table(String tableName) {
this.tableName = tableName;
}
public String getName() {
return tableName;
}
}
public enum Column {
ID(Table.Users, "id"),
NAME(Table.Users, "name");
private Table table;
private String columnName;
Column(Table table, String columnName) {
this.table = table;
this.columnName = columnName;
}
public String getColumnName() {
return columnName;
}
}
First, I do not really understand why are you trying to reinvent the wheel that has been already invented many many times.
So, if you are developing library in order to use it take a look on JOOQ. If however you really want to develop your own query builder you may probably find as useful my project BeanCrumbs.
Enums aren't the way to go; they aren't flexible enough. You need to access the metadata for each database schema you attach to.
I never see why solutions like yours are considered better than just writing SQL and executing it with JDBC. It's just as much work to use your builder as it is to write SQL.
FYI - You realize that you aren't the first or the best attempt to solve this problem. Spring JDBC template takes the boilerplate and work out of interacting with relational databases. iBatis has been around for a long time and is well proven. There are lots of ORM solutions, like Hibernate, but those are a bridge too far. I wouldn't recommend those. JPA is built into Java EE spec.
I would not want to work on a team that reinvented this wheel and forced me to prefer it over commodity, proven solutions.
I have been using ActiveJDBC quite successfully.
See first point in Design principles, this is as far as it goes in applying "convention over configuration" principle
Related
Can jOOQ automatically add an alias prefix for all columns of a table in the select clause?
Can jOOQ also help with the 128 byte/character name length limitation of databases?
The reason for the questions are that SimpleFlatMapper is used for the mapping.
SimpleFlatMapper requires the fetched database column names to map to the model.
Model example (really getter/setter are used):
class Head {
public Integer id;
public List<Position> positions;
...
}
class Position {
public Integer id;
public Integer headId;
...
}
The naming can be done individually:
ResultSet resultSet = dsl.select(..., POSITION.ID.as("positions_id"), ...)
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
List<Head> headers = JdbcMapperFactory.newInstance()
.ignorePropertyNotFound()
.newMapper(Head.class)
.stream(resultSet)
.collect(Collectors.toList())
However if it is a complicated model with several joins/columns it is a bit tedious.
The only solution I found was to program a function manually. Have I however maybe overlooked something and jOOQ can do it by itself or help?
Something the likes of this would be very nice (probably not the best naming, I couldn't come up with something better on the spot):
ResultSet resultSet = dsl.select()
.from(HEAD)
.join(POSITION.as("positions").prefixColumns()).onKey()
.fetchResultSet();
Or:
ResultSet resultSet = dsl.selectWithTableAlias()
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
Resulting in the following SQL:
SELECT head.id, head.***, positions.id AS positions_id, positions.headid AS positions_headid, positions.***
FROM head JOIN position AS positions ON head.id = positions.headid
Furthermore databases like Oracle and MSSQL have a limitation of 128 bytes/characters for names.
In very rare complex scenarios the alias names might reach that limit because of the needed nesting.
jOOQ cannot offer a workaround for this in some form or can it? So basically define a name that is used in the SQL and a name for the actual resulting object.
I know, very niche requirements. But they would help a lot with the mapping.
Can jOOQ automatically add an alias prefix for all columns of a table in the select clause?
There's no such feature, because the possible sets of desired auto-prefix algorithms is quite big, and jOOQ ultimately wouldn't do exactly what you want. Maybe, there's room for an SPI to help you do this, in the future. But there isn't one available yet: https://github.com/jOOQ/jOOQ/issues/11545
However, you can easily do this yourself, because every jOOQ query is just a dynamically constructed expression tree, even if you're not always using jOOQ for dynamic SQL.
You can easily write a utility and use that everywhere:
public static List<Field<?>> autoPrefix(Field<?>... fields) {
return autoprefix(Arrays.asList(fields));
}
public static List<Field<?>> autoPrefix(Field<?>... fields) {
Stream.of(fields).map(f -> f.as(myPrefixLogic(f))).collect(toList());
}
This can now be used explicitly on all DSLContext.select() calls, e.g.
dsl.select(autoPrefix(..., POSITION.ID.as("positions_id"), ...))
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
Alternatively, you can wrap your query in a derived table:
public ResultSet fetchResultSetWithPrefix(Select<?> select) {
Table<?> table = select.asTable("t");
return
dsl.select(autoPrefix(table.fields()))
.from(table)
.fetchResultSet();
}
And now, instead of calling fetchResultSet(), you call your auxiliary function:
try (ResultSet rs = fetchResultSetWithPrefix(
dsl.select(..., POSITION.ID.as("positions_id"), ...)
.from(HEAD)
.join(POSITION.as("positions")).onKey()
)) {
...
}
Another, more complex option would be to do this with a VisitListener.
Remember, irrespective of what your query looks like (because jOOQ's DSL mimicks SQL, syntactically), all your jOOQ queries are dynamic SQL queries, so you can relatively easily transform them to whatever you want, automatically.
Can jOOQ also help with the 128 byte/character name length limitation of databases?
I'll have the same reservations about this automatism as before. Where would you see jOOQ provide automatic help here? Given your naming scheme (table_column) would you like a fair distribution of 63 characters per object type? Or is the table less important than the column, and you'll truncate table names at 42 characters, leaving 85 characters for the column?
What if you wanted to fully qualify identifiers, including the schema name, as in schema_table_column? What if you project UDTs, meaning you'll get schema_table_column_attribute1_attribute2
I don't think any automation can be provided by jOOQ which would suit all needs. However, as shown above, it is very simple to implement this only once for your entire application, making sure the naming is always applied correctly. And if an SPI were available, you could implement your abbreviation logic there.
I am using jOOQ for working with a relational database. I have a SELECT query for which I need to write unit tests with mocking. Based on this doc and this post, I need to define my own data provider, which should look something like this:
class MyProvider implements MockDataProvider {
DSLContext create = DSL.using(SQLDialect.MYSQL);
#Override
public MockResult[] execute(MockExecuteContext mockExecuteContext) throws SQLException {
MockResult[] mock = new MockResult[1];
String sql = mockExecuteContext.sql();
if (sql.startsWith("select")) {
Result<Record2<String, String>> result = create.newResult(COL_1, COL_2);
result.add(create.newRecord(COL_1, COL_2)
.values("val1", "val2"));
mock[0] = new MockResult(1, result);
}
return mock;
}
}
where COL_1 and COL_2 are defined as follows:
Field<String> COL_1 = field("Column1", String.class);
Field<String> COL_2 = field("Column2", String.class);
It's quite simple and straightforward when SELECT is a small one (as in the above example, just 2 columns). I am wondering how it should be done in case of complex and large selects. For instance I have a SELECT statement which selects 30+ columns from multiple table joins. Seems the same approach of
Result<Record_X<String, ...>> result = create.newResult(COL_1, ...);
result.add(create.newRecord(COL_1, ...)
.values("val1", ...));
does not work in case of more than 22 columns.
Any help is appreciated.
Answering your question
There is no such limitation as a maximum of 22 columns. As documented here:
Higher-degree records
jOOQ chose to explicitly support degrees up to 22 to match Scala's typesafe tuple, function and product support. Unlike Scala, however, jOOQ also supports higher degrees without the additional typesafety.
You can still construct a record with more than 22 fields using DSLContext.newRecord(Field...). Now, there is no values(Object...) method on the Record type, because the Record type is the super type of all the Record1 - Record22 types. If such an overload were present, then the type safety on the sub types would be lost, because the values(Object...) method is applicable for all types of arguments. This might be fixed in the future by introducing a new RecordN subtype.
But you can load data into your record with other means, e.g. by calling Record.fromArray(Object...):
Record record = create.newRecord(COL_1, ...);
record.fromArray("val1", ...);
result.add(record);
The values() method being mere convenience (adding type safety) for fromArray().
Disclaimer:
I'm assuming you read the disclaimer on the documentation page you've linked. I'm posting it here anyway for other readers of this question, who might not have read the disclaimer:
Disclaimer: The general idea of mocking a JDBC connection with this jOOQ API is to provide quick workarounds, injection points, etc. using a very simple JDBC abstraction. It is NOT RECOMMENDED to emulate an entire database (including complex state transitions, transactions, locking, etc.) using this mock API. Once you have this requirement, please consider using an actual database instead for integration testing, rather than implementing your test database inside of a MockDataProvider.
It seems you're about to re-implement a database which can "run" any type of query, including a query with 23+ columns, and every time you change the query under test, you will also change this test here. I still recommend you do integration testing instead, using testcontainers or even with H2, which will help cover many more queries than any such unit test approach. Here's a quick example showing how to do that: https://github.com/jOOQ/jOOQ/tree/main/jOOQ-examples/jOOQ-testcontainers-example
Also, integration tests will help test query correctness. Unit tests like these will only provide dummy results, irrespective of the actual query. It is likely that such mocks can be implemented much more easily on a higher level than the SQL level, i.e. by mocking the DAO, or repository, or whatever methods, instead.
I have this domain model:
final class Project {
private final #Id
#With
long projectId;
private final String projectType;
#With
private final ProjectRecipient projectRecipient;
#With
private final Set<PreCalculationCosts> preCalculationCosts;
#With
private final Set<PostCalculationCosts> postCalculationCosts;
}
The problem:
When i call the findById method from the CrudRepository the property projectRecipient gets materialized.
I even see all the issued sql statements being necessary for that in the logs.
When i use my own Query only the One to Many properties are getting materialized (there is no select statement issued for the one to one related projectrecipient):
select p.* from project p
inner join projectrecipient pr on pr.project = p.projectid
where p.projectid = :projectId
EDIT
When i debug the findById method and use this generated SQL as Query value, it gets materialized the right way. Problem with that is, that my Project table has tons of columns, so the Query value string is kind of 5 lines in my IDE(A) ...
On the other hand, i cant use the findById method because i need some postgres specific similar to clause ...
Currently there is no alternative to spelling out the full SQL statement including the columns for the referenced entity prefixed with the relationship name and an _.
If you don't include these columns the referenced entity will be considered to be null
The Spring Data team is thinking about enabling a way to provide everything but the select clause of a query, but it will take quite some time until these thoughts turn into code since the need some other substantial other changes.
Note that you can extract the query or parts thereof into static final values, which might make it easier to digest.
Also note that you can combine * notation with explicit columns (at least in the databases I worked with, so something like this would be fine:
select p.*, pr.id as recipient_id
Of course, the question if this is a good idea might cause some debate.
I am looking for a way to compare two MySQL queries in a unit test. Do you know any library that allows that (all of these asserts should pass):
SQLAssert.assertEquals("select id, name from users", "select id, name from users")
SQLAssert.assertEquals("select id, name from users", "select `id`,`name` from `users`")
While running the queries against an in-memory database and comparing results is the best answer, I think there are less-comprehensive and more brittle options that are nevertheless useful.
In practice it's likely that you can place additional constraints on the syntax of the queries. In your example, there are only select statements, a single table, no where clause, and the only query differences are backticks and spaces, so writing a method that normalizes queries with those constraints would probably be doable. Something like:
private String normalize(String str) {
return str.replaceAll(" +", " ").replaceAll("`", "");
}
These normalized strings can then be compared. This way of doing things is very brittle (and therefore not future proof), but that doesn't mean it can't provide value in certain circumstances. Sure, there are quite a few valid sql statements that would cause this to break, but you don't have to deal with the full set of strings that valid sql entails. You just have to deal with whatever subset of sql your queries use.
If your queries are different enough to make this code unreasonable, it might be easier to use a parser library like JSqlParser to parse out the pieces and then navigate the structure to do your comparison. Again, you don't have to support all of SQL, just whatever subset your queries use. Also, the tests don't have to test full logical equivalence to be useful. A test might just make sure that all the tables mentioned in two queries are the same regardless of joins and ordering. That doesn't make them equivalent, but it does guard against a particular kind of error and is more useful than nothing.
An example of a situation where this could be useful is if you are doing large groups of refactorings on your query-builders and you want to make sure the end queries are equivalent. In this case you aren't testing the queries themselves but the query-building.
I wouldn't suggest doing this as a regular practice in unit tests, but I think it can be useful in very particular circumstances.
You could use JSqlParser to parse your queries. Then you could use the so called Deparser of JSqlParser to get version of your SQL without additional spaces, tabs, linefeeds. From this on, you could use a simple String equality check. Sure you have to process all kinds of quotations like " or [] but it works, like the example code shows.
This does not work, of quotation comes into play or different orders of columns or expression within you SQL. The quotation problem is simple to solve through an extention to the expression deparser.
Statement stmt1 = CCJSqlParserUtil.parse("select id, name from users");
Statement stmt2 = CCJSqlParserUtil.parse("select id, name from users");
Statement stmt3 = CCJSqlParserUtil.parse("select `id`,`name` from `users`");
//Equality
System.out.println(stmt1.toString().equals(stmt2.toString()));
ExpressionDeParser exprDep = new ExpressionDeParser() {
#Override
public void visit(Column tableColumn) {
tableColumn.setColumnName(tableColumn.getColumnName().replace("`", ""));
super.visit(tableColumn);
}
};
SelectDeParser stmtDep = new SelectDeParser() {
#Override
public void visit(Table tableName) {
tableName.setName(tableName.getName().replace("`", ""));
super.visit(tableName);
}
};
exprDep.setBuffer(stmtDep.getBuffer());
stmtDep.setExpressionVisitor(exprDep);
((Select)stmt3).getSelectBody().accept(stmtDep);
String stmt3Txt = stmtDep.getBuffer().toString();
System.out.println(stmt1.toString().equals(stmt3Txt));
I would suggest that the only way to assert that 2 queries return the same result is to actually run them. Of course, what you don't want to do is have unit tests connect to a real database. There are a number of reasons for this:
The tests can affect what is in the database, and you don't want to introduce a load of test data into a production database
Each test should be self contained, and work the same way each time it is run, which requires the database be in the same known state at the start of each run. This requires a reset for each test - not something to do with a production (or dev environment) database.
With these constraints in mind, I suggest you look into DBUnit, which is designed for database-driven JUnit tests. I also suggest, instead of using MySQL for unit tests, use an in-memory database (the examples use HSQLDB), that way, you can test queries without having test data actually persisted.
I can't believe I'm asking this, but...
Is there any way, in Java, to execute a SQL statement (not JPQL) and map the results to a List of Plain Old Java Objects?
I want to be able to create small lightweight POJO objects and then have them populated by raw SQL queries. I'm expressly NOT looking to create complex objects: just primitives, with no relationships.
Everything seems to be centered around JPA/JPQL, but the problem with that is that I do not want to bind my objects to a specific table.
I feel like I'm either:
(a) on crazy pills, or
(b) missing something fundamental
A lightweight mapper is not available as part of the JDK itself. You could either roll-your-own simple mapper using Java's standard JDBC API (in fact JPA implementations build on top of that) or you could have a look at external libraries that provide simple SQL-to-Object mappers. I know MyBatis (formerly known as iBatis).
A) No, I think you're not on crazy pills and B) is it possible that you just missed JDBC?
Sormula may be able to do what you want. You would need to extend Table and override getTableName() and/or getQualifiedTableName() to supply the desired table name since sormula normally associates one POJO to one table. See example 2a and example 3a.
jOOQ has a couple of Record -> POJO mapping capabilities that will probably do the job for you (although jOOQ can do much more). Here's an example:
// A "mutable" POJO class
public class MyBook1 {
public int id;
public String title;
}
// The various "into()" methods allow for fetching records into your POJOs:
List<MyBook1> myBooks = create.select().from(BOOK).fetchInto(MyBook1.class);
Taken from the manual here:
http://www.jooq.org/doc/latest/manual/sql-execution/fetching/pojos/
The mapping algorithm is described in the Javadoc:
http://www.jooq.org/javadoc/latest/org/jooq/impl/DefaultRecordMapper.html
While the above example makes use of jOOQ's DSL API, you can do with plain SQL as well:
List<MyBook1> myBooks = create.resultQuery("SELECT * FROM BOOK")
.fetchInto(MyBook1.class);
You can even operate on a JDBC ResultSet, using jOOQ only for mapping:
ResultSet rs = stmt.executeQuery();
List<MyBook1> myBooks = create.fetch(rs).into(MyBook1.class);