I can't believe I'm asking this, but...
Is there any way, in Java, to execute a SQL statement (not JPQL) and map the results to a List of Plain Old Java Objects?
I want to be able to create small lightweight POJO objects and then have them populated by raw SQL queries. I'm expressly NOT looking to create complex objects: just primitives, with no relationships.
Everything seems to be centered around JPA/JPQL, but the problem with that is that I do not want to bind my objects to a specific table.
I feel like I'm either:
(a) on crazy pills, or
(b) missing something fundamental
A lightweight mapper is not available as part of the JDK itself. You could either roll-your-own simple mapper using Java's standard JDBC API (in fact JPA implementations build on top of that) or you could have a look at external libraries that provide simple SQL-to-Object mappers. I know MyBatis (formerly known as iBatis).
A) No, I think you're not on crazy pills and B) is it possible that you just missed JDBC?
Sormula may be able to do what you want. You would need to extend Table and override getTableName() and/or getQualifiedTableName() to supply the desired table name since sormula normally associates one POJO to one table. See example 2a and example 3a.
jOOQ has a couple of Record -> POJO mapping capabilities that will probably do the job for you (although jOOQ can do much more). Here's an example:
// A "mutable" POJO class
public class MyBook1 {
public int id;
public String title;
}
// The various "into()" methods allow for fetching records into your POJOs:
List<MyBook1> myBooks = create.select().from(BOOK).fetchInto(MyBook1.class);
Taken from the manual here:
http://www.jooq.org/doc/latest/manual/sql-execution/fetching/pojos/
The mapping algorithm is described in the Javadoc:
http://www.jooq.org/javadoc/latest/org/jooq/impl/DefaultRecordMapper.html
While the above example makes use of jOOQ's DSL API, you can do with plain SQL as well:
List<MyBook1> myBooks = create.resultQuery("SELECT * FROM BOOK")
.fetchInto(MyBook1.class);
You can even operate on a JDBC ResultSet, using jOOQ only for mapping:
ResultSet rs = stmt.executeQuery();
List<MyBook1> myBooks = create.fetch(rs).into(MyBook1.class);
Related
Can jOOQ automatically add an alias prefix for all columns of a table in the select clause?
Can jOOQ also help with the 128 byte/character name length limitation of databases?
The reason for the questions are that SimpleFlatMapper is used for the mapping.
SimpleFlatMapper requires the fetched database column names to map to the model.
Model example (really getter/setter are used):
class Head {
public Integer id;
public List<Position> positions;
...
}
class Position {
public Integer id;
public Integer headId;
...
}
The naming can be done individually:
ResultSet resultSet = dsl.select(..., POSITION.ID.as("positions_id"), ...)
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
List<Head> headers = JdbcMapperFactory.newInstance()
.ignorePropertyNotFound()
.newMapper(Head.class)
.stream(resultSet)
.collect(Collectors.toList())
However if it is a complicated model with several joins/columns it is a bit tedious.
The only solution I found was to program a function manually. Have I however maybe overlooked something and jOOQ can do it by itself or help?
Something the likes of this would be very nice (probably not the best naming, I couldn't come up with something better on the spot):
ResultSet resultSet = dsl.select()
.from(HEAD)
.join(POSITION.as("positions").prefixColumns()).onKey()
.fetchResultSet();
Or:
ResultSet resultSet = dsl.selectWithTableAlias()
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
Resulting in the following SQL:
SELECT head.id, head.***, positions.id AS positions_id, positions.headid AS positions_headid, positions.***
FROM head JOIN position AS positions ON head.id = positions.headid
Furthermore databases like Oracle and MSSQL have a limitation of 128 bytes/characters for names.
In very rare complex scenarios the alias names might reach that limit because of the needed nesting.
jOOQ cannot offer a workaround for this in some form or can it? So basically define a name that is used in the SQL and a name for the actual resulting object.
I know, very niche requirements. But they would help a lot with the mapping.
Can jOOQ automatically add an alias prefix for all columns of a table in the select clause?
There's no such feature, because the possible sets of desired auto-prefix algorithms is quite big, and jOOQ ultimately wouldn't do exactly what you want. Maybe, there's room for an SPI to help you do this, in the future. But there isn't one available yet: https://github.com/jOOQ/jOOQ/issues/11545
However, you can easily do this yourself, because every jOOQ query is just a dynamically constructed expression tree, even if you're not always using jOOQ for dynamic SQL.
You can easily write a utility and use that everywhere:
public static List<Field<?>> autoPrefix(Field<?>... fields) {
return autoprefix(Arrays.asList(fields));
}
public static List<Field<?>> autoPrefix(Field<?>... fields) {
Stream.of(fields).map(f -> f.as(myPrefixLogic(f))).collect(toList());
}
This can now be used explicitly on all DSLContext.select() calls, e.g.
dsl.select(autoPrefix(..., POSITION.ID.as("positions_id"), ...))
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
Alternatively, you can wrap your query in a derived table:
public ResultSet fetchResultSetWithPrefix(Select<?> select) {
Table<?> table = select.asTable("t");
return
dsl.select(autoPrefix(table.fields()))
.from(table)
.fetchResultSet();
}
And now, instead of calling fetchResultSet(), you call your auxiliary function:
try (ResultSet rs = fetchResultSetWithPrefix(
dsl.select(..., POSITION.ID.as("positions_id"), ...)
.from(HEAD)
.join(POSITION.as("positions")).onKey()
)) {
...
}
Another, more complex option would be to do this with a VisitListener.
Remember, irrespective of what your query looks like (because jOOQ's DSL mimicks SQL, syntactically), all your jOOQ queries are dynamic SQL queries, so you can relatively easily transform them to whatever you want, automatically.
Can jOOQ also help with the 128 byte/character name length limitation of databases?
I'll have the same reservations about this automatism as before. Where would you see jOOQ provide automatic help here? Given your naming scheme (table_column) would you like a fair distribution of 63 characters per object type? Or is the table less important than the column, and you'll truncate table names at 42 characters, leaving 85 characters for the column?
What if you wanted to fully qualify identifiers, including the schema name, as in schema_table_column? What if you project UDTs, meaning you'll get schema_table_column_attribute1_attribute2
I don't think any automation can be provided by jOOQ which would suit all needs. However, as shown above, it is very simple to implement this only once for your entire application, making sure the naming is always applied correctly. And if an SPI were available, you could implement your abbreviation logic there.
I am using jOOQ for working with a relational database. I have a SELECT query for which I need to write unit tests with mocking. Based on this doc and this post, I need to define my own data provider, which should look something like this:
class MyProvider implements MockDataProvider {
DSLContext create = DSL.using(SQLDialect.MYSQL);
#Override
public MockResult[] execute(MockExecuteContext mockExecuteContext) throws SQLException {
MockResult[] mock = new MockResult[1];
String sql = mockExecuteContext.sql();
if (sql.startsWith("select")) {
Result<Record2<String, String>> result = create.newResult(COL_1, COL_2);
result.add(create.newRecord(COL_1, COL_2)
.values("val1", "val2"));
mock[0] = new MockResult(1, result);
}
return mock;
}
}
where COL_1 and COL_2 are defined as follows:
Field<String> COL_1 = field("Column1", String.class);
Field<String> COL_2 = field("Column2", String.class);
It's quite simple and straightforward when SELECT is a small one (as in the above example, just 2 columns). I am wondering how it should be done in case of complex and large selects. For instance I have a SELECT statement which selects 30+ columns from multiple table joins. Seems the same approach of
Result<Record_X<String, ...>> result = create.newResult(COL_1, ...);
result.add(create.newRecord(COL_1, ...)
.values("val1", ...));
does not work in case of more than 22 columns.
Any help is appreciated.
Answering your question
There is no such limitation as a maximum of 22 columns. As documented here:
Higher-degree records
jOOQ chose to explicitly support degrees up to 22 to match Scala's typesafe tuple, function and product support. Unlike Scala, however, jOOQ also supports higher degrees without the additional typesafety.
You can still construct a record with more than 22 fields using DSLContext.newRecord(Field...). Now, there is no values(Object...) method on the Record type, because the Record type is the super type of all the Record1 - Record22 types. If such an overload were present, then the type safety on the sub types would be lost, because the values(Object...) method is applicable for all types of arguments. This might be fixed in the future by introducing a new RecordN subtype.
But you can load data into your record with other means, e.g. by calling Record.fromArray(Object...):
Record record = create.newRecord(COL_1, ...);
record.fromArray("val1", ...);
result.add(record);
The values() method being mere convenience (adding type safety) for fromArray().
Disclaimer:
I'm assuming you read the disclaimer on the documentation page you've linked. I'm posting it here anyway for other readers of this question, who might not have read the disclaimer:
Disclaimer: The general idea of mocking a JDBC connection with this jOOQ API is to provide quick workarounds, injection points, etc. using a very simple JDBC abstraction. It is NOT RECOMMENDED to emulate an entire database (including complex state transitions, transactions, locking, etc.) using this mock API. Once you have this requirement, please consider using an actual database instead for integration testing, rather than implementing your test database inside of a MockDataProvider.
It seems you're about to re-implement a database which can "run" any type of query, including a query with 23+ columns, and every time you change the query under test, you will also change this test here. I still recommend you do integration testing instead, using testcontainers or even with H2, which will help cover many more queries than any such unit test approach. Here's a quick example showing how to do that: https://github.com/jOOQ/jOOQ/tree/main/jOOQ-examples/jOOQ-testcontainers-example
Also, integration tests will help test query correctness. Unit tests like these will only provide dummy results, irrespective of the actual query. It is likely that such mocks can be implemented much more easily on a higher level than the SQL level, i.e. by mocking the DAO, or repository, or whatever methods, instead.
How can i use jpa for query over an object (not an entity)?
For example this simple code:
String [] theList = {a,b,c,d}.
Query q = new Query("Select tl from theList tl")
Reason behind: the queries are dynamically created and executed, but the objects in the from clause of the jpql query aren't necessarily mapped tables. In some cases there are just an Object, So the actual behavior needed is modify the query during execution of the program to meet the criteria, but i don't know how to modify the query.
Edit: I Don't use native queries because of portability of code. It will be the last option.
What you're looking for is called LINQ, and unfortunately (?) it is available only in C#.
However, you can partially emulate it with Stream(s).
A Stream offers basically all the operators you need
.filter() where
.max() max
.sorted() orderby
.limit() limit
.skip() offset
.collect(groupingBy()) group by
And so on. Just give a look at the Javadoc!
I think 'JdbcTemplate' would suffice your requirement.
JdbcTemplate gives you the flexibility to run native queries and map them to a Java class.
However, you'll have to explicitly map your Java class with the column names in the database.
I have solved using joSQL. Is a powerfull opensource tool that allows you to query over java objects using "sql". It is not jpa but satisfied my needs.
Another tool i have seen that do that is called querydsl.
I'm currently evaluating JOOQ because I believe I started reinventing the wheel which looks very close to part of JOOQ :)
Now, while digging in great JOOQ documentation I've found that my use case lies somewhere between Using JOOQ as SQL Builder and Using JOOQ as SQL Builder with Code generation i.e. I would like to:
Create plain SQL strings like it is shown in Using JOOQ as SQL Builder part
Instead of using hard-coded DSL.fieldByName("BOOK","TITLE") constructs, I prefer storing name of a table along with it's column names and types like it's shown in Using JOOQ as SQL Builder with Code generation part
I prefer not to use code generation (at least not on regular basis), but rather creating TableImpl myself when new table is needed.
While digging in manual, I've found how table implementation should look like in chapter Generated tables. However, TableImpl class as well as Table interface should be parameterized with record type and the same goes for TableField class. I believe this is done for easier type inference when directly querying database and retrieving results, though I may be mistaken.
So my questions are:
Is there a guide in manual on how to create Table and TableField implementations? Or I can simply generate them once for my database schema and use generated code as a guideline?
How can I gracefully "discard" record type parameters in implemented classes? First, I thought about using java.lang.Void class as type parameter but then I noticed that only subclasses of Record are allowed... The reason is that I don't need record types at all because I plan to use generated by JOOQ SQL queries in something like Spring JdbcTemplate so mapping is done by myself.
Thanks in advance for any help!
Given your use-case, I'm not sure why you'd like to roll your own Table and TableField implementations rather than using the ones generated by jOOQ. As you stated yourself, you don't have to regenerate that code every time the DB schema changes. Many users will just generate the schema once in a while and then put the generated artefacts under version control. This will help you keep track of newly added changes.
To answer your questions:
Yes, there are some examples around the use of CustomTable. You may also find some people sharing similar experiences on the user group
Yes you can just use Record. Your minimal custom table type would then be:
class X extends TableImpl<Record> {
public X() {
super("x");
}
}
Note that you will be using jOOQ's internal API (TableImpl), which is not officially supported. While I'm positive that it'll work, it might break in the future, e.g. as the super constructor signature might change.
My Java (JDK6) project uses Spring and JDBCTemplate for all its database access. We recently upgraded from Spring 2.5 to Spring 3 (RC1). The project does not use an ORM like Hibernate nor EJB.
If I need to read a bunch of records, and do some internal processing with them, it seems like there are several (overloaded) methods: query, queryForList and queryForRowSet
What should be the criteria to use one instead of the other? Are there any performance differences? Best practices?
Can you recommend some external references for further research on this topic?
I find that the standard way to access as list is via the query() methods rather than any of the other approaches. The main difference between query and the other methods is that you'll have to implement one of the callback interfaces (either RowMapper, RowCallbackHandler, or ResultSetExtractor) to handle your result set.
A RowMapper is likely what you'll find yourself using most of the time. It's used when each row of the result set corresponds to one object in your list. You only have to implement a single method mapRow where you populate the type of object that goes in your row and return it. Spring also has a BeanPropertyRowMapper which can populate the objects in a list via matching the bean property names to the column names (NB this class is for convenience not performance).
A RowCallbackHandler is more useful when you need your results to be more than just a simple list. You'll have to manage the return object yourself you are using this approach. I usually find myself using this when I need a map structure as my return type (i.e. for grouped data for a tree table or if I'm creating a custom cache based of the primary key).
A ResultSetExtractor is used when you want to control the iteration of the results. You implment a single method extractData that will be the return value of the call to query. I only find myself using this if I have to build some custom data structure that is more complex to build using either of the other callback interfaces.
The queryForList() methods are valuable in that you don't have to implement these callback methods. There are two ways use queryForList. The first is if you're only querying a single column from the database (for example a list of strings) you can use the versions of the method that takes a Class as an argument to automatically give you a list of only objects of those classes.
When calling the other implementations of queryForList() you'll get a list back with each entry being a map of for each column. While this is nice in that you are saved the expense of writing the callback methods, dealing with this data structure is quite unwieldy. You'll find yourself doing a lot of casting since the map's values are of type Object.
I've actually never seen the queryForRowSet methods used in the wild. This will load the entire result of the query into a CachedRowSet object wapped by a Spring SqlRowSet. I see a big downside in using this object in that if you're passing the SqlRowSet around to the other layers of your application, you're coupling those layers to your data access implementation.
You shouldn't see any huge performance differences between any of these calls except as I mentioned with the BeanPropertyRowMapper. If you're working with some complex manipulation of a large result set, you might be able to get some performance gains from writing an optimized ResultSetExtractor for your specific case.
If you want to learn more I would consult the Spring JDBC documentation and the JavaDoc for the classes I've mentioned. You can also take a look at some of the books on the Spring Framework. Though it's a bit dated Java Development with the Spring Framework has a very good section on working with the JDBC framework. Most of all, I would say just try writing some code with each method and see what works best for you.
Since you are in the wonderful Generics land, what you may really want to do is to use SimpleJdbcTemplate and use its query() methods for Lists of objects and queryForObject() for individual objects. Reasoning for this simply is that they're even easier to use than the ones in JdbcTemplate.
One small addition to the excellent answers above: additional methods, like queryForInt, queryForLong, queryForMap, queryForObject, etc. might seem like good options at times if you're running a simple query and expect a single row.
However, if you could get 0 or 1 rows back, the queryForList method is generally easier, otherwise you'd have to catch IncorrectResultSizeDataAccessException. I learned that the hard way.