I'm working with DataNucleus as part of a Google App Engine project and I'm having a bit of trouble with columns in persistence.
#PrimaryKey(column = "user_id")
#Column(name = "user_id")
#Persistent(name = "user_id", column = "user_id", valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key m_id;
#Column(name = "user_name")
#Persistent(name = "user_name", column = "user_name")
private String m_userName;
If you can't tell, I'm trying to name the column something different than the name of the variable because I have two naming conventions (one works better in Java, one works better in SQL). Anyway, I've tried all variations of those marker annotations but the DataNucleus enhancer refuses to honor any of them, so when I run a query like this:
Query q = pm.newQuery(User.class,
"user_name == _username");
I always get an error like this:
org.datanucleus.store.appengine.FatalNucleusUserException: Unexpected expression type while parsing query. Are you certain that a field named user_name exists on your object?
Of course, when a run a query like this:
Query q = pm.newQuery(User.class,
"m_userName == _username");
...everything just works great. So, there would be a field named user_name if any of those annotations were honored, but they're clearly not.
SO my question is: Is there any way to decouple the tokens I use in the query from the name of the field? I'm looking for the ability to change the names of the fields without having to edit the queries by hand.
NOTE: I would sooner just use my SQL naming conventions in the Java classes than write the hideous amounts of XML by hand, so this has to be done with the annotations.
No idea of the talk of SQL, you're using GAE/J hence BigTable and not an RDBMS so SQL just won't work. #Column likely does nothing since it is for ORM. Here you're using JDOQLas the query language, so you use field names ... since it is an Object-Oriented query language. This is NOT SQL. You detest "this" ? JDOQL uses Java syntax, hence "this" makes lots of sense.
If you really want to have a type-safe query extension that allows refactoring then QueryDSL provides JDOQL for use with DataNucleus.
PS The DataNucleus enhancer has nothing to do with column names. It simply adds on extra methods for detecting updates to fields, as per the JDO spec.
Not 100% sure I get what is your problem. If you use m_userName in your query does it get translated as user_name in the SQL query?
You express you query according to the java class names and variables and they get translated to work according to the the SQL schema table and column names. That's most of the time what people want.
BTW, m_id and m_userName is a terrible naming convention for Java code. I would strongly advise you to follow the usual convention.
Related
I have a problem.
I have the following query:
SELECT
Agents.Owner,
Orders.*
FROM
Orders
INNER JOIN Agents ON Agents.id = Orders.agentid
WHERE
Agents.botstate = 'Active' AND Orders.state = 'Active' AND(
Orders.status = 'Failed' OR Orders.status = 'Processing' AND Orders.DateTimeInProgressMicro < DATE_SUB(NOW(), INTERVAL 10 SECOND))
ORDER BY
Orders.agentid
But now I need to convert this to JOOQ language. This is what I came up with:
create.select()
.from(DSL.table("Orders"))
.join(DSL.table("Agents"))
.on(DSL.table("Agents").field("Id").eq(DSL.table("Orders").field("AgentId")))
.where(DSL.table("Agents").field("botstate").eq("Active")
.and(DSL.table("Orders").field("state").eq("Active"))
.and((DSL.table("Orders").field("status").eq("Failed"))
.or(DSL.table("Orders").field("status").eq("Processing")))).fetch().sortAsc(DSL.table("Orders").field("AgentId"));
Now the first problem is that it doesn't like all the .eq() statements, because it gives me the error:
Cannot resolve method: eq(Java.lang.String). And my second problem is that I don't know how to write this statement in JOOQ: Orders.DateTimeInProgressMicro < DATE_SUB(NOW(), INTERVAL 10 SECOND).
The first problem is caused by the fact that I can't just use:
.on(Agents.Id).eq(Orders.AgentId)
But instead I need to enter for every table:
DSL.table("table_name")
And for every column:
DSL.field("column_name")
Without that it doesn't recognize my tables and columns
How can I write the SQL in the JOOQ version correctly or an alternative solution is that I can use normal SQL statements?
Why doesn't your code work?
Table.field(String) does not construct a path expression of the form table.field. It tries to dereference a known field from Table. If Table doesn't have any known fields (e.g. in the case of using DSL.table(String), then there are no fields to dereference.
Correct plain SQL API usage
There are two types of API that allow for working with dynamic SQL fragments:
The plain SQL API to construct plain SQL fragments and templates
The Name API to construct identifiers and jOOQ types from identifiers
Most people use these only when generated code isn't possible (see below), or jOOQ is missing some support for vendor-specific functionality (e.g. some built-in function).
Here's how to write your query with each:
Plain SQL API
The advantage of this API is that you can use arbitrary SQL fragments including vendor specific function calls that are unknown to jOOQ. There's a certain risk of running into syntax errors, SQL injection (!), and simple data type problems, because jOOQ won't know the data types unless you tell jOOQ explicitly
// as always, this static import is implied:
import static org.jooq.impl.DSL.*;
And then:
create.select()
.from("orders") // or table("orders")
.join("agents") // or table("agents")
.on(field("agents.id").eq(field("orders.id")))
.where(field("agents.botstate").eq("Active"))
.and(field("orders.state").eq("Active"))
.and(field("orders.status").in("Failed", "Processing"))
.orderBy(field("orders.agentid"))
.fetch();
Sometimes it is useful to tell jOOQ about data types explicitly, e.g. when using these expressions in SELECT, or when creating bind variables:
// Use the default SQLDataType for a Java class
field("agents.id", Integer.class);
// Use an explicit SQLDataType
field("agents.id", SQLDataType.INTEGER);
Name API
This API allows for constructing identifiers (by default quoted, but you can configure that, or use unquotedName()). If the identifiers are quoted, the SQL injection risk is avoided, but then in most dialects, you need to get case sensitivity right.
create.select()
.from(table(name("orders")))
.join(table(name("agents")))
.on(field(name("agents", "id")).eq(field(name("orders", "id"))))
.where(field(name("agents", "botstate")).eq("Active"))
.and(field(name("orders", "state")).eq("Active"))
.and(field(name("orders", "status")).in("Failed", "Processing"))
.orderBy(field(name("orders", "agentid")))
.fetch();
Using the code generator
Some use cases prevent using jOOQ's code generator, e.g. when working with dynamic schemas that are only known at runtime. In all other cases, it is very strongly recommended to use the code generator. Not only will building your SQL statements with jOOQ be much easier in general, you will also not run into problems like the one you're presenting here.
Your query would read:
create.select()
.from(ORDERS)
.join(AGENTS)
.on(AGENTS.ID.eq(ORDERS.ID))
.where(AGENTS.BOTSTATE.eq("Active"))
.and(ORDERS.STATE.eq("Active"))
.and(ORDERS.STATUS.in("Failed", "Processing"))
.orderBy(ORDERS.AGENTID)
.fetch();
Benefits:
All tables and columns are type checked by your Java compiler
You can use IDE auto completion on your schema objects
You never run into SQL injection problems or syntax errors
Your code stops compiling as soon as you rename a column, or change a data type, etc.
When fetching your data, you already know the data type as well
Your bind variables are bound using the correct type without you having to specify it explicitly
Remember that both the plain SQL API and the identifier API were built for cases where the schema is not known at compile time, or schema elements need to be accessed dynamically for any other reason. They are low level APIs, to be avoided when code generation is an option.
Can jOOQ automatically add an alias prefix for all columns of a table in the select clause?
Can jOOQ also help with the 128 byte/character name length limitation of databases?
The reason for the questions are that SimpleFlatMapper is used for the mapping.
SimpleFlatMapper requires the fetched database column names to map to the model.
Model example (really getter/setter are used):
class Head {
public Integer id;
public List<Position> positions;
...
}
class Position {
public Integer id;
public Integer headId;
...
}
The naming can be done individually:
ResultSet resultSet = dsl.select(..., POSITION.ID.as("positions_id"), ...)
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
List<Head> headers = JdbcMapperFactory.newInstance()
.ignorePropertyNotFound()
.newMapper(Head.class)
.stream(resultSet)
.collect(Collectors.toList())
However if it is a complicated model with several joins/columns it is a bit tedious.
The only solution I found was to program a function manually. Have I however maybe overlooked something and jOOQ can do it by itself or help?
Something the likes of this would be very nice (probably not the best naming, I couldn't come up with something better on the spot):
ResultSet resultSet = dsl.select()
.from(HEAD)
.join(POSITION.as("positions").prefixColumns()).onKey()
.fetchResultSet();
Or:
ResultSet resultSet = dsl.selectWithTableAlias()
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
Resulting in the following SQL:
SELECT head.id, head.***, positions.id AS positions_id, positions.headid AS positions_headid, positions.***
FROM head JOIN position AS positions ON head.id = positions.headid
Furthermore databases like Oracle and MSSQL have a limitation of 128 bytes/characters for names.
In very rare complex scenarios the alias names might reach that limit because of the needed nesting.
jOOQ cannot offer a workaround for this in some form or can it? So basically define a name that is used in the SQL and a name for the actual resulting object.
I know, very niche requirements. But they would help a lot with the mapping.
Can jOOQ automatically add an alias prefix for all columns of a table in the select clause?
There's no such feature, because the possible sets of desired auto-prefix algorithms is quite big, and jOOQ ultimately wouldn't do exactly what you want. Maybe, there's room for an SPI to help you do this, in the future. But there isn't one available yet: https://github.com/jOOQ/jOOQ/issues/11545
However, you can easily do this yourself, because every jOOQ query is just a dynamically constructed expression tree, even if you're not always using jOOQ for dynamic SQL.
You can easily write a utility and use that everywhere:
public static List<Field<?>> autoPrefix(Field<?>... fields) {
return autoprefix(Arrays.asList(fields));
}
public static List<Field<?>> autoPrefix(Field<?>... fields) {
Stream.of(fields).map(f -> f.as(myPrefixLogic(f))).collect(toList());
}
This can now be used explicitly on all DSLContext.select() calls, e.g.
dsl.select(autoPrefix(..., POSITION.ID.as("positions_id"), ...))
.from(HEAD)
.join(POSITION.as("positions")).onKey()
.fetchResultSet();
Alternatively, you can wrap your query in a derived table:
public ResultSet fetchResultSetWithPrefix(Select<?> select) {
Table<?> table = select.asTable("t");
return
dsl.select(autoPrefix(table.fields()))
.from(table)
.fetchResultSet();
}
And now, instead of calling fetchResultSet(), you call your auxiliary function:
try (ResultSet rs = fetchResultSetWithPrefix(
dsl.select(..., POSITION.ID.as("positions_id"), ...)
.from(HEAD)
.join(POSITION.as("positions")).onKey()
)) {
...
}
Another, more complex option would be to do this with a VisitListener.
Remember, irrespective of what your query looks like (because jOOQ's DSL mimicks SQL, syntactically), all your jOOQ queries are dynamic SQL queries, so you can relatively easily transform them to whatever you want, automatically.
Can jOOQ also help with the 128 byte/character name length limitation of databases?
I'll have the same reservations about this automatism as before. Where would you see jOOQ provide automatic help here? Given your naming scheme (table_column) would you like a fair distribution of 63 characters per object type? Or is the table less important than the column, and you'll truncate table names at 42 characters, leaving 85 characters for the column?
What if you wanted to fully qualify identifiers, including the schema name, as in schema_table_column? What if you project UDTs, meaning you'll get schema_table_column_attribute1_attribute2
I don't think any automation can be provided by jOOQ which would suit all needs. However, as shown above, it is very simple to implement this only once for your entire application, making sure the naming is always applied correctly. And if an SPI were available, you could implement your abbreviation logic there.
I have this domain model:
final class Project {
private final #Id
#With
long projectId;
private final String projectType;
#With
private final ProjectRecipient projectRecipient;
#With
private final Set<PreCalculationCosts> preCalculationCosts;
#With
private final Set<PostCalculationCosts> postCalculationCosts;
}
The problem:
When i call the findById method from the CrudRepository the property projectRecipient gets materialized.
I even see all the issued sql statements being necessary for that in the logs.
When i use my own Query only the One to Many properties are getting materialized (there is no select statement issued for the one to one related projectrecipient):
select p.* from project p
inner join projectrecipient pr on pr.project = p.projectid
where p.projectid = :projectId
EDIT
When i debug the findById method and use this generated SQL as Query value, it gets materialized the right way. Problem with that is, that my Project table has tons of columns, so the Query value string is kind of 5 lines in my IDE(A) ...
On the other hand, i cant use the findById method because i need some postgres specific similar to clause ...
Currently there is no alternative to spelling out the full SQL statement including the columns for the referenced entity prefixed with the relationship name and an _.
If you don't include these columns the referenced entity will be considered to be null
The Spring Data team is thinking about enabling a way to provide everything but the select clause of a query, but it will take quite some time until these thoughts turn into code since the need some other substantial other changes.
Note that you can extract the query or parts thereof into static final values, which might make it easier to digest.
Also note that you can combine * notation with explicit columns (at least in the databases I worked with, so something like this would be fine:
select p.*, pr.id as recipient_id
Of course, the question if this is a good idea might cause some debate.
I have two tables, A_TABLE and B_TABLE. in A_TABLE entity class need on formula which has B_TABLE column combination like below code,
Working Code:
A_TABLEEntity {
#Column("BM_NAME_I")
private String bmdName;
#Formula("(select b.LAST_NAME || ', '||b.FIRST_NAME||' ('||b.BM_NAME||')'
from BR_SCHEMA.B_TABLE b where UPPER(b.BM_NAME)=UPPER(BM_NAME_I))")
private string nameCombinationB;
}
Need solution in Formula :
1) Is it possible to provide any way to give B_TABLEEntity class instead of B_TABLE directly and columns from B_table entity class?
And I have tried with entity class its throwing error, - table or view does not exist
2) Is it possible to avoid to give SCHEMA name in B_TABLE before in formula?
And without schema error is throwing - table or view does not exist
Please help me above #Formula JPA code
You should definitely look here:
https://docs.jboss.org/hibernate/orm/5.4/userguide/html_single/Hibernate_User_Guide.html#mapping-column-formula
According to the described comment, #Formula takes only native sql and the article warns you about coupling to the specific database in some cases.
You should be aware that the #Formula annotation takes a native SQL
clause which may affect database portability.
As the #Formula requires native SQL, you should always include schema. I think that some DB's have default schema that does not need to be defined explicitly.
For some advanced operations, I would probably provide some annotation like #PostLoad and load desired properties using good old entitymanager or direct jdbc.
Maybe these links may help:
https://docs.jboss.org/hibernate/orm/5.4/userguide/html_single/Hibernate_User_Guide.html#basic
https://docs.jboss.org/hibernate/orm/5.4/userguide/html_single/Hibernate_User_Guide.html#fetching
I have a Java, GraphQL, Hibernate, PostgreSQL, QueryDSL application that queries a very large PostgreSQL table with over 275 columns.
I've created a GraphQL schema with the 25 most popular columns as query-able fields. I'd like to add a generic "field" input type that consists of a name (the db column name + "_" + operation (like gte, gt, contains, etc.) and a value (the value the user is searching for).
So when the user (in GraphiQL) enters something like (field:{name:"age_gt", value:"50"}) as a search input to the GraphQL query, I can come up with: "age > 50".
All that works fine, but when it's time to create the Predicate and add it to the whole query ( booleanBuilder.and(new Predicate) ), I cannot figure out how to create a Predicate that just contains a raw String of SQL ("age > 50").
I've created several Predicates the "right" way using my entity POJO tied to Hibernate and the jpa generated "Q" object. But I need the ability to add one or more Predicates that are just a String of SQL. I'm not even sure if the ability exists, the documentation for QueryDSL Predicates is non-existent.
I'm thinking PredicateOperation() might be the answer, but again, no documentation and I cannot find any examples online.
My apologies for not posting code, all my stuff is behind a firewall on a different network so there's no cut and paste to my internet machine.
In Hibernate its possible to inject arbitrary SQL using custom functions or the FUNCTION-function (introduced in JPA 2.1). In QueryDSL its possible to inject arbitrary JPQL/HQL through TemplateExpressions. Combined you get:
Expressions.numberTemplate("FUNCTION('SUM', {0}), x)
However, age > 50 as expression is probably valid JPQL as well, so one can just write:
Expressions.numberTemplate("SUM(age)")
Either way, its probably best to create a visitor that traverses the GraphQL query and creates the proper expression in QueryDSL, as TemplateExpressions are prone to SQL injection.