How is this a duplicate as i am specifically asking about JOOQ here?
I am using JOOQ in my JAVA project to handle all my PostgreSQL queries. I read in this article that JOOQ uses prepared statements to execute all queries.
Is it than safe to assume that I don't need to worry about SQL injection or user input when executing my queries?
I don't need to worry about escaping the user input before giving it over to JOOQ?
On the side note, which other vulnerabilities are there to my DB in getting user input (apart from those that are solved by prepared statements), that I should be careful of?
1) Yes, as long as you use the provided API's correctly. It is still possible to inject plain sql queries though so be careful.
All methods in the jOOQ API that allow for plain (unescaped, untreated) SQL contain a warning message in their relevant Javadoc
// This query will use bind values, internally.
create.fetch("SELECT * FROM BOOK WHERE ID = ? AND TITLE = ?", 5, "Animal Farm");
// This query will not use bind values, internally.
create.fetch("SELECT * FROM BOOK WHERE ID = 5 AND TITLE = 'Animal Farm'");
See JOOQ docs here for a more in depth explanation: https://www.jooq.org/doc/3.9/manual/sql-building/bind-values/sql-injection/
2) No, see above.
3) Aside from that just beware of general DB security issues, such as user authentication/roles and storing sensitive data in an unecrypted format etc
Little risk when using jOOQ as intended
When you use jOOQ as intended, then you will run into little risk of SQL injection. The intended usage is:
Using source code generation to generate meta data for your tables / columns, etc.
Using the DSL for type safe embedded SQL
As others have mentioned, jOOQ will always use bind variables, properly escape all inlined values (constants, literals). But again, as others have mentioned, jOOQ still allows for using plain SQL templating for those cases where you need to work around a lack of functionality or vendor specific feature support. In those cases, you have to be as careful as with JDBC and make sure to explicitly use bind variables and avoid string concatenation, yourself.
Preventing accidents with the PlainSQLChecker annotation processor
One way to prevent accidentally using plain SQL templating, and to make sure no one on the team uses it without approval is to use jOOQ's checker framework / error prone integration and disallow all plain SQL usage by default. With Maven, you could configure this (leaving out the JDK version specific details):
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<annotationProcessors>
<annotationProcessor>org.jooq.checker.PlainSQLChecker</annotationProcessor>
</annotationProcessors>
</configuration>
</plugin>
And now your code using methods like DSL.query(String) won't compile anymore, until you explicitly allow it with the #Allow.PlainSQL annotation on the scope of your choice (method, class, package)
It's always possible to write unsafe queries, no matter what language and framework you use.
The naive way of concatenating variables into SQL creates an opportunity for SQL injection:
String unsafeString = "O'Reilly";
create.fetch("SELECT * FROM BOOK WHERE ID = 5 AND TITLE = '"+unsafeString+"'");
// results in SQL syntax error because of unmatched ' marks
Merely using prepared queries does NOT make an unsafe query into a safe query.
Use parameters to separate dynamic values from the SQL query. These are combined within the RDBMS at execution time. There is no way a parameter can cause an SQL injection vulnerability.
String unsafeString = "O'Reilly";
create.fetch("SELECT * FROM BOOK WHERE ID = 5 AND TITLE = ?", unsafeString);
// still OK
When you use parameters, you don't need to do any escaping of the variables. In fact, you must not, because you'll end up with escape symbols in your data.
Parameters are good for combining Java variables into an SQL query, but only in the place of an SQL scalar value. That is, where you would normally use a quoted string literal, quoted date literal, or numeric literal in your SQL, you can replace it with a parameter placeholder.
But you can't use parameters for anything else in SQL:
Table names
Column names
Lists of values, for example for an IN ( ... ) predicate—you must use one ? placeholder per individual value in the list.
SQL expressions
SQL keywords
You might like my presentation SQL Injection Myths and Fallacies (video), or my book, SQL Antipatterns Volume 1: Avoiding the Pitfalls of Database Programming
Re comment from #rehas:
Indeed, using prepared statements does not mean you are using parameters implicitly. I showed an example above (my first example), of concatenating an unsafe variable into an SQL string before it is sent to prepare().
Once the SQL string arrives in the RDBMS server, it has no way of knowing which parts of the string were legitimate and which parts were concatenated from unsafe variables. All it sees is one string containing an SQL statement.
The point of using parameters is to keep the (potentially unsafe) variables separated from the SQL string. Within the RDBMS server, the SQL string—still with parameter placeholders like ?—is parsed. Once it's parsed, it won't be parsed again, so it's safe for strings like "O'Reilly" to be bound to the parameter placeholders without risk of causing mismatched quotes or anything. A parameters is guaranteed to be treated as a single value in the SQL execution, even if the value of the parameter contains characters that would have changed the way the query was parsed, if it had been included before prepare().
It's not true that using prepare() means you're always using parameters.
It's accurate to say that using parameters requires use of prepare() and execute() as separate steps. But some frameworks do both steps for you. I'm sure if you were to read the jOOQ source code, you'd see it.
Related
I have a problem.
I have the following query:
SELECT
Agents.Owner,
Orders.*
FROM
Orders
INNER JOIN Agents ON Agents.id = Orders.agentid
WHERE
Agents.botstate = 'Active' AND Orders.state = 'Active' AND(
Orders.status = 'Failed' OR Orders.status = 'Processing' AND Orders.DateTimeInProgressMicro < DATE_SUB(NOW(), INTERVAL 10 SECOND))
ORDER BY
Orders.agentid
But now I need to convert this to JOOQ language. This is what I came up with:
create.select()
.from(DSL.table("Orders"))
.join(DSL.table("Agents"))
.on(DSL.table("Agents").field("Id").eq(DSL.table("Orders").field("AgentId")))
.where(DSL.table("Agents").field("botstate").eq("Active")
.and(DSL.table("Orders").field("state").eq("Active"))
.and((DSL.table("Orders").field("status").eq("Failed"))
.or(DSL.table("Orders").field("status").eq("Processing")))).fetch().sortAsc(DSL.table("Orders").field("AgentId"));
Now the first problem is that it doesn't like all the .eq() statements, because it gives me the error:
Cannot resolve method: eq(Java.lang.String). And my second problem is that I don't know how to write this statement in JOOQ: Orders.DateTimeInProgressMicro < DATE_SUB(NOW(), INTERVAL 10 SECOND).
The first problem is caused by the fact that I can't just use:
.on(Agents.Id).eq(Orders.AgentId)
But instead I need to enter for every table:
DSL.table("table_name")
And for every column:
DSL.field("column_name")
Without that it doesn't recognize my tables and columns
How can I write the SQL in the JOOQ version correctly or an alternative solution is that I can use normal SQL statements?
Why doesn't your code work?
Table.field(String) does not construct a path expression of the form table.field. It tries to dereference a known field from Table. If Table doesn't have any known fields (e.g. in the case of using DSL.table(String), then there are no fields to dereference.
Correct plain SQL API usage
There are two types of API that allow for working with dynamic SQL fragments:
The plain SQL API to construct plain SQL fragments and templates
The Name API to construct identifiers and jOOQ types from identifiers
Most people use these only when generated code isn't possible (see below), or jOOQ is missing some support for vendor-specific functionality (e.g. some built-in function).
Here's how to write your query with each:
Plain SQL API
The advantage of this API is that you can use arbitrary SQL fragments including vendor specific function calls that are unknown to jOOQ. There's a certain risk of running into syntax errors, SQL injection (!), and simple data type problems, because jOOQ won't know the data types unless you tell jOOQ explicitly
// as always, this static import is implied:
import static org.jooq.impl.DSL.*;
And then:
create.select()
.from("orders") // or table("orders")
.join("agents") // or table("agents")
.on(field("agents.id").eq(field("orders.id")))
.where(field("agents.botstate").eq("Active"))
.and(field("orders.state").eq("Active"))
.and(field("orders.status").in("Failed", "Processing"))
.orderBy(field("orders.agentid"))
.fetch();
Sometimes it is useful to tell jOOQ about data types explicitly, e.g. when using these expressions in SELECT, or when creating bind variables:
// Use the default SQLDataType for a Java class
field("agents.id", Integer.class);
// Use an explicit SQLDataType
field("agents.id", SQLDataType.INTEGER);
Name API
This API allows for constructing identifiers (by default quoted, but you can configure that, or use unquotedName()). If the identifiers are quoted, the SQL injection risk is avoided, but then in most dialects, you need to get case sensitivity right.
create.select()
.from(table(name("orders")))
.join(table(name("agents")))
.on(field(name("agents", "id")).eq(field(name("orders", "id"))))
.where(field(name("agents", "botstate")).eq("Active"))
.and(field(name("orders", "state")).eq("Active"))
.and(field(name("orders", "status")).in("Failed", "Processing"))
.orderBy(field(name("orders", "agentid")))
.fetch();
Using the code generator
Some use cases prevent using jOOQ's code generator, e.g. when working with dynamic schemas that are only known at runtime. In all other cases, it is very strongly recommended to use the code generator. Not only will building your SQL statements with jOOQ be much easier in general, you will also not run into problems like the one you're presenting here.
Your query would read:
create.select()
.from(ORDERS)
.join(AGENTS)
.on(AGENTS.ID.eq(ORDERS.ID))
.where(AGENTS.BOTSTATE.eq("Active"))
.and(ORDERS.STATE.eq("Active"))
.and(ORDERS.STATUS.in("Failed", "Processing"))
.orderBy(ORDERS.AGENTID)
.fetch();
Benefits:
All tables and columns are type checked by your Java compiler
You can use IDE auto completion on your schema objects
You never run into SQL injection problems or syntax errors
Your code stops compiling as soon as you rename a column, or change a data type, etc.
When fetching your data, you already know the data type as well
Your bind variables are bound using the correct type without you having to specify it explicitly
Remember that both the plain SQL API and the identifier API were built for cases where the schema is not known at compile time, or schema elements need to be accessed dynamically for any other reason. They are low level APIs, to be avoided when code generation is an option.
I know that comparing with a normal Select, putting the values directly into the SQL statements, like this way:
Statement command = connection.createStatement( "SELECT * FROM person WHERE name = "+ nameVar);
Doing that with JDBC and positional parameters, like this:
String statment= "SELECT * from Users WHERE name=? and pass =?";
sentence = conexion.prepareStatement(consulta);
sentence.setString(1, nameVar);
sentence.setString(2, passVar);
Is better for several things, like avoid SQL injection, and also uses less memory (for caching the execution plan only once) and performance (for not doing the same execution plan again and again)
But if you have a lots of "?" may be difficult to correlate the parameter with the variable, and that can cause an error.
My question is, if there is a difference between doing the positional parameters as above with doing "named parameters" like this:
String statment= "SELECT * from Users WHERE name=:nameParam and pass =:passParam";
sentence = conexion.prepareStatement(consulta);
sentence.setString("nameParam", nameVar);
sentence.setString("passParam", passVar);
Because is easier and can skip errors
UPDATE
by the comments seems that only using JPA/Hibernate can use :parameters. Well The question remains. There is any difference using Hibernate?
As you point out you are building a PreparedStatement so that the DB can cache it's execution plan etc. So that is where all the performance gain comes from when the driver actually talks to the DB.
The other benefits are for the code. It is DRY-er (Do not Repeat Yourself) you can prevent SQL injection attacks as you can validate the params and they can't add "structural changes" to the SQL. You can keep a ref to the PreparedStatement Object so no need to re-create it etc etc.
But you are doing this on both cases, so the main benefits are the same. Under the covers an Array of params is passed to the Driver to run the prepared statement.
Positional params will most likely just index into the Array (depending on your Drivers' implementation) whereas named params maintain a map of name to position and generate the correct array when required.
Given that the number of params in a SQL statement tends to be small (can usually count them on one or two hands) creating the array is very fast, and negligible compared to the cost of running the query over the network to the DB.
So IMHO use whichever helps you reason about your code best. The performance difference is minuscule.
I have a existing code where the application generates different sql depend of lot of conditions and execute them via hibernate sessions createSQLQuery(). In here the parameters are concat to the sql string which reside in the java class as normal string replacement. The problem here is now i need to prevent sql injections. So for that i have to use getNamedQuery() and bind the parameters so hibernate will take care of special characters. But the problem is moving the string sql's to xml file is a overhead because conditionally generating sql's. So i decide to manually do the special character validation and append it to the string query and execute as it is now.
So then i check the source for PrepareStatement i found, it just throw a exception
byte[] arrayOfByte1 = new byte[0];
try
{
arrayOfByte1 = CharsToBytes(this.OdbcApi.charSet, arrayOfChar);
}
catch (UnsupportedEncodingException localUnsupportedEncodingException) {
}
How can i do same kind of encoding in the java class as above for the parameters before concat them with the string query for eliminate sql injections? Or is there any way i can still keep the string sql as it is an append parameters and use hibernate to execute the query?
As far as I can tell, you want to create SQL queries on the fly because the combination of conditions (from the UI, I guess) can be very complicated. That's fine. All you need to control are the parameters that the user supplies. And for that, you can, and should, still use Hibernate's createSqlQuery(). That function understands either ? for positional parameters (numbered from beginning of query string), or :param_name syntax and then you supply named parameters. You don't need to move anything into an xml file.
Section 16.1.7 has examples.
If you need to assemble custom SQL into a query, I've found writing my own criteria classes that includes the custom SQL works well.
You just need to implement the Criterion interface.
https://docs.jboss.org/hibernate/orm/3.5/api/org/hibernate/criterion/Criterion.html
(See also the Hibernate implementation of 'not null': http://www.grepcode.com/file/repo1.maven.org/maven2/org.hibernate/hibernate/3.2.4.sp1/org/hibernate/criterion/NotNullExpression.java?av=f .)
Then you can simply build up each custom query using the normal hibernate criteria API.
https://docs.jboss.org/hibernate/orm/3.3/reference/en/html/querycriteria.html#querycriteria-creating
Sanitising SQL values properly is painful - try really hard to avoid it! ;-)
I have to process some parameter that is passed in as a quoted SQL string and then quote it again. I am wondering is there any utility in Hibernate or JDBC or some common Java library that can do the unquoting bit?
(Otherwise I guess I can do str.replace("''","'"). Is this correct?)
Edit:
Just some background. I am trying to add functionality to Hibernate. The Hibernate class I am overriding receives its argument as quoted strings. I have no access to the original argument submitted for the bind variable (without rewrite a number of Hibernate classes).
Just don't quote the String object.
there is an issue with your design, you should not be receiving it already quoted, if under any circumstance you need it quoted (avoid that), do it before you are going to use it.
for the JDBC side, you can always do the following, so you don't need SQL quoted strings:
String query ="select myColumn from MyTable where fullname = ?";
PreparedStatement stm = c.prepareStatement(query);
stm.setString(1, "Garis M. Suero");
It's a very good practice to design your software in a simple way in order to mitigate future confusions and errors when developers are coding your software.
For example, if you are receiving this from a third party software you can make an object that will take the string and convert it to the correct value, and maybe have two methods, one 'getValue()and another:getQuotedValue()`
jOOQ runs on generated classes (from database schema). In my apps, there are many tables created while excuting. Then how can I query them using jOOQ? Or can I use something else to do this?
We just use jOOQ to generate SQL, not doing any CRUD operation.
While being the main use case for using jOOQ, you do not have to generate any source code with jOOQ. Several users have used jOOQ the way you do, by supplying dynamic table/column names at runtime to Factory methods such as:
DSL.tableByName(String...)
DSL.fieldByName(String...)
These methods take care of correctly escaping your schema, table, field, alias names, etc, in order to guarantee SQL syntax correctness and SQL injection protection.
Some related discussions on the jOOQ User Group are here:
Escaping SQL identifiers
Intantiating Field and Table by name
Specifying the alias name for a JOIN