Logging PreparedStatements in Java - java

One thing that always been a pain is to log SQL (JDBC) errors when you have a PreparedStatement instead of the query itself.
You always end up with messages like:
2008-10-20 09:19:48,114 ERROR LoggingQueueConsumer-52 [Logger.error:168] Error
executing SQL: [INSERT INTO private_rooms_bans (room_id, name, user_id, msisdn,
nickname) VALUES (?, ?, ?, ?, ?) ON DUPLICATE KEY UPDATE room_id = ?, name = ?,
user_id = ?, msisdn = ?, nickname = ?]
Of course I could write a helper method for retrieving the values and parsing/substitute the question marks with real values (and probably will go down that path if I don't get an outcome of this question), but I just wanted to know if this problem was resolved before by someone else and/or if is there any generic logging helper that would do that automagically for me.
Edited after a few answers:
The libraries provided so far seems to be suitable to logging the statements for debugging, which no doubt is useful. However, I am looking to a way of taking a PreparedStatement itself (not some subclass) and logging its SQL statement whenever an error occur. I wouldn't like to deploy a production app with an alternate implementation of PreparedStatement.
I guess what I am looking for an utility class, not a PreparedStatement specialization.
Thanks!

I tried log4jdbc and it did the job for me.
SECURITY NOTE: As of today August 2011, the logged results of a log4jdbc prepared statement are NOT SAFE to execute. They can be used for analysis, but should NEVER be fed back into a DBMS.
Example of log generated by logjdbc:
2010/08/12 16:30:56 jdbc.sqlonly
org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
8. INSERT INTO A_TABLE
(ID_FILE,CODE1,ID_G,ID_SEQUENCE,REF,NAME,BAR,DRINK_ID,AMOUNT,DESCRIPTION,STATUS,CODE2,REJECT_DESCR,ID_CUST_REJ)
VALUES
(2,'123',1,'2','aa','awe',null,'0123',4317.95,'Rccc','0',null,null,null)
The library is very easy to setup:
My configuration with HSQLDB :
jdbc.url=jdbc:log4jdbc:hsqldb:mem:sample
With Oracle :
jdbc.url=jdbc:log4jdbc:oracle:thin:#mybdd:1521:smt
jdbc.driverClass=net.sf.log4jdbc.DriverSpy
logback.xml :
<logger name="jdbc.sqlonly" level="DEBUG"/>
Too bad it wasn't on a maven repository, but still useful.
From what I tried, if you set
You will only get the statements in error, however, I don't know if this library has an impact on performance.

This is very database-dependent. For example, I understand that some JDBC drivers (e.g. sybase, maybe ms-sql) handle prepared statements by create a temporary stored procedure on the server, and then invoking that procedure with the supplied arguments. So the complete SQL is never actually passed from the client.
As a result, the JDBC API does not expose the information you are after. You may be able to cast your statement objects the internal driver implementation, but probably not - your appserver may well wrap the statements in its own implementation.
I think you may just have to bite the bullet and write your own class which interpolates the arguments into the placeholder SQL. This will be awkward, because you can't ask PreparedStatement for the parameters that have been set, so you'll have to remember them in a helper object, before passing them to the statement.
It seems to me that one of the utility libraries which wrap your driver's implementation objects is the most practical way of doing what you're trying to achieve, but it's going to be unpleasant either way.

Use P6Spy: Its Oracle, Mysql, JNDI, JMX, Spring and Maven friendly. Highly configurable.
Simple and low level integration
Can print the stacktrace.
Can only print heavy calls - time threashold based.

If you are using MySQL, MySQL Connector's PreparedStatement.toString() does include the bound parameters. Though third-party connection pools may break this.
Sub-class PreparedStatement to build up the query string as parameters are added. There's no way to extract the SQL from a PreparedStatement, as it uses a compiled binary form.
LoggedPreparedStatement looks promising, though I haven't tried it.
One advantage of these over a proxy driver that logs all queries is that you can modify the query string before logging it. For example in a PCI environment you might want to mask card numbers.

Related

Is there a java equivalent of Mysql's "multiple-statement-queries"

So basically, I would like to avoid stored procedures, but at the same time I would'nt want multiple round-trips to database to execute sequential statements.
Apparently this blog says Facebook uses mysql's multiple-statement-queries. Unfortunately, its a C API, is there a java equivalent of it?
So in brief, the question "is in java+mysql how can a second jdbc statement use the output of the first statement as input to execute, without a round-trip to database and without a storedproc" ?
If not how do other people approach this problem?
Yes, the JDBC driver for MySQL support the multi-statement queries. It is however disabled by default for security reasons, as multi-statement queries significantly increase the risks associated with eventual SQL injections.
To turn on multi-statement queries support, simply add the allowMultiQueries=true option to your connection string (or pass the equivalent option in map format). You can get more information on that option here: https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html.
Once this option enabled, you can simply execute a call similar to: statement.execute("select ... ; select ... ; select ..."). Returned ResultSets can be iterated from the Statement object: each call to statement.getResultSet() return the next one. Use statement.getMoreResults() to determine if there are indeed more ResultSet available.
It sounds like you want to do batch processing.
here is a duplicate question with an good answer:
How to execute multiple SQL statements from java

Hibernate produce different SQL for every query

I've just tested my application under the profiler and found out that sql strings use about 30% of my memory! This is bizarre.
There are a lot of strings like this stored in app memory. This is SQL queries generated by hibernate, note the different numbers and trailing underscores:
select avatardata0_.Id as Id4305_0_,...... where avatardata0_.Id=? for update
select avatardata0_.Id as Id4347_0_,...... where avatardata0_.Id=? for update
Here is the part I can't understand. Why does hibernate have to generate different sql strings with different identifiers like "Id4305_0_" for each query? Why can't it use one query string for all identical queries? Is this some kind of trick to bypass query caching?
I would greatly appreciate if someone would describe me why it happening and how to avoid such resource wasting.
UPDATE
Ok. I found it. I was wrong assuming memory leak, It was my fault. Hibernate is working as intended.
My app created 121(!) SessionFactories in 10 threads, they produced about 2300 instances of SingleTableEntityPersisters. And each SingleTableEntityPersister generates about 15 SQL queries with different identifiers. Hibernate was forced to generate about 345.000 different SQL queries. Everything is fine, nothing weird :)
There is a logic behind the query string that hibernate generates. Its primary aim is to get unique aliases for tables and columns names.
From your query,
select avatardata0_.Id as Id4305_0_,...... where avatardata0_.Id=?
avatardata0_ ==> avatardata is the alias of the table and 0_ is appended to indicate it is the first table in the query. So if it were the second table(or Entity) in the query it should have been shown as avatardata1_. It uses the same logic for the column aliases.
So, this way all the possible conflicts are avoided.
You are seeing theses queries because you have turns on the show_sql flag the configuration. This is intended for the debugging of queries. Once you application started working you are supposed turn it off.
Read more on the API docs here.
I am not much aware of the memory consumption part, but you repeat your tests with the above flag turned off and see if there is any improvement.
Assuming you are using sql server, you might want to check the parameter type declaration for '?', making sure the declaration results in the same, fixed length declaration every time.
Dynamic length parameters would result in separate execution plans for each query. This could possibly comsume a lot of resources. What we see as the same procedure, get's interpreted by sql server as a different query, rendering a separate execution plan.
Thus,
exec myprocedure #p1 varchar(3)='foo'
and
exec myprocedure #p1 varchar(6)='foobar'
would result in different plans. Simply by the fact that the declarations of #p1, differ in size.
There is a lot to know about this behaviour. If the above applies to you, I would recommend you read up on 'parameter sniffing'.
No... you can generate you common query inside the hibernate. The logic behind is to mapping with table and fetch the record from there. It is used common query for all the database. Please create a common query like that :
Example :
select t.Id as Id4305_0_,...... from t where t.Id=?

which is faster? Statement or PreparedStatement

Often, in the network can be found code like this:
private static final String SQL = "SELECT * FROM table_name";
....
and for this SQL query is used PreparedStatement. Why?
As i know, PreparedStatement spend time to precompile SQL statement. It turns out so that the Statement is faster than a PreparedStatement. Or I'm mistaken?
Prepared statements are much faster when you have to run the same statement multiple times, with different data. Thats because SQL will validate the query only once, whereas if you just use a statement it will validate the query each time.
The other benefit of using PreparedStatements is to avoid causing a SQL injection vulnerability - though in your case your query is so simple you haven't encountered that.
For your query, the difference between running a prepared statement vs a statement is probably negligible.
EDIT: In response to your comment below, you will need to look closely at the DAO class to see what it is doing. If for example, each time the method is called it re-creates the prepared statement then you will lose any benefit of using prepared statements.
What you want to achieve, is the encapsulation of your persistence layer so that their is no specific call to MySQL or Postgres or whatever you are using, and at the same time take advantage of the performance and security benefits of things like prepared statements. To do this you need to rely on Java's own objects such as PreparedStatement,.
I personally would build my own DAO class for doing CRUD operations, using Hibernate underneath and the Java Persistence API to encapsulate it all, and that should use prepared statements for the security benefits. If you have a specific use-case for doing repeated operations, then I would be inclined to wrap that within its own object.
Hibernate can be configured to use whatever database vendor you are using via an XML file, and thus it provides really neat encapsulation of your persistence layer. However, it is quite a complicated product to get right!
Most of the time queries are not as simple as your example. If there is any variation to the query, i.e. any parameters that are not known at compile time, you must use PreparedStatement to avoid SQL injection vulnerabilities. This trumps any performance concerns.
If there is any difference between PreparedStatement and Statement, it would be highly dependent on the particular JDBC driver in question and most of the time the penalty will be negligible compare to the cost of going to the database, executing actual query and fetching results back.
As Per the My knowledge PreparedStatement is much faster then statement. Here some reason why preparedstatement is faster then statement please read for more detail.
JDBC API is provide the functionality of connectivity with database. Then we try to execute the query with the use of statement and preparedstatement.
There are four step to execute the query.
Parsing of sql query.
Compile this Query.
optimization of data acquisition path.
execute the query.
Statement interface is suitable when we will not need to execute the query multiple time.
Disadvantages of Statement Interface.
hacker can easily to hack the data. Like suppose we have one query which have the username and password is a parameters you can give the proper parameters is username='abc#example.com' and password ='abc123' actually this is current But hacker can do username='abc#example.com' or '1'=1 and password='' that means you can logged successfully. so that is happening possible in Statement.
And sql validate every time when we fetch the data from database.
So Java has the solution for this above problem that is PreparedStatement.
This interface has many advantages. the main advantages of preparedstatement is sql is not validate the query every time. so you can get the result fast. please read the below more advantages of preparedstatement.
1) We can safely provide the value of query's parameters with setter method.
2) it prevent the SQL injection because it is automatically escapes the special characters.
3) When we use statement above four steps are execute every time But when we use the PreparedStatement only last steps is execute so this is faster then statement.
Faster is not the consideration here. Parsing of the sql will generally be a tiny part of overall execution. See more at When should we use a PreparedStatement instead of a Statement?

How to reconstruct sql query from a ResultSet

Somewhere way inside JBoss in a hibernate query I'm catching an error that leaves me with a ResultSet. This code is a plugged in custom data type.
It would be nice if I could simple do rs.getStatement().toString() and be done with it, but that unfortunately doesn't give away anything about the sql statement that went into it.
I was thinking doing something with ((PreparedStatement)rs.getStatement()).getMetaData().
I really wished hibernate would be a little more informative when it runs into errors.
Does anyone have a good solution to help reveal which table and which column that was used when the exception occurred?
Simply enable SQL logging in the Hibernate configuration properties by setting the hibernate.show_sql property to true.
This more reliable than examining the result sets metadata since the where clause is not available.
One way you can debug Hibernate is by turning on its detailed logging.
For example, you can log all SQL statements as they are executed by turning on logging for org.hibernate.SQL. From here you should be able to narrow down the last statement executed prior to your exception.
Documentation can be found here.
getting the MetaData for the ResultSet will not allow you to get the info that was passed in. In Hibernate you can have the statements be output to a log file.
Most JDBC drivers allow you to set tracing so that you can debug.

SQL Injection in Java and MySQL when using multiple queries

I've got a web application with an SQL injection as part of an INSERT statement. It looks like this:
INSERT INTO table1 VALUES ('str1', 1, 'INJECTION HERE')
I can insert the regular multiple-query injections such as ');truncate table1;-- but due to the fact that Java + MySQL is used it does not allow stacking multiple queries so the above injection would result in an error from MySQL and the second query never gets executed.
So basically it seems that all one can achieve from such an injection in the aforementioned architecture is injecting "junk data", which is possible without an injection as well.
There are more techniques such as using load_file() but that would still not allow me to manipulate the database to the extent I'm looking for.
Am I missing something here? Is there some other way to use this injection for gaining control over the database?
Of course, if you change your database/driver combination from your current implementation to something supporting multiple requests, then you'll activate a dormant security hole that (no doubt) people will have forgotten about!
Ignoring the nefarious, malicious scenarios, the above will cause you problems with inserting regular data, including quote characters etc. i.e. the above simply won't work for particular sets of data (unless cleansed/esaped etc.). I would correct it simply for functionality purposes.
You should have a look at PreparedStatement, and the data insertion methods for this (setString()) etc.
e.g.:
PreparedStatement pstmt = con.prepareStatement("UPDATE EMPLOYEES
SET SALARY = ? WHERE ID = ?");
pstmt.setBigDecimal(1, 153833.00)
pstmt.setString(2, "Insert what you like here")
The setString() method will support any string without escaping/injection issues.
SQL injection doesn't have to delete something from the database. The attacker might want to retrieve some valuable data that he's not supposed to have access to.
For example, consider the following post-injection form (I'm not familiar with MySQL syntax, but something like this should be possible in general - add casts as needed):
INSERT INTO table1 VALUES ('str1', 1,
-- injected stuff --
'' || (SELECT valuable_info FROM admin_only_table WHERE id=1) || ''
-- end injected stuff --
))
Now table1 - which can be, say, where some publicly accessible info is retrieved from, so anyone can see the values - contains a potentially sensitive value from a presumably secure table admin_only_table.
Of course, this assumes that your server doesn't do any tricks such as user impersonation or otherwise limits permissions on SQL level for the queries, but rather performs them all with full privileges.
As explained in this post, there are more bad things that can happen to your application than the classic table DROP:
call a sleep function so that all your database connections will be busy, therefore making your application unavailable
extracting sensitive data from the DB
bypassing the user authentication
Bottom line, you should never use string concatenation when building SQL statements. Use a dedicated API for that purpose.

Categories