Oracle provides a statement compiler hint, NO_COPY, that can be used to improve performance of SQL statements, particularly those that involve large data structures.
My problem is that, try as I might, I can't find any reference on how to use this feature through the Java API. All I can find is this javadoc for Oracle Fusion which, as I understand, is a separate middleware package that can be applied over the database layer (presumably at great expense), which is not what I need.
EDIT
I am fairly new to database programming and it is quite possible that I have missed something important here!
I am using CallableStatements to call a prepared statement already written in the database. Since I have to tell the Java API the type of each returned parameter with registerOutParameter(), I had assumed that I would also need to tell the Java API to use NO-COPY (ie if the JDBC driver can't work out the types from the procedure definition in the database, I figured I would need to tell it to use NO_COPY as well).
Have I got this all wrong?
NOCOPY is used when you declare and define a PL/SQL subprogram. You don't need to specify it when you call the procedure or function -- much like you don't specify the type or mode (IN/OUT) in a call.
Related
I have a Groovy script that is using the groovy.sql.Sql class to communicate with the DB, but some legacy Java code is expecting to receive a java.sql.ResultSet. Is there any way to execute a SELECT query using groovy.sql.Sql and directly return the java.sql.ResultSet?
Not really. One of the main reasons to use Groovy's Sql is to automatically handle resources (which is why there are methods using closures to interact with the ResultSet, but none that return it -- to ensure they get closed). I'd strongly recommend revisiting why you are passing around references to ResultSet (which could be holding onto server resources) rather than using a POJO/POGO representing those results.
However, if you insist on using ResultSet, and you only need few convenience methods, I guess one option would be to copy those methods into your codebase. As you can see from the source, Sql is a pretty thin wrapper over plain old JDBC. But for what it sounds like you're doing, I question how much value it adds.
The project I'm working on has a REST API written in JRuby/Java, with an endpoint that hits a MySQL database to retrieve a number of records.
We need to allow the client to filter those records using one or more columns, including boolean checks and range values.
The easiest way we can do this is to add a string parameter to the API, then add it into the SQL statement.
Collectively, the development team agree that this is a bad idea but the alternative is to provide an almost identical syntax for filtering, which is translated into SQL. The allure of the SQL injection parameter is strong.
So my question is, are there any circumstances under which this is a safe thing to do?
In particular, might we consider using the WHERE clause safely if it's been fully parsed beforehand, and identified as such. Or at the very least, checking for certain trigger words such as DROP, SELECT etc.
Also if anyone knows of a good library that could act as a go-between (translating or parsing an external expression into a WHERE clause) that would be great.
The OData and GData protocols already implement this functionality in a safe and standard way. You can find server and client implementations for both, for Ruby, PHP, MySQL etc. Check here for the OData libraries
Leaving aside the SQL injection issue, you'll expose your inner implementation (both the database chosen - MySQL and your table structure) directly in the form of your API.
e.g. if you change to some NoSQL-type implementation at the backend, your public-facing API will break immediately. Similarly if you restructure your database. I wouldn't do this even in an environment in which I wasn't worried about the probability/severity of injection attacks.
Besides the security implications, allowing an arbitrary WHERE clause is a bad idea because it takes the "I" out of "API" -- it's not an interface. The API is supposed free the user of the need to know details of the implementation. Like table and column names.
If clients are interacting with your data by constructing their own WHERE clauses, then you can never change the database. There might be code out there with those statements programmed in. If a bug or new feature required you to alter the DB in a way that would break existing client interactions you'd be stuck. The API should provide the filtering capability and translate requests into calls to backend in a way that lets you change the backend without breaking the API.
There are numerous ORM's for this purpose, especially in ruby (activerecord, sequel)
The most basic thing you need to do is escape the string input, which will pretty much prevent sequel injection if you are doing it properly.
It helps to not directly insert parameters directly into the sequel statement if you dont have to either, instead, check their validity and then map them to logical ones (this isn't always possible). For example, if there is an html dropdown list, and when you submit the form it passes some parameter 'firstitem', map 'firstitem' to an id or otherwise that you will then use to search on, versus using the user supplied version (assuming this mapping doesn't involve the db).
Please note that I've searched and searched but I am not able to find an answer.
I'm writing a Java application and I need to support both Oracle and Access databases. I currently have two classes that implement their own version of Connect().
OracleDatabase : IDatabase
AccessDatabase : IDatabase
Let's say that I need to create a table called "MY_TABLE". How would I go about supporting both database types? I thought about something like...
IDatabase.Execute(IDatabase.GetCreateMyTableQuery());
I'm using the factory pattern to select the database implementation. The problem is that I don't know how to store the queries.
I have done something similar: supporting Oracle and HSQLDB for unit tests.
I used a parent class with an abstract protected function modifySql(String sqlText). The interface would call the this function before executing a command or registering a preparedStatement, and the descendant HSQLDB class would change the SQL text accordingly (doing things like changing NUMBER to NUMERIC etc..) before calling standard JDBC functions. The Oracle class would obviously do nothing in the function.
I standardized on the application using Oracle SQL, then changed it for the other types (have also started a MySQL class). The modifySQL function can get a bit lengthy, and you end up having to swap text round to handle sequences etc., but you soon end up with something workable.
Apart from the different JDBC driver issues, this is all I needed to do to treat the databases in exactly the same way in the main application. In this way you don't have to store lots of different query formats for the different database types: and it's obviously viable to have them simply hard-coded into the application.
You could store it like final string variables in your code, or in xml files, like this: http://nicodewet.com/2009/12/29/clean-code-store-native-sql-statements-in-external-file/ (or using some libraries).
We do this at work. We have our own custom DAL and support Access, SQL, and Oracle. We use multiple forms of queries but the focus for your question is we store them in XML Files. Depending on how you write the queries will determine how difficult it is for you. Below is an example of how you can store queries to be retrieved.
<Command ID = "3">
<Provider = "Default">
<QueryText>
Select * from MyTable
</QueryText>
</Provider>
<Provider = "Oracle">
<QueryText>
Select * from dual
</QueryText>
</Provider>
<Command>
Then when you go to run your query in the application - you call the text based on your provider. If you need a custom query for the provider - your code should be able to pull the correct node.
The easiest way to do this is to implement it using the IDB generic interfaces as then cast as needed.
Here is a good explanation of how this can be achieved.
http://stevencalise.wordpress.com/2009/10/16/constructing-a-dal-in-c/
All this being said it really depends on how much control over your DAL you need. In some situations an ORM tool would work better - but you need to confirm data support. You also have to be careful of the data providers when it comes to bit level (32 versus 64 bit) when working with Access and / or Oracle. SQL is much more forgiving.
We're going to write a new web interface for a big system based on Oracle database. All business rules are already coded in PL/SQL stored procedures and we'd like to reuse as much code as possible. We'll write some new stored procedures that will combine the existing business rules and return the final result dataset.
We want to do this on the database level to avoid java-db round trips. The interface layer will be written in Java (we'd like to use GWT), so we need a way of passing data from Oracle stored procedures to Java service side. The data can be e.g. a set of properties of a specific item or a list of items fulfilling certain criteria.
Would anyone recommend a preferable way of doing this?
We're considering one of the 2 following scenarios:
passing objects and lists of objects (DB object types defined on the
schema level)
passing a sys_refcursor
We verified that both approaches are "doable", the question is more about design decision, best practice, possible maintenance problems, flexibility, etc.
I'd appreciate any hints.
I would recommend sticking with a refcursor with well defined keys (agreed on both sides by java devs and pl/sql developers). This is much easier to extend in the future, you can easily convert the refcursor to hashmap and then a hashmap to a POJO using a apache bean utils if needed. I'm working on a big telecom project with many approaches to this issue and refcursor seems to be the best at the end of the day.
In the past I have achieved exactly the same with classic JDBC CallableStatement without any perfomance or maintenance issues. With ORM solutions like Hibernate making persistence much more flexible, you can wrap your solution around Hibernate as achieve in this post. Also see this example if you are not already familiar with the way store procedure and CallableStatement works.
It's been a while since I've done something like that, but the way I remember is that you need to define a view that calls your stored procedure, and you can then easily read the result sets from within java, with the OR-mapper of your choice.
So, this seems close to your scenario 1, which never caused any problems in my experience.
The one thing one needs to be careful is transaction handling: If your stored procedures write data, and you call several of them within a Java EE transaction, you might get into a situation of data inconsistency.
I thought I knew everything about UDTs and JDBC until someone on SO pointed out some details of the Javadoc of java.sql.SQLInput and java.sql.SQLData JavaDoc to me. The essence of that hint was (from SQLInput):
An input stream that contains a stream
of values representing an instance of
an SQL structured type or an SQL
distinct type. This interface, used
only for custom mapping, is used by
the driver behind the scenes, and a
programmer never directly invokes
SQLInput methods.
This is quite the opposite of what I am used to do (which is also used and stable in productive systems, when used with the Oracle JDBC driver): Implement SQLData and provide this implementation in a custom mapping to
ResultSet.getObject(int index, Map mapping)
The JDBC driver will then call-back on my custom type using the
SQLData.readSQL(SQLInput stream, String typeName)
method. I implement this method and read each field from the SQLInput stream. In the end, getObject() will return a correctly initialised instance of my SQLData implementation holding all data from the UDT.
To me, this seems like the perfect way to implement such a custom mapping. Good reasons for going this way:
I can use the standard API, instead of using vendor-specific classes such as oracle.sql.STRUCT, etc.
I can generate source code from my UDTs, with appropriate getters/setters and other properties
My questions:
What do you think about my approach, implementing SQLData? Is it viable, even if the Javadoc states otherwise?
What other ways of reading UDT's in Java do you know of? E.g. what does Spring do? what does Hibernate do? What does JPA do? What do you do?
Addendum:
UDT support and integration with stored procedures is one of the major features of jOOQ. jOOQ aims at hiding the more complex "JDBC facts" from client code, without hiding the underlying database architecture. If you have similar questions like the above, jOOQ might provide an answer to you.
The advantage of configuring the driver so that it works behind the scenes is that the programmer does not need to pass the type map into ResultSet.getObject(...) and therefore has one less detail to remember (most of the time). The driver can also be configured at runtime using properties to define the mappings, so the application code can be kept independent of the details of the SQL type to object mappings. If the application could support several different databases, this allows different mappings to be supported for each database.
Your method is viable, its main characteristic is that the application code uses explicit type mappings.
In the behind the scenes approach the ResultSet.getObject(int) method will use the type mappings defined on the connection rather than those passed by the application code in ResultSet.getObject(int index, Map mapping). Otherwise the approaches are the same.
Other Approaches
I have seen another approach used with JBoss 4 based on these classes:
org.jboss.ejb.plugins.cmp.jdbc.JDBCParameterSetter
org.jboss.ejb.plugins.cmp.jdbc.JDBCResultSetReader.AbstractResultSetReader
The idea is the same but the implementation is non-standard (it probably pre-dates the version of the JDBC standard defining SQLData/SQLInput).
What other ways of reading UDT's in Java do you know of? E.g. what does Spring do? what does Hibernate do? What does JPA do? What do you do?
An example of how something similar to this can be done in Hibernate/JPA is shown in this answer to another question:
Java Enums, JPA and Postgres enums - How do I make them work together?
I know what Spring does: you write implementations of their RowMapper interface. I've never used SQLData with Spring. Your post was the first time I'd ever heard of or thought about that interface.