I have a problem with loading objects from a SQLite database.
First of all, this is my table definition:
CREATE TABLE MyTable (
rowid INTEGER PRIMARY KEY,
data BLOB
);
This is the simple class which I want to store and reload:
public class MyHashMap extends HashMap<String, Integer> {
private static final long serialVersionUID = 0L;
}
Then I'm filling the map with some data and store it with an SQL INSERT statement in the database. Everything works fine and if I execute a SELECT (with the sqlite3 command-line client) I will see the correct information.
Now I'm using the java.sql package to load the object:
String sql = "SELECT data FROM MyTable WHERE rowid = 1";
MyHashMap map = null;
try {
try (Statement stmt = db.createStatement()) {
try (ResultSet rs = stmt.executeQuery(sql)) {
if (rs.next()) {
map = rs.getObject("data", MyHashMap.class);
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
There's no exception thrown but my map variable is null. I debugged the program and I can say that the getObject method is called as expected.
First, you definition of MyHashMap is incorrect
public class MyHashMap extends HashMap<Integer, String> {
private static final long serialVersionUID = 0L;
}
The main issue, though, is that SQL doesn't store Java objects; it merely stores rows of records, which consist of fields. You need to read these records one by one, and store them in your map. Roughly as follows:
MyHashMap map = new MyHashMap();
final String sql = "SELECT rowid, data FROM MyTable";
try (final Statement stmt = connection.createStatement;
final ResultSet rs = stmt.executeQuery(sql)) {
while (rs.next()) {
map.put(rs.getInt(1), rs.getString(2));
}
}
Please note that there's a good chance that reading a Blob into a String will fail. Usually, JDBC drivers are clever enough to convert data types, but if you have raw binary data in your blob, you cannot read it into a string. You would need the getBlob method instead, and deal with the resulting object. But I can't tell from your code what you'll be storing in that blob.
Ok, I found a solution with the following method:
private Object getObjectFromBlob(ResultSet rs, String columnName)
throws ClassNotFoundException, IOException, SQLException {
InputStream binaryInput = rs.getBinaryStream(columnName);
if (binaryInput == null || binaryInput.available() == 0) {
return null;
}
ObjectInputStream in = new ObjectInputStream(binaryInput);
try {
return in.readObject();
} finally {
in.close();
}
}
Related
I'm a newbie in Java and I'm trying to find a way to retrieve a key from Hashtable. The code below was written to retrieve values from in a database and insert in Hashtable in key/value pair format using a stored procedure.
private Hashtable loadRspCodeMappingFromDB(String node_name)
throws SQLException
{
//TO DO:
// Take a look at the SDK user guide -> Programming conventions -> Database access using JDBC.
// Use the stored procedure you've created to fill a Hashtable to return.
// Two events have been created in events.er for your use:
// RspCodeMappingLoaded, and
// ErrorReadingRspCodeMapping
Hashtable resp_mapping_table = new Hashtable();
Connection cn = null;
CallableStatement stmt = null;
ResultSet rs = null;
try
{
cn = JdbcManager.getDefaultConnection();
stmt = cn.prepareCall("{call samplenid_get_rsp_map(?)}");
stmt.setString(1, node_name);
rs = stmt.executeQuery();
while(rs.next()){
tm_rsp_code = rs.getString(1);
Interchange_rsp_code = rs.getString(2);
resp_mapping_table.put(tm_rsp_code, Interchange_rsp_code);
}
JdbcManager.commit(cn, stmt, rs);
}
finally
{
JdbcManager.cleanup(cn, stmt, rs);
}
return resp_mapping_table;
}
The second code block is written to retrieve the mapped key from a Hashtable and include in a transaction message, but unfornately I'm getting this error: "cannot convert from element type Object to Map.Entry.
public Action processMsgFromRemote(AIntegrationDriverEnvironment env,Iso8583Post msg)
throws XPostilion, SQLException
{
//TO DO:
// 1. Check if the message is a response message
// 2.1 Check the integration driver environment to discover the sink node name
// 2.2 If the rsp code mapping has not yet been loaded, load rsp code
// mapping from table now. Use the loadRspCodeMappingFromDB() method
// below.
// 3.1 Check if a mapping exists
// 3.2 If a mapping exists, set the value to transaction manager
// 4. Return the action with the message to Transaction Manager
Action action = new Action();
boolean check_msg_type = Iso8583Post.MsgType.isResponse(msg.getMsgType());
if(check_msg_type){
String node_name = env.getSinkNodeName();
if( rsp_map == null)
{
rsp_map = loadRspCodeMappingFromDB(node_name);
}
else{
String rsp_code_from_cache_loader = (String) rsp_map.get(tm_rsp_code);
String rsp_code_msg_from_interchange = msg.getField(Iso8583.Bit._039_RSP_CODE);
if(rsp_code_from_cache_loader.equals(rsp_code_msg_from_interchange)){
for(Map.Entry entry: rsp_map.entrySet()){
if(rsp_code_from_cache_loader.equals(entry.getValue())){
tm_rsp_code = (String) entry.getKey();
break;
}
}
}
}
}
action.putMsgToTranmgr(msg);
return action;
}
I would like to know what would be the solution to this error. How should I fix it ?
So, I was using a datasource which was defined in Spring, which was working fine. Then I updated my project to take the datasource from the Weblogic server which the application is running on. This too, works fine for most calls to the database, except for one scenario - This scenario is involved sending a list of objects to the database, based on database types which are defined in Java by using Structs.
The full method is:
#Override
public List<String> saveAllocation(String originalId, List<Parcel> parcels) throws SQLException {
if(originalId == null || parcels == null) {
return null;
}
List<String> results = null;
String result = null;
String log = null;
OracleConnection oracleConnection = (OracleConnection)jdbcTemplate.getDataSource().getConnection();
try {
OracleCallableStatement cs = (OracleCallableStatement) oracleConnection.prepareCall("{ call PACKAGE.Update(?, ?, ?, ?) }");
Struct[] cpoList = new Struct[parcels.size()];
for(int i = 0; i < parcels.size(); i++) {
Object[] obj = new Object[] { parcels.get(i).getParcel_id(), parcels.get(i).getPublicID().toUpperCase() };
Struct struct = oracleConnection.createStruct("SCHEME_NAME.PARCEL_OBJ", obj);
cpoList[i] = struct;
}
Array array = oracleConnection.createARRAY("SCHEME_NAME.PARCEL_TAB", cpoList);
cs.setString(1, originalId);
cs.setArray(2, array);
cs.registerOutParameter(3, Types.VARCHAR);
cs.registerOutParameter(4, Types.VARCHAR);
cs.executeUpdate();
log = cs.getObject(3).toString();
result = cs.getObject(4).toString();
results = new ArrayList<>();
results.add(result);
results.add(log);
} catch(SQLException e) {
//Log exception
return results;
} catch(Exception e) {
//Log exception
return results;
} finally {
if (cs != null) {
cs.close();
}
}
return results;
}
}
The database objects are defined as:
PARCEL_OBJ
create or replace TYPE parcel_obj AS OBJECT
(PARCEL_ID VARCHAR2(11),
PUBLIC_ID VARCHAR2(20));
PARCEL_TAB
create or replace TYPE parcel_tab IS TABLE OF parcel_obj;
The application fails on the line
Array array = oracleConnection.createARRAY("SCHEME_NAME.PARCEL_TAB", cpoList);
The exception message is:
java.sql.SQLException: Fail to convert to internal representation: weblogic.jdbc.wrapper.Struct_oracle_sql_STRUCT#187>
My JNDI connection is defined in my application.properties like:
spring.datasource.jndi-name=jdbc/pio
Any help would be appreciated!
As the documentation mentions.
By default, data type objects for Array, Blob, Clob, NClob, Ref,
SQLXML, and Struct, plus ParameterMetaData and ResultSetMetaData
objects are wrapped with a WebLogic wrapper.
In some cases setting the wrapping parameter to false can improve significantly the performance and allows the application to use a native driver.
I don't see a problem disabling that option since it is causing the problem when calling objects like struct in the first place.
I have the below class with 3 resultsets.
public class AllMetricsResultSet {
ResultSet top10Files;
ResultSet top10FilesForUsage;
ResultSet top10DataSet;
}
In another method, I have 3 different select statements(I've given only one select below, but there are 3) which assign result sets into the above.
public AllMetricsResultSet SPDataList (String alldata)
{
...........
String sSQL = "SELECT USERNAME, NUMBEROFFILES FROM FILE_INFO";
PreparedStatement stm = dbConnection.prepareStatement(sSQL);
if (stm.execute())
//TopTenantsForUsage = stm.getResultSet();
rs.top10Files = stm.getResultSet();
rs.top10FilesForUsage = stm.getResultSet();
rs.top10DataSet = stm.getResultSet()
Then finally from another method, I am calling the previous method as follows:
AllMetricsResultSet mrs = SPDataList(alldata);
while (mrs.top10Files.next())
(This while statement fails. I see that there are 10 rows returned. I tried mrs.top10Files.getFetchSize() (this also failed)
Any help would be greatly appreciated.
It's not a good practice to execute the queries in one method and read data from the result sets in a different method.
You want to finish your DB access as quick as possible and close the connection, in order to return the connection to the connection pool (relevant when you have multiple threads accessing the db) and release any db locks your statements may require.
Therefore the result set variables shouldn't br instance variables at all. You should create and consume them in the same method.
You could have, though, a separate method for each of your 3 queries, and each of them can return the data it fetched.
Instead try like this
public AllMetricsResultSet SPDataList (String alldata) {
String sSQL1 = "query1";
String sSQL2 = "query2";
String sSQL3 = "query3";
try {
PreparedStatement stm1 = dbConnection.prepareStatement(sSQL1);
PreparedStatement stm2 = dbConnection.prepareStatement(sSQL2);
PreparedStatement stm3 = dbConnection.prepareStatement(sSQL3);
if (stm1.execute())
top10Files = stm1.getResultSet();
if (stm2.execute())
top10FilesForUsage = stm2.getResultSet();
if (stm3.execute())
top10DataSet = stm3.getResultSet();
while (top10Files.next()) {
//get the resultset1 data
}
while (top10FilesForUsage.next()) {
//get the resultset2 data
}
while (top10DataSet.next()) {
//get the resultset3 data
}
// dont know why as you want to return the classType
//create an object of the class initialize it the the data you obtained
//and return it
}
catch(SQLException e)
{
e.printStackTrace();
}
finally {
//resultsetobject close
//statementObject close
//connection object close
}
}
The db tables that I am using are changing very often meaning new column can be add which will reflect my sql’s.
The solution that I was thinking is first “read” the meta data into some map and use it in order to retrieve the values something like this.
Read meta data:
public class Dynamic {
static public final Map<Integer, String> metadata = initMetaData();
HashMap<String, String> data = new HashMap<String, String>();
private static Map<Integer, String> initMetaData() {
Map<Integer, String> tmpMap = new HashMap<Integer, String>();
try {
Connection connection = DBConnection.getConnection();
try {
Statement stmt = connection.createStatement();
ResultSet result = stmt.executeQuery("SELECT * FROM TMP WHERE ROWNUM = 1");
for (int i = 1; i <= result.getMetaData().getColumnCount(); i++) {
tmpMap.put(new Integer(i), result.getMetaData().getColumnName(i));
}
} finally {
connection.close();
}
} catch (SQLException ex) {
…..
}
return Collections.unmodifiableMap(tmpMap);
}
public static String getColumnName(Integer index) {
return metadata.get(index);
}
And when running the sql:
public static void test()
try {
Connection connection = DBConnection.getConnection()
try {
Statement stmt = connection.createStatement();
ResultSet result = stmt.executeQuery("SELECT * FROM TMP where idx = 'R5'");
while (result.next()) {
Dynamic d = new Dynamic()
for (int i = 1; i <= Dynamic.metadata.size(); i++) {
d.setData(Dynamic.getColumnName(i),result.getString(Dynamic.getColumnName(i)));
}
}
In this approach I have two problems(that I notice):
1) I need to execute two loops
2) I don’t know which get function to use via resultset since the type can also change.
How can I overcome those problems ?
I would also appreciate to get some other suggestion maybe there is a simple why
Thanks
1) What is your alternative to the inner loop? How would you get the field values? Do you think there is a lot of overhead in looping over small number of integers?
2) You can get extra information about field data type from the same metadata where you get the field name and map the method names accordingly.
3) You should really create a map for multiple tables - table/fieldSeq/fieldType/methodName and maybe few extra details - you don't have to get them all dynamically all the time.
I'm trying to write and update a pdf document in a blob column but I'm just able to update the blob only writing more data than the previous stored data.
If I try to update the blob column with a smaller document data I get only a corrupted pdf.
First the blob column has been initialized using empty_blob() function. I wrote the sample Java class below to test this behaviour. I run it the first time with 'true' as first parameter of the main method so in the first row there's stored a document of about 31kB and in the second row there's a document of 278kB.
Then I run it with 'false' as parameter, in this way the two rows should be updated swapping the documents. The result is that I get a correct result only when I write more data than the existing one.
How is it possible to write a method that writes and updates a blob in a reliable way without worring about binary data's size?
import static org.apache.commons.io.IOUtils.copy;
import java.io.FileInputStream;
import java.io.OutputStream;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import oracle.jdbc.OracleDriver;
import oracle.jdbc.OracleResultSet;
import oracle.sql.BLOB;
import org.apache.commons.lang.ArrayUtils;
/**
* Prerequisites:
* 1) a table named 'x' must exists [create table x (i number, j blob);]
* 2) that table should have two columns [insert into x (i, j) values (1, empty_blob()); insert into x (i, j) values (2, empty_blob()); commit;]
* 3) download lsp.pdf from http://www.objectmentor.com/resources/articles/lsp.pdf
* 4) download dotguide.pdf from http://www.graphviz.org/Documentation/dotguide.pdf
*/
public class UpdateBlob {
public static void main(String[] args) throws Exception {
processFiles(new String[]{"lsp.pdf", "dotguide.pdf"}, Boolean.valueOf(args[0]));
}
public static void processFiles(String [] fileNames, boolean forward) throws Exception {
if(!forward){
ArrayUtils.reverse(a);
}
int idx = 1;
for(String fname : fileNames){
insert(idx++, fname);
}
}
private static void insert(int idx, String fname) throws Exception{
Connection conn = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
DriverManager.registerDriver(new OracleDriver());
conn = DriverManager.getConnection("jdbc:oracle:thin:#"+db+":"+port+":"+sid, user, pwd);
ps = conn.prepareStatement("select j from x where i = ? for update");
ps.setLong(1, idx);
rs = ps.executeQuery();
if (rs.next()) {
FileInputStream instream = new FileInputStream(fname);
BLOB blob = ((OracleResultSet)rs).getBLOB(1);
OutputStream outstream = blob.setBinaryStream(1L);
copy(instream, outstream);
instream.close();
outstream.close();
}
rs.close();
ps.close();
conn.close();
} catch (SQLException e) {
e.printStackTrace();
throw new Exception(e);
}
}
}
Oracle version: 11.1.0.7.0 - 64bit
I even tried the standard JDBC API without using Oracle's specific one (like in the example above) without any success.
It's a lot easier:
PreparedStatement pstmt =
conn.prepareStatement("update blob_table set blob = ? where id = ?");
File blob = new File("/path/to/picture.png");
FileInputStream in = new FileInputStream(blob);
// the cast to int is necessary because with JDBC 4 there is
// also a version of this method with a (int, long)
// but that is not implemented by Oracle
pstmt.setBinaryStream(1, in, (int)blob.length());
pstmt.setInt(2, 42); // set the PK value
pstmt.executeUpdate();
conn.commit();
pstmt.close();
It works the same when using an INSERT statement. No need for empty_blob() and a second update statement.
In addition to a_horse_with_no_name's answer (which relies on PreparedStatement.setBinaryStream(...) API), there're at least two more options for BLOBs, and 3 more for CLOBs and NCLOBs:
Explicitly create a LOB, write to it, and use PreparedStatement.setBlob(int, Blob):
int insertBlobViaSetBlob(final Connection conn, final String tableName, final int id, final byte value[])
throws SQLException, IOException {
try (final PreparedStatement pstmt = conn.prepareStatement(String.format("INSERT INTO %s (ID, VALUE) VALUES (?, ?)", tableName))) {
final Blob blob = conn.createBlob();
try (final OutputStream out = new BufferedOutputStream(blob.setBinaryStream(1L))) {
out.write(value);
}
pstmt.setInt(1, id);
pstmt.setBlob(2, blob);
return pstmt.executeUpdate();
}
}
Update an empty LOB (inserted via DBMS_LOB.EMPTY_BLOB() or DBMS_LOB.EMPTY_CLOB()) via SELECT ... FOR UPDATE. This is Oracle-specific and requires two statements executed instead of one. Additionally, this is what you were trying to accomplish in the first place:
void insertBlobViaSelectForUpdate(final Connection conn, final String tableName, final int id, final byte value[])
throws SQLException, IOException {
try (final PreparedStatement pstmt = conn.prepareStatement(String.format("INSERT INTO %s (ID, VALUE) VALUES (?, EMPTY_BLOB())", tableName))) {
pstmt.setInt(1, id);
pstmt.executeUpdate();
}
try (final PreparedStatement pstmt = conn.prepareStatement(String.format("SELECT VALUE FROM %s WHERE ID = ? FOR UPDATE", tableName))) {
pstmt.setInt(1, id);
try (final ResultSet rset = pstmt.executeQuery()) {
while (rset.next()) {
final Blob blob = rset.getBlob(1);
try (final OutputStream out = new BufferedOutputStream(blob.setBinaryStream(1L))) {
out.write(value);
}
}
}
}
}
For CLOBs and NCLOBs, you can additionally use PreparedStatement.setString() and setNString(), respectively.
FWIW, for something that fits in memory, I found I could simply pass in a byte array as the prepared statement parameter, rather than going through the "stream" rigor morale (or worse Oracle specific/suggested things)
Using a Spring "JDBC template" wrapper (org.springframework.jdbc.core.JdbcTemplate) to put the contents of a "large" (or not) string into a BLOB column, the code is something like the following:
jdbc.update( "insert into a_table ( clob_col ) values ( ? )", largeStr.getBytes() );
There is no step 2.