Database encoding change to UTF8 causing java.lang.NullPointerException - java

I'm running a Tomcat WAR, which uses a MySQL database.
The application will run in foreign languages, so I had to change all database character parameters to utf8.
One application string (appPrefix) has to be empty (because the WAR is deployed in the root dir). This worked well, until I created a new database in UTF8 and migrated all the tables.
Now I get a NullPointerException because of the appPrefix being empty:
java.lang.NullPointerException
com.horizon.servlet.PageServlet.doMainPageRequest(PageServlet.java:177)
com.horizon.servlet.PageServlet.doRequest(PageServlet.java:53)
com.horizon.servlet.PageServlet.doGet(PageServlet.java:33)
javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
com.horizon.filters.P3PFilter.doFilter(P3PFilter.java:19)
The above is all the same error causing ripples throughout the application.
It's all caused by appPrefix being empty, but it should..
Should I specify it as empty in another way? Or should I try to hardcode my way around this?
EDIT:
As per the request in the comment below, here is PageServlet.java:177
request.setAttribute("appPrefix", appManager.getAppStringById(11).getValue());
This references AppManager.java:
public static final int APP_STRING_APPLICATION_PREFIX = 11;
which is populated by
public AppString getAppStringById(int id) {
AppString string = (AppString) stringCache.get(id);
if (string == null) {
String query = "SELECT * FROM app_strings WHERE id = ?";
List<Object> params = new LinkedList<Object>();
params.add(id);
string = execQueryLoadSingleRecord(query, params, new LoadAppString());
if (string != null) {
populateCache(stringCache, id, string);
}
}
return string;
}

As per
request.setAttribute("appPrefix", appManager.getAppStringById(11).getValue());
and
The database entry it's getting is empty, as it should.. So shouldn't it return null? Does this conflict with anything? The db string being empty was no problem until I changed the database's encoding to UTF8 from the default latin1 swedish!
I understand that it's not a problem at all if appManager.getAppStringById(11) can possibly return null, right? In that case, you should check for it before calling getValue() on it.
AppString appString = appManager.getAppStringById(11);
if (appString != null) {
request.setAttribute("appPrefix", appString.getValue());
}
As to why it returns null after you changed the table's charset; I have no idea. Perhaps it's just big coincidence or a misinterpretation of the problem. Perhaps you added getValue() call later on because you wanted to use ${appPrefix} instead of ${appPrefix.value} in EL or something. Or perhaps you rewrote execQueryLoadSingleRecord() that it returned null instead of empty string. Or perhaps the column's default value is null instead of an empty string. Or perhaps it's a bug in the JDBC driver used. Who knows. Using null as "no value" is perfectly fine and should be treated as such.

Related

Check if Hashtable.get worked

I'm working with on a Java program that checks a config file integrity.
On some point, I need to ensure that some mandatory values are setted up, either give it a default value.
So, I made a HashTable<String, String> working like a key => value table. On this table I store all the configuration lines on the file, and then I check it.
The problem comes when a specific value does not exist, for example:
String p = null;
/*...*/
//here I'm trying to get the LOG value
p = this.PAIRS.get("LOG");
if(p.equals(null) || p.equals("")){
//set default value
}
The problem is that I'm getting a NullPointerException, so it would be fine if someone can help me on how to determinate if this.PAIRS.get("LOG"); found a key or not...
Thanks!
EDIT: Solved, the right thing was using == and not an equals.
Thanks again!
If p is null, a NullPointerException will be thrown because it is not an instance of an Object (so the equals method doesn't exist). Checking for null should be done the following way : p == null
p = this.PAIRS.get("LOG");
This code return null if the key is not present and when you are doing below statement it will throw exception
if(p.equals(null) || p.equals("")){
//set default value
}
Instead check null first then do .equals
if(p==null || p.equals("")){
//set default value
}

Using JDBC to call a PL/SQL stored procedure with custom type input parameter, all fields are null

I'm using JDBC with createStruct() to call a stored procedure on an Oracle database that accepts a custom type as a parameter. The stored procedure inserts the custom type fields into a table and when I SELECT from the table later I see that all the fields that I tried to insert are NULL.
The custom type looks like this:
type record_rec as object (owner_id varchar2 (7),
target_id VARCHAR2 (8),
IP VARCHAR2 (15),
PREFIX varchar2 (7),
port varchar2 (4),
description VARCHAR2 (35),
cost_id varchar2(10))
The stored procedure looks like this:
package body "PKG_RECORDS"
IS
procedure P_ADD_RECORD (p_target_id in out VARCHAR2,
p_record_rec in record_rec)
is
l_target_id targets.target_id%TYPE;
BEGIN
Insert into targets (target_id,
owner_id,
IP,
description,
prefix,
start_date,
end_date,
cost_id,
port,
server_name,
server_code)
values (f_sequence ('TARGETS'),
p_record_rec.owner_id,
p_record_rec.ip,
p_record_rec.description,
p_record_rec.prefix,
sysdate,
to_date ('01-JAN-2050'),
p_record_rec.cost_id,
p_record_rec.port,
'test-server',
'51')
returning target_id
into p_target_id;
END;
END PKG_RECORDS;
My Java code looks something like this:
try (Connection con = m_dataSource.getConnection()) {
ArrayList<String> ids = new ArrayList<>();
CallableStatement call = con.prepareCall("{call PKG_RECORDS.P_ADD_RECORD(?,?)}");
for (Record r : records) {
call.registerOutParameter("p_target_id", Types.VARCHAR);
call.setObject("p_record_rec",
con.createStruct("SCHEME_ADM.RECORD_REC", new Object[] {
r.getTarget_id(),
null, // will be populated by SP
t.getIp(),
t.getPrefix(),
t.getPort(),
t.getDescription(),
t.getCost_id()
}), Types.STRUCT);
call.execute();
ids.add(call.getString("p_target_id"));
}
return new QueryRunner().query(con,
"SELECT * from TARGETS_V WHERE TARGET_ID IN ("+
ids.stream().map(s -> "?").collect(Collectors.joining(",")) +
")",
new BeanListHandler<Record>(Record.class),
ids.toArray(new Object[] {})
).stream()
.collect(Collectors.toList());
} catch (SQLException e) {
throw new DataAccessException(e.getMessage());
}
Notes:
* That last part is using Apache Commons db-utils - I love their bean stream operations.
* The connection is using C3P0 connection pool - could that be related?
* Just to make it clear - its not that the bean processor populates null values into the Record bean fields - if I use an SQL explorer to load the table (or view) directly, I can see that the fields in the database are indeed set to NULL.
There are no SQLExceptions when the process runs, or any other notice that something is wrong.
Any ideas what to check?
[Update]
After reading on Oracle Objects and SQLData mappings, I rewrote the code to use SQLData.
The Record class now implements SQLData and it's writeSQL() method looks like this:
#Override
public void writeSQL(SQLOutput stream) throws SQLException {
stream.writeString(owner_id);
stream.writeString(target_id);
stream.writeString(Objects.isNull(ip) ? "0" : ip); // weird, but as specified
stream.writeString(prefix);
stream.writeString(String.valueOf(port));
stream.writeString(description);
stream.writeString(cost_id);
}
Then at the start of the calling code, I've added:
con.getTypeMap().put("SCHEME_ADM.RECORD_REC", Record.class);
And instead of using createStruct(), the setObject() call now looks simply like this:
call.setObject("p_record_rec", t, Types.STRUCT)
But the result is the same - no errors and all the passed values are read as NULL. I've traced through the writeSQL() implementation and I can see that it is called and all values are passed correctly into the Oracle code. I've tried to use Types.JAVA_OBJECT in the setObject() call, and got an error: Invalid column type.
[Update 2]
Bordering on insane helplessness I've implemented the OracleData pattern:
public class Record implements SQLData, OracleData, OracleDataFactory {
...
#Override
public Object toJDBCObject(Connection conn) throws SQLException {
return conn.createStruct(getSQLTypeName(), new Object[] {
Objects.isNull(owner_id) ? "" : owner_id,
Objects.isNull(record_id) ? "" : record_id,
Objects.isNull(ip) ? "0" : ip,
Objects.isNull(prefix) ? "" : prefix,
String.valueOf(port),
Objects.isNull(description) ? "" : description,
Objects.isNull(cost_id) ? "" : cost_id
});
}
#Override
public OracleData create(Object jdbcValue, int sqltype) throws SQLException {
if (Objects.isNull(jdbcValue)) return null;
LinkedList<Object> attr = new LinkedList<>(Arrays.asList(((OracleStruct)jdbcValue).getAttributes()));
Record r = new Record();
r.setOwner_id(attr.removeFirst().toString());
r.setRecord_id(attr.removeFirst().toString());
r.setIp(attr.removeFirst().toString());
r.setPrefix(attr.removeFirst().toString());
r.setPort(Integer.parseInt(attr.removeFirst().toString()));
r.setDescription(attr.removeFirst().toString());
r.setCost_id(attr.removeFirst().toString());
return r;
}
public static OracleDataFactory getOracleDataFactory() {
return new Record();
}
Calling code:
...
// unwrap the Oracle object from C3P0 (standard JDBCv4 API)
OracleCallableStatement ops = call.unwrap(OracleCallableStatement.class);
// I'm not sure why I even need to do this - it looks exactly like
// the standard JDBC code
for (Records r : records) {
ops.registerOutParameter(1, Types.VARCHAR);
ops.setObject(2, t);
ops.execute();
ids.add(ops.getString(1));
}
...
And again, same result - no errors, a record is created in the table, with all provided values are null. I've traced through the code and the toJDBCObject() method is called correctly and does pass the values correctly in to createStruct().
Found the problem. Annoyingly, its about character encoding.
If in the toJDBCObject() implementation, I run getAttributes() on the created struct, the resulting Object[] array has all fields set as "???". Which is weird and looks like a character set transcoding failure (although it looks weird for that too - has three question marks for all fields regardless of value length, including empty string values).
According to Oracle's JDBC developer guide, "Globalization Support":
The basic Java Archive (JAR) file ojdbc7.jar, contains all the necessary classes to provide complete globalization support for:
Oracle character sets for CHAR, VARCHAR, LONGVARCHAR, or CLOB data that is not being retrieved or inserted as a data member of an Oracle object or collection type.
CHAR or VARCHAR data members of object and collection for the character sets US7ASCII, WE8DEC, WE8ISO8859P1, WE8MSWIN1252, and UTF8.
To use any other character sets in CHAR or VARCHAR data members of objects or collections, you must include orai18n.jar in the CLASSPATH environment variable:
ORACLE_HOME/jlib/orai18n.jar
And my setup was using the character set "WE8ISO8859P9" (I have no idea why, what it means, or even if it is selected by the client or the server - I just dumped the STRUCT object created by the OracleData API implementation and it was there somewhere).
So when Oracle says that it does not "provide complete globalization support", they mean "all character fields will be silently converted to NULL". Hmpph.
Anyway, adding orai18n.jar to the CLASSPATH indeed fixed the problem, and now records are added correctly to the database.

java,how to consult sql lite if a value of a variable "varConsult" is in column "protocolo" in table "pessoajuridica",

i tryed this:
ResultSet existetabela = stm.executeQuery ("SELECT * FROM pessoajuridica WHERE protocolo =" + varConsult );
System.out.println(existetabela);
but it only return a strange String -> org.sqlite.RS#1f959518
i was expecting the value..
remembering, sql lite and java :S
i want to use the value that it return to compare, if it return any value, means that it exist, so it will not add to the sql, if dont return anything = can add!!!
("if exist" doesnt work for me, says that its a invalid argument in the sql command line --')
You can use ResultSet#next() method to test whether there was any result set returned:
if (existetabela.next()) {
// Result was fetched
// Assuming type of protocol is String (Can be anything)
String protocol = existetabela.getString("protocolo");
} else {
// No result
}
Now, let's move ahead to the major issue. You should use PreparedStatement, to save yourself from SQL Injection.
You need to iterate inside the result set to retrive the actual data that were found:
while (existetabela.next()){
System.out.println(existetabela.getObject("protocolo"));
}
Did you look at PreparedStatement ?

java.sql.SQLException: Numeric Overflow while using IN operator

I implemented a Java application which queries a database based on given set of ids using the query:
select * from STUDENT where ID in (?)
The set of ids will be used to replace ?. However, occasionally, I receive an exception:
Caused by: java.sql.SQLException: Numeric Overflow
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:70)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:133)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:199)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:263)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:271)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:445)
at oracle.jdbc.driver.NumberCommonAccessor.throwOverflow(NumberCommonAccessor.java:4319)
at oracle.jdbc.driver.NumberCommonAccessor.getInt(NumberCommonAccessor.java:187)
at oracle.jdbc.driver.OracleResultSetImpl.getInt(OracleResultSetImpl.java:712)
at oracle.jdbc.driver.OracleResultSet.getInt(OracleResultSet.java:434)
After some testing, I realized that if I divide the list of ids into many sub-lists with smaller size, the exception stops happening. For some reason, jdbc doesn't like putting too many values into IN (?). I wonder if anyone has seen this issue before and has an explanation for it? As this issue never happens on production environment but only on a local one (which has less powerful resources), I suspect it has something to do with server's resources.
Thanks
Update: the source code that I'm using is:
// create a query
private String getQueryString(int numOfParams) {
StringBuilder out = new StringBuilder();
out.append("select * from STUDENT where ID in (");
for (int i = 0; i < numOfParams; i++) {
if (i == numOfParams - 1) {
out.append("?");
} else {
out.append("?, ");
}
}
out.append(")");
}
// set parameters
private void setParams(PreparedStatement ps, Set<String> params) {
int index = 1;
for (String param: params) {
ps.setString(index++, param);
}
}
public void queryStudent(Connection conn, Set<String> ids) throws Exception {
String query = this.getQueryString(ids.size());
PreparedStatement ps = conn.prepareStatement(query);
this.setParams(ps, ids);
ps.executeQuery();
// do some operations with the result
}
The issue was caused by conflict of ojdbc driver between GlassFish and application. In order to fix it, I need to:
Update application's pom.xml (as I'm using maven) to use a latest
ojdbc which is ojdbc6-11.2.0.3
Add ojdbc6-11.2.0.3 to GlassFish lib
If necessary, manually remove the ojdbc jar from deployed applications' lib in glassfish (apparently this is not cleared by undeploy)
Did you check MySQL and/or JDBC max packet size setting? That usually bites you with large IN (...) lists.
This occurs with the ID property or some other integer property type of your entity
look your stacktrace>
at oracle.jdbc.driver.NumberCommonAccessor.getInt(NumberCommonAccessor.java:187)
at oracle.jdbc.driver.OracleResultSetImpl.getInt(OracleResultSetImpl.java:712)
at oracle.jdbc.driver.OracleResultSet.getInt(OracleResultSet.java:434)
Any value returned from the query does not fit on this property!
Change the properties Integer and try to work with the next integer types (long, Long BigInteger) in all fields of Integer type in your entity.

issue dealing the ResultSet Value in Eclipse Rcp

My ResultSet Query is
StrQry = "select SUM(isnull(prn_amount,0))as prn_amount,SUM(isnull(adv_prn,0))as adv_prn, SUM(isnull(prv_prn,0))as prv_prn from loan_transaction_mcg where loan_id='1117'";
It is giving the result as on Sql
prn_amount =NULL
adv_prn =NULL
prv_prn =NULL
when the loan id =1117
ResultSet RsPrincipalDetail = getPaidDetail(loan_id);
while(RsPrincipalDetail.next()){
prn1 = RsPrincipalDetail.getString("prn_amount");
prn2 = RsPrincipalDetail.getString("adv_prn");
prn3 = RsPrincipalDetail.getString("prv_prn");
if(prn1.equals("")){
prn1.equals("0");
}
if(prn2.equalsIgnoreCase("")){
prn2.equals("0");
}
if(prn3.equalsIgnoreCase("")){
prn3.equals("0");
}
I tried putting prn1.equals(null) but still the null pointer exception comes. I tried in debug mode on prn1, it is showing as null as its value.
The problem here is that since your values in database are NULL when you convert them to java values using getString they will also be null.
Since null is not the same as empty string you can not really use prn.equals("")
Also using prn.equals(null) is a bad idea as usually the way that equals is implemented ... it will return false if something that it is compared to is null
Your best bet is to use equality operator to check for null
if(prn == null)
What are the column names of the table?
For me this seems to be possibly cyclic:
SUM(isnull(prn_amount,0))as prn_amount
and there seems to be a group by missing in your statement on loan_id

Categories