I have a requirement. The technology is quite old doesn't support spring at all . It is pure java application with jdbc connection.
Requirement is :
Suppose
select * from employee where empid = <<empid>> and designation = 'Doctor'
I am trying to replace <> with actual int value in java . How I can do it ?
String query = "select * from employee where empid = <<empid>> and designation = 'Doctor'";
if(query.contains("<<empid>>"))
/// Here I want to replace <<empid>> with actual int value in java
Any leads will be helpful
The code you didn't paste, that actually executes the SQL is either [A] a massive security leak that needs serious rewrites, or [B] is using PreparedStatement.
Here's the problem: SQL injection. Creating the SQL string by mixing a template or a bunch of string constants together with a bunch of user input is a security leak. For example, if you try to make SELECT * FROM users WHERE email = 'foo#bar.com' by e.g. String sql = "SELECT * FROM users WHERE email = '" + email + "'";, the problem is, what if the user puts in the web form, in the 'email' field: whatever#foo.com'; DROP TABLE users CASCADE; EXEC 'FORMAT C: /y /force'; --? Then the SQL becomes:
SELECT * FROM users WHERE email = 'whatever#foo.com'; DROP TABLE users CASCADE; EXEC 'FORMAT C: /y /force'; --';
That is legal SQL and you really, really, really don't want your DB engine to execute it.
Each DB engine has its own ideas on what's actually legal, and may do crazy things such as treating curly quotes as real quotes, etc. So, there is no feasible blacklist or whitelist technology you can think of that will properly cover all the bases: You need to ask your DB engine to do this for you, you can't fix this hole yourself.
Java supports this, via java.sql.PreparedStatement. You instead always pass a fully constant SQL string to the engine, and then fill in the blanks, so to speak:
PreparedStatement ps = con.prepareStatement("SELECT * FROM users WHERE email = ?");
ps.setString(1, "foo#whatever.com");
ps.query();
That's how you do it (and add try-with-resources just like you should already be doing here; statements and resultsets are resources you must always close). Even if you call .setString(1, "foo#whatever.com'; DROP TABLE users CASCADE; --"), then it'll simply look for a row in the database that has that mouthful in the email field. It will not delete the entire users table. Security hole eliminated (and this is the only feasible way to eliminate it).
So, check out that code. Is it using preparedstatement? In that case, well, one way or another that code needs to be calling:
ps.setInt(1, 999);
Where ps is the PreparedStatement object created with connection.prepareStatement(...) where ... is either an SQL constant or at least your input string where the <<empid>> was replaced with a question mark and never with any string input from an untrusted source. The 1 in ps.setInt(1, 999) is the position of the question mark (1 = the first question becomes 999), and the 999 is your actual number. It may look like:
if (input instanceof String) {
ps.setString(idx++, (String) input);
} else if (input instanceof Integer) {
ps.setInt(idx++, ((Integer) input).intValue());
} ...
etcetera. If you don't see that, find the setInt invoke and figure out how to get there. If you don't see any setInt, then what you want is not possible without making some updates to this code.
If you don't even see PreparedStatement anywhere in the code, oh dear! Take that server offline right now, research if a security leak has occurred, if this server stored european data you have 72 hours to notify all users if it has or you can't reasonably figure out e.g. by inspecting logs that it hasn't, or you're in breach of the GDPR. Then rewrite that part using PreparedStatement to solve the problem.
So I would like to do a select from an SQL database, using Java, subject to a conditional statement (less than or equal to something) subject to some loop in Java. In other words, something like the following:
for (int i=0; i< 195; i++) {
// Get the results of the SQL query
resultSet = statement.executeQuery(
"SELECT max( millisFromMid ) FROM stockInfo1 WHERE ( millisFromMid <= 34200000 + (120000)*i ) GROUP BY stockID"
);
Now Java is returning an exception here, because it doesn't want me to use "i" in this condition; however, without being able to use it here, I'm not sure how I can change this condition. The problem is this: I'm looking to retrieve data from one database, with the intent of doing some manipulation and putting the results into a new database. These manipulations depend on taking the most recent data available, which is why I'm wanting to increase the number bounding millisFromMid. Does that make sense?
Does anyone have a suggestion on how someone might do something like this? This seems like a fundamental skill to have when using Java and SQL together, so I'd very much like to know it.
The SQL statement is parsed and run in a different environment than your Java code - different language, different scope, and unless you use SQLite they run in different processes or even different machines. Because of that, you can't just refer to the Java variable i from your SQL code - you have to either inject it or use special API.
The first option - injection - is to simply put the value of i inside the string:
"SELECT max( millisFromMid ) FROM stockInfo1 WHERE ( millisFromMid <= 34200000 + (120000)*"+i+" ) GROUP BY stockID"
Personally, I prefer to do it using String.format - but that's just me.
String.format("SELECT max( millisFromMid ) FROM stockInfo1 WHERE ( millisFromMid <= 34200000 + (120000)*%d ) GROUP BY stockID",i)
The second option - via API - is more complex but also faster(especially if you combine it with transactions) - using SQL parameters. You need to create a prepared statement:
PreparedStatement preparedStatement = connection.prepareStatement("SELECT max( millisFromMid ) FROM stockInfo1 WHERE ( millisFromMid <= 34200000 + (120000)*? ) GROUP BY stockID")
Notice the ? that is replacing the i - this is your parameter. You create prepareStatement once - before the loop, and inside the loop you set the parameter every time and execute it:
for (int i=0; i< 195; i++) {
preparedStatement.setInt(1,i); //Set the paramater to the current value of i.
resultSet = preparedStatement.executeQuery(); //Execute the statement - each time with different value of the parameter.
}
The problem os that the database is not told about the value of i. Instead make Java put the value into the string and then submit it to the database.
string sql = "select * from foo where bar=" + i;
You need to remember at all times that the database only knows what you explicitly tell it, and that you are essentially writing code at runtime that will execute on another host.
You can generate an sql string and call it via jdbc, using bind variable is more correct from performance, security. It also saves you from escaping any special characters.
Since you are learning about using sql with jdbc, please read about bind variables. An example is here
I know the advantages of using PreparedStatement, which are
query is rewritten and compiled by the database server
protection against SQL injection
But I want to know when we use it instead of Statement?
Query is rewritten and compiled by the database server
If you don't use a prepared
statement, the database server will
have to parse, and compute an
execution plan for the statement
each time you run it. If you find
that you'll run the same statement
multiple times (with different
parameters) then its worth preparing
the statement once and reusing that
prepared statement. If you are
querying the database adhoc then
there is probably little benefit to
this.
Protected against SQL injection
This is an advantage you almost
always want hence a good reason to
use a PreparedStatement everytime.
Its a consequence of having to
parameterize the query but it does
make running it a lot safer. The
only time I can think of that this
would not be useful is if you were
allowing adhoc database queries; You
might simply use the Statement
object if you were prototyping the
application and its quicker for you,
or if the query contains no
parameters.
Ask Tom's opinion:
The use of a Statement in JDBC should be 100% localized to being used for DDL (ALTER,
CREATE, GRANT, etc) as these are the only statement types that cannot accept BIND
VARIABLES.
PreparedStatements or CallableStatements should be used for EVERY OTHER type of statement
(DML, Queries). As these are the statement types that accept bind variables.
This is a fact, a rule, a law -- use prepared statements EVERYWHERE. Use STATEMENTS
almost no where.
He's specifically talking about Oracle but the same principle applies to any database that caches execution plans.
Database apps that scale and prevent SQL injection attacks at the same time? What's the downside?
I would turn this round: in a publicly distributed app, you should generally always use prepared statements unless you have a really compelling reason not to, and you should always supply parameters "properly" to the prepared statement, and not by splicing them into the query string.
Why? Well, basically because of the reasons you gave (or at least, the second one)...
PreparedStatements should be used very carefully in WHERE clauses.
Suppose that a table is defined as:
create table t (int o, k varchar(100), v varchar(100))
(e.g. "o: object-ID (foreign key), k: attribute-key, v: attribute-value").
Furthermore there is a (non-unique) index on v.
create index ixt on t ( v )
Suppose that this table contains 200 million rows inserted like:
for (i = 0; i < 100*1000*1000; i++) {
insert into t (o,k,v) values (i,'k1','v1');
insert into t (o,k,v) values (i,'k2', Convert(i, varchar));
}
("Thus, every object o has attributes k1=v1 and k2=o")
Then you should not build queries like:
select o,p,v from t as tx, t as ty where tx.o=ty.o and tx.k=? and tx.v=? and ty.k=? and ty.v=?
("find objects that have two given attributes")
My experience with ORACLE and MSSQL is, that those queries might need many minutes to return. This is true even if no row matches the where clause. It depends on wether the SQL-Server decides to lookup tx.v or ty.v first.
One shoud put the values for the columns k and v directy into the statement. I think this is because the SQL-Servers take the values into account when computing the execution plan.
A query look like this returns always after milliseconds:
select o,p,v from t as tx, t as ty where tx.o=ty.o and tx.k='k1' and tx.v='v1' and ty.k='k2' and ty.v='1234'
("The SQL-Server will always search first for v='1234' and then for v='v1' ")
Regards
Wolfgang
Statement: Each time the sql query is running,this sql statement is sent to the DBMS where it is compiled. So, it increases the server loads and decreases the performance.
connection con=null;
String sql="select * from employee where id=5";
Statement st=conn.createStatement();
PreparedStatement: Unlike Statement PreparedStatement is given a sql query as a parameter when it is created.
connection con=null;
String sql="select * from employee where id=?";
PreparedStatement ps=conn.prepareStatement(sql);
This sql statement is sent to Database where it is compiled.
So,in preparedStatement compiled happens only once but in statement compiled happens each time Statement is called.
You can always use PreparedStatement instead of Statment( select, insert , update, delete ). Better performance and protected against SQL injection.
But, don't use it with a dynamic request like a request with WHERE variable IN [ hundreds possibilities ] :
It's counter-productive, you lost performance and memory because you cache every time a new request, and PreparedStatement are not just for SQL injection, it's about performance. In this case, Statement will not be slower.
Your pool have a limit of PreparedStatment ( -1 defaut but you must limit it ), and you will reach this limit ! and if you have no limit or very large limit you have some risk of memory leak, and in extreme case OutofMemory errors. So if it's for your small personnal project used by 3 users it's not dramatic, but you don't want that if you're in a big company and that you're app is used by thousand people and million request.
Some reading.
IBM : Periodical OutOfMemory errors with prepared statement caching
It's simply a Java DESIGN MISTAKE tie "prepared statement" with "parameterized query / bind variables".
Databases does have API to accept "bind variables" in SQL code that just run once time.
It's a big resource wasting force use "prepared statement" everywhere, just to protect from SQL injection. Why not Java just let developers use databases in correct way?
It could be as follows:
Statement Interface - Multiples commands could be run. Not accept bind variables. One execution of SQL command. No SQL injection protection.
PreparedStatement Interface - One command could be run. Accept bind variables.
Multiple executions of SQL command. SQL injection protection.
(MISSING IN JAVA!) RunOnceStatement - One command could be run. Accept bind variables. One execution of SQL command. SQL injection protection.
For exemple, in Postgres performance could be better, by driver mapping to:
Statement Interface - PQExec()
PreparedStatement Interface - PQPrepare() / PQExecPrepare() / ...
(MISSING IN JAVA!) RunOnceStatement - PQExecParams()
Using prepared statement in SQL code that runs just once is a BIG performance problem: more processing in database, waste database memory, by maintaining plans that will not called later. Cache plans get so crowed that actual SQL commands that are executed multiple times could be deleted from cache.
Besides preventing SQL injection, formatting portability (which you can't get from Statement), performance is the obvious reason. However, PreparedStatement doesn't come without any penalty. For example, it is generally slower than Statement if running only once, as there is some overhead. So the general idea is PreparedStatement should be used when you are performing the same query many many times. However, how much overhead is very database server implementation-specific, so exactly when to choose PreparedStatement over Statement, from performance consideration, should really be based on your actual experience/experiments of a specific database server.
I was facing the same issue ,Then I break the query into multiple common table expression (cte) and now it's working fine for me.
SELECT DISTINCT
1 AS RecordSource, --CMD
EA.EmployerKey,
EA.AccountID,
ISNULL(EA.SourceGroupNumber,'NA') AS SourceGroupNumber,
EA.SourceSubGroupNumber,
EA.PurchaserOrgNumber,
ISNULL(EA.GroupName,'NA') AS GroupName,
EA.MemberSourceCode,
CASE WHEN ESA.EmailAddress = '' OR ESA.EmailAddress IS NULL THEN 'UNKNOWN#UNKNOWN.COM'
ELSE ESA.EmailAddress END AS EmailAddress,
CASE WHEN ESA.FirstName='' OR ESA.FirstName IS NULL THEN 'NA' ELSE ESA.FirstName END AS FirstName,
CASE WHEN ESA.MiddleName='' OR ESA.MiddleName IS NULL THEN 'NA' ELSE ESA.MiddleName END AS MiddleInitial,
CASE WHEN ESA.LastName='' OR ESA.LastName IS NULL THEN 'NA' ELSE ESA.LastName END AS LastName,
CASE WHEN ESA.ContactName='' OR ESA.ContactName IS NULL THEN 'NA' ELSE ESA.ContactName END AS ContactName,
ISNULL(ESA.PhoneNumber,'NA') As PhoneNumber,
'NA' AS MobilePhoneNumber,
'GROUP ADMINISTRATOR' AS ContactType,
ESA.StateCode
INTO DCB_A1.Temp_abc
FROM
(SELECT FE.EmployerKey, DE.AccountID, DE.SourceGroupNumber, DE.SourceSubGroupNumber,DE.PurchaserOrgNumber,
DE.GroupName, DE.MemberSourceCode
FROM DCB_A1.FctESF FE
INNER JOIN DCB_A1.DimESF DE
ON FE.EmployerKey = DE.EmployerKey
LEFT JOIN DCB_A1.DimMKBU DM
ON FE.MBUKey = DM.MarketingBusinessUnitKey
where DE.IsActiveVersion = 1 AND DE.PurchaserOrgTerminationReasonCode <> '171'
--and (MBUStateCoverage <> '' and MBUStateCoverage <> 'UNK')
GROUP BY FE.EmployerKey, DE.AccountID, DE.SourceGroupNumber,DE.SourceSubGroupNumber,DE.PurchaserOrgNumber,
DE.GroupName, DE.MemberSourceCode
)EA
INNER JOIN
(SELECT DE.AccountID,DE.SourceGroupNumber,
DE.MemberSourceCode,DE.GroupName,
ISNULL(DC.ContactValue,'') AS EmailAddress,
ISNULL(DN.ContactText2,'') AS FirstName,
ISNULL(DN.ContactText3,'') AS MiddleName,
ISNULL(DN.ContactText4,'') AS LastName,
ISNULL(DN.ContactValue,'') AS ContactName,
ISNULL(DP.ContactValue,'') AS PhoneNumber,
SUBSTRING(MBUStateCoverage, 1, 2) AS StateCode
FROM DCB_A1.FctESF FE
INNER JOIN DCB_A1.DimESF DE
ON FE.EmployerKey = DE.EmployerKey
LEFT JOIN DCA_A1.DimECSF DC
ON DE.MemberSourceCode = DC.MemberSourceCode
AND DE.PurchaserOrgNumber = DC.PurchaserOrgNumber
AND DE.PurchaserOrgTypeCode = DC.PurchaserOrgTypeCode
AND DC.ContactType = 3
--and DC.ContactValue <> '' -- Do we need this?
LEFT JOIN DCA_A1.DimECSF DN
ON DE.MemberSourceCode = DN.MemberSourceCode
AND DE.PurchaserOrgNumber = DN.PurchaserOrgNumber
AND DE.PurchaserOrgTypeCode = DN.PurchaserOrgTypeCode
AND DN.ContactType = 4
AND DN.IsActiveVersion = 1
LEFT JOIN DCA_A1.DimECSF DP
ON DE.MemberSourceCode = DP.MemberSourceCode
AND DE.PurchaserOrgNumber = DP.PurchaserOrgNumber
AND DE.PurchaserOrgTypeCode = DP.PurchaserOrgTypeCode
AND DP.ContactType = 1
LEFT JOIN DCB_A1.DimMKBU DM
ON FE.MBUKey = DM.MarketingBusinessUnitKey
WHERE DE.IsActiveVersion = 2
--and DE.AccountID =2
--AND (DE.CustomerStatusCode = 'A'
-- OR DE.IsEmployerActive = 0)
----and CustomerStatusCode = 'A'
--AND (MBUStateCoverage <> '' and MBUStateCoverage <> 'UNK')
GROUP BY DE.AccountID,DE.SourceGroupNumber,
DE.MemberSourceCode,DE.GroupName,
ISNULL(DC.ContactValue,''), ISNULL(DN.ContactValue,''),
ISNULL(DN.ContactText2,''),ISNULL(DN.ContactText3,''),
ISNULL(DN.ContactText4,''),ISNULL(DP.ContactValue,'') ,SUBSTRING(MBUStateCoverage, 1, 2)
)ESA
ON EA.AccountID = ESA.AccountID ;
[Amazon][JDBC](11220) Parameters cannot be used with normal Statement objects, use PreparedStatements instead.`
****Below check what I done with this code and it iss now working fine.****
with ESA AS
(SELECT DE.AccountID,DE.SourceGroupNumber,
DE.MemberSourceCode,DE.GroupName,
ISNULL(DC.ContactValue,'') AS EmailAddress,
ISNULL(DN.ContactText2,'') AS FirstName,
ISNULL(DN.ContactText3,'') AS MiddleName,
ISNULL(DN.ContactText4,'') AS LastName,
ISNULL(DN.ContactValue,'') AS ContactName,
ISNULL(DP.ContactValue,'') AS PhoneNumber,
SUBSTRING(MBUStateCoverage, 1, 2) AS StateCode
FROM DCB_A1.FctESF FE
INNER JOIN DCB_A1.DimESF DE
ON FE.EmployerKey = DE.EmployerKey
LEFT JOIN DCA_A1.DimECSF DC
ON DE.MemberSourceCode = DC.MemberSourceCode
AND DE.PurchaserOrgNumber = DC.PurchaserOrgNumber
AND DE.PurchaserOrgTypeCode = DC.PurchaserOrgTypeCode
AND DC.ContactType = 3
--and DC.ContactValue <> '' -- Do we need this?
LEFT JOIN DCA_A1.DimECSF DN
ON DE.MemberSourceCode = DN.MemberSourceCode
AND DE.PurchaserOrgNumber = DN.PurchaserOrgNumber
AND DE.PurchaserOrgTypeCode = DN.PurchaserOrgTypeCode
AND DN.ContactType = 4
AND DN.IsActiveVersion = 1
LEFT JOIN DCA_A1.DimECSF DP
ON DE.MemberSourceCode = DP.MemberSourceCode
AND DE.PurchaserOrgNumber = DP.PurchaserOrgNumber
AND DE.PurchaserOrgTypeCode = DP.PurchaserOrgTypeCode
AND DP.ContactType = 1
LEFT JOIN DCB_A1.DimMKBU DM
ON FE.MBUKey = DM.MarketingBusinessUnitKey
WHERE DE.IsActiveVersion = 2
--and DE.AccountID =2
--AND (DE.CustomerStatusCode = 'A'
-- OR DE.IsEmployerActive = 0)
----and CustomerStatusCode = 'A'
--AND (MBUStateCoverage <> '' and MBUStateCoverage <> 'UNK')
GROUP BY DE.AccountID,DE.SourceGroupNumber,
DE.MemberSourceCode,DE.GroupName,
ISNULL(DC.ContactValue,''), ISNULL(DN.ContactValue,''),
ISNULL(DN.ContactText2,''),ISNULL(DN.ContactText3,''),
ISNULL(DN.ContactText4,''),ISNULL(DP.ContactValue,'') ,SUBSTRING(MBUStateCoverage, 1, 2)
), EMK AS (
SELECT DISTINCT
1 AS RecordSource, --CMD
EA.EmployerKey,
EA.AccountID,
ISNULL(EA.SourceGroupNumber,'NA') AS SourceGroupNumber,
EA.SourceSubGroupNumber,
EA.PurchaserOrgNumber,
ISNULL(EA.GroupName,'NA') AS GroupName,
EA.MemberSourceCode
FROM
(SELECT FE.EmployerKey, DE.AccountID, DE.SourceGroupNumber, DE.SourceSubGroupNumber,DE.PurchaserOrgNumber,
DE.GroupName, DE.MemberSourceCode
FROM DCB_A1.FctESF FE
INNER JOIN DCB_A1.DimESF DE
ON FE.EmployerKey = DE.EmployerKey
LEFT JOIN DCB_A1.DimMKBU DM
ON FE.MBUKey = DM.MarketingBusinessUnitKey
where DE.IsActiveVersion = 1 AND DE.PurchaserOrgTerminationReasonCode <> '171'
--and (MBUStateCoverage <> '' and MBUStateCoverage <> 'UNK')
GROUP BY FE.EmployerKey, DE.AccountID, DE.SourceGroupNumber,DE.SourceSubGroupNumber,DE.PurchaserOrgNumber,
DE.GroupName, DE.MemberSourceCode)EA
)
Select Distinct
RecordSource
,EMK.EmployerKey
,EMK.AccountID
,EMK.SourceGroupNumber
,EMK.SourceSubGroupNumber
,EMK.PurchaserOrgNumber
,EMK.GroupName
,EMK.MemberSourceCode
,CASE WHEN ESA.EmailAddress = '' OR ESA.EmailAddress IS NULL THEN 'UNKNOWN#UNKNOWN.COM'
ELSE ESA.EmailAddress END AS EmailAddress
,CASE WHEN ESA.FirstName='' OR ESA.FirstName IS NULL THEN 'NA' ELSE ESA.FirstName END AS FirstName
,CASE WHEN ESA.MiddleName='' OR ESA.MiddleName IS NULL THEN 'NA' ELSE ESA.MiddleName END AS MiddleInitial
,CASE WHEN ESA.LastName='' OR ESA.LastName IS NULL THEN 'NA' ELSE ESA.LastName END AS LastName
,CASE WHEN ESA.ContactName='' OR ESA.ContactName IS NULL THEN 'NA' ELSE ESA.ContactName END AS ContactName
,ISNULL(ESA.PhoneNumber,'NA') As PhoneNumber
,'NA' AS MobilePhoneNumber
,'GROUP ADMINISTRATOR' AS ContactType
,ESA.StateCode
INTO DCB_A1.Temp_abc
FROM EMK
INNER JOIN ESA
ON EMK.AccountID = ESA.AccountID ;
Excerpt from code
PreparedStatement preparedStatement = connection.prepareStatement("SELECT * FROM sch.tab1 where col1 like lower ( 'ABZ' ) ");
preparedStatement.executeQuery();
The above code executes successfully.
But when i try to execute this
PreparedStatement preparedStatement = connection.prepareStatement("SELECT * FROM sch.tab1 where col1 like lower ( ? ) ");
preparedStatement.setString ( myValue );
preparedStatement.executeQuery();
It throws an exception."STRING TO BE PREPARED CONTAINS INVALID USE OF PARAMETER MARKERS"
What could be the problem here?
Answer found, see the comments
I suspect the problem is that you can't apply functions directly to parameters. Is there any particular reason why you want the lower casing to be performed at the database rather than in your code? (I can think of some potential reasons, admittedly.) Unless you really need to do this, I'd just change the SQL to:
SELECT * FROM sch.tab1 where col1 like ?
and call toLower() in Java, preferably specifying the appropriate locale in which to perform the lower-casing.
I think Carlos is on to something. Try
SELECT * FROM sch.tab1 where col1 like lower ( '' + ? )
or whatever passes for string concatenation operator in your version of SQL. Forcing a string context might get you past the error. May require extra parentheses.
For reference: I ran into the same problem while using the NORMALIZE_STRING function:
SELECT NORMALIZE_STRING(?, NFKD) FROM sysibm.sysdummy1
Error message:
THE DATA TYPE, LENGTH, OR VALUE OF ARGUMENT 1 OF NORMALIZE_STRING IS INVALID. SQLCODE=-171, SQLSTATE=42815, DRIVER=4.13.111
Using the following statement solved the problem (CONCAT). Thanks to Paul Chernoch!
SELECT search_normalize(NORMALIZE_STRING(? CONCAT G'', NFKD)) FROM sysibm.sysdummy1
Note the "G" prefix for Unicode compatibility.