I hope receive my answer this time
i wrote below code but don't know where is my mistake
it seem correct i think
this code should insert more than million records into oracle xe
i wrote it by single insert statement when execute PreparedStatement one by one
but it's run took 6 hours !!!!!!
because i was forced use thread.sleep()
package tokenizing;
import java.sql.*;
import java.util.StringTokenizer;
public class TokenExtraction2 {
public static void main(String[] args) throws Exception {
String myText[]=new String[2276];
Jdbc db=new Jdbc();
String st1=null;
int i=0;
int j=0;
String tokens[][]=new String [3000000][2];
st1="select ntext from NEWSTEXT ";
ResultSet result=db.select(st1);
while(result.next())
{
myText[i]=result.getString("ntext");
++i;
}
db.closedb();
i=0;
StringBuilder st= new StringBuilder("insert into tokens5(token,tokenlength) values");
while(i<2276)
{
StringTokenizer s=new StringTokenizer(myText[i]," 0123456789*./»«،~!##$%^&()_-\"+=:;|<>?“؟”’{}[]‘,\\\t\n\r\fabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ...—`—ـ؛–…_");
while(s.hasMoreTokens()){
String key=s.nextToken();
tokens[j][0]=key;
tokens[j][1]=(key.length())+"";
st.append("(?,?)");
if( i<2276 && s.hasMoreTokens())
st.append(", ");
else
st.append(";");
//db.insert(st, key, key.length());
//db.closedb();
System.out.println(key+"\t");
j++;
}
System.out.println("num of news is: "+i);
System.out.println("*****************************************************************************************");
System.out.println("num of tokens is: "+j);
System.out.println("next news"+"\t");
//j=0;
i++;
}
System.out.println(st);
int k=0;
Class.forName("oracle.jdbc.driver.OracleDriver") ;
Connection con = DriverManager.getConnection("jdbc:oracle:thin:#localhost:1521:xe","ALBALOO","myjava123");
PreparedStatement ps=con.prepareStatement(st.toString());
// con.setAutoCommit(false);
//j=1;
i=0;
//j=j-286;
while(k<j)
{
i=i+1;
ps.setString(i, tokens[k][0]);
System.out.println(i);
i=i+1;
ps.setInt(i,Integer.parseInt(tokens[k][1]));
System.out.println(k+2);
k++;
}
ps.executeUpdate();
//con.commit();
}
}
You seem to have trying to insert multiple rows with a single insert statement, by passing multiple sets of values; st appears to end up as:
insert into tokens5(token,tokenlength) values (?,?), (?,?);(?,?), ...;`
with thousands of value pair placeholder. You can't pass multiple sets of values like that. Oracle isn't expecting a comma after the first (?,?), hence the ORA-00933 error. You also have multiple semi-colons in there as you're putting one for each time around the i while loop. As Mark Rotteveel pointed out, you should not have any as Oracle JDBC doesn't allow multiple statements.
You might be better off implementing a string tokenizer as a function on the database and then doing a single insert ... select from newstext, rather than pulling all the data out, converting to and pushing it back. You should at least batch up your updates though. You could pass the tokens as an array argument to a stored procedure, for example.
I'm struggling to understand what you're really doing though, as it looks like you're splitting a string on pretty much any character, which doesn't leave much for the actual keys, does it? It's hard to follow though...
If you look at the Oracle INSERT description in the SQL Language Reference, then you can see that Oracle does not support inserting multiple rows using VALUES. Also as I commented above, using ; in a query doesn't always work as it is usually not part of the query itself, but a terminator for command line or script input.
In your specific case you are even trying to put multiple statements into one prepare. In JDBC a single statement prepare (or execute) should only be one actual statement, not multiple statements separated by ;. Drivers (or the database) usually don't allow it, although some provide options to execute multiple statements, but that is not compliant with JDBC.
Instead you can use JDBC batched updates:
con.setAutoCommit(false);
try (
PreparedStatement pstmt = con.
prepareStatement("insert into tokens5(token,tokenlength) values (?, ?)"
) {
// I use tokens as an abstraction on how you get the token and its length
while (tokens.next()) {
pstmt.setString(1, tokens.token());
pstmt.setInt(2, tokens.length());
pstmt.addBatch();
};
pstmt.executeBatch();
// Optionally do something with result of executeBatch()
con.commit();
}
Depending on the database+driver this will have similar runtime performance as a multi-values insert (I believe with Oracle it does), or simply behave as if you executed a single PreparedStatement multiple times with different values.
Related
This question already has answers here:
How to insert values in a table with dynamic columns Jdbc/Mysql
(2 answers)
Closed 5 years ago.
What is a good design pattern to achieve this without endless code?
Given the scenario whereby the user may input 1...100 columns, maybe 23 one time, 32 on another insert, and 99 fields on another insert etc. All of which may be different fields each time too.
The PreparedStatement in Java needs to know what column names to enter first, how many ?'s to put into the values part of the INSERT query, the data types of the database field names to ensure the correct setInt and setString etc are entered.
For less than around 10 columns, you can kind of get around this challenge with the following logic;
1) If variableEnteredForFieldName is not null, then append to the relevant parts of the query in the form of a String builder type setup;
fieldName_1
?
2) Do the same for all entered field names
3) Strip out the final trailing , that will naturally be present in both the field names and the ?s
4) Create the PreparedStatement
5) Run through the same input parameters again to determine of the variableEnteredForFieldName is not null, if not null, then run a setInt or setString based on the known data type that the database requires and set this to the correct index number for the ?s.
As long as the query builder logic and the query filler logic have the names/values in the correct order in part 1 and part 2, then all works well. It does however mean duplicating the entire code that relates to this logic, one for generating the SQL to use when creating the PreparedStatement and another for filling the PreparedStatement.
This is manageable for a small number of input parameters, but this soon gets unmanageable for larger number of input parameters.
Is there a better design pattern to achieve the same logic?
The code below is an outline of all of the above for reference;
String fieldName1 = request.getParameter("fieldName1");
String fieldName2 = request.getParameter("fieldName2");
//Build Query
String fieldNames = "";
String fieldQuestionMarks = "";
if (fieldName1 != null) {
fieldNames = fieldNames + " FIELD_NAME_1 ,";
fieldQuestionMarks = fieldQuestionMarks + " ? ,";
}
if (fieldName2 != null) {
fieldNames = fieldNames + " FIELD_NAME_2 ,";
fieldQuestionMarks = fieldQuestionMarks + " ? ,";
}
//Trim the trailing ,
fieldNames = fieldNames.substring(1, fieldNames.length() - 1);
fieldQuestionMarks = fieldQuestionMarks.substring(1, fieldQuestionMarks.length() - 1);
try {
String completeCreateQuery = "INSERT INTO TABLE_NAME ( " + fieldNames + " ) VALUES ( " + fieldQuestionMarks + " );";
Connection con = DriverManager.getConnection(connectionURL, user, password);
PreparedStatement preparedStatement = con.prepareStatement(completeCreateQuery);
int parameterIndex = 1;
//Fill Query
if (fieldName1 != null) {
preparedStatement.setString(parameterIndex, fieldName1);
parameterIndex++;
}
if (fieldName2 != null) {
preparedStatement.setInt(parameterIndex, Integer.parseInt(fieldName2));
parameterIndex++;
}
}
As you can see, it's do-able. But even with just 2 optional fields, this code is huge.
The way I see it, if user is able to omit any of the columns from the list, then all columns are optional, and can be safely set to NULL during an insert. Therefore, all you need is one prepared statement with the "monster" INSERT, with all columns listed; then during the actual insert operation, you loop though the user-provided data, setting values for the columns provided, and calling setNull() for omitted columns. You'll need to maintain a structure somewhere (your DAO class most likely) mapping column names to their order in the SQL statement.
Since it seems that I try to learn how to work with SQL and Java the hard way my Question is:
Is it possible to use a variable IN BETWEEN Quotation marks?
I know that if you use the output you can work like this:
System.out.println(_name + " "+_points+ " "+_ID);
Is there a way to make it all in only one Quotation Mark pair?
Something like this:
System.out.println("_name _points _ID");
If yes, how do I mark them so that the Compiler knows that it is a Variable that he should print?
The reason why I want to know it is simple, I try to use executeUpdate
stmt.executeUpdate("INSERT INTO usertable VALUES("+_name+")");
and want it without the addition signs in there.
No you cannot use a variable inside a String literal. There are a couple of options though.
The first is the way you are currently doing it using concatenation with the + sign:
String query = "INSERT INTO table VALUES(" + name + ")";
Another way is to use String.format
String query = String.format("INSERT INTO table VALUES(%s)", name);
But the preferred method for SQL to avoid SQL Injection attacks is using a PreparedStatement:
String query = "INSERT INTO table VALUES(?)";
PreparedStatement statement = con.prepareStatement(query);
statement.setString(1, name);
statement.executeUpdate();
If you have a variable and want to pass it to your query statement with the quotation just add the quotes to your command. If the quotes are single quotes you don't need to scape then but if it is, you gona need to:
stmt.executeUpdate("INSERT INTO usertable VALUES('"+_name+"')");
^
|_Just add the quotes inside the
string
If it is a double quote (which I think is hardly the case) you need to scape then. Scaping is a way to tell the compiler that that specific string is special
stmt.executeUpdate("INSERT INTO usertable VALUES(\""+_name+"\")");
^
|_See the slash before the double quote?
But since you are learning you should learn the proper way to do it, because use variables with quotations will make your code prone to SQL Injection
So The better way to do it is to use Prepared Statements and language willl take care of the quotes for you. It would be:
String sql = "INSERT INTO usertable VALUES (?)"
preparedStatement = dbConnection.prepareStatement(sql);
preparedStatement.setString(1, "name you want");
See here a complete example: http://www.mkyong.com/jdbc/jdbc-preparestatement-example-select-list-of-the-records/
You can use preparedStatement:
Example :
query :
private static String SQL_INSERT_NEW_RULE = "INSERT INTO Table (A, B) VALUES (?, ?)";
then you can put them like this:
PreparedStatement pStmt=null;
pStmt = conn.prepareStatement(SQL_INSERT_NEW_RULE);
int index = 1;
pStmt.setString(index++, "value for A");
pStmt.setLong(index++, "Value for B");
I am trying to improve performance of an api. I need to know whether the second line that's marked will hit database too ? as I want to minimise that.
StringBuffer queryBuf = new StringBuffer("some query in SQL");
--------->StringBuffer queryBuf2 = new StringBuffer(" SELECT DISTINCT PP.ID FROM ( " + queryBuf + ") PP ");
Query query1 = getSession().createSQLQuery(queryBuf2.toString());
query1.setReadOnly(true);
ScrollableResults results = query1.scroll();
if (results.isLast() == false)
results.last();
int total = results.getRowNumber() + 1;
results.close();
logger.debug(">>>>>>TOTAL COUNT<<<<<< = {}", total);
No. Only the line ScrollableResults results = query1.scroll(); executes sql.
Also, you may want to use SQL COUNT
Its plain and simple it wont hit , you are just creating StringBuffer objects.
you are just create a string buffer. its not hitting the db. If i right this code may give compile error.
I need to know whether the second line that's marked will hit database too ?
You are constructing StringBuffer objects in first three lines , why should it hit the DB ! You can use StringBuilder if synchronization is not required !
No it will not hit the database. It will query from the buffer created by first query.
I want to get string from column no. 4 from my database to check user privileges.
Can I use rs.getString(index) to get data from column no.4?
I want to check user´s privileges...so if the column data is equal 4, the page will be redirected to AdminControlPanel.jsp
BUT, this code doesn´t work :(
String user=request.getParameter("login");
String pass=request.getParameter("password");
Class.forName("com.mysql.jdbc.Driver").newInstance();
Connection con=DriverManager.getConnection("jdbc:mysql://localhost:3306/android","root","root");
Statement st=con.createStatement();
ResultSet rs=st.executeQuery("select * from users where login='"+user+"' and password='"+pass+"'");
String p = rs.getString(4);
int count=0;
while(rs.next()){
count++;
}
if(count>0 && p == "4"){
// out.println(rs);
response.sendRedirect("AdminControlPanel.jsp");
}
else{
out.println("aaa");
response.sendRedirect("#");
}
}
catch(Exception e){
System.out.println(e);
}
you are comparing two String objects rather than checking the values in the String.
just change the code to p.equals("4") and try.
String p = rs.getString(4); // This should be inside your while
int count=0;
while(rs.next()){
count++;
}
You should move your first line inside your while loop. You can't
fetch the columns of a row, until you move your cursor to that row
using res.next().
Also, since your database should ideally have only one record for a
combination of username and password. So, you can better use an
if instead of while.
And you don't really need a count variable there.
So, your code should be: -
ResultSet rs=st.executeQuery("select * from users where login='"+user+"' " +
"and password='"+pass+"'");
if (rs.next()) {
String p = rs.getString(4); // Note that using Column name is a better idea
// or rs.getInt(4) if the column type is `int`
if(p.equals("4")) { // Use equals method to compare string content
response.sendRedirect("AdminControlPanel.jsp");
} else{
out.println("aaa");
response.sendRedirect("#");
}
}
Also, note that you should compare your string using equals method. if (p == "4") will give you false result. == operator does not compare the content of the string, rather the content of the reference used in comparison.
You want
while (rs.next()) {
String val = rs.getString(4);
....
Note that iterating through a ResultSet iterates through the rows. For each row, the column indexing starts from '1'.
However it's safer to get by column name, since your SQL query doesn't specify neither the columns nor the order in which they're returned:
String val = rs.getString("COLUMN_NAME");
I see from the below that you need an integer. Check out the doc for ResultSet for more info, but:
int val = rs.getInt("COLUMN_NAME");
As an aside, I don't see you closing your ResultSet/Statement/Connection in the above. If you're not, then you'll need to!
I want to generate a string such as sql command:
"INSERT INTO xxx VALUES(XXX, XXX, XXX)"
currently I use StringBuilder and some String constant like "INSERT INTO" to concatenate input String parameters for the table name and inserted values.
However, other than performance issue, this plain concatenation looks not elegant.
Is there any other way of doing this?
In my opinion, JDBC's prepared statement is one good example of such a "command template":
PreparedStatement pstmt=connection.createPreparedStatement("INSERT INTO ? VALUES(?,?,?)");
then you can set the table name and inserted value.
pstmt.setString(1,"tableA");
pstmt.setInt(2, 100);
...
However, I can not use prepared statement, since what I want is just String...
And someone give me some hint to use java.util.Regex or JavaCC to produce the String.
But as far as I can see, whatever is chosen for some code elegancy issue, Java String must be generated by something like StringBuilder, right???
You could use String.format():
String.format("insert into %s values('%s', '%s', '%s')", "user", "user123", "pass123", "yellow");
It's worth noting though, that any of these "string building" techniques leave you vulnerable to SQL injection attacks. You should really use JDBC parameterised queries wherever possible.
Edited to add quotes around strings.
Maybe you are looking for java.text.MessageFormat
int planet = 7;
String event = "a disturbance in the Force";
String result = MessageFormat.format(
"At {1,time} on {1,date}, there was {2} on planet {0,number,integer}.",
planet, new Date(), event);
Have you tried just using '+' ?
String sql = "INSERT INTO " + table
+" VALUES(" + value1 + ", " + value2 + ", " = value3+")";
Given the variety of other answers and none of them met your approval, perhaps you should accept that the actual String generation (sans JPA, PreparedStatement, etc.) is going to be fairly inelegant and create a utility class with static sql generators.
edit Showing an example of how I'd go about this if a pre-existing class such as PreparedStatement weren't an option. It's not the most elegant, but it does what it's supposed to (assuming I typed it all in correctly).
public class SQLUtil {
public static String generateInsertSQL(String tableName, List<CustomParameter> parmList){
StringBuilder sb = new Stringbuilder();
sb.append("insert into ");
sb.append(tableName);
sb.append(" values (");
for (int i = 0; i < parmList.size(); i++){
customParameter parm = parmList.get(i);
switch (parm.getType()) { // enum with your desired sql types
case ParmTypes.String:
sb.append("'");
sb.append(StringEscapeUtils.escapeSql(String.valueOf(parm.getValue())));
sb.append("'");
break;
case ParmTypes.Integer:
sb.append(Integer.valueOf(parm.getValue()));
break;
}
if (i < parmList.size() - 1) sb.append(",");
}
sb.append(")");
return sb.toString();
}
}
This way, your business code will remain relatively elegant and you can play around with the SQL String generation to your heart's content. You can also use this to "guarantee" all your inserts are protected against such attacks as SQL injection.
Use StringTemplate (http://www.stringtemplate.org/) maybe a good choice:
This looks better, right?
StringTemplate insert = new StringTemplate("INSERT $table$ VALUES ($value; separator=\",\"$)");
insert.setAttribute("table", "aTable");
String[] values = {"1", "1", "'aaa'", "'bbb'"};
for(int i = 0;i < values.length;i++){
insert.setAttribute("value", values[i]);
}
System.out.println(insert.toString());