I have a MySQL table with more than a million rows like this:
id (BIGINT) | id_other (INT)
24334501 | 20123
24334501 | 20324
24334501 | 20111
24334500 | 20123
24334500 | 20324
24334510 | 20111
....
From this table, I want to build a map from a list of ids like this:
id_other -> count of id
my query is: "select * from lsh where id = ?"
To perform those queries, I created an index for the column ìd
Now I want to get a list of all id_other from a list of id.
Currently I have this code:
for (Long id : ids{ // ids has a size between 2000 and 8000
statement.setLong(1, id);
ResultSet rs = statement.executeQuery();
while(rs.next()) {
int idOther = rs.getInt("id_other");
if(!map.containsKey(idOther)){
map.put(idOther, new AtomicInteger(0));
}
map.get(idOther).incrementAndGet();
}
}
any ideas how to perform this?
UPDATE:
Now I have this query: select id_other, count(*) FROM lsh WHERE id ? GROUP BY id_other .
I execute the query with:
Array array = connection.createArrayOf("BIGINT", values.toArray());
statement.setArray(1, array);
final ResultSet rs = statement.executeQuery();
But now I get this exception for connection.createArrayOf: java.sql.SQLFeatureNotSupportedException
thank you
I think you want something like the following (SQL Fiddle):
SELECT id_other, COUNT(*) AS total_ids , GROUP_CONCAT(id) AS list_id
FROM lsh
WHERE id_other IN (20111, 20123, 20324)
GROUP BY id_other
ORDER BY id_other;
Result:
id_other total_ids list_id
20111 2 24334501,24334510
20123 2 24334500,24334501
20324 2 24334500,24334501
Related
Hi I was wondering if it is possible to execute something like this using JDBC as it currently provides an exception even though it is possible in the MySQL query browser.
"SELECT FROM * TABLE;INSERT INTO TABLE;"
While I do realize that it is possible with having the SQL query string being split and the statement executed twice but I was wondering if there is a one time approach for this.
String url = "jdbc:mysql://localhost:3306/";
String dbName = "databaseinjection";
String driver = "com.mysql.jdbc.Driver";
String sqlUsername = "root";
String sqlPassword = "abc";
Class.forName(driver).newInstance();
connection = DriverManager.getConnection(url+dbName, sqlUsername, sqlPassword);
I was wondering if it is possible to execute something like this using JDBC.
"SELECT FROM * TABLE;INSERT INTO TABLE;"
Yes it is possible. There are two ways, as far as I know. They are
By setting database connection property to allow multiple queries,
separated by a semi-colon by default.
By calling a stored procedure that returns cursors implicit.
Following examples demonstrate the above two possibilities.
Example 1: ( To allow multiple queries ):
While sending a connection request, you need to append a connection property allowMultiQueries=true to the database url. This is additional connection property to those if already exists some, like autoReConnect=true, etc.. Acceptable values for allowMultiQueries property are true, false, yes, and no. Any other value is rejected at runtime with an SQLException.
String dbUrl = "jdbc:mysql:///test?allowMultiQueries=true";
Unless such instruction is passed, an SQLException is thrown.
You have to use execute( String sql ) or its other variants to fetch results of the query execution.
boolean hasMoreResultSets = stmt.execute( multiQuerySqlString );
To iterate through and process results you require following steps:
READING_QUERY_RESULTS: // label
while ( hasMoreResultSets || stmt.getUpdateCount() != -1 ) {
if ( hasMoreResultSets ) {
Resultset rs = stmt.getResultSet();
// handle your rs here
} // if has rs
else { // if ddl/dml/...
int queryResult = stmt.getUpdateCount();
if ( queryResult == -1 ) { // no more queries processed
break READING_QUERY_RESULTS;
} // no more queries processed
// handle success, failure, generated keys, etc here
} // if ddl/dml/...
// check to continue in the loop
hasMoreResultSets = stmt.getMoreResults();
} // while results
Example 2: Steps to follow:
Create a procedure with one or more select, and DML queries.
Call it from java using CallableStatement.
You can capture multiple ResultSets executed in procedure.
DML results can't be captured but can issue another select
to find how the rows are affected in the table.
Sample table and procedure:
mysql> create table tbl_mq( i int not null auto_increment, name varchar(10), primary key (i) );
Query OK, 0 rows affected (0.16 sec)
mysql> delimiter //
mysql> create procedure multi_query()
-> begin
-> select count(*) as name_count from tbl_mq;
-> insert into tbl_mq( names ) values ( 'ravi' );
-> select last_insert_id();
-> select * from tbl_mq;
-> end;
-> //
Query OK, 0 rows affected (0.02 sec)
mysql> delimiter ;
mysql> call multi_query();
+------------+
| name_count |
+------------+
| 0 |
+------------+
1 row in set (0.00 sec)
+------------------+
| last_insert_id() |
+------------------+
| 3 |
+------------------+
1 row in set (0.00 sec)
+---+------+
| i | name |
+---+------+
| 1 | ravi |
+---+------+
1 row in set (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Call Procedure from Java:
CallableStatement cstmt = con.prepareCall( "call multi_query()" );
boolean hasMoreResultSets = cstmt.execute();
READING_QUERY_RESULTS:
while ( hasMoreResultSets ) {
Resultset rs = stmt.getResultSet();
// handle your rs here
} // while has more rs
You can use Batch update but queries must be action(i.e. insert,update and delete) queries
Statement s = c.createStatement();
String s1 = "update emp set name='abc' where salary=984";
String s2 = "insert into emp values ('Osama',1420)";
s.addBatch(s1);
s.addBatch(s2);
s.executeBatch();
Hint: If you have more than one connection property then separate them with:
&
To give you somthing like:
url="jdbc:mysql://localhost/glyndwr?autoReconnect=true&allowMultiQueries=true"
I hope this helps some one.
Regards,
Glyn
Based on my testing, the correct flag is "allowMultiQueries=true"
Why dont you try and write a Stored Procedure for this?
You can get the Result Set out and in the same Stored Procedure you can Insert what you want.
The only thing is you might not get the newly inserted rows in the Result Set if you Insert after the Select.
I think this is the easiest way for multy selection/update/insert/delete. You can run as many update/insert/delete as u want after select (you have to make a select first(a dummy if needed)) with executeUpdate(str) (just use new int(count1,count2,...)) and if u need a new selection close 'statement' and 'connection' and make new for next select. Like example:
String str1 = "select * from users";
String str9 = "INSERT INTO `port`(device_id, potition, port_type, di_p_pt) VALUE ('"+value1+"', '"+value2+"', '"+value3+"', '"+value4+"')";
String str2 = "Select port_id from port where device_id = '"+value1+"' and potition = '"+value2+"' and port_type = '"+value3+"' ";
try{
Class.forName("com.mysql.jdbc.Driver").newInstance();
theConnection=(Connection) DriverManager.getConnection(dbURL,dbuser,dbpassword);
theStatement = theConnection.prepareStatement(str1);
ResultSet theResult = theStatement.executeQuery();
int count8 = theStatement.executeUpdate(str9);
theStatement.close();
theConnection.close();
theConnection=DriverManager.getConnection(dbURL,dbuser,dbpassword);
theStatement = theConnection.prepareStatement(str2);
theResult = theStatement.executeQuery();
ArrayList<Port> portList = new ArrayList<Port>();
while (theResult.next()) {
Port port = new Port();
port.setPort_id(theResult.getInt("port_id"));
portList.add(port);
}
I hope it helps
Hi I was wondering if it is possible to execute something like this using JDBC as it currently provides an exception even though it is possible in the MySQL query browser.
"SELECT FROM * TABLE;INSERT INTO TABLE;"
While I do realize that it is possible with having the SQL query string being split and the statement executed twice but I was wondering if there is a one time approach for this.
String url = "jdbc:mysql://localhost:3306/";
String dbName = "databaseinjection";
String driver = "com.mysql.jdbc.Driver";
String sqlUsername = "root";
String sqlPassword = "abc";
Class.forName(driver).newInstance();
connection = DriverManager.getConnection(url+dbName, sqlUsername, sqlPassword);
I was wondering if it is possible to execute something like this using JDBC.
"SELECT FROM * TABLE;INSERT INTO TABLE;"
Yes it is possible. There are two ways, as far as I know. They are
By setting database connection property to allow multiple queries,
separated by a semi-colon by default.
By calling a stored procedure that returns cursors implicit.
Following examples demonstrate the above two possibilities.
Example 1: ( To allow multiple queries ):
While sending a connection request, you need to append a connection property allowMultiQueries=true to the database url. This is additional connection property to those if already exists some, like autoReConnect=true, etc.. Acceptable values for allowMultiQueries property are true, false, yes, and no. Any other value is rejected at runtime with an SQLException.
String dbUrl = "jdbc:mysql:///test?allowMultiQueries=true";
Unless such instruction is passed, an SQLException is thrown.
You have to use execute( String sql ) or its other variants to fetch results of the query execution.
boolean hasMoreResultSets = stmt.execute( multiQuerySqlString );
To iterate through and process results you require following steps:
READING_QUERY_RESULTS: // label
while ( hasMoreResultSets || stmt.getUpdateCount() != -1 ) {
if ( hasMoreResultSets ) {
Resultset rs = stmt.getResultSet();
// handle your rs here
} // if has rs
else { // if ddl/dml/...
int queryResult = stmt.getUpdateCount();
if ( queryResult == -1 ) { // no more queries processed
break READING_QUERY_RESULTS;
} // no more queries processed
// handle success, failure, generated keys, etc here
} // if ddl/dml/...
// check to continue in the loop
hasMoreResultSets = stmt.getMoreResults();
} // while results
Example 2: Steps to follow:
Create a procedure with one or more select, and DML queries.
Call it from java using CallableStatement.
You can capture multiple ResultSets executed in procedure.
DML results can't be captured but can issue another select
to find how the rows are affected in the table.
Sample table and procedure:
mysql> create table tbl_mq( i int not null auto_increment, name varchar(10), primary key (i) );
Query OK, 0 rows affected (0.16 sec)
mysql> delimiter //
mysql> create procedure multi_query()
-> begin
-> select count(*) as name_count from tbl_mq;
-> insert into tbl_mq( names ) values ( 'ravi' );
-> select last_insert_id();
-> select * from tbl_mq;
-> end;
-> //
Query OK, 0 rows affected (0.02 sec)
mysql> delimiter ;
mysql> call multi_query();
+------------+
| name_count |
+------------+
| 0 |
+------------+
1 row in set (0.00 sec)
+------------------+
| last_insert_id() |
+------------------+
| 3 |
+------------------+
1 row in set (0.00 sec)
+---+------+
| i | name |
+---+------+
| 1 | ravi |
+---+------+
1 row in set (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Call Procedure from Java:
CallableStatement cstmt = con.prepareCall( "call multi_query()" );
boolean hasMoreResultSets = cstmt.execute();
READING_QUERY_RESULTS:
while ( hasMoreResultSets ) {
Resultset rs = stmt.getResultSet();
// handle your rs here
} // while has more rs
You can use Batch update but queries must be action(i.e. insert,update and delete) queries
Statement s = c.createStatement();
String s1 = "update emp set name='abc' where salary=984";
String s2 = "insert into emp values ('Osama',1420)";
s.addBatch(s1);
s.addBatch(s2);
s.executeBatch();
Hint: If you have more than one connection property then separate them with:
&
To give you somthing like:
url="jdbc:mysql://localhost/glyndwr?autoReconnect=true&allowMultiQueries=true"
I hope this helps some one.
Regards,
Glyn
Based on my testing, the correct flag is "allowMultiQueries=true"
Why dont you try and write a Stored Procedure for this?
You can get the Result Set out and in the same Stored Procedure you can Insert what you want.
The only thing is you might not get the newly inserted rows in the Result Set if you Insert after the Select.
I think this is the easiest way for multy selection/update/insert/delete. You can run as many update/insert/delete as u want after select (you have to make a select first(a dummy if needed)) with executeUpdate(str) (just use new int(count1,count2,...)) and if u need a new selection close 'statement' and 'connection' and make new for next select. Like example:
String str1 = "select * from users";
String str9 = "INSERT INTO `port`(device_id, potition, port_type, di_p_pt) VALUE ('"+value1+"', '"+value2+"', '"+value3+"', '"+value4+"')";
String str2 = "Select port_id from port where device_id = '"+value1+"' and potition = '"+value2+"' and port_type = '"+value3+"' ";
try{
Class.forName("com.mysql.jdbc.Driver").newInstance();
theConnection=(Connection) DriverManager.getConnection(dbURL,dbuser,dbpassword);
theStatement = theConnection.prepareStatement(str1);
ResultSet theResult = theStatement.executeQuery();
int count8 = theStatement.executeUpdate(str9);
theStatement.close();
theConnection.close();
theConnection=DriverManager.getConnection(dbURL,dbuser,dbpassword);
theStatement = theConnection.prepareStatement(str2);
theResult = theStatement.executeQuery();
ArrayList<Port> portList = new ArrayList<Port>();
while (theResult.next()) {
Port port = new Port();
port.setPort_id(theResult.getInt("port_id"));
portList.add(port);
}
I hope it helps
Hi I was wondering if it is possible to execute something like this using JDBC as it currently provides an exception even though it is possible in the MySQL query browser.
"SELECT FROM * TABLE;INSERT INTO TABLE;"
While I do realize that it is possible with having the SQL query string being split and the statement executed twice but I was wondering if there is a one time approach for this.
String url = "jdbc:mysql://localhost:3306/";
String dbName = "databaseinjection";
String driver = "com.mysql.jdbc.Driver";
String sqlUsername = "root";
String sqlPassword = "abc";
Class.forName(driver).newInstance();
connection = DriverManager.getConnection(url+dbName, sqlUsername, sqlPassword);
I was wondering if it is possible to execute something like this using JDBC.
"SELECT FROM * TABLE;INSERT INTO TABLE;"
Yes it is possible. There are two ways, as far as I know. They are
By setting database connection property to allow multiple queries,
separated by a semi-colon by default.
By calling a stored procedure that returns cursors implicit.
Following examples demonstrate the above two possibilities.
Example 1: ( To allow multiple queries ):
While sending a connection request, you need to append a connection property allowMultiQueries=true to the database url. This is additional connection property to those if already exists some, like autoReConnect=true, etc.. Acceptable values for allowMultiQueries property are true, false, yes, and no. Any other value is rejected at runtime with an SQLException.
String dbUrl = "jdbc:mysql:///test?allowMultiQueries=true";
Unless such instruction is passed, an SQLException is thrown.
You have to use execute( String sql ) or its other variants to fetch results of the query execution.
boolean hasMoreResultSets = stmt.execute( multiQuerySqlString );
To iterate through and process results you require following steps:
READING_QUERY_RESULTS: // label
while ( hasMoreResultSets || stmt.getUpdateCount() != -1 ) {
if ( hasMoreResultSets ) {
Resultset rs = stmt.getResultSet();
// handle your rs here
} // if has rs
else { // if ddl/dml/...
int queryResult = stmt.getUpdateCount();
if ( queryResult == -1 ) { // no more queries processed
break READING_QUERY_RESULTS;
} // no more queries processed
// handle success, failure, generated keys, etc here
} // if ddl/dml/...
// check to continue in the loop
hasMoreResultSets = stmt.getMoreResults();
} // while results
Example 2: Steps to follow:
Create a procedure with one or more select, and DML queries.
Call it from java using CallableStatement.
You can capture multiple ResultSets executed in procedure.
DML results can't be captured but can issue another select
to find how the rows are affected in the table.
Sample table and procedure:
mysql> create table tbl_mq( i int not null auto_increment, name varchar(10), primary key (i) );
Query OK, 0 rows affected (0.16 sec)
mysql> delimiter //
mysql> create procedure multi_query()
-> begin
-> select count(*) as name_count from tbl_mq;
-> insert into tbl_mq( names ) values ( 'ravi' );
-> select last_insert_id();
-> select * from tbl_mq;
-> end;
-> //
Query OK, 0 rows affected (0.02 sec)
mysql> delimiter ;
mysql> call multi_query();
+------------+
| name_count |
+------------+
| 0 |
+------------+
1 row in set (0.00 sec)
+------------------+
| last_insert_id() |
+------------------+
| 3 |
+------------------+
1 row in set (0.00 sec)
+---+------+
| i | name |
+---+------+
| 1 | ravi |
+---+------+
1 row in set (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Call Procedure from Java:
CallableStatement cstmt = con.prepareCall( "call multi_query()" );
boolean hasMoreResultSets = cstmt.execute();
READING_QUERY_RESULTS:
while ( hasMoreResultSets ) {
Resultset rs = stmt.getResultSet();
// handle your rs here
} // while has more rs
You can use Batch update but queries must be action(i.e. insert,update and delete) queries
Statement s = c.createStatement();
String s1 = "update emp set name='abc' where salary=984";
String s2 = "insert into emp values ('Osama',1420)";
s.addBatch(s1);
s.addBatch(s2);
s.executeBatch();
Hint: If you have more than one connection property then separate them with:
&
To give you somthing like:
url="jdbc:mysql://localhost/glyndwr?autoReconnect=true&allowMultiQueries=true"
I hope this helps some one.
Regards,
Glyn
Based on my testing, the correct flag is "allowMultiQueries=true"
Why dont you try and write a Stored Procedure for this?
You can get the Result Set out and in the same Stored Procedure you can Insert what you want.
The only thing is you might not get the newly inserted rows in the Result Set if you Insert after the Select.
I think this is the easiest way for multy selection/update/insert/delete. You can run as many update/insert/delete as u want after select (you have to make a select first(a dummy if needed)) with executeUpdate(str) (just use new int(count1,count2,...)) and if u need a new selection close 'statement' and 'connection' and make new for next select. Like example:
String str1 = "select * from users";
String str9 = "INSERT INTO `port`(device_id, potition, port_type, di_p_pt) VALUE ('"+value1+"', '"+value2+"', '"+value3+"', '"+value4+"')";
String str2 = "Select port_id from port where device_id = '"+value1+"' and potition = '"+value2+"' and port_type = '"+value3+"' ";
try{
Class.forName("com.mysql.jdbc.Driver").newInstance();
theConnection=(Connection) DriverManager.getConnection(dbURL,dbuser,dbpassword);
theStatement = theConnection.prepareStatement(str1);
ResultSet theResult = theStatement.executeQuery();
int count8 = theStatement.executeUpdate(str9);
theStatement.close();
theConnection.close();
theConnection=DriverManager.getConnection(dbURL,dbuser,dbpassword);
theStatement = theConnection.prepareStatement(str2);
theResult = theStatement.executeQuery();
ArrayList<Port> portList = new ArrayList<Port>();
while (theResult.next()) {
Port port = new Port();
port.setPort_id(theResult.getInt("port_id"));
portList.add(port);
}
I hope it helps
I have two tables with a 1 to n relationship.
table hobbies has a foreign key to table users called "userid"
SELECT * FROM hobbies WHERE userid = 7
can have multiple results.
table users contains a list of user profile data and table hobbies contains a list of user hobbies. I want to print a list of profile info of multiple users with hobbies of each user. (hobbies concatenated into one String)
i currently have the following sql queries: (pseudocode):
ResultSet result = executeQuery("SELECT * FROM users;");
for each returned row: executeQuery("SELECT * FROM hobbies WHERE userid =" + result.getInt("ref"));
how do I get the SQL queries out of the loop to optimize performance with a big list of users?
I was trying to do a LEFT JOIN and then doing the WHERE ref= check in java instead of in SQL
problem is that I then get duplicate users and i only want to print one row for each user when processing the result set
also Im not sure if the JOIN is really an improvement in performance, because I have to process more rows.
table users
+--------+------+---------+--------+
| userid | name | country | telno |
+--------+------+---------+--------+
| 1 | John | USA | 123456 |
| 2 | Max | Germany | 345678 |
+--------+------+---------+--------+
+--------------+------------+
| userid |hobby |
+--------------+------------+
| 1 | football |
| 1 | basketball |
| 2 | TV |
| 2 | Music |
| 2 | football |
+--------------+------------+
example output:
John, USA, 123456, {football, basketball}
Max, Germany, 345678, {TV, Music, football}
This is most probably the fastest solution
SELECT name, country, telNo, GROUP_CONCAT(hobby)
FROM users u LEFT JOIN hobbies h ON u.id = h.userId
GROUP BY name, country, telNo
See https://dev.mysql.com/doc/refman/5.7/en/group-by-functions.html#function_group-concat for formatting options.
It's only good for the final output, e.g., if your hobbies contain commas, you wont't be able to parse them uniquely.
You can try :
SELECT h.userid, h.allyouneed FROM hobbies h, users u WHERE h.userid =u.userid
and get the info in one query
"SELECT * FROM hobbies h join users u on h.userid = u.userid WHERE userid = ?"
preparedStatement.setInt(result.getInt("ref"));
You can then save all the hobbies to a list similar to the 'Book' example below.
public static List<String> selectAll() throws SQLException {
PreparedStatement ps = null;
ResultSet rs = null;
// THE LIST OF BOOKS YOU WILL RETURN
List<String> books = new ArrayList<>();
String sql = "SELECT BOOK FROM BOOKs";
try(Connection conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/bookshop", "root", "");){
ps = conn.prepareStatement(sql);
rs = ps.executeQuery();
while(rs.next()){
String book= rs.getString ("BOOK");
// ADD BOOK TO THE LIST
books.add(book);
}
}
finally{
ps.close();
rs.close();
}
// RETURN LIST OF ALL BOOKS
return books;
}
SELECT u.userid,u.name,u.country GROUP_CONCAT(hobby SEPARATOR ',')
FROM hobbies as h join users as u on h.userid=u.userid
where h.userid = ? GROUP BY u.userid,u.name,u.country;
assumming you have relationship on hobbies to users. the query will concat the columns of hobby of the join tables, and there will be one unique user per row.
I have a bit of a problem. I have a table with names, IDs and different geometry like so:
ID | Name| RealID| Geometry|
==========================================
1 | Hampshire | 3 | 0762453...
2 | Hampshire | 3 | 0156245...
3 | Salt Lake | 2 | 312455...
4 | Hampshire | 3 | 016422....
I have a need of selecting all Hampshire rows based on a list of IDs. As you can see my table has different geometries for say Hampshire.
I need all of them somehow by just comparing against one ID I get from a list of them.
I get the list of IDs from Java. It is simply a list with one ID(3) so far.
Doing this:
Select * from table where RealID = any(:listOfIds)
It only returns me one row no matter what if I send in Hampshire's ID in the list (3). I have tried something like this:
Select * from table where RealID IN (any(:listofids))
but the syntax is wrong and I'm not sure what to do to achieve my goal.
If you want to make some thing secure and clean code you can use this way :
String query = "Select * from table where RealID IN (";
for (int i = 0; i < list.lenght(); i++) {
query += (i == 0 ? "" : ", ") + "?";
}
query += ")";
PreparedStatement preparedStatement = dbConnection.prepareStatement(query);
for (int i = 0; i < list.lenght(); i++) {
preparedStatement.setInt(i, list.get(i));
}
ResultSet rs = preparedStatement.executeQuery(query);
while (rs.next()) {
//get your informations here
}
You should to loop throw your list and use PreparedStatement so your query should look like this :
Select * from table where RealID IN(?, ?, ?, ?, ?)
This is safe way.
You can try the following syntax
SELECT * FROM table WHERE RealID in (listOfIDs)
for example:
SELECT * FROM table WHERE RealID in (3,2,6....)
You can try this also:
SELECT * FROM table WHERE FIND_IN_SET('3', RealID);
can you try this one .
Select * from problem where RealID IN (?,?,?);
? =here put on RealID according your requirement
ex:
SELECT * FROM table WHERE RealID in (3);