We Keep Coding

Optimization: Getting SQL Query out of loop

I have two tables with a 1 to n relationship.
table hobbies has a foreign key to table users called "userid"
SELECT * FROM hobbies WHERE userid = 7
can have multiple results.
table users contains a list of user profile data and table hobbies contains a list of user hobbies. I want to print a list of profile info of multiple users with hobbies of each user. (hobbies concatenated into one String)
i currently have the following sql queries: (pseudocode):
ResultSet result = executeQuery("SELECT * FROM users;");
for each returned row: executeQuery("SELECT * FROM hobbies WHERE userid =" + result.getInt("ref"));
how do I get the SQL queries out of the loop to optimize performance with a big list of users?
I was trying to do a LEFT JOIN and then doing the WHERE ref= check in java instead of in SQL
problem is that I then get duplicate users and i only want to print one row for each user when processing the result set
also Im not sure if the JOIN is really an improvement in performance, because I have to process more rows.
table users
+--------+------+---------+--------+
| userid | name | country | telno |
+--------+------+---------+--------+
| 1 | John | USA | 123456 |
| 2 | Max | Germany | 345678 |
+--------+------+---------+--------+
+--------------+------------+
| userid |hobby |
+--------------+------------+
| 1 | football |
| 1 | basketball |
| 2 | TV |
| 2 | Music |
| 2 | football |
+--------------+------------+
example output:
John, USA, 123456, {football, basketball}
Max, Germany, 345678, {TV, Music, football}

This is most probably the fastest solution
SELECT name, country, telNo, GROUP_CONCAT(hobby)
FROM users u LEFT JOIN hobbies h ON u.id = h.userId
GROUP BY name, country, telNo
See https://dev.mysql.com/doc/refman/5.7/en/group-by-functions.html#function_group-concat for formatting options.
It's only good for the final output, e.g., if your hobbies contain commas, you wont't be able to parse them uniquely.

You can try :
SELECT h.userid, h.allyouneed FROM hobbies h, users u WHERE h.userid =u.userid
and get the info in one query

"SELECT * FROM hobbies h join users u on h.userid = u.userid WHERE userid = ?"
preparedStatement.setInt(result.getInt("ref"));
You can then save all the hobbies to a list similar to the 'Book' example below.
public static List<String> selectAll() throws SQLException {
PreparedStatement ps = null;
ResultSet rs = null;
// THE LIST OF BOOKS YOU WILL RETURN
List<String> books = new ArrayList<>();
String sql = "SELECT BOOK FROM BOOKs";
try(Connection conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/bookshop", "root", "");){
ps = conn.prepareStatement(sql);
rs = ps.executeQuery();
while(rs.next()){
String book= rs.getString ("BOOK");
// ADD BOOK TO THE LIST
books.add(book);
}
}
finally{
ps.close();
rs.close();
}
// RETURN LIST OF ALL BOOKS
return books;
}

SELECT u.userid,u.name,u.country GROUP_CONCAT(hobby SEPARATOR ',')
FROM hobbies as h join users as u on h.userid=u.userid
where h.userid = ? GROUP BY u.userid,u.name,u.country;
assumming you have relationship on hobbies to users. the query will concat the columns of hobby of the join tables, and there will be one unique user per row.

Query MySQL with multiple identifier

I have a MySQL table with more than a million rows like this:
id (BIGINT) | id_other (INT)
24334501 | 20123
24334501 | 20324
24334501 | 20111
24334500 | 20123
24334500 | 20324
24334510 | 20111
....
From this table, I want to build a map from a list of ids like this:
id_other -> count of id
my query is: "select * from lsh where id = ?"
To perform those queries, I created an index for the column ìd
Now I want to get a list of all id_other from a list of id.
Currently I have this code:
for (Long id : ids{ // ids has a size between 2000 and 8000
statement.setLong(1, id);
ResultSet rs = statement.executeQuery();
while(rs.next()) {
int idOther = rs.getInt("id_other");
if(!map.containsKey(idOther)){
map.put(idOther, new AtomicInteger(0));
}
map.get(idOther).incrementAndGet();
}
}
any ideas how to perform this?
UPDATE:
Now I have this query: select id_other, count(*) FROM lsh WHERE id ? GROUP BY id_other .
I execute the query with:
Array array = connection.createArrayOf("BIGINT", values.toArray());
statement.setArray(1, array);
final ResultSet rs = statement.executeQuery();
But now I get this exception for connection.createArrayOf: java.sql.SQLFeatureNotSupportedException
thank you

I think you want something like the following (SQL Fiddle):
SELECT id_other, COUNT(*) AS total_ids , GROUP_CONCAT(id) AS list_id
FROM lsh
WHERE id_other IN (20111, 20123, 20324)
GROUP BY id_other
ORDER BY id_other;
Result:
id_other total_ids list_id
20111 2 24334501,24334510
20123 2 24334500,24334501
20324 2 24334500,24334501

SQL Query works in SQL Developer, but not on the Java side

I'm trying to figure this but without success. I'm trying to find every row in a specific table that contains a value.
For example, let's consider this Employee table:
| Id | First_name | Last_name | Date_of_birth | Car_number |
|------------------|------------|------------|---------------|------------|
| 10001 | John | Washington | 28-Aug-43 | 5 |
| 10083 | Arvid | Sharma | 24-Nov-54 | null |
| 10034 | David | Johnson | 12-May-76 | |
I'm building this query on the Java side (this is what I print in the log):
Select * From Employee WHERE Id LIKE ? OR First_name LIKE ? OR Last_name LIKE ? OR Date_of_birth LIKE ? OR Car_number LIKE ?
then I use a prepared statement so that, if I search for the string 'oh' it becomes:
Select * From Employee WHERE Id LIKE '%oh%' OR First_name LIKE '%oh%' OR Last_name LIKE '%oh%' OR Date_of_birth LIKE '%oh%' OR Car_number LIKE '%oh%'
Here's the corresponding code:
String wantedQuery = "Select * From " + tableName + " WHERE";
PreparedStatement preparedStatement;
try(ResultSet rsColumns = columnsForTable(tableName)) {
String keyword = keywordField.getText();
int limit = 0;
while (rsColumns.next()) {
wantedQuery += " " + rsColumns.getString(1) + " LIKE ? OR";
limit++;
}
wantedQuery = wantedQuery.substring(0, wantedQuery.length()-3);
preparedStatement = con.prepareStatement(wantedQuery);
System.out.println(wantedQuery);
System.out.println("\'%"+keyword+"%\'");
for(int i = 1; i <= limit; i++) {;
preparedStatement.setString(i, "\'%"+keyword+"%\'");
}
}
try(ResultSet rs = preparedStatement.executeQuery()) {
//now get the results from here
ResultSetMetaData metaData = rs.getMetaData();
while (rs.next()) {
System.out.println("Fetching row");
...
}
Now the problem is that when I try to get the results, it gives me 0 rows (I never see "Fetching row" printed) but the query works in SQL Developer. So I guess the error is on the Java side but I have any clue what it can be. Any ideas?

When you are using setString(), JDBC knows that it must be a String, so it puts the single quotes around the String as needed, you don't have to do it yourself. But because you do so, it actually a different one, than what you expect.

Try below mentioned methods
Add spaces at the start and at the end in the query in java code
Remove ; from the end of the sql in java code as whenever java code executes sql it appends ; to it.

JAVA mySQL JOIN 2 tables

Hey guys I'm looking to get information out of 2 tables to create a JTABLE with that information.
The tables I am look at are 'shipments' and 'customers'
Where shipments takes the form of
shipNumber | shipperID | destID | size | weight
and customers takes the form of
ID | lastName | firstName | street | city | state | zip
The shipperID and destID both refer to a customer ID.
I am trying to get the city/state information out of the customers table that corresponds to the shipperID and destID.
I have tried the following
query = "SELECT shipments.shipNumber, customers.city, customers.state, customers.city, customers.state FROM shipments, customers WHERE shipments.shipperID = customers.ID";
Realizing that the duplicate customers.city/customers.state is populating the same information twice.
As previously said, I am trying to get the shipper city/state and destination city/state.
I also tried
query = "SELECT shipments.shipNumber, customers.city, customers.state, customers.city, customers.state, shipments.size"
+ " FROM shipments"
+ " INNER JOIN customers ON customers.id = shipments.shipperID";
Where this gives the same information.
I am not sure how to reference the destID = customer.id
Thanks,
Mike

The usual trick is to join with the customers table twice, once for the shipper and once for the destination:
SELECT shipments.shipNumber,
shipper.city, shipper.state,
dest.city, dest.state,
shipments.size
FROM shipments
INNER JOIN customers shipper ON shipper.id = shipments.shipperID
INNER JOIN customers dest ON dest.id = shipments.destID

How do I return a list of joined hibernate entities?

Suppose I have a MySQL table, and entity tag in hibernate, where a tag is a row in the following
id (primary key), tag, entity id
1, food, 77
2, shop, 98
3, food, 32
...
I'd like to return the total counts of the same tag sorted in decreasing number of entries. Aka,
tag, count
food, 2
shop, 1
...
I was told to do something like
SELECT tag, COUNT(*) `count`
FROM table1
GROUP BY tag
ORDER BY `count` DESC
Output:
| TAG | COUNT |
|------|-------|
| food | 2 |
| shop | 1 |
However, how do I read this newly created list of entities out from Hibernate? Do I need to somehow define a new object to read this list out?
The same question goes for say reading an entity (row) that is defined, but just joined with another entity as part of a query. How do I read out the result?
Thanks!

Query:
String query = "SELECT tag, COUNT(*) as count "+
"FROM table1 " +
"GROUP BY tag " +
"ORDER BY count DESC";
Additional class
public class Result
{
private String tag;
private int count;
// getters/ setters
}
Create native query
final Query nativeQuery = entityManager.createNativeQuery(query);
final SQLQuery sqlQuery = (SQLQuery) ((HibernateQuery) nativeQuery ).getHibernateQuery();
sqlQuery.addScalar("tag");
sqlQuery.addScalar("count");
sqlQuery.setResultTransformer(Transformers.aliasToBean(Result.class));