I have a List<String> of categories and for each category, I want to add them to my WHERE clause by combining with AND operator like: SELECT question_id FROM question WHERE category = categ1 AND category = categ2 AND category = ...
Since the size of the categories list is changing, I cannot do something like this:
String sql = "SELECT question_id FROM question WHERE category = ? AND category = ?";
jdbcTemplate.query(sql, stringMapper, "categ1", "categ2");
How can I achieve what I want?
Either check if JDBC Template from Spring handle that for you using a syntax which could be something like (from the doc, I don't think it does)
SELECT question_id FROM question WHERE category in (?...)
Or write your own query with the problems that may arise:
List<Object> parameters = new ArrayList<>(categories.size());
StringBuilder sb = new StringBuilde("SELECT question_id FROM question WHERE 1=1");
if (!categories.isEmpty()) {
if (categories.size() == 1) {
sb.append(" and category = ?");
} else {
sb.append(" and category in ");
sb.append(categories.stream()
.map(ignored -> "?")
.collect(joining(", ", "(", ")")));
sb.append(")");
}
parameters.addAll(categories);
}
Object[] paramArray = parameters.toArray();
jdbcTemplate.query(sb.toString(), stringMapper, paramArray);
Notes:
some security/quality tool may report SQL issues because you are writing a dynamic SQL.
Oracle put a limit on 1000 elements per IN. You would have to partition categories per group of 1000 (or less).
I used a stream() in a more or less strange fashion in order to generate the "?". If you use commons-lang3, you can replace it by "(" + StringUtils.repeat("?", ", ", categories.size()) + ")" (the example in the javadoc was probably done with this kind of use).
if you only have category as single criteria, you may probably remove the 1=1 as well as the and.
I believe this may work for you:
// The SQL Query
String sql = "SELECT question_id FROM question";
// Create the WHERE clause based on the number of items in List...
StringBuilder whereClause = new StringBuilder(" WHERE ");
StringBuilder ps = new StringBuilder("");
for (int i = 0; i < categories.size(); i++) {
if (!ps.toString().isEmpty()) {
ps.append(" AND ");
}
ps.append("category = ?");
}
whereClause.append(ps.toString()).append(";");
//Append the WHERE clause string to the SQL query string
sql = sql + whereClause.toString();
//System.out.println(sql);
/* Convert the categories List to an Object[] Array so as to
pass in as varArgs to the jdbcTemplate.query() method. */
Object[] psArgs = categories.toArray(new Object[categories.size()]);
jdbcTemplate.query(sql, stringMapper, psArgs);
I want to convert the result set from the db into json format, the problem is when the result set returned as multiple rows for the same id with some duplicates data.
DB example:
===========================================================
USER_ID NAME PHONE_NUMBER CITY
===========================================================
1 JACK 079999999999 New York
1 JACK 078888888888 Las Vegas
I want to make my json body looks like:
{ "USER_ID": 1,"NAME": JACK,"PHONE_NUMBER":[ 079999999999, 078888888888 ], "CITY": [ New York, Las Vegas ]}
this is my code:
Statement stmt = con.createStatement();
// Execute the SQL Query. Store results in ResultSet
ResultSet rs = stmt.executeQuery(Query);
System.err.println("Executing the query please wait ...");
ResultSetMetaData rsmd=null;
rsmd=(ResultSetMetaData) rs.getMetaData();
int columnsCount=rsmd.getColumnCount();
for (int j = 1; j <= columnsCount; j++) {
System.err.print(rsmd.getColumnName(j) + " || ");
}
JSONArray jsonArray = new JSONArray();
while (rs.next()) {
int total_rows = rs.getMetaData().getColumnCount();
JSONObject obj = new JSONObject();
for (int i = 0; i < total_rows; i++) {
String columnName = rs.getMetaData().getColumnLabel(i + 1).toLowerCase();
Object columnValue = rs.getObject(i + 1);
// if value in DB is null, then we set it to default value
if (columnValue == null){
columnValue = "null";
}
if (obj.has(columnName)){
columnName += "1";
}
obj.put(columnName, columnValue);
}
jsonArray.put(obj);
System.out.print("\n");
System.out.print("\n");
String gsonBody = gson.toJson(jsonArray);
System.err.println(gsonBody);
}
return jsonArray;
How can i make this general for all possible scenarios with different result set.
which RDBMS are you using? if Postgres/db2 is your database than you could return the resultset directly as JSON.
Using SQL/JSON
In many modern RDBMS, you don't have to implement any Java logic for that. The standard SQL way to do this is by using the SQL/JSON extensions. In your specific case, you could write
SELECT
user_id,
name,
JSON_ARRAYAGG(phone_number) AS phone_number,
JSON_ARRAYAGG(city) AS city
FROM t
GROUP BY user_id, name
Other dialects have different syntax for the same thing
Doing this in Java
Since you're asking how to do this specifically in Java, you could use jOOQ, which has extensive SQL/JSON support and would allow you to write the above query in a type safe way:
String json =
ctx.select(
T.USER_ID,
T.NAME,
jsonArrayAgg(T.PHONE_NUMBER).as(T.PHONE_NUMBER),
jsonArrayAgg(T.CITY).as(T.CITY))
.from(T)
.groupBy(T.USER_ID, T.NAME)
.fetch()
.formatJSON();
The benefit would be that this would take care of emulating the JSON array aggregation for your specific dialect and version, in case you have to work around some caveats.
(Disclaimer: I work for the company behind jOOQ)
A side note on normalisation
In the long term, you should think about normalising your data. You shouldn't have duplicate entries for USER_ID and NAME in this table, as it would allow for data anomalies (e.g. different NAME for the same USER_ID)
I am writing a crawler that scans many urls and then puts all the words found in each webpage into a table. In this same table the ID of the url is stored. If the word is repeated in another page, the ID of the url the word was found on is concatenated with a comma separating them. So if a word appears on multiple pages, all the concatenated ID numbers in the field might look like:
2,3,6,8,9
At the moment, if the number appears multiple times on the same page, the ID number will be added each time the number is found so the URLID field might end up looking like:
2,2,2,4,7,8,8,8,8,8,9,9
Using Java is there a way I get it to check if the number exists in the field already and only add it if it is not already there? I have looked through the api but cannot seem to find a suitable way to do this. Any ideas?
Addition:
public void updateWordTable( String[] array, int urlid ) throws SQLException, IOException {
Statement stat = connection.createStatement();
String wordQuery;
String query;
for (String item : array) {
if(item.matches("[A-Za-z0-9]+")){
wordQuery = "SELECT * FROM word WHERE word = '"+item+"'";
ResultSet rs = stat.executeQuery(wordQuery);
if(!rs.next()){
query = "INSERT INTO word VALUES ('"+item+"',"+urlid+")";
stat.executeUpdate( query );
}
else {
//query = "UPDATE word SET urlid = concat(urlid, ',"+urlid+"') WHERE word = '"+item+"' ";
//query = "UPDATE word SET urlid = CASE WHEN FIND_IN_SET( '"+urlid+"', urlid ) > 0 THEN urlid ELSE CONCAT( urlid, ',', '"+urlid+"' )END WHERE word = '"+item+"' ";
String query2 = "UPDATE word SET urlid = CASE WHEN FIND_IN_SET( ?, urlid ) > 0 THEN urlid ELSE CONCAT( urlid, ',', ? )END WHERE word = ? ";
PreparedStatement pst = connection.prepareStatement( query2 );
pst.setLong( 1, urlid );
pst.setLong( 2, urlid );
pst.setString( 3, item);
int result = pst.executeUpdate();
//stat.executeUpdate( query2 );
}
}
}
stat.close();
}
... is there a way I get it to check if the number exists in the field already and only add it if it is not already there?
You can do it using JAVA, but leave that checking to MySQL as it has such search features.
Using MySQL, you can use FIND_IN_SET function on comma separated values in the column. This will solve your problem to not reprocess in JAVA to find if such id exists.
select
FIND_IN_SET( value_to_find, column_with_cs_values ) > 0 as bool_matched
from table_name
Add where condition and others if any required.
And in the JAVA code you can just read the resultset for getBoolean.
boolean idMatched = rs.getBoolean( "bool_matched" );
if( idMatched ) {
// dont update table
}
else {
// update table
}
Alternatively, you can directly update the table column.
Example:
UPDATE table_name
SET column_name_with_cs_values =
CASE WHEN FIND_IN_SET( value_to_find,
column_name_with_cs_values
) > 0 THEN column_name_with_cs_values
ELSE CONCAT( column_name_with_cs_values, ',', value_to_find )
END
-- add where etc here
;
In JAVA, you can use the above query like the following with PreparedStatement.
String query = "UPDATE word
SET urlid = CASE WHEN FIND_IN_SET( ?, urlid ) > 0 THEN urlid
ELSE CONCAT( urlid, ',', ? )
END
WHERE word = ? ";
PreparedStatement pst = con.prepareStatement( query );
pst.setString( 1, urlid );
pst.setString( 2, urlid );
pst.setString( 3, item);
int result = pst.executeUpdate();
I guess your values are stored in mysql because your question is tagged mysql. In java you can request your database with a select and check if the value is already inserted.
Or if your are not in the mysql world but only java, use a structure that give you the guartantee of unicity as a Set instead of a List.
The easiest way would be just to load those values into Set. Set will take case to have unique elements only.
The idea is whenever you store your IDs this structure should keep uniqueness. Set is the best one when we talk about Java.
If you would like to have some mechanism on database to provide uniqueness that's another story.
That's just the general tip.
If your field is a String then you can use regex
boolean exists = s.matches("(^|.*,)"+ n + "($|,.*)");
Step1: Store new url_id in temp variable.
Step2: now check this url_id existence in your table by select statement, you can do this by below query, suppose new url_id is 7:
SELECT COUNT(url_id) FROM mytable WHERE (url_id LIKE '7,%' OR url_id LIKE '%,7' OR url_id LIKE '%,7,%');
Step3: if you get any count from above query then leave it, otherwise add in your table.
I was having a project coded in pl/sql that comes across the case like yours. My variable is stored in a String and I have to check if the number was already in the String variable. I did it from using
IF instr('2,3,6,8,9,' '2,') <= 0 THEN
' Code to append the '2,'
End If
For JAVA there is something similar to instr method, String.indexOf()
http://www.tutorialspoint.com/java/java_string_indexof.htm
However note that it will return 0 if it's the first character, so probably it will be < 0
String a = "2,3,6,8,9,";
If a.indexOf(ID + ",") < 0 { // -1 equivalent to NOT FOUND
// code to append ID + ",";
}
Note I need to check on ID + "," reason is e.g.
ID = "2";
a = "20,3,6,8,9,";
It will return me 0 due to the 20. Therefore I'm using comma as a delimiter for every number found.
So after I finish append variable a, I would remove the last comma by
a = a.substring(0, a.length()-1); // this will remove the last ","
System.out.println(a); // the output should be - 2,3,6,8,9
This is using Java if your variable is stored in Java.
In Simple JDBC (with no Hibernate)
we can do batch select by changing only place holder, like this
PreparedStatement stmt = conn.prepareStatement(
"select id, name from users where id = ?");
for ( int i=0; i < 10; i++ ) {
stmt.setInt(i); // or whatever values you are trying to query by
// execute statement and get result
}
How we can do it in Hibernate?
Hope this helps you out,
String hql = "from Users s where s.id= :userId";
for(int i=0; i< 10;i++){
List result = session.createQuery(hql)
.setParameter("userId", i)
.list();
}
This is the most common and user friendly way. It use colon followed by a parameter name (:example) to define a named parameter
Example 1: Using setParameter() method
String hql = "from Student s where s.registerNumner = :registerNumner";
List result = session.createQuery(hql).setParameter("registerNumner", "12345").list();
The setParameter() method is smart enough to discover the parameter data type of bind variable.
Example 2: Using setString() method
You can use setString to tell Hibernate this parameter date type is String.
String hql = "from Student s where s.registerNumber = :registerNumber";
List result = session.createQuery(hql).setString("registerNumber", "12345").list();
Example 3: Using setProperties() method
This feature is great ! You can pass an object into the parameter binding. Hibernate will automatic check the object’s properties and match with the colon parameter.
Student student = new Student();
Student.setRegisterNumber("12345");
String hql = "from Strudent s where s.registerNumber = :registerNumber";
List result = session.createQuery(hql).setProperties(student).list();
Example 4:
You can use positional parameters also.
String hql = "from Student s where s.registerNumber = ? and s.studentName = ?";
List result = session.createQuery(hql).setString(0, "12345").setParameter(1, "Harshad").list();
But it’s vulnerable to easy breakage because every change of the position(i.e. index) of the bind parameters requires a change to the parameter binding code
Batch select:
You can use following way for batch select
String hql = "from Users s where s.id= :userId";
List finalResult = new ArrayList();
for(int i=0; i< 10;i++){
List result = session.createQuery(hql).setParameter("userId", i).list();
finalResult.addCollection(result );
}
Is there a direct method to get the all the elements in a row from the ResultSet as String? JDBC
You may use BasicRowProcessor from Apache commons, Dbutils
ResultSet rs = ....
RowProcessor rp = new BasicRowProcessor();
Map m = rp.toMap( rs );
String s = m.toString();
And you'll have then as:
{ id = 1, name = Oscar, lastName = Reyes }
etc.
If you want to get rid of the columNames:
String s = m.values().toString();
There's not a single method call to do it, but you can use something like this to build a String:
StringBuilder sb = new StringBuilder();
ResultSetMetaData rsmd = rs.getMetaData();
int numberOfColumns = rsmd.getColumnCount();
for (int i = 1; i <= numberOfColumns; i++) {
sb.append(rs.getString(i));
if (i < numberOfColumns) {
sb.append(", ");
}
}
String data = sb.toString();
I've got to ask - why would you need something like that?
And no, it's not possible - you would need to call getString() at least once no matter what. The best you can do is to concatenate your fields in SQL, e.g.
SELECT col1 || ', ' || col2 || ', ' || col3 ... FROM my_table
and call resultSet.next().getString(1) to get that.
Update (based on comment)
I don't think (that depends on your JDBC driver and database, really - you'll have to measure to find out) that there is a big difference in data volume between sending data (from DB to your app) by columns vs one concatenated string. In fact, the latter may even be more expensive if you have numbers / dates because they'll likely occupy more bytes formatted as text.