I want to convert the result set from the db into json format, the problem is when the result set returned as multiple rows for the same id with some duplicates data.
DB example:
===========================================================
USER_ID NAME PHONE_NUMBER CITY
===========================================================
1 JACK 079999999999 New York
1 JACK 078888888888 Las Vegas
I want to make my json body looks like:
{ "USER_ID": 1,"NAME": JACK,"PHONE_NUMBER":[ 079999999999, 078888888888 ], "CITY": [ New York, Las Vegas ]}
this is my code:
Statement stmt = con.createStatement();
// Execute the SQL Query. Store results in ResultSet
ResultSet rs = stmt.executeQuery(Query);
System.err.println("Executing the query please wait ...");
ResultSetMetaData rsmd=null;
rsmd=(ResultSetMetaData) rs.getMetaData();
int columnsCount=rsmd.getColumnCount();
for (int j = 1; j <= columnsCount; j++) {
System.err.print(rsmd.getColumnName(j) + " || ");
}
JSONArray jsonArray = new JSONArray();
while (rs.next()) {
int total_rows = rs.getMetaData().getColumnCount();
JSONObject obj = new JSONObject();
for (int i = 0; i < total_rows; i++) {
String columnName = rs.getMetaData().getColumnLabel(i + 1).toLowerCase();
Object columnValue = rs.getObject(i + 1);
// if value in DB is null, then we set it to default value
if (columnValue == null){
columnValue = "null";
}
if (obj.has(columnName)){
columnName += "1";
}
obj.put(columnName, columnValue);
}
jsonArray.put(obj);
System.out.print("\n");
System.out.print("\n");
String gsonBody = gson.toJson(jsonArray);
System.err.println(gsonBody);
}
return jsonArray;
How can i make this general for all possible scenarios with different result set.
which RDBMS are you using? if Postgres/db2 is your database than you could return the resultset directly as JSON.
Using SQL/JSON
In many modern RDBMS, you don't have to implement any Java logic for that. The standard SQL way to do this is by using the SQL/JSON extensions. In your specific case, you could write
SELECT
user_id,
name,
JSON_ARRAYAGG(phone_number) AS phone_number,
JSON_ARRAYAGG(city) AS city
FROM t
GROUP BY user_id, name
Other dialects have different syntax for the same thing
Doing this in Java
Since you're asking how to do this specifically in Java, you could use jOOQ, which has extensive SQL/JSON support and would allow you to write the above query in a type safe way:
String json =
ctx.select(
T.USER_ID,
T.NAME,
jsonArrayAgg(T.PHONE_NUMBER).as(T.PHONE_NUMBER),
jsonArrayAgg(T.CITY).as(T.CITY))
.from(T)
.groupBy(T.USER_ID, T.NAME)
.fetch()
.formatJSON();
The benefit would be that this would take care of emulating the JSON array aggregation for your specific dialect and version, in case you have to work around some caveats.
(Disclaimer: I work for the company behind jOOQ)
A side note on normalisation
In the long term, you should think about normalising your data. You shouldn't have duplicate entries for USER_ID and NAME in this table, as it would allow for data anomalies (e.g. different NAME for the same USER_ID)
Related
I have a List<String> of categories and for each category, I want to add them to my WHERE clause by combining with AND operator like: SELECT question_id FROM question WHERE category = categ1 AND category = categ2 AND category = ...
Since the size of the categories list is changing, I cannot do something like this:
String sql = "SELECT question_id FROM question WHERE category = ? AND category = ?";
jdbcTemplate.query(sql, stringMapper, "categ1", "categ2");
How can I achieve what I want?
Either check if JDBC Template from Spring handle that for you using a syntax which could be something like (from the doc, I don't think it does)
SELECT question_id FROM question WHERE category in (?...)
Or write your own query with the problems that may arise:
List<Object> parameters = new ArrayList<>(categories.size());
StringBuilder sb = new StringBuilde("SELECT question_id FROM question WHERE 1=1");
if (!categories.isEmpty()) {
if (categories.size() == 1) {
sb.append(" and category = ?");
} else {
sb.append(" and category in ");
sb.append(categories.stream()
.map(ignored -> "?")
.collect(joining(", ", "(", ")")));
sb.append(")");
}
parameters.addAll(categories);
}
Object[] paramArray = parameters.toArray();
jdbcTemplate.query(sb.toString(), stringMapper, paramArray);
Notes:
some security/quality tool may report SQL issues because you are writing a dynamic SQL.
Oracle put a limit on 1000 elements per IN. You would have to partition categories per group of 1000 (or less).
I used a stream() in a more or less strange fashion in order to generate the "?". If you use commons-lang3, you can replace it by "(" + StringUtils.repeat("?", ", ", categories.size()) + ")" (the example in the javadoc was probably done with this kind of use).
if you only have category as single criteria, you may probably remove the 1=1 as well as the and.
I believe this may work for you:
// The SQL Query
String sql = "SELECT question_id FROM question";
// Create the WHERE clause based on the number of items in List...
StringBuilder whereClause = new StringBuilder(" WHERE ");
StringBuilder ps = new StringBuilder("");
for (int i = 0; i < categories.size(); i++) {
if (!ps.toString().isEmpty()) {
ps.append(" AND ");
}
ps.append("category = ?");
}
whereClause.append(ps.toString()).append(";");
//Append the WHERE clause string to the SQL query string
sql = sql + whereClause.toString();
//System.out.println(sql);
/* Convert the categories List to an Object[] Array so as to
pass in as varArgs to the jdbcTemplate.query() method. */
Object[] psArgs = categories.toArray(new Object[categories.size()]);
jdbcTemplate.query(sql, stringMapper, psArgs);
I want to create an array for each row.
while (result.next()) {
String[] result1 = {
"Dog ID: " + result.getLong("dogs.id"),
", Dog name: " + result.getString("dogs.first_name"),
", Owner name: "
+ result.getString("owners.first_name"),
", Owner phone: " + result.getString("owners.phone") };
resultList.add(result1);
My code write every one row in one array.
Can i get numbers of columns and put a limit?
while (resultset.next()) {
int i = 1;
while(i <= numberOfColumns) {
It's because i can't send entire table as a result from server to client.
You can query by column number result.getLong(columnIndex) but it doesn't make sense in your case withing a loop because you have columns of different types (unless complicating the code).
If you want to optimize the traffic from server to client the way to go is querying for just the columns you need.
If you want to limit the rows returned, it might be better to put the limiting criteria into the SQL query and only return the rows you want to include.
In order to get number of columns in your ResultSet you can use the following piece of code :
Statement stat = conn.createStatement();
ResultSet rs = stat.executeQuery(myQuery);
ResultSetMetaData metaData = rs.getMetaData();
int numOfColumns = metaData .getColumnCount();
In Simple JDBC (with no Hibernate)
we can do batch select by changing only place holder, like this
PreparedStatement stmt = conn.prepareStatement(
"select id, name from users where id = ?");
for ( int i=0; i < 10; i++ ) {
stmt.setInt(i); // or whatever values you are trying to query by
// execute statement and get result
}
How we can do it in Hibernate?
Hope this helps you out,
String hql = "from Users s where s.id= :userId";
for(int i=0; i< 10;i++){
List result = session.createQuery(hql)
.setParameter("userId", i)
.list();
}
This is the most common and user friendly way. It use colon followed by a parameter name (:example) to define a named parameter
Example 1: Using setParameter() method
String hql = "from Student s where s.registerNumner = :registerNumner";
List result = session.createQuery(hql).setParameter("registerNumner", "12345").list();
The setParameter() method is smart enough to discover the parameter data type of bind variable.
Example 2: Using setString() method
You can use setString to tell Hibernate this parameter date type is String.
String hql = "from Student s where s.registerNumber = :registerNumber";
List result = session.createQuery(hql).setString("registerNumber", "12345").list();
Example 3: Using setProperties() method
This feature is great ! You can pass an object into the parameter binding. Hibernate will automatic check the object’s properties and match with the colon parameter.
Student student = new Student();
Student.setRegisterNumber("12345");
String hql = "from Strudent s where s.registerNumber = :registerNumber";
List result = session.createQuery(hql).setProperties(student).list();
Example 4:
You can use positional parameters also.
String hql = "from Student s where s.registerNumber = ? and s.studentName = ?";
List result = session.createQuery(hql).setString(0, "12345").setParameter(1, "Harshad").list();
But it’s vulnerable to easy breakage because every change of the position(i.e. index) of the bind parameters requires a change to the parameter binding code
Batch select:
You can use following way for batch select
String hql = "from Users s where s.id= :userId";
List finalResult = new ArrayList();
for(int i=0; i< 10;i++){
List result = session.createQuery(hql).setParameter("userId", i).list();
finalResult.addCollection(result );
}
i am trying to insert data from excel sheet into database in java. for that i have connected both the database using ODBC. excelsheet has 2 columns cname and title.
after querying the Excel spreadsheet i stored the resultSet values of cname and title into arraylist.
List Cname = new ArrayList();
List Title=new ArrayList();
Cname.add(rs.getString("Cname"));
Title.add(rs.getString("Title"));
this will result like this
[aaa, bbb,cccc, dddd]
[tree,human,animal,bird]
when i try to insert this into SQL database using insert query it is getting stored like this.
statement1.executeUpdate("INSERT INTO dbo.company (Cname,Title) SELECT
'"+Cname+"','"+Title+"'");
Cname Title
[aaa, bbb,cccc, dddd] [tree,human,animal,bird]
but i want to store this as
Cname Title
___________________________
aaa tree
bbb human
cccc animal
ddd bird
how do i do this???pls help to solve in this.
You'll have to insert/update the list values, actually you're inserting the string representations of the entire lists..
Assuming, both lists do exist (not null) and have the same length, then this is a trivial solution:
for (int i = 0; i < Cname.size(); i++ {
statement1.executeUpdate("INSERT INTO dbo.company (Cname,Title) SELECT
'"+Cname.get(i)+"','"+Title.get(i)+"'");
}
Note - every java class has an implementation of the toString() method, and that method is called, when you "use an object as a string", like in the expression to create the SQL statement. For lists, the method returns a String that simply includes the (String-representations of) the list element in brackets.
Put a for loop around your insert statement:
for(int i = 0; i < Cname.size(); i++)
statement1.executeUpdate("INSERT INTO dbo.company (Cname,Title) values ( '"+Cname.get(i)+"','"+Title.get(i)+"')");
If your db is SQL Server, you can reference the followng Q & A. Just prepare xml data and pass to Stored Procedure.
How to Update SQL Server Table With Data From Other Source (DataTable)
You can create a single SQL statement of this form:
INSERT INTO dbo.company (Cname, Title)
VALUES ('aaa', 'tree'),
('bbb', 'human'),
('cccc', 'animal'),
('dddd', 'bird')
With JDBC:
StringBuilder sql = new StringBuilder();
sql.append("INSERT INTO dbo.company (Cname,Title) VALUES ");
String glue = "";
for (int i = 0; i < Cname.size(); i++) {
sql.append(glue);
sql.append("('");
sql.append(Cname.get(i).toString().replace("'", "''"));
sql.append("', '");
sql.append(Title.get(i).toString().replace("'", "''"));
sql.append("')");
glue = ", ";
}
statement1.executeUpdate(sql.toString());
An alternative syntax (if the above doesn't work for Excel spreadsheets) is this:
INSERT INTO dbo.company (Cname, Title)
SELECT 'aaa', 'tree' UNION ALL
SELECT 'bbb', 'human' UNION ALL
SELECT 'cccc', 'animal' UNION ALL
SELECT 'dddd', 'bird'
Or with JDBC
StringBuilder sql = new StringBuilder();
sql.append("INSERT INTO dbo.company (Cname,Title) ");
String glue = "";
for (int i = 0; i < Cname.size(); i++) {
sql.append(glue);
sql.append("SELECT '");
sql.append(Cname.get(i).toString().replace("'", "''"));
sql.append("', '");
sql.append(Title.get(i).toString().replace("'", "''"));
sql.append("'");
glue = " UNION ALL ";
}
statement1.executeUpdate(sql.toString());
Is there a direct method to get the all the elements in a row from the ResultSet as String? JDBC
You may use BasicRowProcessor from Apache commons, Dbutils
ResultSet rs = ....
RowProcessor rp = new BasicRowProcessor();
Map m = rp.toMap( rs );
String s = m.toString();
And you'll have then as:
{ id = 1, name = Oscar, lastName = Reyes }
etc.
If you want to get rid of the columNames:
String s = m.values().toString();
There's not a single method call to do it, but you can use something like this to build a String:
StringBuilder sb = new StringBuilder();
ResultSetMetaData rsmd = rs.getMetaData();
int numberOfColumns = rsmd.getColumnCount();
for (int i = 1; i <= numberOfColumns; i++) {
sb.append(rs.getString(i));
if (i < numberOfColumns) {
sb.append(", ");
}
}
String data = sb.toString();
I've got to ask - why would you need something like that?
And no, it's not possible - you would need to call getString() at least once no matter what. The best you can do is to concatenate your fields in SQL, e.g.
SELECT col1 || ', ' || col2 || ', ' || col3 ... FROM my_table
and call resultSet.next().getString(1) to get that.
Update (based on comment)
I don't think (that depends on your JDBC driver and database, really - you'll have to measure to find out) that there is a big difference in data volume between sending data (from DB to your app) by columns vs one concatenated string. In fact, the latter may even be more expensive if you have numbers / dates because they'll likely occupy more bytes formatted as text.