I'm trying to query for a set and return it as a hash map object with Spring JdbcTemplate. But some reasons I'm getting empty result set from server. It is possible that I have a problem in the overall configuration but the rest of the queries are working without problems.
This is how I query
public Map<Integer, String> getCompanyDataservers() {
return getTemplate().queryForObject("select id, dataserver from company", new RowMapper<Map<Integer,String>>() {
#Override
public Map<Integer, String> mapRow(ResultSet rs, int rowNum)
throws SQLException {
HashMap<Integer, String> toReturn = new HashMap<Integer, String>();
while(rs.next()) {
int id = rs.getInt("id");
toReturn.put(id, dataserver);
}
return toReturn;
}});
}
After some debugging and logging statements I concluded that my resultset seems to be empty of any rows. When I query the same ("select id, dataserver from company") manually directly from DB I get the desired result. Yet this way I get a resultset with 0 rows.
One of my theories is that can't get get this kind of set when querying for a object this way. But isn't there a possibility to be free at you queries and construct a more elaborate object as a query result, or I have to create a purpose built class to be used by "queryForList" to get the desired data and convert it afterwards?
Or am I just missing something?
You are not supposed to invoke rs.next() inside mapRow. JdbcTemplate will invoke mapRow for you for every row in the resultset
All you need to do on mapRow is just return a HashMap which represent one single database row, spring will do the resultset iteration for you
Related
I was successfully able to execute a jpql query and print the result which is stored in a queryResults variable. What I want to achieve next is storing just the IDs (primary key column) in a list without the date (value), but I am not too sure if this is possible; perhaps using something like a java map. Is it possible? If yes, how can this be easily achieved?
private static final TestDao Test_DAO = new TestDao();
#Test
public void testById() {
List<TestEntity> queryResults = TEST_DAO.findById(""); //The record from the sql query is stored in queryResults and findById("") is the method that executes the query in a TestDao class and it is called here
for (TestEntity qResult: queryResults) { // looping through the query result to print the rows
System.out.println(qResult.getId());
System.out.println(qResult.getDate());
}
System.out.println("This is the sql result " + queryResults );
}
Output:
This is the result [TestEntity(id=101, date=2020-01-19 15:12:32.447), TestEntity(id=102, date=2020-09-01 11:04:10.0)]// I want to get the IDs 101 and 102 and store in a list without the Dates
I tried using a map this way:
Map<Integer, Timestamp> map= (Map<Integer, Timestamp>) queryResults.get(0); but I got an exception:
java.lang.ClassCastException: TestEntity cannot be cast to java.util.Map
There are some points before the implementation.
Why are you defining DAO as static? I think this is a bad implementation unless I am missing a particular reason you declared it static. You should define this as a member variable and not a static member
The naming of the method - findById() translated in English is - find Something by this Id, but you are fetching a list of Records, so naming is not correct.
Point 2 becomes invalid if ID property is not a Primary Key in your table, then it makes sense, but still naming is bad. Id is something we use to define Primary Key in the Database and should be and will be unique. But your comments suggest that ID is unique and the Primary Key. So read about how Databases work
And even if not unique, if you pass an Id to find some records, why will get different ids in the Records !!!
About implementation:
Changing in your existing code:
private TestDao Test_DAO = new TestDao();
#Test
public void testById() {
List<TestEntity> queryResults = TEST_DAO.findById("");
List<Long> listOfIds = new ArrayList<>(); // Assuming Id is Long type, same logic for any type
for (TestEntity qResult: queryResults) {
System.out.println(qResult.getId());
listOfIds.add(qResult.getId()); // Just add it to the list
System.out.println(qResult.getDate());
}
}
In case you want to be efficient with the query:
You can use JPQL and hibernate
You can then write a query like:
String query = "select te.id from TestEntity te";
// Create the TypedQuery using EntityManager and then get ResultSet back
List<Long> ids = query.getResultList();
In case of using Spring-Data-Jpa, you can define the repository and define the method and pass the query with #Query annotation. Spring Data JPA
I am using below bean definition to configure a reader to read some data from the database table in a Spring Batch project. It is using a named param in SQL. I am passing A java.util.List as a parameter. However, I am getting Invalid Column type error when it tries to run the SQL.
If I just hard code one single value (namedParameters.put("keys", "138219"); ) instead of passing a list, it works.
#Bean
public JdbcCursorItemReader<MyDTO> myReader() {
JdbcCursorItemReader<MyDTO> itemReader = new JdbcCursorItemReader<>();
itemReader.setDataSource(myDatasource);
itemReader.setRowMapper(return new RowMapper<MyDTO>() {
#Override
public MyDTO mapRow(ResultSet resultSet, int rowNum) throws SQLException {
return toMyDto(resultSet);
}
};);
Map<String, Object> namedParameters = new HashMap<>();
List<Long> keys= //Some List
Map<String, List<Long>> singletonMap = Collections.singletonMap("keys", keys);
namedParameters.putAll(singletonMap);
itemReader.setSql(NamedParameterUtils.substituteNamedParameters("SELECT A FROM MYTABLE WHERE KEY IN (:keys)",new MapSqlParameterSource(namedParameters)));
ListPreparedStatementSetter listPreparedStatementSetter = new ListPreparedStatementSetter();
listPreparedStatementSetter.setParameters(
Arrays.asList(NamedParameterUtils.buildValueArray("SELECT A FROM MYTABLE WHERE KEY IN (:keys)", namedParameters)));
itemReader.setPreparedStatementSetter(listPreparedStatementSetter);
return itemReader;
}
I referred sample code snippet here as a response to one of the questions asked - it is what seems to be working when we pass one value. However, my issue is with passing a list instead of one value in the param. This is where it seems to fail.
The ListPreparedStatementSetter is not aware of parameters types. If a parameter is an array or a collection, it will set it as is to the first placeholder, leaving other placeholders unset. Hence the error. In your example, if List<Long> keys = Arrays.asList(1, 2), your sql statement will be:
SELECT A FROM MYTABLE WHERE KEY IN (?, ?)
If you pass your singletonMap to the ListPreparedStatementSetter, it will set the value of keys (which is of type List) to the first placeholder and that's it. The second placeholder will still be unset and the preparation of the statement will fail.
You can flatten parameters before passing them to the ListPreparedStatementSetter and it should work fine. I added a sample with how to flatten parameters before passing them to the prepared statement setter here (See flatten method).
Hope this helps.
I want to fetch large datasets from a MySql database, and return it as a List of comma separated Strings via a Webservice (not via a downloadable file, but directly as text).
Therefore I first need all selected rows in CSV format.
Question: What's the best way achieving this with Spring?
The following works with JdbcTemplate, but I don't know if this is the best approach (maybe optimize using Java 8 streams)?
Also if somehow feels wrong having to iterate the ResultSet and call rs.getString(i), concatenating each element of the ResultSet. Isn't there a more elegant way?
RowMapper<String> rowMapper = new RowMapper<String>() {
#Override
public String mapRow(ResultSet rs, int rowNum) throws SQLException {
StringBuilder sb = new StringBuilder();
for (int i = 1; i <= rs.getMetaData().getColumnCount(); i++) {
sb.append(rs.getString(i)).append(",");
}
return sb.toString();
}
};
String sql = "SELECT * from mypersons where age > 12";
List<String> list = new JdbcTemplate(dataSource).query(sql, params, rowMapper);
//...return the list via Webservice
Sidenote: I have to use native SQL for the select. They are much more complicated in my example above.
I recommend using Spring Data JPA to get a SET of persons. Once you have a set of persons, you can use streams to map the persons to the field you want to collect, and join that field with commas.
Assuming a Set, given a person has an attribute called name, this method will return a comma separated list of names.
public String joinName(Set<Person> persons) {
return persons.stream().map(Person::getName).collect(Collectors.joining(", "));
}
Your solution looks fine. I just have two possible improvements:
the indexed based access to the ResultSet already exists in the ColumnMapRowMapper. You could delegate to it and would get (pseudocode)
class CommaSeparatedStringRowMapper implements RowMapper<String> {
ColumnMapRowMapper delegate = new ColumnMapRowMapper();
#Override
public Map<String, Object> mapRow(ResultSet rs, int rowNum){
delegate.mapRow(rs, rowNum).valueSet().toStream.collect(Collectors)
}
Note: this will be less efficient, due to creating and accessing the intermediate map, but it looks nicer.
Note 2: The underlying map should preserve the order of columns, but you better double check.
Alternatively, you might consider using a RowCallbackHandler writing your results directly into a Webservice response thingy. If done right, part of your response might be on the way to your client while you are still processing the rest of the query result. So this might improve latency and memory consumption.
How to load data from JDBCTemplate.queryForMap() and it returns the Map Interface.How the maintained the query data internally in map.I trying to load but i got below exception i.e., org.springframework.dao.IncorrectResultSizeDataAccessException: Incorrect result
Code:-
public List getUserInfoByAlll() {
List profilelist=new ArrayList();
Map m=new HashMap();
m=this.jdbctemplate.queryForMap("SELECT userid,username FROM USER");
Set s=m.keySet();
Iterator it=s.iterator();
while(it.hasNext()){
String its=(String)it.next();
Object ob=(Object)m.get(its);
log.info("UserDAOImpl::getUserListSize()"+ob);
}
return profilelist;
}
Plz help me
queryForMap is appropriate if you want to get a single row. You are selecting without a where clause, so you probably want to queryForList. The error is probably indicative of the fact that queryForMap wants one row, but you query is retrieving many rows.
Check out the docs. There is a queryForList that takes just sql; the return type is a
List<Map<String,Object>>.
So once you have the results, you can do what you are doing. I would do something like
List results = template.queryForList(sql);
for (Map m : results){
m.get('userid');
m.get('username');
}
I'll let you fill in the details, but I would not iterate over keys in this case. I like to explicit about what I am expecting.
If you have a User object, and you actually want to load User instances, you can use the queryForList that takes sql and a class type
queryForList(String sql, Class<T> elementType)
(wow Spring has changed a lot since I left Javaland.)
I know this is really old, but this is the simplest way to query for Map.
Simply implement the ResultSetExtractor interface to define what type you want to return. Below is an example of how to use this. You'll be mapping it manually, but for a simple map, it should be straightforward.
jdbcTemplate.query("select string1,string2 from table where x=1", new ResultSetExtractor<Map>(){
#Override
public Map extractData(ResultSet rs) throws SQLException,DataAccessException {
HashMap<String,String> mapRet= new HashMap<String,String>();
while(rs.next()){
mapRet.put(rs.getString("string1"),rs.getString("string2"));
}
return mapRet;
}
});
This will give you a return type of Map that has multiple rows (however many your query returned) and not a list of Maps. You can view the ResultSetExtractor docs here: http://docs.spring.io/spring-framework/docs/2.5.6/api/org/springframework/jdbc/core/ResultSetExtractor.html
To add to #BrianBeech's answer, this is even more trimmed down in java 8:
jdbcTemplate.query("select string1,string2 from table where x=1", (ResultSet rs) -> {
HashMap<String,String> results = new HashMap<>();
while (rs.next()) {
results.put(rs.getString("string1"), rs.getString("string2"));
}
return results;
});
You can do something like this.
List<Map<String, Object>> mapList = jdbctemplate.queryForList(query));
return mapList.stream().collect(Collectors.toMap(k -> (Long) k.get("userid"), k -> (String) k.get("username")));
Output:
{
1: "abc",
2: "def",
3: "ghi"
}
I'm new to Java and just getting into querying databases. So far I have my results in a ResultSetMetaData. I'm think that for each row in the dataset I should add it to some form of collection? Can anyone tell me the best practice for this?
Thanks,
Jonesy
Create an object to hold the data. Loop through the resultset, creating an object for each one, and store them in an ArrayList or HashMap, depending on how you want to use the data. This allows you to close the database, and it gives you good objects on which you can build methods to manipulate the data.
It also allows you to write code that uses the object that doesn't need to rely on the database. If you ever want to pull out the database later and switch to text files or whatever, it's easy to do and you can still use the same objects and methods.
Usually we have a class with fields that correspond to a table. Then, whenever we have a (full) row in a result set, we create an instance of this class.
Example:
Consider a table created like this:
CREATE TABLE customer (First_Name char(50), Last_Name char(50),
Address char(50), City char(50), Country char(25), Birth_Date date);
A model class would be like this:
public class Customer {
private String firstName;
private String lastName;
private String address;
private String city;
private String country;
private Date date;
public String getFirstName() {
return firstName;
}
// getters for all fields
public void setFirstName(String firstName) {
this.firstName = firstName;
}
// setters for all fields
public String toString() {
return String.format("[%s, %s, %s, %s, %s, %s]", firstName,
lastName, address, city, country, date);
}
}
Now if you read data and have a ResultSet, you would create a new customer object and set the fields:
List<Customer> customers = new ArrayList<Customer>();
ResultSet rs = stmt.executeQuery("SELECT * from CUSTOMER;");
while (rs.next()) {
Customer customer = new Customer();
customer.setFirstName(rs.get("First_Name"));
// ... and so on
customers.add(customer);
}
A List seems quite logical. If you are not going to be having duplicates, and you are not bothered about the order of the results, then perhaps a Set.
A relevant implementation of List:
ArrayList: This is backed by an array, so lookups on particular indices should be quick
Relevant implementations of Set:
HashSet: Backed by a HashMap so O(1) insertion time
TreeSet: Respects the ordering of the data (using the compareTo method) - so iterating over the data will be in order - the trade off is O(log n) insertion time
You can create class which represents real world entities. Later if you wish to choose ORM technology/tool like hibernate you can use same classes.
First, the ResultSetMetaData class holds "information about the types and properties of the columns in a ResultSet object." So the results from your query are in the ResultSet, not in the ResultSetMetaData object.
You can see the Retrieving Values from Result Sets Java tutorial for information on how to extract your data from a ResultSet. You should be able to just loop through the ResultSet as shown and put your records in a List or Map, depending on how you want to access the data later.
I usually follow the same pattern as Andreas_D describes.
The object used to contain each row of data (in this case, the Customer class) is referred to as Data Transfer Object (TO).
The code that gets the database connection, queries the db, populates the TOs and returns them (typically in a List), is referred to as a Data Access Object (DAO).
You can read more about this design pattern here
Many of the answers above advice creating a class to hold the columns of a row and create Array list of the Class Objects. Shouldn't we also worry if the result set is huge though only fewer rows are being processed, would it not over consume memory unless the garbage collector reclaims at the same pace at which the objects are being created.
I had this issue with a ResultSet that had a few dozen columns, writing a class with so many members was way too much work for lazy me. So I iterated the ResultSet each field into a HashMap, the column label way the key and the field being the value. Then each put all the hashmaps from each row into a seperate list and all the lists into a master list.
Worked like a charm.
private ArrayList<ArrayList<HashMap<String, Object>>> allRecords = new ArrayList<>();
public MyTable getRecords()throws IOException, SQLException{
try{
String query = new Utils().readResourceFile("MyQuery.sql");
ResultSet rs = DBUtils.dbConnect().createStatement().executeQuery(query);
ResultSetMetaData rsmd = rs.getMetaData();
int columnCount = rsmd.getColumnCount();
while (rs.next()){
ArrayList<HashMap<String, Object>> row = new ArrayList<>();
for (int i = 1; i < columnCount + 1; i++) {
HashMap<String, Object> data = new HashMap<>();
Object field = rs.getObject(i);
data.put(rsmd.getColumnLabel(i), rs.wasNull()? "": field);
row.add(data);
}
allRecords.add(row);
}
}catch (IOException | ClassNotFoundException | SQLException e){
if(e instanceof SQLException){
DBUtils.printSQLException((SQLException) e);}
e.printStackTrace();
throw e;
}
return this;
}
And here is how I filtered the data:
public MyTable makeChanges(){
for(int i = 0; i < allRecords.size(); i++){
Date startDate = (Date) allRecords.get(i).stream().filter((HashMap<String, Object> field) -> field.containsKey("StartDate")).findFirst().get().get("StartDate");
int product = (int) allRecords.get(i).stream().filter((HashMap<String, Object> field) -> field.containsKey("pk_Product")).findFirst().get().get("pk_Product");
// now do something....
}
return this;
}