I have a question about keeping query result in array. For example I execute a query
SELECT * FROM some_table
Then I want to save it to array and create records. The table contains these columns:
id
user_name
last_name
The result array can be:
[[1, "First user name", "First last name"],
[2, "Second user name", "Second last name"]
...
].
Can you recommend me which array or data type should I use?
You do that like this:
Create a bean class for User
public class User {
private int id;
private String firstName;
private String lastName;
// getter and setter
...
}
And then query all the data from table, and create a User object and set the data.
List<User> users = new ArrayList<>();
while(someValue) {
...
int id = ...
String firstName= ...
String lastName = ...
User user = new User();
user.setId(id);
user.setFirstName(firstName);
user.setLastName(lastName);
users .add(user);
}
// after do what you want with the list
I extend your question to "the best way to keep select result" (with or without array).
It depends on:
how many results
how many fields
what do you want to do after ?
do you want to modify, put in your database again ?
So, several propositions:
just arrays: String[] fields1; String[] fields2, ...
array of arrays: String[][];
better collections: Vector, List or Set: do you want them to be sorted ?, how do you pick them after ? Or Map, (if you want to keep index => data)
or Object you create yourself. For this, you even have tools to map object-database.
you should take a look at these features, and what you want to do .
Hope it helps.
Related
Assuming i have a database table 'UserInformation' with thousands of entries.
id
user_id
type
value
1
1
A
value_1
2
1
B
value_2
3
2
A
value_3
4
2
B
value_4
5
2
C
value_5
...
...
...
...
For simplicity, my task is to select all entries with a specific user_id and type A or B and assign the values to an UserModel Object (in Java).
But i have to select those information for thousands of user ids (e.g. user = 10.000 entries).
I know two ways of achieving this:
1.
public List<UserModel> getUsers(List<User> users){
List<UserModel> userModels = new ArrayList<>();
for (User user : users){
String valueA = findValueByUserIdAndType(user.getId(), "A");
String valueB = findValueByUserIdAndType(user.getId(), "B");
userModels.add(new UserModel(user.getId(), valueA, valueB);
}
return userModels;
}
public List<UserModel> getUsers(List<User> users){
List<UserModel> userModels = new ArrayList<>();
List<UserInformation> entries = findAllByUserIdInAndTypeIn(userIds, List.of("A", "B");
Map<Long, List<UserInformation>> entriesMappedByUserId = groupByUserId(entries);
for (User user : users){
UserInformation userInformation = entriesMappedByUserId.get(user.getId());
userModels.add(new UserModel(user.getId(), userInformation);
}
return userModels;
}
I know the first method yields in 20.000 database queries. But the second one leads me to two questions:
Is there a better way than transforming the database result into a map and retrieving the values based on an identifier
Assuming the second database query uses an in clause in the sql query (where user_id in (1,2,3,...,10000), at which number do i break my database
And the resulting question: Is there a better way to select a huge number of entries?
Edit: Assume the User table and UserInformation table can't be joined because the information is stored on two different places.
Consider this code snippet below:
List<String> usersList = Arrays.asList("john", "jack", "jill", "xxxx", "yyyy");
List<User> userEntities = userRepo.findAllById(usersList);
User class is a simple Entity object annotated with #Entity and has an #Id field which is of String datatype.
Assume that in db I have rows corresponding to "john", "jack" and "jill". Even though I passed 5 items in usersList(along with "xxxx" and "yyyy"), findAllById method would only return 3 items/entities corresponding to "john","jack",and "jill".
Now after the call to findAllById method, what's the best, easy and efficient(better than O(n^2) perhaps) way to find out the missing items which findAllById method did not return?(In this case, it would be "xxxx" and "yyyy").
Using Java Sets
You could use a set as the source of filtering:
Set<String> usersSet = new HashSet<>(Arrays.asList("john", "jack", "jill", "xxxx", "yyyy"));
And now you could create a predicate to filter those not present:
Set<String> foundIds = userRepo.findAllById(usersSet)
.stream()
.map(User::getId)
.collect(Collectors.toSet());
I assume the filter should be O(n) to go over the entire results.
Or you could change your repository to return a set of users ideally using a form of distinct clause:
Set<String> foundIds = userRepo.findDistinctById(usersSet)
.stream()
.map(User::getId)
.collect(Collectors.toSet());;
And then you can just apply a set operator:
usersSet.removeAll(foundIds);
And now usersSet contains the users not found in your result.
And a set has a O(1) complexity to find an item. So, I assume this should be O(sizeOf(userSet)) to remove them all.
Alternatively, you could iterate over the foundIds and gradually remove items from the userSet. Then you could short-circuit the loop algorithm in the event you realize that there are no more userSet items to remove (i.e. the set is empty).
Filtering Directly from Database
Now to avoid all this, you can probably define a native query and run it in your JPA repository to retrieve only users from your list which didn't exist in the database. The query would be somewhat as follows that I did in PostgreSQL:
WITH my_users AS(
SELECT 'john' AS id UNION SELECT 'jack' UNION SELECT 'jill'
)
SELECT id FROM my_users mu
WHERE NOT EXISTS(SELECT 1 FROM users u WHERE u.id = mu.id);
Spring Data: JDBC Example
Since the query is dynamic (i.e. the filtering set could be of different sizes every time), we need to build the query dynamically. And I don't believe JPA has a way to do this, but a native query might do the trick.
You could either pack a JdbcTemplate query directly into your repository or use JPA native queries manually.
#Repository
public class UserRepository {
private final JdbcTemplate jdbcTemplate;
public UserRepository(JdbcTemplate jdbcTemplate) {this.jdbcTemplate = jdbcTemplate;}
public Set<String> getUserIdNotFound(Set<String> userIds) {
StringBuilder sql = new StringBuilder();
for(String userId : userIds) {
if(sql.length() > 0) {
sql.append(" UNION ");
}
sql.append("SELECT ? AS id");
}
String query = String.format("WITH my_users AS (%sql)", sql) +
"SELECT id FROM my_users mu WHERE NOT EXISTS(SELECT 1 FROM users u WHERE u.id = mu.id)";
List<String> myUsers = jdbcTemplate.queryForList(query, userIds.toArray(), String.class);
return new HashSet<>(myUsers);
}
}
Then we just do:
Set<String> usersIds = Set.of("john", "jack", "jill", "xxxx", "yyyy");
Set<String> notFoundIds = userRepo.getUserIdNotFound(usersIds);
There is probably a way to do it with JPA native queries. Let me see if I can do one of those and put it in the answer later on.
You can write your own algorithm that finds missing users. For example:
List<String> missing = new ArrayList<>(usersList);
for (User user : userEntities){
String userId = user.getId();
missing.remove(userId);
}
In the result you will have a list of user-ids that are missing:
"xxxx" and "yyyy"
You can just add a method to your repo:
findByIdNotIn(Collection<String> ids) and Spring will make the query:
See here:
https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods
Note (from the docs):
In and NotIn also take any subclass of Collection as aparameter as well as arrays or varargs.
I have a List of Strings which contains search items for multiple attributes in an Entity. I would like to Query the database and retrieve any Entity which has one of these Strings in any one of the specified attributes. For instance, say I have a List of partial names:
//contents of the list (for visual purposes)
["Mi", "Chr", "Leo", "Jo", "Par"]
Also, let us assume that we do not know the size of the list. Now, say I have an Entity with the following attributes:
String firstName;
String lastName;
How can I retrieve (preferably in one Query) all Entities whose first OR last names contain any of the Strings within the list?
Something like (Pseudo code):
SELECT u FROM User u WHERE u.firstName LIKE (%Mi% OR %Chr% OR %Leo% OR %Jo% OR %Par%)
OR u.lastName LIKE (%Mi% OR %Chr% OR %Leo% OR %Jo% OR %Par%)
If there were something like a LIKE IN clause that'd be perfect but, unfortunately, I don't believe JPQL supports that. So, how could this be done?
This code builds a query using the IN condition.
String query = "";
String whereClause = "";
for(int i = 0; i < LIST.size(); i++) {
// using * wildcard
whereClause += "\"*" + LIST.get(i).toString() + "*\",";
}
query = String.format("SELECT User.firstName, User.lastName FROM User WHERE User.firstName IN (%s) OR User.lastName IN (%s);", whereClause, whereClause);
I have a class that looks like this. I need to populate it from two database tables, which are also shown below. Is there any preferred way to do this?
My thought is to have a service class to select a List<> via a ResultSetExtractor from a DAO. Then do a foreach on that list, and select a List<> of emails for the individual person via another ResultSetExtractor, and attach it from with the foreach loop.
Is there a better way, or is this as good as it gets?
public class Person {
private String personId;
private String Name;
private ArrayList<String> emails;
}
create table Person (
person_id varchar2(10),
name varchar2(30)
);
create table email (
person_id varchar2(10),
email varchar2(30)
);
This is best solved by an ORM. With JDBC, you have to do by hand what an ORM would do for you. Executing N + 1 queries is very inefficient. You should execute a single query, and build your objects manually. Cumbersome, but not hard:
select person.id, person.name, email.email from person person
left join email on person.id = email.person_id
...
Map<Long, Person> personsById = new HashMap<>();
while (rs.next()) {
Long id = rs.getLong("id");
String name = rs.getString("name");
String email = rs.getString("email");
Person person = personsById.get(id);
if (person == null) {
person = new Person(id, name);
personsById.put(person.getId(), person);
}
person.addEmail(email);
}
Collection<Person> persons = personsById.values();
I was looking for something similar, and although the answer is perfectly valid I went with this nice library instead https://simpleflatmapper.org/0203-joins.html
It also integrates perfectly with Spring boot.
main advantage is that you have a clean repository layer, it uses your pojo and makes refactoring much easier, and like hibernate you can still map deep nested and complex one to many and still be in control of what is executed.
It also has a nice jdbctemplate CRUD and Java 13 finally brings support for multi-line string literals which is very good for sql statements readability. hope this helps someone :)
In my case, I had to use the LinkedHashMap to keep the query result ordered by the position field.
From JavaDoc:
LinkedHashMap: "This linked list defines the iteration ordering, which is normally the order in which keys were inserted into the map (insertion-order). Note that insertion order is not affected if a key is re-inserted into the map."
HashMap: "This class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time".
TIP: using the getOrDefault method eliminates the extra check for nullable object.
public List<BucketDto> findAll() {
var sql = """
SELECT
b.uuid bucket_uuid, b.position bucket_position, b.name bucket_name,
c.uuid card_uuid, c.position card_position, c.name card_name
FROM bucket AS b
LEFT JOIN card AS c ON c.bucket_id = b.id
ORDER BY b.position ASC, c.position ASC
""";
return jdbcTemplate.query(sql, rs -> {
Map<Double, BucketDto> resultMap = new LinkedHashMap<>();
while (rs.next()) {
var position = rs.getDouble("bucket_position");
var bucketDto = resultMap.getOrDefault(position, new BucketDto(
UUID.fromString(rs.getString("bucket_uuid")),
position,
rs.getString("bucket_name")));
if (Optional.ofNullable(rs.getString("card_uuid")).isPresent()) {
bucketDto.addCard(new CardDto(
UUID.fromString(rs.getString("card_uuid")),
rs.getDouble("card_position"),
rs.getString("card_name")));
}
resultMap.put(position, bucketDto);
}
return new ArrayList<>(resultMap.values());
});
}
Hi I run a simple select query in oracle over a table and get a resultset. like select username, responsibility, project from mytable.
The resultset contains user details. there are multiple rows returned for each username with different values for responsibility and project.
Now I want to get a list of lists from this resultset which has one List per username and distinct values are concatenated in a comma seperated string.
So if Sam has multiple entries in the resultset then the output of my operation should give me:
UserList =
["Sam", "responsibility1,responsibility2,responsibility3...", "dept1,dept2,dept3.."],
[Some other User],
[and so on..]
Later I will write this to a csv file.
I cannot do this in the query itself for compatibility reasons, we have to support multiple databases, versions in future.
How do I do this in java or groovy?
Thanks
Java is quite easy.
You need a class to model each user.
You need a Map of username to User.
Each User contains a List of responsibility and a List of departments.
Then you iterate your resultset, find the User from the map on each row and add the responsibility and department to that User
Do you need the code or is that good enough?
HTH
Edit: Here's some Java starting code:
(Not checked for syntax or mistakes ;] )
public class User {
private final List<String> responsibility = new ArrayList<String>();
private final List<String> department = new ArrayList<String>();
...standard getters and setters
}
// Your code to do the read
public void executeRead() {
... obtain the resultset somehow here
Map<String, User> usernameToUser = new HashMap<String, User>():
while (rs.next) {
String username = rs.getString("username");
User user = usernameToUser.get(username);
if (user == null) {
user = new User(); // Create and remember a user the first time you see them
usernameToUser.put(username, user);
}
String responsiblity = rs.getString("responsiblity");
String department = rs.getString("department");
user.addResponsibility(responsibility);
user.addDepartment(department);
}
rs.close();
// Now you have the data structure of users in memory you can output
// it in whichever format you like e.g. HTML, CSV, etc
// Probably best to do this step in a totally separate place that can
// be switched out for a different output format in future.
}