We Keep Coding

What is the most efficient way to retrieve specific data from DB without doing loop for all records?

I want to ask about what is the most efficient way to search about specific data from a database without doing a for loop in all of the records?
I have a project on java spring and I have this Entity:
#Entity
#Table(name = "USERS") public class USERS {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "UID")
private Integer id;
#Column(name = "FName")
private String firstName;
#Column(name = "SName")
private String secondName;
#Column(name = "TName")
private String thirdName;
#Column(name = "LName")
private String fourthName;
#Column(name = "Email")
private String email;
#Column(name = "PW")
private String password;
#Column(name = "MNumber")
private String mobileNumber;
#Column(name = "ISDeleted")
private boolean isUserDeleted;
//---------------------- Getters and Setters ----------------------
and I made this service:
public List<USERS> findAllActive() {
List<USERS> usersList = new ArrayList<USERS>();
for (USERS users: usersRepository.findAll()){
if (!users.isUserDeleted()){
usersList.add(users);
}
}
return usersList;
}
For example; I have one property for User, if he is active or not.
So, my question; what is the most efficient way to do get specific data like retrieving all of the active users from the DB without doing a for loop like in the code above? Because if the list of users is a 1 Million or more, it could have performance issues.

Assuming that you are using JpaRepository then you can create custom query.
#Query("SELECT u FROM USERS u WHERE u.userDeleted = false")
List<USERS> findNotDeletedUsers();
and then call usersRepository.findNotDeletedUsers();

First of all, use an index on the field you want to search on (this won't help you much if the column has only two distinct values, but will make a huge difference if the value has high sparsity).
#Entity
#Table(name = "USERS",
indexes = {
// not a huge performance gain, since the column values are true/false
#Index(name = "index_by_active", columnList="ISDeleted", unique = false),
// possible huge performance gain, since only the relevant records are scanned
#Index(name = "index_by_first_name", columnList="FName", unique = false)})
public class USERS {...}
Then, define a query method that uses the indexed field (if you are using spring data it would look as follows).
public interface UsersRepository extends CrudRepository<USERS, Long> {
List<USERS> findUsersByISDeleted(boolean deleted);
List<USERS> findUsersByFName(String name);
List<USERS> findUsersByFNameAndISDeleted(String name, boolean deleted);
}
Queries on indexed fields will leverage the underlying index and provide an efficient access plan (so you won't end up scanning the whole table in order to extract a subset of entities matching a given criteria).

The solution from #Madis is okay. But if you always want to get users which are not deleted in all queries, you can specify it on Entity:
#Entity
#Table(name = "USERS")
#Where("ISDeleted = false")
public class USERS {
So now the condition "ISDeleted = false" is automatically append to all queries from the UserRepository. You can use usersRepository.findAll() instead of.

You don't need to specify any sql query or where clause. CrudRepository will do it for you automatically. Just use below code and pass true/false on need basis
List<Users> findIsUserDeleted(boolean isDeleted)

Can an entity be updated instead of being inserted using spring-data-jpa when entity ID is not known without issuing a query?

We have a Spring Boot/Data-JPA (1.3.3.RELEASE) application using Hibernate implementation where a CSV file is read and inserted into a database table called FIRE_CSV_UPLOAD. For records that are already present we just update them.
We retrieve record ID by querying for unique key (a combination of three columns) but this approach is inefficient for thousands of record in CSV file.
My question is how to update record without querying the table for unique key? If I do not query for ID then the record will be inserted instead of update.
I know one way which is to retrieve all records and store their unique key and ID pairs in a Map. Any other suggestions are very much appreciated. The codes are as below,
Note: They are minimized for brevity.
#Entity
#Table(name = "FIRE_CSV_UPLOAD",
uniqueConstraints={#UniqueConstraint(columnNames = {"account_number" , "account_type", "bank_client_id"})})
public class FireCsv {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#NotNull
#Column(name="account_number")
private String accountNumber;
#NotNull
#Column(name="account_type")
private String accountType;
#NotNull
#Column(name="bank_client_id")
private String bankClientIdNumber;
...
other fields/getters/setters
}
--
public interface FireCsvRepository extends JpaRepository<FireCsv, Long> {
#Query("select u from FireCsv u where u.accountNumber = :accountNumber and u.accountType = :accountType and u.bankClientIdNumber = :bankClientIdNumber ")
FireCsv findRecord(#Param("accountNumber") String accountNumber,
#Param("accountType") String accountType,
#Param("bankClientIdNumber") String bankClientIdNumber);
}
--
#Service
public class FireCsvServiceImpl implements FireCsvService {
other fields/methods
...
#Override
#Transactional
public FireCsv save(final FireCsv fireCsv) {
FireCsv existingFireCsv = fireCsvRepository.findRecord(fireCsv.getAccountNumber(), fireCsv.getAccountType(), fireCsv.getBankClientIdNumber());
// If record exist then mark as update, if not as insert
if (existingFireCsv != null) {
fireCsv.setId(existingFireCsv.getId());
fireCsv.setRecordStatus(CSVUploadRecordStatus.CSV_UPDATE.getStatus());
}
else {
fireCsv.setRecordStatus(CSVUploadRecordStatus.CSV_INSERT.getStatus());
}
fireCsv.setRecordStatusDate(new java.sql.Timestamp(new Date().getTime()));
return fireCsvRepository.save(fireCsv);
}
}

You have to read before deciding to make an update or insert, I dont think there is a way around it.
To make that faster you should add an index to your database
using the three columns "account_number", "account_type", "bank_client_id".
Alternatively you can try to use an composite id using #IdClass as shown in
tutorial-jpa-composite-primary-key
The JPA provider should than automatically create the index for it.

#OneToOne field value not loaded after save

I have a Spring Data JPA Project, my entity is the following
#Entity
#Table(name = "delivery_offers")
public class DeliveryOffer {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
#OneToOne
#JoinColumn(name = "DELIVERY_ID")
private Delivery delivery;
... other fields
The table also has a foreign key for DELIVERY_ID which references to DELIVERY.ID
My Dao is a JpaRepository object, I try to persist a DeliveryOffer with dao.saveAndFlush (deliveryOffer).
I also set delivery.id = xy
I would like to return the persisted DeliveryOffer object, so I make in my controller
return dao.saveAndFlush(deliveryOffer)
The problem is, that in the returned DeliveryOffer the referenced Delivery object fields are all null except of the id, which I set for deliveryOffer.delivery.id (the Delivery exists in the DB with the given id)
If I try to get an "older" deliveryOffer from a recent transaction with findOne, the referenced delivery is then filled correctly.
So I suppose, it has to do something with the transaction. I thought, saveAndFlush commits it immediately, so in the return object I would await the correct entity.
Do you have an idea, why the delivery is empty at this point?

Why I'm getting Javassist objects in the middle of a result set?

I'm having an strange problem when I try to retrieve some entities from the database. The table where the entities lives just have 4 rows. When I try select all rows I get a list where the first and the last elements are loaded correct, however, the second and the third has all properties as null. Here is a print of my debug console:
The entity is simple, as you can see below:
#Entity
#Table(name = "Empresa")
public class Empresa implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "ID_EMPRESA")
private Integer idEmpresa;
#Basic(optional = false)
#Column(name = "NOME_EMPRESA")
#OrderColumn
private String nomeEmpresa;
#Column(name = "CNPJ")
private String cnpj;
#OneToMany(cascade = CascadeType.ALL, mappedBy = "iDEmpresa", fetch = FetchType.LAZY)
private List<Cadastro> cadastroList;
}
If you want know how I am retrieving the entities, here is the code:
#Override
public List<T> recuperarTodos() {
Query query = entityManager.createQuery(criarQueryRecuperarTodos());
limitarQuantidadeDeRegistros(query);
return query.getResultList();
}
private String criarQueryRecuperarTodos() {
StringBuilder builder = new StringBuilder("SELECT e FROM ");
builder.append(classe.getSimpleName());
builder.append(" e");
builder.append(criarParametrosOrdenacao());
return builder.toString();
}

This is perfectly legal and expected situation. Hibernate uses dynamically generated proxies (hence javaassist objects, in the past hibernate used cglib as well) as placeholders for not fully fetched entities to allow lazy fetching. Because of this, generally speaking, you should not attempt to access attribute values directly. Using getters instead allows hibernate to issue an appropriate DB query and fill the entity. This can be a problem in some situations - for example, if the values are first requested outside the Hibernate session.

JPA primary key value is always 0

I have a post class and it kind of works, but there's one problem: the primary key doesn't increase.
#Entity
#Table(name="posts")
public class Post extends GenericModel{
#Id
#Column(name="post_id")
public int id;
#Column(name="post_situation")
public String situation;
#Column(name="post_date")
public Date date;
#Column(name="post_userid")
public int userid;
#OneToMany(mappedBy="post", cascade=CascadeType.ALL)
public List<Block> blocks;
public Post addBlock(String content, int position){
Block b = new Block(this, content, position);
b.save();
this.blocks.add(b);
this.save();
return this;
}
public Post(String situation, Date date){
this.situation = situation;
this.date = date;
this.userid = 2;
}
}
When I call it the first time on an empty table, it works fine, but the second time, I'm getting PersistenceException occured : org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
The post_id column always has 0. Any idea how to fix this? I have the #Id annotation in palce..
This is how I have in my controller:
Post p = new Post("Midden in het middenoosten.", new Date()).save();
Any ideas what's causing this problem?

It seems that you want the primary key values to be auto-generated. If that is the case, if you'll need to add the #GeneratedValue annotation to the id attribute, in addition to the #Id annotation. Your code should therefore be:
#Id
#Column(name="post_id")
#GeneratedValue
public int id;
There are several strategies available to generate the Ids. You would have to read up on those to decide if you want to choose the TABLE -based, SEQUENCE -based or the IDENTITY -based strategy (which depends on what your database supports). If you choose a strategy explicitly, the define strategy will be used, instead of the default AUTO strategy. Explicit strategy decisions, are communicated in code as:
#Id
#Column(name="post_id")
#GeneratedValue(strategy=SEQUENCE, generator="POST_SEQ")
public int id;
Without generated values, the default value for integers in Java, i.e. 0 will be persisted for the post_id column. Due to the primary key constraint, you cannot have a second row with the same key, resulting in the described failure.

There are several strategies available to generate id:
GenerationType.AUTO
GenerationType.SEQUENCE
GenerationType.IDENTITY
GenerationType.TABLE
If you want the primary key values to be auto-generated, use GenerationType.AUTO, it works with MySQL.