We Keep Coding

Postgres Hibernate set session variables for row level security

I am having trouble finding information about this issue I am running into. I am interested in implementing row level security on my Postgres db and I am looking for a way to be able to set postgres session variables automatically through some form of an interceptor. Now, I know that with hibernate you are able to do row-level-security using #Filter and #FilterDef, however I would like to additionally set policies on my DB.
A very simple way of doing this would be to execute the SQL statement SET variable=value prior to every query, though I have not been able to find any information on this.
This is being used on a spring-boot application and every request is expected to will have access to a request-specific value of the variable.

Since your application uses spring, you could try accomplishing this in one of a few ways:
Spring AOP
In this approach, you write an advice that you ask spring to apply to specific methods. If your methods use the #Transactional annotation, you could have the advice be applied to those immediately after the transaction has started.
Extended TransactionManager Implementation
Lets assume your transaction is using JpaTransactionManager.
public class SecurityPolicyInjectingJpaTransactionManager extends JpaTransactionManager {
#Autowired
private EntityManager entityManager;
// constructors
#Override
protected void prepareSynchronization(DefaultTransactionStatus status, TransactionDefinition definition) {
super.prepareSynchronization(status, definition);
if (status.isNewTransaction()) {
// Use entityManager to execute your database policy param/values
// I would suggest you also register an after-completion callback synchronization
// This after-completion would clear all the policy param/values
// regardless of whether the transaction succeeded or failed
// since this happens just before it gets returned to the connection pool
}
}
}
Now simply configure your JPA environment to use your custom JpaTransactionManager class.
There are likely others, but these are the two that come to mind that I've explored.

How to setup Spring Boot to close connection pool after #JdbcTest class?

We have large system using Postgresql DB, with rather complex database structure. And we have many DB-related integration tests for that system.
Because of the complex DB structure and usage of postres specific sql in code, mocking postgres with H2 (or other in memory DB) seems highly unreliable.
So, we are using junit tests of the following structure:
#RunWith(SpringRunner.class)
#JdbcTest
#AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
#Sql( ... schema creation, sample data, etc )
#ContextConfiguration(classes = ... dao and service classes used in the test)
Everything is OK, when you have 2-3 test classes. Problems start to arise when you have 10+ test classes. As I understand it, SpringBoot creates separate connection pool for every distinct context configuration. To keep tests isolated as much as possible, we usually include in context configuration only components, that are used inside the test. So SpringBoot creates dozens of connection pools, that leads to "too many connection"-type errors from connection pool or jdbc driver. You can run your tests one by one, but you cannot run them all at once (so, say farewell to CI).
We are using the following workaround. The following snippet is copy-pasted to every test class:
// <editor-fold name='connection leaking fix'
#Autowired
private DataSource dataSource;
private static HikariDataSource hikariDataSource;
#Before
public void saveDataSource() {
this.hikariDataSource = (HikariDataSource)dataSource;
}
#AfterClass
public static void releaseDataSource() {
if (hikariDataSource != null) {
hikariDataSource.close();
}
}
// </editor-fold>
It works, but you have to remember that you shouldn't paste that snippet to test classes that use the same context configuration.
The question - is there any way to tell spring boot to close connection pool after each test class execution, or any way to limit number of connection pools spring boot creates?

#M.Deinum is right, the only way to solve the problem without hacking some workaround is to use limited number of configurations. So you can use something like this to test just DAO layer:
#RunWith(SpringRunner.class)
#JdbcTest(includeFilters = #ComponentScan.Filter(Repository.class))
#AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
#Sql(...)
or that something like this to test DAO and service layer:
#RunWith(SpringRunner.class)
#JdbcTest(includeFilters = {
#ComponentScan.Filter(Repository.class),
#ComponentScan.Filter(Service.class)
})
#AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
#Sql(...)

Spring Data: Service layer unit testing

In my project I'm having trouble doing unit testing. One issue is that just doing an integration test is much faster to write and also tests that the components actually work together. Unit testing novel "algorithms" or so seems much easier. Unit Testing service classes it just feels wrong and useless.
I'm using mockito to mock spring data repository (and hence DB access). The thing is if i tell the mocked repository to return entity A on method call getById it will obviously return that and the service will return it too. Yes, the service does some extra stuff, but very minor things, like load lazy collections (from hibernate). Obviously I don't have any lazy collections (proxies) in a unit test.
Example:
#Test
public void testGetById() {
System.out.println("getById");
TestCompound expResult = new TestCompound(id, "Test Compound", "9999-99-9", null, null, null);
TestCompoundRepository mockedRepository = mock(TestCompoundRepository.class);
when(mockedRepository.findOne(id)).thenReturn(expResult);
ReflectionTestUtils.setField(testCompoundService, "testCompoundRepository",
mockedRepository, TestCompoundRepository.class);
TestCompound result = testCompoundService.getById(id);
assertEquals(expResult, result);
}
hooray, the rest succeeds. What a surprise! Not really no.
Can some one explain to me what I'm doing wrong? Or else what the point of such a test is? I mean I tell to return expResult and then it is returned. Wow. What a surprise! Feels like I'm testing if mockito works and not my Service.
EDIT:
The only benefit I see if some were stupid error happens like leaving an unwanted line there that sets return value to null or something similar stupid. Such cases would be caught by the unit test. Still the "reward-effort" ratio seems bad?

Question might be a bit old but I will put an answer in case someone stumbles across.
I'm using Mockito and JUnit.
AccountRepository is a plain spring data repository extending JPARepository.
Account is a plain JPA entity.
To test your services and mock Spring Data repositories, you need something like below.
package foo.bar.service.impl;
import foo.bar.data.entity.Account;
import foo.bar.data.repository.AccountRepository;
import foo.bar.service.AccountService;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.runners.MockitoJUnitRunner;
#RunWith(MockitoJUnitRunner.class)
public class AccountServiceImplTest {
#Mock
private static AccountRepository accountRepository;
#InjectMocks
private static AccountService accountService = new AccountServiceImpl();
private Account account;
#Test
public void testFindAccount() {
Integer accountId = new Integer(1);
account = new Account();
account.setId(accountId);
account.setName("Account name");
account.setCode("Accont code");
account.setDescription("Account description");
Mockito.when(accountRepository.findOne(accountId)).thenReturn(account);
Account retrivedAccount = accountService.findAccount(accountId);
Assert.assertEquals(account, retrivedAccount);
}
}

One of the reasons I like testing my Spring Data repositories is to test that I have defined my JPA mappings correctly. I do not use a mocking framework for these tests, I use the Spring Test framework which actually bootstraps the container allowing me to autowire the actual repository into the Junit test so that I may execute tests against it.
I agree with your thoughts that mocking the repository is pretty useless. Since your using Spring I would suggest leveraging the Spring Test framework to perform real tests against your repositories, which can be executed against an embedded database such as H2 in a more unit test based fashion or your actual database implementation such as Oracle or MySql, to conduct more of an integration test. (Execute these against a copy of a development database) These tests will reveal fallacies in your JPA mappings and other items such as improper cascades setup in the database.
Here is an example of one of my tests on GitHub. Notice how the framework actually autowires the repository into the test. The repository also contains an example of how to configure the Spring Test framework, which I have also demonstrated in this blog post.
In conclusion, I do not believe you will receive any of the benefits of testing a repository that I have discussed from using a mock of the repository.
One additional note I wanted to add, is that mocks are not really intended for use in the actual class under test. Their use is for providing required dependencies to a class under test.

You can use this library: https://github.com/agileapes/spring-data-mock
This will mock your repository for you, while allowing you to implement custom functionality for any method as well as your native query methods.

You exactly right. It is clear unit test. And it will never fail (so, it is useless) I think you need at integration test to test real JPA repository with real database (H2 in memory for example) (as I always do).
And it is better to test your services (theirs interfaces). If after some time you will change your storage (to Mongo for example) - you will be able to use your service tests to ensure all works as before.
After some time you will be suprised how many DB\JPA-related problems (constraints, optimistic locks, lazy-loading, duplicate id, some hibernate issues and so on) you find.
Also, try to develop via tests - not just write test after implementation. Instead before creation of new method in service - create test for it, implement service method and only after just recheck it in real application. At least it is much faster to start test than a server.
So, do not create tests to have a lot of them. Find how they may help you.
Usage of mocks for repositories is not good idea. Test how your services work together with Hibernate\JPA\Database. Most part of problems is located beetwen layers.

You can mock the repository and inject it to the service, this is the way; but, if you just instantiate the service with #Mock of repositories, it would be better, if you define the repositories as private final fields in the service and use a constructor of all repositories. That way, if you add another repository to the service, the test will fail and you have to change it, which is the purpose.
Imagine this service:
class OrderService {
private final UserRepository userRepos;
public OrderService(UserRepository userRepos) {
this.userRepos = userRepos;
}
...
}
And this test:
class OrderServiceTests {
#Mock
private UserRepository userRepos;
private OrderService service;
private OrderServiceTests() {
this.service = new OrderService(this.userRepos);
}
}
Now, if we add another dependency to the service:
class OrderService {
private final UserRepository userRepos;
private final AddressRepository addRepos;
public OrderService(UserRepository userRepos, AddressRepository addRepos) {
this.userRepos = userRepos;
this.addRepos = addRepos;
...
}
The previous test will fail because the constructor has changed. If you use #InjectMocks this will not happen; the injection happens behind the curtain and we are not clear what happens; this may not be desirable.
Another thing is, I don't agree that integration test will cover all the cases that unit tests will cover; it may but not always the case. Even the controller can be unit-tested with mocks; after all the tests are meant to cover all the code we have written, so they must be fine-grained; imagine when we follow TTD and we only complete the controller and services level: how we proceed without controller unit testing?

Assuming that we have the below Service class
#Service
public class EmployeeServiceImpl implements EmployeeService {
#Autowired
private EmployeeRepository employeeRepository;
#Override
public Employee getEmployeeByName(String name) {
return employeeRepository.findByName(name);
}
}
Test class:
#RunWith(SpringRunner.class)
public class EmployeeServiceImplIntegrationTest {
#TestConfiguration
static class EmployeeServiceImplTestContextConfiguration {
#Bean
public EmployeeService employeeService() {
return new EmployeeServiceImpl();
}
}
#Autowired
private EmployeeService employeeService;
#MockBean
private EmployeeRepository employeeRepository;
// write test cases here
}
To check the Service class, we need to have an instance of Service class created and available as a #Bean so that we can #Autowire it in our test class. This configuration is achieved by using the #TestConfiguration annotation.
During component scanning, we might find components or configurations created only for specific tests accidentally get picked up everywhere. To help prevent that, Spring Boot provides #TestConfiguration annotation that can be used on classes in src/test/java to indicate that they should not be picked up by scanning.
Another interesting thing here is the use of #MockBean. It creates a Mock for the EmployeeRepository which can be used to bypass the call to the actual EmployeeRepository:
#Before
public void setUp() {
Employee alex = new Employee("alex");
Mockito.when(employeeRepository.findByName(alex.getName()))
.thenReturn(alex);
}
After the setup, we can easily test our service like:
#Test
public void whenValidName_thenEmployeeShouldBeFound() {
String name = "alex";
Employee found = employeeService.getEmployeeByName(name);
assertThat(found.getName())isEqualTo(name);
}
For more in depth knowledge check:
https://www.baeldung.com/spring-boot-testing

Spring Transaction Management Test

I want to test my Dao Class using the SpringContextTests.
In my method class I extended the AbstractTransactionalJUnit4SpringContextTests in order for my test class to integrate with JUnit4. I have also set up the configurations and made the initialization and database clean up in the #Before and tearDown in the #After. My test class works perfectly.
My problem was, when I run my test class and the database is filled with data, the original data was not rolled back and my database is cleared. In the #Before method, I clear the database and populate data, thinking that I will be able to rollback it but its not.
Can anyone site an example that works and rollbacks information in the database.
ADDONS:
Every database manipulation in my test methods are rolled back. But the execution of super.deleteFromTables("person") in the #Before method did not rollback all the previous data from the database.
Spring rollbacks all the CRUD operations but the database clean up before the transaction do not rollback.

Thank you to all those who answered my question. I learned a lot from those answers but it didn't solve my problem.
I knew my test data does a transaction management and it does its job properly.
The mistake is on my part.
I forgot the lesson about database commands that when you execute a DDL statement after a DML statement, it will automatically commit the transaction. I executed a DDL after a DML by deleting all record and then ALTER the AUTO_INCREMENT of the table where in it will cause an auto-commit and delete all records of the table permanently.
FIXING THAT SCENARIO SOLVED MY PROBLEM.

Possible causes:
you're using a database/database engine which does not have proper transactions;
you're using multiple transaction managers and/or data sources and the proper one is not picked up;
you're doing your own, separate, transactions in the test class
As for an example, here's one ( top of my head, not compiled )
public class DBTest extends AbstractTransactionalJUnit4SpringContextTests {
#Autowired
private SomeDAO _aBeanDefinedInMyContextFile;
#Test
public void insert_works() {
assert _aBeanDefinedInMyContextFile.findAll() == 0;
_aBeanDefinedInMyContextFile.save(new Bean());
assert _aBeanDefinedInMyContextFile.findAll() == 1;
}
}
Key points:
the SomeDAO is an interface which corresponds to a bean declared in my context;
the bean does not have any transactional settings ( advice/programmatic), it relies on the caller being transactional - either the service in production, or the test in our situation;
the test does not include any transactional management code, as it's all done in the framework.

I'm not sure what is wrong with your class. Here is an extract of a class that does what you want with dbunit and spring 2.5:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations={
"testContext.xml"})
#TransactionConfiguration
#Transactional
public class SampleDAOTest {
#Autowired
private DataSource dataSource;
#Autowired
private SampleDAO sampleDAO;
#Before
public void onSetUpInTransaction() throws Exception {
//Populate Test data
IDatabaseConnection dbUnitCon = new DatabaseConnection(DataSourceUtils.getConnection(dataSource), "DATASOURCE");
//read in from a dbunit excel file of test data
IDataSet dataSet = new XlsDataSet(new File("src/test/resources/TestData.xls"));
DatabaseOperation.INSERT.execute(dbUnitCon, dataSet);
}
#Test
public void testGetIntermediaryOrganisation() {
// Test getting a user
User object = sampleDAO.getUser(99L);
assertTrue(object.getValue);
}
}
One of the benfits of this method is that you don't need to extend any classes. So you can still have your own hierarchy for tests.
If you really want to stick to your current method instead of using the #before annotation I thinnk you need to overide the below method and put your setup code in there.
#Override
public void onSetUpInTransaction() throws Exception {...}
Hope this helps

Sidestepping your question, I suggest that you use a seperate database instance to run your tests against. That way, you can safely wipe it clean and have your tests initialize it as required.
As far as I know the Spring support classes for database testing only rollback what happens in the tests, not what happens in setup and teardown of tests.

Agree with Confusion-- you should be running your tests against their own database schema.
With this, you can set your hibernate properties to 'create-drop':
With create-drop, the database schema
will be dropped when the
SessionFactory is closed explicitly.
See: Optional Hibernate Config properites
Example snippet:
<bean id="sessionBean" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="hibernateProperties">
<props>
<prop key="hibernate.hbm2ddl.auto">create-drop</prop>
...etc

While I'd agree with the guy's suggesting a deciated db for testing, there isn't any reason why using a populated db shouldn't work, both #Before and #After methods are executed within the transactional context, therefore there changes should be rolledback.
Possibles:
The data setup is doing something that isn't transactional (ie DDL statements)
Something in your test is actually committing the transaction
Can you post the #Before method, I'm just wondering if you are just clearing the tables or actually dropping and recreating them?

As far as I can tell, by looking at the Javadocs and source code for AbstractJUnit4SpringContextTests and TransactionalTestExecutionListener you need to annotate your test methods you want transactionalised with #Transactional.
There are also #BeforeTransaction and #AfterTransaction annotations where you can better control what runs in a transaction.
I suggest you create methods annotated with all these annotations, including #Before and then run the test with breakpoints at these methods. That way you can look at the stack and work out whether spring has started a transaction for you or not. If you see something like "TransactionInterceptor" in the stack then, or anything else with "Transaction" in the name, then chances are you're in a transaction.

You're doing super.deleteFromTables in your #Before method which is within the tx. So if the tx is rolled back doesn't the deletions get rolled back also?

writing test case for a DAO on a J2ee Application

I am trying to write some test cases for my DAO classes in a J2EE applications. Methods in my DAO classes try to get connection to the Database based on a JDBC URL (which is on the app server). So from the front end if I click bunch of stuff and make the DAO trigger it runs fine. However, when I write tests cases for the DAO and the DAO object calls the method then it is not able to get the connection to the database. I think since the JDBC resource is on the App server that is why it is not working from the test class.
because of this when I run my tests instead of pass or fail..it returns bunch of errors.
Has someone encountered this issue? what can I do to overcome this?
Example:
public class DBConnectionManager {
public static final String DB_URL = "jdbc/RSRC/my/connection/mydb"
public Connection getconnection ()
{
DataSource ds = ServiceLocator.getInstance().getDataSource(DB_URL);
return ds.getconnection();
}
}
public class MyDAO extends DBConnectionManager {
publci SomeBean getContents (String id)
{
Connection con = getConnection();
CallableStatement cs = con.prepareCall("{call myStorProc(?)}");
cs.setString(1, id);
...
//code to call resultset and retrieve SomeBean goes here
..
return SomeBean;
}
}
public class MyTests extends TestCase {
public testGetcontents ()
{
MyDAO myd = new MyDAO ();
SomeBean smb = myd.getContents("someparm");
assertEquals (5, smb.getSomeVal());
}
}
Should I be doing something extra in my testcase...? if so what?
EDIT:
error I get is:
java.lang.NoClassDefFoundError: com/iplanet/ias/admin/common/ASException
at java.lang.ClassLoader.defineClass1(Native Method)

Your DAO has a JNDI lookup string hard wired into it. Unless you have a JNDI lookup service available, it won't be able to get a connection.
I don't think a DAO should be responsible for acquiring a database connection. This design won't allow you to set transactions for a unit of work, because a DAO can't know if it's part of a larger unit of work.
I'd recommend passing the connection into the DAO, perhaps into its constructor. That way a service layer can establish appropriate transaction boundaries if there's more than one DAO in a single unit of work.
This design will have the added benefit of making it possible for your application to use its JNDI resource appropriately and your test to get its connection from a DriverManager, without having to use a JNDI lookup. You have two different sources for acquiring the DataSource or Connection - one for the app and another for the test.
UPDATE:
Here's what I mean, expressed in your code:
public class DBConnectionManager
{
public static final String DB_URL = "jdbc/RSRC/my/connection/mydb"
public Connection getConnection (String jndiLookup)
{
DataSource ds = ServiceLocator.getInstance().getDataSource(jndiLookup);
return ds.getconnection();
}
public Connection getConnection(String driver, String url, String username, String password)
throws ClassNotFoundException, SQLException
{
Class.forName(driver);
return DriverManager.getConnection(url, username, password);
}
}
public class MyDAO
{
private Connection connection;
public MyDao(Connection connection)
{
this.connection = connection;
}
public SomeBean getContents (String id)
{
CallableStatement cs = this.connection.prepareCall("{call myStorProc(?)}");
this.connection.setString(1, id);
//code to call resultset and retrieve SomeBean goes here
return someBean;
}
}
You show nothing about closing resources properly or transactions. Judging by this code, you'll be in trouble on both counts. I'd think carefully about your implementation.
I'll recommend Spring JDBC to you. You can write your DAOs in Spring without rewriting your whole app.
I'll also point out that you might also be looking at generics: Don't Repeat The DAO.

Test well your ServiceLocator first. As you mentioned, the problem is probably because the datasource is declared on the server. Here the "bunch of errors" should be helpful, as of whether the problem is in acquiring the DataSource, or the Connectiion itself.
What database are you using? Can you logon to it from your machine from console? If not - configure it so that your host is allowed.

It could be a permissions issue on the database you're trying to access. What errors are you getting?
One useful way for testing database access is to create a clean, local "test" version of your database as part of your test harness. Before you run your tests, use scripts to create a local copy of the database with all the pertinent data, then run your tests against that, rather than the remote server.
People may argue that testing against a database in a unit test is not truly a unit test, since it has an external dependency. If you're able to refactor your DAO classes, you can make it so the actual data source is injectable through some interfaces. In your test code, you'd inject a "mock" data source which provides your test data in some in memory format, then in production, you'd use/inject the actual database source classes. If you can hide the external (non-business code related) parts of your DAO behind interfaces, you can use mocks in your unit tests to test more of your functionality, rather than the actual data access.

Where I work our DAOs have an injectable connection (via constructor injection), and we unit test against a mock connection. To test the code in the DAO, we pass in a mocked (usually using Mockito) connection, and set up expectations in our unit tests as to what methods will be called. This makes for somewhat noisy tests, as the tests look very similar to the code being developed, but it works for us.