I am trying to set up Apache Ignite cache store using PostgreSQL as an external storage.
public class MyCacheStore extends CacheStoreAdapter<String, MyCache> {
private static final String GET_QUERY= "SELECT * FROM ..";
private static final String UPDATE_QUERY = "UPDATE ...";
private static final String DELETE_QUERY = "DELETE FROM ..";
#CacheStoreSessionResource
private CacheStoreSession session;
#Override
public MyCache load(String key) throws CacheLoaderException {
Connection connection = session.attachment();
try (PreparedStatement preparedStatement = connection.prepareStatement(GET_QUERY)) {
// some stuff
}
}
#Override
public void loadCache(IgniteBiInClosure<String, MyCache> clo, Object... args) {
super.loadCache(clo, args);
}
#Override
public void write(Cache.Entry<? extends String, ? extends MyCache> entry) throws CacheWriterException {
Connection connection = session.attachment();
try (PreparedStatement preparedStatement = connection.prepareStatement(UPDATE_QUERY)) {
// some stuff
}
}
#Override
public void delete(Object key) throws CacheWriterException {
Connection connection = session.attachment();
try (PreparedStatement preparedStatement = connection.prepareStatement(DELETE_QUERY)) {
// some stuff
}
}
}
MyCache is a standard class:
public class MyCache implements Serializable {
#QuerySqlField(index = true, name = "id")
private String id;
public MyCache() {
}
public MyCache(String id) {
this.id = id;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
}
Here is a configuration class
import javax.cache.configuration.Factory;
import javax.cache.configuration.FactoryBuilder;
#Configuration
public class ServiceConfig {
// no problems here
#Bean
#ConfigurationProperties(prefix = "postgre")
DataSource dataSource() {
return DataSourceBuilder
.create()
.build();
}
#Bean
public Ignite igniteInstance(IgniteConfiguration igniteConfiguration) {
return Ignition.start(igniteConfiguration);
}
#Bean
public IgniteConfiguration igniteCfg () {
// some other stuff here
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setClientMode(true);
CacheConfiguration myCacheConfiguration = new CacheConfiguration("MY_CACHE")
.setIndexedTypes(String.class, MyCache.class)
.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL)
.setReadThrough(true)
.setReadThrough(true)
.setCacheStoreSessionListenerFactories(new MyCacheStoreSessionListenerFactory(dataSource))
.setCacheStoreFactory(FactoryBuilder.factoryOf(MyCacheStore.class));
cfg.setCacheConfiguration(myCacheConfiguration);
return cfg;
}
private static class MyCacheStoreSessionListenerFactory implements Factory {
DataSource dataSource;
MyCacheStoreSessionListenerFactory(DataSource dataSource) {
this.dataSource = dataSource;
}
#Override
public CacheStoreSessionListener create() {
// Data Source
CacheJdbcStoreSessionListener listener = new CacheJdbcStoreSessionListener();
listener.setDataSource(dataSource);
return listener;
}
}
}
And this is what I get in logs:
...
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to validate cache configuration
(make sure all objects in cache configuration are serializable): MyCache
at org.apache.ignite.internal.processors.cache.GridCacheProcessor$11.applyx(GridCacheProcessor.java:4766)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor$11.applyx(GridCacheProcessor.java:4743)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.withBinaryContext(GridCacheProcessor.java:4788)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.cloneCheckSerializable(GridCacheProcessor.java:4743)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.addCacheOnJoin(GridCacheProcessor.java:818)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.addCacheOnJoinFromConfig(GridCacheProcessor.java:891)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart(GridCacheProcessor.java:753)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor.start(GridCacheProcessor.java:795)
at org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1700)
... 77 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to serialize object: CacheConfiguration [name=MyCache, grpName=null, memPlcName=null, storeConcurrentLoadAllThreshold=5, rebalancePoolSize=2, rebalanceTimeout=10000, evictPlc=null, evictPlcFactory=null, onheapCache=false, sqlOnheapCache=false, sqlOnheapCacheMaxSize=0, evictFilter=null, eagerTtl=true, dfltLockTimeout=0, nearCfg=null, writeSync=null, storeFactory=javax.cache.configuration.FactoryBuilder$ClassFactory#d87782a1, storeKeepBinary=false, loadPrevVal=false, aff=null, cacheMode=PARTITIONED, atomicityMode=TRANSACTIONAL, backups=0, invalidate=false, tmLookupClsName=null, rebalanceMode=ASYNC, rebalanceOrder=0, rebalanceBatchSize=524288, rebalanceBatchesPrefetchCnt=2, maxConcurrentAsyncOps=500, sqlIdxMaxInlineSize=-1, writeBehindEnabled=false, writeBehindFlushSize=10240, writeBehindFlushFreq=5000, writeBehindFlushThreadCnt=1, writeBehindBatchSize=512, writeBehindCoalescing=true, maxQryIterCnt=1024, affMapper=null, rebalanceDelay=0, rebalanceThrottle=0, interceptor=null, longQryWarnTimeout=3000, qryDetailMetricsSz=0, readFromBackup=true, nodeFilter=null, sqlSchema=null, sqlEscapeAll=false, cpOnRead=true, topValidator=null, partLossPlc=IGNORE, qryParallelism=1, evtsDisabled=false, encryptionEnabled=false]
at org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:103)
at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:70)
at org.apache.ignite.marshaller.jdk.JdkMarshaller.marshal0(JdkMarshaller.java:117)
at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:58)
at org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10250)
at org.apache.ignite.internal.processors.cache.GridCacheProcessor$11.applyx(GridCacheProcessor.java:4762)
... 85 more
Caused by: java.io.NotSerializableException: com.zaxxer.hikari.HikariDataSource
at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185)
at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)
I have read all official documentation about it and examined many other examples, but can't make it run.
HikariCP is the most popular connection pool library, I can't understand why Ignite throws an exception about not being able to serialize DataSource.
Any advice or idea would be appreciated, thank you!
Since your Cache Store is not serializable, you should not use Factory.factoryOf (which is a no-op wrapper) but instead supply a real serializable factory implementation which will acquire local HikariCP on node and then construct the Cache Store.
Related
I am getting the following error when I upgrade my Hikari version to 3.4.5
The configuration of the pool is sealed once started. Use HikariConfigMXBean for runtime changes.
I am running the following #BeforeEach test
dataSource.getHikariPoolMXBean().softEvictConnections();
dataSource.setConnectionInitSql("set search_path to " + getTestSchema() + ", public");
This is how I load my DataSource
public class DataSource {
private static HikariConfig config = new HikariConfig();
public static HikariDataSource ds;
static {
config.setJdbcUrl(
System.getProperty("jdbc.url", "jdbc:postgresql://localhost:123/abc"));
config.setUsername(System.getProperty("username", "xyz"));
config.setPassword(System.getProperty("password", "123"));
ds = new HikariDataSource(config);
}
private DataSource() {}
public static Connection getConnection() throws SQLException {
return ds.getConnection();
}
public static void evictConnection(Connection connection) {
ds.evictConnection(connection);
}
}
class TestClass extends DatabaseTestCase {
#Autowired
private HikariDataSource dataSource;
#BeforeEach
void initData() throws SQLException {
DataSource.ds = dataSource;
dataSource.getHikariPoolMXBean().softEvictConnections();
dataSource.setConnectionInitSql("set search_path to " + getTestVitmSchema() + ", public");
}
This is how I setSchema for the Test Cases
public abstract class DatabaseTestCase {
private String testSchema;
#BeforeEach
final void setupSchema() {
int classHash = Math.abs(getClass().getSimpleName().hashCode());
testSchema = String.format("test_%s", classHash);
Map<String, String> placeholders = new HashMap<>();
placeholders.put("schema", testSchema);
placeholders.put("name", "xyz");
placeholders.put("password", "123");
placeholders.putAll(getPlaceholderReplacement());
Flyway flyway =
Flyway.configure()
.dataSource(DataSource.ds)
.schemas(testSchema)
.placeholders(placeholders)
.load();
flyway.migrate();
}
How can I set Connection Init Sql without getting the error. Thank you.
You can't. You will need to create a new dataSource to do this. You can see why from looking at the source code.
I have a problem when adding new test. And the problem is I think related to #DirtiesContext. I tried removing and adding it but nothing works in combination. Test 1 is using Application Context as well.
the following two are running together and no issue.
Test 1
#ActiveProfiles({"aws", "local"})
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class UnauthorizedControllerTest {
private static final Logger LOGGER = LoggerFactory.getLogger(UnauthorizedControllerTest.class);
#Autowired
private TestRestTemplate testRestTemplate;
#LocalServerPort
private int port;
#Autowired
private ApplicationContext context;
private Map<Class<?>, List<String>> excludedMethodsPerController;
#Before
public void setUp() {
excludedMethodsPerController = excludedMethodsPerController();
}
#Test
public void contextStarts() {
assertNotNull(context);
}
#Test
public void controllerCall_WithoutAuthorization_ReturnsUnauthorized() {
Map<String, Object> controllerBeans = context.getBeansWithAnnotation(Controller.class);
for (Object controllerInstance : controllerBeans.values()) {
LOGGER.info("Checking controller {}", controllerInstance);
checkController(controllerInstance);
}
}
public void checkController(Object controllerInstance) {
// Use AopUtils for the case that spring wraps the controller in a proxy
Class<?> controllerClass = AopProxyUtils.ultimateTargetClass(controllerInstance);
Method[] allMethods = controllerClass.getDeclaredMethods();
for (Method method : allMethods) {
LOGGER.info("Checking method: {}", method.getName());
if (!isCallable(controllerClass, method)) {
continue;
}
String urlPrefix = urlPrefix(controllerClass);
Mapping mapping = Mapping.of(method.getAnnotations());
for (String url : mapping.urls) {
for (RequestMethod requestMethod : mapping.requestMethods) {
ResponseEntity<String> exchange = exchange(urlPrefix + url, requestMethod);
String message = String.format("Failing %s.%s", controllerClass.getName(), method.getName());
assertEquals(message, HttpStatus.UNAUTHORIZED, exchange.getStatusCode());
}
}
}
}
private ResponseEntity<String> exchange(String apiEndpoint, RequestMethod requestMethod) {
return testRestTemplate.exchange(url(replacePathVariables(apiEndpoint)), HttpMethod.resolve(requestMethod.name()), null, String.class);
}
private String urlPrefix(Class<?> aClass) {
if (!aClass.isAnnotationPresent(RequestMapping.class)) {
return "";
}
RequestMapping annotation = aClass.getAnnotation(RequestMapping.class);
return annotation.value()[0];
}
private String url(String url) {
return "http://localhost:" + port + url;
}
private boolean isCallable(Class<?> controller, Method method) {
return Modifier.isPublic(method.getModifiers())
&& !isExcluded(controller, method)
&& !isExternal(controller);
}
private boolean isExcluded(Class<?> controller, Method method) {
List<String> excludedMethodsPerController = this.excludedMethodsPerController.getOrDefault(controller, new ArrayList<>());
return excludedMethodsPerController.contains(method.getName());
}
private boolean isExternal(Class<?> controller) {
return controller.getName().startsWith("org.spring");
}
private String replacePathVariables(String url) {
return url.replaceAll("\\{[^\\/]+}", "someValue");
}
/**
* There must be a really good reason to exclude the method from being checked.
*
* #return The list of urls that must not be checked by the security
*/
private static Map<Class<?>, List<String>> excludedMethodsPerController() {
Map<Class<?>, List<String>> methodPerController = new HashMap<>();
methodPerController.put(AuthenticationController.class, Collections.singletonList("generateAuthorizationToken"));
methodPerController.put(SystemUserLoginController.class, Arrays.asList("systemUserLogin", "handleException"));
methodPerController.put(ValidationController.class, Collections.singletonList("isValid"));
return methodPerController;
}
}
Test 2
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#ActiveProfiles({"aws", "local"})
public class RoleAdminControllerAuditTest {
private static final String DOMAIN_NAME = "testDomain";
private static final String APP_NAME_1 = "testApp_1";
private static final String APP_NAME_2 = "testApp_2";
private static final String ROLE_NAME = "testRole";
private static final String USER_NAME = "testUser";
#Autowired
AuditRepository auditRepository;
#Autowired
RoleAdminController roleAdminController;
#MockBean
RoleAdminService roleAdminService;
#MockBean
RoleAdminInfoBuilder infoBuilder;
#MockBean
AppInfoBuilder appInfoBuilder;
#MockBean
BoundaryValueService boundaryValueService;
#MockBean
RoleService roleService;
#MockBean
private SecurityService securityService;
private static final String URS_USER = "loggedInUser";
private static final String BOUNDARY_VALUE_KEY = "11";
private static final String BOUNDARY_VALUE_NAME = "Schulenberg";
private String auditEventDate = LocalDate.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd"));
#BeforeClass
public static void setupTestEnv() {
// https://github.com/localstack/localstack/issues/592
System.setProperty("com.amazonaws.sdk.disableCbor", "true");
}
#Before
public void setUp() throws Exception {
auditRepository.clean();
when(securityService.getLoggedInUser()).thenReturn(new TestHelper.FakeUser(URS_USER));
//when(roleService.addRoleToApp(any(), any(), eq(ROLE_NAME))).thenReturn(TestHelper.initRole(ROLE_NAME));
when(boundaryValueService.findBoundaryValueById(eq(123L))).thenReturn(initBoundaryValue(BOUNDARY_VALUE_KEY, BOUNDARY_VALUE_NAME));
when(boundaryValueService.findBoundaryValueById(eq(666L))).thenReturn(initBoundaryValue(BOUNDARY_VALUE_KEY, BOUNDARY_VALUE_NAME));
}
#Test
public void addUserAsRoleAdminLogged() throws UserIsAlreadyRoleAdminException, RoleNotFoundException, BoundaryValueNotFoundException {
User user = initUser(USER_NAME);
List<RoleAdminInfo> roleAdminInfos = getRoleAdminInfos();
roleAdminController.addUserAsRoleAdmin(user, roleAdminInfos);
List<String> result = auditRepository.readAll();
assertEquals("some data", result.toString());
}
#Test
public void removeUserAsRoleAdminLogged() throws RoleNotFoundException, BoundaryValueNotFoundException {
User user = initUser(USER_NAME);
Long roleId = Long.valueOf(444);
Role role = initRole("test-role");
role.setApp(initApp("test-app"));
role.setDomain(initDomain("test-domain"));
when(roleService.getRoleByIdOrThrow(roleId)).thenReturn(role);
roleAdminController.removeUserAsRoleAdmin(user, roleId, Long.valueOf(666));
List<String> result = auditRepository.readAll();
assertEquals("some data", result.toString());
}
#Test
public void removeRoleAdminPermission() throws RoleNotFoundException, BoundaryValueNotFoundException {
User user = initUser(USER_NAME);
List<RoleAdminInfo> roleAdminInfos = getRoleAdminInfos();
roleAdminController.removeRoleAdminPermission(user, roleAdminInfos);
List<String> result = auditRepository.readAll();
assertEquals(1, result.size());
assertEquals("some data", result.toString());
}
private List<RoleAdminInfo> getRoleAdminInfos() {
RoleAdminInfo info1 = initRoleAdminInfo(DOMAIN_NAME, ROLE_NAME, APP_NAME_1);
RoleAdminInfo info2 = initRoleAdminInfo(DOMAIN_NAME, ROLE_NAME, APP_NAME_2);
info1.setBoundaryValueId(123L);
info1.setBoundaryValueKey(BOUNDARY_VALUE_KEY);
info1.setBoundaryValueName(BOUNDARY_VALUE_NAME);
return Arrays.asList(info1, info2);
}
}
Test 3 (newly added one)
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, classes = FlywayConfig.class)
#DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_CLASS)
#ActiveProfiles({"aws", "local"})
public class BoundaryValueDeltaControllerTest {
private static final String API_V1 = "/api/v1/";
#Autowired
TestRestTemplate testRestTemplate;
#Autowired
private DomainBuilder domainBuilder;
#Autowired
private AppBuilder appBuilder;
#Autowired
private UserBuilder userBuilder;
#Autowired
private DomainAdminBuilder domainAdminBuilder;
#Autowired
private BoundarySetBuilder boundarySetBuilder;
#MockBean
private LoginUserProvider loginUserProvider;
#MockBean
private LoginTokenService loginTokenService;
#MockBean
private BoundaryServiceAdapter serviceAdapter;
#LocalServerPort
private int port;
LoginUserInfo loggedInUser;
#Before
public void setUp() {
clear();
}
#After
public void tearDown() {
clear();
}
#Test
public void updateBoundaryValuesFromApi() throws UrsBusinessException {
Domain domain = domainBuilder.persist();
App app = appBuilder.persist(domain);
BoundarySet boundarySet = boundarySetBuilder.persist(domain);
User user = userBuilder.persist(domain.getAuthor().getUsername());
aLoggedInUser(domain.getAuthor().getUsername());
domainAdminBuilder.persist(user, domain);
mockReadInfoFromApiUsingApp();
ResponseEntity<String> response = callUpdateBoundaryValuesFromApi(domain, boundarySet, app);
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
}
private void mockReadInfoFromApiUsingApp() throws UrsBusinessException {
BoundaryValueInfo boundaryValueInfo = new BoundaryValueInfo();
boundaryValueInfo.setBoundaryValueId(10L);
boundaryValueInfo.setBoundaryValueKey("boundaryValueKey");
boundaryValueInfo.setBoundaryValueName("boundaryValuename");
when(serviceAdapter.readInfoFromApiUsingApp(any(), any(), any())).thenReturn(new BoundaryValueInfo[]{boundaryValueInfo});
}
private ResponseEntity<String> callUpdateBoundaryValuesFromApi(Domain domain, BoundarySet boundarySet, App app) {
String url = url(API_V1 + "domains/" + domain.getName() + "/boundarysets/" + boundarySet.getBoundarySetName() + "/app/" + app.getName()+ "/updatefromapi/");
return testRestTemplate.exchange(url,HttpMethod.GET, null, String.class);
}
private String url(String url) {
return "http://localhost:" + port + url;
}
private void aLoggedInUser(String username) {
Claims claims = Jwts.claims();
claims.put("username", username);
loggedInUser = LoginUserInfo.parse(claims);
when(loginUserProvider.getLoggedInUser()).thenReturn(loggedInUser);
when(loginTokenService.parseToken(any())).thenReturn(loggedInUser);
}
private void clear() {
appBuilder.deleteAll();
boundarySetBuilder.deleteAll();
domainAdminBuilder.deleteAll();
domainBuilder.deleteAll();
userBuilder.deleteAll();
}
}
Flyway config
#TestConfiguration
public class FlywayConfig {
#Bean
public FlywayMigrationStrategy clean() {
return flyway -> {
flyway.clean();
flyway.migrate();
};
}
}
And I am getting below exception while running all together.
java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:125)
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java.....
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.exception.FlywaySqlException:
Unable to obtain connection from database: Too many connections
---------------------------------------------------------------
SQL State : 08004
Error Code : 1040
Message : Too many connections
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1762)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
... 49 more
Caused by: org.flywaydb.core.internal.exception.FlywaySqlException:
Unable to obtain connection from database: Too many connections
I am struggling since yesterday's and you might find duplicate but I tried to add the more details today. please guide me here.
you must add the configuration for flyway
flyway.url=jdbc:postgresql://xxx.eu-west-2.rds.amazonaws.com:5432/xxx
flyway.user=postgres
flyway.password=xxx
I am trying to override Ribbon Server List to get a list of host names from consul. I have the consul piece working properly(when testing with hardcode values) to get the hostname and port for a service. The issue I am having is when I try to autowire in IClientConfig. I get an exception that IClientConfig bean could not be found. How do I override the ribbon configurations and autowire IClientConfig in the ribbonServerList method.
I have tried following the instructions here at http://projects.spring.io/spring-cloud/spring-cloud.html#_customizing_the_ribbon_client on how to customize ribbon client configuration. I keep getting the following error:
Description:
Parameter 0 of method ribbonServerList in com.intradiem.enterprise.keycloak.config.ConsulRibbonSSLConfig required a bean of type 'com.netflix.client.config.IClientConfig' that could not be found.
Which is causing spring-boot to fail.
Bellow are the classes that I am trying to use to create
AutoConfiguration Class:
#Configuration
#EnableConfigurationProperties
#ConditionalOnBean(SpringClientFactory.class)
#ConditionalOnProperty(value = "spring.cloud.com.intradiem.service.apirouter.consul.ribbon.enabled", matchIfMissing = true)
#AutoConfigureAfter(RibbonAutoConfiguration.class)
#RibbonClients(defaultConfiguration = ConsulRibbonSSLConfig.class)
//#RibbonClient(name = "question-answer-provider", configuration = ConsulRibbonSSLConfig.class)
public class ConsulRibbonSSLAutoConfig
{
}
Configuration Class:
#Component
public class ConsulRibbonSSLConfig
{
#Autowired
private ConsulClient client;
private String serviceId = "client";
public ConsulRibbonSSLConfig() {
}
public ConsulRibbonSSLConfig(String serviceId) {
this.serviceId = serviceId;
}
#Bean
#ConditionalOnMissingBean
public ServerList<?> ribbonServerList(IClientConfig clientConfig) {
ConsulSSLServerList serverList = new ConsulSSLServerList(client);
serverList.initWithNiwsConfig(clientConfig);
return serverList;
}
}
ServerList Code:
public class ConsulSSLServerList extends AbstractServerList<Server>
{
private final ConsulClient client;
private String serviceId = "client";
public ConsulSSLServerList(ConsulClient client) {
this.client = client;
}
#Override
public void initWithNiwsConfig(IClientConfig clientConfig) {
this.serviceId = clientConfig.getClientName();
}
#Override
public List<Server> getInitialListOfServers() {
return getServers();
}
#Override
public List<Server> getUpdatedListOfServers() {
return getServers();
}
private List<Server> getServers() {
List<Server> servers = new ArrayList<>();
Response<QueryExecution> results = client.executePreparedQuery(serviceId, QueryParams.DEFAULT);
List<QueryNode> nodes = results.getValue().getNodes();
for (QueryNode queryNode : nodes) {
QueryNode.Node node = queryNode.getNode();
servers.add(new Server(node.getMeta().containsKey("secure") ? "https" : "http", node.getNode(), queryNode.getService().getPort()));
}
return servers;
}
#Override
public String toString() {
final StringBuilder sb = new StringBuilder("ConsulSSLServerList{");
sb.append("serviceId='").append(serviceId).append('\'');
sb.append('}');
return sb.toString();
}
}
I've come up with a working dynamic multi-tenant application using:
Java 8
Java Servlet 3.1
Spring 3.0.7-RELEASE (can't change the version)
Hibernate 3.6.0.Final (can't change the version)
Commons dbcp2
This is the 1st time I've had to instantiate Spring objects myself so I'm wondering if I've done everything correctly or if the app will blow up in my face at an unspecified future date during production.
Basically, the single DataBase schema is known, but the database details will be specified at runtime by the user. They are free to specify any hostname/port/DB name/username/password.
Here's the workflow:
The user logs in to the web app then either chooses a database from a known list, or specifies a custom database (hostname/port/etc.).
If the Hibernate SessionFactory is built successfully (or is found in the cache), then it's persisted for the user's session using SourceContext#setSourceId(SourceId) then the user can work with this database.
If anybody choses/specifies the same database, the same cached AnnotationSessionFactoryBean is returned
The user can switch databases at any point.
When the user switches away from a custom DB (or logs off), the cached AnnotationSessionFactoryBeans are removed/destroyed
So will the following work as intended? Help and pointers are most welcome.
web.xml
<web-app version="3.1" ...>
<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>classpath:applicationContext.xml</param-value>
</context-param>
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>
<listener> <!-- Needed for SourceContext -->
<listener-class>org.springframework.web.context.request.RequestContextListener</listener-class>
</listener>
<web-app>
applicationContext.xml
<beans ...>
<tx:annotation-driven />
<util:properties id="db" location="classpath:db.properties" /> <!-- driver/url prefix -->
<context:component-scan base-package="com.example.basepackage" />
</beans>
UserDao.java
#Service
public class UserDao implements UserDaoImpl {
#Autowired
private TemplateFactory templateFactory;
#Override
public void addTask() {
final HibernateTemplate template = templateFactory.getHibernateTemplate();
final User user = (User) DataAccessUtils.uniqueResult(
template.find("select distinct u from User u left join fetch u.tasks where u.id = ?", 1)
);
final Task task = new Task("Do something");
user.getTasks().add(task);
TransactionTemplate txTemplate = templateFactory.getTxTemplate(template);
txTemplate.execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus status) {
template.save(task);
template.update(user);
}
});
}
}
TemplateFactory.java
#Service
public class TemplateFactory {
#Autowired
private SourceSessionFactory factory;
#Resource(name = "SourceContext")
private SourceContext srcCtx; // session scope, proxied bean
#Override
public HibernateTemplate getHibernateTemplate() {
LocalSessionFactoryBean sessionFactory = factory.getSessionFactory(srcCtx.getSourceId());
return new HibernateTemplate(sessionFactory.getObject());
}
#Override
public TransactionTemplate getTxTemplate(HibernateTemplate template) {
HibernateTransactionManager txManager = new HibernateTransactionManager();
txManager.setSessionFactory(template.getSessionFactory());
return new TransactionTemplate(txManager);
}
}
SourceContext.java
#Component("SourceContext")
#Scope(value="session", proxyMode = ScopedProxyMode.INTERFACES)
public class SourceContext {
private static final long serialVersionUID = -124875L;
private SourceId id;
#Override
public SourceId getSourceId() {
return id;
}
#Override
public void setSourceId(SourceId id) {
this.id = id;
}
}
SourceId.java
public interface SourceId {
String getHostname();
int getPort();
String getSID();
String getUsername();
String getPassword();
// concrete class has proper hashCode/equals/toString methods
// which use all of the SourceIds properties above
}
SourceSessionFactory.java
#Service
public class SourceSessionFactory {
private static Map<SourceId, AnnotationSessionFactoryBean> cache = new HashMap<SourceId, AnnotationSessionFactoryBean>();
#Resource(name = "db")
private Properties db;
#Override
public LocalSessionFactoryBean getSessionFactory(SourceId id) {
synchronized (cache) {
AnnotationSessionFactoryBean sessionFactory = cache.get(id);
if (sessionFactory == null) {
return createSessionFactory(id);
}
else {
return sessionFactory;
}
}
}
private AnnotationSessionFactoryBean createSessionFactory(SourceId id) {
AnnotationSessionFactoryBean sessionFactory = new AnnotationSessionFactoryBean();
sessionFactory.setDataSource(new CutomDataSource(id, db));
sessionFactory.setPackagesToScan(new String[] { "com.example.basepackage" });
try {
sessionFactory.afterPropertiesSet();
}
catch (Exception e) {
throw new SourceException("Unable to build SessionFactory for:" + id, e);
}
cache.put(id, sessionFactory);
return sessionFactory;
}
public void destroy(SourceId id) {
synchronized (cache) {
AnnotationSessionFactoryBean sessionFactory = cache.remove(id);
if (sessionFactory != null) {
if (LOG.isInfoEnabled()) {
LOG.info("Releasing SessionFactory for: " + id);
}
try {
sessionFactory.destroy();
}
catch (HibernateException e) {
LOG.error("Unable to destroy SessionFactory for: " + id);
e.printStackTrace(System.err);
}
}
}
}
}
CustomDataSource.java
public class CutomDataSource extends BasicDataSource { // commons-dbcp2
public CutomDataSource(SourceId id, Properties db) {
setDriverClassName(db.getProperty("driverClassName"));
setUrl(db.getProperty("url") + id.getHostname() + ":" + id.getPort() + ":" + id.getSID());
setUsername(id.getUsername());
setPassword(id.getPassword());
}
}
In the end I extended Spring's AbstractRoutingDataSource to be able to dynamically create datasources on the fly. I'll update this answer with the full code as soon as everything is working correctly. I have a couple of last things to sort out, but the crux of it is as follows:
#Service
public class DynamicRoutingDataSource extends AbstractRoutingDataSource {
// this is pretty much the same as the above SourceSessionFactory
// but with a map of CustomDataSources instead of
// AnnotationSessionFactoryBeans
#Autowired
private DynamicDataSourceFactory dataSourceFactory;
// This is the sticky part. I currently have a workaround instead.
// Hibernate needs an actual connection upon spring startup & there's
// also no session in place during spring initialization. TBC.
// #Resource(name = "UserContext") // scope session, proxy bean
private UserContext userCtx; // something that returns the DB config
#Override
protected SourceId determineCurrentLookupKey() {
return userCtx.getSourceId();
}
#Override
protected CustomDataSource determineTargetDataSource() {
SourceId id = determineCurrentLookupKey();
return dataSourceFactory.getDataSource(id);
}
#Override
public void afterPropertiesSet() {
// we don't need to resolve any data sources
}
// Inherited methods copied here to show what's going on
// #Override
// public Connection getConnection() throws SQLException {
// return determineTargetDataSource().getConnection();
// }
//
// #Override
// public Connection getConnection(String username, String password)
// throws SQLException {
// return determineTargetDataSource().getConnection(username, password);
// }
}
So I just wire up the DynamicRoutingDataSource as the DataSource for Spring's SessionFactoryBean along with a TransactionManager an all the rest as usual. As I said, more code to follow.
First the stacktrace:
org.springframework.dao.InvalidDataAccessApiUsageException:
Exception Description: No transaction is currently active; nested exception is javax.persistence.TransactionRequiredException:
Exception Description: No transaction is currently active
org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:316)
org.springframework.orm.jpa.DefaultJpaDialect.translateExceptionIfPossible(DefaultJpaDialect.java:121)
org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:403)
org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:58)
org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213)
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:163)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.data.jpa.repository.support.LockModeRepositoryPostProcessor$LockModePopulatingMethodIntercceptor.invoke(LockModeRepositoryPostProcessor.java:92)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:155)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
com.sun.proxy.$Proxy299.saveAndFlush(Unknown Source)
I'm pretty sure it's a stupid error but I can't seem to find what I am doing wrong:
The #Service:
#Service
public class BookInfoTracker implements InfoTracker
{
#Autowired
#Qualifier("bookTrackerRepository")
private TrackerRepository bookTrackerRepository;
public BookInfoTracker() {}
public BookInfoTracker (TrackerRepository bookTrackerRepository)
{
this.bookTrackerRepository = bookTrackerRepository;
}
#Transactional
#Override
public void track (long idBooking, String idLeg, String bookingEmail, int sequence, String event)
{
BookInfo ci = new BookInfo(idBooking, idLeg, bookingEmail, new Date(), sequence, event);
bookTrackerRepository.saveAndFlush(ci);
}
#Override
public int getSequenceFrom (long idBooking, String idLeg)
{
BookInfo tracked = findLastTrackedBookFrom(bookTrackerRepository.getLastTrackedBookByIdBookingAndIdLeg(idBooking, idLeg));
return null == tracked? 0 : tracked.getSequence()+1;
}
private BookInfo findLastTrackedBookFrom (List<BookInfo> trackedBooks)
{
return trackedBooks.isEmpty() ? null : trackedBooks.get(0);
}
}
There is a base class for jpa configuration:
#EnableLoadTimeWeaving(aspectjWeaving=AspectJWeaving.ENABLED)
public abstract class JpaConfiguration
{
public JpaConfiguration ()
{
super();
}
public abstract LocalContainerEntityManagerFactoryBean buildEntityManagerFactory () throws SQLException;
#Bean(name = "transactionManager")
public PlatformTransactionManager buildTransactionManager () throws SQLException
{
JpaTransactionManager manager = new JpaTransactionManager();
LocalContainerEntityManagerFactoryBean factory = buildEntityManagerFactory();
manager.setEntityManagerFactory(factory.getObject());
return manager;
}
protected Database toDatabase (String databaseProductName)
{
for (Database database : Database.values())
if (databaseProductName.equalsIgnoreCase(database.toString())) return database;
return null;
}
protected LocalContainerEntityManagerFactoryBean initializeFactory (DataSource datasource, String persistenceUnitName, String... packagesToScan) throws SQLException
{
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setDataSource(datasource);
factory.setPersistenceUnitName(persistenceUnitName);
factory.setPackagesToScan(packagesToScan);
EclipseLinkJpaVendorAdapter jpaAdaptor = new EclipseLinkJpaVendorAdapter();
jpaAdaptor.setDatabase(toDatabase(datasource.getConnection().getMetaData().getDatabaseProductName()));
jpaAdaptor.setShowSql(true);
factory.setJpaVendorAdapter(jpaAdaptor);
Properties jpaProperties = new Properties();
jpaProperties.put("eclipselink.ddl-generation", "none");
jpaProperties.put("eclipselink.ddl-generation.output-mode", "database");
jpaProperties.put("eclipselink.logging.level.sql", "FINE");
jpaProperties.put("eclipselink.logging.parameters", "true");
jpaProperties.put("eclipselink.cache.shared.default", "false");
jpaProperties.put("eclipselink.target-database", "MySQL");
jpaProperties.put("eclipselink.weaving","false");
factory.setJpaProperties(jpaProperties);
factory.afterPropertiesSet();
return factory;
}
}
and then the specific configuration, a subclass:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = {"com.book.tracking.repositories"}, entityManagerFactoryRef="bookManagerFactory")
#Profile({"integration","test","release"})
public class JpaBookConfiguration extends JpaConfiguration
{
#Autowired
#Qualifier("bookDs")
private DataSource datasource;
#Override
#Bean(name = "bookManagerFactory")
public LocalContainerEntityManagerFactoryBean buildEntityManagerFactory () throws SQLException
{
return initializeFactory(datasource, "book.jpa", "com.book.tracking.entity");
}
}
then the restservice:
#Controller
#ControllerAdvice
#RequestMapping("/bookservice")
public class BookRestService implements IBookService
{
private static final String LEG_VALID_PATTERN = "^(20)\\d\\d(0[1-9]|1[012])(0[1-9]|[12][0-9]|3[01])_[A-Z]{3}-[A-Z]{3}$";
#Autowired
#Qualifier("bookBuilder")
private IBookBuilder bookBuilder;
#Autowired
#Qualifier("bookInfoTracker")
private InfoTracker bookInfoTracker;
#Autowired
#Qualifier("bookBookingEventsGenerator")
private BookingEventsGenerator bookBookingEventsGenerator;
public BookRestService (IBookBuilder bookBuilder,
InfoTracker infoTracker, BookingEventsGenerator bookBookingEventsGenerator)
{
this.bookBookingEventsGenerator = bookBookingEventsGenerator;
this.bookBuilder = bookBuilder;
this.bookInfoTracker = infoTracker;
}
public BookRestService ()
{
}
#Override
#RequestMapping(value="/generate/{idBooking}/{idLeg}")
public ResponseEntity<String> generateOneWayEvent (#PathVariable("idBooking") final long idBooking, #PathVariable("idLeg") final String idLeg)
{
return executeEventCommand(new EventCommand() {
#Override
public BookingEvent execute() {
BookingEvent event = bookBookingEventsGenerator.generateEventFrom(idBooking, idLeg);
return event;
}
}, idBooking, idLeg, "PUBLISH");
}
private ResponseEntity<String> executeEventCommand (EventCommand eventCommand, long idBooking, String idLeg, String operation)
{
try {
validate(idLeg);
logFor(idBooking, idLeg, "generateOneWayEvent");
HttpHeaders headers = modifyResponseAccordingTo(idLeg);
BookingEvent event = eventCommand.execute();
int sequence = bookInfoTracker.getSequenceFrom(idBooking, event.getBookingIdentifier());
String bookIcs = generateBookFrom(event, sequence);
logger.debug(bookInfoTracker.getClass());
//exception here
bookInfoTracker.track(idBooking, event.getBookingIdentifier(), bookBookingEventsGenerator.getContactEmail(), sequence, operation);
return new ResponseEntity<String>(bookIcs, headers, HttpStatus.OK );
} catch(LegNotFoundException lnfe){
logger.error(lnfe);
return new ResponseEntity<String>("Something went wrong", HttpStatus.NOT_FOUND);
}
}
/** other methods **/
}
It seems like it's not considering #Transactional annotation maybe? But I used #EnableTransactionManagement. Can you see something wrong in my configuration?
UPDATE:
#Configuration
#Profile({"integration", "release"})
public class IBookSpringConfiguration
{
#Autowired
#Qualifier("bookRestService")
private IBookService bookRestService;
#Autowired
#Qualifier("bookTrackerRepository")
private TrackerRepository bookTrackerRepository;
public ReloadableResourceBundleMessageSource getMessageBundle(){
ReloadableResourceBundleMessageSource bundle = new ReloadableResourceBundleMessageSource();
bundle.setBasename("book-message");
return bundle;
}
#Bean(name = "bookInfoTracker")
public InfoTracker getBookTrackingService(){
return new BookInfoTracker(bookTrackerRepository);
}
#Bean(name = "bookBuilder")
public IBookBuilder getBookBookingService(){
return new BookBuilder(getMessageBundle());
}
#Bean(name = "i18n")
public I18nUtilACL getI18n(){
return new I18nUtilBook(MultiSourceSiteCustomizer.getInstance());
}
#Bean(name = "bookBookingEventsGenerator")
public BookingEventsGenerator getBookBookingEventsGenerator(){
return new BookEventsGenerator();
}
#Bean(name = "bookBookingLegFactory")
public BookingLegEventsGenerator getBookBookingLegFactory(){
return new BookBookingLegEventsGenerator();
}
#Bean(name = "bookBookingFlightEventsGenerator")
public BookingEventsGenerator getBookBookingFlightEventsGenerator(){
return new BookEventsGenerator();
}
#Bean(name = "bookRestService")
#Scope("singleton")
public IBookService getBookRestService ()
{
return new BookRestService();
}
}
SOLVED BUT DON'T KNOW WHY:
I just merge parent and subclass of the JpaConfiguration in JpaBookConfiguration. Now it's working. Don't know why. Can you explain?