In a spring boot application, I have multiple implementations of an interface, every one of which accesses to the database using a JPA repository.
#Autowired
List<? extends CrawlerService> crawlers;
This is the repository:
#Repository
public interface HotelRepository extends JpaRepository<Hotel, Long> {
public Optional<Hotel> findByTextAndSource(String text, String source);
}
And the implementations look like this:
#Service
public class MyService implements CrawlerService {
#Autowired
DocumentService documentService;
#Autowired
HotelRepository hotelRepository;
private Logger logger = LoggerFactory.getLogger("MyService");
private Set<String> visitedLinks = new HashSet<>();
#Override
public void start() {
execute("https://www.example.com/");
}
public void execute(String url) {
if(visitedLinks.contains(url))
return;
try {
logger.info("Connecting '{}' ...", url);
visitedLinks.add(url);
Document document = Jsoup.connect(url).get();
String text = documentService.getText(document);
// this is where I access to the database
if(hotelRepository.findByTextAndSource(text, "EXAMPLE").isPresent()) {
return;
}
Hotel hotel = new Hotel();
hotel.setUrl(url);
hotel.setText(text);
hotel.setSource("EXAMPLE");
hotelRepository.save(hotel);
// find other links
findLinks(document, url).forEach(this::execute);
} catch (IOException e) {
logger.error("Connection error for {}", url);
}
}
}
Now I'm using parallel stream to run each one of them in parallel:
crawlers.stream().parallel().forEach(CrawlerService::start);
They continue to work until the point that they want to access the database through the JPA repository. Then all of them stop, except the one assigned to the main thread. This is observed using the output of my logger:
2018-11-01 21:42:38.408 INFO 7641 --- [ main] MyService ...
Only this one continues to run.
I want all of them to continue to work in parallel. How can I solve this problem?
It is worth mentioning that I'm using MySQL and Spring Boot 2 with default settings for connection pool.
Related
I am doing a work on arango db. Dose arangodb-spring-boot-starter has the transition and rollback support
I have tried #Transition annotation in the custom repo layer. added a error by custom error, the service has a functionality to create multiple document. I was expecting the rollback which is not happened.
This is the arango repository code.
public interface RelationRepository extends ArangoRepository<Relation, String> {
#Transactional
#Query("insert { _from: #from, _to: #to } into #collection return NEW")
Set<Relation> createEdge(#Param("from") String from,#Param("to") String to;}
This is the code snippet for the service
#Service
public class RelationService {
#Autowired
private RelationRepository relationRepository;
private final Logger log = LoggerFactory.getLogger(RelationService.class);
#Transactional(rollbackFor = SQLException.class)
public HashMap<String,String> demoRelation() {
relationRepository.createEdge("vertex1/121286","vertex2/167744","relation",
Instant.now().toEpochMilli(),Long.MAX_VALUE);
if(true)
throw new SQLException("custom exception to check rollback");
return null;
}
}
I was expecting the rollback, instead it is creating records
I have a ProcessRecon usecase class with a single method named execute. It saves an entity Reconciliation using paymentRepository.saveRecon and calls a web service as part of acknowledgement using paymentRepository.sendReconAck.
Now there's a chance that this external web service might fail in which case I want to rollback the changes i.e. the saved entity. Since I am using Unirest, it throws UnirestException which is a checked exception.
There are no errors on the console but this will probably be helpful [UPDATED].
2020-08-20 17:21:42,035 DEBUG [http-nio-7012-exec-6] org.springframework.transaction.support.AbstractPlatformTransactionManager: Creating new transaction with name [com.eyantra.payment.features.payment.domain.usecases.ProcessRecon.execute]:PROPAGATION_REQUIRED,ISOLATION_DEFAULT,-com.mashape.unirest.http.exceptions.UnirestException
...
2020-08-20 17:21:44,041 DEBUG [http-nio-7012-exec-2] org.springframework.transaction.support.AbstractPlatformTransactionManager: Initiating transaction rollback
2020-08-20 17:21:44,044 DEBUG [http-nio-7012-exec-2] org.springframework.orm.jpa.JpaTransactionManager: Rolling back JPA transaction on EntityManager [SessionImpl(621663440<open>)]
2020-08-20 17:21:44,059 DEBUG [http-nio-7012-exec-2] org.springframework.orm.jpa.JpaTransactionManager: Not closing pre-bound JPA EntityManager after transaction
2020-08-20 17:22:40,020 DEBUG [http-nio-7012-exec-2] org.springframework.orm.jpa.support.OpenEntityManagerInViewInterceptor: Closing JPA EntityManager in OpenEntityManagerInViewInterceptor
What I see at the moment is that entity gets pushed to database even if there's a UnirestException. But I expect no data be saved to database.
I am using Spring Boot 2.3.3 with MySQL 5.7. This is the code I have for it.
ProcessRecon.java
#Usecase // this custom annotation is derived from #service
public class ProcessRecon {
private final PaymentRepository paymentRepository;
#Autowired
public ProcessRecon(PaymentRepository paymentRepository) {
this.paymentRepository = paymentRepository;
}
#Transactional(rollbackFor = UnirestException.class)
public Reconciliation execute(final Reconciliation reconciliation) throws UnirestException {
PaymentDetails paymentDetails = paymentRepository.getByReqId(reconciliation.getReqId());
if (paymentDetails == null)
throw new EntityNotFoundException(ExceptionMessages.PAYMENT_DETAILS_NOT_FOUND);
reconciliation.setPaymentDetails(paymentDetails);
Long transId = null;
if (paymentDetails.getImmediateResponse() != null)
transId = paymentDetails.getImmediateResponse().getTransId();
if (transId != null)
reconciliation.setTransId(transId);
if (reconciliation.getTransId() == null)
throw new ValidationException("transId should be provided in Reconciliation if there is no immediate" +
" response for a particular reqId!");
// THIS GETS SAVED
Reconciliation savedRecon = paymentRepository.saveRecon(reconciliation);
paymentDetails.setReconciliation(savedRecon);
// IF THROWS SOME ERROR, ROLLBACK
paymentRepository.sendReconAck(reconciliation);
return savedRecon;
}
}
PaymentRepositoryImpl.java
#CleanRepository
public class PaymentRepositoryImpl implements PaymentRepository {
#Override
public String sendReconAck(final Reconciliation recon) throws UnirestException {
// Acknowledge OP
return sendAck(recon.getRequestType(), recon.getTransId());
}
String sendAck(final String requestType, final Long transId) throws UnirestException {
// TODO: Check if restTemplate can work with characters (requestType)
final Map<String, Object> queryParams = new HashMap<String, Object>();
queryParams.put("transId", transId);
queryParams.put("requestType", requestType);
logger.debug("{}", queryParams);
final HttpResponse<String> result = Unirest.get(makeAckUrl()).queryString(queryParams).asString();
logger.debug("Output of ack with queryParams {} is {}", queryParams, result.getBody());
return result.getBody();
}
#Override
public Reconciliation saveRecon(final Reconciliation recon) {
try {
return reconDS.save(recon);
}
catch (DataIntegrityViolationException ex) {
throw new EntityExistsException(ExceptionMessages.CONSTRAINT_VIOLATION);
}
}
}
ReconciliationDatasource.java
#Datasource // extends from #Repository
public interface ReconciliationDatasource extends JpaRepository<Reconciliation, Long> {
List<Reconciliation> findByPaymentDetails_User_Id(Long userId);
}
To make annotations work you have to use interfaces instead of classes for dependency injection.
interface ProcessRecon {
Reconciliation execute(final Reconciliation reconciliation)
throws UnirestException;
}
Then
#Usecase
public class ProcessReconImpl implements ProcessRecon {
private final PaymentRepository paymentRepository;
#Autowired
public ProcessReconImpl(PaymentRepository paymentRepository) {
this.paymentRepository = paymentRepository;
}
#Transactional(rollbackFor = UnirestException.class)
public Reconciliation execute(final Reconciliation reconciliation) throws UnirestException {
//method implementation...
}
}
Usage
#Autowired
ProcessRecon processRecon;
public void executeServiceMethod(Reconciliation reconciliation) {
processRecon.execute(reconciliation)
}
This way you have got proxy of ProcessReconImpl with provided by annotations additional functionality.
I assumed the default engine for the tables would be InnoDB but to my surpise, the tables were created using MyISAM engine which doesn't support transactions.
I resolved the problem by using the below property as suggested here
spring.jpa.database-platform=org.hibernate.dialect.MySQL5InnoDBDialect
instead of
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
That was the only change required. Thanks!
I would like to load data in cache when my Spring Boot application starts.
I know there is an integrated way using BootstrapCacheLoader in Ehcache2.
How to load data from database to Ehcache when the application starts
But I don't see this in Ehcache3.
I still can do it manually within a #postConstruct method.
But I was wondering if there is an integrating solution (Spring 5, Ehcache 3)
Thank you.
I've ended up doing it after the Spring context has been initialized.
For each element in the DB collection, I call getResourceById() which has the #Cacheable annotation, thus populating the cache for the whole DB collection.
I don't recommend to run this code in a #PostConstruct as proxies may not have been created yet and annotations like #Cacheable may not be working yet.
Running this code when a ContextRefreshedEvent event is triggered (after initialization or after refresh) is a more appropriate place to load the cache.
public class CacheLoader {
private final Logger logger = LoggerFactory.getLogger(CacheLoader.class);
#Autowired
private ResourcePermissionRepository resourcePermissionRepository;
#Autowired
private ResourcePermissionService resourcePermissionService;
#EventListener
public void onApplicationEvent(ContextRefreshedEvent event) {
logger.info("Loading cache following start/refresh event");
for (PermissionGroup permissionGroup : permissionGroupRepository.findAll()) {
permissionGroupService.getGroupById(permissionGroup.getGroupName());
}
for(ResourcePermission resourcePermission: resourcePermissionRepository.findAll()) {
resourcePermissionService.getResourceById(resourcePermission.getResourceId());
}
logger.info("Finished loading cache");
}
public class ResourcePermissionService {
private final Logger logger = LoggerFactory.getLogger(ResourcePermissionService.class);
#Autowired
private ResourcePermissionRepository resourcePermissionRepository;
#Cacheable(value = "resources", sync = true)
public ResourcePermission getResourceById(String resourceId) {
logger.info("Cache miss for resource " + resourceId);
return resourcePermissionRepository.findById(resourceId).orElse(new NullResourcePermission());
}
#CachePut(value = "resources", key = "#result.resourceId")
public ResourcePermission addResourcePermission(ResourcePermission resourcePermission) {
return resourcePermissionRepository.save(resourcePermission);
}
#CacheEvict(value = "resources")
public void deleteById(String resourceId) {
resourcePermissionRepository.deleteById(resourceId);
}
}
So far, the only way I know to set the name of a database, to use with Spring Data ArangoDB, is by hardcoding it in a database() method while extending AbstractArangoConfiguration, like so:
#Configuration
#EnableArangoRepositories(basePackages = { "com.company.mypackage" })
public class MyConfiguration extends AbstractArangoConfiguration {
#Override
public ArangoDB.Builder arango() {
return new ArangoDB.Builder();
}
#Override
public String database() {
// Name of the database to be used
return "example-database";
}
}
What if I'd like to implement multi-tenancy, where each tenant has data in a separate database and use e.g. a subdomain to determine which database name should be used?
Can the database used by Spring Data ArangoDB be determined at runtime, dynamically?
This question is related to the discussion here: Manage multi-tenancy ArangoDB connection - but is Spring Data ArangoDB specific.
Turns out this is delightfully simple: Just change the ArangoConfiguration database() method #Override to return a Spring Expression (SpEL):
#Override
public String database() {
return "#{tenantProvider.getDatabaseName()}";
}
which in this example references a TenantProvider #Component which can be implemented like so:
#Component
public class TenantProvider {
private final ThreadLocal<String> databaseName;
public TenantProvider() {
super();
databaseName = new ThreadLocal<>();
}
public String getDatabaseName() {
return databaseName.get();
}
public void setDatabaseName(final String databaseName) {
this.databaseName.set(databaseName);
}
}
This component can then be #Autowired wherever in your code to set the database name, such as in a servlet filter, or in my case in an Apache Camel route Processor and in database service methods.
P.s. I became aware of this possibility by reading the ArangoTemplate code and a Spring Expression support documentation section
(via), and one merged pull request.
I am having a problem related to JPA & some hibernate listeners I configured to index/deindex the Db entities into Elastic Search. The problem is basically that the listener onPostInsert method is called even if I throw an exception in the method where I am persisting an entity and this method is marked as #Transactional(rollbackFor = {Throwable.class}). My configuration is as follows.
The listener class:
public class ElasticSearchEventListener implements PostDeleteEventListener,
PostInsertEventListener, PostUpdateEventListener {
#Override
public void onPostInsert(PostInsertEvent event) {
log.debug("Listener indexing entity");
try {
updateElasticSearch(event.getEntity());
} catch (Exception e) {
log.debug("Error indexing object from listener");
e.printStackTrace();
}
}
.......
}
The listener configured class:
#Service #Log4j
public class ListenerConfigurerImpl implements ListenerConfigurer {
#Autowired
private EntityManagerFactory entityManagerFactory;
#Autowired
private ElasticSearchEventListener listener;
#PostConstruct #Override
public void registerListeners() {
log.debug("Registering event listeners");
HibernateEntityManagerFactory hibernateEntityManagerFactory = (HibernateEntityManagerFactory) this.entityManagerFactory;
SessionFactoryImpl sessionFactoryImpl = (SessionFactoryImpl) hibernateEntityManagerFactory.getSessionFactory();
EventListenerRegistry registry = sessionFactoryImpl.getServiceRegistry().getService(EventListenerRegistry.class);
registry.getEventListenerGroup(EventType.POST_COMMIT_INSERT).appendListener(listener);
.......
}
}
A service class:
#Service #Log4j
public class ConversationServiceImpl implements ConversationService {
#Override
#Transactional(rollbackFor = {Throwable.class})
public void quotePackage(Long userId, CustomQuoteDTO dto) {
......
Conversation conversation = Conversation.createAndAssign(user, agency, type, subject);
conversation = conversationRepository.save(conversation);
Long conversationId = conversation.getId();
if (1 == 1) throw new RuntimeException();
}
}
Based on this configuration, I would be expecting that the conversation entity is not saved neither in the DB nor Elastic Search. The entity is not persisted in the DB which is correct but for some reason the "onPostInsert" is still executing... and I get the entity in Elastic Search even if it is not in the Database.
Any ideas? I am a bit lost.
Thanks in advance.
EDIT 1 ------
I have found this bug from 2006 and it is still open that seems to be my problem: https://hibernate.atlassian.net/browse/HHH-1582
Is this supposed to work this way?
The pull request added here https://hibernate.atlassian.net/browse/HHH-1582 fixes this issue.