We Keep Coding

How to implement a Spring XD sink?

So far I have implemented Spring XD processors, e.g. like this:
#MessageEndpoint
public class MyTransformer
{
#Transformer( inputChannel = "input", outputChannel = "output" )
public String transform( String payload )
{
...
}
};
However, I am stuck at implementing a custom sink now. The current documentation is not very helpful, since it simply configures something "magically" via XML:
<beans ...>
<int:channel id="input" />
<int-redis:store-outbound-channel-adapter
id="redisListAdapter" collection-type="LIST" channel="input" key="${collection}" auto-startup="false"/>
<beans:bean id="redisConnectionFactory"
class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<beans:property name="hostName" value="${host}" />
<beans:property name="port" value="${port}" />
</beans:bean>
</beans>
This will use the redis store-outbound-channel-adapter as a sink. However, the documentation does not tell me how to create a simple, generic sink that simply has one input channel and consumes a message.
So can anyone provide me with a minimal working example?

A sink is just like a processor but without an output channel; use a #ServiceActivator to invoke your code (which should have a void return).
#MessageEndpoint
public class MyService
{
#ServiceActivator( inputChannel = "input")
public void handle( String payload )
{
...
}
};
EDIT
For sources, there are two types:
Polled (messages are pulled from the source):
#InboundChannelAdapter(value = "output",
poller = #Poller(fixedDelay = "5000", maxMessagesPerPoll = "1"))
public String next() {
return "foo";
}
Message-driven (where the source pushes messages):
#Bean
public MySource source() {
// return my subclass of MessageProducer that has outputChannel injected
// and calls sendMessage
// or use a simple POJO that uses MessagingTemplate.convertAndSend(foo)
}

jdbc update query no working on unit test with derby database and SpringJUnit4ClassRunner

I am having problems trying to do an unit test.
The test class is simple like:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {"/job-runner-context.xml})
public class AtualizarDataServiceTest {
#Autowired
DataSource testDataSource;
#Autowired
Service service;
#Before
public void setUp() throws DatabaseUnitException, SQLException{
service.setDataSource(testDataSource);
}
#Test
public final void testUpdateDate() throws SQLException {
assertTrue(verifyDate());
service.updateDate();
assertFalse(verifyDate()); //Assert brokes here
}
private boolean verifyDate(){
SimpleJdbcTemplate consultaTemplate = new SimpleJdbcTemplate(testDataSource);
int count = consultaTemplate.queryForInt("SELECT COUNT(*) FROM MY_TABLE");
return count == 0;
}
}
The service:
public class Service {
private DataSource dataSource;
public void updateDate(){
SimpleJdbcTemplate jdbcTemplate = new SimpleJdbcTemplate(getDataSource());
String query = "UPDATE MY_TABLE SET DT_UPDATE_OPERATION = ?";
jdbcTemplate.update(query, new Object[]{new java.sql.Date(Calendar.getInstance().getTime().getTime())});
}
public DataSource getDataSource() {
return dataSource;
}
public void setDataSource(DataSource dataSource) {
this.dataSource = dataSource;
}
}
job-runner-context.xml important pieces:
<context:annotation-config/>
<context:component-scan base-package="my.package"/>
<bean class="my.package.Service"/>
<bean id="testDataSource" class="com.read.only.MyBasicDataSource" destroy-method="close" lazy-init="true">
<property name="jdbcReference" value="derby" />
</bean>
the jdbc connection properties:
<com:jdbcReference name="derby" type="DATABASE">
<com:credential user="" password="" />
<com:autocommit arg="false" />
<com:databaseConfig driverClassName="org.apache.derby.jdbc.EmbeddedDriver"
url="jdbc:derby:dbderby;create=true" databaseName="ANY" />
</com:jdbcReference>
At fist I thought the problem was related to commit issues but I tried set the value of autocommit properties to "true" and also manually call testDataSource.getConnection().commit() but it didn't work. The code and methods are working fine but the test isn't updating the derby database. In other test classes data is preset with dbUnit in the same database and the code works. This answer here gives an general list of possible problems I checked and I am reading and writing to the same tables in the same schemas. Am I missing something?

Try setup <com:autocommit arg="true" />.

As the answer to the question I posted I verified the autocommit and if I was writing to the right database, but didn't check the obvious: you can't update a table with no registers! The query UPDATE MY_TABLE SET DT_UPDATE_OPERATION = ? was applied to an empty table and the count query would always return 0. I just configured the test to make DbUnit import an state to the database from a xml file. Sorry for the trouble.

Atomically maintaining service layer transactions and database logging with Spring framework

I have a web application implemented using Spring and Hibernate. A typical controller method in the application looks like the following:
#RequestMapping(method = RequestMethod.POST)
public #ResponseBody
Foo saveFoo(#RequestBody Foo foo, HttpServletRequest request) throws Exception {
// authorize
User user = getAuthorizationService().authorizeUserFromRequest(request);
// service call
return fooService.saveFoo(foo);
}
And a typical service class looks like the following:
#Service
#Transactional
public class FooService implements IFooService {
#Autowired
private IFooDao fooDao;
#Override
public Foo saveFoo(Foo foo) {
// ...
}
}
Now, I want to create a Log object and insert it to database every time a Foo object is saved. These are my requirements:
The Log object should contain userId from the authorised User object.
The Log object should contain some properties from the HttpServletRequest object.
The save operation and log creation operation should be atomic. I.e. if a foo object is saved in the object we should have a corresponding log in the database indicating the user and other properties of the operation.
Since transaction management is handled in the service layer, creating the log and saving it in the controller violates the atomicity requirement.
I could pass the Log object to the FooService but that seems to be violation of separation of concerns principle since logging is a cross cutting concern.
I could move the transactional annotation to the controller which is not suggested in many of the places I have read.
I have also read about accomplishing the job using spring AOP and interceptors about which I have very little experience. But they were using information already present in the service class and I could not figure out how to pass the information from HttpServletRequest or authorised User to that interceptors.
I appreciate any direction or sample code to fulfill the requirements in this scenario.

There are multiple steps which are to be implemented to solve your problem:
Passing Log object non-obtrusively to service classes.
Create AOP based interceptors to start inserting Log instances to DB.
Maintaining the order to AOP interceptors (Transaction interceptor and Log interceptor) such that transaction interceptor is invoked first. This will ensure that user insert and log insert happens in a single transaction.
1. Passing Log object
You can use ThreadLocal to set the Log instance.
public class LogThreadLocal{
private static ThreadLocal<Log> t = new ThreadLocal();
public static void set(Log log){}
public static Log get(){}
public static void clear(){}
}
Controller:saveFoo(){
try{
Log l = //create log from user and http request.
LogThreadLocal.set(l);
fooService.saveFoo(foo);
} finally {
LogThreadLocal.clear();
}
}
2. Log Interceptor
See how spring AOP works (http://docs.spring.io/spring/docs/current/spring-framework-reference/html/aop-api.html)
a) Create an annotation (acts as pointcut), #Log for method level. This annotation will be put on the service methods for which logging is to be done.
#Log
public Foo saveFoo(Foo foo) {}
b) Create an implementation, LogInteceptor (acts as the advice) of org.aopalliance.intercept.MethodInterceptor.
public class LogInterceptor implements MethodInterceptor, Ordered{
#Transactional
public final Object invoke(MethodInvocation invocation) throws Throwable {
Object r = invocation.proceed();
Log l = LogThreadLocal.get();
logService.save(l);
return r;
}
}
c) Wire the pointcut & advisor.
<bean id="logAdvice" class="com.LogInterceptor" />
<bean id="logAnnotation" class="org.springframework.aop.support.annotation.AnnotationMatchingPointcut">
<constructor-arg type="java.lang.Class" value="" />
<constructor-arg type="java.lang.Class" value="com.Log" />
</bean>
<bean id="logAdvisor" class="org.springframework.aop.support.DefaultPointcutAdvisor">
<property name="advice" ref="logAdvice" />
<property name="pointcut" ref="logAnnotation" />
</bean>
3. Ordering of interceptors (transaction and log)
Make sure you implement org.springframework.core.Ordered interface to LogInterceptor and return Integer.MAX_VALUE from getOrder() method. In your spring configuration, make sure your transaction interceptor has lower order value.
So, first your transaction interceptor is called and creates a transaction. Then, your LogInterceptor is called. This interceptor first proceed the invocation (saving foo) and then save log (extracting from thread local).

One more example based Spring AOP but using java configuration, I hate XMLs :) Basically the idea is almost the same as mohit has but without ThreadLocals, Interceptor Orders and XML configs:)
So you will need :
#Loggable annotation to mark methods as the once which create the logs.
TransactionTemplate which we will use to programmatically control the transactions.
Simple Aspect which will put every thing in its place.
So at first lets create the annotation
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.METHOD)
public #interface Loggable {}
If you are missing the TransactionTemplate configuration or EnableAspectJAutoProxy just add following to your Java Config.
#EnableAspectJAutoProxy
#Configuration
public class ApplicationContext {
.....
#Bean
TransactionTemplate transactionTemplate(PlatformTransactionManager transactionManager){
TransactionTemplate template = new TransactionTemplate();
template.setTransactionManager(transactionManager);
return template;
}
}
And next we will need an Aspect which will do all the magic :)
#Component
#Aspect
public class LogAspect {
#Autowired
private HttpServletRequest request;
#Autowired
private TransactionTemplate template;
#Autowired
private LogService logService;
#Around("execution(* *(..)) && #annotation(loggable)")
public void logIt(ProceedingJoinPoint pjp, Loggable loggable) {
template.execute(s->{
try{
Foo foo = (Foo) pjp.proceed();
Log log = new Log();
log.setFoo(foo);
// check may be this is a internal call, not from web
if(request != null){
log.setSomeRequestData(request.getAttribute("name"));
}
logService.saveLog(log);
} catch (Throwable ex) {
// lets rollback everything
throw new RuntimeException();
}
return null;
});
}
}
And finally in your FooService
#Loggable
public Foo saveFoo(Foo foo) {}
Your controller remains the same.

If you use LocalSessionFactoryBean or it's subclass (for instance AnnotationSessionFactoryBean) with inside your Spring context, then the best option would be using entityInterceptor property. You have to pass instance of orh.hibernate.Interceptor interface. For instance:
// java file
public class LogInterceptor extends ScopedBeanInterceptor {
// you may use your authorization service to retrieve current user
#Autowired
private AutorizationService authorizationService
// or get the user from request
#Autowired
private HttpServletRequest request;
#Override
public boolean onSave(final Object entity, final Serializable id, final Object[] state, final String[] propertyNames, final Type[] types) {
// get data from request
// your save logic here
return true;
}
}
// in spring context
<bean id="sessionFactory"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean" destroy-method="destroy">
<property name="dataSource" ref="dataSource"/>
<property name="hibernateProperties">
....
</property>
....
<property name="entityInterceptor" ref="logInterceptor"/>
</bean>
Add the following to your web.xml (or add listener in java code, depending on what you use).
<listener>
<listener-class>
org.springframework.web.context.request.RequestContextListener
</listener-class>
</listener>
Add request scope bean, so it'll be request aware.
<bean id="logInterceptor" class="LogInterceptor" scope="request">
<aop:scoped-proxy proxy-target-class="false" />
</bean>
You can separate log data fetch from interceptor, so there will be a different request scoped component, or also you can use filters to store data in ThreadLocal.

Spring #Transactional rollbackFor not working for checked Exception for method called outside proxy object

I am having a problem with the rollback of Hibernate updates in combination with Spring.
I have the following class:
#Service
#Transactional(readOnly = true)
public class DocumentServiceImpl extends AbstractGenericService<Document, IDocumentDao> implements DocumentService {
#Override
#Transactional(readOnly=false, rollbackFor = DocumentServiceException.class)
public void saveDocument(final DocumentForm form, final BindingResult result, final CustomUserContext userContext) throws DocumentServiceException {
Document document = locateDocument(form, userContext);
if (!result.hasErrors()) {
try {
updateDocumentCategories(form, document);
storeDocument(document, form.getDocumentId(), form.getFile());
solrService.addDocument(document);
} catch (IOException e) {
result.reject("error.uploading.file");
throw new DocumentServiceException("Error trying to copy the uploaded file to its final destination", e);
} catch (SolrServerException e) {
result.reject("error.uploading.file.solr");
throw new DocumentServiceException("Solr had an error parsing your uploaded file", e);
}
}
}
#Override
#Transactional(readOnly = false, rollbackFor = IOException.class)
public void storeDocument(Document document, String documentId, CommonsMultipartFile uploadedFile) throws IOException {
getDao().saveOrUpdate(document);
if (StringUtils.isBlank(documentId)) {
File newFile = documentLocator.createFile(document);
uploadedFile.transferTo(newFile);
// Todo: TEST FOR ROLLBACK ON FILE I/O EXCEPTION
throw new IOException("this is a test");
}
}
The interface is not tagged with any #Transactional annotations. The saveDocument() method is called directly from my Controller, so I expect that the #Transactional configuration of that method is used, particularly the rollbackFor parameter. However, when the DocumentServiceException is thrown, nothing is rolled back (ie the getDao().saveOrUpdate(document) is persisted). For testing purposes I added an "throw new IOException" in the storeDocument method. Hope anybody can help me out how to get this working, it would be greatly appreciated.

The #Transactional annotation is properly placed. You don;t have to set it at interface level, because it's not automatically inherited (what if your concrete class implements two interfaces with a conflicting transactional setting).
When you say the method are called directly, I assume you have the interface being #Autowired and not the concrete implementation.
Place a break point in your service method and check if your have a TransactionInterceptor entry in your stack trace. If you don't have it, then your transaction management configuration is wrong and you are not using Spring transaction management at all.
Update from Dennis
One more thing that could help other people perhaps:
I had the tx:annotation-driven in my applicationContext. The applicationContext contained a component-scan for all beans (no filters).
However, the dispatcherServlet context also contained a component-scan for all beans (legacy code, don't shoot the messenger). So basically I had a copy of all my beans because they were scanned in both contexts.
And because the beans created in the dispatcherServlet context did not contain the tx:annotation-driven element, the services beans in the dispatcherServlet context were not transactional.
I had to change the component-scan in the dispatcherServlet context into:
<context:component-scan base-package="your/base/package" use-default-filters="false">
<context:include-filter type="annotation" expression="org.springframework.stereotype.Controller"/>
</context:component-scan>
So, it will only instantiate the controllers in the dispatcher servlet context (and no autowired dependencies, like its services), and the services/daos in the applicationContext.
The services from the applicationContext are then transactionalized.

One transaction for several JpaRepositories' methods

I have some autowired interfaces extends JpaRepository. Each of them has "update" hql method, for different entities. I'm calling these methods from service's method.
How i can make all of them executing in ONE transaction for rolling back all data, if one of them will fail?
service has attribuites #service & #transactional, but it doesnt help.
------------------------ update
Here an example. repository1.updateMethod() and repository2.updateMethod() works fine, repository3.save throws exception bacause of constraint error. In result, i see that results of repository1 and repository2 methods saved. I need it to roll back.
# Service
# Transactional(rollbackFor = {RuntimeException.class})
public SomeService {
# Autowired SomeRepository repository1;
# Autowired AnotherRepository repository2;
# Autowired ThirdRepository repository3;
...
# Transactional(rollbackFor = {RuntimeException.class})
public void SomeMethod(SomeEntity obj, String someNewValue) {
try {
repository1.updateMethod();
repository2.updateMethod();
obj.setValue(someNewValue);
repository3.save(obj);
} catch (Exception ex) {
throw new RuntimeException();
}
}
}

I think i found one solution, but I still dont understand why it doesnt work with default usage.
First, I declared JpaTransactionManager at applidationData.xml
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory"/>
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
And used it in servise:
#Autowired
JpaTransactionManager jtm;
Before my code i added:
DefaultTransactionDefinition def = new DefaultTransactionDefinition();
def.setName("TxName");
def.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
TransactionStatus ts = jtm.getTransaction(def);
And at the finish
jtm.commit(ts);
Now, if some method produces exception, it throws at the last line and all updates is rolling back. That is what i needed. But, as i said, I still dont understand why it doesnt work with default usage.