We Keep Coding

CouchbasePOJOSetOperator is causing start up problems when running Apache Apex app

I have a basic Apex stream with a Kafka input operator feeding into a Couchbase set operator. The Kafka sides works fine and have proven it by removing the couch operator. I have tried switching around versions of the Malhar library to see if it was broken in the latest.
Also I am using the DataTorrent flavor of Apex.
When I add in the couch operator I get the following exception
java.lang.RuntimeException: Error creating local cluster
at org.apache.apex.engine.EmbeddedAppLauncherImpl.getController(EmbeddedAppLauncherImpl.java:124)
at org.apache.apex.engine.EmbeddedAppLauncherImpl.getController(EmbeddedAppLauncherImpl.java:46)
at com.usaa.data.streams.apex.ApplicationTest.testApplication(ApplicationTest.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be created (missing no-arg constructor): org.codehaus.jackson.map.DeserializationConfig
Serialization trace:
_deserializationConfig (org.codehaus.jackson.map.ObjectMapper)
mapper (com.datatorrent.contrib.couchbase.CouchBaseJSONSerializer)
serializer (com.datatorrent.contrib.couchbase.CouchbasePOJOSetOperator)
at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInstantiatorOf(Kryo.java:1228)
at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049)
at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058)
at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:547)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:523)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.java:192)
at com.datatorrent.stram.plan.logical.LogicalPlan$OperatorMeta.readObject(LogicalPlan.java:898)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2122)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2013)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at java.util.ArrayList.readObject(ArrayList.java:791)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2122)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2013)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2231)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2155)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2013)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at com.datatorrent.stram.plan.logical.LogicalPlan.read(LogicalPlan.java:2326)
at com.datatorrent.stram.StramLocalCluster.cloneLogicalPlan(StramLocalCluster.java:323)
at com.datatorrent.stram.StramLocalCluster.<init>(StramLocalCluster.java:285)
at org.apache.apex.engine.EmbeddedAppLauncherImpl.getController(EmbeddedAppLauncherImpl.java:122)
... 24 more
Here is the associated Application code:
KafkaSinglePortInputOperator kafkaInput = dag.addOperator("kafkaInput", KafkaSinglePortInputOperator.class);
kafkaInput.setTopics("testing2");
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "test");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("enable.auto.commit", "true");
kafkaInput.setConsumerProps(props);
kafkaInput.setClusters("localhost:9092");
CouchbasePOJOSetOperator couchOutput = dag.addOperator("couchOutput", CouchbasePOJOSetOperator.class);
CouchBaseWindowStore store = new CouchBaseWindowStore();
store.setBucket("default");
store.setUriString("localhost:8091,localhost:8091");
store.setUserConfig("");
store.setPasswordConfig("");
store.setPassword("");
couchOutput.setStore(store);
try{
store.connect();
}catch (Exception e){
e.printStackTrace();
}
CouchBaseJSONSerializer serializer = new CouchBaseJSONSerializer();
couchOutput.setSerializer(serializer);
ArrayList<String> expressions = new ArrayList<String>();
expressions.add("getValue()");
couchOutput.setExpressions(expressions);
dag.addStream("kafkaInput", kafkaInput.outputPort, couchOutput.input).setLocality(Locality.CONTAINER_LOCAL);

Spring Batch: Ignore any line that does not match the specified pattern

I have a requirement where I read a file with different kinds of input as below:
*JAMBEG,APP=000007,123456
AC,654321,“ABCD12121212121212”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC,TREX,000000002
AC,654321,“ABCD12121212121213”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC, TREX,000000002
...
AC,654321,“ABCD12121212121214”,23423423423424234,ABCDD,23423423423424234,2424,XYZ,ABC, TREX,000000002
*JAMEND,APP=000007,123456
EOF
I need only the Header line and the records following that, ignoring the line that starts with TREX, *JAMEND, EOF.
Here is how my line mapper is:
public LineMapper<Customer> lineMapper(){
DelimitedLineTokenizer lineTokenizerHeader = new DelimitedLineTokenizer();
lineTokenizerHeader.setNames(new String[]{"association","companyNumber","fileDate"});
lineTokenizerHeader.setIncludedFields(new int[]{0,1,2});
lineTokenizerHeader.setStrict(false);
DelimitedLineTokenizer lineTokenizerBody = new DelimitedLineTokenizer();
lineTokenizerBody.setNames(new String[]{"type","acNumber","orderNumber"});
lineTokenizerBody.setIncludedFields(new int[]{0,1,2});
lineTokenizerBody.setStrict(false);
HashMap<String, DelimitedLineTokenizer> tokenizers = new HashMap<String, DelimitedLineTokenizer>();
tokenizers.put("*BEG*", lineTokenizerHeader);
tokenizers.put("AC*", lineTokenizerBody);
BeanWrapperFieldSetMapper<Customer> beanWrapperFieldSetMapper = new BeanWrapperFieldSetMapper<Customer>();
beanWrapperFieldSetMapper.setTargetType(Customer.class);
beanWrapperFieldSetMapper.setStrict(false);
HashMap<String, BeanWrapperFieldSetMapper<Customer>> fieldSetMappers = new HashMap<String, BeanWrapperFieldSetMapper<Customer>>();
fieldSetMappers.put("*BEG*", beanWrapperFieldSetMapper);
fieldSetMappers.put("AC*", beanWrapperFieldSetMapper);
PatternMatchingCompositeLineMapper patternMatchingCompositeLineMapper = new PatternMatchingCompositeLineMapper();
patternMatchingCompositeLineMapper.setTokenizers(tokenizers);
patternMatchingCompositeLineMapper.setFieldSetMappers(fieldSetMappers);
return patternMatchingCompositeLineMapper;
}
Its my obvious mistake that I don't have any mapping for TREX, *JAMEND, EOF patterns. Hence it throws the below exception:
2014-06-16 16:49:34,746 [main] DEBUG
org.springframework.batch.core.step.item.FaultTolerantChunkProvider -
Parsing error at line: 5 in resource=[class path resource
[0000123456.csv]], input=[EOF] :
org.springframework.batch.item.file.FlatFileParseException 2014-06-16
16:49:34,746 [main] DEBUG
org.springframework.batch.core.step.item.FaultTolerantChunkProvider -
Skipping failed input
org.springframework.batch.item.file.FlatFileParseException: Parsing
error at line: 5 in resource=[class path resource [0000123456.csv]],
input=[EOF] at
org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:183)
at
org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:83)
at
org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91)
at
org.springframework.batch.core.step.item.FaultTolerantChunkProvider.read(FaultTolerantChunkProvider.java:87)
at
org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:114)
at
org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
at
org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at
org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at
org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:108)
at
org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69)
at
org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:402)
at
org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:326)
at
org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
at
org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:267)
at
org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77)
at
org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
at
org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at
org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at
org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:253)
at
org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:198)
at
org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
at
org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:386)
at
org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135)
at
org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:304)
at
org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
at
org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
at
org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:117)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy17.run(Unknown Source) at
com.chofac.pm.batch.CustomerFileToDBJobTest.testLaunchJob(CustomerFileToDBJobTest.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
at
org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
at
org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
at
org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at
org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at
org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at
org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.IllegalStateException: Could not find a matching
pattern for key=[EOF] at
org.springframework.batch.support.PatternMatcher.match(PatternMatcher.java:226)
at
org.springframework.batch.item.file.mapping.PatternMatchingCompositeLineMapper.mapLine(PatternMatchingCompositeLineMapper.java:62)
at
org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:180)
... 67 more
I looked at many examples, this one matched close, and changed my step as below, but still the same issue.
#Bean
public Step step(){
return stepBuilders.get("step")
.<Customer,Customer>chunk(1)
.reader(CustomerAUFileReader())
.faultTolerant()
.skipLimit(3)
.skip(Exception.class)
.processor(CustomerRecordProcessor())
.writer(CustomerDBWriter())
.listener(logProcessListener())
.build();
}
Looked at the Spring.io docs here for skipping records (5.1.5 Configuring Skip Logic), does not work either.
Please let me know the ideal way to get around this issue. Should there not be an easy way to specify skipping records that do not match specific cases? Please advise. Thanks.
---
I have a pattern mapper for '*' which I am mapping with a dummy class. I am returning null at the process stage but its throwing nullpointerexception.
Stack Trace:
2014-06-17 10:03:01,690 [main] DEBUG org.springframework.batch.core.step.item.FaultTolerantChunkProcessor - Skipping after failed process
org.springframework.batch.core.listener.StepListenerFailedException: Error in afterProcess.
at org.springframework.batch.core.listener.MulticasterBatchListener.afterProcess(MulticasterBatchListener.java:136)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.doProcess(SimpleChunkProcessor.java:127)
at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor$1.doWithRetry(FaultTolerantChunkProcessor.java:225)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:263)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:193)
at org.springframework.batch.core.step.item.BatchRetryTemplate.execute(BatchRetryTemplate.java:217)
at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor.transform(FaultTolerantChunkProcessor.java:290)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:192)
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:402)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:326)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:267)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:253)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:198)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:386)
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:304)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:117)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy17.run(Unknown Source)
at com.chofac.pl.batch.CustomerFileToDBJobTest.testLaunchJob(CustomerFileToDBJobTest.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.NullPointerException
at com.chofac.pl.batch.CustomerItemProcessListener.afterProcess(CustomerItemProcessListener.java:13)
at org.springframework.batch.core.listener.CompositeItemProcessListener.afterProcess(CompositeItemProcessListener.java:60)
at org.springframework.batch.core.listener.MulticasterBatchListener.afterProcess(MulticasterBatchListener.java:133)

One approach:
Have your ItemReader simply read a line and return as is. Therefore the items given by the reader will be a simple String.
Write a simple ItemProcessor, which mostly do the work of your LineMapper, base on a pattern for example: if the item matches with a pattern, then translate the input string to your Customer return. If pattern not matching, simply return null and the item will be skipped.
psuedo code for the item processor:
class CustomPatternMatchingItemProcessor<String, Customer>
implements ItemProcessor<String, Customer> {
private String pattern;
public Customer process(String s) {
if (s matches pattern) {
construct Customer object base on s
return customer
} else {
return null;
}
}
}
Or even cleaner: have one processor do the work of mapping from String to Customer, and another processor to do the validation of string base on regex.
just chain your processors using a CompositeItemProcessor. This give a even better separation of concern for each of your processor.

I have a similiar issue, where the file I am sent will always have some lines that don't fit the correct pattern. As I just want to ignore these lines, I log them, and skip them.
You can implement
org.springframework.batch.repeat.exception.ExceptionHandler like such:
class LocalExceptionHandler implements ExceptionHandler {
#Override
public void handleException(RepeatContext rc, Throwable throwable) throws Throwable {
if (throwable instanceof FlatFileParseException) {
FlatFileParseException fe = (FlatFileParseException)throwable;
log.error("!!!! FlatFileParseException, line # is: " + fe.getLineNumber());
log.error("!!!! FlatFileParseException, input is: " + fe.getInput());
}
log.error("!!!! Message : " + throwable.getMessage());
log.error("!!!! Cause : " + throwable.getCause());
}
}
and then add that to your step builder:
faultTolerantStepBuilder.exceptionHandler(new LocalExceptionHandler());
faultTolerantStepBuilder.skipLimit(100);

Your intention is not to skip object due to errors, but skip records with a logic; I think your best option is put a mapper binded to '*' and return a custom object (like a SkippableRecordBean) instead of a Customer and filter out unwanted beans in ItemProcessor.

We started returning Dummy object from the LineMapper instead of null as null causes the reader to skip reading other lines. In the ItemWriter, the write method still gets this dummy object and you need to validate before processing it. We ignore the dummy records in the ItemWriter.

public class FileVerificationSkipper implements SkipPolicy {
#Autowired
private Environment environment;
#Override
public boolean shouldSkip(Throwable exception, int skipCount) throws SkipLimitExceededException {
int skipErrorsRecords = Integer.valueOf(environment.getProperty("max.error.record.count"));
if (exception instanceof FileNotFoundException) {
return false;
} else if (exception instanceof FlatFileParseException && (skipErrorsRecords < 0 || skipCount <= (skipErrorsRecords-1))) {
FlatFileParseException ffpe = (FlatFileParseException) exception;
StringBuilder errorMessage = new StringBuilder();
errorMessage.append((skipCount+1)+", An error occured while processing the " + ffpe.getLineNumber() + " record of the file. the faulty record is:\n");
errorMessage.append(ffpe.getInput() + "\n");
return true;
} else {
return false;
}
}
}
and StepBuilder is:
#Bean
public Step step1() throws Exception{
return stepBuilderFactory.get("step1")
.<User, User> chunk(50)
.reader(reader())
.faultTolerant()
.skipPolicy(fileVerificationSkipper())
.processor(processor())
.writer(writer())
.build();
}

Error in test when updating #ManyToMany relation

I got problem when loading my test:
#PersistenceContext
private EntityManager manager;
#Test
#Transactional
public void testService() {
VideoDvd dvd1 = new VideoDvd("Hell Boy", "horror", 15L);
VideoDvd dvd2 = new VideoDvd("Bonby", "anime", 10L);
VideoDvd dvd3 = new VideoDvd("Riddick", "action", 20L);
VideoDvd dvd4 = new VideoDvd("Butterfly effect", "sci-fi", 5L);
VideoDvd dvd5 = new VideoDvd("Cowboy", "western", 10L);
VideoDvd dvd6 = new VideoDvd("Hell Jigsaw", "horror", 15L);
Customer customer = new Customer("hichem");
manager.persist(dvd1);
manager.persist(dvd2);
manager.persist(dvd3);
manager.persist(dvd4);
manager.persist(dvd5);
manager.persist(dvd6);
manager.persist(customer);
manager.flush();
manager.clear();
customer.addDvd(dvd1);
customer.addDvd(dvd2);
customer.addDvd(dvd3);
customer.addDvd(dvd4);
customer.addDvd(dvd5);
customer.addDvd(dvd6);
manager.persist(customer);
assertEquals(Long.valueOf(75L), service.calculateRentDvDOwner(customer));
}
About Entities : we have Customer and VideoDvd (and each of them have #ManyToMany relationship , that means customer can rent many movies and movie can be rent by many customer),
for table join I have this :
for customer entity:
#ManyToMany(fetch=FetchType.LAZY)
#JoinTable(name="storage",
joinColumns=#JoinColumn(name="custumer_id"),
inverseJoinColumns=#JoinColumn(name="dvd_id"))
private List<VideoDvd> dvdsRent = new ArrayList<VideoDvd>();
PS.: the same thing is done for VideoDvd entity
The stack Error is there:
javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1214)
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1147)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:255)
at org.video.store.dao.impl.VideoDvdDaoImpl.findDvdsRentByCustomer(VideoDvdDaoImpl.java:128)
at org.video.store.services.impl.VideoStoreServiceImpl.calculateRentDvDOwner(VideoStoreServiceImpl.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy29.calculateRentDvDOwner(Unknown Source)
at org.video.store.test.VideoStoreServiceTests.testService(VideoStoreServiceTests.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:82)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:96)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:275)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:268)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:187)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:64)
at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1185)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1261)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:246)
... 42 more
**Caused by: org.h2.jdbc.JdbcBatchUpdateException: NULL not allowed for column "CUSTOMER_ID"; SQL statement**:
insert into storage (custumer_id, dvd_id) values (?, ?) [23502-156]
at org.h2.jdbc.JdbcPreparedStatement.executeBatch(JdbcPreparedStatement.java:1107)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
... 50 more

Error in processing paths generated by XML file using XPath

I have XML file andi'm trying to use XPath to get all the values for it's paths, but it doesn't generate values for the NameSpaces like :
//rss/channel/item/yweather:forecast/#low
OR
//rss/channel/yweather:wind/text()
The CODE :
The Methods Used:
public class XmlSource{
private XPathFactory factory = XPathFactory.newInstance();
private XPath xpath = factory.newXPath();
public Map<String, Object> update(String XML) throws ParserConfigurationException, SAXException, IOException, XPathExpressionException, TransformerException{
XmlSource xml = new XmlSource();
Map<String, Object> result = new HashMap<String, Object>();
Document doc = convert(XML); // converting the String to a document (tested and working efficiently)
xml.doc = doc;
xpath.setNamespaceContext(new NamespaceResolver(xml.doc));
ArrayList<String> paths = xml.getAllPaths(); // to get the listed above paths (tested and working efficiently)
for(int i=0; i < paths.size() ; i++){
result.put(paths.get(i), getValue(paths.get(i), xml.doc));
}
return result;
}
public String getValue(String path, Document doc) throws TransformerException, IOException, ParserConfigurationException, SAXException {
this.doc = doc;
try {
XPathExpression expr = xpath.compile(path);
Object result = expr.evaluate(this.doc, XPathConstants.STRING);
return result.toString();
} catch (XPathExpressionException ex) {
return null;
}
}
}
The NamespaceResolver class:
import java.util.Iterator;
import javax.xml.XMLConstants;
import javax.xml.namespace.NamespaceContext;
import org.w3c.dom.Document;
public class NamespaceResolver implements NamespaceContext {
private Document sourceDocument;
public NamespaceResolver(Document document) {
sourceDocument = document;
}
#Override
public String getNamespaceURI(String prefix) {
if (prefix.equals(XMLConstants.DEFAULT_NS_PREFIX)) {
return sourceDocument.lookupNamespaceURI(null);
} else {
return sourceDocument.lookupNamespaceURI(prefix);
}
}
#Override
public String getPrefix(String namespaceURI) {
return sourceDocument.lookupPrefix(namespaceURI);
}
#Override
public Iterator<String> getPrefixes(String namespaceURI) {
return null;
}
}
The Exceptions Thrown:
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
--------------- linked to ------------------
javax.xml.xpath.XPathExpressionException
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
//rss/channel/item/yweather:condition/#text
//rss/channel/item/yweather:forecast/#code
//rss/channel/item/yweather:forecast/#date
//rss/channel/item/yweather:forecast/#day
//rss/channel/item/yweather:forecast/#high
//rss/channel/item/yweather:forecast/#low
//rss/channel/item/yweather:forecast/#text
//rss/channel/item/yweather:forecast/#code
//rss/channel/item/yweather:forecast/#date
//rss/channel/item/yweather:forecast/#day
//rss/channel/item/yweather:forecast/#high
//rss/channel/item/yweather:forecast/#low
//rss/channel/item/yweather:forecast/#text
--------------- linked to ------------------
javax.xml.xpath.XPathExpressionException
dAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: com.sun.org.apache.xpath.internal.domapi.XPathStylesheetDOM3Exception: Prefix must resolve to a namespace: yweather
at com.sun.org.apache.xpath.internal.compiler.XPathParser.errorForDOM3(Unknown Source)
at com.sun.org.apache.xpath.internal.compiler.Lexer.mapNSTokens(Unknown Source)
at com.sun.org.apache.xpath.internal.compiler.Lexer.tokenize(Unknown Source)
at com.sun.org.apache.xpath.internal.compiler.Lexer.tokenize(Unknown Source)
at com.sun.org.apache.xpath.internal.compiler.XPathParser.initXPath(Unknown Source)
at com.sun.org.apache.xpath.internal.XPath.<init>(Unknown Source)
at com.sun.org.apache.xpath.internal.XPath.<init>(Unknown Source)
... 30 more
com.sun.org.apache.xpath.internal.domapi.XPathStylesheetDOM3Exception: Prefix must resolve to a namespace: yweather
at com.sun.org.apache.xpath.internal.compiler.XPathParser.errorForDOM3(Unknown Source)
at com.sun.org.apache.xpath.internal.compiler.Lexer.mapNSTokens(Unknown Source)
at com.sun.org.apache.xpath.internal.compiler.Lexer.tokenize(Unknown Source)
at com.sun.org.apache.xpath.internal.compiler.Lexer.tokenize(Unknown Source)
at com.sun.org.apache.xpath.internal.compiler.XPathParser.initXPath(Unknown Source)
at com.sun.org.apache.xpath.internal.XPath.<init>(Unknown Source)
at com.sun.org.apache.xpath.internal.XPath.<init>(Unknown Source)
at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.compile(Unknown Source)
at com.nc.inotify.dp.xml.impl.XmlSource.getValue(XmlSource.java:180)
at com.nc.inotify.dp.xml.impl.XmlSource.update(XmlSource.java:284)
at com.nc.inotify.dp.xml.impl.XmlSourceFactory.getSource(XmlSourceFactory.java:155)
at com.nc.inotify.dp.xml.junit.TXmlSourceFactory.tesGetSource(TXmlSourceFactory.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
--------------- linked to ------------------
javax.xml.xpath.XPathExpressionException
at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.compile(Unknown Source)
at com.nc.inotify.dp.xml.impl.XmlSource.getValue(XmlSource.java:180)
at com.nc.inotify.dp.xml.impl.XmlSource.update(XmlSource.java:284)
at com.nc.inotify.dp.xml.impl.XmlSourceFactory.getSource(XmlSourceFactory.java:155)
at com.nc.inotify.dp.xml.junit.TXmlSourceFactory.tesGetSource(TXmlSourceFactory.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
//rss/channel/item/yweather:forecast/text()
//rss/channel/item/guid/text()
DONE

I was able to reproduce your error by using a DocumentBuilderFactory that was not namespace aware.
Suppose your convert method looks something like this:
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(inputSource);
Then you could try this:
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
dbFactory.setNamespaceAware(true);
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(inputSource);
This would explain why the NamespaceResolver isn't able to get the namespaceURI.

How to implement Redis Multi-Exec by using Spring-data-Redis

All Spring Configuration is written properly. Non-Multi-Exec Redis operations work perfectly.
#Autowired
#Qualifier("stringRedisTemplate")
StringRedisTemplate template;
void test(){
template.multi();
template.boundValueOps("somevkey").increment(1);
template.boundZSetOps("somezkey").add("zvalue", timestamp);
template.exec();
}
After running above code through Junit Test, Exceptions are thrown.
org.springframework.data.redis.RedisSystemException: Unknown exception; nested exception is org.springframework.data.redis.RedisSystemException: Unknown jedis exception; nested exception is java.lang.NullPointerException
at org.springframework.data.redis.connection.jedis.JedisUtils.convertJedisAccessException(JedisUtils.java:93)
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.translateExceptionIfPossible(JedisConnectionFactory.java:155)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:58)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:163)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy66.appendUserStream(Unknown Source)
at com.uniu.test.repository.StreamCacheRepositoryTest.testAppendUserStream(StreamCacheRepositoryTest.java:23)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: org.springframework.data.redis.RedisSystemException: Unknown jedis exception; nested exception is java.lang.NullPointerException
at org.springframework.data.redis.connection.jedis.JedisConnection.convertJedisAccessException(JedisConnection.java:119)
at org.springframework.data.redis.connection.jedis.JedisConnection.exec(JedisConnection.java:523)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.data.redis.core.CloseSuppressingInvocationHandler.invoke(CloseSuppressingInvocationHandler.java:58)
at $Proxy70.exec(Unknown Source)
at org.springframework.data.redis.core.RedisTemplate$1.doInRedis(RedisTemplate.java:416)
at org.springframework.data.redis.core.RedisTemplate$1.doInRedis(RedisTemplate.java:412)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:162)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:133)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:121)
at org.springframework.data.redis.core.RedisTemplate.exec(RedisTemplate.java:412)
at com.uniu.repository.impl.StreamCacheRepositoryRedisImpl.appendUserStream(StreamCacheRepositoryRedisImpl.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:155)
... 33 more
Caused by: java.lang.NullPointerException
at redis.clients.jedis.BinaryTransaction.exec(BinaryTransaction.java:31)
at org.springframework.data.redis.connection.jedis.JedisConnection.exec(JedisConnection.java:521)
... 54 more
I check redis server, the above two commands have been executed and the result is correct. So the problem comes at last line of the code (template.exec()). When underlying JedisClient try to get multibulk response from EXEC, it seems to throw NullPointerException
I use spring-data-redis 1.0.0.RELEASE
Thanks for help.

The reason for the exception is probably that the Spring template implementation does not reuse the same connection for .multi() and .exec(). You can try to use execute() via a callback:
private RedisTemplate template = ...;
template.execute(
new RedisCallback() {
#Override
public Object doInRedis(RedisConnection connection)
throws DataAccessException {
connection.multi();
//do whatever you need, like deleting and repopulating some keys
connection.expire(CHANNEL_KEY.getBytes(), EXPIRE_SECS);
connection.exec();
return null;
}
}
);

template.setEnableTransactionSupport(true);
maybe you can use this to enable transaction

Piotr is right. You need to put it into SessionCallback.
Here is a quick summary of doing transactions with Spring Data Redis:
https://github.com/eric-wu/dashboard/wiki/3.-Redis-Transactions-via-Spring-Data-Redis
For your case, scroll down and look at "Optimistic locking using WATCH, MULTI, and EXEC".

Look at the implementatin of org.springframework.data.redis.support.collections.CollectionUtils.rename(K, K, RedisOperations<K, ?>)
and read about transaction details here
static <K> void rename(final K key, final K newKey, RedisOperations<K, ?> operations) {
operations.execute(new SessionCallback<Object>() {
#SuppressWarnings("unchecked")
public Object execute(RedisOperations operations) throws DataAccessException {
do {
operations.watch(key);
if (operations.hasKey(key)) {
operations.multi();
operations.rename(key, newKey);
} else {
operations.multi();
}
} while (operations.exec().isEmpty());
return null;
}
});
}
Note : While loop is important