We Keep Coding

Zero flow files in onTrigger() method of AbstractProcessor of Apache Nifi

I am developing a custom processor for Apache NiFi. I have created nar of my processor and put it in the lib folder of nifi and started the nifi. I have setup the remote debugger in eclipse and enabled breakpoint on first line of onTrigger(). While debugging I am running one processor at a time in my nifi pipeline. I can find single flow file in the input queue of my custom processor, however my custom processor is not receiving any flow file. When I start my custom processor, it hits breakpoint inside onTrigger() method. Inside thie method, when I do:
public class MyCustomProc extends AbstractProcessor {
#Override
public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
List<FlowFile> flowFiles = session.get(5000);
if (flowFiles == null || flowFiles.size() == 0) {
return;
}
//...
flowFiles turns out to be of size zero!!! I am not able to guess in which direction should I check to find why this is happening. Any hint how I can diagnose this?
Edit
Stacktrace
2019-05-02 18:08:09,456 ERROR [Timer-Driven Process Thread-10] c.c.product.module.submodule.MyCustomProcessor MyCustomProcessor[id=016a1008-8956-1dbf-bd66-993e0ce98668] MyCustomProcessor[id=016a1008-8956-1dbf-bd66-993e0ce98668] failed to process due to org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=408fbb3d-7cc2-48bc-be8f-6d0afdbddaf2,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1556800468726-1, container=default, section=1], offset=261, length=591447],offset=0,name=188149730353200,size=591447] transfer relationship not specified; rolling back session: {}
org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=408fbb3d-7cc2-48bc-be8f-6d0afdbddaf2,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1556800468726-1, container=default, section=1], offset=261, length=591447],offset=0,name=188149730353200,size=591447] transfer relationship not specified
at org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:251)
at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:321)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
PS1: This method returns immediately from inside if's body, which gives me following exception:
org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord transfer relationship not specified
This exceptions keeps recurring forever, since flow file in the input queue of my custom processor.
PS2: I am getting following error in the apps.log, though I am unsure if this is the source of the problem:
2019-05-02 18:17:32,394 ERROR [Timer-Driven Process Thread-4] o.a.nifi.groups.StandardProcessGroup Unable to synchronize StandardProcessGroup[identifier=d25747e6-719e-3ed9-c6c5-56794af6555c] with Flow Registry because Process Group was placed under Version Control using Flow Registry with identifier 80016ab0-bfab-152b-ffff-ffffc441867c but cannot find any Flow Registry with this identifier

It is normal behavior to sometimes get zero flow files, which is why processors have the check that you have at the beginning.
The FlowFileHandlingException means that a flow file was obtained from the session, either from get or create, and that flow file was not transferred anywhere and was not removed, so basically it is unaccounted for. This could not happen from just returning at the beginning in that if statement, so the rest of the processor code is executing and producing this error. You haven't provided the rest of the code so we can't see the problem.
The second issue is fairly self-explanatory. You have a process group under version control, but the registry client that was used to start version control somehow no longer exists. I don't know how you created this scenario because I believe the UI/API won't let you delete a registry client that has active flows under version control, but you should be able to stop version control on the process group.

java.lang.ArrayIndexOutOfBoundsException at oracle.jdbc.driver.T4CTTIrxd.readBitVector(T4CTTIrxd.java:135)

Trying to access collection using lazy loading and facing below issue, I'm not able to figure out what is the cause of this issue.
I am using Hibernate3 with Spring and Driver 12.1.0.2 oracle 11.2.0.4.0. Below is the error log.
java.lang.ArrayIndexOutOfBoundsException
at oracle.jdbc.driver.T4CTTIrxd.readBitVector(T4CTTIrxd.java:135)
at oracle.jdbc.driver.T4C8TTIrxh.unmarshalV10(T4C8TTIrxh.java:123)
at oracle.jdbc.driver.T4C8Oall.readRXH(T4C8Oall.java:678)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:587)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:225)
at oracle.jdbc.driver.T4CPreparedStatement.fetch(T4CPreparedStatement.java:1066)
at oracle.jdbc.driver.OracleStatement.fetchMoreRows(OracleStatement.java:3716)
at oracle.jdbc.driver.InsensitiveScrollableResultSet.fetchMoreRows(InsensitiveScrollableResultSet.java:1015)
at oracle.jdbc.driver.InsensitiveScrollableResultSet.absoluteInternal(InsensitiveScrollableResultSet.java:979)
at oracle.jdbc.driver.InsensitiveScrollableResultSet.next(InsensitiveScrollableResultSet.java:579)
at com.ibm.ws.rsadapter.jdbc.WSJdbcResultSet.next(WSJdbcResultSet.java:3120)
at org.hibernate.loader.Loader.doQuery(Loader.java:697)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:236)
at org.hibernate.loader.Loader.loadCollection(Loader.java:1994)
at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:36)
at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:565)
at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:63)
at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1716)
at org.hibernate.collection.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:344)
at org.hibernate.collection.AbstractPersistentCollection.read(AbstractPersistentCollection.java:86)
at org.hibernate.collection.PersistentSet.iterator(PersistentSet.java:163)
at org.apache.commons.collections.CollectionUtils.select(CollectionUtils.java:515)
at com.cmt.acq.service.AcqCampaignService.getGrid(AcqCampaignService.java:1245)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:126)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:50)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:50)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy157.getGrid(Unknown Source)
at com.cmt.acq.action.AcqManageCampaignAction.generateXMLResponse(AcqManageCampaignAction.java:1073)

This is almost certainly due to row compression. The database recognizes sequential duplicate column values and instead of sending the value again just sends "ditto" conceptually. So this problem is highly dependent on the data returned. Unless the query contains ORDER BY even repeat executions of a single query on a single database can return the rows in a different order so this problem can be difficult to reproduce.
There have been several patches to this code in recent releases. I vaguely recall seeing this problem but I can't find a bug for it. Still the code has changed a lot recently so I suggest trying 12.2.0.1.0 which is available on Oracle Maven. If it reproduces with 12.2.0.1.0 please file a bug.

Had the same problem using the 12.1.0.2 driver with an Oracle 10g database. It happens apparently when the driver is performing a fetch - querying the database for more rows while scrolling through a ResultSet.
Looks like a nasty bug in the driver.
Incrementing the fetchSize property worked well as a temporary workaround to avoid the issue, forcing the driver to retrieve all results at once with no data left to fetch, but it is not a long-term solution and may have a large negative impact on performance/memory depending on the size and amount of data the query is retrieving.
If you're using Hibernate, try to identify the query that is raising the exception (use for instance hibernate.show_sql), check how many rows does it return and adjust the hibernate.jdbc.fetch_size property accordingly.
A non-zero value determines the JDBC fetch size (calls Statement.setFetchSize()).
Or try using a different version of the driver.

File name contains # caused Spring Batch StaxEventItemReader failed to open

I have an input file with # in the name (UPX#L.xxxxx001.xml). Spring Batch StaxEventItemReader failed to open this file. Is there anyway to escape this special character? Please advise.
This is my bean
<bean id="myItemReader" parent="myItemReaderParent" scope="step">
<property name="resource" value="file:#{jobParameters['toolf.input.file.name']}"/>
</bean>
<bean id="myItemReaderParent" class="org.springframework.batch.item.xml.StaxEventItemReader" scope="prototype">
<property name="fragmentRootElementName" value="MyRec"/>
<property name="unmarshaller" ref="myUnmarshaller"/>
</bean>
[INFO ]org.springframework.jms.listener.DefaultMessageListenerContainer#0-1[2015-02-05 10:27:23,487] - Sending notification for start of JobExecution: id=52760, version=1, startTime=Thu Feb 05 10:27:23 EST
2015, endTime=null, lastUpdated=Thu Feb 05 10:27:23 EST 2015, status=STARTED, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=52759, version=0, JobParameters=[
{toolf.institutionSymbol=YDX , toolf.projectId=013832, toolf.orderId=678161, toolf.transmissionDate=20150205 10:27:23.392 EST, toolf.input.file.name=/batchdata/dataload/xxx/in/tool/UPX#L.xxxxx001.xml, toolf.out.root.directory=/batchdata/output, toolf.out.input.file.name.without.path=YDX.013832.678161.YbpUsSI#Picklists.d20150204.t140001.xml, toolf.out.working.directory/batchdata/dataload/xxx/in/tool/, timestamp=1423150043392}
], Job=[timeOfOrderLabelJob]]
[INFO ]org.springframework.jms.listener.DefaultMessageListenerContainer#0-1[2015-02-05 10:27:23,576] - Executing step: [toolPrologue]
[ERROR]org.springframework.jms.listener.DefaultMessageListenerContainer#0-1[2015-02-05 10:27:23,607] - Encountered an error executing the step
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:137)
at sun.reflect.GeneratedMethodAccessor260.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy9.open(Unknown Source)
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:93)
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:301)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:192)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:135)
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:61)
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124)
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:281)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:120)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:114)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:129)
at org.oclc.tool.mule.launch.MuleJobLauncher.launchJob(MuleJobLauncher.groovy:74)
at org.oclc.tool.mule.launch.JobLauncher$launchJob.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:137)
at org.oclc.tool.mule.si.JmsMessageUnwrapper.unwrapMessage(JmsMessageUnwrapper.groovy:42)
at sun.reflect.GeneratedMethodAccessor596.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:69)
at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:83)
at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:57)
at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:102)
at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:102)
at org.springframework.integration.util.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:126)
at org.springframework.integration.util.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:225)
at org.springframework.integration.util.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:125)
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:73)
at org.springframework.integration.handler.ServiceActivatingHandler.handleRequestMessage(ServiceActivatingHandler.java:64)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:98)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:110)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:61)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:157)
at org.springframework.integration.core.MessagingTemplate.doSend(MessagingTemplate.java:288)
at org.springframework.integration.core.MessagingTemplate.send(MessagingTemplate.java:149)
at org.springframework.integration.core.MessagingTemplate.convertAndSend(MessagingTemplate.java:189)
at org.springframework.integration.gateway.MessagingGatewaySupport.send(MessagingGatewaySupport.java:183)
at org.springframework.integration.jms.ChannelPublishingJmsMessageListener$GatewayDelegate.send(ChannelPublishingJmsMessageListener.java:424)
at org.springframework.integration.jms.ChannelPublishingJmsMessageListener.onMessage(ChannelPublishingJmsMessageListener.java:278)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:535)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:495)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:325)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalStateException: Input resource must exist (reader is in 'strict' mode)
at org.springframework.batch.item.xml.StaxEventItemReader.doOpen(StaxEventItemReader.java:182)
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:134)
... 73 more
[INFO ]org.springframework.jms.listener.DefaultMessageListenerContainer#0-1[2015-02-05 10:27:23,612] - enter DetailLogStepExecutionListener.afterStep
[INFO ]org.springframework.jms.listener.DefaultMessageListenerContainer#0-1[2015-02-05 10:27:23,612] - exit DetailLogStepExecutionListener.afterStep

In a URL/URI, the # character is reserved, used to delimit the fragment identifier (which would have no meaning to the file: resource scheme). If you want to include this character as part of the path, it must be escaped with %23
Rather than concatenate your value to file: in your bean definition file, I recommend that you specify the parameter as a full URI/URI, rather than a filesystem path:
<property name="resource" value="#{jobParameters['toolf.input.file.url']}"/>
And set toolf.input.file.url to:
file:/batchdata/dataload/xxx/in/tool/UPX%23L.xxxxx001.xml
This will avoid any other escapement pitfalls you might encounter, like with characters #, !, =, etc.
If you must use a filesystem path as a parameter, you should be able to use:
<property name="resource" value="#{new java.io.File(jobParameters['toolf.input.file.name']).toURI().toURL()}"/>

From looking at your error message and the responses, your problem is clear, both Tunaki and Glenn Lane are partially correct, and I'm up-voting Glenn Lane for answering the question you asked, and doing it well.
From your reply to Glenn:
if (fileName.contains("#")) { def (value1, value2) = fileName.split("#") fileName = value1 + "%23#" + value2 } I tried this. Still doesn't work
Escaping will not help the situation.
The character is already being handled somewhere along the execution chain, Tunaki was correct in questioning the type error you were having.
From your error log:
[ERROR]org.springframework.jms.listener.DefaultMessageListenerContainer#0-1[2015-02-05 10:27:23,607] - Encountered an error executing the step
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
Coupled with the resource state exception Tunaki pointed out, we can conclude that is the case. There is insufficient information here to say why or where this occurring, but here are a two general categories:
Depending on how it's arriving at, or from, the file store the character may already removed from the file at that point in the process or aliased on the Store in a way the File handler can't deal with. For the reasons Glenn pointed out.
Depending on the Message or Job Handling in your execution chain it is being stripped out as it transformed into or out of a URI/URL call. For the reasons Glenn pointed out.
Yes. It is the # problem. Once the # is removed it works fine.
Based on the above statement the simple, quick, and dirty solution would be to be aware of this and handle it by stripping and injecting the # directly from the file or uri as needed. Based on your code snippet, you know how to do this.
The correct way to solve this would be to track down where the resource error is occurring and ensure that your file handling is consistent across the entire Job flow.
Without all the right code here and/or knowing more about your system configuration I can't be specific. There is simply not enough information between the bean and the error log to say exactly where this occurring only that this is exactly your problem.
Here are some possible suggestions:
I am assuming you've checked the file Store directly and hand removed the # and changed the file name in the job and it runs, based on your above statement.
Try to access the file on the store from a different java object using the same identifier and it works; Then check how your item reader implementation is handling the file access.
If it failed or that is not the failure point. Check that "#" value is the job file and if it's there and/or the only # when you perform the split.
If that all checks find out how it is arriving to the file store and that resource can be properly located in the store and that # is not also reserved in your implementation or requires special handling/configuration.
This will help narrow done the root cause.

Elasticsearch 2.0.0 security manager with groovy script

I'm using brand new Elasticsearch 2.0.0 from a Java application.
When calling the prepareUpdate() method with an inline groovy script that has the following code:
import org.elasticsearch.common.logging.*
import groovy.json.*
ESLogger logger = ESLoggerFactory.getLogger('events-sequence.groovy')
def TOKEN_SEPARATOR = "###"
def flow = [ ]
try {
ctx._source.events.reverseEach { e ->
def context = e.tuplenized_context ? JsonOutput.toJson(e.tuplenized_context) : "[]"
flow << (e.name.toLowerCase() + TOKEN_SEPARATOR + context.toLowerCase())
}
ctx._source.flow = flow.join(TOKEN_SEPARATOR)
} catch (Throwable t) {
logger.error("Error applying derivation", t)
throw t
}
I'm receiving the following exception:
java.security.AccessControlException: access denied ("java.util.PropertyPermission" "groovy.json.faststringutils.write.to.final.fields" "read")
According to the stacktrace, this happens when calling the JsonOutput.toJson() method:
java.lang.ExceptionInInitializerError
at groovy.json.internal.CharBuf.addJsonFieldName(CharBuf.java:516)
at groovy.json.JsonOutput.writeMap(JsonOutput.java:423)
at groovy.json.JsonOutput.writeObject(JsonOutput.java:267)
at groovy.json.JsonOutput.writeIterator(JsonOutput.java:441)
at groovy.json.JsonOutput.writeObject(JsonOutput.java:269)
at groovy.json.JsonOutput.toJson(JsonOutput.java:187)
at groovy.json.JsonOutput$toJson.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at 3974cbb354b454f7c665982a3a8f854ede6125fb$_run_closure1.doCall(3974cbb354b454f7c665982a3a8f854ede6125fb:18)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019)
at groovy.lang.Closure.call(Closure.java:426)
at groovy.lang.Closure.call(Closure.java:442)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.reverseEach(DefaultGroovyMethods.java:2172)
at org.codehaus.groovy.runtime.dgm$532.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at 3974cbb354b454f7c665982a3a8f854ede6125fb.run(3974cbb354b454f7c665982a3a8f854ede6125fb:17)
at org.elasticsearch.script.groovy.GroovyScriptEngineService$GroovyScript.run(GroovyScriptEngineService.java:248)
at org.elasticsearch.action.update.UpdateHelper.executeScript(UpdateHelper.java:251)
at org.elasticsearch.action.update.UpdateHelper.prepare(UpdateHelper.java:196)
at org.elasticsearch.action.update.UpdateHelper.prepare(UpdateHelper.java:79)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardUpdateOperation(TransportShardBulkAction.java:408)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:203)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.performOnPrimary(TransportReplicationAction.java:579)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase$1.doRun(TransportReplicationAction.java:452)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "groovy.json.faststringutils.write.to.final.fields" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.security.AccessController.checkPermission(AccessController.java:884)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1294)
at java.lang.System.getProperty(System.java:753)
at groovy.json.internal.FastStringUtils.<clinit>(FastStringUtils.java:37)
... 42 more
In my elasticsearch.yml config file, I've added these 2 lines:
script.inline: on
script.indexed: on
Is there anything else I need to configure so that I can serialize an object to JSON within a groovy script?
EDIT: I've also tried to initialize Elasticsearch setting the following option:
export ES_JAVA_OPTS=-Dgroovy.json.faststringutils.write.to.final.fields\=true
But had no luck, since the problem seems to be that there are no permissions to access the groovy.json.faststringutils.write.to.final.fields system property, whatever its value is.
EDIT 2: All modifying the default java.policy file, specifying a new policy file with the -Djava.security.manager and -Djava.security.policy=file:///my.policy options and disabling the security manager via the -Dsecurity.manager.enabled=false option didn't work.
I've reported this as an issue to the Elasticsearch guys, and they've already fixed it. Actually, here's the commit, but it won't be available until version 2.1.
Is there any workaround or configuration option to make it work now?

None of the other suggestions here worked for me either, but I did find that I could disable the ES security manager by adding the following to my elasticsearch.yml file...
security.manager.enabled: false
Note that this is deprecated as of 2.2.0 and will likely be removed soon. This is not a best practice and should be avoided when dynamic scripts are allowed.

You will have to update your policy file with the system property read permission.
Add: permission java.util.PropertyPermission "groovy.json.faststringutils.write.to.final.fields", "read" to the grant block.
The default policy file (java.policy) resides under $JAVA_HOME/lib/security, unless specified otherwise using java.security.policy system property.
Alternatively, run the JVM without a security manager with -Dsecurity.manager.enabled=false

A better way of implementing it is to define your script under /config/scripts. As your script is static, you gain following advantages out of it:
No inline scripting required which makes your application safe.
Script is compiled once and used again. This gives you performance gain.
No need to change any java security policy.

Java Quartz Ibatis Cron Issues

I have a java webapp using an ibatis row handler to load a very large dataset (1 million rows in an innodb table). The process is run as a nightly cron job by quartz scheduler. However, after it processes for 6 minutes, it dies with the following stack trace:
WARN [DefaultQuartzScheduler_Worker-8] MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(168) | Could not invoke method 'doBatch' on target object [org.myCron#4adb34]
org.springframework.jdbc.UncategorizedSQLException: SqlMapClient operation: encountered SQLException [
--- The error occurred in org/myCron/mySqlMap.xml.
--- The error occurred while applying a result map.
--- Check the mySqlMap.outputMapping.
--- The error happened while setting a property on the result object.
--- Cause: com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.io.EOFException
STACKTRACE:
java.io.EOFException
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1903)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2402)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2860)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289)
at com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362)
at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352)
at com.mysql.jdbc.ResultSet.next(ResultSet.java:6106)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:168)
at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:592)
at com.ibatis.common.jdbc.logging.ResultSetLogProxy.invoke(ResultSetLogProxy.java:47)
at $Proxy10.next(Unknown Source)
at com.ibatis.sqlmap.engine.execution.SqlExecutor.handleResults(SqlExecutor.java:380)
at com.ibatis.sqlmap.engine.execution.SqlExecutor.handleMultipleResults(SqlExecutor.java:301)
at com.ibatis.sqlmap.engine.execution.SqlExecutor.executeQuery(SqlExecutor.java:190)
at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.sqlExecuteQuery(GeneralStatement.java:205)
at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithCallback(GeneralStatement.java:173)
at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithRowHandler(GeneralStatement.java:133)
at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryWithRowHandler(SqlMapExecutorDelegate.java:649)
at com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryWithRowHandler(SqlMapSessionImpl.java:156)
at com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.queryWithRowHandler(SqlMapClientImpl.java:133)
at org.springframework.orm.ibatis.SqlMapClientTemplate$5.doInSqlMapClient(SqlMapClientTemplate.java:267)
at org.springframework.orm.ibatis.SqlMapClientTemplate.execute(SqlMapClientTemplate.java:165)
at org.springframework.orm.ibatis.SqlMapClientTemplate.queryWithRowHandler(SqlMapClientTemplate.java:265)
at org.myCron.doBatch(MyCron.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:592)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:248)
at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:165)
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:66)
at org.quartz.core.JobRunShell.run(JobRunShell.java:191)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
** END NESTED EXCEPTION **
The stack trace is very vague. The only hints that I see are 'the error happened while setting a property on the result object'. There are only two properties on the result object: a String and an Integer. Both of them permit null values, but my select statements indicate that neither of them have any null values. They both have a proper gettter/setter (which makes sense since the process runs for a while successfully before dying). Every time that the cron runs, it dies at a random point (so it isn't stuck on a particular row).
Note - The method 'doBatch' does exist since that is the method that starts the cron process. If it couldn't find doBatch, it couldn't successfully process the first thousand rows.
I've also tried runnning the job outside of quartz and it also fails there as well. We tried increasing our MySQL net_read_timeout, net_write_timeout, and delayed_insert_timeout but none of these settings helped with the problem. I also tried setting my log4j setting to DEBUG and I did not get any helpful info.
Any other ideas about what I could try?

Sounds like MySQL closed the connection for some reason. Check the MySQL log see if anything shows up. Turn on various logging options for MySQL if necessary.
Also, start printing debug data (including timestamp) from your app - just print everything, then see what the last action was - perhaps you have some rarely triggered conditions in your code that has a bug.
I.e. every single time you talk to MySQL log it before AND after.