I have used ExecuAndWait in Struts2
I am getting error NPE while setting attribute in request in action(which is long running)
Here is stack track:
java.lang.NullPointerException
at org.apache.catalina.connector.Request.notifyAttributeAssigned(Request.java:1563)
at org.apache.catalina.connector.Request.setAttribute(Request.java:1554)
at org.apache.catalina.connector.RequestFacade.setAttribute(RequestFacade.java:542)
at javax.servlet.ServletRequestWrapper.setAttribute(ServletRequestWrapper.java:239)
at com.os.gfnactions.SiteAction.createSite(SiteAction.java:1298)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:450)
at com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvocation.java:289)
at com.os.interceptor.BackgroundProcess$1.run(BackgroundProcess.java:60)
at java.lang.Thread.run(Unknown Source)
Source snipate :
Action class:
public String createSite() throws Exception
{
----
HttpServletRequest request = ServletActionContext.getRequest();
request.setAttribute("test", "test"); {At this line I got error}
---
}
From ExecuteAndWaitInterceptor.java
231 if ((!executeAfterValidationPass || secondTime) && bp == null) {
232 bp = getNewBackgroundProcess(name, actionInvocation, threadPriority);
233 session.put(KEY + name, bp);
234 performInitialDelay(bp); // first time let some time pass before showing wait page
235 secondTime = false;
236 }
From BackgroundProcess.java
public More ...BackgroundProcess(String threadName, final ActionInvocation invocation, int threadPriority) {
50 this.invocation = invocation;
51 this.action = invocation.getAction();
52 try {
53 final Thread t = new Thread(new Runnable() {
54 public void More ...run() {
55 try {
56 beforeInvocation();
57 result = invocation.invokeActionOnly();
58 afterInvocation();
59 } catch (Exception e) {
60 exception = e;
61 }
62
63 done = true;
64 }
65 });
66 t.setName(threadName);
67 t.setPriority(threadPriority);
68 t.start();
69 } catch (Exception e) {
70 exception = e;
71 }
72 }
Concept of Struts2 ExecuteAndWait
Whenever there is a long-running request, it going to execute in separate thread and result return WAIT, So again client resubmit the same request in some interval to know the status of its process(which is running in the thread)
My Problem: In above case when main request(Which initiate thread to invoke action) return with WAIT and again other request come to know status of action at this time in my action class I have request.setAttribute("test", "test");, at this line it throws an error which I have mention above.
I had a similar problem with execAndWait interceptor throwing NPE. You can find my case study here: execAndWait interceptor not working with validation on how I solved this problem.. In this problem what I find out was, that execAndWait runs in separate thread and keeps throwing wait until action is completed and in mean time it cycles itself. I faced this problem, because I used model driven interceptor with it. Due to that the getModel() of model driven interceptor was repeatedly being called by execAndWait interceptor. In getModel method, it was setting the new POJO object again and again from scratch.
And then it was entering into validate method for validation. Inside validation process it found one of the POJO field to be null. Obviously, it happened due to re-creating of raw new POJO object in getModel. And thus was throwing Null Pointer Exception.
So what I did was I used SessionAware interface. And stored the POJO object for the first time when it entered validate. Because execAndWait will surely call all methods again and re-write an object from scratch. For this purpose I checked the availability of that object in getModel() method. If that found in session, then return that same object, rather of creating a new object.
I hope you'll find a way from this.
Related
I'm very new to Flink, and trying out the Async IO operation by following the doc from here. I've a text file containing bunch of integers. I'm creating a stream on the file, then for each line, I'm making an async http request and finally storing the results into an output file. I created a fastAPI rest endpoint for managing simple get request. In the Flink code, I'm using the java async-http-client library to wrap the http call into an async request. But the problem is, when I run the Flink code, it always times out.
My input file looks something like:
-9
42
2
12
15
18
13
9
45
-15
11
...
The fastAPI code goes something like this:
import time
from random import random
from fastapi import FastAPI
app = FastAPI()
#app.get("/temperatures/{temperature}")
async def read_temperature(temperature: int):
time.sleep(random())
if temperature <= 0:
return {"category": "insanely cold"}
elif temperature <= 15:
return {"category": "cold"}
elif temperature <= 25:
return {"category": "moderate"}
elif temperature <= 35:
return {"category": "moderately hot"}
elif temperature <= 45:
return {"category": "hot"}
else:
return {"category": "insanely hot"}
And finally, this is my Flink code:
import ...
public class AsyncHttpRequest extends RichAsyncFunction<String, Tuple2<String, String>> {
private transient AsyncHttpClient client;
#Override
public void open(Configuration parameters) {
client = asyncHttpClient();
}
#Override
public void close() throws Exception {
client.close();
}
#Override
public void asyncInvoke(String key, final ResultFuture<Tuple2<String, String>> resultFuture) throws Exception {
// issue the asynchronous request, receive a future for result
String getURL = String.format("http://localhost:8000/temperatures/%s", key);
final Future<Response> result = client.executeRequest(get(getURL).build());
// set the callback to be executed once the request by the client is complete
// the callback simply forwards the result to the result future
CompletableFuture.supplyAsync(() -> {
try {
JSONObject responseJson = new JSONObject(result.get().getResponseBody());
return responseJson.getString("category");
} catch (InterruptedException | ExecutionException e) {
return null;
}
}).thenAccept((String httpResult) -> {
resultFuture.complete(Collections.singleton(new Tuple2<>(key, httpResult)));
});
}
public static void main(String[] args) throws Exception {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<String> stream =
env.readTextFile("file:///Users/me/http_input.txt");
env.enableCheckpointing(10);
DataStream<Tuple2<String, String>> resultStream =
AsyncDataStream.unorderedWait(
stream, new AsyncHttpRequest(), 60, TimeUnit.SECONDS, 10);
final StreamingFileSink<Tuple2<String, String>> sink =
StreamingFileSink.forRowFormat(
new Path("file:///Users/me/http_output"),
new SimpleStringEncoder<Tuple2<String, String>>("UTF-8"))
.withRollingPolicy(
DefaultRollingPolicy.builder()
.withRolloverInterval(TimeUnit.MINUTES.toMillis(1)) .withInactivityInterval(TimeUnit.MINUTES.toMillis(5))
.withMaxPartSize(1024 * 1024)
.build())
.build();
resultStream.addSink(sink);
env.execute("Async Http job");
}
}
I get the following stacktrace when running the flink job:
Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
at org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$2(MiniClusterJobClient.java:117)
at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:237)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046)
at akka.dispatch.OnComplete.internal(Future.scala:264)
at akka.dispatch.OnComplete.internal(Future.scala:261)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
at org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68)
at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:284)
at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:284)
at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:284)
at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:573)
at akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
at akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
at scala.concurrent.Future.$anonfun$andThen$1(Future.scala:532)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80)
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233)
at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224)
at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215)
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:669)
at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89)
at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:447)
at jdk.internal.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212)
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at akka.actor.Actor.aroundReceive(Actor.scala:517)
at akka.actor.Actor.aroundReceive$(Actor.scala:515)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
at akka.actor.ActorCell.invoke(ActorCell.scala:561)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
... 4 more
Caused by: java.lang.Exception: Could not complete the stream element: Record # (undef) : 9.
at org.apache.flink.streaming.api.operators.async.AsyncWaitOperator$ResultHandler.completeExceptionally(AsyncWaitOperator.java:383)
at org.apache.flink.streaming.api.functions.async.AsyncFunction.timeout(AsyncFunction.java:97)
at org.apache.flink.streaming.api.operators.async.AsyncWaitOperator.lambda$processElement$0(AsyncWaitOperator.java:197)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1318)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$17(StreamTask.java:1309)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxExecutorImpl.yield(MailboxExecutorImpl.java:86)
at org.apache.flink.streaming.api.operators.async.AsyncWaitOperator.waitInFlightInputsFinished(AsyncWaitOperator.java:284)
at org.apache.flink.streaming.api.operators.async.AsyncWaitOperator.endInput(AsyncWaitOperator.java:254)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:91)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.lambda$close$0(StreamOperatorWrapper.java:128)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:128)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:135)
at org.apache.flink.streaming.runtime.tasks.OperatorChain.closeOperators(OperatorChain.java:439)
at org.apache.flink.streaming.runtime.tasks.StreamTask.afterInvoke(StreamTask.java:627)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:589)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:755)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:570)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.util.concurrent.TimeoutException: Async function call has timed out.
... 20 more
Prining the Interruption/Execution Exception shows the following error:
java.util.concurrent.ExecutionException: java.net.ConnectException: executor not accepting a task
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at org.asynchttpclient.netty.NettyResponseFuture.get(NettyResponseFuture.java:201)
at com.merlot.data.pipeline.jobs.async.AsyncHttpRequest.lambda$asyncInvoke$0(AsyncHttpRequest.java:53)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.net.ConnectException: executor not accepting a task
at org.asynchttpclient.netty.channel.NettyConnectListener.onFailure(NettyConnectListener.java:179)
at org.asynchttpclient.netty.channel.NettyChannelConnector$1.onFailure(NettyChannelConnector.java:108)
at org.asynchttpclient.netty.SimpleChannelFutureListener.operationComplete(SimpleChannelFutureListener.java:28)
at org.asynchttpclient.netty.SimpleChannelFutureListener.operationComplete(SimpleChannelFutureListener.java:20)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:608)
at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:109)
at io.netty.channel.DefaultChannelPromise.setFailure(DefaultChannelPromise.java:89)
at io.netty.bootstrap.Bootstrap.doResolveAndConnect0(Bootstrap.java:197)
at io.netty.bootstrap.Bootstrap.access$000(Bootstrap.java:46)
at io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:180)
at io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:166)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:604)
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84)
at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetSuccess(AbstractChannel.java:989)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:504)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:417)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:474)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: executor not accepting a task
at io.netty.resolver.AddressResolverGroup.getResolver(AddressResolverGroup.java:61)
at io.netty.bootstrap.Bootstrap.doResolveAndConnect0(Bootstrap.java:194)
... 21 more
I'm not really sure why the async function is timing out, because I can see the fastAPI endpoint is being queried in the console. Also, the endpoint is working fine, as all my postman requests go through perfectly fine. Any help in resolving the core issue is greatly appreciated.
I'm on MacOS Big Sur and using the following 3rd party libs:
implementation 'org.json:json:20201115'
implementation 'org.apache.httpcomponents:httpclient:4.5.13'
implementation 'org.asynchttpclient:async-http-client:2.12.2'
implementation 'org.apache.flink:flink-core:1.12.2'
implementation 'org.apache.flink:flink-streaming-java_2.12:1.12.2'
implementation 'org.apache.flink:flink-clients_2.12:1.12.2'
Update 1: If I reduce the capacity to 1 (from 1000), then I don't get any error, but still the output is empty.
Update 2: After the suggestion made by #DavidAnderson I've enabled checkpointing. Now, I'm not seeing the timeout error, and my job is not getting terminated abruptly, which is a good news. But now, the output folder is still empty. I've updated my Flink code to reflect the checkpointing changes.
A common issue with AsyncIO is that every concurrent request limit used in the execution stack should be sized appropriately. Some of these limits are implicit, e.g. if you don't supply your own thread pool to CompletableFuture.supplyAsync(), then it uses the shared commonPool, which is limited to a very small size, see https://dzone.com/articles/be-aware-of-forkjoinpoolcommonpool.
IIRC I use AsyncDataStream.unorderedWait(capacity) <= HTTP client capacity <= executor capacity. And the HTTP client capacity is often a limit of both its connection pool size and the number of connections per host.
Note to community: Please do not close this as duplicate because the particular issue I am researching has manifested as a null pointer exception. As you can see from the stack trace, the NPE is buried 4 layers deep in the Tika library. That means of all the great advice that was given in the existing StackExchange post on NPE, none of the Tika developers saw fit to apply that advice (checking for null pointers) in four modules. Rather than learn Tika and retrofit their code with a patch to do that work, it think it would be more efficient to ask if anyone had achieved the common use case of using the SourcCodeParser.
I am looking for help with a published example for the Tika library here. I did not author the example code. I have seen many similar questions relating to the Tika library, which has 20 contributors and thousands of lines of code. Please do not close this question as I believe this can be quickly easily answered by anyone who used this Parser before. I have already read the post on NullPointerException, and am following this advice from that question:
I still can't find the problem
If you tried to debug the problem and still don't have a solution, you
can post a question for more help, but make sure to include what
you've tried so far. At a minimum, include the stacktrace in the
question, and mark the important line numbers in the code.
As I spent much time authoring this post, retrieving and including relevant stack trace and source code, I would really appreciate it if you would allow this to spend a little bit of time in an unclosed state so that someone who is familiar with Tika might take a look at what appears to be fairly common issue. As you would know as a Java expert, many null pointer exception issues can be non-trivial, particularly when working with a large unfamiliar framework. I really appreciate your help.
I wrote a simple program to test the Tika SourceCodeParser, by substituting it for the AutoDetectParser in the XHTML parsing example from the Tika Examples page. When executing the parse command on line 137, there is a NullPointerException. It appears that there may be a delegate missing from the in on line 180 of the Parser code.
The AutoDetectParser works but does not identify the source code as java.
When I use the Tika desktop app, it works fine and recognizes the code as Java.
How do I initialize the SourceCodeParser to avoid the NullPointerException when operating it?
Example using Tika "Example" Package
LocalFile.toTikaXhtmlString()
123 /** Parses as Tika using source code parser.
124 *
125 * #param filePathParam path to file to parse
126 */
127 public static String toTikaXhtmlString(final String filePathParam)
128 throws IOException, SAXException, TikaException
129 {
130 SourceCodeParser parser = new SourceCodeParser();
131 ContentHandler handler = new ToXMLContentHandler();
132 Metadata metadata = new Metadata();
133 File file = new File(filePathParam);
134 try (InputStream stream
135 = ContentHandlerExample.class
136 .getResourceAsStream(filePathParam)) {
137 parser.parse(stream, handler, metadata);
138 return handler.toString();
139 } catch (Exception e) {
140 System.out.println("Caught exception.");
141 System.out.println(e.toString());
142 e.printStackTrace();
143 throw e;
144 }
145
146 }
I also tried avoiding the Tika 'ContentHandlerExample' class using direct call with InputStreamReader, to the same result:
public static String toTikaXhtmlString(final String filePathParam)
throws IOException, SAXException, TikaException
{
SourceCodeParser parser = new SourceCodeParser();
ContentHandler handler = new ToXMLContentHandler();
Metadata metadata = new Metadata();
File file = new File(filePathParam);
try (InputStream stream = new FileInputStream(file)) {
parser.parse(stream, handler, metadata);
return handler.toString();
} catch (Exception e) {
throw new RuntimeException(e.getMessage());
}
}
JUNIT Test
108 #Test
109 public void parseFile() {
110 String fileName, verifyInput, resultContent;
111
112 //arrange
113 fileName = "/Users/johnmeyer/Projects/code-proc/FileParseTest-run.txt";
114
115 String fileContent = "/** Test */ public MyTestClass {"
116 + "public static void main(String[] args) {"
117 + "System.out.println(\"This is a test.\"); }";
118
119
120 LocalFile.putText(fileName, fileContent);
121
122 verifyInput = LocalFile.getContent(fileName);
123
124 assertEquals(fileContent, verifyInput);
125 //act (and clean up)
126
127 try {
128
129 resultContent = LocalFile.toTikaXhtmlString(fileName);
130 } catch (Exception e) {
131 throw new RuntimeException(e.getMessage());
132 }
133
134 LocalFile.delete(fileName);
135
136 //assert
137 assertEquals(fileContent, resultContent);
138 }
Stack Trace
[INFO] Running us.johnmeyer.test.tools.FileParseTest Caught exception.
java.lang.NullPointerException java.lang.NullPointerException at
org.apache.commons.io.input.ProxyInputStream.markSupported(ProxyInputStream.java:181)
at
org.apache.tika.detect.AutoDetectReader.getBuffered(AutoDetectReader.java:137)
at
org.apache.tika.detect.AutoDetectReader.(AutoDetectReader.java:114)
at
org.apache.tika.parser.code.SourceCodeParser.parse(SourceCodeParser.java:93)
at
org.apache.tika.parser.AbstractParser.parse(AbstractParser.java:53)
at
us.johnmeyer.utilities.LocalFile.toTikaXhtmlString(LocalFile.java:137)
at
us.johnmeyer.test.tools.FileParseTest.parseFile(FileParseTest.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at
org.junit.runners.ParentRunner.run(ParentRunner.java:309) at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:239)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:160)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)
Tika Source Code
17 package org.apache.tika.io;
18
19 import java.io.FilterInputStream;
20 import java.io.IOException;
21 import java.io.InputStream;
22
23 /**
24 * A Proxy stream which acts as expected, that is it passes the method
25 * calls on to the proxied stream and doesn't change which methods are
26 * being called.
27 * <p>
28 * It is an alternative base class to FilterInputStream
29 * to increase reusability, because FilterInputStream changes the
30 * methods being called, such as read(byte[]) to read(byte[], int, int).
31 * <p>
32 * See the protected methods for ways in which a subclass can easily decorate
33 * a stream with custom pre-, post- or error processing functionality.
34 *
35 * #author Stephen Colebourne
36 * #version $Id$
37 */
38 public abstract class ProxyInputStream extends FilterInputStream {
40 /**
41 * Constructs a new ProxyInputStream.
42 *
43 * #param proxy the InputStream to delegate to
44 */
45 public ProxyInputStream(InputStream proxy) {
46 super(proxy);
47 // the proxy is stored in a protected superclass variable named 'in'
48 }
...
174 /**
175 * Invokes the delegate's <code>markSupported()</code> method.
176 * #return true if mark is supported, otherwise false
177 */
178 #Override
179 public boolean markSupported() {
180 return in.markSupported();
181 }
I have the following class
public class MaintanceTools {
public static final ScheduledThreadPoolExecutor THREADSUPERVISER;
private static final int ALLOWEDIDLESECONDS = 1*20;
static {
THREADSUPERVISER = new ScheduledThreadPoolExecutor(10);
}
public static void launchThreadsSupervising() {
THREADSUPERVISER.scheduleWithFixedDelay(() -> {
System.out.println("maintance launched " + Instant.now());
ACTIVECONNECTIONS.forEach((connection) -> {
try {
if ( !connection.isDataConnected() &&
(connection.getLastActionInstant()
.until(Instant.now(), SECONDS) > ALLOWEDIDLESECONDS)) {
connection.closeFTPConnection();
ACTIVECONNECTIONS.remove(connection);
}
} catch (Throwable e) { }
});
System.out.println("maintance finished " + Instant.now());
}, 0, 20, TimeUnit.SECONDS);
}
}
Which iterates over all FTP connections (cause I write FTP server), checks if connection not transmitting any data and idle for some time and closes the connection if so.
The problem is task never runs after some exceptions thrown in the interrupting thread. I know that it's written in docs
If any execution of the task encounters an exception, subsequent executions are suppressed. Otherwise, the task will only terminate via cancellation or termination of the executor.
And I have the exception, but it is caught and do not go outside throwing function.
This function throws AsynchronousCloseException because it hangs on channel.read(readBuffer); and when connection is closed, exception thrown and caught.
The question is how to make THREADSUPERVISER work regardless any thrown and handled exceptions.
Debug output:
maintance launched 2017-08-30T14:03:05.504Z // launched and finished as expected
maintance finished 2017-08-30T14:03:05.566Z
output: FTPConnection id: 176 220 Service ready.
……
output: FTPConnection id: 190 226 File stored 135 bytes.
closing data socket: FTP connection 190, /0:0:0:0:0:0:0:1:1409
maintance launched 2017-08-30T14:03:25.581Z // launched and finished as expected
maintance finished 2017-08-30T14:03:25.581Z
async exception error reading. // got exception
maintance launched 2017-08-30T14:03:45.596Z // launched, but not finished and never run again
output: FTPConnection id: 176 221 Timeout exceeded, closing control and data connection.
closing data socket: FTP connection 176, /0:0:0:0:0:0:0:1:1407
As turns out, the problem was in
ACTIVECONNECTIONS.remove(connection);
I had ConcurrentModifyingException. Solution in http://code.nomad-labs.com/2011/12/09/mother-fk-the-scheduledexecutorservice/ worked perfectly
I'm creating a Custom Java Component for oracle-ucm to do some processing based on passed in Excel Files. The files are keyed off of Content ID.
one of the first things the application does after reading the excel file is call DOC_INFO_BY_NAME.
In short: Why can't DOC_INFO_BY_NAME by name find dProcessingState and how do I fix it?
Full Details Below
However, when I do I get the following Error Message:
Error getting DOC_INFO for 'LOCALHOST16200000001'. Unable to retrieve information for 'LOCALHOST16200000001'. Unable to retrieve file format information. Cannot find the parameter 'dProcessingState'. [ Details ]
An error has occurred. The stack trace below shows more information.
!$Error getting DOC_INFO for 'LOCALHOST16200000001'.!csUnableToGetRevInfo2,LOCALHOST16200000001!csUnableToGetFileFormatInfo!syParameterNotFound,dProcessingState
intradoc.common.ServiceException: !csUnableToGetRevInfo2,LOCALHOST16200000001!csUnableToGetFileFormatInfo
at intradoc.server.ServiceRequestImplementor.buildServiceException(ServiceRequestImplementor.java:2176)
at intradoc.server.Service.buildServiceException(Service.java:2404)
at intradoc.server.Service.createServiceExceptionEx(Service.java:2398)
at intradoc.server.Service.createServiceException(Service.java:2393)
at intradoc.server.DocCommonHandler.getDocFormats(DocCommonHandler.java:271)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at intradoc.common.IdcMethodHolder.invokeMethod(IdcMethodHolder.java:86)
at intradoc.common.ClassHelperUtils.executeMethodReportStatus(ClassHelperUtils.java:324)
at intradoc.server.ServiceHandler.executeAction(ServiceHandler.java:79)
at intradoc.server.Service.doCodeEx(Service.java:622)
at intradoc.server.Service.doCode(Service.java:594)
at intradoc.server.ServiceRequestImplementor.doAction(ServiceRequestImplementor.java:1693)
at intradoc.server.Service.doAction(Service.java:566)
at intradoc.server.ServiceRequestImplementor.doActions(ServiceRequestImplementor.java:1483)
at intradoc.server.Service.doActions(Service.java:561)
at intradoc.server.ServiceRequestImplementor.executeActions(ServiceRequestImplementor.java:1415)
at intradoc.server.Service.executeActions(Service.java:547)
at intradoc.server.ServiceRequestImplementor.doRequestInternalEx(ServiceRequestImplementor.java:958)
at intradoc.server.ServiceRequestImplementor.executeServiceTopLevelSimple(ServiceRequestImplementor.java:1070)
at com.lowes.content.edam.massMetaDataUpdate.service.types.ServicesWrapper.executeService(ServicesWrapper.java:139)
at com.lowes.content.edam.massMetaDataUpdate.service.file.GetFileService.getDocInfo(GetFileService.java:478)
at com.lowes.content.edam.massMetaDataUpdate.service.ServiceFieldMapper.getMappings(ServiceFieldMapper.java:76)
at com.lowes.content.edam.massMetaDataUpdate.file.mapper.impl.FieldMapWorker.doInBackground(FieldMapWorker.java:107)
at com.lowes.content.edam.massMetaDataUpdate.file.mapper.impl.FieldMapWorker.doInBackground(FieldMapWorker.java:37)
at javax.swing.SwingWorker$1.call(SwingWorker.java:277)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at javax.swing.SwingWorker.run(SwingWorker.java:316)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: intradoc.data.DataException: !syParameterNotFound,dProcessingState
at intradoc.data.DataBinder.getEx(DataBinder.java:1258)
at intradoc.data.DataBinder.get(DataBinder.java:1057)
at intradoc.server.DocCommonHandler.getProcessingState(DocCommonHandler.java:277)
at intradoc.server.DocCommonHandler.getDocFormats(DocCommonHandler.java:184)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at intradoc.common.IdcMethodHolder.invokeMethod(IdcMethodHolder.java:87)
at intradoc.common.ClassHelperUtils.executeMethodReportStatus(ClassHelperUtils.java:324)
at intradoc.server.ServiceHandler.executeAction(ServiceHandler.java:79)
at intradoc.server.Service.doCodeEx(Service.java:622)
at intradoc.server.Service.doCode(Service.java:595)
... 20 more
I've tried multiple ways of calling the additional service.
currently I'm using:
public DataBinder executeService(DataBinder binder, Workspace workspace)
throws DataException, ServiceException
{
final String serviceName = binder.getLocal(Services.IdcService);
//check for REMOTE_USER
String username = binder.getEnvironmentValue("REMOTE_USER");
if ( null == username || "".equals(username.trim()))
{
log.warn("username not found, searching the binder.");
try { username = binder.get("dUser"); }
catch (DataException de) { log.error("Unable to get Username.", de); }
if ( null != username && !"".equals(username.trim()))
{ binder.setEnvironmentValue("REMOTE_USER", username); }
else { throw new IllegalStateException("Username required!"); }
}
//clear idcToken
binder.getLocalData().remove("idcToken");
log.info("About to run '" + serviceName + "' as '" + username +"'");
try
{
log.info("Calling service " + serviceName + ": "
+ binder.getLocalData().toString());
// Execute service
service.createRequestImplementor();
service.getRequestImplementor()
.executeServiceTopLevelSimple(binder, serviceName,
service.getUserData());
log.info("Finished calling service");
return service.getBinder();
}
catch (DataException e)
{
String msg = "Unexpected Failure executing service '"+serviceName+"'";
log.info(msg, e);
throw e; //log and duck
}
}
So why can't WCC find dProcessingState, and how do I fix it?
Your way of calling the service is fairly standard and I don't think that is the reason. My first guess is that your other code is interfering with your call to DOC_INFO_BY_NAME. If you disable all your other custom code (perhaps easier said than done) and call the service in a clean component, then I think it'll work. Second candidate is to reconsider if your custom service really should be a FileService. Not sure if that affects the result, but perhaps worth a try.
It almost seems like a bug to me.
getProcessingState() is called from getDocFormats() which is called from the DOC_INFO* service.
The code for getProcessingState() first retrieves dProcessingState.
String curProcState = this.m_binder.get("dProcessingState");
Then, it checks if this is null:
if (curProcState != null)
However, it never gets to the null check. This is because DataBinder.get() is requiring a value for any key passed in (in this case dProcessingState.)
Trying setting dProcessingState to a value in the DataBinder before calling the service.
binder.setLocal("dProcessingState", "ignore");
The problem was totally my fault. I had tried to be fancy, and extend Databinder to make it easier for me to use. Turns out my extension was faulty.
Rolling back to the default and double checking my data in the binder fixed it.
I have long hierarchy of exceptions, one of it add long message with binary data. I can't change it in throw place, since it's in library. How can i truncate it, without losing another exceptions messages?
code example:
throw new RuntimeException("msg",
new RuntimeException(generateLongErrorMessage(),
new RuntimeException("short message",
new RuntimeException("important message"))
)
);
desired output:
Exception in thread "main" java.lang.RuntimeException: msg
at xxx.excpetionMessageTruncator(Erofeev.java:18)
at xxx.main(Erofeev.java:14)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: java.lang.RuntimeException: long message long message ...(truncated, original size: 100)
... 7 more
Caused by: java.lang.RuntimeException: short message
... 7 more
Caused by: java.lang.RuntimeException: important message
... 7 more
Update:
I see two different ways to do this, depending on your needs. I would prefer solution 2, as it is less invasive and allows all parts of the code to work with the complete exception.
1, one exception type only, no logger changes required:
Catch the exception and copy the whole stack exception by exception to set a shorter message:
private static RuntimeException truncate(RuntimeException rt) {
Stack<Throwable> causeStack = new Stack<>();
Throwable currentEx = rt;
while(currentEx != null) {
causeStack.push(currentEx);
currentEx = currentEx.getCause();
}
RuntimeException newEx = null;
while (!causeStack.isEmpty()) {
Throwable t = causeStack.pop();
newEx = new RuntimeException(truncateMessage(t.getMessage()), newEx);
newEx.setStackTrace(t.getStackTrace());
}
return newEx;
}
private static String truncateMessage(String message) {
if (message.length() < 25) {
return message;
}
return message.substring(0, 25) + "...";
}
While you can use it on any type of exception, the result will always be a stack of runtime exceptions. If that is not desireable you can go with reflections + generics to create a new exception of the same type.
If you only have combinations of message / cause inside the constructor this will be manageable, but keep in mind that there are custom expceptions as well.
2, Custom Exception Handler (if you are not able to catch the exception for whatever reason)
See http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Thread.UncaughtExceptionHandler.html to set a threads exception handler. From there you can pass the exception to your logger. Then you can perform the truncation in the logger.