My application is a non-web Spring boot application using the WebClient to make in internal get call.
Every time when I start it it returns me java.lang.IllegalStateException: executor not accepting a task.
The problem is connected with the webclient call: when I do block() everything is going normally. When I try to keep it reactive, it throws an exception.
Here is my main method:
public static void main(String[] args) {
SpringApplication app = new SpringApplication(Myapp.class);
app.setBannerMode(Banner.Mode.OFF);
app.run(args);
}
#Override
public void run(String... args) throws Exception {
///something that calls a webclient
exit(0);
}
Here is my webclient call:
#PostConstruct
private void init() {
log.debug(String.format("diagram path %s", diagramPath));
this.client = WebClient.builder()
.baseUrl("blablabla")
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.build();
}
public Mono<String> getDiagram() {
return client
.get()
.uri("/")
.retrieve()
.bodyToMono(MyResponse.class);
Here is the exception trace:
at io.netty.resolver.AddressResolverGroup.getResolver(AddressResolverGroup.java:60) ~[netty-resolver-4.1.43.Final.jar:4.1.43.Final]
at io.netty.bootstrap.Bootstrap.doResolveAndConnect0(Bootstrap.java:194) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.bootstrap.Bootstrap.access$000(Bootstrap.java:48) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:182) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:168) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:604) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetSuccess(AbstractChannel.java:985) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:505) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:416) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:475) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
It looks like your application is in a shutting down state therefore Netty executor doesn't accept tasks (requests to be executed). How do you keep a process running if it is not a web application?
Source code of AddressResolverGroup
59. if (executor.isShuttingDown()) {
60. throw new IllegalStateException("executor not accepting a task");
61. }
(note: It doesn't look right instantiating a WebClient instance in a #PostConstruct method)
Related
I'm trying to send an multipart/form in a WebClient request that contains files and String fields, but blockhound notify that FileInputStream is blocking the call.
public Mono<ResponseEntity> anotherCall(){
MultipartBodyBuilder builder = new MultipartBodyBuilder();
builder.part("description", "description");
builder.part("email", "email");
builder.part("priority", 1);
builder.part("status", 2);
builder.part("subject", "subject");
builder.part("company_id", 150000240766l);
return Mono.defer(() -> {
return this.webClient.getWebClient().post().uri(URI)
.headers(
httpHeaders -> {
httpHeaders.setBasicAuth("API_KEY", "X");
})
.bodyValue(builder.build())
.contentType(MediaType.MULTIPART_FORM_DATA)
.exchangeToMono(
response -> {
final HttpStatus status = response.statusCode();
return response
.bodyToMono(Map.class)
.defaultIfEmpty(Map.of())
.map(
body -> {
if (status.is2xxSuccessful()) {
return ResponseEntity.ok().body(body);
} else {
return Error(body);
}
});
});
});
I've tried to use builder.asyncPart(), tried to call Flux.then(),
tried .subscribeOn(Schedulers.boundedElastic())
Here is the stacktrace from blockhound, I read that webClient block threads to generate boundary for parts of form, I've tried to generate boundary and add to header, but no success.
reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
at java.base/java.io.FileInputStream.readBytes(FileInputStream.java)
at java.base/java.io.FileInputStream.read(FileInputStream.java:276)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:132)
at java.base/sun.security.provider.NativePRNG$RandomIO.readFully(NativePRNG.java:425)
at java.base/sun.security.provider.NativePRNG$RandomIO.ensureBufferValid(NativePRNG.java:528)
at java.base/sun.security.provider.NativePRNG$RandomIO.implNextBytes(NativePRNG.java:547)
at java.base/sun.security.provider.NativePRNG.engineNextBytes(NativePRNG.java:221)
at java.base/java.security.SecureRandom.nextBytes(SecureRandom.java:758)
at java.base/java.security.SecureRandom.next(SecureRandom.java:815)
at java.base/java.util.Random.nextInt(Random.java:323)
at org.springframework.util.MimeTypeUtils.generateMultipartBoundary(MimeTypeUtils.java:404)
at org.springframework.http.codec.multipart.MultipartWriterSupport.generateMultipartBoundary(MultipartWriterSupport.java:105)
at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.writeMultipart(MultipartHttpMessageWriter.java:185)
at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.lambda$write$0(MultipartHttpMessageWriter.java:158)
at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:152)
at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:441)
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:677)
at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.run(DefaultPooledConnectionProvider.java:274)
at io.netty.util.concurrent.AbstractEventExecutor.runTask$$$capture(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
2023-01-17 09:02:07.040 WARN [,19587802144e28cfd5bee3752b442824,a602a51f0e98a86c] [nio-8080-exec-6] c.i.p.e.api.GlobalExceptionHandler : Blocking call! java.io.FileInputStream#readBytes; nested exception is reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
org.springframework.web.reactive.function.client.WebClientRequestException: Blocking call! java.io.FileInputStream#readBytes; nested exception is reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
*__checkpoint ⇢ Request to POST URI[DefaultWebClient]
Original Stack Trace:
at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141)
at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:55)
at reactor.core.publisher.Mono.subscribe(Mono.java:4490)
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93)
at org.springframework.cloud.sleuth.instrument.reactor.ScopePassingSpanSubscriber.onError(ScopePassingSpanSubscriber.java:96)
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204)
at reactor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:124)
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.whenError(FluxRetryWhen.java:225)
at reactor.core.publisher.FluxRetryWhen$RetryWhenOtherSubscriber.onError(FluxRetryWhen.java:274)
at org.springframework.cloud.sleuth.instrument.reactor.ScopePassingSpanSubscriber.onError(ScopePassingSpanSubscriber.java:96)
at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121)
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:415)
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:251)
at org.springframework.cloud.sleuth.instrument.reactor.ScopePassingSpanSubscriber.onNext(ScopePassingSpanSubscriber.java:89)
at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)
at reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:537)
at reactor.core.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:343)
at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100)
at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:27)
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:190)
at org.springframework.cloud.sleuth.instrument.reactor.ScopePassingSpanSubscriber.onError(ScopePassingSpanSubscriber.java:96)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:201)
at reactor.netty.http.client.HttpClientConnect$HttpObserver.onUncaughtException(HttpClientConnect.java:399)
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onUncaughtException(ReactorNetty.java:670)
at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onUncaughtException(DefaultPooledConnectionProvider.java:205)
at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onUncaughtException(DefaultPooledConnectionProvider.java:454)
at reactor.netty.http.client.HttpClientOperations.onOutboundError(HttpClientOperations.java:583)
at reactor.netty.channel.ChannelOperations.onError(ChannelOperations.java:254)
at reactor.core.publisher.Operators.error(Operators.java:198)
at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:161)
at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:441)
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:677)
at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.run(DefaultPooledConnectionProvider.java:274)
at io.netty.util.concurrent.AbstractEventExecutor.runTask$$$capture(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
at java.base/java.io.FileInputStream.readBytes(FileInputStream.java)
at java.base/java.io.FileInputStream.read(FileInputStream.java:276)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:132)
at java.base/sun.security.provider.NativePRNG$RandomIO.readFully(NativePRNG.java:425)
at java.base/sun.security.provider.NativePRNG$RandomIO.ensureBufferValid(NativePRNG.java:528)
at java.base/sun.security.provider.NativePRNG$RandomIO.implNextBytes(NativePRNG.java:547)
at java.base/sun.security.provider.NativePRNG.engineNextBytes(NativePRNG.java:221)
at java.base/java.security.SecureRandom.nextBytes(SecureRandom.java:758)
at java.base/java.security.SecureRandom.next(SecureRandom.java:815)
at java.base/java.util.Random.nextInt(Random.java:323)
at org.springframework.util.MimeTypeUtils.generateMultipartBoundary(MimeTypeUtils.java:404)
at org.springframework.http.codec.multipart.MultipartWriterSupport.generateMultipartBoundary(MultipartWriterSupport.java:105)
at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.writeMultipart(MultipartHttpMessageWriter.java:185)
at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.lambda$write$0(MultipartHttpMessageWriter.java:158)
at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:152)
at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:441)
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:677)
at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.run(DefaultPooledConnectionProvider.java:274)
at io.netty.util.concurrent.AbstractEventExecutor.runTask$$$capture(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
.body(
BodyInserters.fromProducer(
Mono.just(builder.build()).subscribeOn(Schedulers.boundedElastic()),
MultiValueMap.class));
I am trying to proxy a request to another server. I am using Spring web flux and I have an object of type org.springframework.web.reactive.function.server.ServerRequest that I extract the multipart/form-data body received from using serverRequest.multipartData() method.
public Mono<ServerResponse> uploadItems(ServerRequest serverRequest) {
return catalogService
.uploadItems(serverRequest.multipartData()));
}
After that I use org.springframework.web.reactive.function.client.WebClient to send a request to another server. The object returned from the serverRequest.multipartData() method is of type Mono<MultiValueMap<String,Part>>.
public Mono<ServerResponse> uploadItems(Mono<MultiValueMap<String, Part>> itemsMono) {
return webClient
.post()
.uri(catalogDestination.getUploadItemsUrl())
.body(itemsMono, MultiValueMap.class)
.exchange();
}
The following error is shown when I send the request:
org.springframework.core.codec.CodecException: No suitable writer found for part: zip
at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.encodePart(MultipartHttpMessageWriter.java:299) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.lambda$encodePartValues$4(MultipartHttpMessageWriter.java:252) ~[spring-web-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:367) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:211) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:139) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:63) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.Flux.subscribe(Flux.java:8134) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:441) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:211) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:139) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:63) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.Flux.subscribe(Flux.java:8134) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onComplete(FluxConcatArray.java:207) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.FluxConcatArray.subscribe(FluxConcatArray.java:80) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.Flux.subscribe(Flux.java:8134) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.netty.channel.MonoSendMany.subscribe(MonoSendMany.java:81) ~[reactor-netty-0.9.1.RELEASE.jar:0.9.1.RELEASE]
at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:153) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.ignoreDone(MonoIgnoreThen.java:190) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreInner.onComplete(MonoIgnoreThen.java:240) ~[reactor-core-3.3.0.RELEASE.jar:3.3.0.RELEASE]
at reactor.netty.FutureMono$FutureSubscription.operationComplete(FutureMono.java:188) ~[reactor-netty-0.9.1.RELEASE.jar:0.9.1.RELEASE]
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:604) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:715) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:270) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:350) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.epoll.AbstractEpollChannel.doWriteBytes(AbstractEpollChannel.java:370) ~[netty-transport-native-epoll-4.1.43.Final-linux-x86_64.jar:4.1.43.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel.writeBytes(AbstractEpollStreamChannel.java:260) ~[netty-transport-native-epoll-4.1.43.Final-linux-x86_64.jar:4.1.43.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel.doWriteSingle(AbstractEpollStreamChannel.java:471) ~[netty-transport-native-epoll-4.1.43.Final-linux-x86_64.jar:4.1.43.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel.doWrite(AbstractEpollStreamChannel.java:429) ~[netty-transport-native-epoll-4.1.43.Final-linux-x86_64.jar:4.1.43.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:931) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.flush0(AbstractEpollChannel.java:514) ~[netty-transport-native-epoll-4.1.43.Final-linux-x86_64.jar:4.1.43.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:898) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1384) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:749) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:741) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:727) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.handler.logging.LoggingHandler.flush(LoggingHandler.java:265) ~[netty-handler-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:749) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:741) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:727) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:533) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:125) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:358) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:749) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:741) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.access$2100(AbstractChannelHandlerContext.java:56) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1150) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1073) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413) ~[netty-transport-native-epoll-4.1.43.Final-linux-x86_64.jar:4.1.43.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.43.Final.jar:4.1.43.Final]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
What version of Spring Framework are you using?
Try using org.springframework.web.reactive.function.BodyInserters
public Mono<ServerResponse> uploadItems(Mono<MultiValueMap<String, Part>> itemsMono) {
return Mono.from(itemsMono)//
.flatMap(im -> webClient//
.post()//
.uri(catalogDestination.getUploadItemsUrl())//
.body(BodyInserters.fromMultipartData(im))//
.exchange());
}
I've repeated the example provided in the Creating a Batch Service tutorial
But I want to add second step to the job so it should do following:
1 step(it is already provided in tutorial): read people names from csv and save it in databse in upperCase
2 step: read from database names convert it to lowercase and insert into another database table.
Currently my configuration looks like this:
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
// tag::readerwriterprocessor[]
#Bean
public FlatFileItemReader<Person> reader() {
return new FlatFileItemReaderBuilder<Person>()
.name("personItemReader")
.resource(new ClassPathResource("users.csv"))
.delimited()
.names(new String[]{"firstName", "lastName"})
.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
setTargetType(Person.class);
}})
.build();
}
#Bean
public JdbcCursorItemReader<Person> dbReader(DataSource dataSource) {
JdbcCursorItemReader<Person> reader = new JdbcCursorItemReader<>();
reader.setDataSource(dataSource);
BeanWrapperFieldSetMapper<Person> personBeanWrapperFieldSetMapper = new BeanWrapperFieldSetMapper<Person>() {{
setTargetType(Person.class);
}};
reader.setRowMapper(new RowMapper<Person>() {
#Override
public Person mapRow(ResultSet resultSet, int i) throws SQLException {
Person person = new Person();
person.setFirstName(resultSet.getString(0));
person.setLastName(resultSet.getString(1));
return person;
}
});
reader.setSql("SELECT first_name, last_name FROM people");
reader.open(new ExecutionContext());
return reader;
}
#Bean
public PersonItemProcessor processor() {
return new PersonItemProcessor();
}
#Bean
public ToLowerCaseProcessor processor2() {
return new ToLowerCaseProcessor();
}
#Bean
public JdbcBatchItemWriter<Person> writer(DataSource dataSource) {
return new JdbcBatchItemWriterBuilder<Person>()
.itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
.sql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)")
.dataSource(dataSource)
.build();
}
// end::readerwriterprocessor[]
// tag::jobstep[]
#Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1, Step step2) {
return jobBuilderFactory.get("importUserJob")
//.incrementer(new RunIdIncrementer())
.listener(listener)
.listener(new JobExecutionListener() {
#Override
public void beforeJob(JobExecution jobExecution) {
System.out.println("!!!!!!!!!!!!!SECOND_LISTENER_BEFORE!!!!!!!!!!!!!!!!");
}
#Override
public void afterJob(JobExecution jobExecution) {
System.out.println("!!!!!!!!!!!!!SECOND_LISTENER_AFTER!!!!!!!!!!!!!!!!");
}
})
.flow(step1)
.next(step2)
.end()
.build();
}
#Bean
public Step step1(JdbcBatchItemWriter<Person> writer) {
return stepBuilderFactory.get("step1")
.<Person, Person>chunk(10)
.reader(reader())
.processor(processor())
.writer(writer)
.build();
}
#Bean
public Step step2(JdbcBatchItemWriter<Person> writer, DataSource dataSource) {
return stepBuilderFactory.get("step2")
.<Person, Person>chunk(10)
.reader(dbReader(dataSource))
.processor(processor2())
.writer(writer)
.build();
}
// end::jobstep[]
}
But when I start my spring boot application on the second step I experince the following error:
2019-08-02 14:33:54.164 ERROR 26972 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step step2 in job importUserJob
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:152) ~[spring-batch-infrastructure-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:311) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:313) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:144) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at com.sun.proxy.$Proxy44.run(Unknown Source) ~[na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:206) ~[spring-boot-autoconfigure-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:180) ~[spring-boot-autoconfigure-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:167) ~[spring-boot-autoconfigure-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:162) ~[spring-boot-autoconfigure-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:779) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:763) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:318) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1213) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1202) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at hello.Application.main(Application.java:10) ~[classes/:na]
Caused by: java.lang.IllegalStateException: Stream is already initialized. Close before re-opening.
at org.springframework.util.Assert.state(Assert.java:73) ~[spring-core-5.1.8.RELEASE.jar:5.1.8.RELEASE]
at org.springframework.batch.item.database.AbstractCursorItemReader.doOpen(AbstractCursorItemReader.java:424) ~[spring-batch-infrastructure-4.1.2.RELEASE.jar:4.1.2.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:149) ~[spring-batch-infrastructure-4.1.2.RELEASE.jar:4.1.2.RELEASE]
... 34 common frames omitted
I have no idea what do I wrong. What the stream has mentioned ?
Could please explain how to correct it ?
in your dbReader method you should'nt call :
reader.open(new ExecutionContext());
It is called by springbatch.
So my spring batch job has three steps, the first step(tasklet) will copy files from a shared drive and store them in a folder of the spring project(FileSystems.getDefault()."folder_name"). The second step should read the xml file from this folder and process as per business logic and finally write into the database.The final step(tasklet) will delete the folder created in step 1. But I am getting "java.lang.IllegalArgumentException: Resources must be set and o.s.batch.core.step.AbstractStep : Exception while closing step execution resources in step step1 in job job java.lang.NullPointerException: null." I am using MultiResourceItemReader to read the files in the second step. Can anybody help on how to remove this exception and so that the batch executes as per the mentioned logic.
This is the batch configuration-
#Autowired
public TaskletStep taskletStep;
#Autowired
public MyItemReader myItemReader;
#Autowired
public MyItemWriter myItemWriter;
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Bean
public Job job() throws FileNotFoundException, XMLStreamException, URISyntaxException {
return jobBuilderFactory.get("job")
.incrementer(new RunIdIncrementer())
.start(step1()).next(step2())
.build();
}
#Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.tasklet(taskletStep).listener(promotionListener())
.build();
}
#Bean
public ExecutionContextPromotionListener promotionListener() {
ExecutionContextPromotionListener listener = new ExecutionContextPromotionListener();
listener.setKeys(new String[] {"someKey" });
return listener;
};
#Bean
public Step step2() {
return stepBuilderFactory.get("step2")
.<String, String> chunk(1)
.reader(myItemReader)
.writer(myItemWriter)
.build();
}
This is the taskletStep
private static FileSystem fileSystem = FileSystems.getDefault();
private String locationSourceNetwork = "E:\\assss\\wfwef.txt";
private ChunkContext chunkContext;
String homeOfficeDirectoryUnzip = fileSystem.getPath("HOME").toAbsolutePath().toString();
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
//String homeOfficeDirectoryUnzip = fileSystem.getPath("HOME").toAbsolutePath().toString();
File file = new File(homeOfficeDirectoryUnzip);
Path sourceDirectory = Paths.get(locationSourceNetwork);
//Path targetDirectory = Paths.get(homeOfficeDirectoryUnzip);
Files.copy(sourceDirectory, file.toPath());
ExecutionContext stepContext = chunkContext.getStepContext().getStepExecution().getExecutionContext();
stepContext.put("someKey", homeOfficeDirectoryUnzip);
return RepeatStatus.FINISHED;
}
#BeforeStep
public void saveStepExecution(ChunkContext chunkContext) {
this.chunkContext = chunkContext;
/*ExecutionContext stepContext = this.stepExecution.getExecutionContext();
stepContext.put("someKey", homeOfficeDirectoryUnzip);*/
}
This is the MyItemReader-
private Resource[] resources;
/*#Autowired
ResourcesFactoryBean fileResources;*/
private static String files = new ClassPathResource("HOME").toString();//FileSystems.getDefault().getPath("HOME").toAbsolutePath().toString();
private Resource[] resources;
#BeforeStep
public void retrieveInterstepData(StepExecution stepExecution) throws IOException {
JobExecution jobExecution = stepExecution.getJobExecution();
ExecutionContext jobContext = jobExecution.getExecutionContext();
ResourcePatternResolver patternResolver = new PathMatchingResourcePatternResolver();
this.resources = patternResolver.getResources("file:"+jobContext.get("someKey").toString());
}
public MultiResourceItemReader<String> multiResourceItemReader()
throws FileNotFoundException, XMLStreamException, URISyntaxException {
MyItemReader resourceItemReader = new MyItemReader();
resourceItemReader.setResources(resources);
resourceItemReader.setDelegate(headerXmlReader());
resourceItemReader.setStrict(false);
return resourceItemReader;
}
public StaxEventItemReader<String> headerXmlReader()
throws FileNotFoundException, XMLStreamException, URISyntaxException {
// TODO Auto-generated method stub
StaxEventItemReader<String> staxHeaderDataReader = new StaxEventItemReader<String>();
System.out.println("HIIIIIIIIIII");
int i = resources.length;
while (i > 0) {
staxHeaderDataReader.setResource(resources[i - 1]);
i--;
}
return staxHeaderDataReader;
}
This resource should have the file which is being copied by TaskletStep but the batch is failing during application startup. Note-this is just a dummy poc of the actual scenario.
Stack Trace--
main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=job]] launched with the following parameters: [{run.id=10}]
2018-10-19 20:35:43.959 INFO 3108 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
2018-10-19 20:35:43.998 INFO 3108 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [step2]
2018-10-19 20:35:44.009 ERROR 3108 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step step2 in job job
java.lang.IllegalArgumentException: Resources must be set
at org.springframework.util.Assert.notNull(Assert.java:193) ~[spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.batch.item.file.MultiResourceItemReader.open(MultiResourceItemReader.java:168) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:310) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:197) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:394) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_181]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_181]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_181]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_181]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at com.sun.proxy.$Proxy89.run(Unknown Source) [na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1258) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at com.infotech.batch.SpringBootBatchTaskletApplication.main(SpringBootBatchTaskletApplication.java:12) [classes/:na]
2018-10-19 20:35:44.014 ERROR 3108 --- [ main] o.s.batch.core.step.AbstractStep : Exception while closing step execution resources in step step2 in job job
java.lang.NullPointerException: null
at org.springframework.batch.item.file.MultiResourceItemReader.close(MultiResourceItemReader.java:155) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.close(CompositeItemStream.java:89) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.close(TaskletStep.java:305) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:271) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:394) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_181]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_181]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_181]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_181]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at com.sun.proxy.$Proxy89.run(Unknown Source) [na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1258) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at com.infotech.batch.SpringBootBatchTaskletApplication.main(SpringBootBatchTaskletApplication.java:12) [classes/:na]
2018-10-19 20:35:44.019 INFO 3108 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=job]] completed with the following parameters: [{run.id=10}] and the following status: [FAILED]
You don't need to set the resource manually on the delegate reader here:
private StaxEventItemReader<String> headerXmlReader()
throws FileNotFoundException, XMLStreamException, URISyntaxException {
// TODO Auto-generated method stub
StaxEventItemReader<String> staxHeaderDataReader = new StaxEventItemReader<String>();
int i = resources.length;
while (i > 0) {
staxHeaderDataReader.setResource(resources[i - 1]);
i--;
}
return staxHeaderDataReader;
}
The MultiResourceItemReader will do it automatically while reading the resources. Basically, what you need to do is to configure everything on the delegate (root tag name, unmarshaller, etc) except the resource. The resource will be set dynamically by the MultiResourceItemReader on the delegate.
You can find an example of how to configure a MultiResourceItemReader with a delegate StaxEventItemReader here.
pass value of resource from step 1 to the next step
step1 should dynamically pass the information about the created resource to step2 via the step execution context. For more details about how to do this, see:
Similar SO question: How can we share data between the different steps of a Job in Spring Batch?
"Passing Data to Future Steps" section in the documentation: https://docs.spring.io/spring-batch/4.0.x/reference/html/common-patterns.html#passingDataToFutureSteps.
Hope this helps.
I'm new to Spring Batch and am trying to work with a simple Spring Batch application (http://spring.io/guides/gs/batch-processing/) and am trying to convert it to use Oracle as a storage mechanism.
The problem I'm running into is below;
INFO 22152 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=importUserJob]] failed unexpectedly and fatally with the following parameters: [{run.id=7, -spring.output.ansi.enabled=always}]
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:803) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:771) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1186) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1175) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at hello.Application.main(Application.java:10) [classes/:na]
Caused by: java.lang.IllegalArgumentException: Resource must not be null
at org.springframework.util.Assert.notNull(Assert.java:115) ~[spring-core-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.transaction.support.TransactionSynchronizationUtils.unwrapResourceIfNecessary(TransactionSynchronizationUtils.java:62) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.transaction.support.TransactionSynchronizationManager.getResource(TransactionSynchronizationManager.java:137) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doGetTransaction(DataSourceTransactionManager.java:182) ~[spring-jdbc-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:337) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:430) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:276) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at com.sun.proxy.$Proxy42.update(Unknown Source) ~[na:na]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:357) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_102]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_102]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_102]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
at com.sun.proxy.$Proxy47.run(Unknown Source) ~[na:na]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:216) ~[spring-boot-autoconfigure-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:233) ~[spring-boot-autoconfigure-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:125) ~[spring-boot-autoconfigure-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:119) ~[spring-boot-autoconfigure-1.4.1.RELEASE.jar:1.4.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
The problem is, I can't figure out what resource is null and how to resolve it.
The code for my batch is below;
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Bean
public JobRepository jobRepository(){
JobRepositoryFactoryBean jReposFact = new JobRepositoryFactoryBean();
jReposFact.setDataSource(dataSource());
jReposFact.setTransactionManager(new DataSourceTransactionManager());
jReposFact.setIsolationLevelForCreate("ISOLATION_REPEATABLE_READ");
JobRepository jRepos = null;
try{
jRepos = jReposFact.getObject();
}catch(Exception e){
e.printStackTrace();
}
return jRepos;
}
#Bean
public DataSource dataSource(){
return DataSourceBuilder.create()
.url("jdbc:oracle:thin:#localhost:1521:curamdb")
.driverClassName("oracle.jdbc.OracleDriver")
.username("springbatch")
.password("password")
.build();
}
#Bean
public FlatFileItemReader<Person> reader() {
FlatFileItemReader<Person> reader = new FlatFileItemReader<Person>();
reader.setResource(new ClassPathResource("sample-data.csv"));
reader.setLineMapper(new DefaultLineMapper<Person>() {{
setLineTokenizer(new DelimitedLineTokenizer() {{
setNames(new String[] { "firstName", "lastName" });
}});
setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
setTargetType(Person.class);
}});
}});
return reader;
}
#Bean
public PersonItemProcessor processor() {
return new PersonItemProcessor();
}
#Bean
public JdbcBatchItemWriter<Person> writer() {
JdbcBatchItemWriter<Person> writer = new JdbcBatchItemWriter<Person>();
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Person>());
writer.setSql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)");
writer.setDataSource(dataSource());
return writer;
}
#Bean
public Job importUserJob(JobCompletionNotificationListener listener) {
return jobBuilderFactory.get("importUserJob")
.incrementer(new RunIdIncrementer())
.listener(listener)
.flow(step1())
.end()
.build();
}
#Bean
public Step step1() {
return stepBuilderFactory.get("step1")
.<Person, Person> chunk(10)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
}
I have a suspicion is could be to do with my JobRepository definition, and maybe with the TransactionManager but am not familiar enough with Spring Batch to work out what the problem is.
Thanks in advance!
Bic
Your dataSource is null, when the job starts. (check the stacktrace for DataSourceTransactionManager.doGetTransaction -> this method calls TransactionSynchronizationManager.getResource with a datasource as parameter).
I just can guess, but the problem could be the order in which the beans are being created
In your configuration class, you also define the datasource. But there is a chance, that the datasource been has not been initialized completely when the whole "launching beans of springbatch" are being created.
As far as I understood, the code works if your remove the dataSource and the jobRepository creation from your configuration class.
In this case, springboot will provide a default inmemorydb-datasource and an appropriate jobrepository. I'd suggest, that you remove this two entries from your configuration file and see if the code works (I expect it will) Note: you need to add "#Autowired DataSource dataSource".
Next, write your own configuration class (annotated with '#Configuration') and add only your DataSource definition to it.
#Bean
public DataSource dataSource(){
return DataSourceBuilder.create()
.url("jdbc:oracle:thin:#localhost:1521:curamdb")
.driverClassName("oracle.jdbc.OracleDriver")
.username("springbatch")
.password("password")
.build();
}
Use the "#Import" statement to add this configuration to your class.
#Configuration
#EnableBatchProcessing
#Import(YourDataSourceConfiguration.class)
public class BatchConfiguration {
#Autowired
private DataSource dataSource;
....