I've got a Spring Boot command line application using Spring Integration. The application is running within a service wrapper.
At start up, everything seems to run smoothly with no exceptions. However, if I drop a file in the observed directory nothing triggers the inbound-channel-adapter.
Is it at all possible the service wrapper is causing the polling to not run?
UPDATE
I added a logback.xml configuration
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<logger name="org.springframework.integration" level="DEBUG">
<appender-ref ref="STDOUT" />
</logger>
</configuration>
But now the log is not receiving any input with this config. STDOUT works fine without the config. Makes is difficult to enable DEBUG for the integration package.
I based my wrapper on this project
UPDATE #2
I finally got logging working. I had a mistake in assuming it was not a ERROR level issue.
13:29:52.925 [task-scheduler-10] ERROR o.s.i.handler.LoggingHandler - org.springframework.messaging.MessagingException: The path [\\server\sharepath] does not denote a properly accessible directory.
at org.springframework.integration.file.DefaultDirectoryScanner.listFiles(DefaultDirectoryScanner.java:83)
at org.springframework.integration.file.FileReadingMessageSource.scanInputDirectory(FileReadingMessageSource.java:293)
at org.springframework.integration.file.FileReadingMessageSource.receive(FileReadingMessageSource.java:272)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:175)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:224)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:57)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:176)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:173)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:330)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:51)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:324)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
The issue is that the network path cannot be determined if accessible by Java/Spring.
13:29:52.925 [task-scheduler-10] ERROR o.s.i.handler.LoggingHandler -
org.springframework.messaging.MessagingException: The path
[\server\sharepath] does not denote a properly accessible directory.
Related
I'm trying to create a custom log file for my flow in Nifi; to achieve that I want to use the logback filters.
In my logback.conf I added a new appender called LLAB_FILE which filters the log message by its content:
<appender name="LLAB_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.bootstrap.config.log.dir}/llab-transform.log</file>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator>
<expression>return message.contains("LLAB_TRANSFORM");</expression>
</evaluator>
<OnMismatch>DENY</OnMismatch>
<OnMatch>ACCEPT</OnMatch>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/llab-transform_%d.%i.log</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<maxHistory>30</maxHistory>
</rollingPolicy>
<immediateFlush>true</immediateFlush>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date~%level~%msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="LLAB_FILE"/>
</root>
However this configuration gives the following error when Nifi starts:
Failed to instantiate SLF4J LoggerFactory
Reported exception:
java.lang.NoClassDefFoundError: org/codehaus/janino/ScriptEvaluator
at ch.qos.logback.core.boolex.JaninoEventEvaluatorBase.start(JaninoEventEvaluatorBase.java:57)
at ch.qos.logback.core.joran.action.NestedComplexPropertyIA.end(NestedComplexPropertyIA.java:161)
at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:309)
at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:193)
at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:179)
at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:165)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:152)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:110)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:53)
at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)
at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:150)
at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:84)
at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.apache.nifi.bootstrap.RunNiFi.<init>(RunNiFi.java:134)
at org.apache.nifi.bootstrap.RunNiFi.main(RunNiFi.java:231)
Caused by: java.lang.ClassNotFoundException: org.codehaus.janino.ScriptEvaluator
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 20 more
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/janino/ScriptEvaluator
at ch.qos.logback.core.boolex.JaninoEventEvaluatorBase.start(JaninoEventEvaluatorBase.java:57)
at ch.qos.logback.core.joran.action.NestedComplexPropertyIA.end(NestedComplexPropertyIA.java:161)
at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:309)
at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:193)
at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:179)
at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:165)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:152)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:110)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:53)
at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)
at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:150)
at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:84)
at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.apache.nifi.bootstrap.RunNiFi.<init>(RunNiFi.java:134)
at org.apache.nifi.bootstrap.RunNiFi.main(RunNiFi.java:231)
Caused by: java.lang.ClassNotFoundException: org.codehaus.janino.ScriptEvaluator
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
How can I fix this? I don't want to process the logs throw a nifi's flow.
Thanks,
Temp
http://logback.qos.ch/manual/filters.html#JaninoEventEvaluator
JaninoEventEvaluator requires the Janino library.
Please see the corresponding section of the setup document.
actually any evaluator requires additional library
I have a Java maven application in which I have created a swing GUI.
I am using logback-classic for logging purposes. logback.xml is present under the src/main/resources directory.
When I run the application using eclipse "Run As Java Application", it is working as intended and application is logging to the console and log file.
But after exporting the application as a runnable JAR file and double clicking to run the application, it is not logging to file.
When I opened command prompt and execute application using "java -jar myjarfile.jar", I am able to see the console logs, but it is using the default logback configuration.
I tried giving additional parameter, -Dlogback.configurationFile="path-to-file/logback.xml" and I am getting the below error,
Failed to instantiate [ch.qos.logback.classic.LoggerContext] Reported
exception: java.lang.IllegalArgumentException: name
at sun.misc.URLClassPath$Loader.findResource(Unknown Source)
at sun.misc.URLClassPath$1.next(Unknown Source)
at sun.misc.URLClassPath$1.hasMoreElements(Unknown Source)
at java.net.URLClassLoader$3$1.run(Unknown Source)
at java.net.URLClassLoader$3$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader$3.next(Unknown Source)
at java.net.URLClassLoader$3.hasMoreElements(Unknown Source)
at sun.misc.CompoundEnumeration.next(Unknown Source)
at sun.misc.CompoundEnumeration.hasMoreElements(Unknown Source)
at ch.qos.logback.core.util.Loader.getResources(Loader.java:73)
at ch.qos.logback.classic.util.ContextInitializer.multiplicityWarning(ContextInitializer.java:183)
at ch.qos.logback.classic.util.ContextInitializer.statusOnResourceSearch(ContextInitializer.java:175)
at ch.qos.logback.classic.util.ContextInitializer.findConfigFileURLFromSystemProperties(ContextInitializer.java:111)
at ch.qos.logback.classic.util.ContextInitializer.findURLOfDefaultConfigurationFile(ContextInitializer.java:120)
at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:148)
at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:84)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:55)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at com.esignature.tools.templatemigration.ConverterGUI.(ConverterGUI.java:35)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:59)
I don't know what I am missing and I want to log from runnable JAR to the log file.
It is confusing because it is working from eclipse but not from the exported runnable JAR.
Below is the logback.xml file.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOG_HOME" value="C:/template converter logs" />
<appender name="STDOUT"
class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level - %msg%n</pattern>
</encoder>
</appender>
<appender name="rollingFile"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy
class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_HOME}/applog-%d{yyyy-MM-dd}.log</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level - %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
<appender-ref ref="rollingFile" />
</root>
</configuration>
The error you are facing is because there is two logback.xml configuration, one is in your classpath and you are trying to override/add another one using -Dlogback.configurationFile
In my case when packaging the runnable JAR I changed the pom configuration to this.
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<executable>true</executable>
</configuration>
</plugin>
Note the executable set to true, this makes sure that a MANIFEST.MF file is added to the JAR package.
So in order to make this work with this config I added this property to my application.yml/application.properties
logging:
config: file:D:\\LogConfig\\logback.xml
Then I proceed to generate the JAR file using clean, install mvn commands.
From there running the JAR file using java -jar myJar.jar works fine and there is no need to add arguments to search the application.yml or logback.xml. That is because the properties is being loaded from within the jar and the log config from the specified route on the application.yml .
Using maven shade plugin to generate a shaded JAR file that includes the dependencies and the custom logback.xml is one working solution that I utilized.
Now when I run this JAR file (either from command line or by double-clicking it), logs are generated in the specified location as expected.
So I have a quite simple logstash configuration that should read the host URL from spring. Although for some reason it is not working.
My logback-spring.xml looks sort of like this:
<springProperty name="logstashDestination" scope="context" source="service.logstash.host"/>
<appender class="net.logstash.logback.appender.LogstashTcpSocketAppender" name="LOGSTASH">
<destination>${logstashDestination}</destination>
<!-- encoder is required -->
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<customFields>{"service":"provisioning-engine-jane"}</customFields>
</encoder>
</appender>
In my application.properties I have these lines:
spring.profiles.active:sit
service.logstash.host:NaN
And in application-sit.properties I have this line:
service.logstash.host:api-test.local:9013
This makes the whole application crash with the error
Exception in thread "restartedMain" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: java.lang.IllegalStateException: Logback configuration error detected:
ERROR in ch.qos.logback.core.joran.util.PropertySetter#44cb3536 - Could not invoke method addDestination in class net.logstash.logback.appender.LogstashTcpSocketAppender with parameter of type java.lang.String java.lang.reflect.InvocationTargetException
ERROR in net.logstash.logback.appender.LogstashTcpSocketAppender[LOGSTASH] - No destination was configured. Use <destination> to add one or more destinations to the appender
The odd thing here is that if I add a bootstrap.properties file with some dummy data it starts. Issue with this is that the dummy data is overriding my environment specific logstash URL.
Anyone have any idea of why this might be happening?
I have created a sample application by following the tutorial mentioned at Creating a Basic REST Web Service using Grizzly, Jersey. I have extended the sample with logback.xml. I later found out that jersey uses java.util.logging and configured a slf4j bridge. Things are working fine but I get a unrecoverable exception when having logback level at trace and works fine when level is set at debug or higher (INFO, WARN etc).
Below is the stacktrace -
20:35:24.531 [grizzly-nio-kernel(7) SelectorRunner] WARN o.g.g.filterchain.DefaultFilterChain - GRIZZLY0013: Exception during FilterChain execution
java.lang.NullPointerException: null
at java.lang.String.<init>(String.java:166)
at java.lang.String.valueOf(String.java:3008)
at org.glassfish.grizzly.filterchain.FilterChainContext.toString(FilterChainContext.java:1076)
at java.text.MessageFormat.subformat(MessageFormat.java:1280)
at java.text.MessageFormat.format(MessageFormat.java:865)
at java.text.Format.format(Format.java:157)
at java.text.MessageFormat.format(MessageFormat.java:841)
at org.slf4j.bridge.SLF4JBridgeHandler.getMessageI18N(SLF4JBridgeHandler.java:265)
at org.slf4j.bridge.SLF4JBridgeHandler.callLocationAwareLogger(SLF4JBridgeHandler.java:220)
at org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:303)
at java.util.logging.Logger.log(Logger.java:738)
at java.util.logging.Logger.doLog(Logger.java:765)
at java.util.logging.Logger.log(Logger.java:851)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:280)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:539)
at org.glassfish.grizzly.nio.NIOConnection.preClose(NIOConnection.java:879)
at org.glassfish.grizzly.nio.transport.TCPNIOConnection.preClose(TCPNIOConnection.java:97)
at org.glassfish.grizzly.nio.NIOConnection.terminate0(NIOConnection.java:603)
at org.glassfish.grizzly.nio.transport.TCPNIOConnection.terminate0(TCPNIOConnection.java:291)
at org.glassfish.grizzly.nio.NIOConnection$4.completed(NIOConnection.java:567)
at org.glassfish.grizzly.nio.NIOConnection$4.completed(NIOConnection.java:563)
at org.glassfish.grizzly.asyncqueue.AsyncWriteQueueRecord.notifyCompleteAndRecycle(AsyncWriteQueueRecord.java:173)
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:279)
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:169)
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:71)
at org.glassfish.grizzly.AbstractWriter.write(AbstractWriter.java:76)
at org.glassfish.grizzly.nio.NIOConnection.closeGracefully0(NIOConnection.java:562)
at org.glassfish.grizzly.nio.NIOConnection.closeWithReason(NIOConnection.java:513)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:160)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:539)
at org.glassfish.grizzly.nio.transport.TCPNIOServerConnection$RegisterAcceptedChannelCompletionHandler.completed(TCPNIOServerConnection.java:353)
at org.glassfish.grizzly.nio.transport.TCPNIOServerConnection$RegisterAcceptedChannelCompletionHandler.completed(TCPNIOServerConnection.java:320)
at org.glassfish.grizzly.nio.DefaultSelectorHandler.registerChannel0(DefaultSelectorHandler.java:336)
at org.glassfish.grizzly.nio.DefaultSelectorHandler.access$400(DefaultSelectorHandler.java:64)
at org.glassfish.grizzly.nio.DefaultSelectorHandler$RegisterChannelOperation.run(DefaultSelectorHandler.java:467)
at org.glassfish.grizzly.nio.DefaultSelectorHandler.processPendingTaskQueue(DefaultSelectorHandler.java:301)
at org.glassfish.grizzly.nio.DefaultSelectorHandler.processPendingTasks(DefaultSelectorHandler.java:293)
at org.glassfish.grizzly.nio.DefaultSelectorHandler.preSelect(DefaultSelectorHandler.java:101)
at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:335)
at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:279)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:593)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:573)
at java.lang.Thread.run(Thread.java:748)
My logging.properties file -
handlers = org.slf4j.bridge.SLF4JBridgeHandler
.level=ALL
java.util.logging.SLF4JBridgeHandler.level = ALL
My logback.xml file -
<configuration>
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="trace">
<appender-ref ref="STDOUT" />
</root>
</configuration>
NOTE: I have the following in my main method if it makes any difference
SLF4JBridgeHandler.removeHandlersForRootLogger();
SLF4JBridgeHandler.install();
This is bug FilterChainContext.toString crashes with NullPointerException #1959 in Grizzly.
There are two overloads of String.valueOf, with Object and char[]. (Java 8)
Because of the dangerous unbound generic in getMessage both overloads would seem applicable, however the char[] overload is chosen, probably because it is more specific.
[snip]
However String.valueOf(char[]) will fail if the result of getMessage is null. I would assume a ClassCastException could happen in other cases. This can be fixed by explicitly casting the result to object or ensuring that the correct generics are infered.
You need to upgrade to a newer version of Grizzly (2.4.3).
A regression bug was filed in Nov 10, 2020 under: FilterChainContext.toString causes NullPointerException #2110 which reports fixed in 4.0.0-M2-RELEASE.
Commit is listed as: https://github.com/eclipse-ee4j/grizzly/commit/d7a96f15083a55c38f63746007f664f1ede9f474
I have a multithreaded application and im using log4j for the logging. When i test my application on my windows 7/intel laptop, it works just fine. But when i deploy the same application to the production server(solaris/sparc), it works for 10 minutes then starts to slow down(very slow). I used profiler to see the threads state. There were a lot of threads blocked because of the log4j operations in the thread dump. Im also using async appender but coulnd't manage to get any successful result. Any guess?
Here are some dump:
"pool-7-thread-105" - Thread t#22939
java.lang.Thread.State: BLOCKED
at org.apache.log4j.Category.callAppenders(Category.java:204)
- waiting to lock <794f2dae> (a org.apache.log4j.spi.RootLogger) owned by "pool-7-thread-112" t#22946
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.info(Category.java:666)
"pool-7-thread-112" - Thread t#22946
java.lang.Thread.State: RUNNABLE
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(Unknown Source)
at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
at java.io.BufferedOutputStream.flush(Unknown Source)
- locked <5fcbc329> (a java.io.BufferedOutputStream)
at java.io.PrintStream.write(Unknown Source)
- locked <19e9d0c5> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source)
at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source)
at sun.nio.cs.StreamEncoder.implFlush(Unknown Source)
at sun.nio.cs.StreamEncoder.flush(Unknown Source)
- locked <3680c465> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush(Unknown Source)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
- locked <3fddded7> (a org.apache.log4j.ConsoleAppender)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
- locked <794f2dae> (a org.apache.log4j.spi.RootLogger)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.info(Category.java:666)
Also log4j.xml - version 1.2.17
<appender name="scripts" class="org.apache.log4j.DailyRollingFileAppender">
<param name="threshold" value="INFO"/>
<param name="file" value="log/script.log"/>
<param name="DatePattern" value="'.'yyyy-MM-dd"/>
<layout class="org.apache.log4j.EnhancedPatternLayout">
<param name="ConversionPattern"
value="%-6p[%-d{ISO8601}] [%t] %m (%F:%L) %n"/>
</layout>
</appender>
<appender name="stdout" class="org.apache.log4j.ConsoleAppender">
<param name="threshold" value="INFO"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-6p[%-d{ISO8601}] [%t] %m (%F:%L) %n"/>
</layout>
</appender>
<appender name="ASYNC" class="org.apache.log4j.AsyncAppender">
<param name="BufferSize" value="5000"/>
<appender-ref ref="scripts"/>
</appender>
<root>
<priority value="debug"/>
<appender-ref ref="stdout"/>
<appender-ref ref="ASYNC"/>
</root>
So the threads that are trying to log are blocked. Why? Because a logging thread is currently blocked in java.io.FileOutputStream.writeBytes(Native Method). That strongly suggests that the thing the logging is going to (a file or whatever) cannot keep up with the amount of logging being done.
The way to troubleshoot this is to start with looking at where your logging is going. If it's going to a file, is that file on a fast, local filesystem? Is disk I/O an issue?
You may wish to use a logging scheme that doesn't block on I/O. It doesn't have to -- other threads can log to an internal queue while a thread is doing disk I/O. However, if the logging rate exceeds the I/O rate, eventually you have to stop logging from building up in memory somehow, and that requires either blocking threads or throwing log entries away. That said, this shouldn't be an issue unless you are either logging way too much, have a serious I/O performance issue (like logging over NFS on a crowded network), or have file buffering disabled somehow.
Judging from the stack dump, something is calling flush() ... so that is going to negate a lot of the "goodness" of using BufferedOutputStream. If you can figure out how to avoid the flushes the throughput will improve.
David Schwartz comments thus:
Windows is notorious for not really honoring file flushes as a way to improve its performance. Solaris and Linux really do full flushes. This might help explain the platform-dependent performance difference.
(I didn't know that. And yes it would.)
Turning off immediateFlush in the WriterAppender might help
Excellent advice.