We Keep Coding

Equinox app: log4j broken eclipse console output

I am using log4j with very simple setup in an Eclipse plugin project, that I am building right now.
I call BasicConfigurator.configure(); in the main class and then just use log4j in every class like this:
private static final Logger logger= LogManager.getLogger(ClassName.class.getName());
That worked fine for a while but now I have problems with log4j. I am not sure what happened, but it seems the logging level is now Level.WARN. Also the console output is somewhat broken. The logging messages are logged two times each. One time normal, and one time really weird:
!ENTRY org.apache.log4j 2 0 2017-01-10 13:19:58.192
!MESSAGE package.ClassName - This is a logging message
0 [main] WARN package.ClassName - This is a logging message
I do not use a properties or xml file so I cannot imagine what I changed.
EDIT: I checked out an older point of the program and the problem still exists. That should mean it is not a problem of the code itself. Can Eclipse settings or something like that affect the console behavior?
EDIT 2: The idea of unchecking "Limit console output" did not help.
EDIT 3: These are my dependencies:
Require-Bundle: org.eclipse.ui,
org.eclipse.emf.ecore;bundle-version="2.12.0",
org.eclipse.emf.ecore.xmi;bundle-version="2.12.0",
org.eclipse.core.resources;bundle-version="3.11.0",
org.eclipse.core.runtime;bundle-version="3.12.0",
org.eclipse.jdt.core;bundle-version="3.12.1",
org.eclipse.jdt.launching;bundle-version="3.8.100",
org.junit,
org.apache.log4j;bundle-version="1.2.15"
EDIT 4: A clean Eclipse version fixes the output problem. But using the clean version would mean to install again all the plugins and tools. Also I would have to configure my preferences from scratch. Also perhaps that then causes the problem again, so I should rather find the cause of the problem.

Try removing the "trace_file" directive from the my_package logger statement.
Try this one
static {
PatternLayout layout = new PatternLayout();
String conversionPattern = "%d %-5p [%t] %c{2} (%2F:%M(%L)):%m%n";
layout.setConversionPattern(conversionPattern);
ConsoleAppender consoleAppender = new ConsoleAppender();
consoleAppender.setLayout(layout);
consoleAppender.activateOptions();
Logger rootLogger = Logger.getRootLogger();
rootLogger.setLevel(Level.INFO);
rootLogger.addAppender(consoleAppender);
}

How to enable/setup log4j for oozi java workflows?

I'm running an Oozie Java workflow (the jar file is in HDFS), and I'd like to add logging functionality to my application. Does anybody know how to do it? Where should I put my "log4j.properties" file? How can I make log4j to output the log to a location in HDFS?

Looking in this documentation, you can try adding oozie-log4j.properties in your oozie directory (where workflow.xml is).
Here are the default settings:
log4j.appender.oozie=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.oozie.RollingPolicy=org.apache.oozie.util.OozieRollingPolicy
log4j.appender.oozie.File=${oozie.log.dir}/oozie.log
log4j.appender.oozie.Append=true
log4j.appender.oozie.layout=org.apache.log4j.PatternLayout
log4j.appender.oozie.layout.ConversionPattern=%d{ISO8601} %5p %c{1}:%L - %m%n
log4j.appender.oozie.RollingPolicy.FileNamePattern=${log4j.appender.oozie.File}-%d{yyyy-MM-dd-HH}
log4j.appender.oozie.RollingPolicy.MaxHistory=720
It also outlines the following restrictions:
In order for Oozie logging to work 100% correctly, the following restrictions must be observed (described below and in the oozie-log4j.properties file):
The appender that Oozie uses must be named "oozie" (i.e. log4j.appender.oozie )
log4j.appender.oozie.RollingPolicy.FileNamePattern must end with "-%d{yyyy-MM-dd-HH}.gz" or "-%d{yyyy-MM-dd-HH}". If it ends with ".gz" the old logs will be compressed when rolled
log4j.appender.oozie.RollingPolicy.FileNamePattern must start with the value of log4j.appender.oozie.File

How to disable Log4j for OpenNLP via log4j.properties

I use Log4j and I import OpenNLP via Maven. Now I want to disable the log output (in red) which is printed in syso from opennlp.
I tried to add to my log4j.properties:
# Root logger option
log4j.rootLogger=INFO, stdout
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.org.opennlp=OFF
The last line should hide opennlp messages such as:
Average: 1000,0 sent/s
Total: 1 sent
Runtime: 0.001s
but does not work. Please tell me why?

OpenNLP is not using Log4j, or any other logging framework. The command line interface prints directly to stdout. The command line tools are only useful for training/evaluating and testing.
For production use you should integrate OpenNLP via its Java API, or via some other framework like Apache UIMA.

Application specific log in tomcat 7 using JULI?

I'm using java system logging in tomcat 7, but no logging statements get written to the log. I've added this file to my WEB-INF/classes. The log file "new-xyz-test" gets created (so I have at least some of the config right) but its empty - no log statements get printed to it.
handlers=java.util.logging.ConsoleHandler, org.apache.juli.FileHandler
org.apache.juli.FileHandler.level=ALL
org.apache.juli.FileHandler.directory=${catalina.base}/logs
org.apache.juli.FileHandler.prefix=new-xyz-test-
java.util.logging.ConsoleHandler.level=ALL
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
com.xyz.level=ALL
com.xyz.handlers=org.apache.juli.FileHandler

To configure JULI in the web applications you need have a logging.properties file in the WEB-INF/classes directory. If you use the default handlers, you may lose messages. You need to specify a prefix for the handler in your file.
handlers=1FILE.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler
.handlers=java.util.logging.ConsoleHandler
1FILE.org.apache.juli.FileHandler.level=FINEST
1FILE.org.apache.juli.FileHandler.directory=/app-logs
1FILE.org.apache.juli.FileHandler.prefix=file-1
java.util.logging.ConsoleHandler.level=FINE
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter
com.xyz.level=INFO
com.xyz.handlers=1FILE.org.apache.juli.FileHandler
com.abc.level=INFO
com.abc.handlers=java.util.logging.ConsoleHandler
A
handler prefix (e.g. 1FILE.) starts with a number, then has an arbitrary string, and ends with a period (.).
See more in Logging in Tomcat
Arguments in the JVM
If you are not running the Tomcat from the startup.sh or startup.bat, you need to specify:
The location of the general logging.properties for Tomcat (in the conf directory of Tomcat)
The manager org.apache.juli.ClassLoaderLogManager. This is important because allows you to configure
for each web application different loggin options. By default, a JVM process can only have a single configuration file.) ,
Similar to the next (I'm using eclipse):
-Djava.util.logging.config.file="C:\Users\Paul\workspaces\utils\.metadata\.plugins\org.eclipse.wst.server.core\tmp0\conf\logging.properties" -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
By default, java.util.logging read the file that is included in the JDK or JRE, e.g.:
"C:\Software\jdk1.7.0_17\jre\lib\logging.properties"
Setting Tomcat Heap Size (JVM Heap) in Eclipse, for how to add arguments in the VM

are you sure that you write to the correct logger , i.e. Logger.getLogger("com.xyz")?
I think that you may got wrong when you wrote in logging.properties:com.xyz.level=ALL com.xyz.handlers=org.apache.juli.FileHandler in the case that you actually write to the logger Logger.getLogger(com.xyz.YourClass.class), that because in the logging properties file you should write the logger name which is in your case com.xyz.YourClass

Logstash-Logback Encoder: “IllegalAccessError: failed to access class […]”

I try to send logs from a java application with the Logstash tcp socket appender to logstash. The java app. already works with logback 1.1.9 (slf4j) and other appenders.
Now I added the following lines to the logback-test.xml:
<configuration>
<appender name="logstash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>[IP]:5010</destination>
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<fieldNames>
<message>log_msg</message>
</fieldNames>
</encoder>
<keepAliveDuration>5 minutes</keepAliveDuration>
</appender>
<root>
<level value="info"/>
<appender-ref ref="logstash" />
</root>
</configuration>
But when I add the appender and start my java application, I get an error from the JVM Launcher "A Java Exception has occured" and I get the following console text:
Exception in thread "main" java.lang.IllegalAccessError: failed to access class ch.qos.logback.core.status.OnPrintStreamStatusListenerBase from class net.logstash.logback.status.LevelFilteringStatusListener (ch.qos.logback.core.status.OnPrintStreamStatusListenerBase and net.logstash.logback.status.LevelFilteringStatusListener are in unnamed module of loader 'app')
at net.logstash.logback.status.LevelFilteringStatusListener.setDelegate(LevelFilteringStatusListener.java:67)
at net.logstash.logback.appender.AsyncDisruptorAppender.start(AsyncDisruptorAppender.java:370)
at net.logstash.logback.appender.AbstractLogstashTcpSocketAppender.start(AbstractLogstashTcpSocketAppender.java:1009)
at ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:90)
at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:309)
at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:193)
at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:179)
at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:158)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:145)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:103)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:53)
at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)
at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:150)
at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:84)
at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
I included the logstash 6.3 jar-file and because of this error, I also tried it with an older version, 4.9. Does anybody know if that is compatible to logback 1.1.9?
I'm not using Spring-Boot.
---------Edit 17.02.2020-----------
My input of the logstash.conf file (because I don't get any log from my java application):
input {
tcp {
mode => "server"
port => 5010
codec => json
}
}

You can avoid the exception you encountered by applying any of the following solutions:
A) Use logback version >= 1.1.10
OR
B) Add <addDefaultStatusListener>false</addDefaultStatusListener> to your appender definition (see details here)
OR
C) Add a status listener to your logback configuration (which prevents logstash-logback-encoder from adding a default status listener).
The exception occurs because logstash-logback-encoder versions >= 6.0 reference logback's ch.qos.logback.core.status.OnPrintStreamStatusListenerBase class when adding a default status listener. OnPrintStreamStatusListenerBase was made public in logback 1.1.10, and was package-private before then.
Applying any of the above solutions will avoid the exception you encountered. However, keep in mind that logstash-logback-encoder versions >= 4.8 are compiled and tested against logback 1.2.x. Therefore, you might encounter other errors when using logback 1.1.x. The logstash-logback-encoder readme contains the following statement on this topic:
Older versions than the ones specified in the pom file might work, but the versions in the pom file are what testing has been performed against.

Solution:
I updated logback from 1.1.9 to 1.2.3. As I replaced the jar files and addeded them to the class-path, I got over 600 times the following error-message: Logger cannot be resolved to a type. I fixed this by opening the properties in Eclipse from the logback project, going on "Order and Export" and setting a marker in every checkbox --> Apply and close. Now I don't get the IllegalAccessError any more and my java application is running fine. And I also now can see the logs in Logstash and Kibana :)
I downloaded the following files:
jcl-over-slf4j 1.7.25
jul-to-slf4j 1.7.25
log4j-over-slf4j 1.7.25
slf4j-api 1.7.25
logback-classic 1.2.3
logback-core 1.2.3
I downloaded them from here:
https://mvnrepository.com/artifact/ch.qos.logback/logback-classic/1.2.3