Why is stdout file of a job container in hadoop is always of size 0.
On java
import org.apache.log4j.Logger;
static final Logger MAPLOGGER= Logger.getLogger(MyMap.class.getName());
MAPLOGGER.warn("key is :"+key);
after running jar, 3 files are generated
stderr, stdout,syslog
stderr contains
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.impl.MetricsSystemImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
where exactly MAPLOGGER.warn("key is :"+key); writes the log?.
It doesn't write it anywhere right now, because you have not set up log4j.
From the link in the WARN message:
"Why do I see a warning about "No appenders found for logger" and "Please configure log4j properly"?
This occurs when the default configuration files log4j.properties and log4j.xml can not be found and the application performs no explicit configuration. log4j uses Thread.getContextClassLoader().getResource() to locate the default configuration files and does not directly check the file system. Knowing the appropriate location to place log4j.properties or log4j.xml requires understanding the search strategy of the class loader in use. log4j does not provide a default configuration since output to the console or to the file system may be prohibited in some environments."
So you need to set up the log4j engine, and give it to Java overall, or explicitly declare their location in the application.
Related
I know there is several answers for how to set the file. Non have worked with me.
I'm trying to unifying several conf files for that I need to set the conf of log4j provided in slf4j-log4j12 into the configuration file provided as parameter of the program. Till now I tried by setting as parameter of the jvm like:
java '-Dlog4j.configuration=conf.cfg' -jar my.jar
I'm getting back:
log4j:WARN No appenders could be found for logger (org.apache.commons.beanutils.converters.BooleanConverter).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
log4j:ERROR Could not parse file [].
Setting programatically by:
System.setProperty("log4j.configuration",(new File( Configurator.getDefaultConfig().getString(Const.LoggingDefaultLoggingFile))).toURL().toString());
and
System.setProperty("log4j.configurationFile",(new File( Configurator.getDefaultConfig().getString(Const.LoggingDefaultLoggingFile))).toURL().toString());
Same result.
I tried using DOMConfigurator but is for xml I'm using properties so no success.
I was searching how to reload the log4j but the give complex solution for making subscription for monitoring change and constant loading of changes.
I just want programatically change the configuration file once.
Till now, I was able to set the file just if I put it directly into the jar. But this by using a separated file with the default name.
Is there a way of loading the file directly int the Log4j (no shity system properties)? o there is a simple way of doing this better?
log4j looking for log4j.properties under WEB-INF\classes
PropertyConfigurator class,
example of usage:
import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;
// ...
public static final Logger log=Logger.getLogger(App.class);
// ...
PropertyConfigurator.configure("log4j.properties");
Whenever I debug my code in Netbeans this appears:
log4j:WARN No appenders could be found for logger (org.apache.pdfbox.pdfparser.PDFObjectStreamParser).
log4j:WARN Please initialize the log4j system properly.
Why is this? Is this important?
Why is this?
The reason why you see this message is that your log4j configuration file(i.e. log4j.xml or log4j.properties) is NOT found in the classpath. Placing the log4j configuration file in the applications classpath should solve the issue.
is this important?
Depends on requirement, if you want messages logged to a file with defined levels, then yes you need to fix this warning. Otherwise you may ignore.
For setting Log4j in runtime, do this:
java -Dlog4j.configuration=file:///D:/crawler4j-3.5/log4j.properties -jar newCrawlerV0.1.jar
I have the following log4j.properties file, for an application deployed in WebSphere Portal:
log4j.rootLogger=DEBUG, InfoAppender, DebugAppender
log4j.appender.InfoAppender=org.apache.log4j.RollingFileAppender
log4j.appender.InfoAppender.Threshold=INFO
log4j.appender.InfoAppender.File=C:/info.log
log4j.appender.InfoAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.InfoAppender.layout.ConversionPattern=%d %p [%c] - %m%n
log4j.appender.DebugAppender=org.apache.log4j.RollingFileAppender
log4j.appender.DebugAppender.Threshold=DEBUG
log4j.appender.DebugAppender.File=C:/debug.log
log4j.appender.DebugAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.DebugAppender.layout.ConversionPattern=%d %p [%c] - %m%n
When I code, I define the logger at class level:
private static Logger logger = Logger.getLogger(IWannaLogThis.class);
And I log INFO messages with this:
logger.info(theObjectToLog);
When I deploy my application, the debug.log file gets everything I log with logger.debug() but ignores everything I write with logger.info(). On the other side, the info.log file keeps empty.
The weirdest thing is that in debug.log and info.log appears some INFO and DEBUG messages made by some JARS (like Hibernate Validator) I had in the classpath, but just ignores everything I try to log in my code.
Any ideas?
This is most likely a classloading-related problem. WebSphere Portal uses Log4J internally, so I'm guessing that you end up using WebSphere Portal's provided Log4J JAR file as well as its own Log4J properties.
You can verify that by adding the following to the JVM arguments of the server instance:
-Dlog4j.debug=true
And then inspect the SystemOut.log file. Log4J will spit out lots of tracing information about the configuration file(s) it reads.
The best way to avoid this is to do the following:
Bundle the Log4J JAR file with your application.
Associate a Shared Library with the server. In that Shared Library, place your Log4J configuration file.
As an alternative to step 2, you can bundle your Log4J configuration file with the application itself, however that would carry its own drawbacks (for example, having to repackage your application whenever you perform a Log4J configuration change).
Another common problem is that the JARs you have in your classpath also use log4j and also have their own appenders set. So depending on the settings that they use, and the packages that your classes reside in, this may lead to the problem you describe.
So:
Make sure that your package names are unique and not used by any of the third party libraries.
Check the log4j settings in all libraries in your classpath. They should not contain general settings which override yours.
Make sure your loggers use your log4j.properties (you can be sure if changes you make in your file affect your loggers as expected).
If you can, make sure that your log4j stuff loads last, in case any of the third party libs reset the configuration. They shouldn't, but who can stop them.
Normally, it should be one of these things. Post more explicit example if it doesn't work.
Good luck!
What I have done in the past is set specific logs for the classes I want to log. It sounds like you can try setting your root logger to INFO and see if that gets you the messages you want. Here's a little bit of my log4j property file. I set a logger for each class and assign it to my "data" appender, which defines the log layout. In the loggers I specify specific classes I want to log and set their Log level individually. Any class that logs that is not defined in the Loggers I have use the default log level for the rootCategory.
log4j.rootCategory=INFO, rollingFile, stdout
#GetData Loggers
log4j.logger.com.myapp.data=INFO, data
log4j.logger.com.myapp.data.SybaseConnection=DEBUG, data
log4j.logger.com.myapp.data.GetData=ERROR, data
# data appender
log4j.appender.data=org.apache.log4j.RollingFileAppender
log4j.appender.data.layout=org.apache.log4j.PatternLayout
log4j.appender.data.File=c\:\\Program Files\\MyApp\\logs\\MyApp-data.log
log4j.appender.data.Append=true
log4j.appender.data.layout.ConversionPattern=[%d{ISO8601}]%5p%6.6r[%t]%x - %C.%M(%F:%L) - %m%n
you root logger opens the log properties in the debug mode,
use INFO instead of DEbug in the first line of your properties file.
I get the following:
log4j:WARN No appenders could be found for logger (org.test.Books).
log4j:WARN Please initialize the log4j system properly.
I do not want to print these messages, regardless to whether if my log4j configuration is correct or not.
To turn off the warnings try:
Logger.getRootLogger().setLevel(Level.OFF);
And as said in the comment it might be better using the log4j.xml here is a nice link on its format and a few examples too: http://wiki.apache.org/logging-log4j/Log4jXmlFormat
Reference:
How to turn off log4j warnings?
I'm using apache commons library and log4j.
I have an xml configuration file and a log4j.properties files. I want to specify my log4j properties path inside my xml configuration file.
To load my settings i do:
//Loading my xml file
this.config = new XMLConfiguration(this.xmlFileName);
At this moment the following warnings are raised:
log4j:WARN No appenders could be found for logger (org.apache.commons.configuration.ConfigurationUtils).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
However i haven't yet called any log4j object. Once i have read the xml file i can successfully work with my log4j instance.
Is there any way to remove those warnings?
Check if the log4j.properties file is in the classpath
This link might be useful:
http://www.coderanch.com/t/63230/open-source/log-log-WARN-No-appenders
Log4J outputs this error when a logger is created, but no appender(s) is(are) defined.
In practice this error occurs when a logger is created before log4j is initialized.
You say you haven't called any log4j object. But in the error message you see that org.apache.commons.configuration.ConfigurationUtils creates a logger object (see line 66).
You could turn it off before initialization, see How to turn off log4j warnings?
Logger.getRootLogger().setLevel(Level.OFF);
There should be no need to turn it on again since the initialization sets normally the log level of the root logger.
You should at least set the appender and the logger level for the root logger in the loaded log4j configuration file. Otherwise , you will see this warning message.
Example of setting the appender and logger level for the root logger:
#Set root logger 's level and its appender to an appender called CONSOLE which is defined below.
log4j.rootLogger=DEBUG, CONSOLE
#Set the behavior of the CONSOLE appender
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
I resolve my issue with this workaround:
//Disable log level
Logger.getRootLogger().setLevel(Level.OFF);
Now i can read my xml configuration file without WARNINGS.
After set log level:
Logger.getRootLogger().setLevel(Level.INFO);