How do I add fields to log4j2's JSON logs - java

Say I have a standard JSON log as in the example in the docs (below)
{
"logger":"com.foo.Bar",
"timestamp":"1376681196470",
"level":"INFO",
"thread":"main",
"message":"Message flushed with immediate flush=true"
}
Now I want to add custom info to this log like so:
{
"logger":"com.foo.Bar",
"timestamp":"1376681196470",
"level":"INFO",
"thread":"main",
"message":"Message flushed with immediate flush=true",
"extrainformation":"Some very important stuff I need to include",
"extrainformation2":"Some other very important stuff I need to include"
}
Is there a way to do this? The docs don't seem to mention anything about adding properties to the log object. Do I need to make a custom layout or programmatically add fields or what?
relevant log4j2 docs

This can be archived simply via the the config file.
See my log4j2.yaml:
Configuration:
status: warn
appenders:
Console:
name: STDOUT
JsonLayout:
complete: false
compact: true
eventEol: true
KeyValuePair:
-
key: extrainformation
value: Some very important stuff I need to include
-
key: extrainformation2
value: Some other very important stuff I need to include
Loggers:
Root:
level: "warn"
AppenderRef:
ref: STDOUT
Custom fields are always last, in the order they are declared. The values support lookups

Like #alan7678 said -
A custom layout was also the solution for me.
#Plugin(name = "ExtendedJsonLayout", category = Node.CATEGORY,
elementType = Layout.ELEMENT_TYPE, printObject = true)
public class ExtendedJsonLayout extends AbstractJacksonLayout {
// Lots of code!
}
I created a Log4j2 layout plugin called "extended-jsonlayout". You can include it in your project with Maven or Gradle. Check it out here -
https://github.com/savantly-net/log4j2-extended-jsonlayout

Related

Separate loggers for different methods

I use loggers to print traces that are specific to certain methods. That way when I want to debug my code I can activate traces in only those methods I care about.
I used to be able to do that with log4j v1. I would instrument selected methods with loggers like this:
private static void method1() {
Logger logger = Logger.getLogger("org.examples.method1");
logger.debug("Hello from method1");
etc...
}
private static void method2() {
Logger logger = Logger.getLogger("org.examples.method2");
logger.debug("Hello from method2");
etc...
}
Then in my log4j.properties file, I would activate the traces like this:
log4j.logger.org.examples.method1=debug
log4j.logger.org.examples.method2=debug
etc...
I am trying to replicate that behavior in log4j v2, but am having problems with it.
As far as instrumenting methods with loggers, I am doing OK. I do it the same way as with v1, except that I use LogManager.getLogger() instead of Logger.getLogger() (which does not exist in v2).
For activating the loggers, I need to use the v2 syntax, so I now do this:
logger.app.name = org.examples.method1
logger.app.level = debug
logger.app.name = org.examples.method2
logger.app.level = debug
The problem is that when I run code that invokes both method1 and method2, only the method2 traces appear. In order to see the method1 traces, I have to comment out the method2 properties. Note that I have tried adding the additivity = true to both loggers, to no effect.
Piotr (see answer below) says that I need to have a different value for the '.app' bit in the above code. So instead I should write something like this:
logger.1.name = org.examples.method1
logger.1.level = debug
logger.2.name = org.examples.method2
logger.2.level = debug
But this is really inconvenient for my use case because I have to:
Make sure I use the same number on all lines that pertain to a given logger
Make sure that I never use the same number for a different logger
One solution would be for me to write a macro that would automatically generate the two lines from the name of the logger. So, for a logger called org.examples.method1, it might generate:
logger.org_examples_method1.name = org.examples.method1
logger.org_examples_method1.level = debug
Pietr also suggested that I might use the .yml or .json syntax instead of .properties. But when I tried the .yml syntax, I had the same problem, namely, if I write this:
Loggers:
logger:
- name: org.examples.method1
level: debug
logger:
- name: org.examples.method2
level: debug
Then only the method2 traces are being printed. And here, there does not seem to be a "second id" involved, so I am not sure how to fix it.
As you remarked the properties format is not the easiest one to use, since it requires at least to lines per logger and plenty useless identifiers to generate. By comparison you have:
in the properties format:
logger.<id1>.name=org.example.foo
logger.<id1>.level=INFO
logger.<id2>.name=org.example.bar
logger.<id2>.level=DEBUG
in the XML format:
<Loggers>
<Logger name="org.example.foo" level="INFO"/>
<Logger name="org.example.bar" level="DEBUG"/>
</Loggers>
in the YAML format:
Loggers:
logger:
- name: org.example.foo
level: INFO
- name: org.example.bar
level: DEBUG
in the JSON format:
"loggers": {
"logger": [
{ "name": "org.example.foo", "level": "INFO" },
{ "name": "org.example.bar", "level": "DEBUG" }
]
}
There has been some propositions recently aimed at shortening the properties format: see LOG4J2-3341, LOG4J2-3394 or my own PR #735.

Mute Stanford coreNLP logging

First of all, Java is not my usual language, so I'm quite basic at it. I need to use it for this particular project, so please be patient, and if I have omitted any relevant information, please ask for it, I will be happy to provide it.
I have been able to implement coreNLP, and, seemingly, have it working right, but is generating lots of messages like:
ene 20, 2017 10:38:42 AM edu.stanford.nlp.process.PTBLexer next
ADVERTENCIA: Untokenizable: 【 (U+3010, decimal: 12304)
After some research (documentation, google, other threads here), I think (sorry, I don't know how I can tell for sure) coreNLP is finding the slf4j-api.jar in my classpath, and logging through it.
Which properties of the JVM can I use to set logging level of the messages that will be printed out?
Also, in which .properties file I could set them? (I already have a commons-logging.properties, a simplelog.properties and a StanfordCoreNLP.properties in my project's resource folder to set properties for other packages).
Om’s answer is good, but two other possibly useful approaches:
If it is just these warnings from the tokenizer that are annoying you, you can (in code or in StanfordCoreNLP.properties) set a property so they disappear: props.setProperty("tokenize.options", "untokenizable=NoneKeep");.
If slf4j is on the classpath, then, by default, our own Redwoods logger will indeed log through slf4j. So, you can also set the logging level using slf4j.
If I understand your problem, you want to disable all StanfordNLP logging message while the program is executing.
You can disable the logging message. Redwood logging framework is used as logging framework in Stanford NLP. First, clear the Redwood's default configuration(to display log message) then create StanfordNLP pipeline.
import edu.stanford.nlp.util.logging.RedwoodConfiguration;
RedwoodConfiguration.current().clear().apply();
StanfordCoreNLP pipeline = new StanfordCoreNLP(props);
Hope it helps.
In accordance with Christopher Manning's suggestion, I followed this link
How to configure slf4j-simple
I created a file src/simplelogger.properties with the line org.slf4j.simpleLogger.defaultLogLevel=warn.
I am able to solve it by setting a blank output stream to system error stream.
System.setErr(new PrintStream(new BlankOutputStream())); // set blank error stream
// ... Add annotators ...
System.setErr(System.err); // Reset to default
Accompanying class is
public class BlankOutputStream extends OutputStream {
#Override
public void write(int b) throws IOException {
// Do nothing
}
}
Om's answer disables all logging. However, if you wish to still log errors then use:
RedwoodConfiguration.errorLevel().apply();
I also use jdk logging instead of slf4j logging to avoid loading slfj dependencies as follows:
RedwoodConfiguration.javaUtilLogging().apply();
Both options can be used together and in any order. Required import is:
import edu.stanford.nlp.util.logging.RedwoodConfiguration;

vertx LoggerHandler not adding logback

I am trying to use LoggerHandler to log all incoming requests. I am using logback.xml to specify appenders. I am setting system property for logging.
System.setProperty("org.vertx.logger-delegate-factory-class-name",
"org.vertx.java.core.logging.impl.SLF4JLogDelegateFactory");
Still it is logging everything in console not in file.
This worked for me with Vert.x 3.4.1:
import static io.vertx.core.logging.LoggerFactory.LOGGER_DELEGATE_FACTORY_CLASS_NAME;
import io.vertx.core.logging.LoggerFactory;
// ...
setProperty (LOGGER_DELEGATE_FACTORY_CLASS_NAME, SLF4JLogDelegateFactory.class.getName ());
LoggerFactory.getLogger (LoggerFactory.class); // Required for Logback to work in Vertx
The key is to get a logger, which I guess initializes the Logging subsystem, the class that you use to get a Logger seems irrelevant as I tried with two different ones.
I run these lines as the first ones of the program in production code and in the tests to work properly in both contexts.
I was able to get it to work by setting the VM options as such:
-Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4jLogDelegateFactory
Then in my log4j.properties, I had to add this:
log4j.category.io.vertx = TRACE
I know this question is getting a bit old, but the only way I was able to get the vertx LoggerHandler to not use JUL was to call LoggerFactory.initialise() after setting the system property as described in the question.
Even better, I set the property in my build.gradle, like so:
run {
systemProperty(
"vertx.logger-delegate-factory-class-name",
"io.vertx.core.logging.SLF4JLogDelegateFactory"
)
args = ['run', mainVerticleName, "--redeploy=$watchForChange", "--launcher-class=$mainClassName", "--on-redeploy=$doOnChange",
"-Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.SLF4JLogDelegateFactory" ]
}
And then at the very top of my MainVerticle::start I have:
LoggerFactory.initialise()
And, boom. Everything is now formatted correctly, including all the startup output.

Unable to set log level in a Java web start application?

Some logging levels appear to be broke?
I run a Java web start (which I will begin to call JWS from now on) application straight from a GlassFish 3.1.2.2 instance. The client has a static logger like so:
private final static Logger LOGGER;
static {
LOGGER = Logger.getLogger(App.class.getName());
// Not sure how one externalize this setting or even if we want to:
LOGGER.setLevel(Level.FINER);
}
In the main method, I begin my logic with some simple testing of the logging feature:
alert("isLoggable() INFO? " + LOGGER.isLoggable(Level.INFO)); // Prints TRUE!
alert("isLoggable() FINE? " + LOGGER.isLoggable(Level.FINE)); // ..TRUE
alert("isLoggable() FINER? " + LOGGER.isLoggable(Level.FINER)); // ..TRUE
alert("isLoggable() FINEST? " + LOGGER.isLoggable(Level.FINEST)); // ..FALSE
My alert methods will display a JOptionPane dialog box for "true GUI logging". Anyways, you see the printouts in my comments I added to the code snippet. As expected, the logger is enabled for levels INFO, FINE and FINER but not FINEST.
After my alert methods, I type:
// Info
LOGGER.info("Level.INFO");
LOGGER.log(Level.INFO, "Level.INFO");
// Fine
LOGGER.fine("Level.FINE");
LOGGER.log(Level.FINE, "Level.FINE");
// Finer
LOGGER.finer("Level.FINER");
LOGGER.log(Level.FINER, "Level.FINER");
LOGGER.entering("", "Level.FINER", args); // <-- Uses Level.FINER!
// Finest
LOGGER.finest("Level.FINEST");
LOGGER.log(Level.FINEST, "Level.FINEST");
I go to my Java console and click on the tab "Advanced", then I tick "Enable logging". Okay let's start the application. Guess what happens? Only Level.INFO prints! Here's my proof (look at the bottom):
I've done my best to google for log files on my computer and see if not Level.FINE and Level.FINER end up somewhere on the file system. However, I cannot find the log messages anywhere.
Summary of Questions
Why does it appear that logging of Level.FINE and Level.FINER does not work in the example provided?
I set the logging level in my static initializing block, but I'd sure like to externalize this setting to a configuration file of some sort, perhaps packaged together with the EAR file I deploy on GlassFish. Or why not manually write in some property in the JNLP file we download from the server. Is this possible somehow?
Solution for problem no 1.
After doing a little bit more reading on the topic, I concluded that a logger in Java uses a handler to publish his logs. And this handler in his turn has his own set of "walls" for what levels he handles. But this handler need not be attached directly to our logger! You see loggers are organized in a hierarchical namespace and a child logger may inherit his parents handlers. If so, then By default a Logger will log any output messages to its parent's handlers, and so on recursively up the tree (see Java Logging Overview - Oracle).
I ain't saying I get the full picture just yet, and I sure didn't find any quotes about how all of this relates to a Java Web Start application. Surely there has to be some differences. Anyways, I did manage to write together this static initializing block that solves my immediate problem:
static {
LOGGER = Logger.getLogger(App.class.getName());
/*
* This logic can be externalized. See the next solution!
*/
// DEPRECATED: LOGGER.setLevel(Level.FINER);
if (LOGGER.getUseParentHandlers())
LOGGER.getParent().getHandlers()[0].setLevel(Level.FINER);
else
LOGGER.setLevel(Level.FINER);
}
Solution for problem no 2.
The LogManager API docs provided much needed information for the following solution. In a subdirectory of your JRE installation, there is a subdirectory called "lib" and in there you shall find a "logging.properties" file. This is the full path to my file on my Windows machine:
C:\Program Files (x86)\Java\jre7\lib\logging.properties
In here you can change a lot of flavors. One cool thing you could do is to change the global logging level. In my file, this was done on row 29 (why do we see only a dot in front of the "level"? The root-parent of all loggers is called ""!). That will produce a hole lot of output; on my machine I received about one thousand log messages per second. Thus changing the global level isn't even plausible enough to be considered an option. Instead, add a new row where you specify the level of your logger. In my case, I added this row:
martinandersson.com.malivechat.app.App.level = FINER
However, chances are you still won't see any results. In solution no 1, I talked about how loggers are connected to handlers. The default handler is specified in logging.properties, most likely on row 18. Here's how my line reads:
handlers= java.util.logging.ConsoleHandler
Also previously, I talked about how these handlers in their turn use levels for what should trouble their mind. So, find the line that reads something like this (should now be on row 44?):
java.util.logging.ConsoleHandler.level = INFO
..and in my case I swapped "INFO" to "FINER". Problem solved.
But!
My original inquiry into this matter still hasn't provided an answer how one can set these properties closer in par with the application deployment. More specifically, I would like to attach these properties in a separate file, bundled with the application EAR file I deploy on GlassFish or something like that. Do you have more information? Please share!

log4j - set different loglevel for different packages/classes

I use log4j for logging and i want to print all logger.debug statements in a particular class / selected package.
i set the cfg as below>
log4j.category.my.pkg=info
log4j.category.my.pkg.ab.class1=debug
but still only info messages are shown..
is this not the right way ?
Instead of using 'category' use 'logger'. Hence, these level are configured for entire log4j, and does not depend on appender, etc.
Following change works:
log4j.logger.my.pkg=info
log4j.logger.my.pkg.ab.class1=debug
Copying from my current log4j.properties:
log4j.logger.org.hibernate.tool.hbm2ddl=warn
log4j.logger.org.hibernate.sql=info

Categories