We Keep Coding

Temporarily increase log4j2 logger level in multi-threaded service

Here is a long question for you Log4j2 gurus.
I have a service that:
has very strict performance requirements
is instrumented with a lot of logging calls using log4j2.
A typical call is gated, like:
if ( LOG.isInfoEnabled() ) {
LOG.info("everything's fine");
}
Because of the number of log messages and the performance needs, the service will generally run with logging set to WARN (i.e., not many messages).
However, I have been asked to build in a parameter to the service call that, if given, will cause it to:
Temporarily increase the logging level to whatever was requested in the parameter (e.g., INFO or TRACE)
Add a WriterAppender to capture the logging in a PrintWriter.
Append the PrintWriter log data to the request response.
It seems clear, due to the gating I put around each logging call, that I need to actually increase the logging level temporarily, like this:
LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
Configuration cfg = ctx.getConfiguration();
LoggerConfig loggerCfg = cfg.getLoggerConfig("com.mycompany.scm");
loggerCfg.setLevel(logLevel);
.. other code to add `WriterAppender` ...
ctx.updateLoggers();
But I have an immediate problem with that, in that it causes the logging to ALSO go the log file of the service. That might not be the end of the world, but I'd like to avoid that, if possible.
I did that by having the default logging go through appenders that filter by level, so that even if logging is turned on, it won't write any messages more detailed than are wanted in the default log file. (Like this, from my .properties file):
appenders=scm_warn, scm_info
appender.scm_warn.type = Console
appender.scm_warn.name = SCM_WARN
appender.scm_warn.layout.type = PatternLayout
appender.scm_warn.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
appender.scm_warn.filter.threshold.type = ThresholdFilter
appender.scm_warn.filter.threshold.level = warn
appender.scm_info.type = Console
appender.scm_info.name = SCM_INFO
appender.scm_info.layout.type = PatternLayout
appender.scm_info.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
appender.scm_info.filter.threshold.type = ThresholdFilter
appender.scm_info.filter.threshold.level = info
loggers = coreConfigurator
logger.coreConfigurator.name = com.mycompany.scm
logger.coreConfigurator.level = warn
logger.coreConfigurator.additivity = false # do not let configurator log messages get processed by client application's parent or root logger.
logger.coreConfigurator.appenderRefs = core
logger.coreConfigurator.appenderRef.core.ref = SCM_WARN
... that way, even if the logging level gets increased, the extra messages will not go to the main log file (I only want them to go to my PrintWriter).
And now, the question!
How can I temporarily increase the log level (like I try to do in the code above) for the current thread only?
If there are three (3) simultaneous calls to the service, I...
... want each added Appender to only write log messages generated by the thread that created the Appender.
... want each added Appender to be removed after the request that added it finishes.
... want the logging level to get reset back to what it was, as long as there are no other requests with this logging parameter turned on still in process.
Ideally, I think it sounds like I want each thread to have a completely separate logging context. Is that possible? Any thoughts on how to do all this?

You could potentially use a custom Context Selector to have a different context per thread, but that's probably cause issues when multiple threads want to write to the same log file, so likely not a viable option.
The alternative is to write a custom Appender, that uses a ThreadLocal to store the StringWriter. If a StringWriter has not been established for the thread, the appended will skip logging. This custom Appender should be added in the Log4J config file, so it's always there and receiving log entries.
That way you enable logging for a particular thread by creating and assigning a StringWriter to the ThreadLocal, run the code, then clear the ThreadLocal and get the logged information from the StringWriter. Since there initially is no StringWriter for any thread, the appender will do nothing, so shouldn't affect performance in any noticeable way.
You'd still have to do the level-escalation you're already doing, with filters on the other appenders.

You could:
access to class logger and change the log level (as you suggested):
LogManager.getLogger(Class.forName("your.class.package")).setLevel(Level.FATAL);
use a different logger; just configure two different loggers.

Cassandra query logging through spring configuration

is there any easy way to turn on query logging on cassandra through xml configuration? I'm using namespace:
xmlns:cassandra="http://www.springframework.org/schema/data/cassandra"
but I can't find any suitable solution. I was trying to turn on trace through cqlsh, but it dosen't work for my app.
I was trying also to add line:
<logger name="com.datastax.driver.core.QueryLogger.NORMAL" level="TRACE" />
But also doesn't work.
My versions:
spring-data-cassandra-1.4.0
cassandra: 2.1.5

Add a QueryLogger #Bean and get the Cluster #Autowired in:
#Bean
public QueryLogger queryLogger(Cluster cluster) {
QueryLogger queryLogger = QueryLogger.builder()
.build();
cluster.register(queryLogger);
return queryLogger;
}
(+ obviously configure QueryLogger.Builder as required).
Don't forget to set log levels to DEBUG/TRACE in your application.yml:
logging.level.com.datastax.driver.core.QueryLogger.NORMAL: DEBUG
logging.level.com.datastax.driver.core.QueryLogger.SLOW: TRACE
Voilà!

Please check out this link and check if you added the query logger to your cluster definition like stated:
Cluster cluster = ...
QueryLogger queryLogger = QueryLogger.builder(cluster)
.withConstantThreshold(...)
.withMaxQueryStringLength(...)
.build();
cluster.register(queryLogger);
Let me know if it helped.

If you are using Spring Data Cassandra 2.4+ QueryLogger is not available anymore, it was replaced with RequestTracker which can be configured in application.yml or overridden depending on your needs.
The Java driver provides a RequestTracker interface. You can specify an implementation of your own or use the provided RequestLogger implementation by configuring the properties in the datastax-java-driver.advanced.request-tracker namespace. The RequestLogger tracks every query your application executes and has options to enable logging for successful, failed, and slow queries. Use the slow query logger to identify queries that are not within your defined performance.
Configuration:
datastax-java-driver.advanced.request-tracker {
class = RequestLogger
logs {
# Whether to log successful requests.
success.enabled = true
slow {
# The threshold to classify a successful request as "slow". If this is unset, all
# successful requests will be considered as normal.
threshold = 1 second
# Whether to log slow requests.
enabled = true
}
# Whether to log failed requests.
error.enabled = true
# The maximum length of the query string in the log message. If it is longer than that, it
# will be truncated.
max-query-length = 500
# Whether to log bound values in addition to the query string.
show-values = true
# The maximum length for bound values in the log message. If the formatted representation of
# a value is longer than that, it will be truncated.
max-value-length = 50
# The maximum number of bound values to log. If a request has more values, the list of
# values will be truncated.
max-values = 50
# Whether to log stack traces for failed queries. If this is disabled, the log will just
# include the exception's string representation (generally the class name and message).
show-stack-traces = true
}
More details.

If you're using Spring Data for Apache Cassandra version 2.0 or higher, then you can use your logging configuration to activate CQL logging. Set the log level of org.springframework.data.cassandra.core.cql.CqlTemplate to DEBUG, no need to mess with QueryLogger:
-Dlogging.level.org.springframework.data.cassandra.core.cql.CqlTemplate=DEBUG
This can, of course, be permanently done in the application.properties.

SLF4J logging, different Levels

In SLF4J (Logging) how levels are different in characteristic. i.e. How ERROR message is different than DEBUG message.
import org.apache.log4j.Logger;
public class LogClass {
private static org.apache.log4j.Logger log = Logger.getLogger(LogClass.class);
public static void main(String[] args) {
log.trace("Trace Message!")
log.debug("Debug Message!");
log.info("Info Message!");
log.warn("Warn Message!");
log.error("Error Message!");
log.fatal("Fatal Message!");
}
}
The Output is same regardless of Level, is there any difference in implementation:
Debug Message!
Info Message!
Warn Message!
Error Message!
Fatal Message!
If these levels are producing the same kind of messages then why the implementation didn't have only one method with parameter as level.
Something like:
log("Level","msg");

Starting from the bottom, there's no real benefit to have a log(level, msg) method if you already have all the different methods for all the possible levels. Only if you'd need to log the same message in different levels, but that's a bad practice, since that message should clearly fall into one specific category. And you can always choose how much logging you get out by specifying the level globally or at the package/class.
The message are exactly the same on each level, the only difference is if that message is gonna make to the logging output or not, based on your configuration, and what purpose do you give to each level.

The key purpose to name them levels is to enable you to debug at various levels. Say for example,
INFO level can used to log high level information on the progress of the application during execution.
DEBUG level logged is meant to be even deeper than just high level information. At DEBUG level, you can have more information logged that can include information of what is happening at a module level or component level.
TRACE level is even more granular. You can log message like entering and exiting a method and what information is being returned by each method.
ERROR level is to purely meant to log only errors and exception
You need to be mindful of what kind of message can be logged into their respective level.
To answer your question, these levels can be controlled in log4j.properties or log4j.xml. You can specify at what level the application can debug. If everything goes well in application, I would leave it at INFO level. If something goes wrong and I wanted to dig in deepeur in terms of debugging, I would try to turn on at DEBUG level or TRACE level.
Also, understand that when you run the debugging at DEBUG level, even the INFO level logs will be printed. If you turn on the debugged at TRACE level, even the DEBUG and INFO level logs will be printed. If you turn on debugging at INFO level, only INFO level logs will be printed.
I hope you got some clarify.

Because it is easier to use for you as a user. As the implementation, it might have that very code.

Log4j2, set log level at runtime for Thread specific

We have a webserver and multiple users log in to it. We generally put log level to ERROR or INFO level. But sometimes, for debugging purpose, we need to see logs. There is one way to set it at runtime, but this process is not so good in case of loads of traffic. Important logs will be missed and also we don't know for how much time we need to keep it that way. I have written a wrapper in log4j v1.2, which just ignores the level check if userid belongs to some TestUsersList. So, it opens all logs for a particular user[a thread] only. A snippet is below-
public void trace(Object message) {
Object diagValue = MDC.get(LoggerConstants.IS_ANALYZER_NUMBER);
if (valueToMatch.equals(diagValue)) { // Some condition to check test number
forcedLog(FQCN, Level.TRACE, message, null);
return;
}
if (repository.isDisabled(Level.TRACE_INT))
return;
if (Level.TRACE.isGreaterOrEqual(this.getEffectiveLevel()))
forcedLog(FQCN, Level.TRACE, message, null);
}
But now I have moved to log4j2, I don't want to write this wrapper again. Is there any inbuilt functionality which log4j2 provides for this?

This can be done with filters. Add a logger to the configuration that logs all the messages you want, then add a ThreadContextMapFilter that has a KeyValuePair for each user you want to log.
Then put the user ids in the Thread Context within the code.

Disable unwanted INFO-Messages from SOAP Logger

In question Filter unwanted INFO-Messages from Logger it is proposed to disable unwanted SOAP INFO logging by raising the respective log level to WARNING like so:
// Disable SOAP-internal INFO logging
Logger.getLogger("javax.enterprise.resource.webservices.jaxws.server").setLevel(Level.WARNING)
URL url = new URL("http://localhost:9999/ws/SoapControl?wsdl");
QName qname = new QName("http://example.ch/", "SoapControlImplService");
Service service = Service.create(url, qname);
SoapControl soapControl = service.getPort(SoapControl.class); // Unwanted logging happens here
This usually works, but unfortunately not all the time, i.e. the behavior is not deterministic.
Any ideas? Thanks!

Looks like you might be running into garbage collection of loggers. Pin the soap logger with a static final reference or you can add an entry into your logging.properties file to control the level on demand. Every time the logger is recreated the log level is read from the properties file.