How do I suppress INFO messages when using Kafka in Java? - java

I use Apache Kafka client in Java and now I'm getting a lot of messages with [INFO] mark. All the suggestions I've found on the Internet are of changing log4j output level. But I don't use log4j.
A small part of output:
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 2.2.0
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: 05fcfde8f69b0349
[main] INFO org.apache.kafka.clients.consumer.KafkaConsumer - [Consumer clientId=consumer-1, groupId=group3] Subscribed to topic(s): test.topics.tasks
UPDATE:
I use org.apache.kafka:kafka-clients:2.2.0

But I don't use log4j.
Maybe you don't, but some library that you pull, probably does (for example, Spring).
You should consult the documentation for the framework that you use (if any), otherwise, you can add your own log4j.properties file into your classpath like so.
log4j.rootLogger=WARN, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c)%n
log4j.appender.stdout.Target=System.out

You can do this programmatically by adding this line in your code System.setProperty(org.slf4j.impl.SimpleLogger.DEFAULT_LOG_LEVEL_KEY, "off");
This is because Kafka is using sl4j for logging, so disabling sl4j logging will suppress info messages. If you wish to log for other levels/purposes, you can replace "off" with one of these: "trace", "debug", "info", "warn", "error" based on https://www.slf4j.org/api/org/slf4j/impl/SimpleLogger.html

Related

log4j2 in Wildfly 9

I use log4j2 as my default logging, when I run my webapp with Wildfly, My logs get wrapped as an INFO like follows:
17:28:31,127 INFO [stdout] (default task-6) DEBUG 2015-10-15 17:28:31,127 za.co.manticore.core.aspect.cacing.CachingAspect - Entering Caching Aspect for method getApplicationMeta
This is causing my logs to be cluttered, as all my logs are logged as INFO. How can I configure Wildfly to not do this. I have done some googleing but could not find an answer that worked.
I think, you should properly configure log4j2. It looks like currently it has console appender - try remove console appender from log4j2 config
Ok you can configure Wildfly not to wrap your stdout and stderr logs.
Run the following CLI commands
sh jboss-cli.sh --connect '/subsystem=logging/console-handler=JUST-PRINT:add(formatter="%s%E%n")'
sh jboss-cli.sh --connect '/subsystem=logging/logger=stderr:add(use-parent-handlers="false", handlers=[JUST-PRINT])'
sh jboss-cli.sh --connect '/subsystem=logging/logger=stdout:add(use-parent-handlers="false", handlers=[JUST-PRINT])'

log4j - Configuration, Ignoring the lib/jar Logging

I am working on a java application and I am using log4j for logging. My "log4j.properties" file (in WEB-INF/conf) is here -
log4j.rootLogger=info, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%p %t %c - %m%n
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=C:/billingAutomationService/infotrac20.log
log4j.appender.R.MaxFileSize=30MB
log4j.appender.R.MaxBackupIndex=10
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n
In my application there are a some debug messages with logger.debug("debug message"). But with the configuration I ca not see these debug message in my log file - "stdout.log". To see the debug message in my log file ("stdout.log") I have added the following line in my log4j.properties file -
log4j.rootLogger=debug,stdout
But adding this line enables all 'DEBUG' level on my jar/lib along the whole project. I am using a log of spring, hibernate and other jars/libs. This incident makes my log file ("stdout.log") very large even on a single operation. Because a lots of 'DEBUG' message coming from the standard lib/jars which I don't want to debug. Is there any way to turn off the debug message coming form the lib/jars?
Thanks
Please check the below thread.
log4j: package-specific logging
Usually you would have a common package starting for all your files, only enable debug for those package names.
Use root logger as warn.
Also.
Specify only some packages to have debug output

Unable to stop debug logging in log4J

I use log4j on a project that utilizes SpringAMQP to consume and write to a Rabbit pipe. Despite trying everything, I am unable to stop the debug logging on a couple of classes, specifically: Rabbittemplate and BlockingQueueconsumer
My log files are constantly flooded with these these couple of lines.
2014-10-09 11:19:20,914 DEBUG RabbitTemplate - Executing callback on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,8)
2014-10-09 11:20:24,141 DEBUG BlockingQueueConsumer - Retrieving delivery for Consumer: tag=[amq.ctag-BbFL7Ow0d_fL_aSaZgXMtg], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,6), acknowledgeMode=AUTO local queue size=0
This is what my Log4J configuration looks like now.
# Root logger option
log4j.rootLogger=INFO, stdout
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
#org.springframework.amqp.rabbit.listener=WARN|ERROR|FATAL
log4j.logger.org.springframework.amqp.rabbit.listener=WARN
I have tried everything in the Log4J configuration to turn off the debug logs on these dependencies but I haven't seen any difference in the logging. I use gradle to manage dependencies and builds and I have the Log4j.properties file in the src/main/resources folder. And it is dumped in the classpath when I build/run the project.
Can anyone help me turn off these logs so that I can keep the logs clean.

AppEngine - Log4j and Log-Levels

I want to use Log4j for my logging on AppEngine. I configured the logger like this:
.level = INFO
log4j.rootLogger=DEBUG, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %t %c{1}:%M:%L - %m%n
But the logger then logs everything to INFO and the real log level is just shown inline like this: (Log level error)
Is there a way to get the real log levels to work with log4j so that filtering on log levels works on the appengine UI?
Edit: More detailed example:
So I log an error like this:
import org.apache.log4j.Logger;
private static Logger logger = Logger.getLogger(PingServlet.class);
logger.error("Database was successfully pinged.");
And on the Appengine-UI, it shows up as an INFO (the green I). Only the loggin text indicates the correct log level:
The GAE Java documentation states that everything you write to stdout is logged as INFO and to stderr is WARNING [1]. Your root logger is writing everything to stdout.
I'm not a log4j expert, so I don't know of a way for a logger to write to both stdout and stderr. One workaround is to use two loggers, one for INFO and another one for WARNING.
[1] https://cloud.google.com/appengine/docs/java/requests#Java_Logging

Camel log4j configuration when deployed inside ActiveMQ

I'm running some Camel 2.12.2 routes deployed inside ActiveMQ 5.9.0.
The ActiveMQ log4j is working as expected, but I'm not able to get a log for my camel application when deployed inside ActiveMQ. My Camel log4j.properties looks like this:
#
# The logging properties used
#
log4j.rootLogger=INFO, console, logger
# uncomment the following line to turn on Camel debugging
#log4j.logger.org.apache.camel=DEBUG
# Console appender
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%5p | %m%n
log4j.appender.console.threshold=INFO
# File appender
log4j.appender.logger=org.apache.log4j.RollingFileAppender
log4j.appender.logger.file=camel.log
log4j.appender.logger.maxFileSize=1024KB
log4j.appender.logger.maxBackupIndex=5
log4j.appender.logger.append=true
log4j.appender.logger.layout=org.apache.log4j.PatternLayout
log4j.appender.logger.layout.ConversionPattern=%d | %-5p | %m | %c | %t%n
My camel project is using JavaDSL with SpringRouteBuilders, and when running the application with the maven camel plugin and running with
maven camel:run
the camel.log file is produced as expeceted.
However, when packaging the camel routes and moving the jar to ActiveMQ, the log file is no longer created. I've checked the jars for slf4j, log4j and slf4j-log4j, and the versions are the same in ActiveMQ as in my pom.xml.
Any clues as to what I'm missing here?
EDIT: Just to clarify what I'm after here.
I've experienced issues on more than one occasaion where ActiveMQ is shutting down because there is something wrong with my camel routes, but I don't see any exceptions in the activemq logs, which makes debugging extremely time consuming. Today I had such an issue, and realised that while there was still no trace of error in the log, I got a stacktrace when just doing maven camel:run on my project. A simple example (really just an example, I know why this is happening, but I want it logged when running in ActiveMQ aswell!)
INFO | Apache Camel 2.12.2 (CamelContext: camel-1) is starting
INFO | StreamCaching is enabled on CamelContext: camel-1
INFO | JMX is enabled
INFO | Using EntityManagerFactory configured: org.springframework.orm.jpa.LocalEntityManagerFactoryBean#2127ee90
INFO | Using TransactionManager found in registry with id [transactionTemplate] org.springframework.orm.jpa.JpaTransactionManager#449f40f1
INFO | Apache Camel 2.12.2 (CamelContext: camel-1) is shutting down
INFO | Apache Camel 2.12.2 (CamelContext: camel-1) uptime 0.477 seconds
INFO | Apache Camel 2.12.2 (CamelContext: camel-1) is shutdown in 0.010 seconds
When deployed in ActiveMQ, this is the last line I see from Camel
But when running maven camel:run, I'm also seeing this:
INFO | Apache Camel 2.12.2 stopping
[ERROR] *************************************
[ERROR] Error occurred while running main from: org.apache.camel.spring.Main
[ERROR]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.camel.maven.RunMojo$1.run(RunMojo.java:486)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.camel.RuntimeCamelException: org.apache.camel.FailedToStartRouteException: Failed to start route XPriceChangeToRibRoute because of duplicate id detected: XPriceChangeToRibRoute. Please correct ids to be unique among all your routes.
at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1352)
...
So, what I really want is for the last part to be visible also when deployed in ActiveMQ, either in my activemq.log if possible, or as a separate log, e.g. camel.log.
Have a look at wrapper.log. That's where the STDOUT and STDERR should be found. Try to start ActiveMQ with the wrapper, e.g. for MacOS you could use:
apache-activemq-5.9.0/bin/macosx/activemq
Or if that's not an option, redirect STDOUT/STDERR to a log file.

Categories