We Keep Coding

What happens when a program logs to the console?

I have a java server application running that uses Logback as its primary logging library. Recently I asked a developer to remove the extra console logging they had added for a non-development environment and when they asked me why realized I didn't have solid reasoning for it.
The extra logging I believe would cause more I/O operations but does it also add more memory usage? How large is the buffer that it's writing to in stdout and when is that cleared?
Our standard logging is to a file which we can view or also have it piped into monitoring tools. The application is deployed via an automated process and is headless so generally no one is on the VM looking at things.
Example logging appenders (Dropwizard configurations)
logging:
level: INFO
appenders:
- type: file
currentLogFilename: /var/log/myapplication.log
archive: true
archivedFileCount: 5
- type: console
target: stdout
Essentially, is there a detriment to logging to the console when not using it and what does that take the form of?

Unless you are logging millions of records, logging has no noticeable impact on performance.
Logging to the console is more ephemeral than logging to a file--the log messages are not saved anywhere. This makes it impossible to track down errors and troubleshoot problems, especially in production.

Logging to STDOUT can be useful if you run your application inside a container like Docker. Docker can fetch anything written to STDOUT and STDERR in any container it runs using docker logs or can redirect to a different server. If the application would write to a logfile local to the container it runs it, it would be much more difficult to access this file from outside the container.

How to run custom appender in separate thread

i just created my own appender as base of Logback document chapter 4 (see Writing your own Appender section).
Whatever is being logged at INFO level in my application, My appender gets invoked and post that message as http message to the servlet running on other end.
these kind of logic makes my application to slow down. because the appender runs on same thread which application is running. How do i make my appender to run in separate thread ?

Since Logback is based on Log4J, you should be able to used asynchronous logging option. See here This makes sure that your logging process runs in a separate thread.

How to verify log4j2 is logging asynchronously via LMAX disruptor?

I am developing an Eclipse RCP application and have gone to some pains to get log4j2 to work within the app. All seems to work fine now, and as a finishing touch I wanted to make all loggers asynchronously.
I've managed to get the LMAX Disruptor on the classpath, and think I've solved the issue of providing sun.misc as well. Set the VM argument -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector in the run config and set up log4j2.xml file correctly as well. I think. And that's where the problem is. I'd like to be able to verify that my application logs asynchronously in the proper fashion, so I can enjoy the benefits latency-wise.
How can I - then - verify that my loggers are working asynchronously, utilising the LMAX Dirsuptor in the process?

There are two types of async logger, handled by different classes.
All loggers async: the AsyncLogger class - activated when you use AsyncLoggerContextSelector
Mixing sync with async loggers: the AsyncLoggerConfig class - when your configuration file has <AsyncRoot> or <AsyncLogger> elements nested in the configuration for <Loggers>.
In your case you are making all loggers async, so you want to put your breakpoint in AsyncLogger#logMessage(String, Level, Marker, Message, Throwable).
Another way to verify is by setting <Configuration status="trace"> at the top of your configuration file. This will output internal log4j log messages on log4j is configured. You should see something like "Starting AsyncLogger disruptor...". If you see this all loggers are async.

Put a breakpoint in org.apache.logging.log4j.core.async.AsyncLoggerConfig#callAppenders. Then you can watch as the event is put into the disruptor. Likewise org.apache.logging.log4j.core.config.LoggerConfig#callAppenders should be getting hit for synchronous logging OR getting hit from the other side of the disruptor for async logging (at which point everything is synchronous again).

Log4j2 JMX: Updating the logging level via jConsole doesn't take effect

I am trying to configure log4j2 so that
I can access the loggers via JMX and
change their log levels.
When I hook everything up, I am able to access the LoggerContext via JConsole, which contains all of my LoggerConfigs.
Each LoggerConfig show the correct log level with which the application is running. And if I update a log level in any LoggerConfig it call the MBean and update the logging level correctly, which I have inspected via debugging. But the problem is that updating the log level doesn't take any effect. The application keeps on logging with the old logging level.
For example If I start the application with the XYZ logger with log level DEBUG, and change this log level to FATAL via JConsole, it changes successfully but application keep on logging in DEBUG level.
If instead of updating the single LoggerConfig if I update the LoggerContext by passing the new xml configuration with updated logging levels it works as expected.
What should be the problem? Documentation is quite and google refused to help me.
My Findings:
As far as I understood this problem is that when I update the Log level in the LoggerConfig via JConsole, log4j2 update the log level via MBean correctly but its not updating the LoggerContext, it simply call the setter method and returns. But in case if I update LoggerCoentext log4j2 create the new context to update itself.

This was indeed a bug. Thanks for reporting it. This has been fixed in trunk and will be included in the next release (rc2).

Sounds like a bug in Log4j2. I see you have raised it on their JIRA # https://issues.apache.org/jira/browse/LOG4J2-637 so we'll track the progress there. :)

Can I close and reopen log files with logback at runtime?

I'm new to logback. I quite fascinated by it but I'm not sure if it suits my use-case.
I would like to have a logger that I can stop and start. While it is stopped I would like to remove the log file from the filesystem. When logging is restarted the file should be re-created.
Is logback capable of this? While the logging is paused, should I avoid calling a Logger in my classes, or can logback handle this?
I use a slf4j.Logger currently. In the manual I saw that Appender objects implement the LifeCycle interface, which implies that they implement start(), stop() and isStarted().
I thought this means they can be stopped so I can move the file, but later on it goes:
If the appender could not be started or if it has been stopped, a
warning message will be issued through logback's internal status
management system. After several attempts, in order to avoid flooding
the internal status system with copies of the same warning message,
the doAppend() method will stop issuing these warnings.
Does it mean that I can stop it, then remove the file, then restart?

I would like to have a logger that I can stop and start. While it is stopped I would like to remove the log file from the filesystem. When logging is restarted the file should be re-created.
I'm not sure how to accomplish this programmatically but you can accomplish this via JMX if you've added jmxConfigurator to the logback.xml config file.
<configuration>
<jmxConfigurator />
...
This exposes the ch.qos.logback.classic.jmx.JMXConfigurator bean which has an operation entitled reloadDefaultConfiguration. When I press that at runtime, the logfiles are reopened. See Jconsole image below. This means that a jmx client (such as the one in my SimpleJMX library for example) would be able to do that from the command line.
If you are trying to it programmatically from inside of the same application then you should be able to get ahold of the mbean and trigger the call yourself. Something like seems to work for me:
ManagementFactory.getPlatformMBeanServer().invoke(new ObjectName(
"ch.qos.logback.classic:Name=default,Type=ch.qos.logback.classic.jmx.JMXConfigurator"),
"reloadDefaultConfiguration", null, null);
What I would do is rename the logfile(s) to a different name(s) and then issue the reload configuration command. Then the renamed files can be archived or removed after the new files are created.
Hope this helps.