I'm looking for simplest way to trace execution time of SQL query generated by Hibernate.
Unfortunately it cannot be done in traditional way - just by setting show_sql or hibernate logger, because monitored application is multithread on production environment and sql times tracing should be done only for one service, which is mots problematic.
Service means some component running within Spring application. But in my case it is safe to tell, that it is thread - thread is not changed during invocation. Service implementation is a Java method and this method calls others methods, components, etc, everything i one thread. It is possible for me to change one method source and deploy it, but I cannot release application.
Unfortunately AspectJ cannot be used as is, because I cannot change whole application, recompile nor plug something into JVM.
Unfortunately (next) DB administrators cannot turn on sql queries tracing - they don't know how to do it.
Please help, how to tune Hibernate execution without digging the whole application? What is the simplest way?
Facts: Hibernate 3.x, Spring 2.x, java 1.5
Here is how I would do it, assuming you're using Logback as your logging framework.
Make sure you have scanning enabled on your logback configuration:
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false" scan="true" scanPeriod="30 seconds" >
Make sure your file logger includes thread name in the output (%t):
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%d{yyyy-MM-dd HH:mm:ss} %-5p %C{1} %t - %m%n</pattern>
</encoder>
Start with SQL logging turned off:
<logger name="org.hibernate.SQL" level="OFF">
<appender-ref ref="FILE_APPENDER"/>
</logger>
<logger name="sql-log" level="OFF">
<appender-ref ref="FILE_APPENDER"/>
</logger>
Once your application is up and running, and you're ready to execute your test, edit the logback.xml file of the deployed application, and change those levels to 'DEBUG'. Then execute your tests. When tests are done, set those levels back to 'OFF'.
Look through the log output, and identify the thread name of interest, then grep for that thread name:
grep "thread-1-pool-7" debug.log > sqldebug.log
A bit cumbersome, but it will work.
You are not very specific about filtering criteria: do you want to filter SQLs by thread/HTTP session or from a given service (sql times tracing should be done only for one service)?
Nevertheless everything can be done on logging framework level. First you need to enable logging of all queries and then filter out non-interesting ones. I am assuming you are using Logback (BTW Spring 2.x and Java 1.5.x are becoming obsolete...):
Per thread
Implement Logback filter and discard logs from not interesting thread. Alternatively use SiftingAppender with thread id as a key. All logs from a given thread will be dispatched to a separate file.
Per HTTP session
This is a bit tricky because you need to get access to HTTP session/session id from logging framework level. The easy way is to use MDC (see example: Logging user activity in web app). Having session id you can do filtering similar to Per thread section.
Per service
It's not obvious what do you mean. Do you only want to log SQL queries issued from a given class or from a class and all the methods and classes it calls? In both cases you need to examine a call stack while filtering, which isn't very effective. Also AspectJ has within directive - too heavyweight for you I guess.
If this is what you want to achive please clarify your question, I have some other ideas.
Related
Here is my Hazelcast deployment procedure in two steps:
1) Download the Hazelcast jar at:
https://repo1.maven.org/maven2/com/hazelcast/hazelcast-all/$HAZELCAST_VERSION/hazelcast-all-$HAZELCAST_VERSION.jar
2) Deploy Hazelcast using the command:
java -server $JAVA_OPTS com.hazelcast.core.server.StartServer
Without deviating much from the described procedure, it there a way to set the log level?
More specifically what I'm looking for is something like adding a flag similar to -Djava.util.logging.level=WARN to filter out the info logs, but I have been unsuccessful so far.
I'm trying to avoid using alternative log libraries and xml configuration files for this purpose.
Thank you for your attention
Hazelcast provides multiple ways you can configure the logger. This may be helpful.
Using JVM parameter: java -Dhazelcast.logging.type=log4j
Using System class: System.setProperty( "hazelcast.logging.type", "log4j" );
and in the log4j, add the following.
<logger name="com.hazelcast">
<level value="warn" />
</logger>
Logging Configuration
I believe it's more like a logging config question, rather than Hazelcast. java.util.logging, like other logging frameworks, uses a config file, which you can point using java.util.logging.config.file param. Or you create a LogManager and point it with java.util.logging.config.class param.
Hazelcast doesn't have any internal logging library, just uses java.utl.logging. To configure it, you need to look that logging framework's config options.
I want to log Request/Response of my server in log files. I am using Springboot + jersey 2.x + log4j2. I registered jersey's LoggingFeature like this -
jerseyConfig.register(
new LoggingFeature(
java.util.logging.Logger.getLogger(
LoggingFeature.DEFAULT_LOGGER_NAME),
java.util.logging.Level.SEVERE,
LoggingFeature.Verbosity.PAYLOAD_ANY,
Integer.MAX_VALUE)
);
log4j2.xml
<!-- Jersey logger -->
<AsyncLogger name="org.glassfish" level="all" additivity="false">
<AppenderRef ref="Console" level="off" />
<AppenderRef ref="RollingFileIO" level="error" />
</AsyncLogger>
In pattern, I am injecting transaction-id with help of log4j2's ThreadContext. I have log4j-jul 2.1 in my pom.xml, and I am running it with
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager .
Things are working fine, only problem is, in jersey's logs, I am unable to insert transaction id. I tried utils logging with my custom code, and I am able to put transaction-id in it. But by the time jersey writes these logs, ThreadContext gets cleared and transaction-id values comes empty.
If the log flow is JUL->log4j2->FooAppender->Layout->ThreadContext.pop then
Per Including the ThreadContext when writing logs:
The PatternLayout provides mechanisms to print the contents of the ThreadContext Map and Stack.
Use %X by itself to include the full contents of the Map.
Use %X{key} to include the specified key.
Use %x to include the full contents of the Stack.
If the flow is log4j2->JUL->FooHandler->Formatter->ThreadContext.pop then you have to install or create a formatter that is aware of the ThreadContext. You also have make sure in this configuration that log4j2->ThreadContext.pop->JUL->FooHandler->Formatter is not happening as that would forget all of the information before it gets to JUL.
All of the examples in the documentation show the ThreadContext is set prior to invoking the logger. Which means you have to set the ThreadContext prior to any Jersey code execution.
I haven't looked at Jersey in detail but from the problem description it seems that it has a thread model that makes working with ThreadContext difficult.
Fortunately Log4j 2.7 offers a facility to let you inject key-value pairs (like ID) into log events from another source than the ThreadContext. This was introduced to help with asynchronous frameworks like Finagle, so it may be useful for Jersey as well.
The Log4j2 manual briefly mentions this feature in the section on Custom ContextDataInjectors.
If you want to use this facility you need to write a custom ContextDataInjector and tell Log4j to use that injector instead of the default one by specifying a ContextDataInjectorFactory. Your custom injector needs to get the key-value pairs from somewhere. In Jersey's case, would RequestContext.getProperty and setProperty be appropriate?
I know there has been a lot of question related to this, but i couldn't find one that matches on the scenario that i'm looking at, so here's the question.
Current logging setup: logger coded using Slf4j with Log4j 1.2 bindings. DailyRollingAppender used.
The program: A multi-threading backend Java program processing data from a table and call relevant web services.
A new request came in to have the log file name be based on a certain data, lets call it match_code. With this, whenever a thread is processing say MatchA, then the log file the thread use should be set as MatchA.log
I'd googled for a while and understand that i will need to programmatically configure the Log4j configuration whenever the processes starts, the question is how should i change the log file name settings while not affecting others setting such as the Patterns.
I'm open to switch to Log4j 2.x if it means that can solve my problem, so far have no luck in finding samples for this.
Any suggestion is appreciated. Thank you.
UPDATE on what's tried
Tried using the System.setProperty way to dynamically set the log file. Here's the properties setting:
log4j.appender.file.File=/log/${logfile.name}.log
In main class, added these two lines before anything else:
static{
System.setProperty("logfile.name","output");
}
private static Logger logger = LoggerFactory.getLogger(Engine.class);
Added this right after the process found data to be process:
System.setProperty("logfile.name",match_code+"_output");
where match_code is a value from database such as 'MatchA'
The result is, the main class DID have the log named as output.log. However if i put in data to test, the log will still goes to output.log and there's no new log file created based on the data.
Dunno if I understand your problem: you want your same log message goes to different log file, depending on the data you are processing?
if you use LogBack, you can do it by combination of MDC + SiftingAppender. For example, in your code, you can do:
(in code)
MDC.put("match_code", "MatchA");
logger.debug("whatever message"); // just log it normally
(in logback.xml)
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator>
<key>match_code</key>
<defaultValue>unknown</defaultValue>
</discriminator>
<sift>
<appender name="FILE-${match_code}" class="ch.qos.logback.core.FileAppender">
<file>${match_code}.log</file>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%d [%thread] %level %mdc %logger{35} - %msg%n</pattern>
</layout>
</appender>
</sift>
</appender>
Please note, here I am using LogBack as logging backend. There is no official sifting appender for Log4j 1. I kind of remember something similar in Log4j 2. Wish this can serve as your starting point of searching if you really insist to use Log4j
I am using WebLogic and Log4j for my Struts application. Since Action class is not thread-safe, I assumed Action classes are cached by WebLogic and re-used for every HTTP request.
In this case, if there are multiple clients accessing the same Action class, I assume the events printed on by Log4j will be output by multiple request.
The log information would not be sequential and very difficult to interpret.
How do I resolve such issues?
First, I would like to fix some terms usage in your question. Struts is a MVC framework. Weblogic is a Java EE container. The Action functionality and life cycle does not depend on container. It is the Struts's functionality only.
You are right, since instance of Action is created per request your log will contain a mixture of log messages created by different actions.
The typically used solution is to print thread name into log (log4j supports this configuration), then use grep command on unix or find on windows to filter only relevant messages.
Here is an example of layout configuration that causes log4j to print thread name:
<layout class="org.apache.log4j.EnhancedPatternLayout">
<param name="ConversionPattern" value="%-5p %-23d{ISO8601}{GMT} [%t] %x: %c{1}(%C{1}.%M:%L) - %m%n"/>
</layout>
[%t] does the job.
I'm using Hibernate's c3p0 connection pooling and standard Java 1.4 java.util.logging. Upon startup, my app sets up it's logging properties (including formatter and log levels) in static block. Every time I start my app, I see the following:
2011-04-16 17-43-51 [com.mchange.v2.log.MLog] INFO: {MLog.<clinit>) MLog clients using java 1.4+ standard logging.
2011-04-16 17-43-51 [com.mchange.v2.c3p0.C3P0Registry] INFO: {C3P0Registry.banner) Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true; trace: 10]
2011-04-16 17-43-51 [com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource] INFO: {AbstractPoolBackedDataSource.getPoolManager)
...
I've tried
Logger.getLogger("com.mchange").setLevel(Level.WARNING);
com.mchange.v2.log.MLog.getLogger().setLevel(MLevel.WARNING);
System.setProperty("com.mchange.v2.log.FallbackMLog.DEFAULT_CUTOFF_LEVEL", "WARNING");
but only way to prevent it that I found for now is
Logger.getLogger("").setLevel(Level.WARNING);
which affects everything - not a good side effect. Google didn't help. Could anyone help please?
The way I found is to set the system property
System.setProperty("com.mchange.v2.log.MLog", "com.mchange.v2.log.FallbackMLog");
in addition to
System.setProperty("com.mchange.v2.log.FallbackMLog.DEFAULT_CUTOFF_LEVEL", "WARNING");
I thought, that absence of any other logging system wil make that optional, but it seems, that I was wrong.
P.S.
Damn those wheel-reinvented custom logging implementations, like the one used by c3p0...
The way I found for achieving this
Create in your classpath a file called mchange-log.properties and put into it properties suggested by Frozen Spider.
com.mchange.v2.log.MLog=com.mchange.v2.log.FallbackMLog
com.mchange.v2.log.FallbackMLog.DEFAULT_CUTOFF_LEVEL=WARNING
Thats work fine even when you are not able to set system properties directly.
It appears that c3p0 logging defaults to DEBUG. That can result in a lot of noise.
By adding a line like this to log4j.properties, you are telling the logger not to bother you with c3p0 messages - unless it's something important:
log4j.logger.com.mchange.v2=WARN
Do you not want to see any c3p0 logging?
If so try:
Logger.getLogger("com.mchange.v2.c3p0").setLevel(Level.WARNING);
OR, if you don't even want to see the first line of the log:
Logger.getLogger("com.mchange.v2").setLevel(Level.WARNING);
This is probably really late, but according to the c3p0 project website it is possible to configure the logging inside the mchange-log.properties so that you can capture the information using slf4j or log4j (and thus also with Logback).
The link http://www.mchange.com/projects/c3p0/#configuring_logging provides this information that in your mchange-log.properties file set the property com.mchange.v2.log.MLog to equal com.mchange.v2.log.slf4j.Slf4jMLog then in your logback.xml you can provide a logger like this:
<logger name="com.mchange" level="warn" additivity="false">
<appender-ref ref="c3p0-log" />
</logger>
Note: you will need to create a logback appender called c3p0-log before you can use this exact piece of code.
create a file called log4j.properties in your root classpath
set the following in there,
# Configure the name of the file for the LOGGER appender
log4j.appender.LOGGER=org.apache.log4j.ConsoleAppender
log4j.appender.LOGGER.layout=org.apache.log4j.PatternLayout
log4j.appender.LOGGER.layout.ConversionPattern=%d{MM-dd#HH:mm:ss} %-5p (%13F:%L) %3x - %m%n
log4j.appender.LOGGER.append=false
# this line logs everything from hibernate package at info level, you can refine this to include only some pachages like log4j.logger.org.hibernate.hql etc.,
log4j.logger.org.hibernate=INFO, LOGGER
log4j.logger.org.jboss.cache=INFO, LOGGER
this is a much better way of implementing the logging because if you set the logging strategy programmatically, then the config sometimes might not take effect at all (like in your case).. if you use the log4j.properties file ,the config is applied at application startup & everything works smoothly.
This only happens on older c3p0 version. So it might also be worth checking if you can just update to a newer version.