We Keep Coding

Accessing application code from overridden Tomcat class

I'm currently creating my own JDBCRealm and more specifically, I'm overriding the Authenticate method. The override was no problem, and I got it to work using SHA authentication.
The problems starting arising when I needed to access a different column that isn't covered by Tomcat's own methods, like getPassword(username) and getPrincipal(username).
I have a Hibernate method that retrieves a user based on their username and it looks like:
UsersDTO user = UsersDAO.getUser(username);
The problem is, that because the code is part of the Tomcat library and not the application, calling the method doesn't work. It throws no exceptions and nothing gets logged to catalina.out, it just presents me with a 500 HTTP error.
I have tried numerous things, but I'm at a loss at the moment. Can anyone shed a light on my problem?

You cannot call classes loaded by the app classloader from classes loaded by the server classloader.
You should deploy your jars in a way that all the invocations are done between classes loaded by the same classloader, or from classes loaded by a lower level classloader to classes loaded by a higher level classloader.
Take a look at this. It's related to Weblogic, but the same concepts apply to Tomcat.
http://docs.oracle.com/cd/E24329_01/web.1211/e24368/classloading.htm

Tomcat is implemented with Apache Commons Logging library. So with proper implementation of that logging library in your code, tomcat should be able to log your error properly.
see reference here

Logging same class to different file, if war is different

I need to split 1 batch application to 3 different. Code is almost the same, I have just modified ANT build script, and excluded or included some dependencies for different app. Than I have set different web.xml for each war. Each web.xml defines different spring application context with different beans for different behaviour.
All wars run on one tomcat server. Application used log4j, but now I refactored it to use slf4j instead. Thought I still need to use log4j under slf4j.
The problem I have is that each application log must appear in different log file,
even though class names are the same.
I can't write different log4j.properties file, because administrators placed it in tomcat/lib folder for all applications.
I have tried to place 3 files in tomcat/lib and change configuration file name for each application when initializing servlet, but it changed for all applications at same time.
Only solution I can think of now is to wrap log4j-over-slf4j, create 3 different slf4j log factories, that would append some prefix for each log name. For example, if I have this log:
private final Logger logger = LoggerFactory.getLogger(MainProcessor.class);
Each logging factory would genarate these logging names (with prefixes app1,app2 and app3) :
app1.com.test.MainProcessor
app2.com.test.MainProcessor
app3.com.test.MainProcessor
Is there any better way to deal with this problem ?

Try using a hook method, fire event, etc. so the logging doesn't happen in the class that is shared across the applications, but in some (top) class that is unique per application.
Variation is to statically access some logging class, use a singleton, etc. from the class where the logging should occur, but set context to that logging class on app initialization.

GlassFish: email log handler

We need to send mail from a log handler in GlassFish v3.1.2.2.
We've tried to use smtphandler-0.6 and -0.7 with limited success. We install the jarfile to domain/lib/ext, and configure smtphandler's properties in domain/config/logging.properties. We've tried two ways of satisfying smtphandler's reliance on mail imports: 1) Editing its manifest classpath to point to ../../../../modules/javax.mail.jar, and 2) Putting javax.mail.jar in domain/lib/ext alongside the smtphandler jar. (We prefer the former approach so that the same javamail classes are used throughout the system. But it seems to make no difference, no worky either way.)
With either of these arrangements the behavior is the same:
The handler loads ok as shown by the JVM's verbose:class output.
Sometimes it sends mail for errors and warnings that occur during domain startup (like the expired certificate). Other times it fails as described below before ever sending mail.
It always fails once we've deployed our application and logged some application errors. We can tell that the handler is invoked - that's evidenced by the debugger and some primitive System.out "logging".
The root problem is a NoSuchMethodException: com.sun.mail.smtp.SMTPTransport.[init](Session, URLName). The failure to find the c'tor means the transport object can't be created; that eventually manifests as a NoSuchProtocolException.
We can see that the SMTPTransport class is loaded and that it has the requested c'tor. Our best theory is that class loading is somehow involved, but we've not been able to figure out exactly how. (Yet. We're working that angle now.)
Questions:
Are we deploying the handler to the correct location (domain/lib/ext) ?
Why does it (sometimes) send mail during an (empty, no apps) domain startup, but fail for logs emitted by our application? That's a race condition, surely, but what are those threads doing that occasionally works for a bit then reliably fails?
Are we right to think that all these indications together imply a class loading issue?
We've reproduced these symptoms on GlassFish versions 3.1.2.2 and 4, with JavaMail 1.4.4 and 1.5.
Thanks in advance for any help.

Looks like a bug was reported for this NoSuchMethodException when attempting to install log handler in GlassFish server. This issue was raised with the JavaMail team under Bug K6668 | GH144 - skip unusable Store and Transport classes which is fixed in JavaMail 1.5.3. Upgrading the JavaMail module in glassfish and all other copies deployed should correct the issue. It is also helpful to start glassfish with "glassfish/bin/asadmin start-domain -v" so you can see all bootstrapping messages.
•Are we deploying the handler to the correct location (domain/lib/ext)
Yes. You can deploy to domain/lib/ext but you have to include JavaMail 1.5.3 so you find the correct transport service. Otherwise, you can deploy the jar to the modules dir and add the HK2 metadata to make the smtphandler look like a service. This can be done by including new jar or modifying the existing smtphandler jar.
For GlassFish 3, the file META-INF/inhabitants/default must be added containing the following:
class=smtphandler.SMTPHandler,index=java.util.logging.Handler
For GlassFish 4, the file META-INF/hk2-locator/default must be added containing the following:
[smtphandler.SMTPHandler]
contract={java.util.logging.Handler}
scope=javax.inject.Singleton
The jar then has to be placed in the glassfish/modules folder along with upgrading javax.mail.jar.
Another option is to subclass smtphandler to look like a HK2 log handler service and add a preDestroy method to trigger the email on shutdown. This is described in the Oracle GlassFish Server 3.1 Administration Guide Part I section 7 titled Adding a Custom Logging Handler.
Under GlassFish 4 there are two different ways to locate a handler in the logging.properties. Handlers placed in domain/lib/ext use the standard handlers key in the properties file. Handlers placed in the glassfish/modules as a HK2 Service are loaded using the handlerServices key in the properties file.
#GF3 ext/endorsed or OSGI. GF4 ext/endorsed only.
handlers=smtphandler.SMTPHandler
#GF4 OSGI only, 'handlerServices' should not contain any whitespace characters between handlers.
handlerServices=com.sun.enterprise.server.logging.GFFileHandler,smtphandler.SMTPHandler
•Why does it (sometimes) send mail during an (empty, no apps) domain startup, but fail for logs emitted by our application? That's a race condition, surely, but what are those threads doing that occasionally works for a bit then reliably fails?
Difference in context class loader between messages logged by GF vs your web app. The CCL is used in JavaMail to locate the transport. Patching the 'sendBuffer' method should fix the behavior:
#Override
protected void sendBuffer() {
final Thread thread = Thread.currentThread();
ClassLoader ccl = null;
try {
ccl = thread.getContextClassLoader();
thread.setContextClassLoader(
javax.mail.Transport.class.getClassLoader());
} catch (SecurityException ignore) {
}
try {
super.sendBuffer();
} finally {
try {
thread.getContextClassLoader();
thread.setContextClassLoader(ccl);
} catch (SecurityException ignore) {
}
}
}
•Are we right to think that all these indications together imply a class loading issue?
Yes. The only way to really make this work right is to subclass or patch the smtphandler to look like an HK2 component and modify the CCL.
We need to send mail from a log handler in GlassFish v3.1.2.2.
We've tried to use smtphandler-0.6 and -0.7 with limited success.
Disclaimer: I'm a content developer for MailHandler included with JavaMail project.
An alternative to the smtphandler is the com.sun.mail.util.logging.MailHandler included with the JavaMail reference implementation.
Under GlassFish 4 you have to have glassfish/modules/javax.mail.jar that is JavaMail 1.5.3 or newer. An updated version can be downloaded from the JavaMail API homepage and can be used to replace the version bundled with GlassFish.
Next you have to modify the logging.properties for the domain. Here is a sample configuration you can include to get you started.
#Ensure no whitespace between handler class names.
handlerServices=com.sun.enterprise.server.logging.GFFileHandler,com.sun.mail.util.logging.MailHandler
com.sun.mail.util.logging.MailHandler.subject=com.sun.mail.util.logging.CollectorFormatter
#com.sun.mail.util.logging.CollectorFormatter.format=GlassFish 4.x:{0}{1}{2}{4,choice,-1#|0#|0<... {4,number,integer} more}
#com.sun.mail.util.logging.CompactFormatter.format=[%4$-7.7s] %7$#.140s
com.sun.mail.util.logging.MailHandler.level=WARNING
com.sun.mail.util.logging.MailHandler.filter=com.sun.mail.util.logging.DurationFilter
com.sun.mail.util.logging.MailHandler.pushLevel=WARNING
com.sun.mail.util.logging.MailHandler.mail.smtp.host=some-smtp-host
#com.sun.mail.util.logging.MailHandler.mail.user=some-user
#com.sun.mail.util.logging.MailHandler.authenticator=some-password
com.sun.mail.util.logging.MailHandler.mail.from=app#server.com
#com.sun.mail.util.logging.MailHandler.mail.sender=team#list.com
com.sun.mail.util.logging.MailHandler.mail.to=devs#bugfixers.com
com.sun.mail.util.logging.MailHandler.verify=resolve
com.sun.mail.util.logging.MailHandler.mail.smtp.quitwait=false
com.sun.mail.util.logging.MailHandler.mail.smtps.quitwait=false
com.sun.mail.util.logging.MailHandler.mail.smtp.connectiontimeout=45000
com.sun.mail.util.logging.MailHandler.mail.smtps.connectiontimeout=45000
com.sun.mail.util.logging.MailHandler.mail.smtp.timeout=45000
com.sun.mail.util.logging.MailHandler.mail.smtps.timeout=45000
For GlassFish 3 you have to install JavaMail (javax.mail.jar) under the domain/lib/ext or glassfish/lib/endorsed and test that this doesn't break any of your applications. This classloader configuration also works under GlassFish 4 and allows combining the MailHandler with MemoryHandler which can emulate the behavior of the smtphandler.
Next you have to modify the logging.properties for the domain. You can use the same sample as the GlassFish 4 except that you have to use the standard handlers tag instead of the handlerServices.
handlers=java.util.logging.MemoryHandler
java.util.logging.MemoryHandler.target=com.sun.mail.util.logging.MailHandler
java.util.logging.MemoryHandler.size=512
java.util.logging.MemoryHandler.level=INFO
java.util.logging.MemoryHandler.push=WARNING
com.sun.mail.util.logging.MailHandler.capacity=512
com.sun.mail.util.logging.MailHandler.level=INFO
com.sun.mail.util.logging.MailHandler.pushLevel=WARNING
com.sun.mail.util.logging.MailHandler.filter=com.sun.mail.util.logging.DurationFilter
com.sun.mail.util.logging.DurationFilter.records=512
com.sun.mail.util.logging.DurationFilter.duration=5*60*1000

Log(ger) Variable Declaration

In the majority of cases I see Log instances declared as follows:
public static final Log LOG = LogFactory.getLog(MyClass.class);
I assume this means that the log configuration is loaded when the MyClass is loaded and is therefore set in stone until the MyClass is either reloaded or the JVM restarted?
So, if this assumption is correct what is the best way to ensure changes to the log configuration are picked up as (or as soon after) they happen?

I assume that you are using commons-logging from the LogFactory class? As far as I know, none of the usual logging implementations (Log4J, java.util.logging) allow you to reload a configuration file in a running application (regardless of whether the actuall Loggers are declared as static variables). (EDIT: Peter's answer below proves that I was wrong about this in the case of Log4J)
However, they do allow for the dynamic changing of logging levels (e.g. via an MBean). These level-changes will be picked up by any Logger (including those declared as static variables). If you use java.util.logging you get the MBean for free in the JConsole.
Is it just the changing of levels you care about, or do you wish to provide completely different logging configurations (e.g. files, logger definitions) on the fly?

I guess this depends on the underlying implementation, as pointed out by oxbow_lanes. Generally, it might be difficult to reconfigure your logging subsystem if you are relying on config files that are available via the classpath. To get around this limitation, we do all our config programmatically, and do not rely on only static config files. But I don't know whether your implementation support programmatic reconfiguration.

No, the log configuration is loaded typically when the logging implementation classes are initialized. When your class is (re)loaded, all that happens is that the logging API is called to get a logger (which may or may not be present in any configuration) and stored as a class variable.
To reload your logging configuration, then you typically would have to get the logging implementation to reload.

Depends on the backend.
Logback has a very niftly feature where the reload can be triggered by JMX, i.e. in jvisualvm or jconsole.

Actually, if you are using "java.util.logging" logging directly, then you CAN reload the logger configurations on the fly. There are two methods on the LogManager class for doing this:
public void readConfiguration()
throws IOException, SecurityException
This reloads the default logging properties and reinitializes the logging configuration.
public void readConfiguration(InputStream ins)
throws IOException, SecurityException
This loads the logging properties from a stream (in Property file format) and reinitializes the logging configuration.
See the LogManager javadoc.

Log4j can reload your config file whenever it changes.
see the faq here
Is there a way to get log4j to
automatically reload a configuration
file if it changes?
Yes. Both the DOMConfigurator and the
PropertyConfigurator support automatic
reloading through the
configureAndWatch method. See the API
documentation for more details.
Because the configureAndWatch launches
a separate wathdog thread, and because
there is no way to stop this thread in
log4j 1.2, the configureAndWatch
method is unsafe for use in J2EE
envrironments where applications are
recycled.

Logging Java web applications?

I am planning to implement logging into a web application that I am currently working on but I am struggling with some of the details. What is the best way to go about logging a Java web application?
Specifically;
Where does the configuration file go in .war package file?
Where do people log to, relative or absolute path flat file, or a database?
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
Any other gotchas I should be aware of for web application logging?
Currently I am using Log4J but I imagine that the best practices would apply universally to all logging implementations.
EDIT:
One addition to the questions up top.
Where do you initilize the log
configuration?
In a traditional app, I do this at the entry point;
DOMConfigurator.configureAndWatch("log4j.xml");
What would the web application equivalent be?

I would recommend you to use SLF4J. This is simple logging facade which supports most of popular logging systems (Log4j, commons-logging, Java Logging API and Logback).
Using it, you will able to replace your underline logging system to any other, by simple CLASSPATH update.
The other benefit of SLF4J are parameterized calls, which reduces ugly logging code.
Actually, they recommends to use SLF4J with Logback.
Logback is a successor of Log4J. And it was designed by the same author.

Where does the configuration file go in .war package file?
Root of the classpath.
Where do people log to, relative or absolute path flat file, or a database?
Depends on the need. It's always better to use relative paths. Databases are good if you implement another application which will fetch logs from it and send them using email/sms
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
If you use Console appender, yes, it's going to be logged in your servlet container log file.
Any other gotchas I should be aware of for web application logging?
If you are logging from different threads use logback, it's thread-safe and it exposes parameterized log messages.

Logging to a DB adds another failure point. We had a situation where the prod servers logged to a DB and someone ran an expensive query on that DB that slowed it so much that the prod servers got very, very slow. Logging to a file can cause issues if you run out of space but it seems less likely to slow down the whole app.

I place my configuration on the default package: src/
and log to files using the ${catalina.home} system property:
log4j.appender.???.file=${catalina.home}/logs/system.log

It might be a good idea to place the config file somewhere where an admin can modify it without rebuilding your web app (e.g., so they can turn on detailed logging without waking you up in the middle of the night).
Unfortunately, there's no "official" way for locating externalized resources from a web app (correct me if I'm wrong). The most common way of doing it I've seen is to look through the directories in the classpath.

The excellent paper How to Do Application Logging Right has a bunch of gotchas.
I believe that your other questions have been answered by other people on this page.
I also recommend that you use SLF4J.
One last thing: having speakable representations of objects can save some time.

I recommend to call log API (log4j) via slf4j. Even if you use log4j, web container or depending modules may use different log API such as Java.util.logging or Jakarta commons logging. Slf4j provides bridge modules that redirect them to slf4j API. As a result, all log messages are written by log4j in that case.

put the log4j in the container (server) and create proper appenders per application
relative to server path, but that depends on your needs
we use appenders which log to different files, depends on your needs, e.g. one file for hibernate info/statistics, one for application only, etc.
don't log to much, it slows the application down

Personally I put the log4j.properties in the WEB-INF directory and use an init servlet with the following code :
public class InitServlet extends javax.servlet.http.HttpServlet implements javax.servlet.Servlet {
private static final String LOG4J_FILE = "WEB-INF/log4j.properties";
public InitServlet() {
super();
}
#Override
public void init() throws ServletException {
super.init();
PropertyConfigurator.configure(getServletContext().getRealPath(LOG4J_FILE));
LogFactory.getLog(InitServlet.class).info("LOG 4J configured");
}
}

Where does the configuration file go in .war package file?
At the root of the classpath but... Don't put the configuration file in the war package. You don't want to repackage and redeploy the application if you change the logging configuration, do you ? A better practice would be to put the configuration file somewhere in the classpath outside the war.
Where do people log to, relative or absolute path flat file, or a database?
I usually log to the file system on a separate partition (log files can grow very fast and should never block the application or the operating system if they become too big). I use most of time an absolute path based on the following model: /var/projects/<PROJECT_NAME>/<PRODUCT>/<CLUSTER_NAME>/logs/<INSTANCE_NAME>.log where <PROJECT_NAME> is the project name, <PRODUCT> can be Apache, Tomcat, Weblogic,..., <CLUSTER_NAME> the name of the cluster and <INSTANCE_NAME> the name of the instance inside the cluster. Logging to the file system is faster than in a database. The drawback is that logs aren't centralized if you are using several instances and physical machines. But merging can easily be done with a script.
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
Application server logs are application server logs, not application logs. Don't write to them but set up a logger tool (e.g. log4j) and write to application logs (understand dedicated).
Any other gotchas I should be aware of for web application logging?
If you are using log4j, don't forget to use the isDebugEnabled() before to log:
if(logger.isDebugEnabled()) {
logger.debug("Bla Bla Bla");
}

Where does the configuration file go in .war package file?
Usually, I do not place any logging configuration into the application, rather leaving that to the appserver admins to configure logging server-wide. In the rare cases I want the log4j configuration deployed with a webapp, WEB-INF is the usual path.
Where do people log to, relative or absolute path flat file, or a database?
Again, depends on appserver settings. One common log file for a appserver and rotating on a daily basis is the usual setup. If there are any app-specific needs, the admin may configure a separate logfile for an app (distinguished by package / class names).
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
See above. For tomcat used for development purposes, I'd just look for its logging (log4j) configuration and add app-specific specific there.
Any other gotchas I should be aware of for web application logging?
Performance. Limit the ,log level to a minimum (i.e. WARN or ERROR) once you go live. Use
if (log.isDebugEnabled()) { log.debug("..."); } and alike constructs in your code.

Note that if you just need a bit of logging, the servlet standard specifies that you can get the ServletContext and use the log methods there. That is the generic servlet equivalent of System.out.println.