I have method where I want to add specific logging:
#Slf4j
#Service
public class SomethingService {
public void doSomething(Something data, String comment, Integer limit) {
Long id = saveSomethingToDatabase(data, comment);
boolean sentNotification = doSomething(id);
// ...
// Log what you done.
// Variables that always have important data: data.getName(), id
// Variables that are optional: sentNotification, comment, limit
// (optional means they aren't mandatory, rarely contains essential data, often null, false or empty string).
}
}
I can simply log all:
log.info("Done something '{}' and saved (id {}, sentNotification={}) with comment '{}' and limit {}",
something.getName(), id, sentNotification, comment, limit);
// Done something 'Name of data' and saved (id 23, sentNotification=true) with comment 'Comment about something' and limit 2
But most of the time most of the parameters are irrelevant. With the above I get logs like:
// Done something 'Name of data' and saved (id 23, sentNotification=false) with comment 'null' and limit null
That makes logs hard to read, long and unnecessarily complicated (in most cases other parameters aren't present).
I want to handle all cases with preserving only essential data. Examples:
// Done something 'Name of data' and saved (id 23)
// Done something 'Name of data' and saved (id 23) with comment 'Comment about something'
// Done something 'Name of data' and saved (id 23) with limit 2
// Done something 'Name of data' and saved (id 23) with comment 'Comment about something' and limit 2
// Done something 'Name of data' and saved (id 23, sent notification)
// Done something 'Name of data' and saved (id 23, sent notification) with limit 2
// Done something 'Name of data' and saved (id 23, sent notification) with comment 'Comment about something'
// Done something 'Name of data' and saved (id 23, sent notification) with comment 'Comment about something' and limit 2
I can code it by hand:
String notificationMessage = sentNotification ? ", sent notification" : "";
String commentMessage = comment != null ? String.format(" with comment '%s'", comment) : "";
String limitMessage = "";
if (limit != null) {
limitMessage = String.format("limit %s", limit);
limitMessage = comment != null ? String.format(" and %s", limitMessage) : String.format(" with %s", limitMessage);
}
log.info("Done something '{}' and saved (id {}{}){}{}",
something.getName(), id, notificationMessage, commentMessage, limitMessage);
But it's hard to write, hard to read, complicated and causes errors.
I would like something like specify part of logs.
Example pseudocode:
log.info("Done something '{}' and saved (id {} $notification) $parameters",
something.getName(), id,
$notification: sentNotification ? "sent notification" : "",
$parameters: [comment, limit]);
It should supports optional parameters, replace boolean/condition with given string, supports separating spaces, commas and words with and and.
Maybe are there existing library for this? Or maybe is there at least a simpler way for coding this?
If not, it remains for me nothing else to write my own library for messages to logging. Additionally, this kind of library will provide that all logs would be consistent.
If you don't see a problem with three optional parameters, just imagine there are more (and you can't always pack them into a class - another class layer only for parameter logging cause even more complications).
At the end, I know I can log each action separately. But with this I get many more logs and I won't have the most important information in one place. Other logs are in the debug level, not info.
both of these are possible. You can either:
register a component with the Logger to do the work for you
write a wrapper class for your logger to use
I will demonstrate both and explain why I think the second is the better choice. Let's start with that:
Instead of having the Logger own the knowledge of how to format your specific properties, let your code own this responsibility.
For example, rather than logging each parameter, collect them and define their logging separately. See this code:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class LoggingExample {
private static final Logger LOGGER = LoggerFactory.getLogger(LoggingExample.class);
public static void main(String[] args) {
LogObject o = new LogObject();
LOGGER.info("{}", o);
o.first = "hello";
LOGGER.info("{}", o);
o.second = "World";
LOGGER.info("{}", o);
o.last = "And finally";
LOGGER.info("{}", o);
}
public static class LogObject {
String first;
String second;
String last;
#Override
public String toString() {
StringBuffer buffer = new StringBuffer();
buffer.append("Log Object: ");
if (first != null) {
buffer.append("First: " + first + " ");
}
if (second != null) {
buffer.append("Second: " + second + " ");
}
if (last != null) {
buffer.append("Second: " + last + " ");
}
return buffer.toString();
}
}
}
We define LogObject as a container and this container implements toString. All Loggers will call toString() on their objects, that is how they figure out what they should print (unless special formatters applied etc).
With this, the log statements print:
11:04:12.465 [main] INFO LoggingExample - Log Object:
11:04:12.467 [main] INFO LoggingExample - Log Object: First: hello
11:04:12.467 [main] INFO LoggingExample - Log Object: First: hello Second: World
11:04:12.467 [main] INFO LoggingExample - Log Object: First: hello Second: World Second: And finally
Advantages:
this works with any Logger. You won't have to implement specifics depending on what you want to use
the knowledge is encapsulated in 1 object that can be easily tested. This should mitigate the error prone formatting problem you stated.
no need for a complex formatter library or implementation
It will make the logging look much nicer and compact in the end. log.info("{}", object);
Disadvantage:
You are required to write the Bean.
Now the same can be achieved using for example a custom Layout. I am using logback, so this is an example for logback.
We may define a Layout that owns the knowledge of what to do with your custom formatting instructions.
import org.slf4j.LoggerFactory;
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.LoggerContext;
import ch.qos.logback.classic.encoder.PatternLayoutEncoder;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.ConsoleAppender;
import ch.qos.logback.core.LayoutBase;
public class LoggingExample2 {
private static final Logger CUSTOM_LOGGER = createLoggerFor("test");
public static void main(String[] args) {
LogObject o = new LogObject();
CUSTOM_LOGGER.info("{}", o);
o.first = "hello";
CUSTOM_LOGGER.info("{}", o);
o.second = "World";
CUSTOM_LOGGER.info("{}", o);
o.last = "And finally";
CUSTOM_LOGGER.info("{}", o);
}
public static class LogObject {
String first;
String second;
String last;
#Override
public String toString() {
StringBuffer buffer = new StringBuffer();
buffer.append("Log Object: ");
if (first != null) {
buffer.append("First: " + first + " ");
}
if (second != null) {
buffer.append("Second: " + second + " ");
}
if (last != null) {
buffer.append("Second: " + last + " ");
}
return buffer.toString();
}
}
public static class ModifyLogLayout extends LayoutBase<ILoggingEvent> {
#Override
public String doLayout(ILoggingEvent event) {
String formattedMessage = event.getFormattedMessage() + "\n";
Object[] args = event.getArgumentArray();
return String.format(formattedMessage, args);
}
}
private static Logger createLoggerFor(String string) {
LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory();
PatternLayoutEncoder ple = new PatternLayoutEncoder();
ple.setPattern("%date %level [%thread] %logger{10} [%file:%line] %msg%n");
ple.setContext(lc);
ple.start();
ConsoleAppender<ILoggingEvent> consoleAppender = new ConsoleAppender<ILoggingEvent>();
consoleAppender.setEncoder(ple);
consoleAppender.setLayout(new ModifyLogLayout());
consoleAppender.setContext(lc);
consoleAppender.start();
Logger logger = (Logger) LoggerFactory.getLogger(string);
logger.addAppender(consoleAppender);
logger.setLevel(Level.DEBUG);
logger.setAdditive(false); /* set to true if root should log too */
return logger;
}
}
I borrowed the Logger instatiation from: Programmatically configure LogBack appender
Note that I have not found a library that can parse the complex expressions that you have listed. I think you may have to write your own implementation.
In my example, i only illustrate how to intercept and modify the message based on the arguments.
Why I would not recommend this unless it is really needed:
the implementation is specific to logback
writing correct formatting is hard ... it will produce more errors than creating a custom object to format
It is harder to test because you literally have unlimited objects that may pass through this (and formatting). Your code must be resilient to this now, and in the future since any developer may add the weirdest things at any time.
The last (unasked) answer:
Why don't you use a json encoder? And then use something like logstash to aggregate (or cloudlwatch, or anything else).
This should solve all your problems.
This is what I have done in the past:
Define 1 bean that you like to log "differently". I call it metadata. This bean can be i.e.
public class MetaHolder {
// map holding key/values
}
This basically just stores all your variables with a key. It allows you to effectively search on these keys, sink them into databases, etc. etc.
In your log, you simply do:
var meta = // create class
meta.put("comment", comment);
// put other properties here
log.info("formatted string", formattedArguments, meta); // meta is always the last arg
In the Layout this can then be converted quite nicely. Because you are no longer logging "human language", there are no "withs" and "in" to replace. Your log will simply be:
{
"time" : "...",
"message" : "...",
"meta" : {
"comment" : "this is a comment"
// no other variables set, so this was it
}
}
And one last (last) one in just pure java, if you wanted that. You could write:
public static void main(String[] args) {
String comment = null;
String limit = "test";
String id = "id";
LOGGER.info(
"{} {} {}",
Optional.ofNullable(comment).map(s -> "The comment " + s).orElse(""),
Optional.ofNullable(limit).map(s -> "The Limit " + s).orElse(""),
Optional.ofNullable(id).map(s -> "The id " + s).orElse(""));
}
Which effectively moves the conditional logic you want in your formatting into Java's Optional.
I find this also is hard to read and test and would still recommend the first solution
Related
I am trying to implement a unit test using jOOQ's mocking tools but failing to understang how to create custom records and put some data on them.
This is my MockDataProvider:
private static class MyProvider implements MockDataProvider {
#Override
public MockResult[] execute(MockExecuteContext ctx) throws SQLException {
log.trace("SQL = {}", ctx.sql());
DSLContext create = DSL.using(SQLDialect.MARIADB);
MockResult[] mockResults = new MockResult[1];
if (ctx.sql().startsWith("select `gc`.`article`.`id`, `gc`.`familia`.`iva` from")) {
Record2<Integer, Byte> record2 = create.newRecord(ARTICLE.ID, FAMILIA.IVA);
log.debug("record2 = " + record2);
log.debug("record2.value1()={}" + record2.value1());
Record2<Integer, Byte> values = record2.values(1, (byte) 10);
log.debug("record2.value1()={}" + record2.value1());
log.debug("values = " + values);
mockResults[0] = new MockResult(record2);
}
return mockResults;
}
}
And this is the output of the different log statements:
12:21:05.670 [main] DEBUG net.sargue.gc.test.TestArticleDAO - record2 = +------+------+
| id| iva|
+------+------+
|{null}|{null}|
+------+------+
12:21:05.671 [main] DEBUG net.sargue.gc.test.TestArticleDAO - record2.value1()={}null
12:21:05.671 [main] DEBUG net.sargue.gc.test.TestArticleDAO - record2.value1()={}null
12:21:05.671 [main] DEBUG net.sargue.gc.test.TestArticleDAO - values = null
So it seems to ignore setting values (third log output) and fully returns null from that method. Not sure if it's a bug or I am doing something wrong.
By the way, setting values one by one seems to work ok.
record2.value1(1)
Which it's what I am doing now as a workaround.
There was an unfortunate bug #5042, which was fixed in jOOQ 3.7.3 (and will be fixed in 3.6.5 and 3.5.5).
That values() method was only added the API, not to the implementation, which is empty ;)
If upgrading is not yet a possibility, you can still use individual setters (as you did) or the unsafe Record.from() to which you can pass an array:
// Using from()
record2.from(new Object[] { 1, (byte) 10 });
// Or simpler, using fromArray()
record2.fromArray(1, (byte) 10);
I use following pattern [%file:%line] %msg%n to output file + number to my log.
I as well use a simple wrapper class, that I call L.java. Now it does not make sense to output [L.java:74] Message... to my log. Instead, I would like to output the calling file name and line number...
Is that somehow possible with log4j?
The PatternLayout are slighty different between log4j 1.x and 2.x, and I do not know which version are you using, but I think you can't achieve this by configuration in neither versions.
You can achieve that programmatically (but this is going to affect your performance), I think in your L.java method you will have to use a method like:
private Logger logger = getYourLoggerAsYouAreCurrentlyDoing();
public enum LogLevel { INFO,DEBUG, ERROR, ETC }
void log(String msg, LogLevel level) {
StackTraceElement[] stackTraceElements = Thread.currentThread().getStackTrace();
String callerClass = stackTraceElements[1].getClassName();
String callerLine = "" + stackTraceElements[1].getLineNumber();
String msg = callerClass + ":" + callerLine + "-" + msg;
switch(LogLevel) {
case INFO: logger.info(msg); break;
case DEBUG: logger.debug(msg); break;
//etc.
}
}
And in case another method with the Throwable argument to log stacktraces.
By default, log4j will print the class name as the prefix of log. Now my case is that I have multiple instances of class A in one JVM, and there's one log in this class. I'd like the log as following to allow me know which instance is printing this line of log ? How can I achieve this ?
2014-09-09 13:07:08,512 INFO com.myexample.A(id1)
2014-09-09 13:07:08,514 INFO com.myexample.A(id2)
I think you can not do it by settings in log4j.properties file. See possible variants here.
But you can solve the problem like this:
class A {
Logger LOGGER;
A(String id) {
LOGGER = Logger.getLogger(getClass() + "(" + id + ")");
}
void myMethod() {
LOGGER.info("Hello!");
}
}
Easiest way is to add desired ID to your log message:
if (logger.isDebugEnabled()) {
int id = this.hashCode(); // Invoke ID getter method
logger.debug( String.format("[ %d ] My log message with ID", id) );
}
I am making a simple phishing scanner tool for a university project. One of my detection methods includes checking if the DNS within the email are valid, and I also want to check their age. This is example code of how I check if they are existing:
import javax.naming.NamingException;
import javax.naming.directory.Attribute;
import javax.naming.directory.Attributes;
import javax.naming.directory.DirContext;
import javax.naming.directory.InitialDirContext;
import java.util.Hashtable;
public class DNSExample {
static int doLookup( String hostName ) throws NamingException {
Hashtable env = new Hashtable();
env.put("java.naming.factory.initial",
"com.sun.jndi.dns.DnsContextFactory");
DirContext ictx = new InitialDirContext( env );
Attributes attrs =
ictx.getAttributes( hostName, new String[] { "MX" });
Attribute attr = attrs.get( "MX" );
if( attr == null ) return( 0 );
return( attr.size() );
}
public static void main( String args[] ) {
String [] array = {"google.com","dsad33114sssaxzx.com"} ;
for( int i = 0; i < array.length; i++ ) {
try {
System.out.println( array[i] + " has " +
doLookup( array[i] ) + " mail servers" );
}
catch( Exception e ) {
System.out.println(array[i] + " : " + e.getMessage());
}
}
}
}
How would I need to modify the above code to include a check of age
for servers that exist?
I think you've chosen a problem that cannot be solved in the general case ... using current generation internet standards:
The information you need cannot be obtained from DNS itself.
In some cases information about DNS registrations can be obtained from WHOIS. However, the information returned by WHOIS servers is not standardised:
There is no standard information model.
There is no standard format.
There are no guarantees as to the accuracy of the information.
It is not even clear if "age of server" is going to be available. (For instance, the closest that APNIC's WHOIS provides to that is the last modification timestamp for the DNS record. And that is NOT a good proxy for server age.)
There is a set of RFC's that define something called CRISP, but as far as I can make out the purpose of that standard is for registrar to registrar exchange of information. (I couldn't find any public-facing services based on CRISP.)
There is also an IETF working group called WEIRDS which I think is intended to define a web-enabled replacement for WHOIS. (Don't confuse WEIRDS with the IETF WEIRD WG!) But that was formed very recently, and it is too soon to make any predictions of the outcome. (Or how long it will take for the NICs to implement any specs that come out of the WG.)
Summary: your chances of implementing something in this space that really works are currently low. Probably the best you can hope to achieve is something based on screen-scraping one or two WHOIS services.
This might change in a few years, but that is of no help for your current project.
It seems based on your description and comments above you are trying to gather whois information.
download APIs from http://commons.apache.org/proper/commons-net/
change the nameToQuery below and run it.
public class WhoisIt {
public static final String WHOIS_SERVER = "whois.internic.net";
public static final int WHOIS_PORT = 43;
public static void main(String[] args) throws Exception {
String nameToQuery = "avajava.com";
WhoisClient whoisClient = new WhoisClient();
whoisClient.connect(WHOIS_SERVER, WHOIS_PORT);
String results = whoisClient.query(nameToQuery);
System.out.println(results);
}
}
good luck
I'd like to remove the timestamp from GWT logging output on the console.
What's the simplest way to do this? Ideally, in the .gwt.xml configuration would be great.
Here is an example output currently with the timestamp:
Wed Mar 21 08:23:57 EDT 2012 Job
FINE: Job: 'updater': end
EDIT: I am only interested in the client side.
This logging capability is not really configurable. You need to write your own formatter:
call this at the beginning of onModuleLoad():
Handler[] handlers = Logger.getLogger("").getHandlers();
for(Handler h : handlers){
h.setFormatter(new MyCustomLogFormatter());
}
And here is an example of a formatter:
public class MyCustomLogFormatter extends TextLogFormatter{
private static DateTimeFormat timeFormat = DateTimeFormat.getFormat("HH:mm:ss.SSS");
public MyCustomLogFormatter() {
super(true);
}
#Override
public String format(LogRecord event) {
StringBuilder message = new StringBuilder();
message.append(getRecordInfo(event, " "));
message.append(event.getMessage());
message.append(getStackTraceAsString(event.getThrown(), "\n", "\t"));
return message.toString();
}
#Override
protected String getRecordInfo(LogRecord event, String newline) {
Date date = new Date(event.getMillis());
StringBuilder s = new StringBuilder();
s.append(timeFormat.format(date));
s.append(" GWT ");
s.append(event.getLevel().getName());
String loggerName = event.getLoggerName();
String[] split = loggerName.split("\\.");
s.append(" ");
s.append(split[split.length-1]);
s.append(newline);
s.append(": ");
return s.toString();
}
}
More: http://code.google.com/webtoolkit/doc/latest/DevGuideLogging.html
The accepted answer shows how to customize GWT log messages. The example is longer than necessary, though. In case anyone wants to just remove the timestamp (the original question), here is a shorter snippet:
Handler[] handlers = Logger.getLogger("").getHandlers();
for (Handler h : handlers) {
h.setFormatter(new TextLogFormatter(false) {
#Override
public String format(LogRecord event) {
return event.getLoggerName() + ": " +
event.getLevel().getName() + ": " +
event.getMessage();
}
});
}
Watch out for the "gotcha" of doing this in the constructor of a singleton dependency-injected class---you want to make sure GWT has had the chance to actually add handlers to the logger first.
It uses the same pattern config as log4j, see here: http://code.google.com/p/gwt-log/wiki/GettingStarted#Control_the_format_of_your_log_messages
In the gwt-log wiki it says:
Server side logging automatically detects Apache log4j, falling back
to JDK 1.4 logging
As there is a "FINE" log level in your post, it should be the second case. Either include Log4J in your classpath or check "logging.properties" from your current jdr/jre-conf directory.