I'd like to remove the timestamp from GWT logging output on the console.
What's the simplest way to do this? Ideally, in the .gwt.xml configuration would be great.
Here is an example output currently with the timestamp:
Wed Mar 21 08:23:57 EDT 2012 Job
FINE: Job: 'updater': end
EDIT: I am only interested in the client side.
This logging capability is not really configurable. You need to write your own formatter:
call this at the beginning of onModuleLoad():
Handler[] handlers = Logger.getLogger("").getHandlers();
for(Handler h : handlers){
h.setFormatter(new MyCustomLogFormatter());
}
And here is an example of a formatter:
public class MyCustomLogFormatter extends TextLogFormatter{
private static DateTimeFormat timeFormat = DateTimeFormat.getFormat("HH:mm:ss.SSS");
public MyCustomLogFormatter() {
super(true);
}
#Override
public String format(LogRecord event) {
StringBuilder message = new StringBuilder();
message.append(getRecordInfo(event, " "));
message.append(event.getMessage());
message.append(getStackTraceAsString(event.getThrown(), "\n", "\t"));
return message.toString();
}
#Override
protected String getRecordInfo(LogRecord event, String newline) {
Date date = new Date(event.getMillis());
StringBuilder s = new StringBuilder();
s.append(timeFormat.format(date));
s.append(" GWT ");
s.append(event.getLevel().getName());
String loggerName = event.getLoggerName();
String[] split = loggerName.split("\\.");
s.append(" ");
s.append(split[split.length-1]);
s.append(newline);
s.append(": ");
return s.toString();
}
}
More: http://code.google.com/webtoolkit/doc/latest/DevGuideLogging.html
The accepted answer shows how to customize GWT log messages. The example is longer than necessary, though. In case anyone wants to just remove the timestamp (the original question), here is a shorter snippet:
Handler[] handlers = Logger.getLogger("").getHandlers();
for (Handler h : handlers) {
h.setFormatter(new TextLogFormatter(false) {
#Override
public String format(LogRecord event) {
return event.getLoggerName() + ": " +
event.getLevel().getName() + ": " +
event.getMessage();
}
});
}
Watch out for the "gotcha" of doing this in the constructor of a singleton dependency-injected class---you want to make sure GWT has had the chance to actually add handlers to the logger first.
It uses the same pattern config as log4j, see here: http://code.google.com/p/gwt-log/wiki/GettingStarted#Control_the_format_of_your_log_messages
In the gwt-log wiki it says:
Server side logging automatically detects Apache log4j, falling back
to JDK 1.4 logging
As there is a "FINE" log level in your post, it should be the second case. Either include Log4J in your classpath or check "logging.properties" from your current jdr/jre-conf directory.
Related
I have method where I want to add specific logging:
#Slf4j
#Service
public class SomethingService {
public void doSomething(Something data, String comment, Integer limit) {
Long id = saveSomethingToDatabase(data, comment);
boolean sentNotification = doSomething(id);
// ...
// Log what you done.
// Variables that always have important data: data.getName(), id
// Variables that are optional: sentNotification, comment, limit
// (optional means they aren't mandatory, rarely contains essential data, often null, false or empty string).
}
}
I can simply log all:
log.info("Done something '{}' and saved (id {}, sentNotification={}) with comment '{}' and limit {}",
something.getName(), id, sentNotification, comment, limit);
// Done something 'Name of data' and saved (id 23, sentNotification=true) with comment 'Comment about something' and limit 2
But most of the time most of the parameters are irrelevant. With the above I get logs like:
// Done something 'Name of data' and saved (id 23, sentNotification=false) with comment 'null' and limit null
That makes logs hard to read, long and unnecessarily complicated (in most cases other parameters aren't present).
I want to handle all cases with preserving only essential data. Examples:
// Done something 'Name of data' and saved (id 23)
// Done something 'Name of data' and saved (id 23) with comment 'Comment about something'
// Done something 'Name of data' and saved (id 23) with limit 2
// Done something 'Name of data' and saved (id 23) with comment 'Comment about something' and limit 2
// Done something 'Name of data' and saved (id 23, sent notification)
// Done something 'Name of data' and saved (id 23, sent notification) with limit 2
// Done something 'Name of data' and saved (id 23, sent notification) with comment 'Comment about something'
// Done something 'Name of data' and saved (id 23, sent notification) with comment 'Comment about something' and limit 2
I can code it by hand:
String notificationMessage = sentNotification ? ", sent notification" : "";
String commentMessage = comment != null ? String.format(" with comment '%s'", comment) : "";
String limitMessage = "";
if (limit != null) {
limitMessage = String.format("limit %s", limit);
limitMessage = comment != null ? String.format(" and %s", limitMessage) : String.format(" with %s", limitMessage);
}
log.info("Done something '{}' and saved (id {}{}){}{}",
something.getName(), id, notificationMessage, commentMessage, limitMessage);
But it's hard to write, hard to read, complicated and causes errors.
I would like something like specify part of logs.
Example pseudocode:
log.info("Done something '{}' and saved (id {} $notification) $parameters",
something.getName(), id,
$notification: sentNotification ? "sent notification" : "",
$parameters: [comment, limit]);
It should supports optional parameters, replace boolean/condition with given string, supports separating spaces, commas and words with and and.
Maybe are there existing library for this? Or maybe is there at least a simpler way for coding this?
If not, it remains for me nothing else to write my own library for messages to logging. Additionally, this kind of library will provide that all logs would be consistent.
If you don't see a problem with three optional parameters, just imagine there are more (and you can't always pack them into a class - another class layer only for parameter logging cause even more complications).
At the end, I know I can log each action separately. But with this I get many more logs and I won't have the most important information in one place. Other logs are in the debug level, not info.
both of these are possible. You can either:
register a component with the Logger to do the work for you
write a wrapper class for your logger to use
I will demonstrate both and explain why I think the second is the better choice. Let's start with that:
Instead of having the Logger own the knowledge of how to format your specific properties, let your code own this responsibility.
For example, rather than logging each parameter, collect them and define their logging separately. See this code:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class LoggingExample {
private static final Logger LOGGER = LoggerFactory.getLogger(LoggingExample.class);
public static void main(String[] args) {
LogObject o = new LogObject();
LOGGER.info("{}", o);
o.first = "hello";
LOGGER.info("{}", o);
o.second = "World";
LOGGER.info("{}", o);
o.last = "And finally";
LOGGER.info("{}", o);
}
public static class LogObject {
String first;
String second;
String last;
#Override
public String toString() {
StringBuffer buffer = new StringBuffer();
buffer.append("Log Object: ");
if (first != null) {
buffer.append("First: " + first + " ");
}
if (second != null) {
buffer.append("Second: " + second + " ");
}
if (last != null) {
buffer.append("Second: " + last + " ");
}
return buffer.toString();
}
}
}
We define LogObject as a container and this container implements toString. All Loggers will call toString() on their objects, that is how they figure out what they should print (unless special formatters applied etc).
With this, the log statements print:
11:04:12.465 [main] INFO LoggingExample - Log Object:
11:04:12.467 [main] INFO LoggingExample - Log Object: First: hello
11:04:12.467 [main] INFO LoggingExample - Log Object: First: hello Second: World
11:04:12.467 [main] INFO LoggingExample - Log Object: First: hello Second: World Second: And finally
Advantages:
this works with any Logger. You won't have to implement specifics depending on what you want to use
the knowledge is encapsulated in 1 object that can be easily tested. This should mitigate the error prone formatting problem you stated.
no need for a complex formatter library or implementation
It will make the logging look much nicer and compact in the end. log.info("{}", object);
Disadvantage:
You are required to write the Bean.
Now the same can be achieved using for example a custom Layout. I am using logback, so this is an example for logback.
We may define a Layout that owns the knowledge of what to do with your custom formatting instructions.
import org.slf4j.LoggerFactory;
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.LoggerContext;
import ch.qos.logback.classic.encoder.PatternLayoutEncoder;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.ConsoleAppender;
import ch.qos.logback.core.LayoutBase;
public class LoggingExample2 {
private static final Logger CUSTOM_LOGGER = createLoggerFor("test");
public static void main(String[] args) {
LogObject o = new LogObject();
CUSTOM_LOGGER.info("{}", o);
o.first = "hello";
CUSTOM_LOGGER.info("{}", o);
o.second = "World";
CUSTOM_LOGGER.info("{}", o);
o.last = "And finally";
CUSTOM_LOGGER.info("{}", o);
}
public static class LogObject {
String first;
String second;
String last;
#Override
public String toString() {
StringBuffer buffer = new StringBuffer();
buffer.append("Log Object: ");
if (first != null) {
buffer.append("First: " + first + " ");
}
if (second != null) {
buffer.append("Second: " + second + " ");
}
if (last != null) {
buffer.append("Second: " + last + " ");
}
return buffer.toString();
}
}
public static class ModifyLogLayout extends LayoutBase<ILoggingEvent> {
#Override
public String doLayout(ILoggingEvent event) {
String formattedMessage = event.getFormattedMessage() + "\n";
Object[] args = event.getArgumentArray();
return String.format(formattedMessage, args);
}
}
private static Logger createLoggerFor(String string) {
LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory();
PatternLayoutEncoder ple = new PatternLayoutEncoder();
ple.setPattern("%date %level [%thread] %logger{10} [%file:%line] %msg%n");
ple.setContext(lc);
ple.start();
ConsoleAppender<ILoggingEvent> consoleAppender = new ConsoleAppender<ILoggingEvent>();
consoleAppender.setEncoder(ple);
consoleAppender.setLayout(new ModifyLogLayout());
consoleAppender.setContext(lc);
consoleAppender.start();
Logger logger = (Logger) LoggerFactory.getLogger(string);
logger.addAppender(consoleAppender);
logger.setLevel(Level.DEBUG);
logger.setAdditive(false); /* set to true if root should log too */
return logger;
}
}
I borrowed the Logger instatiation from: Programmatically configure LogBack appender
Note that I have not found a library that can parse the complex expressions that you have listed. I think you may have to write your own implementation.
In my example, i only illustrate how to intercept and modify the message based on the arguments.
Why I would not recommend this unless it is really needed:
the implementation is specific to logback
writing correct formatting is hard ... it will produce more errors than creating a custom object to format
It is harder to test because you literally have unlimited objects that may pass through this (and formatting). Your code must be resilient to this now, and in the future since any developer may add the weirdest things at any time.
The last (unasked) answer:
Why don't you use a json encoder? And then use something like logstash to aggregate (or cloudlwatch, or anything else).
This should solve all your problems.
This is what I have done in the past:
Define 1 bean that you like to log "differently". I call it metadata. This bean can be i.e.
public class MetaHolder {
// map holding key/values
}
This basically just stores all your variables with a key. It allows you to effectively search on these keys, sink them into databases, etc. etc.
In your log, you simply do:
var meta = // create class
meta.put("comment", comment);
// put other properties here
log.info("formatted string", formattedArguments, meta); // meta is always the last arg
In the Layout this can then be converted quite nicely. Because you are no longer logging "human language", there are no "withs" and "in" to replace. Your log will simply be:
{
"time" : "...",
"message" : "...",
"meta" : {
"comment" : "this is a comment"
// no other variables set, so this was it
}
}
And one last (last) one in just pure java, if you wanted that. You could write:
public static void main(String[] args) {
String comment = null;
String limit = "test";
String id = "id";
LOGGER.info(
"{} {} {}",
Optional.ofNullable(comment).map(s -> "The comment " + s).orElse(""),
Optional.ofNullable(limit).map(s -> "The Limit " + s).orElse(""),
Optional.ofNullable(id).map(s -> "The id " + s).orElse(""));
}
Which effectively moves the conditional logic you want in your formatting into Java's Optional.
I find this also is hard to read and test and would still recommend the first solution
This is the thing: I want to improve the Logging system in Our Apps.
We have multiple services Running on Cloud (irrelevant).
Those services each one Log everything in their Logs, for example:
public class class1 {
protected Logger logger =
Logger.getLogger(Service1 .class.getName());
public method1 (){
...
logger.info("....."); //It creates a new String
...
logger.debug("..."); //It creates a new String
.
.
.
}
public method2 (){
...
logger.info("....."); //It creates a new String
...
logger.debug("..."); //It creates a new String
.
.
.
}
}
I want to improove this code and standardize the Logging System, but I'm not a Java Expert and I need help to know what's the best approach for this?
I got an Idea, here I go:
I'm thinking to create a Utility Class that manage all Logging Messaging (I think this approach would be with StringBuffer for synchronize) or create an StringBuilder on each method and reuse it, something like this:
public class class1 {
protected Logger logger =
Logger.getLogger(Service1 .class.getName());
public method1 (){
StringBuilder msg = new StringBuilder();
msg.append(" ...").append(" xx ").append(" yy "); //new message
logger.info(msg);
...
msg.setLength(0); //Reset
msg.append(" ...").append(" xx ").append(" yy "); //new message
logger.debug("...");
.
.
.
}
public method2 (){
StringBuilder msg = new StringBuilder();
msg.append(" ...").append(" xx ").append(" yy "); //new message
reusing builder
logger.info(msg);
...
msg.setLength(0); //Reset
msg.append(" ...").append(" xx ").append(" yy "); //new message
reusing builder
logger.debug(msg);
.
.
.
}
}
What do you think?
PD: We are using Log4j and my english is not perfect. This approach is to avoid Sonar fails for doing this:
logger.debug("..." + " " + someVariable + " " + otherVariable);
I hope you understand to me =)
If you are able to use log4j 2.x, you can let the library take care of it by using parameters:
logger.debug("... {} {}", someVariable, otherVariable);
See Logger javadocs for more info.
Note that even if you have a dependency bound to 1.x, the log4j team explicitly changed the package structure for 2.x so as to avoid collisions. You can use both together freely without fear of compatibility issues (just make sure that you are importing the correct Logger class).
I have such log4j.properties file:
#Wed Jan 18 12:55:30 EET 2017
log4j.rootLogger=ERROR, stdout, gui, clientFile
log4j.logger.app=DEBUG
...
And when I start my application first line with timestamp (#Wed Jan 18 12:55:30 EET 2017) is always changing. It causes some problems with Git commits (I can not add this file to .gitignore).
Found what is adding the timestamp: this method is calling in app linkedProperties.store(fileOutputStream, null); The implementation of store() method is from java.util.Properties.
package java.util;
...
public class Properties extends Hashtable<Object,Object> {
...
public void store(OutputStream out, String comments)
throws IOException
{
store0(new BufferedWriter(new OutputStreamWriter(out, "8859_1")),
comments,
true);
}
private void store0(BufferedWriter bw, String comments, boolean escUnicode)
throws IOException
{
if (comments != null) {
writeComments(bw, comments);
}
bw.write("#" + new Date().toString());
bw.newLine();
synchronized (this) {
for (Enumeration<?> e = keys(); e.hasMoreElements();) {
String key = (String)e.nextElement();
String val = (String)get(key);
key = saveConvert(key, true, escUnicode);
/* No need to escape embedded and trailing spaces for value, hence
* pass false to flag.
*/
val = saveConvert(val, false, escUnicode);
bw.write(key + "=" + val);
bw.newLine();
}
}
bw.flush();
}
...
How can I avoid this bw.write("#" + new Date().toString());? Is there something similar to java.util.Properties?
Edit: This answer is now laregely redundant given the OP's edits, following my suggestion to find what was adding the timestamp to the file. However I'll keep it here as it may help someone, perhaps.
Firstly, it's not really possible to instruct Git to ignore individual lines in a file.
My first recommendation would be to find what is adding the timestamp to the file and stop it.
The only thing that comes to mind that could help you in Git specifically is removing the file from Gits working tree.
git update-index --skip-worktree <file>
This will instruct Git that a changed version of this file shouldn't be committed and so will not include it in its working tree, but will still keep the tracked copy in the repository. Look here for official docs
Obivously, this won't work if you require developers to regularly update/commit this file.
I have just overrided public void store(OutputStream out, String comments) (removed bw.write("#" + new Date().toString())). For more information about this problem you can use this link (it fully dublicates my issue): Properties.store() - suppress timestamp comment .
I'm testing a web crawler with JSoup. The issue comes when I test the crawler on a regular PC, and works as expected, then I export this web crawler as a jar to work in a server in a cron job. This where the things go wrong.
The code is the same, no changes. The data I'm trying to extract is different comments from the users of how they rate a service, the problem is that the web crawler behaves differently when it's executed in the server, for example: the comments are duplicated, something that doesn't happened when I'm testing the program locally.
Also the web crawler differentiates what language the comments are written (I take that info from the URL, .de for German, .es for Spanish, etc). This info get mixed for example, a comment in Spanish is classified as Portuguese one.
Again I repeat the logic behind the crawler is correct, I tested many times with different input.
What could be the problem behind these issues?
Additional notes:
No exceptions/crashes.
I'm using jsoup 1.9.2.
This is how I get the data from the website:
Document doc = Jsoup.connect(link).userAgent(FakeAgentBooking.getAgent()).timeout(60 * 4000).get();
I already tried to use a proxy just in case the server was banned.
System.getProperties().put("https.proxyHost", "PROXY");
System.getProperties().put("https.proxyPort", "PORT");
System.getProperties().put("https.proxyUser", "USER");
System.getProperties().put("https.proxyPassword", "PASSWORD");
This is the code of the cron job:
#Crawler(name = "Booking comments", nameType = "BOOKING_COMMENTS", sentimetal = true, cron = "${cron.booking.comments}")
public class BookingCommentsJob extends MotherCrawler {
private static final org.slf4j.Logger logger = LoggerFactory.getLogger(BookingCommentsJob.class);
#Value("${full.booking.comments}")
private String full;
#Autowired
private ComentariosCDMXRepository comentariosCDMXRepository;
#Override
#PostConstruct
public void init() {
setInfo(this.getClass().getAnnotation(Crawler.class));
}
#Override
public void exec(int num) {
// <DEBUG>
String startTime = time.format(new Date());
// </DEBUG>
Set<CrawQuery> li = queryManager.getMeQueries(type, num, threadnum);
Gson gson = new GsonBuilder().create();
for (CrawQuery s : li) {
String query = s.getQuery().get("query");
try {
//the crawling begins here-->
String result = BookingComentarios.crawlBookingComentarios(query, Boolean.parseBoolean(full));
//get the result from json to a standarized class
ComentarioCDMX[] myComments = gson.fromJson(result, ComentarioCDMX[].class);
for (ComentarioCDMX myComment : myComments) {
//evaluates if the comment is positive, neutral or negative.
Integer sentiment = sentimentAnalysis.classifyVector(myComment.getComment());
myComment.setSentiment(sentiment);
myComment.setQuery(query);
/* <Analisis de sentimiento /> */
comentariosCDMXRepository.save(myComment);
}
s.setStatus(true);
} catch (Exception e) {
logger.error(query, e);
s.setStatus(false);
mailSend.add(e);
} finally {
s.setLastUse(new Date());
//Saves data to Solr
crawQueryDao.save(s);
}
}
update();
// <DEBUG>
String endTime = time.format(new Date());
logger.info(name + " " + num + " > Inicio: " + startTime + ", Fin: " + endTime);
// </DEBUG>
}
#Scheduled(cron = "${cron.booking.comments}")
public void curro0() throws InterruptedException {
exec(0);
}
}
and this is when the code should be executed:
cron.booking.comments=00 30 02 * * *
Additional notes:
The test PC OS is Windows 7 and the server OS is linux Debian 3.16.7. and tghe java version in the test PC is 1.7 oracle JDK and on the server is 1.8.0 JRE.
I use following pattern [%file:%line] %msg%n to output file + number to my log.
I as well use a simple wrapper class, that I call L.java. Now it does not make sense to output [L.java:74] Message... to my log. Instead, I would like to output the calling file name and line number...
Is that somehow possible with log4j?
The PatternLayout are slighty different between log4j 1.x and 2.x, and I do not know which version are you using, but I think you can't achieve this by configuration in neither versions.
You can achieve that programmatically (but this is going to affect your performance), I think in your L.java method you will have to use a method like:
private Logger logger = getYourLoggerAsYouAreCurrentlyDoing();
public enum LogLevel { INFO,DEBUG, ERROR, ETC }
void log(String msg, LogLevel level) {
StackTraceElement[] stackTraceElements = Thread.currentThread().getStackTrace();
String callerClass = stackTraceElements[1].getClassName();
String callerLine = "" + stackTraceElements[1].getLineNumber();
String msg = callerClass + ":" + callerLine + "-" + msg;
switch(LogLevel) {
case INFO: logger.info(msg); break;
case DEBUG: logger.debug(msg); break;
//etc.
}
}
And in case another method with the Throwable argument to log stacktraces.