We currently generically log all XML documents coming in and going out of our system, and some of them contain passwords in the clear. We would like to be able to configure the logback logger/appender that is doing this to do some pattern matching or similar and if it detects a password is present to replace it (with asterisks most likely). Note we don't want to filter out the log entry, we want to mask a portion of it. I would appreciate advice on how this would be done with logback. Thanks.
The logback version 0.9.27 introduced replacement capability. Replacements support regular expressions. For example, if the logged message was "userid=alice, pswd='my secret'", and the output pattern was
"%d [%t] $logger - %msg%n",
you just modify the pattern to
"%d [%t] $logger - %replace(%msg){"pswd='.*'", "pswd='xxx'"}%n"
Note that the above makes use of option quoting.
The previous log message would be output as "userid=alice, pswd='xxx'"
For blazing performance, you could also mark the log statement as CONFIDENTIAL and instruct %replace to perform replacement only for log statements marked as CONFIDENTIAL. Example,
Marker confidential = MarkerFactory.getMarker("CONFIDENTIAL");
logger.info(confidential, "userid={}, password='{}'", userid, password);
Unfortunately, the current version of logback does not yet support conditional replacements (based on markers or otherwise). However, you could easily write your own replacement code by extending ReplacingCompositeConverter. Shout on the logback-user mailing list if you need further assistance.
I believe Masking is an aspect of your business, not the aspect of any technology or logging system. There are situations where the passwords, national identities etc should be masked while storing them in the DB as well due to legal reasons. You should be able to mask the xml before giving it to the logger.
One way to do it is to run the XML through XSLT that does that making and then give it to logger for logging.
If you doesn't want to do this then LogBack has Filters support that is one of the option (not the right one though).
But understand that any generic out of the box solution you are trying to find at the logging infrastructure level is going to be suboptimal as every log message is going to be checked for masking.
Related
I'm told to write a code that get a string text and check if its encoding is equal the specific encoding that we want or not. I've searched a lot but I didn't seem to find anything. I found a method (getEncoding()) but it just works with files and that is not what I want. and also I'm told that i should use java library not methods of mozilla or apache.
I really appreciate any help. thanks in advance.
What you are thinking of is "Internationalization". There are libraries for this like, Loc4j, but you can also get this using java.util.Locale in Java. However in general text is just text. It is a token with a certain value. No localization information is stored in the character. This is why a file normally provides the encoding in the header. A console or terminal can also provide localization using certain commands/functions.
Unless you know the source encoding and the token used you will have a limited ability to guess what encoding is used in the other end. If you still would want to do this you will need to go into deeper areas such as decryption where this kind of stuff usually is done using statistic analysis. This in turn requires databases on the usage of different tokens and depending on the quality of the text, databases and algorithms a specific amount of text is required. Special stuff, like writing Swedish with eg. US encoding (like using a for å and ä or o for ö) will require more advanced analysis.
EDIT
Since I got a comment that encoding and internationalization is different entities I will add some comments. It is possible to work with different encodings working plainly with English (like some English special characters). It is also possible to work with encodings using for example Charset. However for many applications using different encodings it may still be efficient to use Locale, since this library can do a lot of operations on text with different encodings.
Thanks for ur answers and contribution but these two link did the trick. I had already seen these two pages but it didn't seem to work for me cause I was thinking about get the encoding directly and then compare it with the specific one.
This is one of them
This is another one.
I'm trying to generate a 'nice' log message with using escaping characters or an xml pattern etc.
What I'd like to output is something along the lines of:
ABAS : value
B : value
Cse : value
I've achieved this using \t but I figure there must be a cleaner way. I've looked at .info which takes an argument and using the {} as a way of inserting the values but I can't seem to find out how to add the line breaks or tabbing.
so far I have
logger.info(A : {} \nBasdas : {} \nC : asds ) and so on.
thanks for any help.
slf4j is the log frontend and only intended to provide log level, message etc. to the backend, most likely logback in your case. You shouldn't format your messages in the frontend expecting any special format in the actual log output, because exactly that can be somewhat freely configured by the backend one uses. Especially indentation over some independent lines doesn't work, because you can't know how lines start, if your logger names are part of lines, where the msg is printed within a line and all that stuff. Just look at the logback configuration and what is possible, how do you want to tell as the log message issuing programmer which of those possibilities are used during runtime in any environment of your software? You simply can't and therefore shouldn't assume too much.
So what you want is simply not possible, besides embedding tabs or newlines there's nothing to format log messages in slf4j for a good reason. And you can't count on your tabs as well, because how those are presented to a user looking at your log file depends totally on the text editor or whatever one uses. It may even convert tabs to spaces, show them with a width of 1 or 10 or whatever.
Log statements spanning multiple lines may be considered bad practice at all.
I have a requirement to include the name of the log file in the log entry itself.
For example say final name of the log file is something like trx_log.2014-09-22-12-42 the log entries I'm printing that log should have that same name. Following is a example log entry.
123456|test value|xyz|trx_log.2014-09-22-12-42
I'm using Log4j DailyRollingFileAppender to print the log at the moment. Is there a way which I can get this requirement implemented using some log4j/logback configuration.
Not that I'm aware of.
But a solution does exist nevertheless: write your own custom extension of the DailyRollingFileAppender.
Please note though the filename will be available to your custom appender only: in case you want to use such information in another appender (the only use case I can think of this might be of any use) then you need a more convoluted solution using a shared data storage (shared memory, file system, database, whatever) with the simplest solution being a static member of your just made appender. In this case the other appender (lat's say Console) need to be extended as well in order to append the new information to the log statement.
Use this method logger.getName()
logger.log(Level.SEVERE,"Exception in "+e.getMessage()+logger.getName());
After doing logging for many years I have now reached a point where I need to be able to postprocess the log files with the long term goal of using the log files as a "transport medium" allowing me to persist objects et. al. so I can replay the backend requests. In order to do so I need to persist objects into a loggable form.
The recommended way to do this by Sun, is to use java.beans.XMLEncoder to create an XML snippet which is quite agreeable with me, but the problem is that it is sent to an UTF-8 encoded OutputStream including an UTF-8 header, and OutputStreams are byte oriented. Log files are character oriented (strings) and logfiles are typically encoded in the default encoding for that platform. Our XML may include any Unicode character.
I need a robust way of handling this, with preferation to an approach which generates humanly readable files.
I have thought about converting the XML OuptutStream to a String, removing the unusable header, and flattening to ASCII (with any non-ASCII character encoded as a numeric entity). I have also thought about using XML transformations but I have a gut feeling that this will require more resources than I want a logger to do.
Suggestions?
More a hint than a real answer: maybe have a look at this thread on the logback-dev mailing list and especially the messages from Joern Huxhorn (which is the author of Lilith). More generally, I think that you should look at logback, the "successor" of log4j from the same author, Ceki Gülcü. This is where things happen in my opinion.
Is there a way to avoid or set up a schema to display better user friendly messages?
I am parsing the string and using reg ex to interpret them, but there might be a better way.
Ex.
"cvc-complex-type.2.4.b: The content of element 'node' is not complete. One of '{\"\":offer,\"\":links}' is expected."
Instead I want:
"The element 'node' is not complete. The child elements 'offer' and 'links' are expected."
Again, I've solved the problem by creating an extra layer that validates it. But when I have to use a XML tool with a schema validation, the crypt messages are the ones displayed.
Thanks
Not that I know of. You will probably have to create some custom code to adapt your error messages. One way might be to define a set of regular expressions that can pull out the relevant pieces of the validator's error messages and then plug them back into your own error messages. Something like this comes to mind (not optimized, doesn't handle general case, etc. but I think you'll get the idea):
String uglyMessage = "cvc-complex-type.2.4.b: The content of element 'node' is not complete. One of '{\"\":offer,\"\":links}' is expected.";
String findRegex = "cvc-complex-type\\.2\\.4\\.b: The content of element '(\\w+)' is not complete\\. One of '\\{\"\":(\\w+),\"\":(\\w+)}' is expected\\.";
String replaceRegex = "The element '$1' is not complete. The child elements '$2' and '$3' are expected.";
String userFriendlyMessage = Pattern.compile(findRegex).matcher(uglyMessage).replaceAll(replaceRegex);
System.out.println(userFriendlyMessage);
// OUTPUT:
// The element 'node' is not complete. The child elements 'offer' and 'links' are expected.
I suspect those validator error messages are vendor-specific so if you don't have control over the XML validator in your deployed app, this may not work for you.
We're using Schematron for displaying user a friendly error messages if XML he send us is wrong. Our current implementation is a bit simplistic, notably in the following points:
Error messages text is hadcoded into a schematron rules
For each new XML type (i.e. new XSD schema) there is a need to manually add schematron rules
This, however, can be easily fixed, by the following rework:
Schematron rules should contain a unique error message codes, while actual message text selection (including I18n issues) should be done out of validation framework scope
Basic rules can be generated from XSD schema using XSD to Schematron converter (available at http://www.schematron.com/resources.html)
I asked a similar question a while ago.
My conclusion was there is no provided way of mapping the errors, and that it's something you need to do yourself.
Hope someone out there can do better!