I am very much new to multi-tenancy. We have an application based on Java, Spring, Hibernate/JPA etc. which doesn't support multi-tenancy.
Now, we want to convert that application into multi-tenant one. I have read about multi-tenancy and even wrote one standalone application using hibernate with separate schema approach. Link referred is here.
I think about the logging part which is bound to be changed now as log files will be maintained per tenant(client) now. So, for each tenant a separate log file will be there.Also, log file for a particular tenant shouldn't be accessed by another tenant.
Is there any logging API specific to support multi-tenancy? If not, how should i go ahead with implementing logging in multi-tenant application? What should be taken care of while implementing logging in multi-tenant application.
you can use MDC (mapped diagnostic context) support to route logging for each tenant into a separate file/dir/whatever.
you can read up on the concept here. it exists in slf4/logback and log4j
simply put, you set some property like tenantName in the MDC at the beginning of every request processing according to the specific tenant making the request and then use this property in your logging configuration to determine the log file into which log messages are written.
Related
To give you some context, we have various microservices and a config server where we store store all the application.yml files.
The problem is when some issue happens, we have to temporarily enable logs for every application so that we can trace the activities done by the user. But this creates a hassle as we have to change the log configs.
So we want to enable logging for only certain users (say, when some issue happens and we want to debug) via changing the yml files so we don't have to touch the code or redeploy the application. Basically we are thinking of creating a application-logging.yml file for all the microservices and in this when we mention
logging:
users:
id: 123
level: warn
all the microservices will pick it up and start generating logs only for this user.
How can I implement this in Spring Boot, if its possible?
Put user ID into the thread context then use a MutableThreadContextMapFilter.
You can specify values via a JSON file, which log4j polls periodically to pick up changes.
I am using OpenTelemetry java auto instrumentation in my spring boot app. Is there a way to make the application logs part of the spans that are created?
My autoconfig settings are as below:
-Dotel.traces.exporter=jaeger
-Dotel.metrics.exporter=none
-Dotel.exporter.jaeger.endpoint=http://localhost:14250
-Dotel.resource.attributes=service.name=myService
-javaagent:C:/path/to/opentelemetry-javaagent-1.0.1-all.jar
OpenTelemetry ships logs separately to the telemetry data obtained from auto instrumentation, and does not interleave log data I'm afraid. We ship our logs via the use of FluentBit (https://medium.com/opentelemetry/introducing-the-fluentbit-exporter-for-opentelemetry-574ec133b4b4).
You may wish to use manual instrumentation and add spans, span attributes and/or events to pertinent code blocks, to add log like context to the metadata utilised downstream.
As you are using Spring Boot, it would be advisable to use one of the starter dependencies, such as opentelemetry-otlp-exporter-starter (https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/spring/starters/otlp-exporter-starter), which should get you most of the way there. You want to use the #WithSpan annotation to decorate your methods, which will enable you to obtain the current span easily. See https://opentelemetry.lightstep.com/java/.
The official docs have a few examples, that may help, but be aware that the API and SDK are changing rapidly, so examples don't always work - https://opentelemetry.io/docs/java/manual_instrumentation/.
Detailed information regarding OpenTelemetry and logging: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/overview.md
Adding logs to spans depends a bit on the backend you are using for collecting the traces/spans and visualizing them.
I used Jaegar, which interprets OTEL events as logs in the UI, and so I wrote a custom log appender which put app logs into an event, which was picked up subsequently in the UI.
More details here:
https://stackoverflow.com/a/68739794/2715083
I have just discovered that Apache commons-configuration can read properties from a DataSource, but it does not cache them. My application needs to read properties a lot of times and it is to slow to access the database each time.
I have a Camel application that sends all messages to routes that ends with my custom beans. These beans are created with scope prototype (I believe in OOP) and they will/need to read some properties and a data source (which reads from properties url/name/etc) that depends from the current user from a SQL db. Each message I receive creates a bean and so properties are reread.
Unfortunately, I am not free to choose where to read properties from because now there is another software (GUI) not written by me that is a User/properties manager that writes to db. So I need to read properties from it.
Can you suggest me an alternative?
You could use the Netflix Archaius project, which adds the caching behavior you are looking for as well as dynamic refresh capabilities. Archaius is built around Commons Configuration.
So, rather than subclassing the DatabaseConfiguration, you could use Archaius' DynamicConfiguration, which extends Commons' AbstractConfiguration. This class will cache whatever source you would like, and refresh the properties at an interval you specify using their poll scheduling class.
The only class you would have to implement is a PolledConfigurationSource which pulls data from the database and places it in a Map. Should be pretty simple.
https://github.com/Netflix/archaius/wiki/Users-Guide
I was going through details of CAS project and found that it is using something called inspektr. I googled for some time and tried to find more details about its usage. But I did not get any information.
Can anyone provide more details about it and its usage.
Thanks in advance.
Inspektr can be found here: https://github.com/dima767/inspektr with details for usage here: https://github.com/dima767/inspektr/wiki/Inspektr-Auditing
As I understand the project, it collects information from your web flow and allows you to save said data through the use of the #Audit annotations provided. If the configuration is copied from that CAS project you linked, nearly everything's configured to log to a file. Sample data logged would be the Client's IP, remote IP, the action being performed (as configured via Spring and the #Audit annotation), as well as various other things.
If you're familiar with Spring Aspects, it should be a breeze to look through the Inspektr source code to find other uses.
Inspektr is a framework that allows us to drive audit records from Annotations utilizing an Aspect that is provided with the framework. This works for Spring Managed Beans only!
Here the github project website:
https://github.com/dima767/inspektr/wiki/Inspektr-Auditing
A good practical reference for config: https://wiki.jasig.org/display/CASUM/Auditing+and+Statistics+Via+Inspektr
The base principal here is that Inspektr allows for logging of these audit frames into the console, database, the application server log ,we can even define our own managers to log to a different medium if required.
There is only one file. And it is written simultaneously as web app copies run.
How do you filter only one session log messages from other log lines?
Using a servlet filter with either NDC or MDC information is the best way I've seen. A quick comparison of the two is available at http://wiki.apache.org/logging-log4j/NDCvsMDC.
I've found MDC has worked better for me in the past. Remember that you'll need to update your log4j properties file to include whichever version you prefer (pattern definitions at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html).
A full example of configuring MDC with a servlet filter is available at http://veerasundar.com/blog/2009/11/log4j-mdc-mapped-diagnostic-context-example-code/.
A slightly easier to configure, but significantly inferior option: You could opt to just print out the thread ID (via the properties file) for each request and make sure that the first thing you log about each request is a session identifier. It isn't as proper (or useful), but it can work for low-volume applications.
You could set a context message including the identifier of the specific app instance using org.apache.log4j.NDC, like this:
String appInstanceId = "My App Instance 1";
org.apache.log4j.NDC.push(appInstanceId);
// handle request
org.apache.log4j.NDC.clear();
You can set up the context during the initialization of your web app instance, or inside the doPost() method of your servlets. As its name implies, you can also nest contexts within contexts with multiple push calls at different levels.
See the section "Nested Diagnostic Contexts" in the Log4J manual.
Here is a page that sets up an MDC filter for web-app -> http://rtner.de/software/MDCUserServletFilter.html
Being a servlet filter it will free you from managing MDC/NDC in each of your servlets.
Of course, you should modify it to save information more pertinent to your web-app.
If you want to differentiate sessions in the same application then the MDC is the way to go. But if you want to differentiate the web applications writing to the same file, then MDC won't help because it works on a thread basis. In such case I used to make my own appender which knows which application instance it serves. This can be done through appender configuration properties. Such appender would stick application name into each logging event as a property before writing it into the media, and then you can use a layout to show this property value in the text file it writes to. Using MDC in such case won't work because every thread will have to MDC.put(applicationName) and that is quite ugly. MDC is only good for single process, not for several processes. If someone knows the other way, I'd like to hear.