Quarkus - configure logging with kubernetes cluster - java

According to the official resource, logging configuration relies on application.properties file.
Now I need to have several configuration according to the cluster in use (let's say we have the typical dev, staging and production environments, thus dev should have a DEBUG level and production at least INFO).
At first I thought using Kubernetes ConfigMaps, but I can't see any connection with quarkus logging.
How can I solve this issue?
EDIT:
This is my ConfigMap
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-cm-config-map
namespace: default
uid: d992d86f-c247-471d-8e31-53e9a1858b76
resourceVersion: '8484'
creationTimestamp: '2021-04-22T13:12:43Z'
managedFields:
- manager: kubectl-create
operation: Update
apiVersion: v1
time: '2021-04-22T13:12:43Z'
fieldsType: FieldsV1
fieldsV1:
'f:data':
.: {}
'f:myenv': {}
'f:myname': {}
- manager: kubectl-edit
operation: Update
apiVersion: v1
time: '2021-04-22T16:52:18Z'
fieldsType: FieldsV1
fieldsV1:
'f:data':
'f:log.file.level': {}
- manager: dashboard
operation: Update
apiVersion: v1
time: '2021-04-23T08:03:06Z'
fieldsType: FieldsV1
fieldsV1:
'f:data':
'f:quarkus.log.file.level': {}
data:
log.file.level: DEBUG
myenv: cl1
myname: cluster1
quarkus.log.file.level: DEBUG
EDIT2
This is my config map (through command kubectl edit cm ):
apiVersion: v1
data:
QUARKUS_LOG_FILE_ENABLE: "true"
QUARKUS_LOG_FILE_FORMAT: '%d{HH:mm:ss} %-5p [%c{2.}] (%t) %s%e%n'
QUARKUS_LOG_FILE_LEVEL: ERROR
QUARKUS_LOG_FILE_PATH: /tmp/kube-cm.log
myenv: cl1
myname: cluster 1b
kind: ConfigMap
metadata:
creationTimestamp: "2021-04-22T13:12:43Z"
name: kube-cm-config-map
namespace: default
resourceVersion: "39810"
uid: d992d86f-c247-471d-8e31-53e9a1858b76

If you are using Kubernetes resource yaml to deploy your app, use the snippet below to push your custom ConfigMap as environment variables to your applicaton (https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#configure-all-key-value-pairs-in-a-configmap-as-container-environment-variables):
spec:
containers:
- name:
image:
envFrom:
- configMapRef:
name: kube-cm-config-map
Use a different ConfigMap for each environment but with the same name. If your environments (dev/qa/etc) are Kubernetes namespaces, then it is very easy to setup. Just duplicate the ConfigMap in each namespace, and change the log level value in each namespace.
Also, change the naming convention for your ConfigMap properties from log.file.level to LOG_FILE_LEVEL
See https://quarkus.io/guides/config-reference#environment_variables

Ok solved by editing the cm in the following way:
data:
QUARKUS_LOG_FILE_ENABLE: "true"
QUARKUS_LOG_FILE_FORMAT: '%d{HH:mm:ss} %-5p [%c{2.}] (%t) %s%e%n'
QUARKUS_LOG_FILE_LEVEL: ERROR
QUARKUS_LOG_FILE_PATH: /tmp/kube-cm.log
Then I set application.properties on quarkus with:
quarkus.kubernetes.env.configmaps=kube-cm-config-map

There are 2 ways to use a ConfigMap in Quarkus to read runtime configuration.
The first is to let Quarkus query the API server using the quarkus-kubernetes-config extension which is described here.
The second way to configure the Kubernetes Deployment to turn ConfigMap values into environment variables for the Pod. This can be done with the quarkus-kubernetes extension which is described here.
So you would add the proper quarkus logging configuration (i.e a key value pair) in the ConfigMap and then use one of the above methods to use that at runtime

Related

Java Spring Active profile in kubernetes Cluster

i want to start Java spring app with active profile...
I build Docker image in Gitlab CI/CD using maven wrapper ,
./mvnw compile jib:build -Dimage=image/sms-service:1
after that i deploy app in k8s....
now i want to run with active profile , what is best way? how can i define in k8s to run specific user
apiVersion: apps/v1
kind: Deployment
metadata:
name: sms-service
namespace: sms-service
spec:
selector:
matchLabels:
app: sms-service
replicas: 4 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: sms-service
spec:
template:
spec:
containers:
- name: sms-service
image: image/sms-service:1
imagePullPolicy: Always
ports:
- containerPort: 8080
imagePullSecrets:
- name: sms-service
Set the SPRING_PROFILES_ACTIVE environment variable to the profile(s) you want to run.
You can set it in the deployment yaml or at build time in your image but usually better to add it to deployment.
Create a new file, named configmap.yaml under the k8s config folder and add the following lines:
apiVersion: v1
kind: ConfigMap
metadata:
name: blabla
namespace: bla
data:
application.yaml: |
spring:
profiles:
active: prod (here goes the profile)
This tells Kubernetes to set this configuration when starting the container

Dockerized Spring Boot app not using mounted Kubernetes ConfigMap (application.properties)

I have a problem where in my dockerized Spring Boot application is not using the application.properties I stored in a configMap.
However, I can see and confirm that my configMap has been mounted properly in the right directory of my Spring Boot app when I enter the pod's shell.
Note that I have an application.properties by default wherein Kubernetes mounts / overwrites it later on.
It seems that the Spring Boot uses the first application.properties and when k8s overwrites it, apparently, it doesn't use it.
It seems that, apparently, what happens is:
run the .jar file inside the Dockerized Spring Boot app
use the first/default application.properties file on runtime
Kubernetes proceeds to mount the configMap
mount / overwrite success, but how will Spring Boot use this one since it's already running?
Here is the Dockerfile of my Spring Boot / Docker image for reference:
FROM maven:3.5.4-jdk-8-alpine
# Copy whole source code to the docker image
# Note of .dockerignore, this ensures that folders such as `target` is not copied
WORKDIR /usr/src/myproject
COPY . /usr/src/myproject/
RUN mvn clean package -DskipTests
WORKDIR /usr/src/my-project-app
RUN cp /usr/src/myproject/target/*.jar ./my-project-app.jar
EXPOSE 8080
CMD ["java", "-jar", "my-project-app.jar"]
Here's my Kubernetes deployment .yaml file for reference:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-project-api
namespace: my-cluster
labels:
app: my-project-api
spec:
replicas: 1
selector:
matchLabels:
app: my-project-api
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
template:
metadata:
labels:
app: my-project-api
spec:
containers:
- name: my-project-api
image: "my-project:latest"
imagePullPolicy: Always
env:
.
.
.
volumeMounts:
- name: my-project-config
mountPath: /usr/src/my-project/my-project-service/src/main/resources/config/application.properties
ports:
- containerPort: 8080
name: my-project-api
protocol: TCP
volumes:
# Name of the volume
- name: my-project-config
# Get a ConfigMap with this name and attach to this volume
configMap:
name: my-project-config
And my configMap for reference:
kind: ConfigMap
apiVersion: v1
data:
application.properties: |-
# This comment means that this is coming from k8s ConfigMap. Nice!
server.port=8999
.
.
.
.
metadata:
name: my-project-config
namespace: my-cluster
Any help is greatly appreciated... Thank you so much.. :)
The thing is that /src/main/resources/application.properties that your application uses is the one that is inside the jar file by default. If you open your jar, you should see it there.
That being said, your expectations to mount a /src/main/resources directory where your jar is are not going to be fulfilled, unfortunately.
These are the docs you should be looking at.
I won't go into much detail as it's explained pretty good in the docs but I will say that you are better off explicitly declaring your config location so that new people on the project know from where the config is coming from right off the bat.
You can do something like this:
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-project-api
labels:
app: my-project-api
spec:
selector:
matchLabels:
app: my-project-api
template:
metadata:
labels:
app: my-project-api
spec:
containers:
- name: my-project-api
image: "my-project:latest"
imagePullPolicy: Always
env:
- name: JAVA_OPTS
value: "-Dspring.config.location=/opt/config"
.
.
.
volumeMounts:
- name: my-project-config
mountPath: /opt/config
ports:
- containerPort: 8080
volumes:
- name: my-project-config
configMap:
name: my-project-config
Hope that helps,
Cheers!
I did slightly differently. I made sure I have mounted application.properties at config/. i.e; below is my example mounted application.properties (below commands show the values in pod - i.e; after kubectl exec -it into the pod)
/ # pwd
/
/ # cat config/application.properties
logback.access.enabled=false
management.endpoints.web.exposure.include=health, loggers, beans, configprops, env
Basically, the trick is based on the link in the above answer. Below is an excerpt from the link in which it does say application.properties will be picked from config/. So, I made sure my environment (dev, test, prod) specific config map was mounted at config/. Do note there is precedence for the below list (per the link: locations higher in the list override lower items)
A /config subdir of the current directory.
The current directory
A classpath /config package
The classpath root
Below is the config map definition (just pasted data section)
data:
application.properties: |+
logback.access.enabled={{.Values.logacbkAccessEnabled}}
management.endpoints.web.exposure.include=health, loggers, beans, configprops, env
And you can also see from actuator/env endpoint SpringBootApp did pick those values.
{
"name": "Config resource 'file [config/application.properties]' via location 'optional:file:./config/'",
"properties": {
"logback.access.enabled": {
"value": "false",
"origin": "URL [file:config/application.properties] - 1:24"
},
"management.endpoints.web.exposure.include": {
"value": "health, loggers, beans, configprops, env",
"origin": "URL [file:config/application.properties] - 2:43"
}
}
},

Spring Cloud Kubernetes fails setting ConfigMap values

I am trying to support K8s ConfigMap API using Spring to handle my externalized configuration. I've been using a configuration service without any issues, so I don't believe it is a cloud-config implementation problem. I am using version 1.0.2.RELEASE of spring-cloud-starter-kubernetes-config.
My ConfigMap consists of two configurations: global (application.yml) and app-specific yaml (myapp.yml). Here is what that looks like:
apiVersion: v1
kind: ConfigMap
metadata:
name: myapp-config
data:
application.yml: |-
kafka:
hosts: kafka:9092
topics:
events: myapp-events
normalized: myapp-normalized
#...
storage:
uri: mongodb://mongo-0.mongo:27017,mongo-1.mongo:27017,mongo-2.mongo:27017
database: do-stuff-db
#...
spring:
cloud:
config:
override-system-properties: false
#...
myapp.yml: |-
myapp:
broadcast-address: 0.0.0.0:666
#...
application.yml seems to work just fine. I can see the configuration values in my application but for whatever reason, myapp.yml does not get set. I can see Spring is successfully grabbing the entire result from the ConfigMap API but it does not do anything with myapp.yml.
Turning on debug logging exposed how Spring is interpreting the ConfigMap result. It is treating the entire myapp.yml section as one property and not its own property source. This explains why the application.yml is taking and not the myapp.yml. Here is what that log looks like (I have truncated the message):
2019-09-04 14:01:37.083 [main] INFO config.PropertySourceBootstrapConfiguration - Located property source: CompositePropertySource {name='composite-configmap', propertySources=[ConfigMapPropertySource#92864491 {name='configmap.myapp-config.default', properties={kafka.hosts=kafka:9092, kafka.topics.events=myapp-events, kafka.topics.normalized=myapp-normalized, myapp.yml=myapp:
bind-address: 0.0.0.0:666
#...
storage.uri: mongodb://mongo-0.mongo:27017,mongo-1.mongo:27017,mongo-2.mongo:27017, storage.database: do-stuff-db}}]}
Do I need to separate these configurations? The examples I've seen from Spring suggest you could consolidate your configuration into a single YAML ConfigMap file. I am stumped.

Single docker image thats being deployed as 2 different services thru kubernetes and helm. Change context path of app

We have a single docker image thats being deployed as 2 different services thru kubernetes and helm with names like "ServiceA" and "ServiceB". At the point deploy happens need to set the context path of Tomcat to be something different like /ServiceA and /ServiceB. How can this be done ? is there anything that can be set directly on the yaml ?
ex: Looks like below
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "fullname" . }}-bg
{{- include "labels" . }}
spec:
replicas: {{ .replicaCount }}
selector:
matchLabels:
app.kubernetes.io/name: {{ include "name" . }}-bg
app.kubernetes.io/instance: {{ .Release.Name }}
strategy:
type: Recreate
rollingUpdate: null
template:
metadata:
labels:
app.kubernetes.io/name: {{ include "name" . }}-bg
app.kubernetes.io/instance: {{ .Release.Name }}
spec:
containers:
- name: {{ .Chart.Name }}
image: "{{ .image.repository }}:{{ .image.tag }}"
imagePullPolicy: {{ .image.pullPolicy }}
env:
- name: SERVICE_NAME
value: "ServiceB"
- name: background.jobs.enabled
value: "true"
envFrom:
- configMapRef:
name: {{ include "commmonBaseName" . }}-configmap
-
There are few approaches into setting up the context path of an app.
From the app itself: This depends on the language/framework/runtime your application uses. For example, if it's a traditional Java web application that runs on Tomcat, it would be served by default from the context path of the name of the .war file you put in the webapp directory. Or, if it is a Spring Boot 2.X app, you could set it up with the Spring Boot property server.servlet.context-path, which can be also passed via an environment variable, specifically SERVER_SERVLET_CONTEXT_PATH. So, to give an example, in the container in your deployment pod spec:
env:
- name: SERVER_SERVLET_CONTEXT_PATH
value: "/ServiceB"
However, this kind of app-specific settings are most of the times not needed in Kubernetes, since you can handle those concerns on the outer layers.
Using Ingress objects: If you have an Ingress controller running and properly configured, you can create an Ingress that will manage path prefix stripping, and other HTTP Layer7 concerns. This means, you can leave your application itself as-is (like serving from root context /) but configure the context path from the Ingress. An example is the following, assuming you use Nginx Ingress Controller
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: service-a-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: service-a.yourdomain.com
http:
paths:
- path: /ServiceA/(.*)
backend:
serviceName: service-a
servicePort: service-a-port
Note the capture group (.*) in the path, and $1 in the rewrite target - it will rewrite the request paths like /ServiceA/something to /something before forwarding the packet to your backend.
See this page to lear more about ingresses.
You can use an HTTP router software such as skipper to handle all this HTTP traffic configuration in-cluster.
If you use a service mesh solution such as Istio, they give you many ways to manage the traffic inside the mesh.

reference variables from pom.xml to deployment.yaml

i want to use maven pom.xml variables in a kubernetes deployment.yaml file. the variables i want to reference are ${project.artifactId} and ${project.version} which is pulled from
pom.xml
<groupId>my-project</groupId>
<artifactId>>my-project</artifactId>
<version>1.0.0-SNAPSHOT</version>
and this is what i want to achieve
deploment.yaml
apiVersion: v1
kind: Pod
spec:
containers:
- name: my-project
image: ${project.artifactId}:${project.version}
with this attempt i get an InvalidImageName error.
please advice on which better way of doing this.
I would say its issue with your deploment.yaml content.
I have use it (with nginx image) on K8s and get error below:
error: error when retrieving current configuration of:
Resource: "/v1, Resource=pods", GroupVersionKind: "/v1, Kind=Pod"
Name: "", Namespace: "default"
Object: &{map["apiVersion":"v1" "kind":"Pod" "spec":map["containers":[map["name":"test" "image":"nginx"]]] "metadata":map["namespace":"default" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""]]]}
from server for: "pod.yaml": resource name may not be empty
In your current file you have named only container. You have to specify your POD name using metadata.name. In metadata section you can also specify namespace.
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
spec:
containers:
- name: nginx
image: nginx
In addition keep in mind that kind: Pod and kind: Deployment are two different things (bit confused regarding your YAML file name).

Categories