I'm trying to implement a simple liveness probe in my helm chart deployment template. Below is my liveness probe configuration. Spring boot /actuator/health endpoint is used as the health check endpoint.
containers:
- name: {{ .Release.Name }}-container
image: {{ .Values.container.image }}
ports:
- containerPort: 8080
livenessProbe:
httpGet:
path: /actuator/health
port: 8080
failureThreshold: 5
periodSeconds: 10
initialDelaySeconds: 30
timeoutSeconds: 25
This is the error I'm encountering (Tried adding a large initialDelay and also tried adding a startupProbe. Both did not work)
Liveness probe failed: Get http://x.x.x.x:8080/actuator/health: dial tcp x.x.x.x:8080: connect: connection refused
However I'm able to get a 200 response from different pods via this endpoint which are in the same ec2 instance and also different ec2 instances.
$k exec -it pod/test sh
# curl http://x.x.x.x:8080/actuator/health -I
HTTP/1.1 200 OK
Connection: keep-alive
Transfer-Encoding: chunked
Content-Type: application/vnd.spring-boot.actuator.v3+json
correlation-id: x-x-x-x-x
Date: Fri, 09 Oct 2020 14:04:56 GMT
Without the liveness probe, the app is working fine and I can access all the endpoints via port 8080.
Tried setting up livenessprobe to an nginx image and it works fine (So ruling out network issues)
Containers:
liveness:
Container ID: docker://0af63462845d6a2b44490308147c73277d22aff56f993ca7c065a495ff97fcfa
Image: nginx
Image ID: docker-pullable://nginx#sha256:c628b67d21744fce822d22fdcc0389f6bd763daac23a6b77147d0712ea7102d0
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Tue, 29 Sep 2020 15:53:17 +0530
Ready: True
Restart Count: 0
Liveness: http-get http://:80/ delay=2s timeout=1s period=2s #success=1 #failure=3
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-57smz (ro)
Related
I have this problem that's driving me insane. I have two deployment and two service yaml files created by kompose convert from a docker-compose. The app that I'm trying to run in Google Cloud is a Spring Boot web app with a mariadb backend. After I apply the four yamls with kubectl, I expose the frontend deployment (on port 8081) by running
TL;DR for anyone coming to this question via search:
OP's service was a ClusterIP and not LoadBalancer. Setting this as LoadBalancer still did not fix issue. Checking logs of pod determined code was unable to connect to DB, so never actually started up successfully.
Output from OP:
kubectl get svc -n default
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.72.0.1 <none> 443/TCP 147m
load-balancer LoadBalancer 10.72.15.246 34.69.204.138 80:30870/TCP 86s
mysqldb ClusterIP 10.72.3.186 <none> 3308/TCP 3m20s
web-app ClusterIP 10.72.13.41 <none> 8081/TCP 3m19s
Please share those in your question, making sure to prepend "```" before and after each yaml to preserve formatting
Now, a few points.
You specified you have service yamls. If you have a yaml that describes a service such as a LoadBalancer, you shouldn't need to kubectl expose afterwards, as your yaml should have done that for you.
Assuming the service named "load-balancer" is the one you have created via your yamls, the IP:port combination you should be using is 34.69.204.138:80. What IP have you been trying to access? Are you trying to access this IP and port? Or a different one?
UPDATE
Based on the pasted yamls, I see this:
In your docker-compose yaml:
web-app:
build: .
image: mihaialexandruteodor/featherwriter
ports:
- "8081:8081"
expose:
- "8081"
This is exposing port 8081 and connecting it to the underlying container.
This is reflected in the service yaml:
apiVersion: v1
kind: Service
metadata:
...
name: web-app
spec:
ports:
- name: "8081"
port: 8081
targetPort: 8081
selector:
io.kompose.service: web-app
status:
loadBalancer: {}
However, I do not see a service called "web-app" in your listing. It's possible therefore, you may have deployed it into a different namespace.
Try kubectl get svc --all-namespaces and see where the service "web-app" is. Find the IP from that, the port should be 8081 and you can then do x.x.x.x:8081 to access the service.
UPDATE 2
The web-app service is of type ClusterIP (documentation) which cannot be accessed outside of the cluster, you need to change the service to be a LoadBalancer type, or use port-forwarding.
To make the service a LoadBalancer, change the service yaml as follows (documentation here):
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: kompose convert
kompose.version: 1.26.1 (HEAD)
creationTimestamp: null
labels:
io.kompose.service: mysqldb
name: mysqldb
spec:
ports:
- name: "3308"
port: 3308
targetPort: 3306
selector:
io.kompose.service: mysqldb
type: LoadBalancer
This will provision a service that will have an external IP you can use.
Alternatively, use port forwarding to connect a local port with the port being listened on by the service:
kubectl port-forward -n {namespace} svc/web-app 8081:8081
Then you can use localhost:8081 to connect to your service. This option does not require an externally-accessible endpoint, but you will need to run the port forward command (and have it active) each time you want to access the service via the localhost endpoint.
If you want to be able to access the service from somewhere outside of your cluster, that is not your local machine, and is not within the same cluster, you will need to use a LoadBalancer service type.
UPDATE 3
Right, I can't build that Dockerfile as I do not have the src folder, but I can run the image from mihaialexandruteodor/featherwriter, and can see it is indeed listening on 8081
Tomcat initialized with port(s): 8081 (http)
so the next thing to see is see if there's any issues with the pod functionality itself. First check the pod status:
kubectl get pods -n {namespace}
The pod should be called web-app-xxxxx where xxxxx is a random sequence of letters and numbers.
Is the web-app pod running? Does it have a restart counter that is not zero like some of the pods in my prometheus namespace:
$ kubectl get pods -n prometheus
NAME READY STATUS RESTARTS AGE
alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 3h51m
prometheus-grafana-66cb8bcf4f-428d8 3/3 Running 0 3h51m
prometheus-kube-prometheus-operator-749fc8899b-dnvft 1/1 Running 0 3h51m
prometheus-kube-state-metrics-77698656df-btq4k 1/1 Running 20 3h51m
prometheus-prometheus-kube-prometheus-prometheus-0 2/2 Running 0 3h51m
prometheus-prometheus-node-exporter-jj9z5 1/1 Running 30 3h51m
prometheus-prometheus-node-exporter-lbk6p 1/1 Running 0 3h51m
prometheus-prometheus-node-exporter-vqfhk 1/1 Running 20 3h51m
Next get the logs from the pods like so:
kubectl logs -n {namespace} web-app-xxxxx
See if you can find any errors.
My hunch, given that we've connected everything through on 8081 and Tomcat is indeed running on 8081, is that the spring app is crashing repeatedly and Kubernetes is restarting it, the app then fails again, and it tries again over and over, eventually failing into a CrashLoopBackOff state where Kubernetes will delay restarting by a longer period.
I am trying to use Spring Cloud Gateway for routing my microservice, but even if the microservice is working expectedly gateway routing returns an empty response.
My microservice is a simple application and it's running on port 5861. I routed my Gateway to it simply by predicting all cases to be sure it routed, after my routing trials with specific paths.
That's my Gateway configuration file:
spring:
cloud:
gateway:
routes:
- id: product_service
uri: localhost:5861/
predicates:
- Path=/product-service/**
After running both service and hitting them, my microservice returns response properly:
$ curl -v http://localhost:5861/product/
* Trying 127.0.0.1:5861...
* Connected to localhost (127.0.0.1) port 5861 (#0)
> GET /product/ HTTP/1.1
> Host: localhost:5861
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200
< Content-Type: application/json
< Transfer-Encoding: chunked
< Date: Wed, 01 Jun 2022 19:41:40 GMT
<
* Connection #0 to host localhost left intact
[{"id":1,"name":"Apple","price":2},{"id":2,"name":"Apple","price":2},{"id":3,"name":"Apple","price":2}]
But if I try to do this from my API gateway it returns nothing, if I try to reach it from my browser a blank page occurs.
$ curl -v http://localhost:5860/product-service/product
* Trying 127.0.0.1:5860...
* Connected to localhost (127.0.0.1) port 5860 (#0)
> GET /product-service/product HTTP/1.1
> Host: localhost:5860
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 0
<
* Connection #0 to host localhost left intact
Can someone help me with what I am missing? I want to get the same response from the gateway.
It seems like your application has /product/ endpoint exposed.
From gateway, you are trying to redirect from /product-service/product to /product/ of service.
By default, spring cloud gateway redirects to Uris as they are. So currently, I believe that http://localhost:5860/product-service/product is being redirected to http://localhost:5861/product-service/product.
If you need to redirect from product-service/** to /** of product-service then use RewritePath filter.
here is an example usage that may work for you:
spring:
cloud:
gateway:
routes:
- id: product_service
uri: localhost:5861/
predicates:
- Path=/product-service/**
filters:
- RewritePath=/product-service/(?<segment>/?.*),$\{segment}
Working update:
spring:
cloud:
gateway:
routes:
- id: product_service
uri: http://localhost:5861/
predicates:
- Path=/product
I have a spring boot application which needs to serve range requests for resuming downloads. Things works fine when this spring boot app is hosted in local windows machine and "Accept-Ranges" header is set to "bytes" by default. But when the same application is hosted in a remote virtual machine running windows, "Accept-Ranges" header is set to "none" by default and resuming download is not working. What can be the issue here?
>curl -I http://localhost:8080
HTTP/1.1 200
Accept-Ranges: bytes
Content-Type: application/json
Content-Length: 36121548
Date: Tue, 14 Sep 2021 13:00:19 GMT
>curl -I http://{IPofRemoteVM}:8080
HTTP/1.1 200
Accept-Ranges: none
Content-Type: application/json
Content-Length: 36121548
Date: Tue, 14 Sep 2021 13:00:23 GMT
I have a containerized Java Spring Boot web app deployed using Azure App Service. I have enabled Actuator Health Check endpoint which is accessible at https:///actuator/health.
I enabled Health Check for the service and it cribs that 'Your app is unhealthy'. Following statements keep on appearing in the log:
2021-07-26T06:52:44.592883929Z 172.16.0.1 - [26/Jul/2021:06:52:44 +0000] GET HTTP/1.1 "GET /actuator/health HTTP/1.1" 302 154 0.000 "-" "HealthCheck/1.0" "-"
2021-07-26T06:52:45.436822668Z 172.16.0.1 - [26/Jul/2021:06:52:45 +0000] GET HTTP/1.1 "GET /actuator/health HTTP/1.1" 302 154 0.000 "-" "ReadyForRequest/1.0 (HealthCheck)" "-"
2021-07-26T06:52:51.107134976Z 172.16.0.1 - [26/Jul/2021:06:52:51 +0000] GET HTTP/1.1 "GET /actuator/health HTTP/1.1" 302 154 0.000 "-" "HealthCheck/1.0" "-"
2021-07-26T06:52:51.124Z WARN - Container for <instance> site <application-name> is unhealthy, recycling site.
I have set 'HTTPS Only' in the TLS/SSL settings as well. When I ran 'Diagnose and solve problems' it didn't give any problems with Health Check.
What should I do?
Thanks.
The Healthcheck URL needs to return a success status code (200-299), otherwise App Services will consider it failed. Also, it works only on HTTP, iirc.
If your app has a redirect to https built-in you should make sure to whitelist the Healthcheck user agent so the healthcheck url does not redirect and returns a success status code instead.
I've been deploying a Java application and trying to push it to the Cloud using the PaaS Cloud Foundry but it seems like one of the routes can't be accessed when I am trying to push my application.
Also, I am using Anynines (https://www.anynines.com/) for the Cloud Foundry part (and I checked, this is not an authentication problem).
The exact error log I get when putting the verbose mode on is :
REQUEST: [2020-07-27T14:48:16+02:00]
GET /v2/routes/reserved/domain/21d14133-2acd-462e-84ff-2a0d56bbd9ae?host=logicielgestionformations HTTP/1.1
Host: api.de.a9s.eu
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
User-Agent: cf/6.51.0+2acd15650.2020-04-07 (go1.13.8; amd64 windows)
RESPONSE: [2020-07-27T14:48:16+02:00]
HTTP/1.1 404 Not Found
Connection: keep-alive
Content-Length: 86
Content-Type: application/json;charset=utf-8
Date: Mon, 27 Jul 2020 12:48:15 GMT
Keep-Alive: timeout=20
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 5eb75dbe-96ec-40df-61ec-ac37b158d47c::5a0552f3-22cb-4c76-b234-c72b29010a1e
{
"code": 10000,
"description": "Unknown request",
"error_code": "CF-NotFound"
}
I've been searching everywhere I could to get an answer to my problem but no one seems to have the same as me...
If you could help me that would be extremely nice of you :)
EDIT :
I have found the problem, I only had to remove the "-" at the beginning of my manifest.yml, as it was written here : https://docs.cloudfoundry.org/devguide/deploy-apps/manifest-attributes.html (Note: If your app name begins with the dash character (-), you cannot interact with the app using the cf CLI. This is because the cf CLI interprets the dash as a flag).
Now I have another problem to solve though : it seems like I have to choose a buildpack...
Error staging application: An app was not successfully detected by any available buildpack
Regards,
Déborah Jabès
I have found the problem, I only had to remove the "-" at the beginning of my manifest.yml, as it was written here : https://docs.cloudfoundry.org/devguide/deploy-apps/manifest-attributes.html (Note: If your app name begins with the dash character (-), you cannot interact with the app using the cf CLI. This is because the cf CLI interprets the dash as a flag).