We Keep Coding

Google Cloud Kubernetes Load Balancer Service: ERR_CONNECTION_REFUSED

I have this problem that's driving me insane. I have two deployment and two service yaml files created by kompose convert from a docker-compose. The app that I'm trying to run in Google Cloud is a Spring Boot web app with a mariadb backend. After I apply the four yamls with kubectl, I expose the frontend deployment (on port 8081) by running

TL;DR for anyone coming to this question via search:
OP's service was a ClusterIP and not LoadBalancer. Setting this as LoadBalancer still did not fix issue. Checking logs of pod determined code was unable to connect to DB, so never actually started up successfully.
Output from OP:
kubectl get svc -n default
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.72.0.1 <none> 443/TCP 147m
load-balancer LoadBalancer 10.72.15.246 34.69.204.138 80:30870/TCP 86s
mysqldb ClusterIP 10.72.3.186 <none> 3308/TCP 3m20s
web-app ClusterIP 10.72.13.41 <none> 8081/TCP 3m19s
Please share those in your question, making sure to prepend "```" before and after each yaml to preserve formatting
Now, a few points.
You specified you have service yamls. If you have a yaml that describes a service such as a LoadBalancer, you shouldn't need to kubectl expose afterwards, as your yaml should have done that for you.
Assuming the service named "load-balancer" is the one you have created via your yamls, the IP:port combination you should be using is 34.69.204.138:80. What IP have you been trying to access? Are you trying to access this IP and port? Or a different one?
UPDATE
Based on the pasted yamls, I see this:
In your docker-compose yaml:
web-app:
build: .
image: mihaialexandruteodor/featherwriter
ports:
- "8081:8081"
expose:
- "8081"
This is exposing port 8081 and connecting it to the underlying container.
This is reflected in the service yaml:
apiVersion: v1
kind: Service
metadata:
...
name: web-app
spec:
ports:
- name: "8081"
port: 8081
targetPort: 8081
selector:
io.kompose.service: web-app
status:
loadBalancer: {}
However, I do not see a service called "web-app" in your listing. It's possible therefore, you may have deployed it into a different namespace.
Try kubectl get svc --all-namespaces and see where the service "web-app" is. Find the IP from that, the port should be 8081 and you can then do x.x.x.x:8081 to access the service.
UPDATE 2
The web-app service is of type ClusterIP (documentation) which cannot be accessed outside of the cluster, you need to change the service to be a LoadBalancer type, or use port-forwarding.
To make the service a LoadBalancer, change the service yaml as follows (documentation here):
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: kompose convert
kompose.version: 1.26.1 (HEAD)
creationTimestamp: null
labels:
io.kompose.service: mysqldb
name: mysqldb
spec:
ports:
- name: "3308"
port: 3308
targetPort: 3306
selector:
io.kompose.service: mysqldb
type: LoadBalancer
This will provision a service that will have an external IP you can use.
Alternatively, use port forwarding to connect a local port with the port being listened on by the service:
kubectl port-forward -n {namespace} svc/web-app 8081:8081
Then you can use localhost:8081 to connect to your service. This option does not require an externally-accessible endpoint, but you will need to run the port forward command (and have it active) each time you want to access the service via the localhost endpoint.
If you want to be able to access the service from somewhere outside of your cluster, that is not your local machine, and is not within the same cluster, you will need to use a LoadBalancer service type.
UPDATE 3
Right, I can't build that Dockerfile as I do not have the src folder, but I can run the image from mihaialexandruteodor/featherwriter, and can see it is indeed listening on 8081
Tomcat initialized with port(s): 8081 (http)
so the next thing to see is see if there's any issues with the pod functionality itself. First check the pod status:
kubectl get pods -n {namespace}
The pod should be called web-app-xxxxx where xxxxx is a random sequence of letters and numbers.
Is the web-app pod running? Does it have a restart counter that is not zero like some of the pods in my prometheus namespace:
$ kubectl get pods -n prometheus
NAME READY STATUS RESTARTS AGE
alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 3h51m
prometheus-grafana-66cb8bcf4f-428d8 3/3 Running 0 3h51m
prometheus-kube-prometheus-operator-749fc8899b-dnvft 1/1 Running 0 3h51m
prometheus-kube-state-metrics-77698656df-btq4k 1/1 Running 20 3h51m
prometheus-prometheus-kube-prometheus-prometheus-0 2/2 Running 0 3h51m
prometheus-prometheus-node-exporter-jj9z5 1/1 Running 30 3h51m
prometheus-prometheus-node-exporter-lbk6p 1/1 Running 0 3h51m
prometheus-prometheus-node-exporter-vqfhk 1/1 Running 20 3h51m
Next get the logs from the pods like so:
kubectl logs -n {namespace} web-app-xxxxx
See if you can find any errors.
My hunch, given that we've connected everything through on 8081 and Tomcat is indeed running on 8081, is that the spring app is crashing repeatedly and Kubernetes is restarting it, the app then fails again, and it tries again over and over, eventually failing into a CrashLoopBackOff state where Kubernetes will delay restarting by a longer period.

Name resolution failure with Spring Gateway and Eureka

Good day,
At this moment I am working on a very simple gateway that (for now) only needs to redirect incoming HTTP POST and GET requests.
THE SETUP:
The Eureka Server: the location where my Spring Boot microservices are registered;
The Spring Gateway: maps all incoming HTTP POST and GET requests and routes them to the proper microservice;
The Spring Boot microservices: doing just some thingies as requested :)
Note: I'm kinda new to this gateway stuff, just you know :).
The microservice is registered fine with the Eureka server. Its webbased GUI shows me that the instance "MY-MICRO-SERVICE" is registered with the Eureka server. Other (Spring Boot) services can use that name ("MY-MICRO-SERVICE") without issues, so for them it works fine. Just this gateway can't handle the instance name; it seems it only accepts IP addresses (which I just want to prevent, as the microservice can change from servers and therefor their IP address). And the Eureka server is not configured to only allow/use IP addresses.
THE ISSUE:
All runs smooth when the Gateway has a route that holds an IP address of the microservice. But what I want is to let the Gateway resolve the service ID from the Eureka server. And if I do that, it throws me a java.net.UnknownHostException: MY-MICRO-SERVICE: Temporary failure in name resoultion.
THE QUESTION:
Now why can't I use the name of the Spring application "MY-MICRO-SERVICE" (being the registered Spring Boot microservice) in the Spring Gateway (while that construction works fine when used in other microservices)? Can't a Yaml config file handle such instance names, just only IP addresses?
THE DETAILS
The gateway is mostly configured via a yaml config file. There is only one simple Java class that kicks off the gateway application. The routing is all set in the yaml config file.
The Spring Gateway application class
#SpringBootApplication
#EnableEurekaClient
public class MyGatewayApplication {
public static void main(String[] args) {
SpringApplication.run(MyGatewayApplication.class, args);
}
}
The Gateway Yaml configuration file (application.yml)
spring:
application:
name: my-gateway
cloud:
gateway:
discovery:
locator:
lowerCaseServiceId: true
enabled: true
globalcors:
corsConfigurations:
'[/**]':
allowedOrigins: "*"
allowedMethods:
- GET
- POST
routes:
- id: my_route
uri: http://MY-MICRO-SERVICE
predicates:
- Path=/test/**
server:
port: 8999
info:
app:
properties: dev
The Error
java.net.UnknownHostException: MY-MICRO-SERVICE: Temporary failure in name resolution
at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[na:na]
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929) ~[na:na]
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515) ~[na:na]
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848) ~[na:na]
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505) ~[na:na]
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364) ~[na:na]
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298) ~[na:na]
at java.base/java.net.InetAddress.getByName(InetAddress.java:1248) ~[na:na]
at io.netty.util.internal.SocketUtils$8.run(SocketUtils.java:146) ~[netty-common-4.1.36.Final.jar:4.1.36.Final]
...

Issue has been fixed.
I changed the "http" to "lb" protocol and that fixed my issue. To my understanding, "lb" stands for LoadBalancing. I have no loadbalancer active on my local machine, but anyway: this works.
- POST
routes:
- id: my_route
uri: lb://MY-MICRO-SERVICE
predicates:
- Path=/test/**

Run/curl simple Java application deployed and exposed to Kubernetes cluster hosted on AWS

I am newbie to Kubernetes and had a long time configuring my application to be hosted on Kubernetes cluster hosted on AWS EKS.
Status-quo: I am pretty sure that the service of type LoadBalancer is up and running. It has its pod and all the stuff running. The application is simple Java application with input. You can try accessing it by pulling an image from Docker Hub via:
docker run -i ardulat/mckinsey
Question: how can I run the Java application (not Spring, not REST) that is being hosted on Kubernetes cluster?
Already tried:
curl -v <EXTERNAL-IP>:<PORT> that outputs:
* Trying 3.134.148.191...
* TCP_NODELAY set
* Connected to a8154210d09da11ea9c3806983848f2f-1085657314.us-east-2.elb.amazonaws.com (3.134.148.191) port 8080 (#0)
> GET / HTTP/1.1
> Host: a8154210d09da11ea9c3806983848f2f-1085657314.us-east-2.elb.amazonaws.com:8080
> User-Agent: curl/7.63.0
> Accept: */*
>
* Empty reply from server
* Connection #0 to host a8154210d09da11ea9c3806983848f2f-1085657314.us-east-2.elb.amazonaws.com left intact
curl: (52) Empty reply from server
nc -v <EXTERNAL-IP> <PORT> that outputs:
found 0 associations
found 1 connections:
1: flags=82<CONNECTED,PREFERRED>
outif en0
src 172.20.22.42 port 63865
dst 3.13.128.24 port 8080
rank info not available
TCP aux info available
Connection to a8154210d09da11ea9c3806983848f2f-1085657314.us-east-2.elb.amazonaws.com port 8080 [tcp/http-alt] succeeded!
Therefore, I assume that connection works and the service is up and running except I am trying to connect to the Java (.jar) application in the wrong way. Do you have any suggestions?

You should change your dockerfile and change CMD to ENTRYPOINT which is nicely explained here.
I would also recommend reading Define a Command and Arguments for a Container.
CMD sets default command and/or parameters, which can be overwritten from command line when docker container runs.
ENTRYPOINT configures a container that will run as an executable.
Your dockerfile might look like this:
FROM java:8
WORKDIR /
ADD Anuar.jar Anuar.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","Anuar.jar"]
Your service might look like this:
apiVersion: v1
kind: Service
metadata:
name: javaservice
labels:
app: javaservice
spec:
type: LoadBalancer
selector:
app: javaservice
ports:
- protocol: TCP
port: 8080
name: http
Also it's important which LoadBalancer you want to use as on AWS there is Classic Load Balancer which is default and Network Load Balancer. You can read more about it on Internal load balancer and check the AWS documentation for Load Balancing.
Amazon EKS supports the Network Load Balancer and the Classic Load Balancer through the Kubernetes service of type LoadBalancer. The configuration of your load balancer is controlled by annotations that are added to the manifest for your service.
By default, Classic Load Balancers are used for LoadBalancer type services. To use the Network Load Balancer instead, apply the following annotation to your service:
service.beta.kubernetes.io/aws-load-balancer-type: nlb
For more information about using Network Load Balancer with Kubernetes, see Network Load Balancer support on AWS in the Kubernetes documentation.
By default, services of type LoadBalancer create public-facing load balancers. To use an internal load balancer, apply the following annotation to your service:
service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
For internal load balancers, your Amazon EKS cluster must be configured to use at least one private subnet in your VPC. Kubernetes examines the route table for your subnets to identify whether they are public or private. Public subnets have a route directly to the internet using an internet gateway, but private subnets do not.

How to access postgresql pod/service in google kubernetes

I am deploying a simple web application. I divided it into 3 pods:front end, back end, and postgres db. I successfully deployed my front end and back end to google kubernetes service and they works as expected. But for my postgresql db server, I used the following yamls. The postgres image is created by me using standard postgres images from dockerhub. I created some tables, and inserted some data and pushed to DockerHub. My backend is not able to make connection to my db. I think I might need to change my Java connection code.Not sure on using localhost. It works without any issue on my local Eclipse Jee and Tomcat.
//my pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: postgres-app-pod
labels:
name: postgres-app-pod
app: demo-geo-app
spec:
containers:
- name: postgres
image: myrepo/example:v1
ports:
- containerPort: 5432
//my service.yaml
apiVersion: v1
kind: Service
metadata:
name: db
labels:
name: db-service
app: demo-geo-app
spec:
ports:
- port: 5432
targetPort: 5432
selector:
name: postgres-pod
app: demo-geo-app
//from my java backend, I access my db server this way.
String dbURL = "jdbc:postgresql://localhost:5432/Location?user=postgres&password=mysecretpassword";

there are two issues to be fixed.
the service selector key:value should match with pod labels
replace localhost with postgresql service dns

As per comments you have an error on your connection string, localhost is referring to the same pod where you are running the Java code, you need to change to db as the same name you put on the service yaml to work.
I recommend you to use a deployment instead of Pod on the type, but in that case that you are trying to deploy a database you need to use a StatefulSet please review the documentation
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
Also, I recommend you to check https://helm.sh you have a lot of chart ready to use instead of having to code from scratch a service like a database.
https://github.com/helm/charts/tree/master/stable/postgresql
On that chart you have all the necessary yaml ready including the PVC provisioning.

How to make Spring Cloud Consul register with external IP?

How to make Spring Cloud Consul register with external IP?
I run spring boot applications with spring cloud consul in docker.
The application is running inside a docker network on a docker swarm.
This means that if spring consul registers with the hostname or the IP address it will get an hostname or ip address that is internal to the docker swarm. My consul server is outside the docker swarm.

You can set spring.cloud.consul.discovery.ipAddress externally and set the property spring.cloud.consul.discovery.preferIpAddress=true.
Alternatively, you can control which network interfaces to derive an ip address from.
In Brixton you can ignore interfaces:
spring:
cloud:
inetutils:
ignoredInterfaces:
- docker0
- veth.*
In Camden (currently RC1) you can also whitelist networks:
spring:
cloud:
inetutils:
preferredNetworks:
- 192.168
- 10.0

Run the container with -p 12345:12345 to publish the port so that you can access the container from your host machine through the specified port