We Keep Coding

How to send Kafka message to multiple client racks?

I have a Spring Boot Application that uses kafka to produce and consume messages to/from other applications.
I implemented a new producer whose messages should be sent to different clients located in different development servers. Kafka configuration it's specificated in the application.yml of the project. This was the previous configuration:
spring:
kafka:
bootstrap-servers: server.a:port
producer:
properties:
client.rack: server.a
consumer:
clientId: a-client-id
groupId: a-group-id
properties:
client.rack: server.a
jaas:
options:
username: an-username
password: a-password
Now with the new producer I need to produce message to a second server, server.b so:
spring:
kafka:
bootstrap-servers:
- server.a:port
- server.b:port
producer:
properties:
client.rack:
- server.a
- server.b
consumer:
clientId: a-client-id
groupId: a-group-id
properties:
client.rack: server.a
jaas:
options:
username: an-username
password: a-password
However, this seems to be sending the produced messages to server.b only.
I'm not sure if my config it's wrong or not. As far as I read this seems to be the properly way of doing it but, obviously, I did something wrong because it's not working. Bit lost here.

This isn't how Kafka works. Clients always send to the leader broker. Leaders can exist on any rack, and clients cannot control where they send data other than that leader first, and then configure acks to allow for replication to other followers on other racks.
Also, client.rack is a string, not a list.

Ingress not working on Kubernetes Infrastructure

I created a microservices infrastructure on Kubernetes (version: 1.20.9-gke.1001) on Google Cloud Platform using the Spring Cloud.
First I created the following deployments: Eureka (service discovery), Zuul (API Gateway), Zipkin (Distributed tracing system), User Service and Auth Service.
Then I created the following services: eureka-service with “Cluster IP” type which allows other pods to connect to Eureka, zipkin-service with “Cluster IP” type which allows other pods to connect to Zipkin and loadbalancer-service with “External load balancer” type which is connected to the Zuul.
Finally I tried to create an Ingress using the attached yaml file but at every request I tried to execute, I received the following error: “response 404 (backend NotFound), service rules for the path non-existent”. While if I try to invoke the APIs using the external IP of the loadbalancer-service the backend works correctly.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: project-ingress
annotations:
kubernetes.io/ingress.class: "gce"
spec:
rules:
- host: project.test.com
http:
paths:
- path: /*
pathType: ImplementationSpecific
backend:
service:
name: loadbalancer-service
port:
number: 8765
Do you have any idea why the Ingress is not working?
Also I would need to expose the services with HTTPS, could you kindly explain to me how to use an existing security certificate in the Ingress?
Thanks, this is my first experience with Kubernetes and of course any advice on how to improve the infrastructure is welcome.

Name resolution failure with Spring Gateway and Eureka

Good day,
At this moment I am working on a very simple gateway that (for now) only needs to redirect incoming HTTP POST and GET requests.
THE SETUP:
The Eureka Server: the location where my Spring Boot microservices are registered;
The Spring Gateway: maps all incoming HTTP POST and GET requests and routes them to the proper microservice;
The Spring Boot microservices: doing just some thingies as requested :)
Note: I'm kinda new to this gateway stuff, just you know :).
The microservice is registered fine with the Eureka server. Its webbased GUI shows me that the instance "MY-MICRO-SERVICE" is registered with the Eureka server. Other (Spring Boot) services can use that name ("MY-MICRO-SERVICE") without issues, so for them it works fine. Just this gateway can't handle the instance name; it seems it only accepts IP addresses (which I just want to prevent, as the microservice can change from servers and therefor their IP address). And the Eureka server is not configured to only allow/use IP addresses.
THE ISSUE:
All runs smooth when the Gateway has a route that holds an IP address of the microservice. But what I want is to let the Gateway resolve the service ID from the Eureka server. And if I do that, it throws me a java.net.UnknownHostException: MY-MICRO-SERVICE: Temporary failure in name resoultion.
THE QUESTION:
Now why can't I use the name of the Spring application "MY-MICRO-SERVICE" (being the registered Spring Boot microservice) in the Spring Gateway (while that construction works fine when used in other microservices)? Can't a Yaml config file handle such instance names, just only IP addresses?
THE DETAILS
The gateway is mostly configured via a yaml config file. There is only one simple Java class that kicks off the gateway application. The routing is all set in the yaml config file.
The Spring Gateway application class
#SpringBootApplication
#EnableEurekaClient
public class MyGatewayApplication {
public static void main(String[] args) {
SpringApplication.run(MyGatewayApplication.class, args);
}
}
The Gateway Yaml configuration file (application.yml)
spring:
application:
name: my-gateway
cloud:
gateway:
discovery:
locator:
lowerCaseServiceId: true
enabled: true
globalcors:
corsConfigurations:
'[/**]':
allowedOrigins: "*"
allowedMethods:
- GET
- POST
routes:
- id: my_route
uri: http://MY-MICRO-SERVICE
predicates:
- Path=/test/**
server:
port: 8999
info:
app:
properties: dev
The Error
java.net.UnknownHostException: MY-MICRO-SERVICE: Temporary failure in name resolution
at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[na:na]
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929) ~[na:na]
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515) ~[na:na]
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848) ~[na:na]
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505) ~[na:na]
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364) ~[na:na]
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298) ~[na:na]
at java.base/java.net.InetAddress.getByName(InetAddress.java:1248) ~[na:na]
at io.netty.util.internal.SocketUtils$8.run(SocketUtils.java:146) ~[netty-common-4.1.36.Final.jar:4.1.36.Final]
...

Issue has been fixed.
I changed the "http" to "lb" protocol and that fixed my issue. To my understanding, "lb" stands for LoadBalancing. I have no loadbalancer active on my local machine, but anyway: this works.
- POST
routes:
- id: my_route
uri: lb://MY-MICRO-SERVICE
predicates:
- Path=/test/**

Springboot Microservice is registered with Eureka but cannot hit it

Basic question so just want to ensure I understand it all correctly.
I have created a discovery server:
#SpringBootApplication
#EnableEurekaServer
public class DisocveryServiceApplication {
public static void main(String[] args) {
SpringApplication.run(DisocveryServiceApplication.class, args);
}
and registered microservices with it successfully; If I hit localhost:8761 I can see my discovery service has found the microservices. The microservices run fine if i hit them on their designated port. For example, I have one called creds and if i hit localhost:9000 it returns. However, My understanding is I should now be able to hit localhost:8761/creds and it will show the same output but this isnt working.
Am I misunderstanding? Any suggestions on what I should try?
creds bootstrap.yml:
spring:
application:
name: creds
creds application.yml
server:
port: 9000
eureka:
client:
serviceUrl:
defaultZone: http://localhost:8761/eureka/
discover application.yml
server:
port: ${PORT:8761}
eureka:
client:
registerWithEureka: false
fetchRegistry: false
server:
waitTimeInMsWhenSyncEmpty: 0

Another #EnableEurekaClient annotated Spring boot webservice can access your creds webservice by using an injected RestTemplate with http://creds/..., where creds is the spring.application.name registered with Eureka.
If you want to access the creds webservice from the outside of your web application, then what you want is a proxy like Zuul http://github.com/Netflix/zuul.

Just registering micro service to Eureka server wont make sure that you can access the microservice under a gateway. Eureka Server is not a gateway server , its just a service registry. You can think Eureka as just one more service that holds information about all the other services in the cluster. It doesnt do anything extra other than getting information to the Clients registered.
You may need a Gateway service for routing your request under the Eureka Server. Zuul Proxy routes a request coming to it to the under lying microservices using its service id or the URL configured.
Add this dependency in your classpath.
dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-zuul</artifactId>
<version>1.0.4.RELEASE</version>
</dependency>
Add this config in your properties filezuul:
routes:
serv1:
path: /serv1/**
serviceId: http://localhost:8080/serv1
serv2:
path: /serv2/**
serviceId: serv2
This will create a dynamic router that routes your request to appropriate service instances. This also provides a server end load balancer for your services

Spring Cloud Turbine Amqp doesn't work

I have a problem with spring cloud: Turbine Amqp doesn't work. I exploit a local rabbit broker out of the box.
one of my services:
#EnableFeignClients(basePackages = "com.sample")
#EnableEurekaClient
#EnableAutoConfiguration
#SpringBootApplication
dependencies:
spring-cloud-config-client
spring-cloud-starter-eureka
spring-cloud-starter-feign
spring-boot-starter-actuator
spring-cloud-starter-hystrix
spring-cloud-netflix-hystrix-amqp
spring-cloud-starter-bus-amqp
bootstrap.yml:
spring:
rabbitmq:
host: 127.0.0.1
port: 5672
my turbine service:
#EnableHystrixDashboard
#EnableTurbineAmqp
#EnableEurekaClient
#EnableAutoConfiguration
#SpringBootApplication
spring-cloud-config-client
spring-cloud-starter-eureka
spring-cloud-starter-bus-amqp
spring-cloud-starter-turbine-amqp
spring-cloud-starter-hystrix-dashboard
spring:
rabbitmq:
host: 127.0.0.1
port: 5672
turbine:
amqp:
port: 8989
management:
port: 8090
when i try to execute http://127.0.0.1:8989/turbine.stream, i've got:
data: {"type":"Ping"} in cycle
rabbitmqctl list_queues - shows a new queue ...f1d5cfa119f5 0 at the same time when my "client" service is executed. when i turn it off the queue is removed.
turbine server writes:
SSE Request Received
Unsubscribing RxNetty server connection
What am i doing wrong? thanks.