We Keep Coding

Cannot override properties in Hazelcast.yaml for Spring Boot Application

I have Spring Boot application with Hazelcast (use 4.2 and also try to use 4.0.3 of hazelcast-all dependency). Hazelcast is configured for embedded topology. I use TCP-IP network join way. Properties for Hazelcast are set using file Hazelcast.yaml and Spring Boot property: spring.hazelcast.config (Spring by default use this name for Hazelcast config, it is redundant).
In property member-list I indicate IP addresses of two machines in one subnet (e.g. 192.0.0.1 and 192.0.0.2).
I build application in Docker using image based on Alpine on OpenJdk. Image includes start java -jar command as ENTRYPOINT.
PROBLEM PREMISE:
I run two docker containers on two machines described earlier. I forward only port 5701 (using -p) on both containers. And containers don't see each other. Spring Boot logs show that the container network is being used.
PS:
All work if run docker with --net host.
Also all work if I package Spring Boot application with property public-address in Hazelcast.yaml for two containers - once package with value 192.0.0.1, the other with value 192.0.0.2. Spring Boot Hazelcast instances see each other using network of machine (192.0.0.1 and 192.0.0.2).
PROBLEM:
I try to override property public-address in Hazelcast.yaml using:
docker run -e HZ_NETWORK_PUBLICADDRESS=192.0.0.1
export HZ_NETWORK_PUBLICADDRESS=192.0.0.1 && docker run
JAVA_OPTS="-Dhz.network.public-address=192.0.0.1"
JAVA_OPTS="-Dhazelcast.local.publicAddress=192.0.0.1"
JAVA_OPTS="-Dhazelcast.config=/mnt/overrided_hazelcast.yaml"
ENV HZ_NETWORK_PUBLICADDRESS=192.0.0.1 - in Dockerfile
ENTRYPOINT java -jar -Dhz.network.public-address=192.0.0.1 my-app.jar
Nothing works. Does anyone know why it is not possible to override property public-address in the Hazelcast.yaml at startup?
Or maybe anyone knows how I can run two Spring Boot applications with embedded Hazelcast in separate Docker containers on separate machines.

The way you set up Hazelcast public address is correct, at least starting from Hazelcast 4.1 where the Config Override feature was added.
To check the working version, you can have a look at Hazelcast Guide: Embedded Hazelcast on Kubernetes. Instead of Hazelcast Kubernetes configuration, you can use TCP-IP. The following Hazelcast configuration worked for me (my host IP is 172.22.41.210).
hazelcast:
cluster-name: hazelcast-cluster
network:
join:
tcp-ip:
enabled: true
member-list:
- 172.22.41.210:5701
- 172.22.41.210:5702
Then, building and starting two applications should form a cluster.
$ mvn package && docker build -t hazelcast-embedded .
$ docker run --rm -e HZ_NETWORK_PUBLICADDRESS=172.22.41.210 -p 5701:5701 hazelcast-embedded
$ docker run --rm -e HZ_NETWORK_PUBLICADDRESS=172.22.41.210:5702 -p 5702:5701 hazelcast-embedded
You should see that the cluster was formed in the application logs.
Members {size:2, ver:2} [
Member [172.22.41.210]:5701 - 21af9e1a-7e98-4305-905c-451ee23486c3 this
Member [172.22.41.210]:5702 - 0507d970-1f31-4df3-9ea5-8c3981eb7c98
]

Running Spring Boot docker instance with Postgres docker instance

I'm attempting to run a Spring Boot app that connects a Postgres DB using:
docker-compose.yml (for Postgres) :
version: '3'
services:
postgres-db:
container_name: postgres-db
image: postgres:latest
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_USER: my_user
POSTGRES_PASSWORD: my_password
POSTGRES_DB: shorten-db
To run the Postgres DB:
docker-compose up
.Dockerfile (for the Spring Boot app) :
FROM openjdk:12-jdk-alpine
RUN addgroup -S spring && adduser -S spring -G spring
USER spring:spring
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
In order to run the Spring app using Docker I use:
mvn package
docker build -t url-shorten/url-shorten-docker .
docker run -p 8080:8080 url-shorten/url-shorten-docker
But I receive the error when starting when running above docker command:
Caused by: org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
In Spring application.properties I connect to the DB using:
spring.datasource.url=jdbc:postgresql://localhost:5432/shorten-db
I think this error is due to the Spring Boot app is running in a different container to DB so it cannot find the DB on localhost. Is there an idiomatic way of connecting the Spring Book docker container to the DB container. Or do I have do access the IP address of my machine and use this address to connect to the Postgres DB running on Docker?

Yes, you can't use localhost in this situation
spring.datasource.url=jdbc:postgresql://postgres-db:5432/shorten-db

In Spring application.properties, try to change DB config to:
spring.datasource.url=jdbc:postgresql://postgres-db:5432/shorten-db
In container networks, You need to use the container name as a host.

You can add both DB and app containers to one Docker network and change PostgreSQL host in datasource URL to postgres-db. Then Spring app will work with your DB.

Spring boot running as container issues

I am running a spring boot with REST API inside a docker container. Everything seems to work fine when i run from eclipse or as jar. But when i dockerize it and run i am facing below issues
First
Not able to access REST Endpoint within container.
http://localhost:9000/ --> works But
http://localhost:9000/api/v1/test --> it does not identify.
However i can run it from swagger.
Second issue org.postgresql.util.PSQLException: ERROR: permission denied for schema < schema_name >
However i have given all permissions for the schema like
GRANT ALL ON SCHEMA < schemaname> TO < username>;
GRANT USAGE ON SCHEMA < schemaname> TO < username>;
These issues are only when i try to run from a container.
Commands use for docker
docker run -p 9000:9000 < image name >
Am using spring boot 2.1.9
Dockerfile
FROM openjdk:8-jdk-alpine
VOLUME /tmp
ADD run-app.sh run-app.sh
RUN chmod +x run-app.sh
EXPOSE 9000
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} dummy.jar
ENTRYPOINT ./run-app.sh
run-app.sh
java $JAVA_OPTS -jar /dummy.jar
My postgresql Db is running in aws.
My spring boot is able to start, but only while my API is querying i am facing the exception

Can you share dockerfile content if possible, would like to see what commands you have given with endpoints. and for postgresql which you are using, is it embeded in docker with spring boot app or another server?
I do already setup spring boot with nginx and postgresql dockerized but in different servers/containers which working pretty smooth on production.

For first issue I will need more details.
For second issue, the problem is two containers are not on the same network and hence service container can't communicate with psql container. You can actually create a docker-compose.yml file to run those on same network. Or create a network and join the containers on that network.

Dockerized Mac/Java app can't talk to localhost

MacOS + Docker (Version 17.12.0-ce-mac49 (21995)) here. I am trying to Dockerize an existing Spring Boot app. Here's my Dockerfile:
FROM openjdk:8
RUN mkdir /opt/myapp
ADD build/libs/myapp.jar /opt/myapp
ADD application.yml /opt/myapp
ADD logback.groovy /opt/myapp
WORKDIR /opt/myapp
EXPOSE 9200
ENTRYPOINT ["java", "-Dspring.config=.", "-jar", "myapp.jar"]
Here's my Spring Boot application.yml config file. As you can see it expects Docker to inject environment variables from an env file:
logging:
config: 'logback.groovy'
server:
port: 9200
error:
whitelabel:
enabled: true
spring:
cache:
type: none
datasource:
driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://${DB_HOST}:3306/myapp_db?useSSL=false&nullNamePatternMatchesAll=true
username: ${DB_USERNAME}
password: ${DB_PASSWORD}
testWhileIdle: true
validationQuery: SELECT 1
jpa:
show-sql: false
hibernate:
ddl-auto: none
naming:
physical-strategy: org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
implicit-strategy: org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
properties:
hibernate.dialect: org.hibernate.dialect.MySQL5InnoDBDialect
hibernate.cache.use_second_level_cache: false
hibernate.cache.use_query_cache: false
hibernate.generate_statistics: false
hibernate.hbm2ddl.auto: validate
myapp:
detailsMode: ${DETAILS_MODE}
tokenExpiryDays:
alert: 5
jwtInfo:
secret: ${JWT_SECRET}
expiry: ${JWT_EXPIRY}
topics:
adminAlerts: admin-alerts
Here's my myapp-local.env file:
DB_HOST=localhost
DB_USERNAME=root
DB_PASSWORD=
DETAILS_MODE=Terse
JWT_SECRET=12345==
JWT_EXPIRY=86400000
It's worth noting that above in the env file, I have tried localhost, 127.0.0.1 and 172.17.0.1 and all of them produce identical errors below.
Then I build the container:
docker build -t myapp .
Success! Then I run the container:
docker run -it -p 9200:9200 --net="host" --env-file myapp-local.env --name myapp myapp
...and I watch as the container quickly dies with MySQL connection-related exceptions (can't connect to the MySQL machine running locally). I can confirm that the Spring Boot app has no problem connecting to MySQL when it runs as an executable ("fat") jar outside of Docker, and I can confirm that the local MySQL instance is up and running and is perfectly healthy.
Unable to connect to database. }com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:590)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:57)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:1606)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:633)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:347)
When I turn TRACE-level logging on, I see it is trying to connect to:
url=jdbc:mysql://localhost:3306/myapp?useSSL=false&nullNamePatternMatchesAll=true
So it does look like Docker is properly injecting the env file's vars into the Spring YAML-based config. So this doesn't feel like a config issue, moreover an isse with the container speaking to the MySQL port running on the Docker host.
Can anybody see where I'm going awry?

Accessing the host machine from within a container is not recommended. Usually it can be solved by wrapping service you need into a container and accessing it via container name.
There is no solution, there are only workarounds, you can use one of them:
On Mac you can access the host services using docker.for.mac.host.internal DNS name.
You need to set environment variable like this:
DB_HOST=docker.for.mac.host.internal
And refer to the DB_HOST from your connection string.
For more details see the documentation:
From 17.12 onwards our recommendation is to connect to the special
Mac-only DNS name docker.for.mac.host.internal, which resolves to the
internal IP address used by the host.
Note: Having --net="host" doesn't let you reach the host machine via localhost. localhost always points to local machine, but in case if it is invoked from within a container it points to the container itself.

So basically Docker app is not in the same network as the host you're running it from and that's why you can't access MySQL by pointing to localhost (because this is another network from Docker's point of view).
What you could try is to run docker with --net="host" option and then it will share the network with its host.
You can find better explanation on this issue in this topic From inside of a Docker container, how do I connect to the localhost of the machine?

How to override embedded DNS server for Docker in /etc/resolv.conf from a docker-compose file

I am using a docker-compose command to create and start my containers.
My Docker Version
docker --version
Docker version 17.09.0-ce, build afdb6d4
My Docker-Compose version
docker-compose --version
docker-compose version 1.16.1, build 6d1ac21
The .yml file that I'm using looks something like this:
(Note that I've just shortened it to take sensitive things out)
---
services:
zookeeper:
image: "zookeeper"
server-1:
cap_add:
- "NET_ADMIN"
server-0:
cap_add:
- "NET_ADMIN"
dns:
- 8.8.8.8
- 9.9.9.9
environment:
SERVER_ID: 0
NETEM_HOSTS: ""
LOSS_VALUES: ""
MAX_RATE_VALUES: ""
DELAY_VALUES: ""
image: "cloud.mycompany.com:5000/server-0:latest"
fakedns:
image: "cloud.mycompany.com:5000/fakedns:latest"
version: "3.3"
Then I start using:
docker-compose --file compose.yml up -d
My Question is this:
1) After containers come up... when I go into a container, for e.g. in this case server-0, I don't see the /etc/resolv.conf file updated to use these nameservers. Instead it uses the embedded dns of docker which is 127.0.0.11
2) How do I make sure that it uses what I specify in file that is used by docker-compose
3) I tried to do this with the command and it seems to work, but I need to do from compose-file
docker run -p 4000:53 --dns=8.8.8.8 cloud.mycompany.com:5000/server-0:latest
4) Ideally, I want it to have the IP address of the container 'fakedns' so that it uses this one instead of the embedded one #127.0.0.11

You won't see custom DNS servers in /etc/resolv.conf but Docker's resolver will forward DNS requests to them.
User Defined Networks and DNS
Docker compose definitions that are v2+ create a user defined network by default.
Docker with a user defined network uses an embedded DNS server so that Docker can respond for local container requests (service discovery).
For any DNS hosts Docker can not resolve, the request will be forwarded onto a DNS server. This is either the system default server, the server configured in dockerd or the DNS server configured for the container at run time.
Docker DNS
Be careful when using internal DNS servers. Things in the Docker daemon will break if you point the systems DNS at a container as you create a chicken or the egg problem, Docker needs DNS to start but can't start the container to provide DNS.
As your example config is only setting the DNS for one app container it should be ok, but make sure the DNS container is up and healthy before your application.