I have a problem about defining database in some services in my Spring Boot Microservice example.
When I run docker-compose.yml file through this command (docker-compose up -d), I have a datasource issue in user service, advertisement service and lastly report service.
All these services have their own database defined in their own properties file under configuration folder of config server.
Here is the database part of docker-compose.yml
database:
container_name: mysql-database
image: 'mysql:latest'
ports:
- "3366:3306"
restart: always
environment:
MYSQL_DATABASE: "springbootuser"
MYSQL_USER: "springmicroserviceuser"
MYSQL_PASSWORD: "111111"
MYSQL_ROOT_PASSWORD: "111111"
volumes:
- db-data:/var/lib/mysql
networks:
backend:
aliases:
- "database"
healthcheck:
test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"]
timeout: 20s
retries: 10
How can I fix the datasource issue in all services defined in docker-compose file
Here is my docker-compose.yml : Link
Here is the user service properties file : Link
Here is the advertisement service properties file : Link
Here is the report service properties file : Link
Edited 1st :
Here is my new docker-compose.yml : Link
Here is the problem with your datasource connection port localhost:3306.
user 3366 instead of 3306.
when you define internal port on same docker network you need to use database hostname instead of localhost.
if you use localhost or IP then you need to use docker to expose a port for database connections that route from the external network.
if needed update docker compose database section:
ports:
- "3366:3306"
expose:
- "3366"
Related
I'm trying to connect my containerized spring boot application with another containerized MQTT broker. Both of them are on their own projects as follows:
mqtt docker-compose.yml:
version: '3.9'
services:
mqttbroker:
container_name: mqttbroker
restart: always
volumes:
- ./config:/mosquitto/config
- ./data:/mosquitto/data
- ./log:/mosquitto/log
ports:
- 8883:8883
networks:
- mynetwork
volumes:
config:
data:
log:
mqtt Dockerfile
FROM eclipse-mosquitto
WORKDIR /mosquitto
COPY . .
EXPOSE 8883
And then the spring boot project is like:
spring boot docker-compose.yml
version: '3.8'
services:
myapp:
build: .
container_name: myapp
ports:
- '8082:8082'
stdin_open: true
tty: true
networks:
- mynetwork
In my application.properties I try to connect to the MQTT broker like:
mosquitto.url=tcp://mqttbroker:8883 and I get connection refused. However, if I run the spring boot application locally, I can connect to the docker container with mosquitto.url=tcp://localhost:8883.
I would rather have all the configurations in my docker-compose files to decrease manual codes.
I really appreciate your help in advance!
Yes, You do have to share the network as external.
you cab check the networks you have with this command : docker network ls
You would notice that they each have their own networks.
adding this to spring boot docker-compose file should fix the issue :
networks:
default:
name: mynetwork
external: true
For more information about docker compose networks
I have a SpringBoot application that works perfectly. Now I am trying to Dockerise it, but have some issues when connecting to the database.
The application has two datasources:
application.properties
server.port= 8081
# pims datasource
spring.datasource1.driver-class-name=org.postgresql.Driver
spring.datasource1.jdbc-url=jdbc:postgresql://localhost:5432/pims
spring.datasource1.username=postgres
spring.datasource1.password=
spring.jpa.database-platform=postgres
#spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.show-sql=false
# approval datasource
spring.datasource2.driver-class-name=org.postgresql.Driver
spring.datasource2.jdbc-url=jdbc:postgresql://localhost:5432/approval
spring.datasource2.username=postgres
spring.datasource2.password=
In the application, I use Spring to access the two datasources:
#Configuration
#ComponentScan(basePackages = "com.nexct")
public class MultipleDBConfig {
#Bean(name = "datasource1")
#ConfigurationProperties("spring.datasource1")
#Primary
public DataSource dataSource1(){
return DataSourceBuilder.create().build();
}
#Bean(name = "datasource2")
#ConfigurationProperties("spring.datasource2")
public DataSource dataSource2(){
return DataSourceBuilder.create().build();
}
}
So I have created:
Dockerfile
FROM openjdk:14
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} nexct-approval-service.jar
EXPOSE 8081
ENTRYPOINT ["java","-jar","/nexct-approval-service.jar"]
docker-compose.yml
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
ports:
- "8081:8081"
db:
image: postgres
environment:
POSTGRES_DB_PORT: "5432"
POSTGRES_DB_HOST: "localhost"
POSTGRES_PASSWORD:
POSTGRES_USER: postgres
POSTGRES_DB: pims
However, I am not sure how to configure the two Postgres databases.
Any advise would be appreciated.
Note: the databases are not running in a container, as they are used by other legacy applications.
If I remove the db from docker-compose.yml. When I run docker-compose up, the Spring Boot application starts, and I can access a RESTful service. However, I get the following error:
PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
ports:
- "8081:8081"
server.port=8081
# pims datasource
spring.datasource1.driver-class-name=org.postgresql.Driver
spring.datasource1.jdbc-url=jdbc:postgresql://<postgres_host_private_ip>:5432/pims
spring.datasource1.username=postgres
spring.datasource1.password=
spring.jpa.database-platform=postgres
#spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.show-sql=false
# approval datasource
spring.datasource2.driver-class-name=org.postgresql.Driver
spring.datasource2.jdbc-url=jdbc:postgresql://<postgres_host_private_ip>:5432/approval
spring.datasource2.username=postgres
spring.datasource2.password=
Replace <postgres_host_private_ip> with the private IP of the machine hosting the postgres databases.
I'm not sure what you were trying to achieve with the nested db service, but since your database are already running somewhere outside there is no need for this.
Your Spring application will run on a isolated network from your host because of the docker isolation (unless explicitly asked to use host network with network_mode: "host"). Knowing that, the "localhost" can not work inside the container, you need to use the real IP of the machine that hosts the databases. Most likely something like 192.168.xxx.xxx
Also, you need to make sure the postgres is configured to accept connection from "non-localhost" sources (see https://www.andrew-kirkpatrick.com/2017/05/allow-connection-postgresql-server-outside-localhost/).
To access an application running on ur Host there are few options
Use the Bridge network. (Doc)
Create a bridge network and assign the subnet address and gateway address.
Now you can reach your application running at the host from the container at <gateway address>:<port> i.e 172.28.0.1:<port>
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
ports:
- "8081:8081"
networks:
hostnet: {}
networks:
hostnet:
driver: bridge
ipam:
config:
- subnet: 172.28.0.0/16
Use Attach service to host network. (Doc)
add network_mode: host to your service (only works on Linux)
OR create an external network
Now you can reach your application running at the host from the container at localhost:<port>
With network_mode: (Linux)
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
ports:
- "8081:8081"
network_mode: host
Try configuring your docker-compose file to use bridged networking:
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
environment:
POSTGRES_DB_PORT: "5432"
POSTGRES_DB_HOST: "localhost"
POSTGRES_PASSWORD:
POSTGRES_USER: postgres
POSTGRES_DB: pims
networks:
- product_service_network
networks:
product_service_network:
driver: bridge
You also have to move the ENVIRONMENT variables to product-service if you are going to use them in the product-service container.
If localhost doesn't work, use 127.0.0.1 instead.
I'm trying to set up a Spring Boot application that depends on a MySQL database called teste in docker-compose. After issuing docker-compose up, I'm getting:
Caused by: java.net.ConnectException: Connection refused (Connection refused)
I'm running on Linux Mint, my docker-compose version is 1.23.2, my Docker version is 18.09.0.
application.properties
# JPA PROPS
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.hibernate.ddl-auto=update
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
spring.jpa.hibernate.naming-strategy=org.hibernate.cfg.ImprovedNamingStrategy
spring.datasource.url=jdbc:mysql://db:3306/teste?useSSL=false&serverTimezone=UTC
spring.datasource.username=rafael
spring.datasource.password=password
spring.database.driverClassName =com.mysql.cj.jdbc.Driver
docker-compose.yml
version: '3.5'
services:
db:
image: mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=rootpass
- MYSQL_DATABASE=teste
- MYSQL_USER=rafael
- MYSQL_PASSWORD=password
ports:
- 3306:3306
web:
image: spring-mysql
depends_on:
- db
links:
- db
ports:
- 8080:8080
environment:
- DATABASE_HOST=db
- DATABASE_USER=rafael
- DATABASE_NAME=teste
- DATABASE_PORT=3306
and the Dockerfile
FROM openjdk:8
ADD target/app.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "app.jar"]
Docker compose always starts and stops containers in dependency order, or sequential order in the file if not given. But docker-compose does not guarantee that it will wait till the dependency container is running. You can refer here for further details. So the problem here is that your database is not ready when your spring-mysql container tries to access the database. So, the recommended solution is you could use wait-for-it.sh or similar script to wrap your spring-mysql app starting ENTRYPOINT.
As example if you use wait-for-it.sh your ENTRYPOINT in your Dockerfile should change to following after copying above script to your project root:
ENTRYPOINT ["./wait-for-it.sh", "db:3306", "--", "java", "-jar", "app.jar"]
And two other important thing to consider here is:
Do not use links they are deprecated you should use user-defined network instead. All services in docker-compose file will be in single user-defined network if you don't explicitly define any network. So you just have to remove the links from compose file.
You don't need to publish the port for docker container if you only use it inside the user-defined network.
I was facing the same issue and in case you do not want to use any custom scripts, this can easily be resolved using health checks along with depends on. A sample using these is as follows:
services:
mysql-db:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=vikas1234
- MYSQL_USER=vikas
ports:
- 3306:3306
restart: always
healthcheck:
test: [ "CMD", "mysqladmin" ,"ping", "-h", "localhost" ]
timeout: 20s
retries: 10
app:
image: shop-keeper
container_name: shop-keeper-app
build:
context: .
dockerfile: Dockerfile
ports:
- 8080:8080
depends_on:
mysql-db:
condition: service_healthy
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://mysql-db:3306/shopkeeper?createDatabaseIfNotExist=true
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: vikas1234
Your config looks nice, I would just recommend:
Remove links: db. It has no value in user-defined bridge networking
Remove port exposing for db unless you want to connect from outside docker-compose - all ports are exposed automatically inside user-defined bridge network.
I think the problem is that database container takes more time to start than web. depends_on just controls the order, but does not guarantee you database readiness. If possible, set several connection attempts or put socket-wait procedure in your web container.
I am trying to set up a Spring boot application with Redis Sentinel 3.2.11 using docker. However I am getting
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /172.27.0.2:6379
My docker compose configuration
version: '3.1'
services:
master:
image: redis:3
container_name: redis-master
hostname: host_dev
networks:
- docker_dev
slave:
image: redis:3
command: redis-server --slaveof redis-master 6379
hostname: host_dev
links:
- master:redis-master
container_name: redis-slave
networks:
- docker_dev
sentinel:
build: sentinel
environment:
- SENTINEL_DOWN_AFTER=5000
- SENTINEL_FAILOVER=5000
- MASTER_NAME=mymaster
hostname: host_dev
image: sentinel:3
links:
- master:redis-master
- slave
container_name: sentinel
ports:
- "26379:26379"
networks:
- docker_dev
networks:
docker_dev:
Docker file
FROM redis:3
EXPOSE 26379
ADD sentinel.conf /etc/redis/sentinel.conf
RUN chown redis:redis /etc/redis/sentinel.conf
ENV SENTINEL_QUORUM 2
ENV SENTINEL_DOWN_AFTER 30000
ENV SENTINEL_FAILOVER 180000
COPY sentinel-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/sentinel-entrypoint.sh
ENTRYPOINT ["sentinel-entrypoint.sh"]
Spring configuration in application.properties:
redis.cluster.name=mymaster
redis.sentinel.nodes=localhost:26379
redis.timeout=2000
Issue:
The spring boot app(run from outside docker-machine) is able to connect with Sentinel node. The sentinel node provides the master information with IP 172.27.0.2 i.e docker n/w IP. The spring boot app tries to connect with redis-master at IP 172.27.0.2 and fails as the IP is not visible outside the docker machine.
Possible fix:
How can I make sentinel node provide an master IP as localhost instead of internal docker-machine n/w ip?
I'm using Docker to get my micro services architecture ready.
I'm facing some problem trying to link one container with another using docker-compose.
Basically I have a container for a postgressql image, and the a java micro service developed with spring boot that should connect to the database container.
So I'm setting a link in docker-compose.yml and referencing the db container ip as 'db' using :
- "JAVA_OPTS=-Dpostgres.host=db"
However I'm getting the following error starting the microservice with docker:
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'postgress.host' in value "jdbc:postgresql://${postgress.host}:5432/docker"
So basically this placeholder cannot be resolved, but normally docker-composed should take care of setting this system variable to point to the db container IP address right?
What I'm doing wrong?
Below the files involved:
docker-compose.yml:
version: "2"
services:
microservices:
build: ./microservices
container_name: microservices
links:
- db
- consul
environment:
- "JAVA_OPTS=-Dpostgres.host=db"
consul:
image: consul
container_name: consul
ports:
- "8500:8500"
db:
image: postgres
container_name: local-postgres9.6
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: docker
POSTGRES_USER: docker
application.yml of the microservice:
server:
port: 8081
project:
jdbc:
url: jdbc:postgresql://${postgres.host}:5432/docker
driver: org.postgresql.Driver
username: docker
password: docker
Try ${db.host}, according to the documentation:
Containers for the linked service will be reachable at a hostname
identical to the alias, or the service name if no alias was specified.