I have a SpringBoot application that works perfectly. Now I am trying to Dockerise it, but have some issues when connecting to the database.
The application has two datasources:
application.properties
server.port= 8081
# pims datasource
spring.datasource1.driver-class-name=org.postgresql.Driver
spring.datasource1.jdbc-url=jdbc:postgresql://localhost:5432/pims
spring.datasource1.username=postgres
spring.datasource1.password=
spring.jpa.database-platform=postgres
#spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.show-sql=false
# approval datasource
spring.datasource2.driver-class-name=org.postgresql.Driver
spring.datasource2.jdbc-url=jdbc:postgresql://localhost:5432/approval
spring.datasource2.username=postgres
spring.datasource2.password=
In the application, I use Spring to access the two datasources:
#Configuration
#ComponentScan(basePackages = "com.nexct")
public class MultipleDBConfig {
#Bean(name = "datasource1")
#ConfigurationProperties("spring.datasource1")
#Primary
public DataSource dataSource1(){
return DataSourceBuilder.create().build();
}
#Bean(name = "datasource2")
#ConfigurationProperties("spring.datasource2")
public DataSource dataSource2(){
return DataSourceBuilder.create().build();
}
}
So I have created:
Dockerfile
FROM openjdk:14
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} nexct-approval-service.jar
EXPOSE 8081
ENTRYPOINT ["java","-jar","/nexct-approval-service.jar"]
docker-compose.yml
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
ports:
- "8081:8081"
db:
image: postgres
environment:
POSTGRES_DB_PORT: "5432"
POSTGRES_DB_HOST: "localhost"
POSTGRES_PASSWORD:
POSTGRES_USER: postgres
POSTGRES_DB: pims
However, I am not sure how to configure the two Postgres databases.
Any advise would be appreciated.
Note: the databases are not running in a container, as they are used by other legacy applications.
If I remove the db from docker-compose.yml. When I run docker-compose up, the Spring Boot application starts, and I can access a RESTful service. However, I get the following error:
PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
ports:
- "8081:8081"
server.port=8081
# pims datasource
spring.datasource1.driver-class-name=org.postgresql.Driver
spring.datasource1.jdbc-url=jdbc:postgresql://<postgres_host_private_ip>:5432/pims
spring.datasource1.username=postgres
spring.datasource1.password=
spring.jpa.database-platform=postgres
#spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.show-sql=false
# approval datasource
spring.datasource2.driver-class-name=org.postgresql.Driver
spring.datasource2.jdbc-url=jdbc:postgresql://<postgres_host_private_ip>:5432/approval
spring.datasource2.username=postgres
spring.datasource2.password=
Replace <postgres_host_private_ip> with the private IP of the machine hosting the postgres databases.
I'm not sure what you were trying to achieve with the nested db service, but since your database are already running somewhere outside there is no need for this.
Your Spring application will run on a isolated network from your host because of the docker isolation (unless explicitly asked to use host network with network_mode: "host"). Knowing that, the "localhost" can not work inside the container, you need to use the real IP of the machine that hosts the databases. Most likely something like 192.168.xxx.xxx
Also, you need to make sure the postgres is configured to accept connection from "non-localhost" sources (see https://www.andrew-kirkpatrick.com/2017/05/allow-connection-postgresql-server-outside-localhost/).
To access an application running on ur Host there are few options
Use the Bridge network. (Doc)
Create a bridge network and assign the subnet address and gateway address.
Now you can reach your application running at the host from the container at <gateway address>:<port> i.e 172.28.0.1:<port>
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
ports:
- "8081:8081"
networks:
hostnet: {}
networks:
hostnet:
driver: bridge
ipam:
config:
- subnet: 172.28.0.0/16
Use Attach service to host network. (Doc)
add network_mode: host to your service (only works on Linux)
OR create an external network
Now you can reach your application running at the host from the container at localhost:<port>
With network_mode: (Linux)
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
ports:
- "8081:8081"
network_mode: host
Try configuring your docker-compose file to use bridged networking:
version: '3.7'
services:
product-service:
build: ../nexct-approval-service
volumes:
- ../nexct-approval-service:/usr/src/app
environment:
POSTGRES_DB_PORT: "5432"
POSTGRES_DB_HOST: "localhost"
POSTGRES_PASSWORD:
POSTGRES_USER: postgres
POSTGRES_DB: pims
networks:
- product_service_network
networks:
product_service_network:
driver: bridge
You also have to move the ENVIRONMENT variables to product-service if you are going to use them in the product-service container.
If localhost doesn't work, use 127.0.0.1 instead.
Related
I'm trying to connect my containerized spring boot application with another containerized MQTT broker. Both of them are on their own projects as follows:
mqtt docker-compose.yml:
version: '3.9'
services:
mqttbroker:
container_name: mqttbroker
restart: always
volumes:
- ./config:/mosquitto/config
- ./data:/mosquitto/data
- ./log:/mosquitto/log
ports:
- 8883:8883
networks:
- mynetwork
volumes:
config:
data:
log:
mqtt Dockerfile
FROM eclipse-mosquitto
WORKDIR /mosquitto
COPY . .
EXPOSE 8883
And then the spring boot project is like:
spring boot docker-compose.yml
version: '3.8'
services:
myapp:
build: .
container_name: myapp
ports:
- '8082:8082'
stdin_open: true
tty: true
networks:
- mynetwork
In my application.properties I try to connect to the MQTT broker like:
mosquitto.url=tcp://mqttbroker:8883 and I get connection refused. However, if I run the spring boot application locally, I can connect to the docker container with mosquitto.url=tcp://localhost:8883.
I would rather have all the configurations in my docker-compose files to decrease manual codes.
I really appreciate your help in advance!
Yes, You do have to share the network as external.
you cab check the networks you have with this command : docker network ls
You would notice that they each have their own networks.
adding this to spring boot docker-compose file should fix the issue :
networks:
default:
name: mynetwork
external: true
For more information about docker compose networks
I have a problem about defining database in some services in my Spring Boot Microservice example.
When I run docker-compose.yml file through this command (docker-compose up -d), I have a datasource issue in user service, advertisement service and lastly report service.
All these services have their own database defined in their own properties file under configuration folder of config server.
Here is the database part of docker-compose.yml
database:
container_name: mysql-database
image: 'mysql:latest'
ports:
- "3366:3306"
restart: always
environment:
MYSQL_DATABASE: "springbootuser"
MYSQL_USER: "springmicroserviceuser"
MYSQL_PASSWORD: "111111"
MYSQL_ROOT_PASSWORD: "111111"
volumes:
- db-data:/var/lib/mysql
networks:
backend:
aliases:
- "database"
healthcheck:
test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"]
timeout: 20s
retries: 10
How can I fix the datasource issue in all services defined in docker-compose file
Here is my docker-compose.yml : Link
Here is the user service properties file : Link
Here is the advertisement service properties file : Link
Here is the report service properties file : Link
Edited 1st :
Here is my new docker-compose.yml : Link
Here is the problem with your datasource connection port localhost:3306.
user 3366 instead of 3306.
when you define internal port on same docker network you need to use database hostname instead of localhost.
if you use localhost or IP then you need to use docker to expose a port for database connections that route from the external network.
if needed update docker compose database section:
ports:
- "3366:3306"
expose:
- "3366"
I am building a Spring Boot application which uses PostgreSQL with docker-compose.
When I run my containers using docker-compose up --build, my Spring Boot application fails to start because it does not find the PostgreSQL container's hostname.
Spring Boot Dockerfile
FROM maven:3.6.3-openjdk-14-slim AS build
COPY src /usr/src/app/src
COPY pom.xml /usr/src/app
RUN mvn -f /usr/src/app/pom.xml clean package
FROM openjdk:14-slim
COPY --from=build /usr/src/app/target/server-0.0.1-SNAPSHOT.jar /usr/app/server-0.0.1-SNAPSHOT.jar
EXPOSE 9000
ENTRYPOINT ["java","-jar","/usr/app/server-0.0.1-SNAPSHOT.jar"]
docker-compose.yml
version: '3'
services:
db:
image: postgres
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: my_db
ports:
- "5432:5432"
volumes:
- db-data:/var/lib/postgresql/data
networks:
- db-network
restart: always
server:
build: './server'
depends_on:
- db
restart: always
ports:
- "9000:9000"
networks:
- db-network
volumes:
- ./server:/server
networks:
db-network:
volumes:
db-data:
application.properties
spring.datasource.url=jdbc:postgresql://db:5432/my_db
spring.datasource.username=postgres
spring.datasource.password=postgres
Error output
Caused by: java.net.UnknownHostException: db
My guess is that docker-compose's virtual network isn't created yet during the build stage of the Spring Boot Dockerfile.
Any idea on how to solve this issue ?
Lots of info here: https://docs.docker.com/compose/networking/
Within the web container, your connection string to db would look like
postgres://db:5432, and from the host machine, the connection string
would look like postgres://{DOCKER_IP}:8001.
What this is saying is db:5432 is fine to use within docker-compose.yaml and the IP address will be passed (not "db"), but using it externally within your application code isn't going to work. You could however pass from docker-compose.yaml db as an application input variable, which your application could fill in in the configuration file. This would enable you then to connect.
Externalising configuration like this is fairly common practice so should be a relatively easy fix.
eg:
Docker-compose.yaml
version: '3'
services:
db:
container_name: db
image: postgres
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: my_db
ports:
- "5432:5432"
volumes:
- db-data:/var/lib/postgresql/data
networks:
- db-network
restart: always
server:
build: './server'
depends_on:
- db
environment:
DB_HOST: db # untested, but there should be a way to pass this in
DB_PORT: 5432
DB_DATABASE: my_db
DB_USER: postgres
DB_PASSWORD: postgres
restart: always
ports:
- "9000:9000"
networks:
- db-network
volumes:
- ./server:/server
networks:
db-network:
volumes:
db-data:
Then have an application.properties file located under src/main/java/resources/application.properties
spring.datasource.url=jdbc:postgresql://${DB_HOST}:${DB_PORT}/${DB_DATABASE}
spring.datasource.username=${DB_USERNAME}
spring.datasource.password=${DB_PASSWORD}
This post completely solved my issue.
It turns out that maven was trying to establish the connection to the database while building the .jar file. All I had to do is modify my Dockerfile with this line RUN mvn -f /usr/src/app/pom.xml clean package -DskipTests.
Please do note while building the images the service will not have access to the database as it is not yet running . Only after the images are built and the containers are running do the services have access . So when you try to pass a host as db , it is not yet available in the build stage . It is available only once the db container starts running .
I'm trying to set up a Spring Boot application that depends on a MySQL database called teste in docker-compose. After issuing docker-compose up, I'm getting:
Caused by: java.net.ConnectException: Connection refused (Connection refused)
I'm running on Linux Mint, my docker-compose version is 1.23.2, my Docker version is 18.09.0.
application.properties
# JPA PROPS
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.hibernate.ddl-auto=update
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
spring.jpa.hibernate.naming-strategy=org.hibernate.cfg.ImprovedNamingStrategy
spring.datasource.url=jdbc:mysql://db:3306/teste?useSSL=false&serverTimezone=UTC
spring.datasource.username=rafael
spring.datasource.password=password
spring.database.driverClassName =com.mysql.cj.jdbc.Driver
docker-compose.yml
version: '3.5'
services:
db:
image: mysql:latest
environment:
- MYSQL_ROOT_PASSWORD=rootpass
- MYSQL_DATABASE=teste
- MYSQL_USER=rafael
- MYSQL_PASSWORD=password
ports:
- 3306:3306
web:
image: spring-mysql
depends_on:
- db
links:
- db
ports:
- 8080:8080
environment:
- DATABASE_HOST=db
- DATABASE_USER=rafael
- DATABASE_NAME=teste
- DATABASE_PORT=3306
and the Dockerfile
FROM openjdk:8
ADD target/app.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "app.jar"]
Docker compose always starts and stops containers in dependency order, or sequential order in the file if not given. But docker-compose does not guarantee that it will wait till the dependency container is running. You can refer here for further details. So the problem here is that your database is not ready when your spring-mysql container tries to access the database. So, the recommended solution is you could use wait-for-it.sh or similar script to wrap your spring-mysql app starting ENTRYPOINT.
As example if you use wait-for-it.sh your ENTRYPOINT in your Dockerfile should change to following after copying above script to your project root:
ENTRYPOINT ["./wait-for-it.sh", "db:3306", "--", "java", "-jar", "app.jar"]
And two other important thing to consider here is:
Do not use links they are deprecated you should use user-defined network instead. All services in docker-compose file will be in single user-defined network if you don't explicitly define any network. So you just have to remove the links from compose file.
You don't need to publish the port for docker container if you only use it inside the user-defined network.
I was facing the same issue and in case you do not want to use any custom scripts, this can easily be resolved using health checks along with depends on. A sample using these is as follows:
services:
mysql-db:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=vikas1234
- MYSQL_USER=vikas
ports:
- 3306:3306
restart: always
healthcheck:
test: [ "CMD", "mysqladmin" ,"ping", "-h", "localhost" ]
timeout: 20s
retries: 10
app:
image: shop-keeper
container_name: shop-keeper-app
build:
context: .
dockerfile: Dockerfile
ports:
- 8080:8080
depends_on:
mysql-db:
condition: service_healthy
environment:
SPRING_DATASOURCE_URL: jdbc:mysql://mysql-db:3306/shopkeeper?createDatabaseIfNotExist=true
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: vikas1234
Your config looks nice, I would just recommend:
Remove links: db. It has no value in user-defined bridge networking
Remove port exposing for db unless you want to connect from outside docker-compose - all ports are exposed automatically inside user-defined bridge network.
I think the problem is that database container takes more time to start than web. depends_on just controls the order, but does not guarantee you database readiness. If possible, set several connection attempts or put socket-wait procedure in your web container.
I'm using Docker to get my micro services architecture ready.
I'm facing some problem trying to link one container with another using docker-compose.
Basically I have a container for a postgressql image, and the a java micro service developed with spring boot that should connect to the database container.
So I'm setting a link in docker-compose.yml and referencing the db container ip as 'db' using :
- "JAVA_OPTS=-Dpostgres.host=db"
However I'm getting the following error starting the microservice with docker:
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'postgress.host' in value "jdbc:postgresql://${postgress.host}:5432/docker"
So basically this placeholder cannot be resolved, but normally docker-composed should take care of setting this system variable to point to the db container IP address right?
What I'm doing wrong?
Below the files involved:
docker-compose.yml:
version: "2"
services:
microservices:
build: ./microservices
container_name: microservices
links:
- db
- consul
environment:
- "JAVA_OPTS=-Dpostgres.host=db"
consul:
image: consul
container_name: consul
ports:
- "8500:8500"
db:
image: postgres
container_name: local-postgres9.6
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: docker
POSTGRES_USER: docker
application.yml of the microservice:
server:
port: 8081
project:
jdbc:
url: jdbc:postgresql://${postgres.host}:5432/docker
driver: org.postgresql.Driver
username: docker
password: docker
Try ${db.host}, according to the documentation:
Containers for the linked service will be reachable at a hostname
identical to the alias, or the service name if no alias was specified.