I have a spring-boot java application running in a docker container on my linux host machine.
I have a postgresql instance installed on the host that I want to connect to from the running container.
I've tried multiple different approaches (--network="host" is not what I want).
My Dockerfile looks like this:
FROM openjdk:13-ea-9-jdk-alpine3.9
EXPOSE 8080
CMD mkdir /opt/StatisticalRestService
COPY target/StatisticalRestService-0.0.1-SNAPSHOT.jar
/opt/StatisticalRestService/
COPY DockerConfig/application.yml /opt/StatisticalRestService/
RUN chmod 777 /opt/StatisticalRestService/StatisticalRestService-0.0.1-SNAPSHOT.jar \
&& ls -l /opt/StatisticalRestService/StatisticalRestService-0.0.1-SNAPSHOT.jar \
&& INTERNAL_HOST_IP=$(ip route show default | awk '/default/ {print $3}') \
&& echo "$INTERNAL_HOST_IP host.docker.internal" >> /etc/hosts \
&& chmod +r /etc/hosts \
&& cat /etc/hosts
ENTRYPOINT [ "java", "-jar", "-Dspring.config.location=/opt/StatisticalRestService/application.yml", "/opt/StatisticalRestService/StatisticalRestService-0.0.1-SNAPSHOT.jar" ]
application.yml:
spring:
application:
name: StatisticalRestService
jpa:
database: POSTGRESQL
show-sql: true
hibernate:
ddl-auto: create-drop
datasource:
platform: postgres
#url: jdbc:postgresql://host.docker.internal:5432/StatisticalRestService
url: jdbc:postgresql://172.17.0.1:5432/StatisticalRestService
username: statEntityUser
password: test123
driverClassName: org.postgresql.Driver
I have configured postresql's setting listen_addressess = '*' and the following entry is in the pg_hba.conf:
host all all 172.17.0.0/16 md5
host all all 192.168.1.0/24 md5
ifconfig docker0:
arizon#tuxpad:~/Utveckling/StatisticalRestService$ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
inet6 fe80::42:3bff:fe4f:ed34 prefixlen 64 scopeid 0x20<link>
ether 02:42:3b:4f:ed:34 txqueuelen 0 (Ethernet)
RX packets 28 bytes 1506 (1.5 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 198 bytes 25515 (25.5 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
This is the build output:
arizon#tuxpad:~/Utveckling/StatisticalRestService$ sudo docker build . -t arizon/statisticalrestservice:1.0.0-SNAPSHOT
Sending build context to Docker daemon 223.7MB
Step 1/7 : FROM openjdk:13-ea-9-jdk-alpine3.9
---> 6a6c49978498
Step 2/7 : EXPOSE 8080
---> Running in df7ebc70e950
Removing intermediate container df7ebc70e950
---> 417e50a9f5fd
Step 3/7 : CMD mkdir /opt/StatisticalRestService
---> Running in f33ca0acddf7
Removing intermediate container f33ca0acddf7
---> 59ae394176f3
Step 4/7 : COPY target/StatisticalRestService-0.0.1-SNAPSHOT.jar /opt/StatisticalRestService/
---> 4fbcfeb039f8
Step 5/7 : COPY DockerConfig/application.yml /opt/StatisticalRestService/
---> 244d31fc4755
Step 6/7 : RUN chmod 777 /opt/StatisticalRestService/StatisticalRestService-0.0.1-SNAPSHOT.jar && ls -l /opt/StatisticalRestService/StatisticalRestService-0.0.1-SNAPSHOT.jar && INTERNAL_HOST_IP=$(ip route show default | awk '/default/ {print $3}') && echo "$INTERNAL_HOST_IP host.docker.internal" >> /etc/hosts && chmod +r /etc/hosts && cat /etc/hosts
---> Running in 241f43aebbdc
-rwxrwxrwx 1 root root 35266534 Mar 16 19:52 /opt/StatisticalRestService/StatisticalRestService-0.0.1-SNAPSHOT.jar
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2 241f43aebbdc
172.17.0.1 host.docker.internal
Removing intermediate container 241f43aebbdc
---> 5c6c53d8011d
Step 7/7 : ENTRYPOINT [ "java", "-jar", "-Dspring.config.location=/opt/StatisticalRestService/application.yml", "/opt/StatisticalRestService/StatisticalRestService-0.0.1-SNAPSHOT.jar" ]
---> Running in 213a87164e8f
Removing intermediate container 213a87164e8f
---> 802cd987771f
Successfully built 802cd987771f
Successfully tagged arizon/statisticalrestservice:1.0.0-SNAPSHOT
When I run this with the datasource url pointed to host.docker.internal, i get unknownHostException, despite the output from the /etc/hosts file confirming it's there. From what I understand, there might be an issue with /etc/nsswitch.conf under alpine. I've tried adding the file and pasting this line from my host:
hosts: files mdns4_minimal [NOTFOUND=return] dns myhostname
to no avail.
When I run it with the datasource url pointed to 172.17.0.1:5432, I get connection timed out.
I verified access to psql from my host by pointing pgadmin to the 192.168 ip to verify that listen_addresses = '*' works:
host all all 192.168.1.0/24 md5
which it does. It's a different entry tho.
Docker version:
Client:
Version: 18.09.2
API version: 1.39
Go version: go1.10.4
Git commit: 6247962
Built: Tue Feb 26 23:52:23 2019
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.09.2
API version: 1.39 (minimum version 1.12)
Go version: go1.10.4
Git commit: 6247962
Built: Wed Feb 13 00:24:14 2019
OS/Arch: linux/amd64
Experimental: false
Postgresql version:
arizon#tuxpad:~/Utveckling/StatisticalRestService$ dpkg --list | grep postgresql
ii postgresql-10 10.6-0ubuntu0.18.04.1 amd64 object-relational SQL database, version 10 server
So, TL;DR: Two questions:
1. How do I get host.docker.internal to work on docker under linux?
2. How do I connect my containerized application to my host postgresql instance?
I solved this not in the way I intended when I asked the question but It's solved.
I ended up creating a postgres container too with a volume to keep the persisted data persitent.
I made a Dockerfile for postgres that looks like this:
FROM postgres:10-alpine
RUN mkdir /docker-entrypoint-initdb.d/
#COPY initdb.sql /docker-entrypoint-initdb.d/
COPY my-postgres.conf /usr/local/share/postgresql/postgresql.conf
#ENV POSTGRES_USER statEntityUser
#ENV POSTGRES_PASSWORD test123
#ENV POSTGRES_DB StatisticalRestService
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 5432
CMD [ "postgres" ]
The initdb.sql script is just creating the same thing that the commented environment variables are, the user, password and database. Sql-scripts placed in that folder is run by the entry point script when the database is started and ready (it's an out-of-the-box feature in the docker image I derive from). The reason they are commented out is that it's in the docker-compose file (see below). The postgresql.conf is basically the template that is included in the container but with listen_addresses = '*' uncommented.
I also made a docker-compose.yml to run both of these containers in a good way together:
version: "3"
services:
statistical-rest-service:
build: ./StatisticalRestService
ports:
- 8081:8080
depends_on:
- postgres
networks:
- statisticsNet
postgres:
container_name: postgres
build: ./Postgres
ports:
- 5433:5432
volumes:
- postgres-volume:/var/lib/postgresql/data
command: postgres -c 'config_file=/usr/local/share/postgresql/postgresql.conf'
networks:
- statisticsNet
environment:
POSTGRES_USER: statEntityUser
POSTGRES_PASSWORD: test123
POSTGRES_DB: StatisticalRestService
networks:
statisticsNet:
volumes:
postgres-volume:
I'm not sure if you have to create the volume before hand or if it's included in docker-compose but if you need to, it's just docker volume create postgres-volume.
Postgres documentation on how to use the image and/or derive from it: Postgres on docker hub
NOTE: When you start the container with an appointed volume, make some mistake and shut it down, when you start it again, it will not mess with the existing database on the volume. You might get into a position where you have "dangling volumes" that are stale versions of old run-time containers that you've killed and removed but they can produce unexpected behavior (for me, the user and database wasn't created because of this).
You can clear them with this command: docker volume rm $(docker volume ls -qf dangling=true) (or run the command inside $() to list eventual dangling volumes.
Since I am creating a dedicated docker network in the docker-compose file, the containers can find each other by name (note container_name). That makes the connection url in the application.yml for my java like this: url: jdbc:postgresql://postgres:5432/StatisticalRestService
I hope someone is helped by this :)
Related
I am trying to set up oracle data once my oracle container is up and running, below is my compose file:
version: '3'
services:
oracle:
image: absolutapps/oracle-12c-ee
container_name: oracle-docker
ports:
- 8080:8080
- 1521:1521
environment:
CASSANDRA_BROADCAST_ADDRESS: 127.0.0.1
ORACLE_ALLOW_REMOTE: "true"
volumes:
- ./scripts/oracle-init.sh:/oracle-init.sh
oracle-init.sh:
#!/usr/bin/env bash
echo "### SETUP EXECUTION! ###"
sqlplus -s "system/oracle#localhost:1521/orcl /scripts/init-oracle.sql"
echo "### SETUP EXECUTED! ###"
init-oracle.sql :
CREATE USER ot_consumer_tryout IDENTIFIED BY ot_consumer_tryout;
Can anyone help me what is wrong with above snippets. My docker container getting started but sql script does not execute ,here is the log from terminal:
oracle-docker | Running init scripts...
oracle-docker | Init scripts in /oracle.init.d/: Ignoring /oracle.init.d/*
oracle-docker |
oracle-docker | Done with scripts we are ready to go
Look at the volume you created
- ./scripts/oracle-init.sh:/oracle-init.sh
which means inside your container the file is in '/oracle-init.sh' but your
container might be expecting in a different location something with folder name oracle.init.d as per your container log.
oracle-init.sh is trying to run /scripts/init-oracle.sql , hope your image already has it otherwise you will have to mount it as well in compose file.
I was given a multy-steps task and im stuck !!
im trying to connect my Java container to my MYSQL container,but im getting 503 ERROR
HTTP ERROR 503
Problem accessing /. Reason:
Service Unavailable
docker-compose file :
version: "3.3"
services:
lavagna:
build: .
ports:
- "8080:8080"
networks:
- back_net
depends_on:
- my_db
environment:
spring.datasource.url: "jdbc:mysql://my-db:3306/lavagna"
my_db:
image: mysql:5.7
ports:
- "3306:3306"
networks:
- back_net
volumes:
- $PWD/mysql:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: 123
MYSQL_USER: eyal
MYSQL_PASSWORD: 123
networks:
back_net:
driver: bridge
I got the JAVA src files,i just used maven localy to build it and use target for the Java Dockerfile
java app dockerfile :
FROM openjdk:8-jre-alpine
EXPOSE 8080
COPY ./target/. .
COPY ./entrypoint.sh .
ENV DB_DIALECT MYSQL
ENV DB_URL jdbc:mysql://localhost:3306/lavagna
ENV DB_USER "root"
ENV DB_PASS "123"
ENV SPRING_PROFILE dev
RUN apk update \
&& apk add ca-certificates \
&& update-ca-certificates && apk add openssl
RUN chmod 774 entrypoint.sh
ENTRYPOINT [ "./entrypoint.sh" ]
I think you need a combination of comments and answers given already. Your containers are on the same network, so it appears to boil down to configuration.
In your docker file update your DB_URL to:
ENV DB_URL jdbc:mysql://my_db:3306/lavagna
If you use localhost your container will loopback to itself, and never hit the network.
In your docker-compose yml file, you have a typo in the url, try updating to:
spring.datasource.url: "jdbc:mysql://my_db:3306/lavagna"
As an aside, using depends_on does not wait for the service to be ready. It simply dictates start order as the documentation states:
There are several things to be aware of when using depends_on:
depends_on does not wait for db and redis to be “ready” before starting web - only until they have been started. If you need to wait for a service to be ready...
I am building my first Springboot 2.0 application. I am trying to put my Springboot application into one docker container and my PostgresDB into another container.
My Dockerfile
FROM frolvlad/alpine-oraclejdk8:slim
VOLUME /tmp
ADD springboot-api-demo-0.1*.jar app.jar
RUN sh -c 'touch /app.jar'
EXPOSE 9443
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/urandom -jar /app.jar" ]
My docker-compose.yml file
version: "2.1"
services:
springboot-api-demo:
image: "fw/springboot-api-demo"
mem_limit: 1024m
ports:
- "8080:8080"
environment:
- SPRING_PROFILES_ACTIVE=local
- AWS_REGION=local
- ENVIRONMENT=local
- AUTH_ENABLED=false
postgres:
container_name: pgdb
image: postgres:9.6-alpine
environment:
- 'POSTGRES_ROOT_PASSWORD=postgres'
- 'POSTGRES_USER=postgres'
- 'POSTGRES_PASSWORD=postgres'
ports:
- "54321:5432"
I am using Springboot JPA Data 2.0 with below config data in my application.properties
spring.datasource.url= jdbc:postgresql://localhost:54321/java_learning
spring.datasource.username=postgres
spring.datasource.password=postgres
I can test that Both of the Images are up. Also from docker log and docker events, I see that postgres Container is running fine, even I can access it and also created a DB too.
But springboot container started but i died because it could not connect to postgress and throwing error below.
Unable to obtain connection from database: The connection attempt
failed
Note that my host machine already has Postgres on port 5432 thats why I did a port mapping ofr 54321:5432 on my postgres container. Here is Proof :) -
➜ springboot-api-demo git:(master) ✗ lsof -i:54321
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
com.docke 44345 shailendra.singh 18u IPv4 0xf62897fbdd69e31d 0t0 TCP *:54321 (LISTEN)
com.docke 44345 shailendra.singh 21u IPv6 0xf62897fbdd119975 0t0 TCP localhost:54321 (LISTEN)
➜ springboot-api-demo git:(master) ✗ lsof -i:5432
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
postgres 715 shailendra.singh 5u IPv6 0xf62897fbb43e03b5 0t0 TCP localhost:postgresql (LISTEN)
postgres 715 shailendra.singh 6u IPv4 0xf62897fbbaeea9bd 0t0 TCP localhost:postgresql (LISTEN)
I am not sure what is the problem. But my Springboot application is not able to connect my postgres container which is running fine with proper creadentials.
Try with :
spring.datasource.url= jdbc:postgresql://pgdb:5432/java_learning
The postgres database is not running on localhost, it's running in the other container which has an other IP (yet unknown).
Thanksfully, docker-compose automatically create a network shared among all the containers in the docker-compose.yml (unless explicitly said to do not), as a result you can magically use the service name as an hostname.
Also, you have a typo in the port, Postgres use 5432 by default, not 54321
You are pointing your application towards localhost, but this is not shared between containers.
To access another container you have to refer to its hostname.
you should use the following datasource url:
spring.datasource.url=jdbc:postgresql://pgdb:5432/java_learning
See this simple tutorial about connecting to a container from another container with docker compose: https://docs.docker.com/compose/gettingstarted/
You're missing networking configuration in your docker-compose.yml specification. By using "networks" you can effectively communicate between containers by their service name (using dns, the service name as the hostname).
Here is an updated docker-compose.yml:
version: "2.1"
services:
springboot-api-demo:
image: "fw/springboot-api-demo"
mem_limit: 1024m
ports:
- "8080:8080"
environment:
- SPRING_PROFILES_ACTIVE=local
- AWS_REGION=local
- ENVIRONMENT=local
- AUTH_ENABLED=false
networks:
- mynet
postgres:
container_name: pgdb
image: postgres:9.6-alpine
environment:
- 'POSTGRES_ROOT_PASSWORD=postgres'
- 'POSTGRES_USER=postgres'
- 'POSTGRES_PASSWORD=postgres'
ports:
- "54321:5432"
networks:
- mynet
networks:
mynet:
driver: bridge
Your database url should look like spring.datasource.url=jdbc:postgresql://postgres:5432/java_learning (notice the hostname, postgres, is equal to that of the service name.
Apart from the above solutions provided JDK 11 java container with the mentioned configuration (connecting postgres via IP, localhost, servicename .. with postgres container exposed to LAN) still doesn't work. Upgrade to JDK latest version (17 currently) works for me - do consider this also when you use JDK 11 and trying java container (docker) communicating with postgres container.
I'm trying to set Envoy as a load balancer for a Java application. Following is the setup I'm trying to implement.
I followed Envoy Tutorial to get some idea and found a code. There, they use a front end envoy container as a single front end service, to distribute traffic among many backend containers. But for my work, I do not need such a front end container. So I modified the code and following are the files I created.
service-envoy.yaml
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.http_connection_manager
config:
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: service
domains:
- "*"
routes:
- match:
prefix: "/service"
route:
cluster: local_service
http_filters:
- name: envoy.router
config: {}
clusters:
- name: local_service
connect_timeout: 0.25s
type: strict_dns
lb_policy: round_robin
hosts:
- socket_address:
address: 127.0.0.1
port_value: 9000
admin:
access_log_path: "/dev/null"
address:
socket_address:
address: 0.0.0.0
port_value: 8081
Dockerfile-Service
FROM envoyproxy/envoy-alpine:latest
FROM java:8
ADD gs-actuator-service-0.1.0.jar gs-actuator-service-0.1.0.jar
ADD ./start_service.sh /usr/local/bin/start_service.sh
RUN chmod u+x /usr/local/bin/start_service.sh
ENTRYPOINT /usr/local/bin/start_service.sh
start-service.sh
#!/bin/sh
java -jar gs-actuator-service-0.1.0.jar &
envoy -c /etc/service-envoy.yaml --service-cluster service${SERVICE_NAME}
docker-compose.yml
version: '2'
services:
service1:
build:
context: .
dockerfile: Dockerfile-service
volumes:
- ./service-envoy.yaml:/etc/service-envoy.yaml
networks:
envoymesh:
aliases:
- service1
environment:
- SERVICE_NAME=1
expose:
- "80"
networks:
envoymesh: {}
gs-actuator-service-0.1.0.jar is the jar file which contains the web service. When run in localhost, it can be called using
localhost:9000/prime?number=10
When I run
sudo docker-compose up --build
it gives the following output with an error line at the end.
Building service1
Step 1/6 : FROM envoyproxy/envoy-alpine:latest
---> 44b4a2e0acd8
Step 2/6 : FROM java:8
---> d23bdf5b1b1b
Step 3/6 : ADD gs-actuator-service-0.1.0.jar gs-actuator-service-0.1.0.jar
---> fee1f402b547
Step 4/6 : ADD ./start_service.sh /usr/local/bin/start_service.sh
---> 52c2cc17c4dd
Step 5/6 : RUN chmod u+x /usr/local/bin/start_service.sh
---> Running in 95ee0537c3d1
Removing intermediate container 95ee0537c3d1
---> d8e9bdb2f95e
Step 6/6 : ENTRYPOINT /usr/local/bin/start_service.sh
---> Running in 27b20f261c0f
Removing intermediate container 27b20f261c0f
---> 26080be0b5ea
Successfully built 26080be0b5ea
Successfully tagged frontproxy_service1:latest
Recreating frontproxy_service1_1 ...
Recreating frontproxy_service1_1 ... done
Attaching to frontproxy_service1_1
service1_1 | /usr/local/bin/start_service.sh: 3:
/usr/local/bin/start_service.sh: envoy: not found
frontproxy_service1_1 exited with code 127
Can someone please point out where I'm getting this wrong?
Thank you
Are you on Windows? If you are, check all of your files with Notepad++ and at the bottom right you can change from CR LF to LF. Change it across all files, save it and try to run it again.
Hiho
Spring Settings
spring.jpa.hibernate.ddl-auto=update
spring.datasource.url= jdbc:mysql://vetdb:3306/vetdb?useSSL=false&allowPublicKeyRetrieval=true
spring.datasource.username=root
spring.datasource.testWhileIdle = true
spring.datasource.validationQuery = SELECT 1
spring.datasource.password=root
spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver
spring.datasource.platform=mysql
spring.datasource.initialize=true
spring.jpa.show-sql=false
spring.jpa.hibernate.naming-strategy = org.hibernate.cfg.ImprovedNamingStrategy
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
spring.jmx.default-domain=TierarztDatenController
endpoints.jmx.uniquie-names=true
Docker Compose
vetdb:
container_name: vetdb
hostname: vetdb
image: mariadb
restart: always
command: --default-authentication-plugin=mysql_native_password --innodb-flush-method=O_DSYNC --innodb-use-native-aio=0 --log_bin=ON
environment:
MYSQL_ROOT_PASSWORD: ${VET_DB_ROOT_PASSWORD}
MYSQL_DATABASE: ${VET_DB_NAME}
MYSQL_PASSWORD: ${VET_DB_PASSWORD}
MYSQL_ROOT_HOST: '%'
volumes:
# - "${VET_DB_PATH}/mysql-conf:/etc/mysql/conf.d"
- "${VET_DB_PATH}:/var/lib/mysql"
- "${VET_DB_PATH}/mysql-log:/var/log/mysql"
.env
VET_DB_PATH=./dockervolumes/microservices/VET/db
VET_DB_NAME=vetdb
VET_DB_ROOT_PASSWORD=root
VET_DB_PASSWORD=root
Exception: `java.sql.SQLException: null, message from server: "Host '172.23.0.9' is not allowed to connect to this MariaDB server"`
i added MYSQL_ROOT_HOST: '%' to allow everyone to connect to the db but it didnt work
i tried to use a config-file.cnf containing bind-address = 0.0.0.0 but it didnt work
funny thing is everything worked well before i restarted the container today and i didnt change this service in 3 days
i tried see if name/password = root work
$ docker exec -it vetdb bash
root#vetdb:/# mysql -u root -p
Enter password:
ERROR 1045 (28000): Access denied for user 'root'#'localhost' (using password: YES)
but root without password works (inside the container)
root#vetdb:/# mysql -u root
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 21
Server version: 10.3.9-MariaDB-1:10.3.9+maria~bionic-log mariadb.org binary distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]>
if i just remove spring.datasource.password=root
or set it to spring.datasource.password=
then it results in the same exception
what am i doing wrong?
i changed the environment variables to this
environment:
- "MYSQL_ROOT_PASSWORD=${VET_DB_ROOT_PASSWORD}"
- "MYSQL_DATABASE=${VET_DB_NAME}"
- "MYSQL_ROOT_HOST=%"
and deleted everything rerun it and it worked
docker-compose down
docker rm -f $(docker container ls -aq)
docker-compose rm -v
docker volume prune
docker system prune
rm dockerdockervolumes/ -rf
and it worked