We Keep Coding

Containerized kafka - cannot create transactions due to "Timeout expired after 60000 milliseconds while awaiting InitProducerId"

I am testing out a containerized kafka instance which I created with the following command:
docker run -d --name kafkacontainer -p 9093:9093
-e KAFKA_BROKER_ID=1
-e KAFKA_TRANSACTION_STATE_LOG_MIN_ISR=1
-e KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR=1
-e KAFKA_ZOOKEEPER_CONNECT=172.17.0.2:2181
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://mytestvm:9093,BROKER://localhost:9092
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9093,BROKER://0.0.0.0:9092
-e KAFKA_INTER_BROKER_LISTENER_NAME=BROKER
-e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=BROKER:PLAINTEXT,PLAINTEXT:PLAINTEXT
-e KAFKA_OFFSETS_TOPIC_NUM_PARTITIONS=1
-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
-e KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS=0
confluentinc/cp-kafka:7.0.1
The instance works fine when I add a topic, send messages to the topic, and read from that topic.
Then, I attempted to create a transaction with the following code:
((KafkaProducer<?, ?>) producer).getInternalKafkaProducer().initTransactions();
This throws the following exception after hanging for 60 seconds:
org.apache.kafka.common.errors.TimeoutException: Timeout expired after 60000 milliseconds while awaiting InitProducerId
After some googling, I found many posts suggesting that this error occurs because transactions require special configuration. Namely, there has to be multiple brokers, and multiple in-sync replicators (ISR). So I've tried configuring this setting pair below to no avail:
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR=1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR=1
I have tried 1/1, 3/2. What should they be in order to resolve this timeout error?

If you want to connect to kafka outside container, you need to publish outside port in docker container. Please, add -p 9092:9092 to your container.

Postgresql docker container connection failure

I'm using this line of code to try to connect(w/ jdbc) to a psql docker container:
Connection conn = DriverManager.getConnection(
"jdbc:postgresql://localhost:5431/postgres?allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC",
"postgres", "postgres");
db name, username, and pw are all postgres.
The container was created with
docker run --name practice -e POSTGRES_PASSWORD=postgres -p 5431:5431 postgres
Error here:
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:150)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:393)
at org.postgresql.Driver.connect(Driver.java:267)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
at com.dehmer.JdbcSelectTest.main(JdbcSelectTest.java:9)
Caused by: java.io.EOFException
at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:276)
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:269)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
... 11 more
docker container ls:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
be8931d1621c postgres "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:5431->5431/tcp, 5432/tcp practice
This same line of code will connect to a different psql container I made a while ago, the only difference I can see between them is that the other one is on port 5432 (so just using 5432 in the getConnection args and running my old container will make it work). I can access the db through the docker cli just fine and it is running. But I'll add that I'm very new to networking concepts, so I could easily be overlooking something here. Any help is much appreciated.

I think you want the container port 5432 which is the default postgres port to be published to localhost port 5431. Instead of
docker run --name practice -e POSTGRES_PASSWORD=postgres -p 5431:5431 postgres
Use
docker run --name practice -e POSTGRES_PASSWORD=postgres -p 5431:5432 postgres

Cannot access docker postgres image via intellij and quarkus?

I run a docker postgres container via command line:
docker run --name some-postgres -p 5432:5432 -e POSTGRES_PASSWORD=passme -d postgres
But in IntelliJ I can't access this container
The following error message occurs:
[08001] Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
java.net.ConnectException: Connection refused: connect.
Do you know any problem, maybe with the port?

jdbc url for mysql from docker to localhost does not work without host mode

I have a local spring boot application that connects to local MySQL and it works fine.
For connection I use the following property:
spring.datasource.url=jdbc:mysql://localhost:3306/pitstop?useSSL=false&useUnicode=true&characterEncoding=utf8&serverTimezone=UTC
I would like to put my app in docker and try to connect to local DB.
So I need to modify MySQL url.
I used this command to obtain local IP
ip route show | grep "default" | awk '{print $3}' the result is 192.168.1.1. I modify my url like this
spring.datasource.url=jdbc:mysql://192.168.1.1:3306/pitstop?useSSL=false&useUnicode=true&characterEncoding=utf8&serverTimezone=UTC
and try to start docker container with my app by command docker run -p 9001:9001 --network=bizon4ik --rm bizon4ik/mycontainer the result is the exception:
Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
I had tried to find another IP. I used ip addr show command and found the next record:
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 30:52:cb:db:8d:e0 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.102/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp3s0
So based on it I took 192.168.1.102 and modify my url but the result is the same exception.
To be sure that founded IPs are correct I run new one container with pure ubuntu docker run -it --rm --network=bizon4ik ubuntu and checked the mentioned above IP:
root#268c4d544328:/# nmap -p 3306 192.168.1.1
Starting Nmap 7.60 ( https://nmap.org ) at 2019-07-28 17:28 UTC
Nmap scan report for 192.168.1.1
Host is up (0.053s latency).
PORT STATE SERVICE
3306/tcp filtered mysql
root#268c4d544328:/# nmap -p 3306 192.168.1.102
Starting Nmap 7.60 ( https://nmap.org ) at 2019-07-28 15:56 UTC
Nmap scan report for my-host (192.168.1.102)
Host is up (0.000088s latency).
PORT STATE SERVICE
3306/tcp closed mysql
So looks fine, the ubuntu container can ping my DB. Do you have any ideas on why I cannot connect through the app to DB?
NB:
I checked in DB SHOW GRANTS; the result is GRANT ALL PRIVILEGES ON *.* TO 'root'#'%' WITH GRANT OPTION
I also checked /etc/mysql/my.cnf file. It has only these records:
!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/
I don't have ~/.my.cnf

The problem was in bind-address for MySQL. I looked it in the /etc/mysql/my.cnf however, the right place is /etc/mysql/mysql.conf.d/mysqld.cnf

Connection refused: accessing a spring boot application running in docker container

I have my spring boot application and mysql database running in separate docker containers. I am able to access server database from my host.
My application.properties for Spring boot application looks like below:
spring.datasource.url=jdbc:mysql://benefitsmysql:3308/benefitsmysql
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
# ====================================================================================
# = SSL Configuration
# ====================================================================================
#security.basic.enabled=false
server.port=8443
server.ssl.key-store=keystore.jks
server.ssl.key-store-password=*******
server.ssl.keyStoreType=jks
server.ssl.keyAlias=tomcatselfsigned
I am building a docker container image by using maven plugin for docker. My Dockerfile looks like below:
FROM java:8
VOLUME /tmp
ADD Benefits.jar Benefits.jar
EXPOSE 8443
RUN bash -c 'touch /Benefits.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/Benefits.jar"]
I am starting docker container for spring boot application like below:
docker run -p 8443:8443 --name benefits --link benefitsmysql:mysql -d c794a4d0c634
and if I do docker ps -a, I get following output
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8070c575b6dd c794a4d0c634 "java -Djava.secur..." 2 minutes ago Up 2 minutes 0.0.0.0:8443->8443/tcp benefits
aa417df08b94 mysql:5.6 "docker-entrypoint..." 2 days ago Up 2 days 0.0.0.0:3308->3306/tcp benefitsmysql
f55a2a7ac487 hello-world "/hello" 2 days ago Exited (0) 2 days ago gifted_lalande
Now when I access my spring boot application running inside docker container from my windows machine like https://192.168.99.103:8443/home, I get connection refused error ERR_CONNECTION_REFUSED.
What am I missing in this configuration?

yogsma
I read your blog, and apply your solve, but docker-machine ip didn't solve my problem.
Then I realize docker containers can't communicate with 127.0.0.1 and I use their container ip
docker inspect <container_id>
then find IpAddress.
This ip address is solves my problem.I dont need to use docker-machine ip