We Keep Coding

org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for host Linux error that works on Windows

I have looked at many (too many) similar questions here and tried over and over to solve this one but I can't quite get it to resolve.
I have a java code that connects to a PostGreSQL server on a RHEL 8.2 box to query and return a value. When I run it on my Windows 10 laptop; it works with no issues. However, when I run it locally on the Linux box it fails with the dreaded org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for host.
The java code is made by a ETL software called Talend; and the code is run on Linux by RunDeck.
To make matters more confusing, I can login on Linux as the postgres user and use the psql interface to connect to the db with the same host, port, db, and user I'm using in Java without error-
psql -h [server ip] -p 5432 -d my_special_db -U my_db_user
I have checked the server settings, the pg_hba.conf file, the user permissions, the log files and tried changing all of the above to get some sort of return; all to no avail.
I have added to pg_hba.conf:
host my_special_db my_db_user [server ip]/32 md5
host my_special_db my_db_user [server ip with .0 as the last quatraine]/24 md5
host my_special_db root ::1/128 trust
host all all [server ip]/32 trust
I have tried multiple connection modifiers like:
sslfactory=org.postgresql.ssl.NonValidatingFactory&sslmode=prefer
or
sslmode=prefer
or
sslmode=Require
or
no modifier at all
Please advise.
Details:
OS
NAME : Linux RHEL 8.2
VERSION : 4.18.0-372.16.1.el8_6.x86_64
JVM
IMPLEMENTATIONVERSION : 11.0.15+9-LTS
NAME : OpenJDK 64-Bit Server VM
VENDOR : Amazon.com Inc.
VERSION : 11.0.15
PostGreSQL
postgresql11-libs-11.7-1PGDG.rhel8.x86_64.rpm \
postgresql11-11.7-1PGDG.rhel8.x86_64.rpm \
postgresql11-server-11.7-1PGDG.rhel8.x86_64.rpm
Under server > properties > SSL
All Defaults:
SSL Mode: Prefer
Client Cert: [blank]
Client Cert Key: [blank]
Root Cert: [blank]
Cert Rev List: [blank]
SSL compression?: no
//////////////////
Under /etc/systemd/system/postgresql-11.service
[Unit]
Description=PostgreSQL 11 database server
Documentation=https://www.postgresql.org/docs/11/static/
After=syslog.target
After=network.target
[Service]
Type=notify
User=postgres
Group=postgres
# Note: avoid inserting whitespace in these Environment= lines, or you may
# break postgresql-setup.
# Location of database directory
Environment=PGDATA=[my data directory path]
# Where to send early-startup messages from the server (before the logging
# options of postgresql.conf take effect)
# This is normally controlled by the global default set by systemd
# StandardOutput=syslog
# Disable OOM kill on the postmaster
OOMScoreAdjust=-1000
Environment=PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj
Environment=PG_OOM_ADJUST_VALUE=0
ExecStartPre=[path to server]/pgsql-11/bin/postgresql-11-check-db-dir ${PGDATA}
ExecStart=[path to server]/pgsql-11/bin/postgres -D ${PGDATA}
#ExecReload=/bin/kill -HUP $MAINPID
KillMode=mixed
KillSignal=SIGINT
# Do not set any timeout value, so that systemd will not kill postmaster
# during crash recovery.
TimeoutSec=0
[Install]
WantedBy=multi-user.target
[Service]
Environment=PGDATA=[my data directory path]
//////////////////
//////////////////
Top of the pg_hba.conf file:
# TYPE DATABASE USER ADDRESS METHOD
# linux requires a 'local' entry
#local all all md5
local all all md5
# IPv4 local connections:
host all all 0.0.0.0/0 md5
# IPv6 local connections:
host all all ::0/0 md5
# no ssl connection
hostnossl all all 0.0.0.0/0 trust
# IPv4 local connections:
hostnossl all all 127.0.0.1/32 md5
# IPv6 local connections:
hostnossl all all ::1/128 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
host replication all 127.0.0.1/32 md5
host replication all ::1/128 md5
# This Servers IP and Ranges as Talend doesn't use localhost
host postgres postgres [server ip]/32 md5
host postgres all [server ip with .0 as the last quatraine]/24 md5
host all postgres [server ip]/32 md5
host all all [server ip with .0 as the last quatraine]/24 md5
host my_special_db my_db_user [server ip]/32 md5
host my_special_db my_db_user [server ip with .0 as the last quatraine]/24 md5
host my_special_db root ::1/128 trust
host all all [server ip]/32 trust
//////////////////
//////////////////
Part of My Java Code:
conn_tDBInput_1 = (java.sql.Connection) globalMap.get("conn_tDBConnection_2");
java.sql.Statement stmt_tDBInput_1 = conn_tDBInput_1.createStatement();
String dbquery_tDBInput_1 = "\n(SELECT ID AS \"id\"\n, ATTRIBUTE AS \"attribute\"\n, DEFINITION AS \"definition\"\n, DESCRIPTION AS \"description\""
+ "\nFROM " + context.SEC_DB_SCHEMA + "." + context.SEC_DB_TABLE
+ "\nWHERE \"description\" = 'my_filter_here'\nLIMIT 1);\n\n";
globalMap.put("tDBInput_1_QUERY", dbquery_tDBInput_1);
java.sql.ResultSet rs_tDBInput_1 = null;
try {
rs_tDBInput_1 = stmt_tDBInput_1.executeQuery(dbquery_tDBInput_1);
java.sql.ResultSetMetaData rsmd_tDBInput_1 = rs_tDBInput_1.getMetaData();
int colQtyInRs_tDBInput_1 = rsmd_tDBInput_1.getColumnCount();
String tmpContent_tDBInput_1 = null;
while (rs_tDBInput_1.next()) {
nb_line_tDBInput_1++;
if (colQtyInRs_tDBInput_1 < 1) {
row1.ID = 0;
} else {
row1.ID = rs_tDBInput_1.getInt(1);
if (rs_tDBInput_1.wasNull()) {
throw new RuntimeException("Null value in non-Nullable column");
}
}
if (colQtyInRs_tDBInput_1 < 2) {
row1.ATTRIBUTE = null;
} else {
row1.ATTRIBUTE = routines.system.JDBCUtil.getString(rs_tDBInput_1, 2, false);
}
if (colQtyInRs_tDBInput_1 < 3) {
row1.DEFINITION = null;
} else {
row1.DEFINITION = routines.system.JDBCUtil.getString(rs_tDBInput_1, 3, false);
}
if (colQtyInRs_tDBInput_1 < 4) {
row1.DESCRIPTION = null;
} else {
row1.DESCRIPTION = routines.system.JDBCUtil.getString(rs_tDBInput_1, 4, false);
}
/**
* [tDBInput_1 begin ] stop
*/
/**
* [tDBInput_1 main ] start
*/
currentComponent = "tDBInput_1";
tos_count_tDBInput_1++;
/**
* [tDBInput_1 main ] stop
*/
/**
* [tDBInput_1 process_data_begin ] start
*/
currentComponent = "tDBInput_1";
/**
* [tDBInput_1 process_data_begin ] stop
*/
/**
* [tReplicate_1 main ] start
*/
currentComponent = "tReplicate_1";
if (execStat) {
runStat.updateStatOnConnection(iterateId, 1, 1
, "row1"
);
}
row5 = new row5Struct();
row5.ID = row1.ID;
row5.ATTRIBUTE = row1.ATTRIBUTE;
row5.DEFINITION = row1.DEFINITION;
row5.DESCRIPTION = row1.DESCRIPTION;
globalMap.put("row5.ATTRIBUTE", row5.ATTRIBUTE);
globalMap.put("row5.DEFINITION", row5.DEFINITION);
// Set Reports DB values
context.DB_HOST = [my_server_ip];
context.DB_NAME = [my_db];
context.DB_PORT = 5432;
// This will have to be changed for each tower will have it's own 'reports' table
context.DB_SCHEMA = "my_schema";
context.DB_TABLE = "my_table";
context.DB_USER = (String)(globalMap.get("row5.ATTRIBUTE"));
context.DB_PWD = (String)(globalMap.get("row5.DEFINITION"));
//------------
currentComponent="tDBConnection_1";
int tos_count_tDBConnection_1 = 0;
String dbProperties_tDBConnection_1 = "sslfactory=org.postgresql.ssl.NonValidatingFactory&sslmode=prefer";
String url_tDBConnection_1 = "jdbc:postgresql://"+context.DB_HOST+":"+context.DB_PORT+"/"+context.DB_NAME;
if(dbProperties_tDBConnection_1 != null && !"".equals(dbProperties_tDBConnection_1.trim())) {
url_tDBConnection_1 = url_tDBConnection_1 + "?" + dbProperties_tDBConnection_1;
}
String dbUser_tDBConnection_1 = context.DB_USER;
//////////////////
//////////////////
Log entry from the time of trying to connect both locally and remotely-
2022-08-23 15:20:45.452 EDT [67687] LOG: database system is ready to accept connections
2022-08-23 15:22:00.719 EDT [67687] LOG: received fast shutdown request
2022-08-23 15:22:00.720 EDT [67687] LOG: aborting any active transactions
2022-08-23 15:22:00.722 EDT [67687] LOG: background worker "logical replication launcher" (PID 67696) exited with exit code 1
2022-08-23 15:22:00.722 EDT [67691] LOG: shutting down
2022-08-23 15:22:00.754 EDT [67687] LOG: database system is shut down
2022-08-23 15:22:00.814 EDT [67739] LOG: database system was shut down at 2022-08-23 15:22:00 EDT
2022-08-23 15:22:00.818 EDT [67735] LOG: database system is ready to accept connections
2022-08-23 15:52:25.666 EDT [67735] LOG: received fast shutdown request
2022-08-23 15:52:25.668 EDT [67735] LOG: aborting any active transactions
2022-08-23 15:52:25.669 EDT [67735] LOG: background worker "logical replication launcher" (PID 67745) exited with exit code 1
2022-08-23 15:52:25.669 EDT [67740] LOG: shutting down
2022-08-23 15:52:25.692 EDT [67735] LOG: database system is shut down
2022-08-23 15:52:34.188 EDT [79803] LOG: database system was shut down at 2022-08-23 15:52:25 EDT
2022-08-23 15:52:34.194 EDT [79801] LOG: database system is ready to accept connections
//////////////////////////////////////
UPDATE:
Here is a much of the java error message as I can share; given some of the restrictions on what I can post;
error message :: FATAL: no pg_hba.conf entry for host "[my_server_ip]", user "my_db_user", database "my_db", SSL off|4
Exception in component tDBConnection_2 (Generic_Talend_Job)
org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for host "[my_server_ip]", user "my_db_user", database "my_db", SSL off
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
at org.postgresql.Driver.makeConnection(Driver.java:465)
at org.postgresql.Driver.connect(Driver.java:264)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.tDBConnection_2Process(Generic_Talend_Job.java:5752)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.tSystem_1Process(Generic_Talend_Job.java:5614)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.tJava_3Process(Generic_Talend_Job.java:5342)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.tWarn_1Process(Generic_Talend_Job.java:5094)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.tFileInputDelimited_1Process(Generic_Talend_Job.java:4851)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.tLibraryLoad_1Process(Generic_Talend_Job.java:4089)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.tPrejob_1Process(Generic_Talend_Job.java:3856)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.runJobInTOS(Generic_Talend_Job.java:11916)
at local_project.generic_talend_job_0_1.Generic_Talend_Job.main(Generic_Talend_Job.java:11245)
Suppressed: org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for host "[my_server_ip]", user "my_db_user", database "my_db", SSL off
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)
... 15 more
sec db connection failure, error message :: FATAL: no pg_hba.conf entry for host "[my_server_ip]", user "my_db_user", database "my_db", SSL off
Update 2:
Entry from this mornings logs at /data_folder/log after trying again and getting the same error
[postgres#my_server_name log]$ vim postgresql-Wed.log
2022-08-24 10:00:17.714 EDT [79801] LOG: received fast shutdown request
2022-08-24 10:00:17.717 EDT [79801] LOG: aborting any active transactions
2022-08-24 10:00:17.717 EDT [508177] FATAL: terminating connection due to administrator command
2022-08-24 10:00:17.717 EDT [508176] FATAL: terminating connection due to administrator command
2022-08-24 10:00:17.718 EDT [79801] LOG: background worker "logical replication launcher" (PID 79809) exited with exit code 1
2022-08-24 10:00:17.718 EDT [79804] LOG: shutting down
2022-08-24 10:00:17.740 EDT [79801] LOG: database system is shut down
2022-08-24 10:00:35.780 EDT [513920] LOG: database system was shut down at 2022-08-24 10:00:17 EDT
2022-08-24 10:00:35.788 EDT [513918] LOG: database system is ready to accept connections
Update 3:
From the postgresql.conf file:
#------------------------------------------------------------------------------
# CONNECTIONS AND AUTHENTICATION
#------------------------------------------------------------------------------
# - Connection Settings -
listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost'; use '*' for all
# (change requires restart)
port = 5432 # (change requires restart)
max_connections = 100 # (change requires restart)
#superuser_reserved_connections = 3 # (change requires restart)
#unix_socket_directories = '/var/run/postgresql, /tmp' # comma-separated list of directories
# (change requires restart)
#unix_socket_group = '' # (change requires restart)
#unix_socket_permissions = 0777 # begin with 0 to use octal notation
# (change requires restart)
#bonjour = off # advertise server via Bonjour
# (change requires restart)
#bonjour_name = '' # defaults to the computer name
# (change requires restart)
# - TCP Keepalives -
# see "man 7 tcp" for details
#tcp_keepalives_idle = 0 # TCP_KEEPIDLE, in seconds;
# 0 selects the system default
#tcp_keepalives_interval = 0 # TCP_KEEPINTVL, in seconds;
# 0 selects the system default
#tcp_keepalives_count = 0 # TCP_KEEPCNT;
# 0 selects the system default
# - Authentication -
#authentication_timeout = 1min # 1s-600s
#password_encryption = md5 # md5 or scram-sha-256
#db_user_namespace = off
# GSSAPI using Kerberos
#krb_server_keyfile = ''
#krb_caseins_users = off
# - SSL -
#ssl = off
#ssl_ca_file = ''
#ssl_cert_file = 'server.crt'
#ssl_crl_file = ''
#ssl_key_file = 'server.key'
#ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL' # allowed SSL ciphers
#ssl_prefer_server_ciphers = on
#ssl_ecdh_curve = 'prime256v1'
#ssl_dh_params_file = ''
#ssl_passphrase_command = ''
#ssl_passphrase_command_supports_reload = off
Update 4:
After doing some more searching, I have found that the default for the postgres logging isn't verbose; so this is the verbose part:
[636130] [630676b4.9b4e2] [2022-08-24 15:06:28.850 EDT] [] [F0000] [] [] [] [] [] [7] [0]: LOG: hostssl record cannot match because SSL is disabled
[636130] [630676b4.9b4e2] [2022-08-24 15:06:28.850 EDT] [] [F0000] [] [] [] [] [] [8] [0]: HINT: Set ssl = on in postgresql.conf.
[636130] [630676b4.9b4e2] [2022-08-24 15:06:28.850 EDT] [] [F0000] [] [] [] [] [] [9] [0]: CONTEXT: line 103 of configuration file "[data folder path]/pg_hba.conf"
[636132] [630676b4.9b4e4] [2022-08-24 15:06:28.854 EDT] [] [00000] [] [] [] [] [] [1] [0]: LOG: database system was shut down at 2022-08-24 15:06:18 EDT
[636130] [630676b4.9b4e2] [2022-08-24 15:06:28.859 EDT] [] [00000] [] [] [] [] [] [10] [0]: LOG: database system is ready to accept connections
I have gotten this error even after the following updates-
update to the postgresql.conf file:
edit 1
ssl = off
edit 2
ssl = on
edit 3
#ssl = off
updated the log level in the postgresql.conf file to
log_line_prefix = '[%p] [%c] [%m] [%v] [%e] [%i] [%d] [%u] [%a] [%r] [%l] [%x]: '
...
log_statement = 'all'
updated the pg_hba.conf file to start with:
# Allow any user on the local system to connect to any database with
# any database user name using Unix-domain sockets (the default for local
# connections).
#
# TYPE DATABASE USER ADDRESS METHOD
local all all trust
# The same using local loopback TCP/IP connections.
#
# TYPE DATABASE USER ADDRESS METHOD
host all all 127.0.0.1/32 trust
# The same as the previous line, but using a separate netmask column
#
# TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD
host all all 127.0.0.1 255.255.255.255 trust
# The same over IPv6.
#
# TYPE DATABASE USER ADDRESS METHOD
host all all ::1/128 trust
# The same using a host name (would typically cover both IPv4 and IPv6).
#
# TYPE DATABASE USER ADDRESS METHOD
host all all localhost trust
hostssl all all 0.0.0.0/0 md5
also updated the connection params with-
sslfactory=org.postgresql.ssl.NonValidatingFactory
According to postgres this is supposed to stop any and all SSL issues and just let the connections through-
https://jdbc.postgresql.org/documentation/head/ssl-client.html
"A non-validating connection is established via a custom SSLSocketFactory class that is provided with the driver. Setting the connection URL parameter sslfactory=org.postgresql.ssl.NonValidatingFactory will turn off all SSL validation."
Update 5:
Logging settings from postgresql.conf file:
#------------------------------------------------------------------------------
# REPORTING AND LOGGING
#------------------------------------------------------------------------------
# - Where to Log -
log_destination = 'stderr' # Valid values are combinations of
# stderr, csvlog, syslog, and eventlog,
# depending on platform. csvlog
# requires logging_collector to be on.
# This is used when logging to stderr:
logging_collector = on # Enable capturing of stderr and csvlog
# into log files. Required to be on for
# csvlogs.
# (change requires restart)
# These are only used if logging_collector is on:
log_directory = 'log' # directory where log files are written,
# can be absolute or relative to PGDATA
log_filename = 'postgresql-%a.log' # log file name pattern,
# can include strftime() escapes
#log_file_mode = 0600 # creation mode for log files,
# begin with 0 to use octal notation
log_truncate_on_rotation = on # If on, an existing log file with the
# same name as the new log file will be
# truncated rather than appended to.
# But such truncation only occurs on
# time-driven rotation, not on restarts
# or size-driven rotation. Default is
# off, meaning append to existing files
# in all cases.
log_rotation_age = 1d # Automatic rotation of logfiles will
# happen after that time. 0 disables.
log_rotation_size = 0 # Automatic rotation of logfiles will
# happen after that much log output.
# 0 disables.
# These are relevant when logging to syslog:
#syslog_facility = 'LOCAL0'
#syslog_ident = 'postgres'
#syslog_sequence_numbers = on
#syslog_split_messages = on
# This is only relevant when logging to eventlog (win32):
# (change requires restart)
#event_source = 'PostgreSQL'
# - When to Log -
#log_min_messages = warning # values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic
#log_min_error_statement = error # values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic (effectively off)
#log_min_duration_statement = -1 # -1 is disabled, 0 logs all statements
# and their durations, > 0 logs only
# statements running at least this number
# of milliseconds
# - What to Log -
#debug_print_parse = off
#debug_print_rewritten = off
#debug_print_plan = off
#debug_pretty_print = on
#log_checkpoints = off
#log_connections = off
#log_disconnections = off
#log_duration = off
#log_error_verbosity = default # terse, default, or verbose messages
#log_hostname = off
log_line_prefix = '[%p] [%c] [%m] [%v] [%e] [%i] [%d] [%u] [%a] [%r] [%l] [%x]: ' # special values:
# %a = application name
# %u = user name
# %d = database name
# %r = remote host and port
# %h = remote host
# %p = process ID
# %t = timestamp without milliseconds
# %m = timestamp with milliseconds
# %n = timestamp with milliseconds (as a Unix epoch)
# %i = command tag
# %e = SQL state
# %c = session ID
# %l = session line number
# %s = session start timestamp
# %v = virtual transaction ID
# %x = transaction ID (0 if none)
# %q = stop here in non-session
# processes
# %% = '%'
# e.g. '<%u%%%d> '
#log_lock_waits = off # log lock waits >= deadlock_timeout
log_statement = 'all' # none, ddl, mod, all
#log_replication_commands = off
#log_temp_files = -1 # log temporary files equal or larger
# than the specified size in kilobytes;
# -1 disables, 0 logs all temp files
log_timezone = 'America/New_York'
Update 6:
I made edits to the logging part of the config, and now have a lot of output from the log
new logging settings:
#------------------------------------------------------------------------------
# REPORTING AND LOGGING
#------------------------------------------------------------------------------
# - Where to Log -
log_destination = 'stderr' # Valid values are combinations of
# stderr, csvlog, syslog, and eventlog,
# depending on platform. csvlog
# requires logging_collector to be on.
# This is used when logging to stderr:
logging_collector = on # Enable capturing of stderr and csvlog
# into log files. Required to be on for
# csvlogs.
# (change requires restart)
# These are only used if logging_collector is on:
log_directory = 'log' # directory where log files are written,
# can be absolute or relative to PGDATA
log_filename = 'postgresql-%a.log' # log file name pattern,
# can include strftime() escapes
#log_file_mode = 0600 # creation mode for log files,
# begin with 0 to use octal notation
log_truncate_on_rotation = on # If on, an existing log file with the
# same name as the new log file will be
# truncated rather than appended to.
# But such truncation only occurs on
# time-driven rotation, not on restarts
# or size-driven rotation. Default is
# off, meaning append to existing files
# in all cases.
log_rotation_age = 1d # Automatic rotation of logfiles will
# happen after that time. 0 disables.
log_rotation_size = 0 # Automatic rotation of logfiles will
# happen after that much log output.
# 0 disables.
# These are relevant when logging to syslog:
#syslog_facility = 'LOCAL0'
#syslog_ident = 'postgres'
#syslog_sequence_numbers = on
#syslog_split_messages = on
# This is only relevant when logging to eventlog (win32):
# (change requires restart)
#event_source = 'PostgreSQL'
# - When to Log -
log_min_messages = debug5 # values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic
log_min_error_statement = debug5 # values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic (effectively off)
log_min_duration_statement = 0 # -1 is disabled, 0 logs all statements
# and their durations, > 0 logs only
# statements running at least this number
# of milliseconds
# - What to Log -
#debug_print_parse = off
#debug_print_rewritten = off
#debug_print_plan = off
#debug_pretty_print = on
#log_checkpoints = off
log_connections = on
log_disconnections = on
#log_duration = off
log_error_verbosity = verbose # terse, default, or verbose messages
log_hostname = on
log_line_prefix = '[%p] [%c] [%m] [%v] [%e] [%i] [%d] [%u] [%a] [%r] [%l] [%x]: ' # special values:
# %a = application name
# %u = user name
# %d = database name
# %r = remote host and port
# %h = remote host
# %p = process ID
# %t = timestamp without milliseconds
# %m = timestamp with milliseconds
# %n = timestamp with milliseconds (as a Unix epoch)
# %i = command tag
# %e = SQL state
# %c = session ID
# %l = session line number
# %s = session start timestamp
# %v = virtual transaction ID
# %x = transaction ID (0 if none)
# %q = stop here in non-session
# processes
# %% = '%'
# e.g. '<%u%%%d> '
#log_lock_waits = off # log lock waits >= deadlock_timeout
log_statement = 'all' # none, ddl, mod, all
#log_replication_commands = off
#log_temp_files = -1 # log temporary files equal or larger
# than the specified size in kilobytes;
# -1 disables, 0 logs all temp files
log_timezone = 'America/New_York'
I couldn't paste the error log here because after making it verbose it was very long- so I made a gist
Last update:
I removed the Gist because the peeps here helped me find my error:
okay, so the issue was 3 things; thing 1, we have a 'properties' file that I thought was set to this server this db; and it wasn't // thing 2, the default logging was set to 'warning' and didn't have enough information so after putting it to debug level 5 (the most data) and on 'all' events; I finally saw a error message that made me realize it was pointed at the wrong db // thing 3, in the many (too many) iterations of my pg_hba.conf file I had added then removed the IP address.
so now it's working, lol
Sometimes you are too close to the tracks to see the train, lol

I removed the Gist because the peeps here helped me find my error:
okay, so the issue was 3 things; thing 1, we have a 'properties' file that I thought was set to this server this db; and it wasn't // thing 2, the default logging was set to 'warning' and didn't have enough information so after putting it to debug level 5 (the most data) and on 'all' events; I finally saw a error message that made me realize it was pointed at the wrong db // thing 3, in the many (too many) iterations of my pg_hba.conf file I had added then removed the IP address. so now it's working, lol Sometimes you are too close to the tracks to see the train, lol

Cannot connect java mail to postfix smtp server

I'm facing an issue right now. I configure my postfix server together with dovecot and everything works well so far. I could connect some accounts using my phone email client with the parameters:
Host: imap.mydomain.com
email: xxx#mydomain.com
password: xxxxxxxx
port: 993
and
Host: smtp.mydomain.com
email: xxx#mydomain.com
password: xxxxxxxx
port: 465
Accept all SSL certificates
Now I want to connect my java app with the smtp parameters to send emails. Unfortunately, it takes too long and then I get a time out exception.
On the server, I get the following from the logs:
Sep 01 12:55:05 mail.mydomain.com postfix/smtps/smtpd[125179]: lost connection after AUTH from unknown[212.70.149.71]
Sep 01 12:55:05 mail.mydomain.com postfix/smtps/smtpd[125179]: disconnect from unknown[212.70.149.71] ehlo=1 auth=0/1 rset=1 commands=2/3
Sep 01 12:55:11 mail.mydomain.com postfix/smtps/smtpd[125179]: connect from pd9fff457.dip0.t-ipconnect.de[217.255.244.87]
Sep 01 12:55:11 mail.mydomain.com postfix/smtps/smtpd[125179]: SSL_accept error from pd9fff457.dip0.t-ipconnect.de[217.255.244.87]: -1
Sep 01 12:55:11 mail.mydomain.com postfix/smtps/smtpd[125179]: warning: TLS library problem: error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:../ssl/record/rec_layer_s3.c:1543:SSL alert number 46:
Sep 01 12:55:11 mail.mydomain.com postfix/smtps/smtpd[125179]: lost connection after CONNECT from pd9fff457.dip0.t-ipconnect.de[217.255.244.87]
Sep 01 12:55:11 mail.mydomain.com postfix/smtps/smtpd[125179]: disconnect from pd9fff457.dip0.t-ipconnect.de[217.255.244.87] commands=0/0
Sep 01 12:55:11 mail.mydomain.com postfix/smtps/smtpd[125179]: connect from pd9fff457.dip0.t-ipconnect.de[217.255.244.87]
Here is my config in java:
mail.host=smtp.mydomain.com
mail.port=465
mail.username=xxxx#mydomain.com
mail.password=xxxx
Here the java stacktrace I get:
Exception in thread "main" org.springframework.mail.MailSendException: Mail server connection failed; nested exception is javax.mail.MessagingException: Could not connect to SMTP host: smtp.mydomain.com, port: 465, response: -1. Failed messages: javax.mail.MessagingException: Could not connect to SMTP host: smtp.mydomain.com, port: 465, response: -1; message exception details (1) are:
Failed message 1:
javax.mail.MessagingException: Could not connect to SMTP host: smtp.mydomain.com, port: 465, response: -1
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2197)
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:740)
at javax.mail.Service.connect(Service.java:366)
My dovecot config
##
## Authentication processes
##
# Disable LOGIN command and all other plaintext authentications unless
# SSL/TLS is used (LOGINDISABLED capability). Note that if the remote IP
# matches the local IP (ie. you're connecting from the same computer), the
# connection is considered secure and plaintext authentication is allowed.
# See also ssl=required setting.
disable_plaintext_auth = yes
# Authentication cache size (e.g. 10M). 0 means it's disabled. Note that
# bsdauth and PAM require cache_key to be set for caching to be used.
#auth_cache_size = 0
# Time to live for cached data. After TTL expires the cached record is no
# longer used, *except* if the main database lookup returns internal failure.
# We also try to handle password changes automatically: If user's previous
# authentication was successful, but this one wasn't, the cache isn't used.
# For now this works only with plaintext authentication.
#auth_cache_ttl = 1 hour
# TTL for negative hits (user not found, password mismatch).
# 0 disables caching them completely.
#auth_cache_negative_ttl = 1 hour
# Space separated list of realms for SASL authentication mechanisms that need
# them. You can leave it empty if you don't want to support multiple realms.
# Many clients simply use the first one listed here, so keep the default realm
# first.
#auth_realms =
# Default realm/domain to use if none was specified. This is used for both
# SASL realms and appending #domain to username in plaintext logins.
#auth_default_realm =
# List of allowed characters in username. If the user-given username contains
# a character not listed in here, the login automatically fails. This is just
# an extra check to make sure user can't exploit any potential quote escaping
# vulnerabilities with SQL/LDAP databases. If you want to allow all characters,
# set this value to empty.
#auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_#
# Username character translations before it's looked up from databases. The
# value contains series of from -> to characters. For example "##/#" means
# that '#' and '/' characters are translated to '#'.
#auth_username_translation =
# Username formatting before it's looked up from databases. You can use
# the standard variables here, eg. %Lu would lowercase the username, %n would
# drop away the domain if it was given, or "%n-AT-%d" would change the '#' into
# "-AT-". This translation is done after auth_username_translation changes.
auth_username_format = %n
# If you want to allow master users to log in by specifying the master
# username within the normal username string (ie. not using SASL mechanism's
# support for it), you can specify the separator character here. The format
# is then <username><separator><master username>. UW-IMAP uses "*" as the
# separator, so that could be a good choice.
#auth_master_user_separator =
# Username to use for users logging in with ANONYMOUS SASL mechanism
#auth_anonymous_username = anonymous
# Maximum number of dovecot-auth worker processes. They're used to execute
# blocking passdb and userdb queries (eg. MySQL and PAM). They're
# automatically created and destroyed as needed.
#auth_worker_max_count = 30
# Host name to use in GSSAPI principal names. The default is to use the
# name returned by gethostname(). Use "$ALL" (with quotes) to allow all keytab
# entries.
#auth_gssapi_hostname =
# Kerberos keytab to use for the GSSAPI mechanism. Will use the system
# default (usually /etc/krb5.keytab) if not specified. You may need to change
# the auth service to run as root to be able to read this file.
#auth_krb5_keytab =
# Do NTLM and GSS-SPNEGO authentication using Samba's winbind daemon and
# ntlm_auth helper. <doc/wiki/Authentication/Mechanisms/Winbind.txt>
#auth_use_winbind = no
# Path for Samba's ntlm_auth helper binary.
#auth_winbind_helper_path = /usr/bin/ntlm_auth
# Time to delay before replying to failed authentications.
#auth_failure_delay = 2 secs
# Require a valid SSL client certificate or the authentication fails.
#auth_ssl_require_client_cert = no
# Take the username from client's SSL certificate, using
# X509_NAME_get_text_by_NID() which returns the subject's DN's
# CommonName.
#auth_ssl_username_from_cert = no
# Space separated list of wanted authentication mechanisms:
# plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp
# gss-spnego
# NOTE: See also disable_plaintext_auth setting.
auth_mechanisms = plain login
##
## Password and user databases
##
#
# Password database is used to verify user's password (and nothing more).
# You can have multiple passdbs and userdbs. This is useful if you want to
# allow both system users (/etc/passwd) and virtual users to login without
# duplicating the system users into virtual database.
#
# <doc/wiki/PasswordDatabase.txt>
#
# User database specifies where mails are located and what user/group IDs
# own them. For single-UID configuration use "static" userdb.
#
# <doc/wiki/UserDatabase.txt>
#!include auth-deny.conf.ext
#!include auth-master.conf.ext
!include auth-system.conf.ext
#!include auth-sql.conf.ext
#!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-static.conf.ext
My master.cf
#
# Postfix master process configuration file. For details on the format
# of the file, see the master(5) manual page (command: "man 5 master" or
# on-line: http://www.postfix.org/master.5.html).
#
# Do not forget to execute "postfix reload" after editing this file.
#
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (no) (never) (100)
# ==========================================================================
smtp inet n - y - - smtpd
#smtp inet n - y - 1 postscreen
#smtpd pass - - y - - smtpd
#dnsblog unix - - y - 0 dnsblog
#tlsproxy unix - - y - 0 tlsproxy
submission inet n - y - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_sasl_type=dovecot
-o smtpd_sasl_path=private/auth
-o smtpd_tls_wrappermode=no
-o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
-o smtpd_relay_restrictions=permit_sasl_authenticated,reject
# -o smtpd_tls_auth_only=yes
# -o smtpd_reject_unlisted_recipient=no
# -o smtpd_client_restrictions=$mua_client_restrictions
# -o smtpd_helo_restrictions=$mua_helo_restrictions
# -o smtpd_sender_restrictions=$mua_sender_restrictions
# -o milter_macro_daemon_name=ORIGINATING
smtps inet n - y - - smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
-o smtpd_sasl_type=dovecot
-o smtpd_sasl_path=private/auth
-o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
-o smtpd_relay_restrictions=permit_sasl_authenticated,reject
# -o smtpd_reject_unlisted_recipient=no
# -o smtpd_client_restrictions=$mua_client_restrictions
# -o smtpd_helo_restrictions=$mua_helo_restrictions
# -o smtpd_sender_restrictions=$mua_sender_restrictions
# -o milter_macro_daemon_name=ORIGINATING
#628 inet n - y - - qmqpd
pickup unix n - y 60 1 pickup
cleanup unix n - y - 0 cleanup
qmgr unix n - n 300 1 qmgr
#qmgr unix n - n 300 1 oqmgr
tlsmgr unix - - y 1000? 1 tlsmgr
rewrite unix - - y - - trivial-rewrite
bounce unix - - y - 0 bounce
defer unix - - y - 0 bounce
trace unix - - y - 0 bounce
verify unix - - y - 1 verify
flush unix n - y 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - y - - smtp
relay unix - - y - - smtp
-o syslog_name=postfix/$service_name
# -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq unix n - y - - showq
error unix - - y - - error
retry unix - - y - - error
discard unix - - y - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - y - - lmtp
anvil unix - - y - 1 anvil
scache unix - - y - 1 scache
postlog unix-dgram n - n - 1 postlogd
#
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# Many of the following services use the Postfix pipe(8) delivery
# agent. See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
#
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in main.cf: maildrop_destination_recipient_limit=1
#
maildrop unix - n n - - pipe
flags=DRXhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
#
# ====================================================================
#
# Recent Cyrus versions can use the existing "lmtp" master.cf entry.
#
# Specify in cyrus.conf:
# lmtp cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
#
# Specify in main.cf one or more of the following:
# mailbox_transport = lmtp:inet:localhost
# virtual_transport = lmtp:inet:localhost
#
# ====================================================================
#
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in main.cf: cyrus_destination_recipient_limit=1
#
#cyrus unix - n n - - pipe
# flags=DRX user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
#
# ====================================================================
# Old example of delivery via Cyrus.
#
#old-cyrus unix - n n - - pipe
# flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
#
# ====================================================================
#
# See the Postfix UUCP_README file for configuration details.
#
uucp unix - n n - - pipe
flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
#
# Other external delivery methods.
#
ifmail unix - n n - - pipe
flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp unix - n n - - pipe
flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix - n n - 2 pipe
flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman unix - n n - - pipe
flags=FRX user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${user}
My main.cf
# TLS parameters
smtpd_tls_cert_file=/my/cert
smtpd_tls_key_file=/my/key
smtpd_tls_security_level=may
smtpd_tls_loglevel = 1
smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
#Enforce TLSv1.3 or TLSv1.2
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = XXX
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, XXX, YYYYY, localhost.localdomain, localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
mailbox_transport = lmtp:unix:private/dovecot-lmtp
smtputf8_enable = no
Can someone please help me?

Statement.setQueryTimeout doesn't work on Oracle 18c jdbc driver

We have a J2EE application on a payara 5.2020 server that executes a long running query (PL/SQL that executes for a couple of hours).
To avoid a timeout exception, we use this sentence at StatementLevel:
statement.setQueryTimeout(0);
This works using Oracle jdbc drivers version 12c, but when we have migrated to Oracle 18c, and we changed the driver to the version 18c, the query execution stops after 15 minutes with this exception. The code works with Oracle 12 and Oracle 18 is the change in the driver's jar what brings up the problem.
The problem has been reproduced in Linux and Windows machines:
2021-06-14T07:50:01.762+0200|SEVERE: java.sql.SQLRecoverableException: Error de E/S: Socket read interrupted
at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:946)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1136)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3640)
at oracle.jdbc.driver.T4CCallableStatement.executeInternal(T4CCallableStatement.java:1318)
at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3752)
at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:4242)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1079)
at com.sun.gjc.spi.base.PreparedStatementWrapper.execute(PreparedStatementWrapper.java:532)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.sun.gjc.spi.jdbc40.ProfiledConnectionWrapper40$1.invoke(ProfiledConnectionWrapper40.java:437)
at com.sun.proxy.$Proxy324.execute(Unknown Source)
at org.apache.jsp.index_jsp.callPL(index_jsp.java:49)
at org.apache.jsp.index_jsp._jspService(index_jsp.java:108)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:411)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:473)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:377)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1636)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:259)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.InterruptedIOException: Socket read interrupted
at oracle.net.nt.TimeoutSocketChannel.handleInterrupt(TimeoutSocketChannel.java:262)
at oracle.net.nt.TimeoutSocketChannel.read(TimeoutSocketChannel.java:184)
at oracle.net.ns.NSProtocolNIO.doSocketRead(NSProtocolNIO.java:544)
at oracle.net.ns.NIOPacket.readHeader(NIOPacket.java:234)
at oracle.net.ns.NIOPacket.readPacketFromSocketChannel(NIOPacket.java:174)
at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:122)
at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:100)
at oracle.net.ns.NIONSDataChannel.readDataFromSocketChannel(NIONSDataChannel.java:86)
at oracle.jdbc.driver.T4CMAREngineNIO.prepareForUnmarshall(T4CMAREngineNIO.java:762)
at oracle.jdbc.driver.T4CMAREngineNIO.unmarshalUB1(T4CMAREngineNIO.java:427)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:394)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:255)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:610)
at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:249)
at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:82)
at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:924)
It seems the transport layer has been migrated to java.nio and the method setQueryTimeout is no longer
Things We've tried:
Setting the default Statement timeout to -1 in the JDBC Connection Pool Advanced Attributes screen in payara console.
Trying to set the time directly in the connection with conn.setNetworkTimeout(Executors.newFixedThreadPool(1), 0) didn't make any effect.
In different sources we have found that these properties below should affect the network timeout evaluation. We set them as JVM properties for payara startup (-Doracle.net.CONNECT_TIMEOUT=xxx) and as JDBC Connection Pool Properties, both cases with values 0 and -1. Didn't work in any case.
oracle.net.CONNECT_TIMEOUT
oracle.net.READ_TIMEOUT
oracle.jdbc.ReadTimeout
Sources:
Oracle 18c Net services best practices
Oracle 18c java jdbc reference. E.1.5 Using JDBC with Firewalls
4.- As we are accessing the DataSource through payara DataSource pool, we cannot cast the com.sun.gjc.spi.jdbc40.DataSource40 (class provided by payara) to an OracleDataSource, but we created the DataSorce programatically to set the connection properties as shown here and setting the properties seen in the image above but it doesn't work:
public static Properties oracleProperties() {
// Already tried -1 and 0
Properties properties = new Properties();
properties.put("Oracle.net.CONNECT_TIMEOUT", 0);
properties.put("Oracle.net.READ_TIMEOUT", 0);
properties.put("Oracle.jdbc.ReadTimeout", 0);
return properties;
}
public static OracleDataSource createDataSource() throws Exception {
OracleDataSource ods = new OracleDataSource();
ods.setURL("jdbc:oracle:thin:#itauc4602x:1521/BDExp");
ods.setUser("enevac");
ods.setPassword("enevac");
ods.setDataSourceName("OracleXADataSource");
ods.setLoginTimeout(0);
// default connection properties to avoid timeoutException
ods.setConnectionProperties(oracleProperties());
return ods;
}
Has anyone faced this problem, any idea on how to avoid the timeOut restriction?
Why 15 minutes?, according to the reference, the default value for oracle.net.ReadTimeout is 10 minutes.
Update:
To explain in more detail why I think the problem is in the driver and why I discard other possible origins of the exception, I assume the timeout can be raised from three sources:
Network timeout: I discard it cause I'm testing a payara server in my local machine against the development database, with no firewall in between.
Database server: the DBA has checked the Oracle net services configuration and there's no limit set that explains the 15 minutes cut. Besides, in these case, an SQLException would be expected with some kind of ORA-xxx error code.
JDBC: this can be set at connection level, statement level and transaction level. As I said at the beginning, the code works with oracle 12c drivers against Oracle 12 and Oracle 18 servers, it was the change of the driver's jar what make the code stop working.

Finally the problem was fixed configuring in the payara pool the "connectionProperties" custom property of the OracleDataSource. As #ibre5041 pointed, setting the property oracle.jdbc.javaNetNio=false changes the transport layer used by the driver and it starts working as the oracle 12c previous version.
According to Oracle reference, the OracleDataSource implementors can receive the connection properties as a java.util.Properties object.
Table 8-2 Oracle Extended Data Source Properties
Name: connectionProperties
Type: java.util.Properties
Description: Specifies the connection properties.
To set a multivalued property to the jdbc pool in the Payara Admin Console, you have to set the properties as (prop1=value1,prop2=value2), (Thank you again Ondro Mihályi). So in our case we set:
connectionProperties = (oracle.jdbc.ReadTimeout=0, oracle.jdbc.javaNetNio=false)
As a summary of what works and doesn't using Oracle 18c jdbc driver (every step tested separately)::
Setting timeout at statement level doesn't work:
statement.setQueryTimeout(0)
Setting timeout at connection level, with -1 or 0, doesn't work:
conn.setNetworkTimeout(Executors.newFixedThreadPool(1), timeout in ms)
Setting timeout properties programmatically in the java.util.connection using the OracleDataSource makes it work as indicated in the question.
Setting timeout properties as JVM properties makes it work if the limit is below 15min, but if you set a value > 15 minutes the exceptions is thrown, so setting to 0 or -1 has no effect:
So this makes the query stop after 10 secs:
-Doracle.net.CONNECT_TIMEOUT=10000 -Doracle.net.READ_TIMEOUT=10000 -Doracle.jdbc.ReadTimeout=10000
But with this stops after 15 minutes:
-Doracle.net.CONNECT_TIMEOUT=-1 -Doracle.net.READ_TIMEOUT=-1 -Doracle.jdbc.ReadTimeout=-1
Setting oracle.net.keepAlive=true as JVM property as #Nirmala suggested doesn’t work.
Setting oracle.jdbc.javaNetNio=false as JVM property as #ibre5041 makes it work. So it points to some problem with the java.nio transport layer.
Anyway, we opened a support issue to Oracle, cause the jdbc api statement.setQueryTimeout(0) should work without having to configure the datasource, I'll put the response when the case is closed.

The query execution could be stopped because of default tcp connection timeout. Can you set keepalive property "oracle.net.keepAlive" to “true” and verify?

Windows OpenNMS - PostgreSQL/Java database issue

OpenNMS - PostgreSQL/Java database issue
I understand that the jicmp files aren't causing fatal errors, but when OpenNMS goes to create a user, there seems to be a Java exception.
Does anybody with experience know what's causing this at all?
Errors dumped from the OpenNMS installer
- using SQL directory... C:\Program Files\OpenNMS\etc
- using create.sql... C:\Program Files\OpenNMS\etc\create.sql
* using 'postgres' as the PostgreSQL user for OpenNMS
* using 'opennms' as the PostgreSQL database name for OpenNMS
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.opennms.bootstrap.Bootstrap$4.run(Bootstrap.java:460)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.opennms.core.schema.MigrationException: an error occurred creating the OpenNMS user
at org.opennms.core.schema.Migrator.createUser(Migrator.java:339)
at org.opennms.core.schema.Migrator.prepareDatabase(Migrator.java:447)
at org.opennms.install.Installer.install(Installer.java:254)
at org.opennms.install.Installer.main(Installer.java:989)
... 6 more
Caused by: org.postgresql.util.PSQLException: ERROR: unrecognized role option "createuser"
Position: 54
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2284)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2003)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:200)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:424)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:321)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:313)
at org.opennms.core.schema.Migrator.createUser(Migrator.java:337)
... 9 more

If you're still interested: the postgres database has changed the CREATE USER command. The argument CREATEUSER is no longer valid and should be changed to CREATEROLE. This should be done in the create.sql file where it tries to create the user.
I had a similar case for OpenCMS in opencms\setup\database\postgresql\create_db.sql where I changed:
#
# replacer = "${database}"
############################
# Create the user;
CREATE USER ${user}
PASSWORD '${password}'
CREATEDB CREATEUSER;
#create the database
CREATE DATABASE ${database}
WITH ENCODING='UNICODE' OWNER=${user};
#commit all (if connection is not autocommit)
commit;
for
#
# replacer = "${database}"
############################
# Create the user;
CREATE USER ${user}
PASSWORD '${password}'
CREATEDB CREATEROLE;
#create the database
CREATE DATABASE ${database}
WITH ENCODING='UNICODE' OWNER=${user};
#commit all (if connection is not autocommit)
commit;
This was after unzipping the war. I rezipped it and then it worked.

How to configure pgpool for reasonable speed

I have a problem with pgpool II configured in master-slave mode with replication on slony.
I have configured postgres databases on each server to run as fast as they can and this was verified by me when I set up connection directly to the db server. However when connection is established by pgpool sth goes wrong, my webapp on tomcat starts very slowly(30min in compare to 5min without pgpool).
I can't figure out what is wrong. On working webapp communication is a bit slower, but difference is so much smaller than at startup.
PGPool use ssh tunnels to communicate with servers.
I've read sth about pgpool and if I understood correctly number of connections(pools) is determine by tuples(database, user). I use only one database and user, is that a reason? If so is there anything I can do to speed up pgpool?
Configuration info:
CentOS 6 x64
Postgres 9.2.4
PGPool II 3.2.5
There are 3 servers, two with postgres database which is replicated by slony and on another one pgpool is running.
My pgpool.conf
#------------------------------------------------------------------------------
# CONNECTIONS
#------------------------------------------------------------------------------
# - pgpool Connection Settings -
listen_addresses = '*'
# Host name or IP address to listen on:
# '*' for all, '' for no TCP/IP connections
# (change requires restart)
port = 5432
# Port number
# (change requires restart)
socket_dir = '/var/run'
# Unix domain socket path
# The Debian package defaults to
# /var/run/postgresql
# (change requires restart)
# - pgpool Communication Manager Connection Settings -
pcp_port = 9898
# Port number for pcp
# (change requires restart)
pcp_socket_dir = '/var/run'
# - Backend Connection Settings -
backend_hostname0 = '127.0.0.1'
# Host name or IP address to connect to for backend 0
backend_port0 = 27930
# Port number for backend 0
backend_weight0 = 1
# Weight for backend 0 (only in load balancing mode)
backend_data_directory0 = '/usr/local/pgsql/data'
# Data directory for backend 0
backend_flag0 = 'ALLOW_TO_FAILOVER'
# Controls various backend behavior
# ALLOW_TO_FAILOVER or DISALLOW_TO_FAILOVER
backend_hostname1 = '127.0.0.1'
backend_port1 = 27931
backend_weight1 = 1
backend_data_directory1 = '/usr/local/pgsql/data'
backend_flag1 = 'ALLOW_TO_FAILOVER'
# - Authentication -
enable_pool_hba = on
# Use pool_hba.conf for client authentication
pool_passwd = ''
# File name of pool_passwd for md5 authentication.
# "" disables pool_passwd.
# (change requires restart)
authentication_timeout = 30
# Delay in seconds to complete client authentication
# 0 means no timeout.
# - SSL Connections -
ssl = off
# Enable SSL support
# (change requires restart)
ssl_key = './server.key'
# Path to the SSL private key file
# (change requires restart)
ssl_cert = './server.cert'
# Path to the SSL public certificate file
# (change requires restart)
ssl_ca_cert = './root.crt'
# Path to a single PEM format file
# containing CA root certificate(s)
# (change requires restart)
#ssl_ca_cert_dir = ''
# Directory containing CA root certificate(s)
# (change requires restart)
#------------------------------------------------------------------------------
# POOLS
#------------------------------------------------------------------------------
# - Pool size -
num_init_children = 512
# Number of pools
# (change requires restart)
max_pool = 2
# Number of connections per pool
# (change requires restart)
# - Life time -
child_life_time = 300
# Pool exits after being idle for this many seconds
child_max_connections = 0
# Pool exits after receiving that many connections
# 0 means no exit
connection_life_time = 0
# Connection to backend closes after being idle for this many seconds
# 0 means no close
client_idle_limit = 0
# Client is disconnected after being idle for that many seconds
# (even inside an explicit transactions!)
# 0 means no disconnection
#------------------------------------------------------------------------------
# LOGS
#------------------------------------------------------------------------------
# - Where to log -
log_destination = 'stderr'
# Where to log
# Valid values are combinations of stderr,
# and syslog. Default to stderr.
# - What to log -
print_timestamp = on
# Print timestamp on each line
# (change requires restart)
log_connections = on
# Log connections
log_hostname = on
# Hostname will be shown in ps status
# and in logs if connections are logged
log_statement = on
# Log all statements
log_per_node_statement = on
# Log all statements
# with node and backend informations
log_standby_delay = 'none'
# Log standby delay
# Valid values are combinations of always,
# if_over_threshold, none
# - Syslog specific -
syslog_facility = 'LOCAL0'
# Syslog local facility. Default to LOCAL0
syslog_ident = 'pgpool'
# Syslog program identification string
# Default to 'pgpool'
# - Debug -
debug_level = 1
# Debug message verbosity level
# 0 means no message, 1 or more mean verbose
#------------------------------------------------------------------------------
# FILE LOCATIONS
#------------------------------------------------------------------------------
pid_file_name = '/usr/local/etc/pgpool.pid'
# PID file name
# (change requires restart)
logdir = '/usr/local/etc/'
# Directory of pgPool status file
# (change requires restart)
#------------------------------------------------------------------------------
# CONNECTION POOLING
#------------------------------------------------------------------------------
connection_cache = on
# Activate connection pools
# (change requires restart)
# Semicolon separated list of queries
# to be issued at the end of a session
# The default is for 8.3 and later
reset_query_list = 'ABORT; DISCARD ALL'
# The following one is for 8.2 and before
#reset_query_list = 'ABORT; RESET ALL; SET SESSION AUTHORIZATION DEFAULT'
#------------------------------------------------------------------------------
# REPLICATION MODE
#------------------------------------------------------------------------------
replication_mode = off
# Activate replication mode
# (change requires restart)
replicate_select = off
# Replicate SELECT statements
# when in replication or parallel mode
# replicate_select is higher priority than
# load_balance_mode.
insert_lock = off
# Automatically locks a dummy row or a table
# with INSERT statements to keep SERIAL data
# consistency
# Without SERIAL, no lock will be issued
lobj_lock_table = ''
# When rewriting lo_creat command in
# replication mode, specify table name to
# lock
# - Degenerate handling -
replication_stop_on_mismatch = off
# On disagreement with the packet kind
# sent from backend, degenerate the node
# which is most likely "minority"
# If off, just force to exit this session
failover_if_affected_tuples_mismatch = off
# On disagreement with the number of affected
# tuples in UPDATE/DELETE queries, then
# degenerate the node which is most likely
# "minority".
# If off, just abort the transaction to
# keep the consistency
#------------------------------------------------------------------------------
# LOAD BALANCING MODE
#------------------------------------------------------------------------------
load_balance_mode = on
# Activate load balancing mode
# (change requires restart)
ignore_leading_white_space = on
# Ignore leading white spaces of each query
white_function_list = ''
# Comma separated list of function names
# that don't write to database
# Regexp are accepted
black_function_list = 'currval,lastval,nextval,setval'
# Comma separated list of function names
# that write to database
# Regexp are accepted
#------------------------------------------------------------------------------
# MASTER/SLAVE MODE
#------------------------------------------------------------------------------
master_slave_mode = on
# Activate master/slave mode
# (change requires restart)
master_slave_sub_mode = 'slony'
# Master/slave sub mode
# Valid values are combinations slony or
# stream. Default is slony.
# (change requires restart)
# - Streaming -
sr_check_period = 0
# Streaming replication check period
# Disabled (0) by default
sr_check_user = 'nobody'
# Streaming replication check user
# This is neccessary even if you disable streaming
# replication delay check by sr_check_period = 0
sr_check_password = ''
# Password for streaming replication check user
delay_threshold = 0
# Threshold before not dispatching query to standby node
# Unit is in bytes
# Disabled (0) by default
# - Special commands -
follow_master_command = ''
# Executes this command after master failover
# Special values:
# %d = node id
# %h = host name
# %p = port number
# %D = database cluster path
# %m = new master node id
# %H = hostname of the new master node
# %M = old master node id
# %P = old primary node id
# %r = new master port number
# %R = new master database cluster path
# %% = '%' character
#------------------------------------------------------------------------------
# PARALLEL MODE
#------------------------------------------------------------------------------
parallel_mode = off
# Activates parallel query mode
# (change requires restart)
pgpool2_hostname = ''
# Set pgpool2 hostname
# (change requires restart)
# - System DB info -
#system_db_hostname = 'localhost'
# (change requires restart)
#system_db_port = 5432
# (change requires restart)
#system_db_dbname = 'pgpool'
# (change requires restart)
#system_db_schema = 'pgpool_catalog'
# (change requires restart)
#system_db_user = 'pgpool'
# (change requires restart)
#system_db_password = ''
# (change requires restart)
#------------------------------------------------------------------------------
# HEALTH CHECK
#------------------------------------------------------------------------------
health_check_period = 10
# Health check period
# Disabled (0) by default
health_check_timeout = 20
# Health check timeout
# 0 means no timeout
health_check_user = 'postgres'
# Health check user
health_check_password = ''
# Password for health check user
health_check_max_retries = 0
# Maximum number of times to retry a failed health check before giving up.
health_check_retry_delay = 1
# Amount of time to wait (in seconds) between retries.
#------------------------------------------------------------------------------
# FAILOVER AND FAILBACK
#------------------------------------------------------------------------------
failover_command = ''
# Executes this command at failover
# Special values:
# %d = node id
# %h = host name
# %p = port number
# %D = database cluster path
# %m = new master node id
# %H = hostname of the new master node
# %M = old master node id
# %P = old primary node id
# %r = new master port number
# %R = new master database cluster path
# %% = '%' character
failback_command = ''
# Executes this command at failback.
# Special values:
# %d = node id
# %h = host name
# %p = port number
# %D = database cluster path
# %m = new master node id
# %H = hostname of the new master node
# %M = old master node id
# %P = old primary node id
# %r = new master port number
# %R = new master database cluster path
# %% = '%' character
fail_over_on_backend_error = on
# Initiates failover when reading/writing to the
# backend communication socket fails
# If set to off, pgpool will report an
# error and disconnect the session.
#------------------------------------------------------------------------------
# ONLINE RECOVERY
#------------------------------------------------------------------------------
recovery_user = 'nobody'
# Online recovery user
recovery_password = ''
# Online recovery password
recovery_1st_stage_command = ''
# Executes a command in first stage
recovery_2nd_stage_command = ''
# Executes a command in second stage
recovery_timeout = 90
# Timeout in seconds to wait for the
# recovering node's postmaster to start up
# 0 means no wait
client_idle_limit_in_recovery = 0
# Client is disconnected after being idle
# for that many seconds in the second stage
# of online recovery
# 0 means no disconnection
# -1 means immediate disconnection
#------------------------------------------------------------------------------
# WATCHDOG
#------------------------------------------------------------------------------
use_watchdog = off
# Activates watchdog
# (change requires restart)
trusted_servers = ''
# trusted server list which are used
# to confirm network connection
# (hostA,hostB,hostC,...)
# (change requires restart)
delegate_IP = ''
# delegate IP address
# (change requires restart)
wd_hostname = ''
# Host name or IP address of this watchdog
# (change requires restart)
wd_port = 9000
# port number for watchdog service
# (change requires restart)
wd_interval = 10
# lifecheck interval (sec) > 0
# (change requires restart)
ping_path = '/bin'
# ping command path
# (change requires restart)
ifconfig_path = '/sbin'
# ifconfig command path
# (change requires restart)
if_up_cmd = 'ifconfig eth0:0 inet $_IP_$ netmask 255.255.255.0'
# startup delegate IP command
# (change requires restart)
if_down_cmd = 'ifconfig eth0:0 down'
# shutdown delegate IP command
# (change requires restart)
arping_path = '/usr/sbin' # arping command path
# (change requires restart)
arping_cmd = 'arping -U $_IP_$ -w 1'
# arping command
# (change requires restart)
wd_life_point = 3
# lifecheck retry times
# (change requires restart)
wd_lifecheck_query = 'SELECT 1'
# lifecheck query to pgpool from watchdog
# (change requires restart)
# Other pgpool Connection Settings
#other_pgpool_hostname0 = 'host0'
# Host name or IP address to connect to for other pgpool 0
#other_pgpool_port0 = 5432
# Port number for othet pgpool 0
#other_wd_port0 = 9000
# Port number for othet watchdog 0
#other_pgpool_hostname1 = 'host1'
#other_pgpool_port1 = 5432
#other_wd_port1 = 9000
#------------------------------------------------------------------------------
# OTHERS
#------------------------------------------------------------------------------
relcache_expire = 0
relcache_size = 256
# Number of relation cache
# entry. If you see frequently:
# "pool_search_relcache: cache replacement happend"
# in the pgpool log, you might want to increate this number.
check_temp_table = on
# If on, enable temporary table check in SELECT statements.
# This initiates queries against system catalog of primary/master
# thus increases load of master.
# If you are absolutely sure that your system never uses temporary tables
# and you want to save access to primary/master, you could turn this off.
# Default is on.
#------------------------------------------------------------------------------
# ON MEMORY QUERY MEMORY CACHE
#------------------------------------------------------------------------------
memory_cache_enabled = off
# If on, use the memory cache functionality, off by default
memqcache_method = 'shmem'
# Cache storage method. either 'shmem'(shared memory) or
# 'memcached'. 'shmem' by default
# (change requires restart)
memqcache_memcached_host = 'localhost'
# Memcached host name or IP address. Mandatory if
# memqcache_method = 'memcached'.
# Defaults to localhost.
# (change requires restart)
memqcache_memcached_port = 11211
# Memcached port number. Mondatory if memqcache_method = 'memcached'.
# Defaults to 11211.
# (change requires restart)
memqcache_total_size = 67108864
# Total memory size in bytes for storing memory cache.
# Mandatory if memqcache_method = 'shmem'.
# Defaults to 64MB.
# (change requires restart)
memqcache_max_num_cache = 1000000
# Total number of cache entries. Mandatory
# if memqcache_method = 'shmem'.
# Each cache entry consumes 48 bytes on shared memory.
# Defaults to 1,000,000(45.8MB).
# (change requires restart)
memqcache_expire = 0
# Memory cache entry life time specified in seconds.
# 0 means infinite life time. 0 by default.
# (change requires restart)
memqcache_auto_cache_invalidation = on
# If on, invalidation of query cache is triggered by corresponding
# DDL/DML/DCL(and memqcache_expire). If off, it is only triggered
# by memqcache_expire. on by default.
# (change requires restart)
memqcache_maxcache = 409600
# Maximum SELECT result size in bytes.
# Must be smaller than memqcache_cache_block_size. Defaults to 400KB.
# (change requires restart)
memqcache_cache_block_size = 1048576
# Cache block size in bytes. Mandatory if memqcache_method = 'shmem'.
# Defaults to 1MB.
# (change requires restart)
memqcache_oiddir = '/var/log/pgpool/oiddir'
# Temporary work directory to record table oids
# (change requires restart)
white_memqcache_table_list = ''
# Comma separated list of table names to memcache
# that don't write to database
# Regexp are accepted
black_memqcache_table_list = ''
# Comma separated list of table names not to memcache
# that don't write to database
# Regexp are accepted
Thanks in advance.