We Keep Coding

Trouble establishing a simple connection with JDBC to MySql, java.sql.SQLNonTransientConnectionException

I fail to establish a connection to a MySql database on an external server when I use the mysql-connector-java version 8.0.11+ (I was currently testing with 8.0.25). However I am able to create a connection when I use the older MySql connector mysql-connector-java version 5.1.49.
The version of the MySql Database is "8.0.25-15"
The code fails on the first line:
try (Connection conn = DriverManager.getConnection(DB_URL, USER, PASS);
{...}
Since I can connect though mysql-connector-java version 5.1.49, I first thought I could solve this problem by adding parameters to the DB_URL. I have experimented with a lot of parameters, but no luck:
useSSL=false
serverTimezone=GMT
useUnicode=true
characterEncoding=utf-8
passwordCharacterEncoding=utf-8
connectionCollation=utf8mb4_bin
autoReconnect=true
failOverReadOnly=false
maxReconnects=10
cacheServerConfiguration=false
The error I always receive is:
java.sql.SQLNonTransientConnectionException: Could not create connection to database server.
Caused by: java.lang.NullPointerException: Cannot invoke "String.toUpperCase(java.util.Locale)" because "javaEncoding" is null
at com.mysql.cj.CharsetMapping.getMysqlCharsetForJavaEncoding(CharsetMapping.java:552)
at com.mysql.cj.CharsetMapping.getCollationIndexForJavaEncoding(CharsetMapping.java:585)
at com.mysql.cj.protocol.a.NativeServerSession.configureCharacterSets(NativeServerSession.java:452)
at com.mysql.cj.jdbc.ConnectionImpl.initializePropsFromServer(ConnectionImpl.java:1329)
at com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:866)
A month ago, I did not have this problem, so I suspect the cause is a change in the database (I am not tbe database Administrator).
When I debug, the error seems to be caused because the program requests the "character_set_system" variable from MySql, which is"utf8mb3" (instead "utf8" or "utf8mb4").
I suspect that changing the database variable "utf8mb3" to "utf8" might solve this problem? But I am not certain and I cannot simply change this value because the database is hosted by an external company. I have typed "Show Variables" and "Show Global Variables" in a MySql editor for your information:
A screenshot of the DB variables concerning language.
Alternatively I would like to tell java to ignore "character_set_system". It seems that Java first looks for "local.character_set_results", but this variable returns null despite that "character_set_results" is defined in the database.
I hope someone can help me with this problem.
Kind Regards
Steven

Kafka "Login module not specified in JAAS config"

I have a problem communicating with Kafka secured with sasl using console scripts. Kafka is secured with sasl, listener is SASL_PLAINTEXT and mechanism is PLAIN.
What I did:
I tried listing some data using one of kafka scripts:
bin/kafka-consumer-groups.sh --bootstrap-server (address) --list
However I get
WARN Bootstrap broker (address) disconnected (org.apache.kafka.clients.NetworkClient)
and command fails, which is understandable because it's secured with sasl.
So I tried how to add client username/password to that command.
First, I tried to run kafka-console-consumer script, I used --command-config to add necessary file. I quickly discovered that I can't add jaas file directly and I needed to use .properties file, so I did.
My properties file(keep in mind that brackets indicate "censored" data, I can't put all real data here):
bootstrap.servers=(address)
zookeeper.connect=127.0.0.1:2181
zookeeper.connection.timeout.ms=6000
sasl.jaas.config=(path)/consumer_jaas.conf
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
group.id=(group)
My jaas file:
KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=(username)
password=(password);
};
This jaas file works in my standard java applications.
However, when I'm trying to run either kafka-consumer-groups script or kafka-console-consumer, I get this error:
Exception in thread "main" org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException: Login module not specified in JAAS config
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:94)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:93)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:51)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:84)
at kafka.admin.AdminClient$.create(AdminClient.scala:229)
at kafka.admin.AdminClient$.create(AdminClient.scala:223)
at kafka.admin.AdminClient$.create(AdminClient.scala:221)
at kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.createAdminClient(ConsumerGroupCommand.scala:454)
at kafka.admin.ConsumerGroupCommand$KafkaConsumerGroupService.<init>(ConsumerGroupCommand.scala:389)
at kafka.admin.ConsumerGroupCommand$.main(ConsumerGroupCommand.scala:65)
at kafka.admin.ConsumerGroupCommand.main(ConsumerGroupCommand.scala)
Caused by: java.lang.IllegalArgumentException: Login module not specified in JAAS config
at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:68)
at org.apache.kafka.common.security.JaasUtils.jaasConfig(JaasUtils.java:59)
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:85)
This jaas file is a direct copy of a file that I'm using in java app that communicates with kafka and it works, however here, using console tools, it just doesn't work. I tried searching for a solution but I can't find anything useful.
Can anyone help me with this?

There are 2 ways to provide the JAAS configuration to the Kafka clients.
Via the client property: sasl.jaas.config. In that case you set it to the actual JAAS configuration entry. For example, your configuration file becomes:
bootstrap.servers=(address)
zookeeper.connect=127.0.0.1:2181
zookeeper.connection.timeout.ms=6000
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="(username)" password="(password)";
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
group.id=(group)
As you've already figured out, you can use --command-config to pass a properties file to kafka-consumer-groups.sh.
Via the Java property: java.security.auth.login.config. In this case, you set it to the path of your JAAS file. Also if you set it in KAFKA_OPTS, kafka-consumer-groups.sh will pick it up automatically.
export KAFKA_OPTS="-Djava.security.auth.login.config=(path)/consumer_jaas.conf"

Error when trying to open a JDBC connection to Google Cloud Spanner

I am trying to open a JDBC connection to a Google Cloud Spanner database, but I get the following error message:
java.lang.IllegalArgumentException: A project ID is required for this
service but could not be determined from the builder or the
environment. Please set a project ID using the builder.
My JDBC URL is as follows:
jdbc:cloudspanner://localhost;Project=project-id;Instance=instance-id;Database=database-name;PvtKeyPath=path-to-key-file
If I remove the Project property from the URL, I get the following exception:
java.sql.SQLNonTransientConnectionException: [Simba]JDBC
Connection Refused: [Simba]JDBC Required Connection Key(s):
Project; [Simba]JDBC Optional Connection Key(s): Language,
Mode
So it seems that the driver does pick up my Project ID, but somehow does not accept it. I have checked and double checked that my project id does equal the project id that I created on Google, I have also tried to change the value to the project name instead of the project id, but to no avail.
Does anyone have a URL example that works?
EDIT: It appears to be related to the reference to the private key file. If I make an environment variable GOOGLE_APPLICATION_CREDENTIALS pointing to my private key file, the connection can successfully be made. If I remove this environment variable, I get the above exception.

Which version of the driver are you using? In the latest version, if you are specifying the path to the credentials file in the URL then you need not set GOOGLE_APPLICATION_CREDENTIALS.

As the JDBC Driver supplied by Google is severely limited (does not support DML and DDL statemetns), I have written my own JDBC Driver. The driver is designed to work with JPA/Hibernate-enabled applications. The driver can be found here: https://github.com/olavloite/spanner-jdbc
This driver supports the same kind of URL's as the driver supplied by Google, including the PvtKeyPath property.

Spring boot RabbitMQ Host string format

I am using spring boot to listen for messages on my rabbitmq instance. I have it working locally using an application.properties file.
However, when I want to connect to the remote rabbit instance I am getting number format exceptions and unknown host exceptions because of my url.
I've tried setting spring.rabbitmq.host to:
amqp://myurl/dev and myurl/dev and amqp%3A%2F%2Fmyurl%2Fdev
Nothing is working. Any ideas what could be up. I have set my user name and pass for the remote instance as well.

Try removing the amqp:// part. The host property should just be that, the hostname:
spring.rabbitmq.host=myurl
Not sure what the /dev part is. Are you saying that is your virtual-host? If so then just set this property too:
spring.rabbitmq.virtual-host=dev
For username/password, set these properties:
spring.rabbitmq.username=
spring.rabbitmq.password=

A better way to do this is to set
spring.rabbitmq.addresses=amqps://user:password#10.0.0.123/virtualhost
This will get auto-resolved, and correctly set your hostname, password etc
Github issue that fixes this:
https://github.com/spring-projects/spring-boot/issues/6401

Glassfish - No suitable driver for mysql when trying to connect with datasource

I am trying to complete this tutorial:
https://netbeans.org/kb/docs/javaee/ecommerce/connect-db.html
Part of it is setting up a database and trying to establish a connection using datasource and connection pooling.
I did everything that is in the tutorial but when i try to run my code i get the following error:
javax.servlet.jsp.JspException: Unable to get connection, DataSource invalid: "java.sql.SQLException: No suitable driver found for jdbc/affablebean"
Here is the code that throws it:
<sql:query var = "result" dataSource = "jdbc/affablebean">
SELECT * FROM category, product
WHERE category.id = product.category_id
</sql:query>
I tried to connect to the database without using connection pooling and datasource and it worked like a charm.
try {
Class.forName("com.mysql.jdbc.Driver");
System.out.println("Driver registered");
}
catch (ClassNotFoundException ex) {
Logger.getLogger(SqlService.class.getName()).log(Level.SEVERE, null, ex);
}
That means that the driver is in the right lib folder in the glassfish directory so the problem must be somewhere else.
The tutorial has a troubleshooting section where they describe that if i get this kind of error "No suitable driver found for jdbc/affablebean" that means that i do not have resource reference in my web.xml. Well... I DO HAVE ONE and here it is:
<resource-ref>
<description>Connects to database for AffableBean application</description>
<res-ref-name>jdbc/affablebean</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
<res-sharing-scope>Shareable</res-sharing-scope>
</resource-ref>
I have been trying to solve this problem 16 hours a day for more than two day but still NO luck.
Do you guys have some idea what is wrong?
I browsed all the google and stackoverflow and found similar problems but the solution they give is "Make sure the mysql driver is in the right folder on the server".
Why do i connect to the database without dataSource object but can not connect with one?

So i found the solution to the exact same problem I was having.
So it appears that the issue was within web.xml not containing a reference to the datasource.
Simply add,
<resource-ref>
<description>AffableBean DataSource</description>
<res-ref-name>jdbc/affablebean</res-ref-name>
<res-type>javax.sql.ConnectionPoolDataSource</res-type>
<res-auth>Container</res-auth>
</resource-ref>
to your web.xml file and save and run file(right click inside the file and press "Run File").
Afterwards you should see the database tab pop up in your browser of choice.

I had same error,
the problem is on when you set resource-type:
Resource Type: javax.sql.ConnectionPoolDataSource
try using:
Resource Type:=javax.sql.DataSource
thats strange in tutorial text is (the problem):
Click Next. In Step 5, Add Connection Pool Properties, specify the
following details: Datasource Classname:
com.mysql.jdbc.jdbc2.optional.MysqlDataSource Resource Type:
javax.sql.ConnectionPoolDataSource Description: (Optional) Connects to
the affablebean database
but on screenshot is correct.

I had several issues with this
https://netbeans.org/kb/docs/javaee/ecommerce/connect-db.html
part of ecommerce tutorial.
First problem (no suitable driver fourd) appeared due to improper jdbc resource creation -
i used 'new file \ glassfish \ jdbc connection pool'
instead of 'new file \ glassfish \ jdbc resource'.
Redoing this step registered driver correctly and created pool
(was indicated in glassfish servel log that mysql driver did register on server instance).
After that i couldn't get data via testDataSource.jsp - error was org.apache.derby.client.am.SqlException: Table/View 'CATEGORY' does not exist.
I couldn't figure out why org.apache.derby.client was even mentioned and
after some lookup found discussion on netbeans forum:
https://forums.netbeans.org/ntopic61746.html
This part of discussion was solution for my case:
try to change in step setup file sun-resources.xml or glassfish-resources.xml
Setting up a JDBC data source and connection pool
at step 6 -> change Resource Type to javax.sql.ConnectionPoolDataSource
and change in step setup file web.xml
Referencing the data source from the application
at step 4 -> change Resource Type to javax.sql.ConnectionPoolDataSource
p.s. i'm using netbeans 8.0.2, glassfish 4.1 and jdk1.7.0_21.
There was no javax.sql.ConnectionPoolDataSource option in Netbeans'
interface for 'edit resource reference \ resource type' combobox for web.xml.
So i put this value there manually.

After a very tiresome research, having checked servers tomcat, wildfly,and Jboss, none seemed to work,I had problem with registering new datasource with glassfish, adding new jdbc connectionpool would throw java.lang.runtimeexception, the workaround for this issue was to reconfigure DerbyPool under jdbcconnection pools in admin console and feeding it with required Datasource Classname, url,username,and password to point it the database on mysql server.and it did work.

I think you did not add Mysql JDBC Driver in your project libraries.
Add manually Mysql JDBC Driver in your project libraries and try again.
I think it will work.

I have the exactly problem, and even thru Netbeans IDE, the DB connection test is fine while in Glassfish 4, it is not working, try all possible like include lib in project, put here and there. Finally the problem is fixed by remove Glassfish and install Glassfish 3.1.2.2, the project files is exactly same and working fine. Guess this is a bug or specail setting may needed in Glassfish 4.

I had the same issue. The last 4 hours I was looking for a solution and it was so easy for me...
Make sure the "Status" in JDBC Resource (at Glassfish Admin Console) is Enabled (checkbox is checked) :-(

Same problem here. I spent hours with this, tried some of the solutions offered here, and only one got me a step further: changing the Resource Type from "javax.sql.ConnectionPoolDataSource" to "javax.sql.DataSource.
I immediately stumbled upon the next problem: the table "category" didn't exist. It actually does exist.
I gave up on glassfish (this was my first time trying to use glassfish) and went back to Tomcat. Succes! So I decided it might be helpful to point people to another possible solution that worked for me.
For those who are interested in the Tomcat solution, visit this page http://tomcat.apache.org/tomcat-8.0-doc/jndi-datasource-examples-howto.html#MySQL_DBCP_Example. Very clear explanation with an example. And don't forget to add the JSTL library to your project.
Good luck.