Requirement
Remove DB credentials from Java Code(property files) to AWS SM.
Implement autorotation of DB credentials.
Problem Statement
Though we are able to retrieve DB credentials from AWS SM from our application, but we are facing below issues during auto rotation of passwords:
How Java Code will identify that DB passwords are rotated by AWS SM
All the instances of application should be updated with new DB credentials after automatic password rotation from AWS SM.
Proposed Solution
Solution 1
Whenever passwords are rotated, java application won’t be able to
connect to DB.
At that time, we will get SQL Connection exception (Connection lost
exception) in our application.
Java Application will catch the exception & then add a mechanism to
retrieve the DB secrets again from AWS SM.
Set up new Db connection with the updated credentials.
Step 3 & 4 would be done for all the instances of the application
Solution 2
We can call refresh method and will set up new DB connection
automatically & avoid SQL Connection exception .
Is there any way without any db connection issues? we can rotate db password using aws SM
Yes, there is an AWS Secrets Manager JDBC Library which is basically a wrapper to common JDBC drivers with support to AWS Secrets.
This wrapper tries to connect to the database. If an authentication exception is caught, it will refresh the secrets to provide a valid connection.
Here are the two steps to configure your spring boot application.
1 - Add the dependency to your pom.xml
<dependency>
<groupId>com.amazonaws.secretsmanager</groupId>
<artifactId>aws-secretsmanager-jdbc</artifactId>
<version>1.0.7</version>
</dependency>
2 - Setup the database connection on your application.yaml
spring:
datasource:
url: jdbc-secretsmanager:mysql://database-host:3306/rotate_db
username: secret/rotation
driver-class-name: com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver
The username is actually the secret name created at AWS Secrets.
Make sure to use the right URL, in this example it is a URL for MySQL.
Related
I was try to doing different connections with keycloak and MySQL, and always i get the next error:
Operation ("add") failed - address: ([("subsystem" => "microprofile-metrics-smallrye")]): java.lang.NullPointerException
I read differents articles about this problem but anyone don't give me a concrete answer, I try with mariadb and the problem is the same and once i used the integrate gui that have keycloack on the url: http://127.0.0.1:9990/ but nothing changed, the connection take some minutes and create some of tables but finished with the last error
In this post https://keycloak.discourse.group/t/wildfly-nullpointerexception/747, the user says that each time keycloak is deployed, it generates a new random password for the database.
Therefore, while the database password remained the same, keycloak always tried to use its own newly generated password to connect to the DB.
That person's solution was to set the DB password explicitly in the keycloak config file (values.yml).
I have setted up an Oracle cloud Autonomous transaction Processing (ATP) database, but when I am trying to connect it to my Java Spring boot application, it is throwing error:
java.sql.SQLRecoverableException: IO Error: An existing connection was forcibly closed by the remote host, Authentication lapse 0 ms."
I have correctly configured tnsnames.ora file and also sqlnet.ora file with proper location of credentials folder. But still getting above error.
My application.properties file is as below:
spring.datasource.url=jdbc:oracle:thin:#sampledb_high?TNS_ADMIN=E:\\Oracle cloud POC\\Wallet_sampleDB
spring.datasource.username=ADMIN
spring.datasource.password=********
spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
Please let me know how to solve this.
Download the wallet from the UI
then unzip the wallet zip into a folder
for spring boot configuration
spring.datasource.url=jdbc:oracle:thin:#ddf9jcwvp9o3z7aa_high?TNS_ADMIN=/home/mintozzy/Downloads/Wallet_DDF9JCWVP9O3Z7AA
spring.datasource.username=READONLY_USER
spring.datasource.password=qw27dLDW4uDZf9D
You can see running spring boot petclinic example here for full properties
here you can see how to connect sql developer to cloud db.
Thanks in advance!
I am new to gcp and am trying to use Cloud SQL Postgres.
I have a database that is created and would want to create/update/delete tables into it using either java or scala.
I am trying to use postgres-socket-factory from 'com.google.cloud.sql' package.
But while creating a connection I get an error that
"java.sql.SQLException: No suitable driver found for jdbc:postgres://google/"
Here is my code
def getConnection(url:String) : Connection = {
Class.forName("org.postgresql.Driver")
import java.sql.DriverManager
DriverManager.getConnection(url)
}
What is the means for Java/Scala clients to access db from Cloud SQL?
Yes after following instructions from comments and changing the url it worked but fails with
Caused by: java.lang.RuntimeException: Unable to obtain credentials to communicate with the Cloud SQL API
at com.google.cloud.sql.core.SslSocketFactory$ApplicationDefaultCredentialFactory.create(SslSocketFactory.java:600)
at com.google.cloud.sql.core.SslSocketFactory.getInstance(SslSocketFactory.java:147)
at com.google.cloud.sql.postgres.SocketFactory.createSocket(SocketFactory.java:91)
at org.postgresql.core.PGStream.<init>(PGStream.java:62)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:454)
at org.postgresql.Driver.connect(Driver.java:256)
... 6 more
Caused by: java.io.IOException: The Application Default Credentials are not available. They are available if running on Google App Engine, Google Compute Engine, or Google Cloud Shell. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getDefaultCredential(DefaultCredentialProvider.java:98)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:213)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:191)
at com.google.cloud.sql.core.SslSocketFactory$ApplicationDefaultCredentialFactory.create(SslSocketFactory.java:598)
... 15 more
How to address this failure?
Regards,
Most libraries on GCP use the Application Default Credentials (ADC) strategy for handling credentials. The README for this project mentions this library does as well. The Cloud SQL JDBC Socket Factory uses these credentials to authenticate the connections against your database.
The quickest way is to use gcloud auth application-default login, which will set your personal credentials as the default. However, the safest and most secure way is to create a Service Account for the application, grant it the "Cloud SQL Client" IAM role, and use GOOGLE_APPLICATION_CREDENTIALS environment variable to pass the location of the service account key to the library.
First of all, I created PostgreSQL DB in Heroku. I manually created my DB/schema/tables through pgAdmin remote access. I know I succeeded since it updated my row limit.
Now, I am deploying a Spring Boot application.
No DB props/credentials are found in my application.properties file since I am supposed to do this in Heroku Config Vars. For example, my DB username is janxgspmlpjgbn
Project builds successfully, however, in the logs I see that it has this exception.
2016-09-28 16:12:31.184 [ERROR] org.apache.juli.logging.DirectJDKLog.log:181 - Unable to create initial connections of pool.
2016-09-28T16:12:31.185772+00:00 app[web.1]: org.postgresql.util.PSQLException: FATAL: password authentication failed for user "u24453"
Question
Why does it fail for the user u24453 (by the way this user changes every time I redeploy) when in my Config Var, I typed the user janxgspmlpjgbn?
Edit
Here's how my Config Vars look right now:
u24453 is probably the OS user name (the one Heroku creates for your app as it runs on Ubuntu Linux).
I believe the Postgres Driver will use the OS username as your DB username by default (if a username is not provided). Thus, I suspect the DB username is not getting into the configuration properly.
In your application.yml you should have this:
spring:
datasource:
url: ${JDBC_DATABASE_URL}
username: ${JDBC_DATABASE_USERNAME}
password: ${JDBC_DATABASE_PASSWORD}
For more info see Connecting to Relational Databases on Heroku with Java
I have created a java web servlet using app engine, the servlet makes requests to a database. I have tested the servlet locally using a local database and it worked perfectly, i then proceeded to test the servlet locally but istead accessed the Cloud SQL database, this also worked perfectly.
My problem arises after i deploy the servlet. Once deployed all database requests return the following:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not
received any packets from the server.
I checked within the cloud console, and my app was properly added to the cloud SQL Authorized App Engine Applications under the Access Control tab.
Has anyone had similar problems with deployed app engine servlets? Any solutions or advice out there? I would appreciate any and all help!!!
UPDATE:
The above error was generated using the following code to access the db
Class.forName("com.mysql.jdbc.Driver");
Url = "jdbc:mysql://<ip-address-cloudsql>:3306/<dbname>";
Connection con = DriverManager.getConnection (Url,"root",<password>);
the same error was acheived using this code, note that it is very similar to the code shown in the example here https://developers.google.com/appengine/docs/java/cloud-sql/
Class.forName("com.mysql.jdbc.GoogleDriver");
Url = "jdbc:google:mysql://<appID:instanceID>/<dbname>?
user=root&password=<password>";
Connection con = DriverManager.getConnection (Url);
I followed the formatting tips show in this stackoverflow post when it came to setting the url using appid and instance id:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Using this code resulted in the following different error:
java.sql.SQLException: Access denied for user 'root'#'localhost' (using password: YES)
I'm assuming it says localhost because my cloudsql database is set to follow the app engine servlet. Also, know that both of these methods worked fine when running the servlet locally and accessing the cloud sql database.
any thoughts? i don't know what else to try:[
When connecting to Cloud SQL from an authorized App Engine application, the password is not required (actually it will fail if you try to connect with password).
Change your connection string to jdbc:google:mysql://<appID:instanceID>/<dbname>?
user=root omitting the &password=<password> part
If you have Authorized App Engine Applications you app engine on the access control settings you do not need a password since it is local so just make you password= ""; However if you are using something remote for example phpmyadmin that is run from another host, your command line or a GCE VM that runs through a TCP , SSH or HTML you will need to have a password ="something"; where something is set by you in your access control.
To everyone from Google who are looking as to why you might be getting "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure" on a connection.
Make sure your IP is allowed if you are calling from a test server.
I was testing at a friends house, and this unhelpful error kept showing up.
When connecting to Google Cloud Sql you should be careful:
-To close your opened connections
-To use Exponential backoff algorithm when trying to create new connection.
For more information see: https://cloud.google.com/sql/faq
If you're using application.properties in Spring Boot app, then just put the below line into application.properties:
spring.datasource.url: jdbc:mysql://google/<dbname>?cloudSqlInstance=<InstanceName>&socketFactory=com.google.cloud.sql.mysql.SocketFactory&user=****&password=****