Thanks in advance!
I am new to gcp and am trying to use Cloud SQL Postgres.
I have a database that is created and would want to create/update/delete tables into it using either java or scala.
I am trying to use postgres-socket-factory from 'com.google.cloud.sql' package.
But while creating a connection I get an error that
"java.sql.SQLException: No suitable driver found for jdbc:postgres://google/"
Here is my code
def getConnection(url:String) : Connection = {
Class.forName("org.postgresql.Driver")
import java.sql.DriverManager
DriverManager.getConnection(url)
}
What is the means for Java/Scala clients to access db from Cloud SQL?
Yes after following instructions from comments and changing the url it worked but fails with
Caused by: java.lang.RuntimeException: Unable to obtain credentials to communicate with the Cloud SQL API
at com.google.cloud.sql.core.SslSocketFactory$ApplicationDefaultCredentialFactory.create(SslSocketFactory.java:600)
at com.google.cloud.sql.core.SslSocketFactory.getInstance(SslSocketFactory.java:147)
at com.google.cloud.sql.postgres.SocketFactory.createSocket(SocketFactory.java:91)
at org.postgresql.core.PGStream.<init>(PGStream.java:62)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:454)
at org.postgresql.Driver.connect(Driver.java:256)
... 6 more
Caused by: java.io.IOException: The Application Default Credentials are not available. They are available if running on Google App Engine, Google Compute Engine, or Google Cloud Shell. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getDefaultCredential(DefaultCredentialProvider.java:98)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:213)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:191)
at com.google.cloud.sql.core.SslSocketFactory$ApplicationDefaultCredentialFactory.create(SslSocketFactory.java:598)
... 15 more
How to address this failure?
Regards,
Most libraries on GCP use the Application Default Credentials (ADC) strategy for handling credentials. The README for this project mentions this library does as well. The Cloud SQL JDBC Socket Factory uses these credentials to authenticate the connections against your database.
The quickest way is to use gcloud auth application-default login, which will set your personal credentials as the default. However, the safest and most secure way is to create a Service Account for the application, grant it the "Cloud SQL Client" IAM role, and use GOOGLE_APPLICATION_CREDENTIALS environment variable to pass the location of the service account key to the library.
Related
Requirement
Remove DB credentials from Java Code(property files) to AWS SM.
Implement autorotation of DB credentials.
Problem Statement
Though we are able to retrieve DB credentials from AWS SM from our application, but we are facing below issues during auto rotation of passwords:
How Java Code will identify that DB passwords are rotated by AWS SM
All the instances of application should be updated with new DB credentials after automatic password rotation from AWS SM.
Proposed Solution
Solution 1
Whenever passwords are rotated, java application won’t be able to
connect to DB.
At that time, we will get SQL Connection exception (Connection lost
exception) in our application.
Java Application will catch the exception & then add a mechanism to
retrieve the DB secrets again from AWS SM.
Set up new Db connection with the updated credentials.
Step 3 & 4 would be done for all the instances of the application
Solution 2
We can call refresh method and will set up new DB connection
automatically & avoid SQL Connection exception .
Is there any way without any db connection issues? we can rotate db password using aws SM
Yes, there is an AWS Secrets Manager JDBC Library which is basically a wrapper to common JDBC drivers with support to AWS Secrets.
This wrapper tries to connect to the database. If an authentication exception is caught, it will refresh the secrets to provide a valid connection.
Here are the two steps to configure your spring boot application.
1 - Add the dependency to your pom.xml
<dependency>
<groupId>com.amazonaws.secretsmanager</groupId>
<artifactId>aws-secretsmanager-jdbc</artifactId>
<version>1.0.7</version>
</dependency>
2 - Setup the database connection on your application.yaml
spring:
datasource:
url: jdbc-secretsmanager:mysql://database-host:3306/rotate_db
username: secret/rotation
driver-class-name: com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver
The username is actually the secret name created at AWS Secrets.
Make sure to use the right URL, in this example it is a URL for MySQL.
I'm hosting mysql server on a Digital Ocean droplet, and I am trying to figure out how to grant a Spring Boot web application access to the droplet, so that it can connect to the database. I configured the droplet so that it can only be reached via an ssh tunnel (i.e.: I disabled password authentication), but the database server itself can be connected with a username and password.
I know how to connect to the database using Connector/J, configuring datasources, and so forth. However, the extra security layer of Digital Ocean is new to me, and I'm not sure how to approach this problem.
EDIT: When I run the application and try to hit an endpoint, I get the following error:
The error occurred while executing a query
### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLException: null, message from server: "Host '<my network hostname>' is not allowed to connect to this MySQL server"] with root cause
So, I was mistaken in thinking that the authentication issue was with the Digital Ocean droplet. As the error message (see question above) indicates, the Spring Boot application was able to get to the remote mysql server, and the authentication issue occurred there. It turns out that the mysql user I was attempting to connect with could only be used on localhost.
So, I created a new mysql user and tied it to my local machine's public IP address, and that solved the issue. For details on how to accomplish this, please read the following answer:
Host 'xxx.xx.xxx.xxx' is not allowed to connect to this MySQL server
I am writing a simple Java client that goes to a workspace server to retrieve some data. Workspace server does not allow public connections and the authentication mechanism is "SAS Token Authentication".
When I attempt to connect to the server, I get an error for invalid credentials. The same set of credentials work just fine when I attempt to connect to SAS Metadata server in Java code. I have full access to the workspace in questions in SAS EG.
SAS institute (I filed a support track) says that my connection needs to be SAS Metaserver aware, in other words I should have an active connection with SAS Metadata server and then connect with the same credentials to a workspace server of choice. The workspace server in turn will validate credentials against a list of active connections on Metadataserver.
I can do the first part no problem, I get a connection with metadata server going, but when I attempt to connect to a workspace server while maintaining connection with metadata, my connection is denied saying username/password are incorrect.
I have used this sample code where I deleted methods for creating data tables/columns on metadata and instead put another method that starts a workspace factory with the same credentials after initiating a successful connection with metadata server, but before closing it. http://support.sas.com/documentation/cdl/en/omaref/63063/HTML/default/viewer.htm#p0zlnmq1vv8xgbn1hshp4rmw2btd.htm
I had posted this on LinkedIn where some suggestions were around token authentication, that I don't need to send a second set of credentials and I should obtain a token from metadata, use that to connect. The only reference to a token object that I found is on here: http://support.sas.com/rnd/javadoc/93/Foundation/com/sas/services/user/UserContextInterface.html#SHAREDRESOURCEKEY_ENVIRONMENT
Any general suggestions? Have I got something conceptually wrong here? Does anyone have a working sample of connecting to workplace server which uses SAS Token Authentication?
Regards,
Vasilij
I have created a java web servlet using app engine, the servlet makes requests to a database. I have tested the servlet locally using a local database and it worked perfectly, i then proceeded to test the servlet locally but istead accessed the Cloud SQL database, this also worked perfectly.
My problem arises after i deploy the servlet. Once deployed all database requests return the following:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not
received any packets from the server.
I checked within the cloud console, and my app was properly added to the cloud SQL Authorized App Engine Applications under the Access Control tab.
Has anyone had similar problems with deployed app engine servlets? Any solutions or advice out there? I would appreciate any and all help!!!
UPDATE:
The above error was generated using the following code to access the db
Class.forName("com.mysql.jdbc.Driver");
Url = "jdbc:mysql://<ip-address-cloudsql>:3306/<dbname>";
Connection con = DriverManager.getConnection (Url,"root",<password>);
the same error was acheived using this code, note that it is very similar to the code shown in the example here https://developers.google.com/appengine/docs/java/cloud-sql/
Class.forName("com.mysql.jdbc.GoogleDriver");
Url = "jdbc:google:mysql://<appID:instanceID>/<dbname>?
user=root&password=<password>";
Connection con = DriverManager.getConnection (Url);
I followed the formatting tips show in this stackoverflow post when it came to setting the url using appid and instance id:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
Using this code resulted in the following different error:
java.sql.SQLException: Access denied for user 'root'#'localhost' (using password: YES)
I'm assuming it says localhost because my cloudsql database is set to follow the app engine servlet. Also, know that both of these methods worked fine when running the servlet locally and accessing the cloud sql database.
any thoughts? i don't know what else to try:[
When connecting to Cloud SQL from an authorized App Engine application, the password is not required (actually it will fail if you try to connect with password).
Change your connection string to jdbc:google:mysql://<appID:instanceID>/<dbname>?
user=root omitting the &password=<password> part
If you have Authorized App Engine Applications you app engine on the access control settings you do not need a password since it is local so just make you password= ""; However if you are using something remote for example phpmyadmin that is run from another host, your command line or a GCE VM that runs through a TCP , SSH or HTML you will need to have a password ="something"; where something is set by you in your access control.
To everyone from Google who are looking as to why you might be getting "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure" on a connection.
Make sure your IP is allowed if you are calling from a test server.
I was testing at a friends house, and this unhelpful error kept showing up.
When connecting to Google Cloud Sql you should be careful:
-To close your opened connections
-To use Exponential backoff algorithm when trying to create new connection.
For more information see: https://cloud.google.com/sql/faq
If you're using application.properties in Spring Boot app, then just put the below line into application.properties:
spring.datasource.url: jdbc:mysql://google/<dbname>?cloudSqlInstance=<InstanceName>&socketFactory=com.google.cloud.sql.mysql.SocketFactory&user=****&password=****
I am using the jTDS driver in order to connect to an SQL Server database from my Android application, which uses the Windows Authentication. As advised in the FAQs, I read the READMESSO file and as told, I placed the native SPPI library (ntlmauth.dll) in the system path (defined by the PATH system variable)
However, when I try to connect to the database using the following code:
String driver = "net.sourceforge.jtds.jdbc.Driver";
Class.forName(driver).newInstance();
String connString = "jdbc:jtds:sqlserver://192.168.56.1/MyMovies;";
Conncection conn = DriverManager.getConnection(connString);
I get the following exception:
java.sql.SQLException: Single-Sign-On is only supported on Windows. Please specify a user name.
Since you are connecting from an android device, you would not be able to get the SSO credentials required by the driver to connect to SQL server. The setting you referred to works only if the java program trying to connect to the DB is on a windows machine, which is clearly mentioned by the error message.
Unless your application has authorization based on the SSO user connecting to the DB, you should have an SQL Server user-based authentication mechanism to connect to the server and all authorization procedures should be tied to this user.
You might have to give the username also.
"jdbc:jtds:sqlserver://192.168.56.1/MyMovies;instance=SQLEXPRESS;user=foo"