I am attempting to verify Google ID tokens with a backend server as per:
https://developers.google.com/identity/sign-in/android/backend-auth
The tokens are initially retrieved by an android app, and are then passed to a backend login server via sockets which attempts verification. As things stand, there is an error thrown at runtime within the GoogleIdTokenVerifier code I am importing.
ServerThread.java:
GoogleIdToken idToken = GoogleAuthenticator.authenticateToken(tokenJson.getToken());
GoogleAuthenticator.java:
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.Collections;
import java.util.Properties;
import com.google.api.client.googleapis.auth.oauth2.GoogleIdToken;
import com.google.api.client.googleapis.auth.oauth2.GoogleIdTokenVerifier;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.extensions.appengine.http.UrlFetchTransport;
public class GoogleAuthenticator {
public GoogleAuthenticator(){
}
public static Properties prop;
public static GoogleIdToken authenticateToken(String inputToken) throws IOException{
final JacksonFactory jacksonFactory = new JacksonFactory();
prop = new Properties();
prop.load(GoogleAuthenticator.class.getClassLoader().getResourceAsStream("config.properties"));
GoogleIdTokenVerifier verifier = new GoogleIdTokenVerifier.Builder(UrlFetchTransport.getDefaultInstance(), jacksonFactory)
// Specify the CLIENT_ID of the app that accesses the backend:
.setAudience(Collections.singletonList(prop.getProperty("google.web.client.id")))
// Or, if multiple clients access the backend:
//.setAudience(Arrays.asList(CLIENT_ID_1, CLIENT_ID_2, CLIENT_ID_3))
.build();
GoogleIdToken idToken;
try {
idToken = verifier.verify(inputToken);
return idToken;
} catch (GeneralSecurityException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return null;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return null;
}
}
}
pom.xml
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.25.0</version>
</dependency>
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client-appengine</artifactId>
<version>1.25.0</version>
</dependency>
I am currently seeing the following stack trace:
Exception in thread "Thread-0" java.lang.NoClassDefFoundError: com/google/appengine/api/urlfetch/HTTPMethod
at com.google.api.client.extensions.appengine.http.UrlFetchTransport.buildRequest(UrlFetchTransport.java:118)
at com.google.api.client.extensions.appengine.http.UrlFetchTransport.buildRequest(UrlFetchTransport.java:50)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:872)
at com.google.api.client.googleapis.auth.oauth2.GooglePublicKeysManager.refresh(GooglePublicKeysManager.java:172)
at com.google.api.client.googleapis.auth.oauth2.GooglePublicKeysManager.getPublicKeys(GooglePublicKeysManager.java:140)
at com.google.api.client.googleapis.auth.oauth2.GoogleIdTokenVerifier.verify(GoogleIdTokenVerifier.java:174)
at com.google.api.client.googleapis.auth.oauth2.GoogleIdTokenVerifier.verify(GoogleIdTokenVerifier.java:192)
at com.omarhegazi.login.GoogleAuthenticator.authenticateToken(GoogleAuthenticator.java:40)
at com.omarhegazi.login.ServerThread.run(ServerThread.java:45)
It looks like there's no HTTPMethod class available amongst the dependencies. Any thoughts?
UPDATE 1:
Adding the appengine dependency below has made some progress. I now have the following stack trace error:
Exception in thread "Thread-0" com.google.apphosting.api.ApiProxy$CallNotFoundException: The API package 'urlfetch' or call 'Fetch()' was not found.
at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98)
at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:37)
at com.google.api.client.extensions.appengine.http.UrlFetchRequest.execute(UrlFetchRequest.java:74)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.api.client.googleapis.auth.oauth2.GooglePublicKeysManager.refresh(GooglePublicKeysManager.java:172)
at com.google.api.client.googleapis.auth.oauth2.GooglePublicKeysManager.getPublicKeys(GooglePublicKeysManager.java:140)
at com.google.api.client.googleapis.auth.oauth2.GoogleIdTokenVerifier.verify(GoogleIdTokenVerifier.java:174)
at com.google.api.client.googleapis.auth.oauth2.GoogleIdTokenVerifier.verify(GoogleIdTokenVerifier.java:192)
at com.omarhegazi.login.GoogleAuthenticator.authenticateToken(GoogleAuthenticator.java:40)
at com.omarhegazi.login.ServerThread.run(ServerThread.java:45)
This was using v1.6.1
I've also attempted to use v1.9.70 which results in the following:
Exception in thread "Thread-0" com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager
at com.google.apphosting.api.ApiProxy$CallNotFoundException.foreignThread(ApiProxy.java:800)
at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:112)
at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:40)
at com.google.api.client.extensions.appengine.http.UrlFetchRequest.execute(UrlFetchRequest.java:74)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.api.client.googleapis.auth.oauth2.GooglePublicKeysManager.refresh(GooglePublicKeysManager.java:172)
at com.google.api.client.googleapis.auth.oauth2.GooglePublicKeysManager.getPublicKeys(GooglePublicKeysManager.java:140)
at com.google.api.client.googleapis.auth.oauth2.GoogleIdTokenVerifier.verify(GoogleIdTokenVerifier.java:174)
at com.google.api.client.googleapis.auth.oauth2.GoogleIdTokenVerifier.verify(GoogleIdTokenVerifier.java:192)
at com.omarhegazi.login.GoogleAuthenticator.authenticateToken(GoogleAuthenticator.java:40)
at com.omarhegazi.login.ServerThread.run(ServerThread.java:45)
It looks like the fetch package gets included later than 1.6.1, but there's some issues relating to the threads in which the API calls are made.
Try to include the appengine dependency in your pom.xml:
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-api-1.0-sdk</artifactId>
<version>1.6.1</version> <!-- Check your version -->
</dependency>
Link to the docs
As per ngueno's post above, I needed appengine-api-1.0-sdk v1.9.70 to get this working:
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-api-1.0-sdk</artifactId>
<version>1.9.70</version>
</dependency>
To resolve the "Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager" error that was being thrown with this version of the appengine API, I had to change the HttpTransport used from UrlFetchTransport.getDefaultInstance() to GoogleNetHttpTransport.newTrustedTransport()
i.e.:
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
prop = new Properties();
prop.load(GoogleAuthenticator.class.getClassLoader().getResourceAsStream("config.properties"));
GoogleIdTokenVerifier verifier = new GoogleIdTokenVerifier.Builder(httpTransport, gsonFactory)
// Specify the CLIENT_ID of the app that accesses the backend:
.setAudience(Collections.singletonList(prop.getProperty("google.client.id")))
.setIssuer("https://accounts.google.com")
// Or, if multiple clients access the backend:
//.setAudience(Arrays.asList(CLIENT_ID_1, CLIENT_ID_2, CLIENT_ID_3, etc))
.build();
// (Receive idTokenString by HTTPS POST)
GoogleIdToken idToken;
idToken = verifier.verify(inputToken);
All references and guides used had pointed to the UrlFetchTransport.getDefaultInstance() being Thread safe/the best option to use but it did not work for me here.
Related
I have some code that unit tests fine using https.proxyHost and https.proxyPort to access Google Cloud Storage Buckets on on a corporate network which needs to go via a proxy:
log.info("resolving service....");
Storage storage = StorageOptions.newBuilder()
.setCredentials(
ServiceAccountCredentials.fromStream(
new FileInputStream(fullPath)))
.build()
.getService();
log.info("resolving bucket....");
bucket = storage.get(bucketName);
Yet when I run it in a larger app that starts a lot of other internal services (e.g., RMI) the proxy settings stop working.
Running as:
java -Dhttps.proxyHost=googleapis-dev.gcp.cloud.corporate -Dhttps.proxyPort=3128 ...
When it tries to resolve a bucket with the unit tested code it hangs for ages then throws:
com.google.cloud.storage.StorageException: Error getting access token for service account: oauth2.googleapis.com
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:231) ~[htu-gcs-plugin.jar:?]
...
Caused by: java.io.IOException: Error getting access token for service account: oauth2.googleapis.com
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:444) ~[?:?]
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:157) ~[?:?]
...
Caused by: java.net.UnknownHostException: oauth2.googleapis.com
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) ~[?:1.8.0_231]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[?:1.8.0_231]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_231]
at java.net.Socket.connect(Socket.java:606) ~[?:1.8.0_231]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666) ~[?:1.8.0_231]
I can get exactly the same error without proxy settings by adding this to the top of the main method:
String hostname = "https://oauth2.googleapis.com";
HttpURLConnection con = (HttpURLConnection) new URL(hostname).openConnection();
int code = con.getResponseCode();
System.out.println("> https://oauth2.googleapis.com returned: "+code);
Yet if the proxy settings are pass that runs fine then later throws the java.net.UnknownHostException as through the proxy settings are cleared.
To make things a bit more complex a custom URLClassLoader is used to load the code in question. Yet I have made a standalone app that uses the classloader and runs the code fine with the proxy settings passed as normal.
So it appears that something in the larger app is messing with the proxy system settings. Searching the codebase I can see no trace of that.
I have looked at https://googleapis.github.io/google-http-java-client/http-transport.html to see if there is way to plugin in a transport that has a proxy but cannot find a clear example.
Is there a way to coerce the use of a proxy when using google-cloud-storage?
To explicitly force a proxy to not rely upon the standard java System properties add the client libraries:
<!-- https://mvnrepository.com/artifact/com.google.http-client/google-http-client-apache-v2 -->
<dependency>
<groupId>com.google.http-client</groupId>
<artifactId>google-http-client-apache-v2</artifactId>
<version>1.37.0</version>
</dependency>
Then you can create a custom HttpTransportFactory with something like:
public class ProxyAwareTransportFactory implements HttpTransportFactory {
public static SSLContext trustAllSSLContext() throws Exception {
SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
sslContext.init(null, null, null);
return sslContext;
}
#Override
public HttpTransport create() {
InetSocketAddress socketAddress = new InetSocketAddress(this.host,this.port);
Proxy proxy = new Proxy(Proxy.Type.HTTP, socketAddress);
try {
return new NetHttpTransport.Builder()
.setProxy(proxy)
.setConnectionFactory(new DefaultConnectionFactory(proxy) )
.setSslSocketFactory(trustAllSSLContext().getSocketFactory())
.setHostnameVerifier(new DefaultHostnameVerifier())
.build();
} catch (Exception e) {
final String msg = "Could not build HttpTransport due to; " + e.getMessage();
log.error(msg, e);
throw new RuntimeException(e);
}
}
}
You can then use it using something like:
StorageOptions.Builder builder = StorageOptions.newBuilder();
if( useProxy ) {
HttpTransportFactory httpTransportFactory = new ProxyAwareTransportFactory(proxyHost, proxyPort);
HttpTransportOptions options = HttpTransportOptions.newBuilder().setHttpTransportFactory(httpTransportFactory).build();
builder.setTransportOptions(options);
}
Storage storage = builder
.setCredentials(
ServiceAccountCredentials.fromStream(
new FileInputStream(fullPath)))
.build()
.getService();
I am triggering a Lambda function from an SQS event with the following code:
#Override
public Void handleRequest(SQSEvent sqsEvent, Context context) {
for (SQSMessage sqsMessage : sqsEvent.getRecords()) {
final String body = sqsMessage.getBody();
try {
//do stuff here
} catch (Exception ex) {
//send to DLQ
}
}
return null;
}
The "do stuff" is calling another Lambda function with the following code:
private final AWSLambda client;
private final String functionName;
public LambdaService(AWSLambdaAsync client, String functionName) {
this.client = client;
this.functionName = functionName;
}
public void runWithPayload(String payload) {
logger.info("Invoking lambda {} with payload {}", functionName, payload);
final InvokeRequest request = new InvokeRequest();
request.withFunctionName(functionName).withPayload(payload);
final InvokeResult invokeResult = client.invoke(request);
final Integer statusCode = invokeResult.getStatusCode();
logger.info("Invoked lambda {} with payload {}. Got status code {} and response payload {}",
functionName,
payload,
statusCode,
StandardCharsets.UTF_8.decode(invokeResult.getPayload()).toString());
if(statusCode.equals(200) == false) {
throw new IllegalStateException(String.format("There was an error executing the lambda function %s with payload %s", functionName, payload));
}
}
I am using the following libraries:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-core</artifactId>
<version>1.2.0</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-events</artifactId>
<version>2.2.6</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-sqs</artifactId>
<version>1.11.505</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-lambda</artifactId>
<version>1.11.505</version>
</dependency>
The problem is that it looks like the SQS message is not removed from the queue and it gets reprocessed over and over. It happens every 30 seconds which is exactly the value of Default Visibility Timeout. Now, as far as I know, if the lambda consuming the sqs messages is terminating properly it should automatically delete the message from the queue, but this is not happening.
I don't think there is any error happening in the lambda because I am not getting any message in the DLQ (and I have a catch-all block) and I cannot see any stacktrace in the logs in Cloudwatch. I am bit confused about what's happening here, anyone has some good idea?
Unless something changed recently, I don't think the AWS SDK for Java automatically deletes the message from the queue. You need to write the code to do that.
I would love to be proven wrong on that one, please share the doc excerpt I missed.
Code sample :
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-sqs-messages.html
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/standard-queues-getting-started-java.html
I have a root CA certificate along with its private key (CAcert.pem and CApvtkey.key).
The root CA certificate has been registered on the AWS IoT core. This will be used to self-sign and validate other certificates for authentication.
I am trying to create certificates signed by my root CA certificate using Java, but not much luck.
The AWS IoT Java SDK provides functions to generate certificates, and to register/activate them on AWS, but I can't figure out how to sign them with my root CA certificate and activate them.
What I have is this only :
//Previous code sets up thing name etc...
CreateThingResult resp1 = client.createThing(thingRequest);
CreateKeysAndCertificateRequest req = new CreateKeysAndCertificateRequest();
req.setSetAsActive(true);
CreateKeysAndCertificateResult resp2 = client.createKeysAndCertificate(req);
client.attachThingPrincipal(new AttachThingPrincipalRequest().
withPrincipal(resp2.getCertificateArn()).withThingName("Java-App_Thing"));
Would anybody have an idea how I can create my certificates which will be signed by my CA certificate?
So the documentation is pretty vague on AWS. I had the same issue. Here is how I fixed it. Assuming you registered your CA with AWS IoT,
even if you enable auto registration for the CA you uploaded, AWS IoT will not allow devices to connect. The Certs will have a couple of options for JiT(Just in Time) registration.
Create Lamba to activate devices with certain conditions;
List to Event on MQTT to activate the cert;
Register the Public key.
Options 1 and 2 are described in AWS Docs
https://docs.aws.amazon.com/iot/latest/developerguide/auto-register-device-cert.html
Steps to do option 3:
Register the Thing
software.amazon.awssdk.services.iot.IotClient iotClient = IotClient.create()
//This allows AWS Credentials to be picked up using DefaultAWSCredentialsProviderChain
CreateThingRequest thingToBeCreated =
CreateThingRequest.builder().thingName("Unique Id of Device").build();
iotClient.createThing(thingToBeCreated);
Register and activate the Public Key of the device.
RegisterCertificateRequest registerCertificateRequest = RegisterCertificateRequest.builder()
.caCertificatePem("CA Pem as String")
.certificatePem("Device Public Key in Pem as String")
.setAsActive(true)
.build();
final RegisterCertificateResponse registerCertificateResponse = iotClient.registerCertificate(registerCertificateRequest);
Attach the Cert to the thing.
AttachThingPrincipalRequest attachThingPrincipalRequest = AttachThingPrincipalRequest.builder()
.thingName("Unique Id of Device")
.principal(registerCertificateResponse.certificateArn())
.build();
iotClient.attachThingPrincipal(attachThingPrincipalRequest);
Optional, attach policies to the thing so it can connect.
AttachPolicyRequest attachPolicyRequest = AttachPolicyRequest.builder()
.policyName("policy_that_allow_device_connections")
.target(registerCertificateResponse.certificateArn())
.build();
iotClient.attachPolicy(attachPolicyRequest);
Thanks!!!
above steps are working for me and code requires below dependency,
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>iot</artifactId>
<version>2.17.121</version>
</dependency>
Also here I used another library for the same process.
Steps are,
Register the RootCA with Verification certificate in AWS IoT core
Create the thing and attach device certificate and policy to connect programatically
So now in detail:
Register the RootCA with Verification certificate in AWS IoT core
Follow the link
https://docs.aws.amazon.com/iot/latest/developerguide/register-CA-cert.html
Create the thing and attach device certificate and policy to connect programatically
Below steps needed to do the process programatically,
Dependencies needed:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-iot-device-sdk-java</artifactId>
<version>1.3.9</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>1.12.150</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-iot</artifactId>
<version>1.12.150</version>
</dependency>
Aws config class:
public class AwsConfig {
#Bean
public AWSIot getIotClient() {
return AWSIotClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("users_aws_access_key", "users_aws_secret_key")))
.withRegion("users_aws_region").build();
}
}
Service class:
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import com.amazonaws.services.iot.model.AttachPolicyRequest;
import com.amazonaws.services.iot.model.AttachPolicyResult;
import com.amazonaws.services.iot.model.AttachThingPrincipalRequest;
import com.amazonaws.services.iot.model.AttachThingPrincipalResult;
import com.amazonaws.services.iot.model.CertificateStatus;
import com.amazonaws.services.iot.model.CreateThingRequest;
import com.amazonaws.services.iot.model.CreateThingResult;
import com.amazonaws.services.iot.model.DescribeThingRequest;
import com.amazonaws.services.iot.model.DescribeThingResult;
import com.amazonaws.services.iot.model.RegisterCertificateRequest;
import com.amazonaws.services.iot.model.RegisterCertificateResult;
import com.amazonaws.services.iot.model.ResourceNotFoundException;
#Service
public class RegisterService {
#Autowired
private AwsConfig iotClient;
public String RegisterDevice() {
// check if thing Already exists
if (!describeThing("Unique Id of Device")) {
// Thing Creation
CreateThingResult response = iotClient.getIotClient()
.createThing(new CreateThingRequest().withThingName("Unique Id of Device/Thing"));
// Register and activate the Public Key of the device
RegisterCertificateResult registerCert = iotClient.getIotClient()
.registerCertificate(new RegisterCertificateRequest().withCaCertificatePem("CA Pem as String")
.withCertificatePem("Device Public Key in Pem as String").withStatus(CertificateStatus.ACTIVE));
// Attach the Cert to the thing
AttachThingPrincipalResult attachThingPrincipalResult = iotClient.getIotClient().attachThingPrincipal(
new AttachThingPrincipalRequest()
.withThingName("Unique Id of Device/Thing").withPrincipal(registerCert.getCertificateArn()));
// Attach policies to the thing so it can connect
AttachPolicyResult policyResult = iotClient.getIotClient()
.attachPolicy(new AttachPolicyRequest()
.withPolicyName("policy_that_allow_device_connections").withTarget(registerCert.getCertificateArn()));
return "Thing Created Successfully";
}
// Thing exists
return "Thing Already Exists on IoT Console";
}
private boolean describeThing(String thingName) {
if (thingName == null) {
return false;
}
try {
describeThingResponse(thingName);
return true;
} catch (ResourceNotFoundException e) {
// e.printStackTrace();
return false;
}
}
private DescribeThingResult describeThingResponse(String thingName) {
DescribeThingRequest describeThingRequest = new DescribeThingRequest();
describeThingRequest.setThingName(thingName);
return iotClient.getIotClient().describeThing(describeThingRequest);
}
}
I am currently working with Eclipse and the AWS Toolkit for Eclipse. My project already works and it is doing its job, which is to connect to an RDS instance and return JSON objects to API Gateway calls.
I just got a new requirement, we are to use the service SecretsManager to authomatically rotate RDS configuration such as Users, passwords and so on.
The problem is when I try to import classes such as GetSecretValueResponse, I get a The import com.amazonaws.services.secretsmanager cannot be resolved. When I explore the documentation and the SDK, there exists a GetSecretValueRequest but not a GetSecretValueResponse, so I am not being able to make sense on what should I do, nor I have found anything similar to an example I can study.
The following code is what I am trying to implement and is given by Amazon itself (in the Secrets Manager page there is a button you can click to see how it would go with Java, in this case), and it is presented without any modification yet because as I said I do not know how to import several classes:
// Use this code snippet in your app.
public static void getSecret() {
String secretName = "secretName";
String endpoint = "secretEndpoint";
String region = "region";
AwsClientBuilder.EndpointConfiguration config = new AwsClientBuilder.EndpointConfiguration(endpoint, region);
AWSSecretsManagerClientBuilder clientBuilder = AWSSecretsManagerClientBuilder.standard();
clientBuilder.setEndpointConfiguration(config);
AWSSecretsManager client = clientBuilder.build();
String secret;
ByteBuffer binarySecretData;
GetSecretValueRequest getSecretValueRequest = GetSecretValueRequest.builder()
.withSecretId(secretName)
.build();
GetSecretValueResponse getSecretValueResponse = null;
try {
getSecretValueResponse = client.getSecretValue(getSecretValueRequest);
} catch(ResourceNotFoundException e) {
System.out.println("The requested secret " + secretName + " was not found");
} catch (InvalidRequestException e) {
System.out.println("The request was invalid due to: " + e.getMessage());
} catch (InvalidParameterException e) {
System.out.println("The request had invalid params: " + e.getMessage());
}
if(getSecretValueResponse == null) {
return;
}
// Decrypted secret using the associated KMS CMK
// Depending on whether the secret was a string or binary, one of these fields will be populated
if(getSecretValueResponse.getSecretString() != null) {
secret = getSecretValueResponse.getSecretString();
}
else {
binarySecretData = getSecretValueResponse.getSecretBinary();
}
// Your code goes here.
}
I had the same problem, the code that is present on AWS page doesn't work out of the box. The class you are looking for is GetSecretValueResult
Here are the latest java docs
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/secretsmanager/model/GetSecretValueResult.html
Here is a piece that shall work:
public void printRdsSecret() throws IOException {
String secretName = "mySecretName";
System.out.println("Requesting secret...");
AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard().build();
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest().withSecretId(secretName);
GetSecretValueResult getSecretValueResult = client.getSecretValue(getSecretValueRequest);
System.out.println("secret retrieved ");
final String secretBinaryString = getSecretValueResult.getSecretString();
final ObjectMapper objectMapper = new ObjectMapper();
final HashMap<String, String> secretMap = objectMapper.readValue(secretBinaryString, HashMap.class);
String url = String.format("jdbc:postgresql://%s:%s/dbName", secretMap.get("host"), secretMap.get("port"));
System.out.println("Secret url = "+url);
System.out.println("Secret username = "+secretMap.get("username"));
System.out.println("Secret password = "+secretMap.get("password"));
}
This was tested with aws-java-sdk-secretsmanager of version 1.11.337
I think the main problem was in lack of dependencies to AWS SDK v2.
Adding a code snippet here that utilizes AWS SDK v2. Just in case anybody is looking for this.
package com.may.util;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient;
import software.amazon.awssdk.services.secretsmanager.model.DecryptionFailureException;
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueRequest;
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueResponse;
import software.amazon.awssdk.services.secretsmanager.model.InternalServiceErrorException;
import software.amazon.awssdk.services.secretsmanager.model.InvalidParameterException;
import software.amazon.awssdk.services.secretsmanager.model.InvalidRequestException;
import software.amazon.awssdk.services.secretsmanager.model.ResourceNotFoundException;
public class SecretsManagerUtil {
public static String obtainSecret() {
String secretName = "db_secret_name";
String region = "us-east-1";
SecretsManagerClient client = SecretsManagerClient.builder().region(Region.of(region)).build();
GetSecretValueResponse response = null;
try {
response = client.getSecretValue(GetSecretValueRequest.builder().secretId(secretName).build());
} catch (DecryptionFailureException e) {
// Secrets Manager can't decrypt the protected secret text using the provided KMS key.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InternalServiceErrorException e) {
// An error occurred on the server side.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InvalidParameterException e) {
// You provided an invalid value for a parameter.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InvalidRequestException e) {
// You provided a parameter value that is not valid for the current state of the resource.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (ResourceNotFoundException e) {
// We can't find the resource that you asked for.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
}
return response.secretString();
}
}
Deserialize and print secret:
public class SecretPrinter {
private static final Logger logger = LoggerFactory.getLogger(SecretPrinter.class);
public void printSecret() {
String json = SecretsManagerUtil.obtainSecret(); // secret in json format
RdsSecret secret;
try {
secret = new ObjectMapper().disable(FAIL_ON_UNKNOWN_PROPERTIES).readValue(json, RdsSecret.class);
} catch (IOException e) {
logger.error("Couldn't parse secret obtained from AWS Secrets Manager!");
throw new RuntimeException(e);
}
System.out.println("username: " + secret.getUsername());
System.out.println("password: " + secret.getPassword());
}
static class RdsSecret {
private String username;
private String password;
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public String getPassword() {
return password;
}
public void setPassword(String password) {
this.password = password;
}
}
}
Maven:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>2.6.3</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>secretsmanager</artifactId>
</dependency>
Gradle:
implementation platform('software.amazon.awssdk:bom:2.6.3')
implementation 'software.amazon.awssdk:secretsmanager'
aws-secretsmanager-jdbc can be used to access AWS RDS via AWS secrete manager.
https://github.com/aws/aws-secretsmanager-jdbc
Below is the content of my application.properties
spring.datasource.url=jdbc-secretsmanager:mysql://dev-xxxx-database.cluster-xxxxxxxxx.ap-southeast-1.rds.amazonaws.com:3306/dev_xxxxxx
spring.datasource.username=/secret/application
spring.datasource.driver-class-name=com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver
spring.jpa.database-platform = org.hibernate.dialect.MySQL5Dialect
Below is the secret in AWS secret manager.
By using this method, you don't need to obtain username and password manually and create data sources.
just add the lib to your pom:
https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-secretsmanager
I recommend using aws secret manger jdbc wrapper to access the RDS (https://github.com/aws/aws-secretsmanager-jdbc). You don't need to deal with fetching the secret, decode and move around with password in text before passing to RDS client.
Just pass in secret id to RDS client and the jdbc wrapper will handle the rest.
I have faced the same issue. just delete the scope line i.e "test" from the dependency. It'll work
You're missing "aws-java-sdk-secretsmanager" dependency, you have to add this to your pom.xml and then import on your java class.
From Maven:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-secretsmanager</artifactId>
<version>1.11.355 </version>
</dependency>
AWS Reference
If you are not using maven, you have add AWS SDK to your existing project.
I want to upload files using the java API for DropBox. The following code gets me the oauth_token and oauth_secret. but when ever I try to upload a file I get a exception.
Java Class
package com.dropbox.client;
import com.dropbox.client.DropboxAPI.Config;
import java.io.File;
import java.net.URL;
import java.util.HashMap;
import java.util.Map;
/**
*
* #author Charan
*/
public class DBmain {
public DBmain() {
System.setProperty("java.net.useSystemProxies", "true");
Map configuration= new HashMap();
configuration.put("consumer_key", "XXXXXXXXXXXXXXXX");
configuration.put("consumer_secret", "XXXXXXXXXXXXXXXX");
configuration.put("request_token_url", "http://api.dropbox.com/0/oauth/request_token");
configuration.put("access_token_url", "http://api.dropbox.com/0/oauth/access_token");
configuration.put("authorization_url", "http://api.dropbox.com/0/oauth/authorize");
configuration.put("port",80);
//configuration.put("trusted_access_token_url","http://api.getdropbox.com/0/token");
configuration.put("server","api.getdropbox.com");
configuration.put("content_server","api-content.dropbox.com");
String username="myMailAddress#domain.com";
String password="myPassword";
try {
Authenticator auth = new Authenticator(configuration);
String url = auth.retrieveRequestToken("");
String access_key = auth.getTokenKey();
String access_secret = auth.getTokenSecret();
System.out.println(access_key);
System.out.println(access_secret);
DropboxAPI api = new DropboxAPI();
DropboxAPI.Config conf = api.new Config(configuration);
api.authenticateToken("XXXXXXXXXXXX", "XXXXXXXXXXX", conf);
System.out.println(api.isAuthenticated());
URL resource = this.getClass().getResource("/config/testing.json");
File f= new File(resource.toURI());
api.putFile("dropbox", "/Project", f);
//api.accountInfo(); //even this method gives me a exception
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
new DBmain();
}
}
Exception
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.http.protocol.BasicHttpContext: method <init>()V not found
at org.apache.http.impl.client.DefaultHttpClient.createHttpContext(DefaultHttpClient.java:205)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:532)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:465)
at com.dropbox.client.DropboxClient.putFile(DropboxClient.java:299)
at com.dropbox.client.DropboxAPI.putFile(DropboxAPI.java:463)
at com.dropbox.client.DBmain.<init>(DBmain.java:58)
at com.dropbox.client.DBmain.main(DBmain.java:70)
Java Result: 1
Some extra info
Edit Date:18-6-2011
I changed the httpclient-4.0-beta1.jar and httpcore-4.0-alpha6.jar to httpclient-4.0.jar and httpcore-4.0.1.jar respectively and I no longer get the above exception [ java.lang.NoSuchMethodError ]
This stackoverflow question helped me in solving this : java.lang.NoSuchMethodError: org.apache.http.protocol.BasicHttpContext: method <init>()V not found
But Now I get UnknownhostException on execution of any methods of the API
com.dropbox.client.DropboxException: java.net.UnknownHostException: api.getdropbox.com:80
at com.dropbox.client.RESTUtility.request(RESTUtility.java:250)
at dump.DropboxClient.accountInfo(DropboxClient.java:121)
at com.charan.client.DBmain.<init>(DBmain.java:57)
at com.charan.client.DBmain.main(DBmain.java:65)
Caused by: java.net.UnknownHostException: api.getdropbox.com:80
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:436)
at java.net.Socket.connect(Socket.java:525)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:123)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:147)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:101)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:381)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)
at com.dropbox.client.RESTUtility.executeRequest(RESTUtility.java:172)
at com.dropbox.client.RESTUtility.request(RESTUtility.java:248)
... 3 more
You should use http-client 4.0.3 jar
Your class should NOT be in package com.dropbox.client. Try moving it to a different (i.e. com.yourname.client). It looks like you might be creating a name clash and inadvertently overriding something in the Dropbox client API.
You have specified api.getdropbox.com:80 as the host name. Try using api.getdropbox.com instead.