I have following code snippet, that is supposed to run in a AWS Lambda function:
AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard().withRegion(AWS_REGION).build();
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest().withSecretId(SECRET_NAME);
GetSecretValueResult secretValue = client.getSecretValue(getSecretValueRequest);
As the lambda function is going to be run in the same VPC as the secret manager I don't have to provide credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) for it.
I use Localstack with Testcontainers for integration tests and set up the secret in the test setup like this:
AWSSecretsManager secretsManager = AWSSecretsManagerClientBuilder.standard()
.withEndpointConfiguration(secretsmanager.getEndpointConfiguration(SECRETSMANAGER))
.withCredentials(secretsmanager.getDefaultCredentialsProvider())
.build();
String secretString = "{'engine':'mysql','port':" + mysql.getMappedPort(3306) + ",'host':'" + mysql.getContainerIpAddress() + "'}";
CreateSecretRequest request = new CreateSecretRequest().withName("aurora")
.withSecretString(secretString)
.withRequestCredentialsProvider(secretsmanager.getDefaultCredentialsProvider());
secretsManager.createSecret(request);
Now the test crashes with an error:
com.amazonaws.services.secretsmanager.model.AWSSecretsManagerException:
The security token included in the request is invalid.
(Service: AWSSecretsManager;
Status Code: 400; Error Code:
UnrecognizedClientException;
Request ID: ...
Here is also the definition of the localstack container used in the test:
#ClassRule
public static LocalStackContainer secretsmanager = new LocalStackContainer("0.10.4")
.withServices(LocalStackContainer.Service.SECRETSMANAGER)
.withEnv("DEFAULT_REGION", "eu-west-1")
.withExposedPorts(4584);
How could I configure the LocalStackContainer to accept requests without any credentials validation going on?
Assuming you are a reasonable java developer which prefers spring boot test and junit5 over alternatives, #DynamicPropertySource can be quite handy here
private static final LocalStackContainer LOCALSTACK = ...;
#DynamicPropertySource
static void setCredentials(DynamicPropertyRegistry registry) {
var credentials = LOCALSTACK.getDefaultCredentialsProvider().getCredentials();
registry.add("cloud.aws.region.static", () -> "eu-central-1");
registry.add("cloud.aws.credentials.access-key", () -> credentials.getAWSAccessKeyId());
registry.add("cloud.aws.credentials.secret-key", () -> credentials.getAWSSecretKey());
registry.add("cloud.aws.s3.endpoint", () -> LOCALSTACK.getEndpointOverride(S3));
}
Also please doublecheck you've overridden the endpoints you rely on (s3 in my example), otherwise you may request real AWS API instead of the containerized one
Related
I have a Aws lambda that try to access to S3 bucket to get head object and then their metadata.
I am using Serverless to test locally.
But, when I execute locally the test, i get:
software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Read timed
If I implemented the lambda with node js, then it work fine. But I need to use Java.
Technologies:
Java8
software.amazon.awssdk:bom:2.17.103
software.amazon.awssdk:lambda
software.amazon.awssdk:s3
"serverless": "^2.71.0"
"serverless-offline": "^8.4.0"
"serverless-s3-local": "^0.6.21"
Environment:
Local
I don't have VPC
Code:
public class FileS3Handler implements RequestHandler<Map<String,String>, String> {
#Override
public String handleRequest(Map<String,String> event, Context context)
{
String response;
URI uri = new URI("http://localhost:3333");
S3Configuration config = S3Configuration.builder()
.pathStyleAccessEnabled(true)
.build();
S3Client s3Client = S3Client.builder()
.endpointOverride(uri)
.serviceConfiguration(config)
.build();
HeadObjectRequest headObjectRequest = HeadObjectRequest.builder()
.bucket("myBucket")
.key("myFile")
.build();
s3Client.
HeadObjectResponse headObjectResponse = s3Client.headObject(headObjectRequest);
response = "Metadata: "+ headObjectResponse.metadata().toString();
return response;
}
}
Environment Variables:
AWS_ACCESS_KEY_ID=S3RVER
AWS_SECRET_ACCESS_KEY=S3RVER
AWS_REGION=us-west-2
serverless.yml
service: example
provider:
name: aws
region: us-west-2
runtime: java8
lambdaHashingVersion: 20201221
package:
individually: true
functions:
file-handler:
handler: com.example.FileS3Handler
events:
- http:
path: /api/filetest
method: get
package:
artifact: build/example.zip
plugins:
- serverless-offline
- serverless-dotenv-plugin
- serverless-s3-local
custom:
serverless-offline:
httpPort: 4000
s3:
port: 3333
host: localhost
resources:
Resources:
Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: myBucket
Test:
Run: serverless start offline
Postman -> GET http://localhost:4000/dev/api/filetest
Does anyone know what is happening?
Thanks!
What we have on start
Let's say there is a simple Spring Boot application which provides an API for some frontend. Tech stack is quite regular: Kotlin, Gradle, Spring WebMVC, PostgreSQL, Keycloak.
The frontend interacts with the app synchronously via HTTP. Client authenticates with JWT token.
The business task
There is a list of events that could be raised somewhere in the system. User should be notified about them.
User is able to subscribe to one or more event notification. Subscriptions is just a pair of user_id + event_type_id persisted in dedicated Postgres table.
When event X is being raised we should find all the users subscribed to it and send them some data via Websocket
Configuration
Let's configure Spring first. Spring uses a STOMP over Websocket protocol.
Add dependencies to build.gradle.kts
implementation("org.springframework.boot:spring-boot-starter-websocket")
implementation("org.springframework:spring-messaging")
Add a Websocket config
#Configuration
#EnableWebSocket
#EnableWebSocketMessageBroker
class WebsocketConfig(
private val websocketAuthInterceptor: WebsocketAuthInterceptor
) : WebSocketMessageBrokerConfigurer {
override fun configureMessageBroker(config: MessageBrokerRegistry) {
config.enableSimpleBroker("/queue/")
config.setUserDestinationPrefix("/user")
}
override fun registerStompEndpoints(registry: StompEndpointRegistry) {
registry.addEndpoint("/ws").setAllowedOrigins("*")
registry.addEndpoint("/ws").setAllowedOrigins("*").withSockJS()
}
override fun configureClientInboundChannel(registration: ChannelRegistration) {
registration.interceptors(websocketAuthInterceptor) // we'll talk later about this
}
}
registerStompEndpoints is about how to establish connection with our websocket. The config allows frontend interact with us via SockJS of Websocket libraries. What they are and what the difference is not a today's topic
configureMessageBroker is about how we will interact with the frontend after the connection established.
configureClientInboundChannel is about message interception. Let talk about it later.
Add /ws/** path to ignored patterns in the Spring Security config.
Connection establishing
On the frontend side it should looks something like that (JavaScript)
const socket = new WebSocket('ws://<api.host>/ws')
//const socket = new SockJS('http://<api.host>/ws') // alternative way
const headers = {Authorization: 'JWT token here'};
const stompClient = Stomp.over(socket, {headers});
stompClient.connect(
headers,
function (frame) {
stompClient.subscribe('/user/queue/notification',
function (message) { ...message processing... });
});
The key moment is to pass an authorization header. It is not a HTTP header. And initial HTTP handshake itself will not be authorized. This is the reason of adding /ws/** to ignored patterns.
We need the header and the token because we want to allow websocket connection only for authorized users and also we want to know which exactly user is connected.
Authentication
Now we are adding the authentication mechanism
#Component
class WebsocketAuthInterceptor(
private val authService: AuthService, //your implementation
private val sessionStore: WebsocketUserSessionStore
) : ChannelInterceptor {
override fun preSend(message: Message<*>, channel: MessageChannel): Message<*>? {
val accessor: StompHeaderAccessor = MessageHeaderAccessor.getAccessor(message, StompHeaderAccessor::class.java)
val sessionId: String = accessor.messageHeaders["simpSessionId"].toString()
if (StompCommand.CONNECT == accessor.command) {
val jwtToken: String = accessor.getFirstNativeHeader("Authorization")
val token: AccessToken = authService.verifyToken(jwtToken)
val userId: Long = token.otherClaims["user_id"].toString().toLong()
sessionStore.add(sessionId, userId)
} else if (StompCommand.DISCONNECT == accessor.command) {
sessionStore.remove(sessionId)
}
return message
}
}
The point is to link a random generating websocket session ID with a user ID from our Spring Security store and persist that pair during the session life. JWT token should be parsed from the message headers.
Then by the given token a user ID should be obtained. Implementation of that part depends on what Spring Security config you exactly have. In case of Keycloack there is a useful static method org.keycloak.adapters.rotation.AdapterTokenVerifier::verifyToken
WebsocketUserSessionStore is just a map for linking session_id with user_id It may looks like the following code. Remember the concurrent access of course
#Component
class WebsocketUserSessionStore {
private val lock = ReentrantLock()
private val store = HashMap<String, Long>()
fun add(sessionId: String, userId: Long) = lock.withLock {
store.compute(sessionId) { _, _ -> userId }
}
fun remove(sessionId: String) = lock.withLock {
store.remove(sessionId)
}
fun remove(userId: Long) = lock.withLock {
store.values.remove(userId)
}
}
Notification actually
So event A was raised somewhere inside business logic. Let's implement a websocket publisher.
#Component
class WebsocketPublisher(
private val messagingTemplate: SimpMessagingTemplate,
private val objectMapper: ObjectMapper,
private val sessionStore: WebsocketUserSessionStore,
private val userNotificationRepository: UserNotificationRepository
) {
suspend fun publish(eventType: EventType, eventData: Any) {
val userIds = userNotificationRepository.getUserSubscribedTo(eventType)
val sessionIds = sessionStore.getSessionIds(userIds)
sessionIds.forEach {
messagingTemplate.convertAndSendToUser(
it,
"/queue/notification",
objectMapper.writeValueAsString(eventData),
it.toMessageHeaders()
)
}
}
private fun String.toMessageHeaders(): MessageHeaders {
val headerAccessor = SimpMessageHeaderAccessor.create(SimpMessageType.MESSAGE)
headerAccessor.sessionId = this
headerAccessor.setLeaveMutable(true)
return headerAccessor.messageHeaders
}
}
EventType is an enumeration of event types the system has.
UserNotificationRepository is just a part of data persistence layer (Hibernate|JOOQ repository, MyBatis mapper or smth). Function getUserSubscribedTo should do something like select user_id from user_subscription where event_type_id = X.
The rest of code is plenty straightforward. By the giving userIds it is possible to obtain living websocket sessions. Then for every session convertAndSendToUser function should be called.
I think they have the tutorial to build the WebSocket push-notification service.
I'm try reproduce the follow code Deploying a Camel Route in AWS Lambda : A Camel Quarkus example in my own code Quarkus Camel AWS Lambda, but the ProducerTemplate returns NullPointerExcetion, as can see in this link BUG_CAMEL_QUARKUS_LAMBDA
#Named("languageScoreLambda")
public class LanguageScoreLambda implements RequestHandler<Language, LanguageScoreDto> {
#Inject
ProducerTemplate template;
#Override
public LanguageScoreDto handleRequest(Language input, Context context) {
System.out.println("#Template isNull ===> " + (null == template)); // true
return new LanguageScoreDto("5", input.getLanguage());
}
}
I Found the issue, as I've been using Terraform to provide the AWS Lambda function the handler MUST BE io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest
bellow the raw code snippted
resource "aws_lambda_function" "hello_lambda" {
function_name = var.AWS_LAMBDA_FUNCTION_NAME
filename = "${path.module}/function.zip"
role = aws_iam_role.hello_lambda_role.arn
depends_on = [aws_cloudwatch_log_group.hello_lambda_logging]
runtime = "java11"
handler = io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest
timeout = 10
memory_size = 256
}
Iam trying to authenticate a Java app with Cognito.
I have used for python the warrant library that worked very good. But i want to do the same in java now.
My Python function i used for authentication with the warrant library
def SRPauthentication(organizationAdmin,
password,
pool_id,
client_id,
client):
aws = AWSSRP(username=organizationAdmin,
password=password,
pool_id=pool_id,
client_id=client_id,
client=client)
tokens = aws.authenticate_user()
authorization_token= tokens['AuthenticationResult']['IdToken']
return authorization_token
with this i could easily acces some secured APIs.
Now i want to do the same with Java but i have problems.
This is my solution so far is this method:
public static void GetCreds()
{
AWSCognitoIdentityProvider identityProvider = AWSCognitoIdentityProviderClientBuilder.defaultClient();
AdminInitiateAuthRequest adminInitiateAuthRequest = new AdminInitiateAuthRequest().
withAuthFlow(AuthFlowType.USER_SRP_AUTH).
withClientId("234234234234").withUserPoolId("eu-central-1_sdfsdfdsf")
.addAuthParametersEntry("USERNAME", "UserK").
addAuthParametersEntry("PASSWORD","#######);
adminInitiateAuthRequest.getAuthFlow();
AdminInitiateAuthResult adminInitiateAuth = identityProvider.adminInitiateAuth(adminInitiateAuthRequest);
System.out.println(adminInitiateAuth.getAuthenticationResult().getIdToken());
}
When i run this i get an Exception:
Exception in thread "main" `com.amazonaws.services.cognitoidp.model.AWSCognitoIdentityProviderException: User: arn:aws:iam::XXXXXXXXXXXXXXXXX:user/khan is not authorized to perform: cognito-idp:AdminInitiateAuth on resource: arn:aws:cognito-idp:eu-central-1:XXXXXXXX:userpool/eu-central-1_XXXXXXX with an explicit deny (Service: AWSCognitoIdentityProvider; Status Code: 400; Error Code: AccessDeniedException; Request ID: 21be0b8e-adec-11e8-ad45-234234234)`
It says iam not authorized to perform this kind of instruction. So i guess iam doing something generally wrong. Because its working with my python code and in Java it recognizes my username from the credentials. The Cognito call should actually be independent from my aws credentials/useraccount right?
How to authenticate with Cognito using Java to get an Token to access secured aws services?
EDIT:
AWSCognitoIdentityProvider identityProvider = AWSCognitoIdentityProviderClientBuilder.standard()
.build();
InitiateAuthRequest adminInitiateAuthRequest = new InitiateAuthRequest()
.withAuthFlow(AuthFlowType.USER_SRP_AUTH)
.withClientId("XXXXXXXXXXXXXXXXX")
.addAuthParametersEntry("USERNAME", "user").
addAuthParametersEntry("PASSWORD","za$Lwn")
.addAuthParametersEntry("SRP_A",new AuthenticationHelper("eu-central-1XXXXXXXXX").getA().toString(16));
adminInitiateAuthRequest.getAuthFlow();
InitiateAuthResult adminInitiateAuth = identityProvider.initiateAuth(adminInitiateAuthRequest);
System.out.println(adminInitiateAuth);
I changed the AdminInitateAuthRequest to InitateAuthRequest. After that i had the Error missing SRP_A parameter that i somehow fixed with a similiar question here
And now i recive this :
{ChallengeName: PASSWORD_VERIFIER,ChallengeParameters: {SALT=877734234324234ed68300f39bc5b, SECRET_BLOCK=lrkwejrlewrjlewkjrewlrkjwerlewkjrewlrkjewrlkewjrlewkrjZ+Q==, USER_ID_FOR_SRP=user, USERNAME=user, SRP_B=43ecc1lwkerjwelrkjewlrjewrlkewjrpoipweoriwe9r873jr34h9r834hr3455f7d079d71e5012f1623ed54dd10b832792dafa3438cca3f59c0f462cbaee255d5b7c2werwerwerkjweorkjwerwerewrf5020e4f8b5452f3b89caef4a797456743602b80b5259261f90e52374adc06b456521a9026cce9c1cbe8b9ffd6040e8c1589d35546861422110ac7e38c1c93389b802a03e3e2e4a50e75d088275195f836f66e25f1a431dd56bb2},}
I have shorten the result with all the keys, but what to do next ?
Finally i could solve it with this code class.
There are multiple challenges involved in SRP authentication. The InitiateAuthRequest is one first request that is necessary.
This similiar question helped me :
stackoverflow
stackoverfow
String PerformSRPAuthentication(String username, String password) {
String authresult = null;
InitiateAuthRequest initiateAuthRequest = initiateUserSrpAuthRequest(username);
try {
AnonymousAWSCredentials awsCreds = new AnonymousAWSCredentials();
AWSCognitoIdentityProvider cognitoIdentityProvider = AWSCognitoIdentityProviderClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCreds))
.withRegion(Regions.fromName(this.region))
.build();
InitiateAuthResult initiateAuthResult = cognitoIdentityProvider.initiateAuth(initiateAuthRequest);
if (ChallengeNameType.PASSWORD_VERIFIER.toString().equals(initiateAuthResult.getChallengeName())) {
RespondToAuthChallengeRequest challengeRequest = userSrpAuthRequest(initiateAuthResult, password);
RespondToAuthChallengeResult result = cognitoIdentityProvider.respondToAuthChallenge(challengeRequest);
//System.out.println(result);
System.out.println(CognitoJWTParser.getPayload(result.getAuthenticationResult().getIdToken()));
authresult = result.getAuthenticationResult().getIdToken();
}
} catch (final Exception ex) {
System.out.println("Exception" + ex);
}
return authresult;
}
private InitiateAuthRequest initiateUserSrpAuthRequest(String username) {
InitiateAuthRequest initiateAuthRequest = new InitiateAuthRequest();
initiateAuthRequest.setAuthFlow(AuthFlowType.USER_SRP_AUTH);
initiateAuthRequest.setClientId(this.clientId);
//Only to be used if the pool contains the secret key.
//initiateAuthRequest.addAuthParametersEntry("SECRET_HASH", this.calculateSecretHash(this.clientId,this.secretKey,username));
initiateAuthRequest.addAuthParametersEntry("USERNAME", username);
initiateAuthRequest.addAuthParametersEntry("SRP_A", this.getA().toString(16));
return initiateAuthRequest;
}
I'm using amazon SDK 1.5.6 and trying to initialize the connection using a proxy server.
The initialization succeed, but when I try to use the AmazonEC2Client it fails with the following
error:
Caught Exception: Status Code: 407, AWS Service: AmazonEC2, AWS
Request ID: null, AWS Error Code: 407 Unauthorized, AWS Error Message:
Unable to unmarshall error response (Premature end of file.) Reponse
Status Code: 407 Error Code: 407 Unauthorized Request ID: null
Initialization code:
protected AmazonEC2 initAmazonSDKClient(String endpoint) {
AWSCredentials awsCredentials =
new BasicAWSCredentials(_account.getAccessKey(), _account.getSecretKey());
ClientConfiguration config = getProxySettings();
AmazonEC2 ret = CloudServicesEC2Api.getAmazonEC2Client(awsCredentials, config);
ret.setEndpoint(endpoint);
_endPointToAmazonEC2Client.put(endpoint, ret);
return ret;
}
private ClientConfiguration getProxySettings() {
ClientConfiguration ret = new ClientConfiguration();
String host = _proxySettings.getServer();
if (!StringUtils.isNullOrEmpty(host )) {
ret.setProxyHost(host );
}
if (!StringUtils.isNullOrEmpty(_proxySettings.getPort())) {
ret.setProxyPort(Integer.valueOf(_proxySettings.getPort()));
}
String proxyUserName = _proxySettings.getUserName();
if (!StringUtils.isNullOrEmpty(proxyUserName)) {
ret.setProxyUsername(proxyUserName);
}
String proxyPassword = _proxySettings.getPassword();
if (!StringUtils.isNullOrEmpty(proxyPassword)) {
ret.setProxyPassword(proxyPassword);
}
ret.setProxyWorkstation(host );
ret.setProxyDomain(host );
return ret;
}
Code that uses the client and causes the error:
private List<InstanceStatus> getStatusChecks(AmazonEC2 ec2Client, String[] hostIds) {
DescribeInstanceStatusRequest describeInstanceRequest =
new DescribeInstanceStatusRequest().withInstanceIds(hostIds);
DescribeInstanceStatusResult describeInstanceResult =
ec2Client.describeInstanceStatus(describeInstanceRequest);
return describeInstanceResult.getInstanceStatuses();
}
I know the root cause is the NTLM authentication, I need to somehow configure the NTLM host & NTLM domain, in my code sample its the lines:
ret.setProxyWorkstation(host );
ret.setProxyDomain(host );
I have tried several versions of workstation & domain, i tried leaving it blank, no use...
Please advise!
Well, after some research it turns out that something is not right in the way Amazon SDK handles NTLM proxy requests. We've compared the proxy parameters passed to other libraries (e.g JClouds) where the connection succeeded and tried applying the same on Amazon SDK with no luck.
We have even compared the packets using wireshark, everything is the same. Something is just not right with this library...