I am trying to create a lambda funciton through serverless framewrok and want to expose a REST endpoint that will list all the s3 buckets in a region mentioned if the lambda gets triggered through api gateway call.
I successfully created a REST api with hard coded values as mentioned below -
SUCCESS SCENARIO
lambda code:
#RestController
public class LanguageResource {
#RequestMapping(path = "/languages", method = RequestMethod.GET)
public List<Language> listLambdaLanguages() {
return Arrays.asList(new Language("node"), new Language("java"), new Language("python"));
}
}
SAM.yml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Lambda with spring boot.
Resources:
LambdaSpringBootFunction:
Type: AWS::Serverless::Function
Properties:
Handler: com.example.test.lambda.LambdaHandler::handleRequest
Runtime: java8
CodeUri: target/spring-boot-lambda-1.0.0-SNAPSHOT.jar
MemorySize: 1512
Role: arn:aws:iam::XXXXXX:role/rol-lambda-servicenow
# Policies: AWSLambdaBasicExecutionRole
Timeout: 60
Events:
GetResource:
Type: Api
Properties:
Path: /{proxy+}
Method: any
Outputs:
LambdaSpringBootApi:
Description: URL for application
Value: !Sub 'https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Stage/languages'
Export:
Name: LambdaSpringBootApi
endpoint:
/languages
output:
Mon Aug 31 14:13:55 UTC 2020 : Endpoint request body after transformations: {"resource":"/{proxy+}","path":"/languages","httpMethod":"GET","headers":null,"multiValueHeaders":null,"queryStringParameters":null,"multiValueQueryStringParameters":null,"pathParameters":{"proxy":"languages"},"stageVariables":null,"requestContext":{"resourceId":"b04re4","resourcePath":"/{proxy+}","httpMethod":"GET","extendedRequestId":"SI3RfEFWDoEFltg=","requestTime":"31/Aug/2020:14:13:55 +0000","path":"/{proxy+}","accountId":"XXXXXX","protocol":"HTTP/1.1","stage":"test-invoke-stage","domainPrefix":"testPrefix","requestTimeEpoch":1598883235028,"requestId":"caa5dcc6-60ca-4a35-9a6b-a0f3c2a2555a","identity":{"cognitoIdentityPoolId":null,"cognitoIdentityId":null,"apiKey":"test-invoke-api-key","principalOrgId":null,"cognitoAuthenticationType":null,"userArn":"arn:aws:sts::XXXXXX:assumed-role/rol-mgt-dta-administrators/aniruddha.ghosh","apiKeyId":"test-invoke-api-key-id","userAgent":"aws-internal/3 aws-sdk-java/1.11.829 Linux/4.9.217-0.1.ac.205.84.332.metal1. [TRUNCATED]
Mon Aug 31 14:13:55 UTC 2020 : Sending request to https://lambda.eu-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-west-1:XXXXXX:function:ssc-mgt-dta-servicenow-la-LambdaSpringBootFunction-DI8EHLE9ME1U/invocations
Mon Aug 31 14:14:05 UTC 2020 : Received response. Status: 200, Integration latency: 10595 ms
Mon Aug 31 14:14:05 UTC 2020 : Endpoint response headers: {Date=Mon, 31 Aug 2020 14:14:05 GMT, Content-Type=application/json, Content-Length=188, Connection=keep-alive, x-amzn-RequestId=1338096f-3eec-4bf4-bd0f-9873250b4e8b, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-5f4d05a3-789c6e282a4f7f92e19a77ff;sampled=0}
Mon Aug 31 14:14:05 UTC 2020 : Endpoint response body before transformations: **{"statusCode":200,"multiValueHeaders":{"Content-Type":["application/json; charset=UTF-8"]},"body":"[{\"name\":\"node\"},{\"name\":\"java\"},{\"name\":\"python\"}]","isBase64Encoded":false}**
Mon Aug 31 14:14:05 UTC 2020 : Method response body after transformations: [{"name":"node"},{"name":"java"},{"name":"python"}]
Mon Aug 31 14:14:05 UTC 2020 : Method response headers: {Content-Type=application/json; charset=UTF-8, X-Amzn-Trace-Id=Root=1-5f4d05a3-789c6e282a4f7f92e19a77ff;Sampled=0}
Mon Aug 31 14:14:05 UTC 2020 : Successfully completed execution
Mon Aug 31 14:14:05 UTC 2020 : Method completed with status: 200
However, when i try to list s3 buckets what i understood that i need to use aws s3 SDK and while used it as mentioned below it is throwing me gateway timeout error with no specific details. I am very new into these things that is why need help
FAILURE SCENARIO
lambda code:
#RequestMapping(path = "/languages", method = RequestMethod.GET)
public ListBucketsResponse listLambdaLanguages() {
Region region = Region.EU_WEST_1;
S3Client s3 = S3Client.builder().region(region).build();
ListBucketsRequest listBucketsRequest = ListBucketsRequest.builder().build();
ListBucketsResponse listBucketsResponse = s3.listBuckets(listBucketsRequest);
listBucketsResponse.buckets().stream().forEach(x -> System.out.println(x.name()));
return listBucketsResponse;
}
SAM.yml
endpoint:
/languages
Output:
Mon Aug 31 14:17:40 UTC 2020 : Endpoint request body after transformations: {"resource":"/{proxy+}","path":"/languages","httpMethod":"GET","headers":null,"multiValueHeaders":null,"queryStringParameters":null,"multiValueQueryStringParameters":null,"pathParameters":{"proxy":"languages"},"stageVariables":null,"requestContext":{"resourceId":"2n2pvy","resourcePath":"/{proxy+}","httpMethod":"GET","extendedRequestId":"SI30xFwRDoEFRPA=","requestTime":"31/Aug/2020:14:17:40 +0000","path":"/{proxy+}","accountId":"XXXXXX","protocol":"HTTP/1.1","stage":"test-invoke-stage","domainPrefix":"testPrefix","requestTimeEpoch":1598883460839,"requestId":"df87b0f3-57de-45f8-9c11-d64000be3d61","identity":{"cognitoIdentityPoolId":null,"cognitoIdentityId":null,"apiKey":"test-invoke-api-key","principalOrgId":null,"cognitoAuthenticationType":null,"userArn":"arn:aws:sts::XXXXXX:assumed-role/rol-mgt-dta-administrators/aniruddha.ghosh","apiKeyId":"test-invoke-api-key-id","userAgent":"aws-internal/3 aws-sdk-java/1.11.829 Linux/4.9.217-0.1.ac.205.84.332.metal1. [TRUNCATED]
Mon Aug 31 14:17:40 UTC 2020 : Sending request to https://lambda.eu-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-west-1:XXXXXX:function:ssc-mgt-dta-servicenow-la-LambdaSpringBootFunction-U14LQ9SEJQ4I/invocations
Mon Aug 31 14:17:57 UTC 2020 : Received response. Status: 200, Integration latency: 16221 ms
Mon Aug 31 14:17:57 UTC 2020 : Endpoint response headers: {Date=Mon, 31 Aug 2020 14:17:57 GMT, Content-Type=application/json, Content-Length=143, Connection=keep-alive, x-amzn-RequestId=1c9e34f3-fa10-4db0-b825-5e40e3ac9e94, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-5f4d0684-a47151cdb1421b4494698ebf;sampled=0}
Mon Aug 31 14:17:57 UTC 2020 : Endpoint response body before transformations: {"statusCode":502,"multiValueHeaders":{"Content-Type":["application/json"]},"body":"{\"message\":\"Gateway timeout\"}","isBase64Encoded":false}
Mon Aug 31 14:17:57 UTC 2020 : Method response body after transformations: {"message":"Gateway timeout"}
Mon Aug 31 14:17:57 UTC 2020 : Method response headers: {Content-Type=application/json, X-Amzn-Trace-Id=Root=1-5f4d0684-a47151cdb1421b4494698ebf;Sampled=0}
Mon Aug 31 14:17:57 UTC 2020 : Successfully completed execution
Mon Aug 31 14:17:57 UTC 2020 : Method completed with status: 502
Note:
this role rol-lambda-servicenow has full access on s3
Your lambda does not have permission to do that so it's blocked. You can read more here https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-policy-templates.html
Also try to go to CloudWatch log groups in your AWS console. Logs should tell you clearly about that problem
This is a typical issue while Lambda sitting behind API gateway. I've googled almost all the posts but still can't not figure out why it fails.
I've read the instruction on what the response should be look like
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-output-format
It is looking for
{
"isBase64Encoded": true|false,
"statusCode": httpStatusCode,
"headers": { "headerName": "headerValue", ... },
"multiValueHeaders": { "headerName": ["headerValue", "headerValue2", ...], ... },
"body": "..."
}
Here is the log of my test from API gateway console (I have double checked make sure the name of fields are exactly the same)
Thu Mar 19 10:07:58 UTC 2020 : Endpoint response body before transformations: "{\"statusCode\":200,\"body\":\"RESULTS\",\"isBase64Encoded\":false}"
Thu Mar 19 10:07:58 UTC 2020 : Execution failed due to configuration error: Malformed Lambda proxy response
Thu Mar 19 10:07:58 UTC 2020 : Method completed with status: 502
Can anyone help?
EDIT
I've removed "body" from response and still get the same error. The following is the actual log
Thu Mar 19 23:29:54 UTC 2020 : Received response. Status: 200, Integration latency: 24 ms
Thu Mar 19 23:29:54 UTC 2020 : Endpoint response headers: {Date=Thu, 19 Mar 2020 23:29:54 GMT, Content-Type=application/json, Content-Length=48, Connection=keep-alive, x-amzn-RequestId=f2c2c752-a5e0-45e4-9ff0-d91826b51c7b, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-5e740072-46cee9045a56e04b8023816d;sampled=0}
Thu Mar 19 23:29:54 UTC 2020 : Endpoint response body before transformations: "{\"statusCode\":200,\"isBase64Encoded\":false}"
Thu Mar 19 23:29:54 UTC 2020 : Execution failed due to configuration error: Malformed Lambda proxy response
Thu Mar 19 23:29:54 UTC 2020 : Method completed with status: 502
The actual JSON response catptured in java app & lambda is
{"statusCode":200,"isBase64Encoded":false}
After comparing to Node.js example, seems that it is to do with the handler.
When using a RequestHandler, the response is a String
public class MyLambda implements RequestHandler<Object, String> {
#Override
public String handleRequest(Object input, Context context) {
From API gateway, The response is showing as follows, note that the whole response is a String, and API gateway complains
Thu Mar 19 23:29:54 UTC 2020 : Endpoint response body before transformations: "{\"statusCode\":200,\"isBase64Encoded\":false}"
Thu Mar 19 23:29:54 UTC 2020 : Execution failed due to configuration error: Malformed Lambda proxy response
Changed that to RequestStreamHandler , the error is gone
public class APIgatewayTest implements RequestStreamHandler {
#Override
public void handleRequest(InputStream input, OutputStream output, Context context) throws IOException {
From API gateway, the response is as follows
Thu Apr 09 09:13:08 UTC 2020 : Endpoint response body before transformations: {"statusCode":200,"body":"\"Hello from Lambda!\""}
Thu Apr 09 09:13:08 UTC 2020 : Method response body after transformations: "Hello from Lambda!"
Here's the story:
Sending a request from SOAPUI (5.2.1) gives me either correct response or, in case of invalid parameters, the error in form of xml.
Looking at SoapUI log this is what i see:
Wed Jan 18 16:32:39 EST 2017:DEBUG:Attempt 1 to execute request
Wed Jan 18 16:32:39 EST 2017:DEBUG:Sending request: POST/XBC/services/TranslateGeometryService HTTP/1.1
Wed Jan 18 16:32:48 EST 2017:DEBUG:Receiving response: HTTP/1.1 401 Unauthorized
Wed Jan 18 16:32:48 EST 2017:DEBUG:Connection can be kept alive for 10000 MILLISECONDS
Wed Jan 18 16:32:48 EST 2017:DEBUG:Target requested authentication
Wed Jan 18 16:32:48 EST 2017:DEBUG:Authorization challenge processed
Wed Jan 18 16:32:48 EST 2017:DEBUG:Authentication scope: BASIC 'Spring Security Application'#mywebservice:80
Wed Jan 18 16:32:48 EST 2017:INFO:mywebservice:80 requires authentication with the realm 'Spring Security Application'
Wed Jan 18 16:32:48 EST 2017:DEBUG:Found credentials
Wed Jan 18 16:32:48 EST 2017:DEBUG:Attempt 2 to execute request
Wed Jan 18 16:32:48 EST 2017:DEBUG:Sending request: POST/XBC/services/TranslateGeometryService HTTP/1.1
Wed Jan 18 16:33:14 EST 2017:DEBUG:Receiving response: HTTP/1.1 500 Internal Server Error
Wed Jan 18 16:33:14 EST 2017:DEBUG:Connection shut down
Wed Jan 18 16:33:14 EST 2017:INFO:Got response for [TranslateGeometrySoap11.getLocationFromTramPoleTrackOffset:Request 1] in 35794ms (312 bytes)
(Note that those 312 bytes come after error 500 and connection shut down messages)
Response window, at the same time, in SOAPUI shows this:
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<SOAP-ENV:Fault>
<faultcode>SOAP-ENV:Server</faultcode>
<faultstring xml:lang="en">Error: No Track exists with Track Code: CHS-CSK-UP-TW</faultstring>
</SOAP-ENV:Fault>
</SOAP-ENV:Body>
This is perfect and works as i expect it to work.
Now for my problem that is seriously driving me mad.
When i attempt to send exactly the same request from my java code that is supposed to return the error above, i receive IOException that says:
Server returned HTTP response code: 500 for URL: http://mywebservice/XBC/services/TranslateGeometryService
So, pretty much same as what SOAPUI shows in its log.
But...
No matter what i do in my code i cannot get that message in faultstring that SOAPUI displays: "No Track exists with Track Code: CHS-CSK-UP-TW".
How is SOAPUI doing it after it receives error 500? Or, more importantly - how do i get it in JAVA after receiving error 500?
Tried playing with SoapFaultException class and SoapFault, but i could not find anything more anywhere.
Thanks to anyone who tries to answer this question.
Cheers,
Pierre
The next code in Java handler uses SOAPMessageContext TO detect SOAP fault and extract fault string from SOAPBody:
public bool handleFault(SOAPMessageContext context)
{
try {
if (context.getMessage().getSOAPBody().hasFault()) {
SOAPFault fa = context.getMessage().getSOAPBody().getFault();
System.err.println(fa.getFaultCode());
System.err.println(fa.getFaultString());
}
return true;
} catch (SOAPException ex) {
System.err.println("Fault parsing Exception:" + ex.getMessage());
return true;
}
return false;
}
Could you please explain the conditions under which Yoda time gives different result while working with same input data:
LocalDate value = LocalDate.now()
Here is what debug shows in Junit test
value
2015-10-16
((LocalDate) value).toDateTimeAtStartOfDay().toDate()
Fri Oct 16 00:00:00 EEST 2015
((LocalDate) value).toDate()
Fri Oct 16 00:00:00 EEST 2015
Here is what real environment debug gives (Karaf)
value
2015-10-16
((LocalDate) value).toDateTimeAtStartOfDay().toDate()
Thu Oct 15 20:00:00 EDT 2015
((LocalDate) value).toDate()
Fri Oct 16 00:00:00 EDT 2015
The question is why we have different results when toDateTimeAtStartOfDay() is used?
I have a Jersey REST service with a resource class that calls methods on a service class. During testing we've noticed a latency between the time the resource method's "Entering" log statement and the service's. This latency can be as much as 5 minutes although normally it's in the 2 minute range. Once in awhile, the latency is minimal (milliseconds).
Here's what our classes look like:
Resource
#Stateless
#Path("/provision")
public class ProvisionResource
{
private final Logger logger = LoggerFactory.getLogger(ProvisionResource.class);
#EJB
private ProvisionService provisionService;
#GET
#Produces(MediaType.APPLICATION_XML)
#Path("/subscriber")
public SubscriberAccount querySubscriberAccount(
#QueryParam("accountNum") String accountNum)
{
logger.debug("Entering querySubscriberAccount()");
final SubscriberAccount account;
try
{
account = provisionService.querySubscriber(accountNum);
}
catch (IllegalArgumentException ex)
{
logger.error("Illegal argument while executing query for subscriber account",
ex);
throw new WebApplicationException(Response.Status.BAD_REQUEST);
}
catch (Exception ex)
{
logger.error("Unexpected exception while executing query for subscriber account",
ex);
throw new WebApplicationException(Response.Status.INTERNAL_SERVER_ERROR);
}
logger.debug("Exiting querySubscriberAccount()");
return account;
}
}
Service:
#Singleton
public class ProvisionService
{
private final Logger logger = LoggerFactory.getLogger(ProvisionService.class);
public SubscriberAccount querySubscriber(final String accountNum) throws IllegalArgumentException, Exception
{
logger.debug("Entering querySubscriber()");
if (null == accountNum)
{
throw new IllegalArgumentException("The argument {accountNum} must not be NULL");
}
SubscriberAccount subscriberAccount = null;
try
{
// do stuff to get subscriber account
}
catch (Exception ex)
{
throw new Exception("Caught exception querying {accountNum}=["
+ accountNum + "]", ex);
}
finally
{
logger.debug("Exiting querySubscriber()");
}
return subscriberAccount;
}
Here's some samples from our logs showing the timestamps of when we enter the methods.
2012 Feb 07 15:31:06,303 MST [http-thread-pool-80(1)] DEBUG my.package.ProvisionResource - Entering querySubscriberAccount()
2012 Feb 07 15:31:06,304 MST [http-thread-pool-80(1)] DEBUG my.package.ProvisionService - Entering querySubscriber()
2012 Feb 07 15:35:06,359 MST [http-thread-pool-80(1)] DEBUG my.package.ProvisionResource - Entering querySubscriberAccount()
2012 Feb 07 15:40:33,395 MST [http-thread-pool-80(1)] DEBUG my.package.ProvisionService - Entering querySubscriber()
2012 Feb 07 15:34:06,345 MST [http-thread-pool-80(2)] DEBUG my.package.ProvisionResource - Entering querySubscriberAccount()
2012 Feb 07 15:37:24,372 MST [http-thread-pool-80(2)] DEBUG my.package.ProvisionService - Entering querySubscriber()
2012 Feb 07 15:33:06,332 MST [http-thread-pool-80(4)] DEBUG my.package.ProvisionResource - Entering querySubscriberAccount()
2012 Feb 07 15:34:15,349 MST [http-thread-pool-80(4)] DEBUG my.package.ProvisionService - Entering querySubscriber()
2012 Feb 07 15:37:24,371 MST [http-thread-pool-80(4)] DEBUG my.package.ProvisionResource - Entering querySubscriberAccount()
2012 Feb 07 15:40:36,004 MST [http-thread-pool-80(4)] DEBUG my.package.ProvisionService - Entering querySubscriber()
2012 Feb 07 15:32:06,317 MST [http-thread-pool-80(5)] DEBUG my.package.ProvisionResource - Entering querySubscriberAccount()
2012 Feb 07 15:34:15,325 MST [http-thread-pool-80(5)] DEBUG my.package.ProvisionService - Entering querySubscriber()
2012 Feb 07 15:36:06,373 MST [http-thread-pool-80(5)] DEBUG my.package.ProvisionResource - Entering querySubscriberAccount()
2012 Feb 07 15:40:34,956 MST [http-thread-pool-80(5)] DEBUG my.package.ProvisionService - Entering querySubscriber()
As you can see, the first one called the querySubscriber method in the service almost immediately after entering the resource's querySubscriberAccount. However, subsequent calls to the webservice take ~1 to 5 minutes. There's really nothing happening in the resource that would hold up processing/calling the service.
The webservice is deployed on a Linux server in Glassfish 3.1.1.
Has anyone seen anything like this before?? Any suggestions on what's going on?
EDIT
Just a little more information...
The domain into which the web service war is deployed has 4 applications deployed in it:
the Jersey webservice war which is having issues
an ear which uses a client to the Jersey webservice
a servlet war used to test connections etc. used by the ear (including the Jersey webservice)
another servlet war which does not use the webservice
When we disabled the ear and "other" war file (only the Jersey war and the test servlet were enabled), the latency issue goes away. We re-enabled the war and ear and things still continued to respond in a timely manner. When we redeployed Jersey webservice war (made some logging changes), the latency problem immediately came back.
Thread dump can be used to find out what code (including stack traces) is running and the current moment inside Java process. jps tool will help in get getting PID of the required JVM instance.