We Keep Coding

Looking for ways to detect AWS Lambda timeouts(few seconds before timeout) in Java and to test the same

My current Lambda function is calling a 3rd party web service Synchronously.This function occasionally times out (current timeout set to 25s and cannot be increased further)
My code is something like:
handleRequest(InputStream input, OutputStream output, Context context) throws IOException {
try{
response = calling 3rd party REST service
}catch(Exception e){
//handle exceptions
}
}
1)I want to custom handle the timeout (tracking the time and handling few milli seconds before actual timeout) within my Lambda function by sending a custom error message back to the client.
How can I effectively use the
context.getRemainingTimeInMillis()
method to track the time remaining while my synchronous call is running? Planning to call the context.getRemainingTimeInMillis() asynchronously.Is that the right approach?
2)What is a good way to test the timeout custom functionality ?

I solved my problem by increasing the Lambda timeout and invoking my process in a new thread and timing out the Thread after n seconds.
ExecutorService service = Executors.newSingleThreadExecutor();
try {
Runnable r = () ->{
try {
myFunction();
} catch (Exception e) {
e.printStackTrace();
}
};
f = service.submit(r);
f.get(n, TimeUnit.MILLISECONDS);// attempt the task for n milliseconds
}catch(TimeoutException toe){
//custom logic
}

Another option is to use the
readTimeOut
property of the RestClient(in my case Jersey) to set the timeout.But I see that this property is not working consistently within the Lambda code.Not sure if it's and issue with the Jersey client or the Lambda.

You can try with cancellation token to return custom exceptions with lambda before timeout.
try
{
var tokenSource = new CancellationTokenSource(TimeSpan.FromMinutes(1)); // set timeout value
var taskResult = ApiCall(); // call web service method
while (!taskResult.IsCompleted)
{
if (tokenSource.IsCancellationRequested)
{
throw new OperationCanceledException("time out for lambda"); // throw custom exceptions eg : OperationCanceledException
}
}
return taskResult.Result;
}
catch (OperationCanceledException ex)
{
// handle exception
}

How to register a service too ZooKeeper using Curator.x.discovery

I am trying to register a simple REST service on int port,
to ZooKeeper server at localhost:2181.
I checked path ls / using zooClient too.
Any ideas?
private static void registerInZookeeper(int port) throws Exception {
CuratorFramework curatorFramework = CuratorFrameworkFactory
.newClient("localhost:2181", new RetryForever(5));
curatorFramework.start();
ServiceInstance<Object> serviceInstance = ServiceInstance.builder()
.address("localhost")
.port(port)
.name("worker")
.uriSpec(new UriSpec("{scheme}://{address}:{port}"))
.build();
ServiceDiscoveryBuilder.builder(Object.class)
.basePath("myNode")
.client(curatorFramework)
.thisInstance(serviceInstance)
.build()
.start();
Optional.ofNullable(curatorFramework.checkExists().forPath("/zookeeper")).ifPresent(System.out::println);
Optional.ofNullable(curatorFramework.checkExists().forPath("/myNode")).ifPresent(System.out::println);
}

I kept receiving Received packet at server of unknown type 15 from Zoo Server, because of compatibility issues

the registration code here looks correct. In order to print registered instances the following code can be executed:
Optional.ofNullable(curatorFramework.getChildren().forPath("/myNode/worker"))
.orElse(Collections.emptyList())
.forEach(childNode -> {
try {
System.out.println(childNode);
System.out.println(new String(curatorFramework.getData().forPath("/myNode/worker/" + childNode)));
} catch (Exception e) {
e.printStackTrace();
}
});
The result will be like
07:23:12.353 INFO [main-EventThread] ConnectionStateManager:228 - State change: CONNECTED
48202336-e89b-4724-912b-89620f7c9954
{"name":"worker","id":"48202336-e89b-4724-912b-89620f7c9954","address":"localhost","port":1000,"sslPort":null,"payload":null,"registrationTimeUTC":1515561792319,"serviceType":"DYNAMIC","uriSpec":{"parts":[{"value":"scheme","variable":true},{"value":"://","variable":false},{"value":"address","variable":true},{"value":":","variable":false},{"value":"port","variable":true}]}}

Creating your curator framework with zk34 (the version used by kafka) compatibility should fix your problem
private CuratorFramework buildFramework(String ip) {
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
return CuratorFrameworkFactory.builder().zk34CompatibilityMode(true).connectString(ip + ":2181")
.retryPolicy(retryPolicy).build();
}
Please note that curator will just try its best and some new methods (eg. creatingParentsIfNeeded (ok) vs creatingParentContainersIfNeeded (ko)) will fail.

MockRestServiceServer simulate backend timeout in integration test

I am writing some kind of integration test on my REST controller using MockRestServiceServer to mock backend behaviour.
What I am trying to achieve now is to simulate very slow response from backend which would finally lead to timeout in my application. It seems that it can be implemented with WireMock but at the moment I would like to stick to MockRestServiceServer.
I am creating server like this:
myMock = MockRestServiceServer.createServer(asyncRestTemplate);
And then I'm mocking my backend behaviour like:
myMock.expect(requestTo("http://myfakeurl.blabla"))
.andExpect(method(HttpMethod.GET))
.andRespond(withSuccess(myJsonResponse, MediaType.APPLICATION_JSON));
Is it possible to add some kind of a delay or timeout or other kind of latency to the response (or maybe whole mocked server or even my asyncRestTemplate)? Or should I just switch to WireMock or maybe Restito?

You can implement this test functionality this way (Java 8):
myMock
.expect(requestTo("http://myfakeurl.blabla"))
.andExpect(method(HttpMethod.GET))
.andRespond(request -> {
try {
Thread.sleep(TimeUnit.SECONDS.toMillis(1));
} catch (InterruptedException ignored) {}
return new MockClientHttpResponse(myJsonResponse, HttpStatus.OK);
});
But, I should warn you, that since MockRestServiceServer simply replaces RestTemplate requestFactory any requestFactory settings you'd make will be lost in test environment.

If you control timeout in your http client and use for example 1 seconds you can use mock server delay
new MockServerClient("localhost", 1080)
.when(
request()
.withPath("/some/path")
)
.respond(
response()
.withBody("some_response_body")
.withDelay(TimeUnit.SECONDS, 10)
);
If you want to drop connection in Mock Server use mock server error action
new MockServerClient("localhost", 1080)
.when(
request()
.withPath("/some/path")
)
.error(
error()
.withDropConnection(true)
);

Approach that you can go for:
Specifying the responsebody either with Class Path resource or normal string content. More detailed version of what Skeeve suggested above
.andRespond(request -> {
try {
Thread.sleep(TimeUnit.SECONDS.toMillis(5)); // Delay
} catch (InterruptedException ignored) {}
return withStatus(OK).body(responseBody).contentType(MediaType.APPLICATION_JSON).createResponse(request);
});

In Restito, there is a buil-in function to simulate timeout:
import static com.xebialabs.restito.semantics.Action.delay
whenHttp(server).
match(get("/something")).
then(delay(201), stringContent("{}"))

In general, you can define your custom request handler, and do a nasty Thread.sleep() there.
This would be possible in Restito with something like this.
Action waitSomeTime = Action.custom(input -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return input;
});
whenHttp(server).match(get("/asd"))
.then(waitSomeTime, ok(), stringContent("Hello World"))
Not sure about Spring, however. You can easily try. Check DefaultResponseCreator for inspiration.

Mongo client {Mongo-Java-driver} creation hangs duration . No exception thrown

I have a cron-job running at a Linux machine running after every 5 minutes. The job executes a Java class.
private MongoClient createConnection(int retry,List<ServerAddress> host){
try {
System.out.println("Retrying----------"+retry);
MongoClient client = new MongoClient(host, MongoClientOptions.builder()
.connectionsPerHost(10)
.threadsAllowedToBlockForConnectionMultiplier(5)
.connectTimeout(5000).writeConcern(WriteConcern.NORMAL).build());
client.getDB("admin").command("ping").throwOnError();
retry = 0;
return client;
} catch (Exception e) {
retry++;
if (retry < retryLimit) {
createConnection(retry,host);
} else {
System.out.println("Connection could not be established to host-"+host);
}
return null;
}
}
retry is the integer value denoting how many times client creation can be tried in case host is unreachable.
The host list that i am passing is -
public static List<ServerAddress> HOST_SCRIPT = new ArrayList<ServerAddress>() {
private static final long serialVersionUID = 1L;
{
try {
add(new ServerAddress("PrimaryHost23", 27017));
} catch (UnknownHostException e) {
e.printStackTrace();
}
}
};
Code is Stuck when i MongoClient is being created. It does not happen always. Code works fine and NEVER hangs when i run on my local machine. There is no exception thrown.
I recently upgraded Linux machine OS (from CentOS 5 to CentOS 6). Can this be responsible for this because this script was working fine earlier.
Please help.
Regards,
Vibhav

The thing what you can do is you can throw mongo exception try out that of mongo client is stuck you will get to know try out this https://api.mongodb.org/java/2.6/com/mongodb/MongoException.html

Yes of course, actually i was creating crawler in java which fetch all the links of any particular website and validate the css and html structure Using the Jsoup and jcabi api but when i used to store links to the database it was not throwing any exception and even not storing the data also. so i did this
catch (MongoException e){
System.err.print(e.getClass().getName()+": "+e.getMessage());
}

Have you checked the compatibility like of jar that you have uploaded for your project like before it was like Mongo mongo = new Mongo(host,port); but That is deprecated. Try to check that and even your MongoDb jar.

try-catch-finally block in java

As per my understanding, I want to follow the best practice for releasing the resources at the end to prevent any connection leaks. Here is my code in HelperClass.
public static DynamoDB getDynamoDBConnection()
{
try
{
dynamoDB = new DynamoDB(new AmazonDynamoDBClient(new ProfileCredentialsProvider()));
}
catch(AmazonServiceException ase)
{
//ase.printStackTrace();
slf4jLogger.error(ase.getMessage());
slf4jLogger.error(ase.getStackTrace());
slf4jLogger.error(ase);
}
catch (Exception e)
{
slf4jLogger.error(e);
slf4jLogger.error(e.getStackTrace());
slf4jLogger.error(e.getMessage());
}
finally
{
dynamoDB.shutdown();
}
return dynamoDB;
}
My doubt is, since the finally block will be executed no matter what, will the dynamoDB returns empty connection because it will be closed in finally block and then execute the return statement? TIA.

Your understanding is correct. dynamoBD.shutdown() will always execute before return dynamoDB.
I'm not familiar with the framework you're working with, but I would probably organize the code as follows:
public static DynamoDB getDynamoDBConnection()
throws ApplicationSpecificException {
try {
return new DynamoDB(new AmazonDynamoDBClient(
new ProfileCredentialsProvider()));
} catch(AmazonServiceException ase) {
slf4jLogger.error(ase.getMessage());
slf4jLogger.error(ase.getStackTrace());
slf4jLogger.error(ase);
throw new ApplicationSpecificException("some good message", ase);
}
}
and use it as
DynamoDB con = null;
try {
con = getDynamoDBConnection();
// Do whatever you need to do with con
} catch (ApplicationSpecificException e) {
// deal with it gracefully
} finally {
if (con != null)
con.shutdown();
}
You could also create an AutoCloseable wrapper for your dynamoDB connection (that calls shutdown inside close) and do
try (DynamoDB con = getDynamoDBConnection()) {
// Do whatever you need to do with con
} catch (ApplicationSpecificException e) {
// deal with it gracefully
}

Yes,dynamoDB will return an empty connection as dynamoBD.shutdow() will be executed before return statement, Always.

Although I am not answering your question about the finally block being executed always (there are several answers to that question already), I would like to share some information about how DynamoDB clients are expected to be used.
The DynamoDB client is a thread-safe object and is intended to be shared between multiple threads - you can create a global one for your application and re-use the object where ever you need it. Generally, the client creation is managed by some sort of IoC container (Spring IoC container for example) and then provided by the container to whatever code needs it through dependency injection.
Underneath the hood, the DynamoDB client maintains a pool of HTTP connections for communicating the DynamoDB endpoint and uses connections from within this pool. The various parameters of the pool can be configured by passing an instance of the ClientConfiguration object when constructing the client. For example, one of the parameters is the maximum number of open HTTP connections allowed.
With the above understanding, I would say that since the DynamoDB client manages the lifecycle of HTTP connections, resource leaks shouldn't really be concern of code that uses the DynamoDB client.

How about we "imitate" the error and see what happens ? This is what I mean:
___Case 1___
try{
// dynamoDB = new DynamoDB(new AmazonDynamoDBClient(new ProfileCredentialsProvider()));
throw new AmazonServiceException("Whatever parameters required to instantiate this exception");
} catch(AmazonServiceException ase)
{
//ase.printStackTrace();
slf4jLogger.error(ase.getMessage());
slf4jLogger.error(ase.getStackTrace());
slf4jLogger.error(ase);
}
catch (Exception e)
{
slf4jLogger.error(e);
slf4jLogger.error(e.getStackTrace());
slf4jLogger.error(e.getMessage());
}
finally
{
//dynamoDB.shutdown();
slf4jLogger.info("Database gracefully shutdowned");
}
___Case 2___
try{
// dynamoDB = new DynamoDB(new AmazonDynamoDBClient(new ProfileCredentialsProvider()));
throw new Exception("Whatever parameters required to instantiate this exception");
} catch(AmazonServiceException ase)
{
//ase.printStackTrace();
slf4jLogger.error(ase.getMessage());
slf4jLogger.error(ase.getStackTrace());
slf4jLogger.error(ase);
}
catch (Exception e)
{
slf4jLogger.error(e);
slf4jLogger.error(e.getStackTrace());
slf4jLogger.error(e.getMessage());
}
finally
{
//dynamoDB.shutdown();
slf4jLogger.info("Database gracefully shutdowned");
}
These exercise could be a perfect place to use unit tests and more specifically mock tests. I suggest you to take a close look at JMockit, which will help you write such tests much more easily.