We Keep Coding

In Java, how to test a private code running in parallel with executor service wrapped in public method

I had an existing non-parallel code that we recently made concurrent by using executor service. Adding concurrency ensured limit the number of requests sent to another API in our scenario. So we are calling an external service and limiting requests, waiting for all requests to complete so as merge the responses later before sending the final response.
I am stuck on how to add a unit test/mock test to such a code, considering the private method is parallelized. I have added below my code structure to explain my situation.
I am trying to test here
#Test
public void processRequest() {
...
}
Code
int MAX_BULK_SUBREQUEST_SIZE = 10;
public void processRequest() {
...
// call to private method
ResponseList responseList = sendRequest(requestList);
}
private void sendRequest(List<..> requestList) {
List<Response> responseList = new ArrayList<>();
ExecutorService executorService = Executors.newFixedThreadPool(10);
int numOfSubRequests = requestList.size();
for (int i = 0; i < numOfSubRequests; i += MAX_BULK_SUBREQUEST_SIZE) {
List<Request> requestChunk;
if (i + MAX_BULK_SUBREQUEST_SIZE >= numOfSubRequests) {
requestChunk = requestList.subList(i, numOfSubRequests);
} else {
requestChunk = requestList.subList(i, i + MAX_BULK_SUBREQUEST_SIZE);
}
// parallelization
executorService.submit(() -> {
Response responseChunk = null;
try {
responseChunk = callService(requestChunk); // private method
} catch (XYZException e) {
...
try {
throw new Exception("Internal Server Error");
} catch (Exception ex) {
...
}
}
responseList.add(responseChunk);
});
}
executorService.shutdown();
try {
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
} catch (InterruptedException e) {..}
}
return responseList;
}
private Response callService(..) {
// call to public method1
method1(..);
// call to public method2
method2(..);
}

I was able to do so with unit tests and adding a mockito verify on how many times a method is called. If it's running in parallel after chunkifying, then the method will be called more than once equal to number of chunks it's going to process.

Netty ChannelFuture timeout when response received

I am new in netty I have a tcp client application developed with netty. When i use future get async response from server some response returning but future is not completing into timeout. TCPClient class like following;
public TcpClient {
public boolean connect(Host host) {
try {
Bootstrap clientBootstrap = new Bootstrap()
.group(group)
.channel(NioSocketChannel.class)
.option(ChannelOption.SO_KEEPALIVE,true)
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 50)
.remoteAddress(new InetSocketAddress(host.getIp(), host.getPort()))
.handler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) {
socketChannel.config().setRecvByteBufAllocator(new FixedRecvByteBufAllocator(2146));
FalconClientHandler falconClientHandler = new FalconClientHandler(host);
host.setFalconClientHandler(falconClientHandler);
socketChannel.pipeline().addLast(falconClientHandler);
}
});
channelFuture = clientBootstrap.connect().sync(); //BAŞARI İLE BAĞLANDI
channelFuture.channel().closeFuture().sync();
return host.isActive();
} catch (Exception e) {
log.info("Connection timed out --> " + e);
host.setActive(false);
return false;
} finally {
host.setActive(false);
}
}
public synchronized ResponseFuture send(long transactionId,String message) {
final Map<Long,ResponseFuture> responseFuture = new ConcurrentHashMap<>();
responseFuture.put(transactionId,new ResponseFuture());
if (!hostSelector.getUpHostList().isEmpty()) {
int hostCount = hostSelector.getUpHostList().size();
Host host;
host = hostSelector.getUpHostList().get(index.incrementAndGet() % hostCount);
if (host.isActive()) {
int headerLength = Integer.parseInt(message.substring(8, 12));
log.info("[{}] Host {} Tcp Request",message.substring(52, 52 + headerLength),host.getIp());
channelFuture.addListener((GenericFutureListener<ChannelFuture>) future -> {
log.info("[{}] Tcp request added to map",transactionId);
channelFuture.channel().pipeline().get(FalconClientHandler.class).setResponseFuture(responseFuture);
byte[] byteBuffer = message.getBytes();
channelFuture.channel().writeAndFlush(Unpooled.copiedBuffer(byteBuffer));
});
}
} else {
log.error("AYAKTA HOST YOK");
}
return responseFuture.get(transactionId);
}
}
Send method have transactionId and request message, When i send this message with transaction id response will return with this thransaction id. I am calling this send like following;
ResponseFuture responseFuture = falconClient.send(Long.valueOf(transactionId), finalMessage);
try {
Object obj = responseFuture.get(ddaTimeoutParam, TimeUnit.MILLISECONDS);
if(obj!=null) {
response = obj.toString();
ddaDelta = System.currentTimeMillis()-ddaRequestStartTime;
}
} catch (InterruptedException | ExecutionException | TimeoutException e) {
log.warn("[{}] DDA timeout. Timeout parameter: {}",transactionId,ddaTimeoutParam);
responseFuture.cancel(true);
response = "TIMEOUT";
ddaDelta = System.currentTimeMillis()-ddaRequestStartTime;
}
Response future is a basic Future implementation class. Put and get methods like that;
public class ResponseFuture implements Future<String> {
private volatile State state = State.WAITING;
ArrayBlockingQueue<String> blockingResponse = new ArrayBlockingQueue<String>(1);
private enum State {
WAITING,
DONE
}
#Override
public String get(long timeout, TimeUnit unit) throws InterruptedException,
ExecutionException, TimeoutException {
final String responseAfterWait = blockingResponse.poll(timeout, unit);
if (responseAfterWait == null) {
throw new TimeoutException();
}
return responseAfterWait;
}
public void set(String msg) {
if (state == State.DONE) {
return;
}
try {
blockingResponse.put(msg);
state = State.DONE;
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
My Handler class for receive server response message like following;
public class FalconClientHandler extends SimpleChannelInboundHandler<ByteBuf> {
private ChannelHandlerContext ctx;
private Map<Long,ResponseFuture> responseFuture;
public synchronized void setResponseFuture(Map<Long,ResponseFuture> responseFuture) {
log.info("{} ResponseFuture setted",responseFuture.keySet());
this.responseFuture = responseFuture;
}
#Override
public void channelRead0(ChannelHandlerContext channelHandlerContext, ByteBuf in) {
String input = in.toString(CharsetUtil.UTF_8);
String transactionKey = input.substring(52, 66).trim();
if(responseFuture.get(Long.valueOf(transactionKey))!=null)
responseFuture.get(Long.valueOf(transactionKey)).set(input);
else
log.info("[{}] Tcp Response map is empty",transactionKey);
}
}
When i run this code under high load like 30 transaction per second, tcp response returned from netty server but future get method received timeout.This situation not occuring every request for example %20 request is fail when 30 tps %50 request fail in 40 tps. What can be occur under load?

Java Using CountDownLatch to poll a method until a success response

I am trying to call a method multiple times every 60 seconds until a success response from the method which actually calls a rest end point on a different service. As of now I am using do while loop and using
Thread.sleep(60000);
to make the main thread wait 60 seconds which I feel is not the ideal way due to concurrency issues.
I came across the CountDownLatch method using
CountDownLatch latch = new CountDownLatch(1);
boolean processingCompleteWithin60Second = latch.await(60, TimeUnit.SECONDS);
#Override
public void run(){
String processStat = null;
try {
status = getStat(processStatId);
if("SUCCEEDED".equals(processStat))
{
latch.countDown();
}
} catch (Exception e) {
e.printStackTrace();
}
}
I have the run method in a different class which implements runnable. Not able to get this working. Any idea what is wrong?

You could use a CompletableFuture instead of CountDownLatch to return the result:
CompletableFuture<String> future = new CompletableFuture<>();
invokeYourLogicInAnotherThread(future);
String result = future.get(); // this blocks
And in another thread (possibly in a loop):
#Override
public void run() {
String processStat = null;
try {
status = getStat(processStatId);
if("SUCCEEDED".equals(processStat))
{
future.complete(processStat);
}
} catch (Exception e) {
future.completeExceptionally(e);
}
}
future.get() will block until something is submitted via complete() method and return the submitted value, or it will throw the exception supplied via completeExceptionally() wrapped in an ExecutionException.
There is also get() version with timeout limit:
String result = future.get(60, TimeUnit.SECONDS);

Finally got it to work using Executor Framework.
final int[] value = new int[1];
pollExecutor.scheduleWithFixedDelay(new Runnable() {
Map<String, String> statMap = null;
#Override
public void run() {
try {
statMap = coldService.doPoll(id);
} catch (Exception e) {
}
if (statMap != null) {
for (Map.Entry<String, String> entry : statMap
.entrySet()) {
if ("failed".equals(entry.getValue())) {
value[0] = 2;
pollExecutor.shutdown();
}
}
}
}
}, 0, 5, TimeUnit.MINUTES);
try {
pollExecutor.awaitTermination(40, TimeUnit.MINUTES);
} catch (InterruptedException e) {
}

How to gracefully wait to job task finish in BlockingQueue java

I am writing a job queue using BlockingQueue and ExecutorService. It basically waiting new data in the queue, if there are any data put into the queue, executorService will fetch data from queue. But the problem is that i am using a loop that loops to wait the queue to have data and thus the cpu usage is super high.
I am new to use this api. Not sure how to improve this.
ExecutorService mExecutorService = Executors.newSingleThreadExecutor();
BlockingQueue<T> mBlockingQueue = new ArrayBlockingQueue();
public void handleRequests() {
Future<T> future = mExecutorService.submit(new WorkerHandler(mBlockingQueue, mQueueState));
try {
value = future.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
if (mListener != null && returnedValue != null) {
mListener.onNewItemDequeued(value);
}
}
}
private static class WorkerHandler<T> implements Callable<T> {
private final BlockingQueue<T> mBlockingQueue;
private PollingQueueState mQueueState;
PollingRequestHandler(BlockingQueue<T> blockingQueue, PollingQueueState state) {
mBlockingQueue = blockingQueue;
mQueueState = state;
}
#Override
public T call() throws Exception {
T value = null;
while (true) { // problem is here, this loop takes full cpu usage if queue is empty
if (mBlockingQueue.isEmpty()) {
mQueueState = PollingQueueState.WAITING;
} else {
mQueueState = PollingQueueState.FETCHING;
}
if (mQueueState == PollingQueueState.FETCHING) {
try {
value = mBlockingQueue.take();
break;
} catch (InterruptedException e) {
Log.e(TAG, e.getMessage(), e);
break;
}
}
}
Any suggestions on how to improve this would be much appreciated!

You don't need to test for the queue to be empty, you just take(), so the thread blocks until data is available.
When an element is put on the queue the thread awakens an value is set.
If you don't need to cancel the task you just need:
#Override
public T call() throws Exception {
T value = mBlockingQueue.take();
return value;
}
If you want to be able to cancel the task :
#Override
public T call() throws Exception {
T value = null;
while (value==null) {
try {
value = mBlockingQueue.poll(50L,TimeUnit.MILLISECONDS);
break;
} catch (InterruptedException e) {
Log.e(TAG, e.getMessage(), e);
break;
}
}
return value;
}

if (mBlockingQueue.isEmpty()) {
mQueueState = PollingQueueState.WAITING;
} else {
mQueueState = PollingQueueState.FETCHING;
}
if (mQueueState == PollingQueueState.FETCHING)
Remove these lines, the break;, and the matching closing brace.

How can I terminate Tasks that have timed out in multithreading?

I need to make a library in which I will have synchronous and asynchronous methods in it.
executeSynchronous() - waits until I have a result, returns the result.
executeAsynchronous() - returns a Future immediately which can be processed after other things are done, if needed.
Core Logic of my Library
The customer will use our library and they will call it by passing DataKey builder object. We will then construct a URL by using that DataKey object and make a HTTP client call to that URL by executing it and after we get the response back as a JSON String, we will send that JSON String back to our customer as it is by creating DataResponse object. Some customer will call executeSynchronous() and some might call executeAsynchronous() so that's why I need to provide two method separately in my library.
Interface:
public interface Client {
// for synchronous
public DataResponse executeSynchronous(DataKey key);
// for asynchronous
public Future<DataResponse> executeAsynchronous(DataKey key);
}
And then I have my DataClient which implements the above Client interface:
public class DataClient implements Client {
private RestTemplate restTemplate = new RestTemplate();
private ExecutorService executor = Executors.newFixedThreadPool(10);
// for synchronous call
#Override
public DataResponse executeSynchronous(DataKey key) {
DataResponse dataResponse = null;
Future<DataResponse> future = null;
try {
future = executeAsynchronous(key);
dataResponse = future.get(key.getTimeout(), TimeUnit.MILLISECONDS);
} catch (TimeoutException ex) {
PotoLogging.logErrors(ex, DataErrorEnum.TIMEOUT_ON_CLIENT, key);
dataResponse = new DataResponse(null, DataErrorEnum.TIMEOUT_ON_CLIENT, DataStatusEnum.ERROR);
// does this look right the way I am doing it?
future.cancel(true); // terminating tasks that have timed out.
} catch (Exception ex) {
PotoLogging.logErrors(ex, DataErrorEnum.CLIENT_ERROR, key);
dataResponse = new DataResponse(null, DataErrorEnum.CLIENT_ERROR, DataStatusEnum.ERROR);
}
return dataResponse;
}
//for asynchronous call
#Override
public Future<DataResponse> executeAsynchronous(DataKey key) {
Future<DataResponse> future = null;
try {
Task task = new Task(key, restTemplate);
future = executor.submit(task);
} catch (Exception ex) {
PotoLogging.logErrors(ex, DataErrorEnum.CLIENT_ERROR, key);
}
return future;
}
}
Simple class which will perform the actual task:
public class Task implements Callable<DataResponse> {
private DataKey key;
private RestTemplate restTemplate;
public Task(DataKey key, RestTemplate restTemplate) {
this.key = key;
this.restTemplate = restTemplate;
}
#Override
public DataResponse call() {
DataResponse dataResponse = null;
String response = null;
try {
String url = createURL();
response = restTemplate.getForObject(url, String.class);
// it is a successful response
dataResponse = new DataResponse(response, DataErrorEnum.NONE, DataStatusEnum.SUCCESS);
} catch (RestClientException ex) {
PotoLogging.logErrors(ex, DataErrorEnum.SERVER_DOWN, key);
dataResponse = new DataResponse(null, DataErrorEnum.SERVER_DOWN, DataStatusEnum.ERROR);
} catch (Exception ex) {
PotoLogging.logErrors(ex, DataErrorEnum.CLIENT_ERROR, key);
dataResponse = new DataResponse(null, DataErrorEnum.CLIENT_ERROR, DataStatusEnum.ERROR);
}
return dataResponse;
}
// create a URL by using key object
private String createURL() {
String url = somecode;
return url;
}
}
Problem Statement:-
When I started working on this solution, I was not terminating the tasks that have timed out. I was reporting the timeout to the client, but the task continues to run in the thread pool (potentially occupying one of my limited 10 threads for a long time). So I did some research online and I found that I can cancel my tasks those have timed out by using cancel on future as shown below -
future.cancel(true);
But I wanted to make sure, does it look right the way I am doing in my executeSynchronous method to cancel the tasks that have got timedout?
Since I am calling cancel() on theFuture which will stop it from running if tasks is still in the queue so I am not sure what I am doing is right or not? What is the right approach to do this?
If there is any better way, then can anyone provide an example for that?

If task is still in the queue then cancelling it by simply calling future.cancel() is ok but obviously you don't know if that is in the queue. Also even if you ask future to interrupt the task it may not work as your task can still be doing something which is ignoring the thread interrupted status.
So you can use the future.cancel(true) but you need to make sure that your task (thread) does regard the thread interrupted status. For example as you mentioned you make http call, so you might need to close the http client resource as soon as thread is interrupted.
Please refer to the example below.
I have tried to implement the task cancellation scenario. Normally a thread can check isInterrupted() and try to terminate itself. But this becomes more complex when you are using thread pool executors, callable and if the task is not really like while(!Thread.isInterrupted()) {// execute task}.
In this example, a task is writing a file (I did not use http call to keep the it simple). A thread pool executor starts running the task but the caller wants to cancel it just after 100 milli seconds. Now future sends the interrupt signal to the thread but the callable task can not check it immediately while writing to file. So to make this happen callable maintains a list of IO resources it is going to use and as soon as future wants to cancel the task it just calls cancel() on all IO resources which terminates the task with IOException and then thread finishes.
public class CancellableTaskTest {
public static void main(String[] args) throws Exception {
CancellableThreadPoolExecutor threadPoolExecutor = new CancellableThreadPoolExecutor(0, 10, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>());
long startTime = System.currentTimeMillis();
Future<String> future = threadPoolExecutor.submit(new CancellableTask());
while (System.currentTimeMillis() - startTime < 100) {
Thread.sleep(10);
}
System.out.println("Trying to cancel task");
future.cancel(true);
}
}
class CancellableThreadPoolExecutor extends ThreadPoolExecutor {
public CancellableThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
}
#Override
protected <T> RunnableFuture<T> newTaskFor(Callable<T> callable) {
return new CancellableFutureTask<T>(callable);
}
}
class CancellableFutureTask<V> extends FutureTask<V> {
private WeakReference<CancellableTask> weakReference;
public CancellableFutureTask(Callable<V> callable) {
super(callable);
if (callable instanceof CancellableTask) {
this.weakReference = new WeakReference<CancellableTask>((CancellableTask) callable);
}
}
public boolean cancel(boolean mayInterruptIfRunning) {
boolean result = super.cancel(mayInterruptIfRunning);
if (weakReference != null) {
CancellableTask task = weakReference.get();
if (task != null) {
try {
task.cancel();
} catch (Exception e) {
e.printStackTrace();
result = false;
}
}
}
return result;
}
}
class CancellableTask implements Callable<String> {
private volatile boolean cancelled;
private final Object lock = new Object();
private LinkedList<Object> cancellableResources = new LinkedList<Object>();
#Override
public String call() throws Exception {
if (!cancelled) {
System.out.println("Task started");
// write file
File file = File.createTempFile("testfile", ".txt");
BufferedWriter writer = new BufferedWriter(new FileWriter(file));
synchronized (lock) {
cancellableResources.add(writer);
}
try {
long lineCount = 0;
while (lineCount++ < 100000000) {
writer.write("This is a test text at line: " + lineCount);
writer.newLine();
}
System.out.println("Task completed");
} catch (Exception e) {
e.printStackTrace();
} finally {
writer.close();
file.delete();
synchronized (lock) {
cancellableResources.clear();
}
}
}
return "done";
}
public void cancel() throws Exception {
cancelled = true;
Thread.sleep(1000);
boolean success = false;
synchronized (lock) {
for (Object cancellableResource : cancellableResources) {
if (cancellableResource instanceof Closeable) {
((Closeable) cancellableResource).close();
success = true;
}
}
}
System.out.println("Task " + (success ? "cancelled" : "could not be cancelled. It might have completed or not started at all"));
}
}
For your REST Http client related requirement you can modify the factory class something like this -
public class CancellableSimpleClientHttpRequestFactory extends SimpleClientHttpRequestFactory {
private List<Object> cancellableResources;
public CancellableSimpleClientHttpRequestFactory() {
}
public CancellableSimpleClientHttpRequestFactory(List<Object> cancellableResources) {
this.cancellableResources = cancellableResources;
}
protected HttpURLConnection openConnection(URL url, Proxy proxy) throws IOException {
HttpURLConnection connection = super.openConnection(url, proxy);
if (cancellableResources != null) {
cancellableResources.add(connection);
}
return connection;
}
}
Here you need to use this factory while creating RestTemplate in your runnable task.
RestTemplate template = new RestTemplate(new CancellableSimpleClientHttpRequestFactory(this.cancellableResources));
Make sure that you pass the same list of cancellable resources that you have maintained in CancellableTask.
Now you need to modify the cancel() method in CancellableTask like this -
synchronized (lock) {
for (Object cancellableResource : cancellableResources) {
if (cancellableResource instanceof HttpURLConnection) {
((HttpURLConnection) cancellableResource).disconnect();
success = true;
}
}
}