We Keep Coding

How to manage concurrent API Calls throughout the lifetime of an application

At the moment i am trying to find the best way to manage concurrent API Calls within my application. Currently i have been using HTTPURLConnection to make my HTTP method calls and although it works fine, eventually i would come across some 'Socket exception: connection reset' whilst calls are being made. however, i am using multithreading as i have many different api calls running concurrently.
I have looked into using AsyncRestTemplate and although it is working i find that in the console a list of the pool and thread is shown i.e [pool-6-thread-1] however when it becomes [pool-2018-thread-1] that is when it decides to stop making any more api calls.
This is the code that i am using:
//This method is inside another class in my actual application but here for simplicity
public static ListenableFuture<ResponseEntity<String>> getLastPrice( AsyncRestTemplate asyncRestTemplate) {
String url = "https://bittrex.com/api/v1.1/public/getmarketsummary?market=btc-dar";
asyncRestTemplate = new AsyncRestTemplate(new ConcurrentTaskExecutor(Executors.newCachedThreadPool()));
return asyncRestTemplate.exchange(url, HttpMethod.GET, new HttpEntity<>("result"), String.class);
}
public PriceThread(JTextField lastPriceJT) {
this.lastPriceJT = lastPriceJT;
}
#Override
public void run() {
AsyncRestTemplate asyncRestTemplate = new AsyncRestTemplate(new ThreadPoolTaskExecutor());
while (true) {
try {
getLastPrice(coin, asyncRestTemplate)
.addCallback(new ListenableFutureCallback<ResponseEntity<String>>() {
#Override
public void onSuccess(ResponseEntity<String> response) {
//TODO: Add real response handling
try {
JSONObject result = new JSONObject(response.getBody());
String status = LOGGER.printResponseToLogger(result);
BigDecimal last = result.getJSONArray("result").getJSONObject(0).getBigDecimal("Last");
lastPriceJT.setText(last.toPlainString());
} catch (Exception e) {
LOGGER.printResponseToLogger(e.getMessage());
}
}
#Override
public void onFailure(Throwable ex) {
//TODO: Add real logging solution
LOGGER.printResponseToLogger(ex.getMessage());
}
});
} catch (Exception e) {
LOGGER.printResponseToLogger(e.getMessage());
}
}
}
Currently i'm thinking the solution to this issue would be for me to reuse the pools so that it doesn't increment to 2018 if that is possible but i have not found a way to do so.

App Engine URL fetch callback

I am relatively new to App Engine. I do not understand how to asynchronously make an HTTP request with Java. I would have thought that this was a very simple matter using Thread and Runnable. But it seems that App Engine does not permit their use.
public Hashtable someApiMethod(..) {
SomeEntity entity = new SomeEntity(..);
ObjectifyService.ofy().save().entity(entity).now();
makeSomeHttpRequest(entity);
return launchResponse;
}
My question is this: how do I implement the method makeSomeHttpRequest(..) such that it returns without waiting for the URLFetchService.fetchAsync to return. I have tried the following without success:
protected void makeSomeHttpRequest(SomeEntity entity) {
URLFetchService fetcher = URLFetchServiceFactory.getURLFetchService();
try {
URL url = new URL("https://www.example.com");
Future future = fetcher.fetchAsync(url);
HTTPResponse response = (HTTPResponse) future.get();
byte[] content = response.getContent();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bos.write(content);
String responseString = new String(bos.toByteArray());
int responseCode = response.getResponseCode();
// Here I will do something with the responseCode and responseString
if (responseCode == 200) entity.someValue = responseString;
} catch (IOException e) {
// handle this
} catch (InterruptedException e) {
// handle this
} catch (ExecutionException e) {
// handle this
}
}
Really what I am trying to do is perform this HTTP request without forcing the method someApiMethod to wait for the response.

Few things:
First. Future doesn't work this way. Method .get waits for result of Feature execution, so basically you're stopping current thread until other thread finishes its execution. You're making it synchronous, it doesn't make any sense. Usually you call .get much later, when all other work in current thread is finished
Second. Threads in Appengine are limited to current request, you have to complete all async processing during current request. So updating an entity in such way doesn't make much sense, it's still bounded to current request. I mean in your case makeSomeHttpRequest(entity); should work faster than return launchResponse;
What you really need is to send this data to TaskQueue and make processing of SomeEntity entity from there (but don't send entity itself, just send ID and load by id from queue task). Basically it's going to be a new request handler (servlet/controller/etc) that should load entity by id, execute makeSomeHttpRequest (synchronously) and return http status 200.
See TaskQueue docs: https://cloud.google.com/appengine/docs/java/taskqueue/
You need Push Queue most likely: https://cloud.google.com/appengine/docs/java/taskqueue/overview-push

NTLM Authentication failing in MultiThreaded application

I have been trying to put together some code that will- among other things - upload files to a Sharepoint site that uses NTLM authentication. Earlier versions of the code were single threaded, and worked perfectly. They uploaded the file exactly as expected without the slightest issue. However, I eventually tried to multi-thread this application, so that it could upload many files at once, while still going about the rest of its business.
However when I tried to multithread the code, it fails every single time, throwing an IndexOutOfBoundsException. This is singularly unhelpful to me in diagnosing the actual cause of the problem.
In case you are wondering, if I change out the CachedThreadExecutor for a SingleThreadExecutor - forcing the code bask to a single-threaded state - it once again works fine.
Creating the executor and connection manager, and constructing threads:
class OrderProcessor implements Runnable {
//Other variables for object
private final ExecutorService executorService = Executors
.newCachedThreadPool();
// .newSingleThreadExecutor();
private HttpClientConnectionManager conManager;
private void setup() {
//always called before execution of anything else in object
conManager = new PoolingHttpClientConnectionManager();
}
//lots of other code
}
The actual code for submitting the threads is complicated, so this version is somewhat simplified, but gets the point across.
for(Request request : requests){
//Do other stuff
simpleSubmitFile(request);
//Do other stuff
}
Here is the simplified file submission method
public Future<Boolean> simpleSubmitFile(Request request){
transferer = new SharePointTransferer(extractionRequest, conManager);
Future<Boolean> future = executorService.submit(transferer);
return future;
}
SharePointTransferer code
//actual values scrubbed
private final String USERNAME = "";
private final String PASSWORD = "";
private final String DOMAIN = "";
private final File sourceFile;
private final String destinationAddress;
private final CloseableHttpClient client;
public SharePointTransferer(final Request extractionRequest, HttpClientConnectionManager conManager) {
super(extractionRequest);
this.sourceFile = this.extractionRequest.getFile();
this.destinationAddress = this.extractionRequest.getDestinationAddress();
this.client = HttpClients.custom()
.setConnectionManager(conManager).build();
}
public Boolean call() throws Exception {
String httpAddress = correctSharePointAddress(destinationAddress);
HttpPut put = new HttpPut(httpAddress + sourceFile.getName());
// construct basic request
put.setEntity(new FileEntity(sourceFile));
HttpClientContext context = HttpClientContext.create();
// set credentials for the SharePoint login
CredentialsProvider credProvider = new BasicCredentialsProvider();
credProvider.setCredentials(AuthScope.ANY, new NTCredentials(USERNAME,
PASSWORD, "", DOMAIN));
context.setCredentialsProvider(credProvider);
// execute request
try {
HttpResponse response = client.execute(put, context);
logger.info("response code was: "
+ response.getStatusLine().getStatusCode());
if (response.getStatusLine().getStatusCode() != 201) {
throw new FileTransferException(
"Could not upload file. Http response code 201 expected."
+ "\nActual status code: "
+ response.getStatusLine().getStatusCode());
}
} catch (ClientProtocolException e) {
throw new FileTransferException(
"Exception Occurred while Transferring file "
+ sourceFile.getName(), e);
} catch (IOException e) {
throw new FileTransferException(
"Exception Occurred while Transferring file "
+ sourceFile.getName(), e);
}finally{
logger.info("deleting source file: " + sourceFile.getName());
sourceFile.delete();
client.close();
}
logger.info("successfully transfered file: "+sourceFile.getName());
return true;
}
If I submit multiple files it throws essentially the exact same exception for all of the files. The trace is below
Exception Stack Trace
2015-04-16 11:49:26 ERROR OrderProcessor:224 - error processing file: FILE_NAME_SCRUBBED
PACKAGE_SCRUBBED.FileProcessingException: Could not process file: FILE_NAME_SCRUBBED
at PACKAGE_SCRUBBED.OrderProcessor.finishProcessingOrder(OrderProcessor.java:223)
at PACKAGE_SCRUBBED.OrderProcessor.run(OrderProcessor.java:124)
at PACKAGE_SCRUBBED.FileTransferDaemon.process(FileTransferDaemon.java:48)
at PACKAGE_SCRUBBED.FileTransferDaemon.start(FileTransferDaemon.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 41
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at PACKAGE_SCRUBBED.OrderProcessor.finishProcessingOrder(OrderProcessor.java:208)
... 8 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 41
at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte(NTLMEngineImpl.java:924)
at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addUShort(NTLMEngineImpl.java:946)
at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getResponse(NTLMEngineImpl.java:1052)
at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTLMEngineImpl.java:148)
at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NTLMEngineImpl.java:1641)
at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme.java:139)
at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSchemeBase.java:138)
at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthenticator.java:239)
at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResponse(HttpAuthenticator.java:202)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:262)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at PACKAGE_SCRUBBED.SharePointTransferer.call(SharePointTransferer.java:74)
at PACKAGE_SCRUBBED.SharePointTransferer.call(SharePointTransferer.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
If anyone can figure out what is causing this problem, I would greatly appreciate it.
EDIT: I managed to find a workaround that fixes the issue for me, but would still appreciate an explanation of exactly what is going on.

this is a bug, solved in httpclient version 4.5.2
http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES-4.5.x.txt
Release 4.5.2
Changelog:
[HTTPCLIENT-1715] NTLMEngineImpl#Type1Message not thread safe but declared as a constant. Contributed by Olivier Lafontaine , Gary Gregory

You can't reuse nor HttpClientContext neither NTLMScheme in a concurrent environment because they are both marked as #NotThreadSafe (see javadoc).
In my environment I got the same error, solved with something like:
synchronized(context) {
HttpResponse response = client.execute(put, context);
}
The authenticated context is reused, but one thread at time.

I eventually managed to solve this problem by setting the number of connections per route to 1, as below.
conManager.setDefaultMaxPerRoute(1);
I'm still not exactly sure why the problem occured, or what the proper way to fix this is, but this solution worked for me.

Java Kryonet servers, client not receiving server response

I am trying to teach myself some networking in Java using the Kryonet library. The following code is almost identical to the code in the kyronet tutorial. https://code.google.com/p/kryonet/#Running_a_server
The client is successfully sending the message "Here is the request!" to the server (the server is printing it out) however the client is not receiving any response from the server even though the server is sending one.
I've tried unsuccessfully to fix it, can anyone see or suggest a possible problem/solution with the code?
(The code follows)
Client
public class Client_test {
Client client = new Client();
public Client_test() {
Kryo kryo = client.getKryo();
kryo.register(SomeRequest.class);
kryo.register(SomeResponse.class);
client.start();
try {
client.connect(50000, "127.0.0.1", 54555, 54777);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
client.addListener(new Listener() {
public void received (Connection connection, Object object) {
if (object instanceof SomeResponse) {
SomeResponse response = (SomeResponse)object;
System.out.println(response.text);
}
}
});
SomeRequest request = new SomeRequest();
request.text = "Here is the request!";
client.sendTCP(request);
}
}
Server
public class ServerGame {
Server server = new Server();
public ServerGame() {
Kryo kryo = server.getKryo();
kryo.register(SomeRequest.class);
kryo.register(SomeResponse.class);
server.start();
try {
server.bind(54555, 54777);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
server.addListener(new Listener() {
public void received (Connection connection, Object object) {
if (object instanceof SomeRequest) {
SomeRequest request = (SomeRequest)object;
System.out.println(request.text);
SomeResponse response = new SomeResponse();
response.text = "Thanks!";
connection.sendTCP(response);
}
}
});
}
}
Response & Request classes
public class SomeRequest {
public String text;
public SomeRequest(){}
}
public class SomeResponse {
public String text;
public SomeResponse(){}
}

After many hours watching youtube videos and sifting through the web I found the answer. Which I will post on here as it seems that quite a few people have had this problem so I would like to spread the word.
Basically the client would shut down immediately, before it could receive and output the message packet. This is because "Starting with r122, client update threads were made into daemon threads, causing the child processes to close as soon as they finish initializing.", the solution is "Maybe you could use this? new Thread(client).start();".
So basically instead of using
client.start();
to start the client thread you must use
new Thread(client).start();
Which I believe stops the thread being made into a daemon thread which therefore stops the problem.
Source: https://groups.google.com/forum/?fromgroups#!topic/kryonet-users/QTHiVmqljgE

Yes, inject a tool like Fiddler in between the two so you can see the traffic going back and forth. It's always easier to debug with greater transparency, more information.

Synchronization in threads for Java

I have a home grown web server in my app. This web server spawns a new thread for every request that comes into the socket to be accepted. I want the web server to wait until a specific point is hit in the thread it just created.
I have been through many posts on this site and examples on the web, but cant get the web server to proceed after I tell the thread to wait. A basic code example would be great.
Is the synchronized keyword the correct way to go about this? If so, how can this be achieved? Code examples are below of my app:
Web Server
while (true) {
//block here until a connection request is made
socket = server_socket.accept();
try {
//create a new HTTPRequest object for every file request
HttpRequest request = new HttpRequest(socket, this);
//create a new thread for each request
Thread thread = new Thread(request);
//run the thread and have it return after complete
thread.run();
///////////////////////////////
wait here until notifed to proceed
///////////////////////////////
} catch (Exception e) {
e.printStackTrace(logFile);
}
}
Thread code
public void run() {
//code here
//notify web server to continue here
}
Update - Final code is as below. The HttpRequest does just call resumeListener.resume() whenever I send a response header (of course also adding the interface as a separate class and the addResumeListener(ResumeListener r1) method in HttpRequest):
Web Server portion
// server infinite loop
while (true) {
//block here until a connection request is made
socket = server_socket.accept();
try {
final Object locker = new Object();
//create a new HTTPRequest object for every file request
HttpRequest request = new HttpRequest(socket, this);
request.addResumeListener(new ResumeListener() {
public void resume() {
//get control of the lock and release the server
synchronized(locker) {
locker.notify();
}
}
});
synchronized(locker) {
//create a new thread for each request
Thread thread = new Thread(request);
//run the thread and have it return after complete
thread.start();
//tell this thread to wait until HttpRequest releases
//the server
locker.wait();
}
} catch (Exception e) {
e.printStackTrace(Session.logFile);
}
}

You can use java.util.concurrent.CountDownLatch with a count of 1 for this. Arrange for an instance of it to be created and shared by the parent and child thread (for example, create it in HttpRequest's constructor, and have it retrievable by a member function). The server then calls await() on it, and the thread hits countDown() when it's ready to release its parent.

You probably need to use a Java Condition. From the docs:
Conditions (also known as condition
queues or condition variables) provide
a means for one thread to suspend
execution (to "wait") until notified
by another thread that some state
condition may now be true.

First of all, I echo the sentiment of others that re-inventing the wheel here will most likely lead to a variety of issues for you. However, if you want to go down this road anyway what you are trying to do is not difficult. Have you experimented with Jetty?
Maybe something like this:
public class MyWebServer {
public void foo() throws IOException {
while (true) {
//block here until a connection request is made
ServerSocket socket = new ServerSocket();
try {
final Object locker = new Object();
//create a new HTTPRequest object for every file request
MyRequest request = new MyRequest(socket);
request.addResumeListener(new ResumeListener() {
public void resume() {
locker.notify();
}
});
synchronized(locker){
//create a new thread for each request
Thread thread = new Thread(request);
//start() the thread - not run()
thread.start();
//this thread will block until the MyRequest run method calls resume
locker.wait();
}
} catch (Exception e) {
}
}
}
}
public interface ResumeListener {
public void resume();
}
public class MyRequest implements Runnable{
private ResumeListener resumeListener;
public MyRequest(ServerSocket socket) {
}
public void run() {
// do something
resumeListener.resume(); //notify server to continue accepting next request
}
public void addResumeListener(ResumeListener rl) {
this.resumeListener = rl;
}
}

Run under a debugger and set a breakpoint?
If unfeasible, then read a line from System.in?