So,
Lately I started to play with non-blocking, async servlets (Servlet 3.1). As far as I understood,
by using async servlets there is a possibility to release a working thread while a non-blocking
operation is being done, so basically none of the threads is blocked. So i started doing some
load tests using Gatling stress tol, I used Spring RestController to create two simple controllers -
one doing a blocking HTTP get using RestTemplate (Spring's HTTP client), and one doing non-blocking
requests using AsyncRestTemplate and after processing a message with a callback, so those 2 follow:
#RequestMapping("/non-blocking")
public DeferredResult<String> nonBlocking() throws IOException {
DeferredResult<String> deferredResult = new DeferredResult<String>();
AsyncRestTemplate restTemplate = new AsyncRestTemplate();
String URL ="http://localhost:8080/static.html";
HttpMethod method = HttpMethod.GET;
Class<String> responseType = String.class;
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.TEXT_PLAIN);
HttpEntity<String> requestEntity = new HttpEntity<String>("params", headers);
ListenableFuture<ResponseEntity<String>> future = restTemplate.exchange(URL, method, requestEntity, responseType);
future.addCallback(new ListenableFutureCallback<ResponseEntity<String>>() {
#Override
public void onSuccess(ResponseEntity<String> result) {
deferredResult.setResult(result.getBody());
}
#Override
public void onFailure(Throwable ex) {
System.out.println("Failure while processing");
}
});
return deferredResult;
}
#RequestMapping("/blocking")
public String blockingRouter() {
HttpStatus status = HttpStatus.INTERNAL_SERVER_ERROR;
LOG.logStartBlocking();
try {
ResponseEntity<String> result = restTemplate.getForEntity("http://localhost:8080/static.html");
status = result.getStatusCode();
return result.getBody();
} finally {
System.out.println("Processing done");
}
}
So, as you can see non-blocking is returning DefferedResult (Spring's Future abstraction) and if I
get it right, after sending a HTTP request, thread should be freed and some (probably) other thread
would return a response when it's available. Well, after doing some load tests with 200 concurrent users, test went terribly wrong for a non-blocking version. Threads were fine, but upon reaching 400 req/s, controller just stopped returning responses - it got stuck, probably because of too high response time. Blocking one responded to all of the requests with cca 1000 req/s. So the question is did I do a good testing or missed something because results were totally unexpected for me. Server used was Tomcat (+NIO connector) with 50 threads - hoped to prove that non-blocking won't use many threads and reach it till the end and that blocking will fail due to thread exhaustion but it didn't happen. Any suggestions?
Thanks in advance.
Related
How can i limit threads for task, that being executed in parallel? The issue is simple - while my scheduler working, i can't do anything else (fetch some info using postman etc). Is there any way to solve this problem?
Also, i've tryed to set number of threads in flux, for example, using parallel(3).runOn(Schedulers.parallel()) and still my programm is blocked.
#Scheduled(fixedRate = 60000L)
#PostConstruct
public void fillMap() {
Flux.fromIterable(proxyParserService.getProxyList())
.parallel()
.runOn(Schedulers.parallel())
.flatMap(geoDataService::getData)
//some logic here...
Also worh mentioning, that i have flatmap method with opening connections in parallel:
public Mono<Address> getData(Address proxy) {
WebClient webClient = WebClient.builder()
.baseUrl(String.format(URL, proxy.getHost()))
.build();
WebClient.RequestBodyUriSpec request = webClient.method(HttpMethod.GET);
return request.retrieve()
.onStatus(HttpStatus::isError, clientResponse -> {
log.error("Error while calling endpoint {} with status code {}",
URL, clientResponse.statusCode());
throw new RuntimeException("Error while calling geolocation endpoint");
})
.bodyToMono(Address.class)
I am getting 504 Gateway Time-out issue Spring Boot Rest call using HTTP GET call for heavy record(more than 80K), I am calling other service to get data using RestTemplate object resClient, code as below:
public ResponseEntity<String> getData(String endPointUrl, Map<String, Object> parameterMap, String smToken) throws Exception {
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));
headers.add("Cookie", smToken);
//headers.setContentType(MediaType.APPLICATION_JSON_UTF8);
HttpEntity<Map<String, Object>> entity = new HttpEntity<Map<String, Object>>(parameterMap, headers);
ResponseEntity<String> responseEntity = null;
try {
SSLUtil.turnOffSslChecking();
LOGGER.info("Start call to end point : " +endPointUrl+ " at :"+ (new Date().toString()) );
//resClient.getMessageConverters()
//.add(0, new StringHttpMessageConverter(Charset.forName("UTF-8")));
responseEntity = resClient.exchange(endPointUrl, HttpMethod.POST, entity,String.class);
LOGGER.info("End of call to end point : " +endPointUrl+ " at :"+ (new Date().toString()) );
LOGGER.debug("Response from end point: " + responseEntity);
} catch (Exception e) {
LOGGER.error("Exception while making a http call to " + endPointUrl,e);
throw e;
}
return responseEntity;
}
While debugging I am seeing Response from other service call It takes more than 4 minutes, but rather than waiting it by that time to get response it use to come out only after 3 minutes. How can we make it to wait for response coming from other service call?
I tried to resolve this issue with increasing timeout time to 5 minutes using attribute server.connection-timeout=300000 in application.properties, but I am getting empty response. I am not sure If this is right approach or not. Please help me on this issue.
504 Gateway Time-out issue is generally thrown by proxy server that means server is closing connection. If client close connection then you will get Connectiontimeout error.
server.connection-timeout= #
Time that connectors wait for another HTTP request before closing the
connection. When not set, the connector's container-specific default
is used.
Use a value of -1 to indicate no (that is, an infinite) timeout. (might be bad fix)
OR Try to set it from Application
#SpringBootApplication
public class Application {
public static void main(String[] args) throws Exception {
System.setProperty("server.port","8080"));
System.setProperty("server.connection-timeout","300000");
System.setProperty("server.tomcat.max-threads","yourValue"); //in-case if you want to chaneg number of thredas
SpringApplication.run(Application.class, args);
}
}
Also, Refer This
Try this
#Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder)
{
return restTemplateBuilder
.setConnectTimeout(...)
.setReadTimeout(...)
.build();
}
I have a controller which receives a request to check status of a game, I want the whole thing to be done asynchronously. The controller seems to be working asynchronous, however postman hangs when I do the post request.
Here is my controller:
#RequestMapping(value = "/status", method = RequestMethod.POST, consumes="application/json")
private Callable<Byte> getStatus(#RequestBody final Matrix board){
System.out.println("Entering controller");
Callable<Byte> asyncResult = new Callable<Byte>() {
#Override
public Byte call() throws Exception {
return status.checkWinner(board);
}
};
System.out.println("Leaving controller");
return asyncResult ;
}
and here is my method in Status class:
public Byte checkWinner(Matrix board) {
System.out.println("start Slow work");
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("finish Slow work");
return 0;
}
the program output shows this:
Entering controller
Leaving controller
start Slow work
finish Slow work
So i tried to put #Async on top of checkWinner method, postman shows 200 status ok, without showing the result 0 later on. And without the #Async annotation, postman (webpage) freezes for 5 seconds and the result 0 is shown.
This is an expected behaviour. The Callable returned is executed in a separate thread pool and the request thread is free for another processing. However, the connection to that specific client will be open and the client will be responded to only after callable is complete. Client will wait for the response or will timeout if the response is not obtained in the expected time.
The above model is only for server side load handling not client side. So this model will not work for you.
If you want the client to return, then you should return a jobid as the return value. Provide that client with a poll url to check for response. When the job is complete, return the response.
I try to understand the way asynchronous responses work with Jersey. I read chapter 10 of the Jersey documentation (https://jersey.java.net/documentation/latest/async.html) but it doesn't help with my problem. Also research here on stackoverflow didn't result in satisfying answers (that I can understand).
What I'm trying to do is similar to the one question in this post (Use http status 202 for asynchronous operations). I want to upload a large file to the server using a HTML form document. After the request is send to the server the web service should immediately response with status 202 and a URI where the file can be found after the request has finished.
After reading the post abive it seems possible but sadly no hints how to implement such a behavior where given.
I wrote a small web service to test the functionality:
#Path("/test/async/")
public class TestAsyncResponse {
#GET
#Path("get")
public Response asyncGet(#Suspended final AsyncResponse response) {
new Thread(new Runnable() {
#Override
public void run() {
DateFormat df = new SimpleDateFormat("dd/MM/yy HH:mm:ss");
System.out.println("#### thread started: "
+ df.format(new Date()) + " ####");
String result = veryExpensiveOperation();
System.out.println("#### thread finished: "
+ df.format(new Date()) + " ####");
response.resume(result);
}
private String veryExpensiveOperation() {
try {
Thread.sleep(10000);
}
catch (InterruptedException e) {
e.printStackTrace();
}
return "Woke up!";
}
}).start();
return Response.status(202).entity("Request accepted. " +
"Long running operation started")
.build();
}
}
The service works but as a response I get the "Woke Up!" message after the 10 second wait rather than the 202 response which seems logical because the AsyncResponse is the one that handles the response (as I understand it).
After reading the documentation I got the impression that this is suppose to happen because all Jersey does with the asynchronous server response is to outsource the thread from the response thread pool to another one to free processing time for more responses to the service.
So my two questions would be: Is my understanding correct and can I use the asynchronous server response to get the desired behavior?
I tried to start a new thread without the AsyncResponse and I get a NullPointerExceptionbecause Jersey already closed the response and thus closed the InputStream that contains the file data. Is this the expected behavior? This post (https://stackoverflow.com/a/17559684/1416602) seems to indicate that it might work.
Any response is greatly appreciated.
Greetings
Your question is mixing two topics.
From HTTP perspective, 202 is technically a completed request. And the result of the request is 202, server telling you it will do it on the side. You will have to make another HTTP request to get updated status.
From the perspective of your application, async means that you will execute the request in a separate thread (or other async way). But also, this means that you will not return a result, not even 202, until the other "veryExpensiveOperation" finishes. The whole point in jumping through this hoop is to free up the calling thread. Your web server has a limited number, e.g. 20, and if each of your requests took a very long time, all 20 would be hanging. Using #Suspended you transfer execution from the web server thread to some other means, (another thread in your case). This is really only the first step. The idea behind async servers is that even the veryExpensiveOperation is implemented in some async way so that waiting for a DB or a file does not occupy a whole thread.
I have been through the same pain recently. Jersey keeps claiming it supports Asynchronous REST calls, but I think it's being disingenuous.
And in fact, once I started to work out the correct way of doing this, Jersey actually got in the way.
private static ExecutorService executorService = Executors.newFixedThreadPool( Integer.valueOf( numberOfThreads ) );
#POST
#Path("async")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response async( #FormDataParam("file") InputStream inputStream,
#FormDataParam("file") FormDataContentDisposition des ) throws Throwable {
String uniqueID = UUID.randomUUID().toString();
executorService.execute( new Runnable() {
#Override
public void run() {
try {
// do long performing action
} catch (Exception ex) {
}
}
} );
return Response.accepted().location( getResultsURI( uniqueID ) ).build();
}
#GET
#Path("results/{uniqueID}")
#Produces("application/zip")
public Response results( #PathParam(value = "uniqueID ") String uniqueID ) {
// Check status of job
// If not finished...
if (notFinished) {
return Response.status( 202 ).location( getResultsURI( uniqueID ) )
.entity( status ).build();
}
return Response.ok( FileUtils.readFileToByteArray( zip.toFile() ) ).type( "application/zip" )
.header( "Content-Disposition", "attachment; filename=\"filename.zip\"" ).build();
}
protected URI getResultsURI( String uniqueID ) throws URISyntaxException {
return new URI( Constants.WS_VERSION + "/results/" + uniqueID );
}
The biggest pain was that when you set Response.location(), even if you set it to "./results" or "/results", Jersey expands it to the full URL. Which would be fine, except that it ignores any class-level #Path:
#Path(Constants.WS_VERSION)
public class MyEndpoint {
So instead of fighting it, I used the above code to at least make it correct. Ideally I'd like Jersey to leave the "Location" header alone.
Anyway - the above code is what I used (excluding the business logic bits ;) )
Lead by several examples and questions answered here ( mainly
http://www.javaworld.com/javaworld/jw-02-2009/jw-02-servlet3.html?page=3 ), I want to have server sending the response multiple times to a client without completing the request. When request times out, I create another one and so on.
I want to avoid long polling, since I have to recreate request every time I get the response. (and that quite isn't what async capabilities of servlet 3.0 are aiming at).
I have this on server side:
#WebServlet(urlPatterns = {"/home"}, name = "async", asyncSupported = true)
public class CometServlet extends HttpServlet {
public void doGet(final HttpServletRequest request, final HttpServletResponse response) throws IOException, ServletException {
AsyncContext ac = request.startAsync(request, response);
HashMap<String, AsyncContext> store = AppContext.getInstance().getStore();
store.put(request.getParameter("id"), ac);
}
}
And a thread to write to async context.
class MyThread extends Thread {
String id, message;
public MyThread(String id, String message) {
this.id = id;
this.message = message;
}
public void run() {
HashMap<String, AsyncContext> store = AppContext.getInstance().getStore();
AsyncContext ac = store.get(id);
try {
ac.getResponse().getWriter().print(message);
} catch (IOException e) {
e.printStackTrace();
}
}
}
But when I make the request, data is sent only if I call ac.complete(). Without it request will always timeout. So basically I want to have data "streamed" before request is completed.
Just to make a note, I have tried this with Jetty 8 Continuation API, I also tried with printing to OutputStream instead of PrintWriter. I also tried flushBuffer() on response. Same thing.
What am I doing wrong?
Client side is done like this:
var xhr = new XMLHttpRequest();
xhr.open('GET', 'http://localhost:8080/home', true);
xhr.onreadystatechange = function () {
if (xhr.readyState == 3 || xhr.readyState == 4) {
document.getElementById("dynamicContent").innerHTML = xhr.responseText;
}
}
xhr.send(null);
Can someone at least confirm that server side is okay? :)
Your server-side and client-side code is indeed ok.
The problem is actually with your browser buffering text/plain responses from your web-server.
This is the reason you dont see this issue when you use curl.
I took your client-side code and I was able to see incremental responses, with only just one little change:
response.setContentType("text/html");
The incremental responses showed up immediately regardless of their size.
Without that setting, when my output was a small message, it was considered as text/plain and wasnt showing up at the client immediately. When I kept adding more and more to the client responses, it got accumulated until the buffer size reached about 1024 bytes and then the whole thing showed up on the client side. After that point, however, the small increments showed up immediately (no more accumulation).
I know this is a bit old, but you can just flushBuffer on the response as well.