I am trying to get accustomed to rxjava and I am trying to call the below QuoteReader in an Observable. I am not sure how to handle the exception thrown,
public class QuoteReader {
public Map<String, Object> getQuote() throws IOException{
OkHttpClient okHttpClient = new OkHttpClient();
Request request = new Request.Builder().url("http://quotes.rest/qod.json").build();
Gson gson = new Gson();
Map<String, Object> responseMap = null;
try(Response response = okHttpClient.newCall(request).execute()) {
responseMap = gson.fromJson(response.body().string(), Map.class);
System.out.println("response map : "+responseMap);
} catch(IOException ioe) {
ioe.printStackTrace();
throw ioe;
} finally {
okHttpClient = null;
request = null;
}
return responseMap;
}
}
The following is the rx code I am trying to write,
rx.Observable.just(new QuoteReader().getQuote()) //compile time error saying unhandled exception
.subscribe(System.out::println);
How should I update the code to handle the exception. Thanks!
Use fromCallable that allows your method to throw (plus, it gets evaluated lazily and not before you even get into the Observable world):
rx.Observable.fromCallable(() -> new QuoteReader().getQuote())
.subscribe(System.out::println, Throwable::printStackTrace);
There is another factory method for an Observable, which is called create. This gives you an Observer as input of a lambda expression. Observer is a kind of callback object with three methods: onNext(T t), onError(Throwable e) and onCompleted. I read you're new to RxJava, so here is a little extra as a sidenote: the Rx contract specifies that an Observer can receive zero or more onNext calls, followed by maybe either an onError or onCompleted call. In regular expression for this looks like: onNext* (onError|onCompleted)?.
Now that you know this, you can implement your operation using Observable.create:
Observable.create(observer -> {
try {
observer.onNext(new QuoteReader.getQuote());
observer.onCompleted();
}
catch (Throwable e) {
observer.onError(e);
}
}
Notice that this code doesn't do anything until you subscribe to this Observable as you already did in your question!
Observable pipeline does not allow you throw Exceptions. You must use runtimeExceptions. So changing your code it should looks like.
try(Response response = okHttpClient.newCall(request).execute()) {
responseMap = gson.fromJson(response.body().string(), Map.class);
System.out.println("response map : "+responseMap);
} catch(IOException ioe) {
ioe.printStackTrace();
new RuntimeException(ioe);
You can see a practical example here https://github.com/politrons/reactive/blob/master/src/test/java/rx/observables/errors/ObservableExceptions.java
Related
I have a specific code which is working fine if webclient response is OK. If any error, then the get() method throwing error and the thread blocked forever.
#SneakyThrows
public List<ResponseData> validateExpression(List<RequestData> RequestDataList, Data data) {
System.out.println(Instant.now());
final List<Mono<ResponseData>> monoList = new ArrayList<>();
RequestDataList.parallelStream().forEach(requestData -> {
try {
ObjectMapper mapper = new ObjectMapper();
log.info("Diversity API request data:");
log.info(mapper.writeValueAsString(requestData));
Mono<ResponseData> monoResponse = webClient
.post()
.uri("http://...")
.contentType(MediaType.APPLICATION_JSON)
.header(API_KEY_HEADER, config.getApiKey())
.body(Mono.just(requestData), RequestData.class)
.retrieve()
.bodyToMono(ResponseData.class);
System.out.println("create mono response lazy initialization");
monoList.add(monoResponse);
} catch (Exception e) {
log.info(e.getMessage());
}
});
System.out.println(Instant.now());
CompletableFuture<List<ResponseData>> futureCount = new CompletableFuture<>();
List<ResponseData> responseDataList = new ArrayList<>();
Mono.zip(monoList, Arrays::asList)
.flatMapIterable(objects -> objects)
.doOnComplete(() -> {
futureCount.complete(responseDataList);
}).subscribe(responseData -> {
responseDataList.add((ResponseData) responseData);
});
return futureCount.get();
}
It is working fine with successful case. If there is any error from the webclient it is throwing error and thread blocked forever.
How to skip the errors and get only validate response data ?
How to avoid deadLock on this case?
You should look at CompletableFuture::get method's doc:
It throws three different checked exceptions. Lombok's #SneakyThrows annotation hides them so they aren't managed by your method. You should probably add a try/catch block to manage these exceptions and skip the errors if you want so.
I have a controller advice that handles the exceptional behavior in my REST controller and I came across a situation when I have to conditionally process SQLIntegrityConstraintViolationException that have a certain message (the one for duplicate keys), returning a 409, letting the other ones be handled by the default handler (returning a 500 error code).
I am thinking of 2 possible ways to achieve that:
Throwing a new bare-boned Exception on the else branch on my condition, so the handling is done by Spring.
Explicitly calling the general exception handler (like return handleGeneralException(exception) from inside my else branch).
I there a "proper" way to pass on a fraction of exceptions of one kind in my ControllerAdvice to another handler than the "original" one?
EDIT 1:
I would like to do something like this in my ControllerAdvice:
if (exception.getMessage.contains("something")) {
// handle exception
} else {
// pass to other handler
}
Have a custom exception class and then when you throw the SQLIntegrityConstraintViolationException wrap it in your custom exception classs with additional fields whatever you want to be accessible in controller advice. Handle the custom exception in your controller advice class.
#ControllerAdvice
public class CustomExceptionHandler {
#ExceptionHandler(YourCustomException.class)
public final ResponseEntity<ExceptionResponse> handleNotFoundException(YourCustomExceptionex,
WebRequest request) {
ExceptionResponse exceptionResponse = new ExceptionResponse(new Date(), ex.getMessage(),
request.getDescription(false), HttpStatus.NOT_ACCEPTABLE.getReasonPhrase());
return new ResponseEntity<>(exceptionResponse, HttpStatus.CONFLICT);
}
}
While having the try catch block to handle this exception in your code , make sure that you handle DataIntegrityViolationException instead of SQLIntegrityConstraintViolationException if you are using Spring Data JPA. So , if you are using Spring Data Jpa then :
try {
anyRepository.save(new YourModel(..));
} catch (DataIntegrityViolationException e) {
System.out.println("history already exist");in res
throw New YourCustomException("additional msg if you need it ", e);
}
Below code will capture the error message of exception SQLIntegrityConstraintViolationException in ControllerAdbvice without having to handle in code
#ControllerAdvice
public class CustomGlobalExceptionHandler extends ResponseEntityExceptionHandler {
#ExceptionHandler(value = DataIntegrityViolationException.class)
public ResponseEntity<ExceptionResponse> dataIntegrityViolationExceptionHandler(Exception ex) {
ExceptionResponse response = new ExceptionResponse();
Throwable throwable = ex.getCause();
while (throwable != null) {
if (throwable instanceof SQLIntegrityConstraintViolationException) {
String errorMessage = throwable.getMessage();
response.setErrors(new ArrayList<>(Arrays.asList(errorMessage)));
}
throwable = throwable.getCause();
}
return new ResponseEntity<Object>(response, HttpStatus.CONFLICT);
}
}
We're using org.springframework.web.reactive.function.client.WebClient with
reactor.netty.http.client.HttpClient as part of Spring 5.1.9 to make requests using the exchange() method. The documentation for this method highlights the following:
... when using exchange(), it is the responsibility of the application
to consume any response content regardless of the scenario (success,
error, unexpected data, etc). Not doing so can cause a memory leak.
Our use of exchange() is rather basic, but the documentation for error scenarios is unclear to me and I want to be certain that we are correctly releasing resources for all outcomes. In essence, we have a blocking implementation which makes a request and returns the ResponseEntity regardless of the response code:
try {
...
ClientResponse resp = client.method(method).uri(uri).syncBody(body).exchange().block();
ResponseEntity<String> entity = resp.toEntity(String.class).block();
return entity;
} catch (Exception e) {
// log error details, return internal server error
}
If I understand the implementation, exchange() will always give us a response if the request was successfully dispatched, regardless of response code (e.g. 4xx, 5xx). In that scenario, we just need to invoke toEntity() to consume the response. My concern is for error scenarios (e.g. no response, low-level connection errors, etc). Will the above exception handling catch all other scenarios and will any of them have a response that needs to be consumed?
Note: ClientResponse.releaseBody() was only introduced in 5.2
The response have to be consumed when the request was made, but if you can't do the request probably an exception was be throwed before, and you will no have problems with response.
In the documentation says:
NOTE: When using a ClientResponse through the WebClient exchange() method, you have to make sure that the body is consumed or released by using one of the following methods:
body(BodyExtractor)
bodyToMono(Class) or bodyToMono(ParameterizedTypeReference)
bodyToFlux(Class) or bodyToFlux(ParameterizedTypeReference)
toEntity(Class) or toEntity(ParameterizedTypeReference)
toEntityList(Class) or toEntityList(ParameterizedTypeReference)
toBodilessEntity()
releaseBody()
You can also use bodyToMono(Void.class) if no response content is expected. However keep in mind the connection will be closed, instead of being placed back in the pool, if any content does arrive. This is in contrast to releaseBody() which does consume the full body and releases any content received.
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/reactive/function/client/ClientResponse.html
You can try to use .retrieve() instead .exchange() and handle errors as your preference.
public Mono<String> someMethod() {
return webClient.method(method)
.uri(uri)
.retrieve()
.onStatus(
(HttpStatus::isError), // or the code that you want
(it -> handleError(it.statusCode().getReasonPhrase())) //handling error request
)
.bodyToMono(String.class);
}
private Mono<? extends Throwable> handleError(String message) {
log.error(message);
return Mono.error(Exception::new);
}
In this example I used Exception but you can create some exception more specific and then use some exception handler to return the http status that you want.
Is not recommended to use block, a better way is pass the stream forward.
create some exception classes
Autowired ObjectMapper
Create a method that returns Throwable
Create a custom class for Error.
return webClient
.get()
.uri(endpoint)
.retrieve()
.bodyToMono(Model.class)
.onErrorMap(WebClientException.class, this::handleHttpClientException);
private Throwable handleHttpClientException(Throwable ex) {
if (!(ex instanceof WebClientResponseException)) {
LOG.warn("Got an unexpected error: {}, will rethrow it", ex.toString());
return ex;
}
WebClientResponseException wcre = (WebClientResponseException)ex;
switch (wcre.getStatusCode()) {
case NOT_FOUND -> throw new NotFoundException(getErrorMessage(wcre));
case BAD_REQUEST -> throw new BadRequestException(getErrorMessage(wcre));
default -> {
LOG.warn("Got a unexpected HTTP error: {}, will rethrow it", wcre.getStatusCode());
LOG.warn("Error body: {}", wcre.getResponseBodyAsString());
return ex;
}
}
}
private String getErrorMessage(WebClientResponseException ex) {
try {
return mapper.readValue(ex.getResponseBodyAsString(), HttpErrorInfo.class).getMessage();
} catch (IOException ioe) {
return ex.getMessage();
}
}
I need to make a library in which I will have synchronous and asynchronous feature.
executeSynchronous() - waits until I have a result, returns the result.
executeAsynchronous() - returns a Future immediately which can be processed after other things are done, if needed.
Core Logic of my Library
The customer will use our library and they will call it by passing DataKey builder object. We will then construct a URL by using that DataKey object and make a HTTP client call to that URL by executing it and after we get the response back as a JSON String, we will send that JSON String back to our customer as it is by creating DataResponse object. Some customer will call executeSynchronous() and some might call executeAsynchronous() so that's why I need to provide two method separately in my library.
Interface:
public interface Client {
// for synchronous
public DataResponse executeSynchronous(DataKey key);
// for asynchronous
public Future<DataResponse> executeAsynchronous(DataKey key);
}
And then I have my DataClient which implements the above Client interface:
public class DataClient implements Client {
private RestTemplate restTemplate = new RestTemplate();
private ExecutorService executor = Executors.newFixedThreadPool(10);
// for synchronous call
#Override
public DataResponse executeSynchronous(DataKey key) {
DataResponse dataResponse = null;
Future<DataResponse> future = null;
try {
future = executeAsynchronous(key);
dataResponse = future.get(key.getTimeout(), TimeUnit.MILLISECONDS);
} catch (TimeoutException ex) {
PotoLogging.logErrors(ex, DataErrorEnum.TIMEOUT_ON_CLIENT, key);
dataResponse = new DataResponse(null, DataErrorEnum.TIMEOUT_ON_CLIENT, DataStatusEnum.ERROR);
// does this looks right?
future.cancel(true); // terminating tasks that have timed out
} catch (Exception ex) {
PotoLogging.logErrors(ex, DataErrorEnum.CLIENT_ERROR, key);
dataResponse = new DataResponse(null, DataErrorEnum.CLIENT_ERROR, DataStatusEnum.ERROR);
}
return dataResponse;
}
//for asynchronous call
#Override
public Future<DataResponse> executeAsynchronous(DataKey key) {
Future<DataResponse> future = null;
try {
Task task = new Task(key, restTemplate);
future = executor.submit(task);
} catch (Exception ex) {
PotoLogging.logErrors(ex, DataErrorEnum.CLIENT_ERROR, key);
}
return future;
}
}
Simple class which will perform the actual task:
public class Task implements Callable<DataResponse> {
private DataKey key;
private RestTemplate restTemplate;
public Task(DataKey key, RestTemplate restTemplate) {
this.key = key;
this.restTemplate = restTemplate;
}
#Override
public DataResponse call() {
DataResponse dataResponse = null;
String response = null;
try {
String url = createURL();
response = restTemplate.getForObject(url, String.class);
// it is a successful response
dataResponse = new DataResponse(response, DataErrorEnum.NONE, DataStatusEnum.SUCCESS);
} catch (RestClientException ex) {
PotoLogging.logErrors(ex, DataErrorEnum.SERVER_DOWN, key);
dataResponse = new DataResponse(null, DataErrorEnum.SERVER_DOWN, DataStatusEnum.ERROR);
} catch (Exception ex) {
PotoLogging.logErrors(ex, DataErrorEnum.CLIENT_ERROR, key);
dataResponse = new DataResponse(null, DataErrorEnum.CLIENT_ERROR, DataStatusEnum.ERROR);
}
return dataResponse;
}
// create a URL by using key object
private String createURL() {
String url = somecode;
return url;
}
}
When I started working on this solution, I was not terminating the tasks that have timed out. I was reporting the timeout to the client, but the task continues to run in the thread pool (potentially occupying one of my limited 10 threads for a long time). So I did some research online and I found out that I can cancel my tasks those have timed out by using cancel on future as shown below -
future.cancel(true);
But if I am doing like this as shown in my above solution, then do I need to close any other resources like RestTemplate as soon as thread is interrupted? If yes, then how would I do that? Also, can we interrupt RestTemplate calls? Since I tried calling cancel on my future as soon as the task got timed out but I guess my thread doesn't got interrupted.
Should we always be terminating the tasks that have got timed out? If we don't do that then what might be the impact I will have? Will it affect my performance?
Is there any better solution to deal with this case with my current setup?
It appears that a call to a RestTemplate cannot be interrupted or canceled. Even if the "kludge" using a callback is utilized, the RestTemplate might have resources locked up internally, waiting for the response before invoking the callback.
When the underlying socket is accessible, network I/O can be aborted by closing the socket from another thread. For example, a timer can be started to close the socket after a timeout elapses. Or, if you want an indefinite timeout that is sensitive to interrupts (due to a user pressing a "Cancel" button, for example), you can submit a task that waits indefinitely but responds to interrupts by closing the socket.
Unfortunately, it doesn't look like the authors of RestTemplate provided this capability.
Yes, you should clean up resources that are no longer needed because of task cancellation or expiration. Yes, it will affect performance. If your thread pool has a limited number of threads, eventually all will be stuck in defunct tasks. If it has an unlimited number of threads, eventually memory will become exhausted.
Sometimes it is not possible to interrupt thread especially when thread performs blocking operations on Socket.
So instead of cancelling the task when it timeouts, you should rather set timeouts on http connection.
Unfortunately timeousts are set per Connection Factory and RestTemplate, thus each request must use it's own RestTemplate.
You can create new RestTemplate per task, or reuse previusly created templates using ThreadLocal or resource pooling.
For example the task using Thread local might look like below:
public class Task implements Callable<DataResponse> {
private DataKey key;
private ThreadLocal<RestTemplate> restTemplateThreadLocal =
ThreadLocal.withInitial(()->new RestTemplate(new SimpleClientHttpRequestFactory()));
public Task(DataKey key) {
this.key = key;
}
private SimpleClientHttpRequestFactory getConnectionFactory(){
return (SimpleClientHttpRequestFactory)restTemplateThreadLocal.get().getRequestFactory();
}
#Override
public DataResponse call() {
DataResponse dataResponse = null;
String response = null;
try {
String url = createURL();
//it is up to you, how to set connection and read timeouts from provided key.getTimeout
getConnectionFactory().setConnectTimeout(1000);
getConnectionFactory().setReadTimeout(key.getTimeout());
response = restTemplateThreadLocal.get().getForObject(url, String.class);
// it is a successful response
dataResponse = new DataResponse(response, DataErrorEnum.NONE, DataStatusEnum.SUCCESS);
} catch (RestClientException ex) {
PotoLogging.logErrors(ex, DataErrorEnum.SERVER_DOWN, key);
dataResponse = new DataResponse(null, DataErrorEnum.SERVER_DOWN, DataStatusEnum.ERROR);
} catch (Exception ex) {
PotoLogging.logErrors(ex, DataErrorEnum.CLIENT_ERROR, key);
dataResponse = new DataResponse(null, DataErrorEnum.CLIENT_ERROR, DataStatusEnum.ERROR);
}
return dataResponse;
}
// create a URL by using key object
private String createURL() {
String url = somecode;
return url;
}
}
BTW.
Spring also provides AsyncRestTemplate, which may make your code simpler.
If used with Netty4ClientHttpRequestFactory you can get NIO based client connections. In such case, you should be able to interrupt your tasks even while it makes Http connection.
Short sample below. It uses NIO thus you does not have to care if the request is really cancelled after Timeout.
URI url = new URI("http://www.chicagotribune.com/news/ct-college-of-dupage-investigation-met-20150330-story.html");
Netty4ClientHttpRequestFactory asyncRequestFactory = new Netty4ClientHttpRequestFactory();
AsyncRestTemplate asyncRestTemplate = new AsyncRestTemplate(asyncRequestFactory);
ListenableFuture<ResponseEntity<String>> entity = asyncRestTemplate.getForEntity(url, String.class);
System.out.println("entity.get() = " + entity.get());
asyncRequestFactory.destroy();
I have a high volume java application in which I have to send http posts to another server.
Currently I'm using org.apache.commons.httpclient library:
private static void sendData(String data) {
HttpClient httpclient = new HttpClient();
StringRequestEntity requestEntity;
try {
requestEntity = new StringRequestEntity(data, "application/json", "UTF-8");
String address = "http://<my host>/events/"
PostMethod postMethod = new PostMethod(address);
postMethod.setRequestEntity(requestEntity);
httpclient.executeMethod(postMethod);
} catch (Exception e) {
LOG.error("Failed to send data ", e);
}
}
This means I'm sending my http requests synchronously, which doesn't fit my multithreaded high volume app. So I would like to change those calls to asynchronous non-blocking http calls.
I was going through number of options such as apache async client and xsocket but was not able to make it work.
Tried ning:
private static void sendEventToGrpahiteAsync(String event) {
LOG.info("\n" + "sendEventToGrpahiteAsync");
try (AsyncHttpClient asyncHttpClient = new AsyncHttpClient()) {
BoundRequestBuilder post = asyncHttpClient.preparePost();
post.addHeader("Content-Type", "application/json");
post.setBodyEncoding("UTF-8");
post.setBody(event);
post.execute(new HttpRequestCompletionHandler());
} catch (Exception e) {
LOG.error("Failed to sending event", e);
}
}
I tried Apache HttpAsyncClient:
private static void sendEventToGrpahiteAsync(String event) {
LOG.info("\n" + "sendEventToGrpahiteAsync");
try (CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault()) {
httpclient.start();
HttpPost request = new HttpPost(addr);
StringEntity entity = new StringEntity(event, ContentType.create("application/json", Consts.UTF_8));
request.setEntity(entity);
httpclient.execute(request, null);
} catch (Exception e) {
LOG.error("Failed to sending event", e);
}
}
I tried xsocket:
private static void sendEventToGrpahiteAsync2(String event) {
LOG.info("\n" + "sendEventToGrpahiteAsync");
try (INonBlockingConnection con = new NonBlockingConnection(<SERVER_IP>, 80);
IHttpClientEndpoint httpClientConnection = new HttpClientConnection(con)) {
IHttpResponseHandler responseHandler = new MyResponseHandler();
IHttpRequest request = new PostRequest(url_address, "application/json", Consts.UTF_8.toString(), event);
request.setTransferEncoding(Consts.UTF_8.toString());
httpClientConnection.send(request, responseHandler);
} catch (Exception e) {
LOG.error("Failed to sending event", e);
}
}
I get no exceptions but the post doesn't get to the target as well.
To be clear, the target is a graphite server so once a post arrives it is clearly seen in a graph. The synchronous posts works well, I can see the result on the graph, but none of the asynchronous posts shows on my destination graph.
What am I missing?
Thanks
Got it.
All the libraries I'n using are implemented using an extra IO thread, so my process probably ends before a full handshake.
Once I added Thread.sleep(2000) after the http calls things worked just fine.
So for a web app (which is my case) my suggested implementations are just fine (but for a java process you might consider NickJ's answer).
You could use the Java Executor framework:
First, create a Callable to do your work:
public class MyCallable implements Callable<MyResult> {
#Override
public MyResult call() throws Exception {
//do stuff
return result;
}
}
Get an Exectutor which will run your Callable. There are various way to get one, here's one example:
ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
Finally, run it:
MyCallable callable = new MyCallable();
Future<MyResult> futureResult = executor.submit(callable);
Getting the result:
boolean resultReady = futureResult.isDone(); //is the result ready yet?
Result r = futureResult.get(); //wait for result and return it
try {
Result r = futureResult.get(10, TimeUnit.SECONDS); //wait max. 10 seconds for result
} catch (TimeOutException e) {
//result still not ready after waiting 10 seconds
}