I am new to asynchronus coding in Java.
This is my code in spring boot application:
public void func(String someString){
if(someString != null){
doAsync(someString);
publish(topic, someString);
}
}
public void doAsync(String someString) {
log.info("Inside do Async");
CompletableFuture.runAsync(() -> {
try {
execute(someString);
} catch (Throwable throwable) {
log.error("Error");
}
}, executorService);
}
private void execute(String someString) {
try {
log.info("Inside Execute");
DBcall(someString);
} catch (Throwable e) {
log.info("Error");
}
}
The func() is being called around 200k times through an event in a Queue, In logs the log Inside do Async appeared for 200k times but the log Inside Execute appeared just for 195k times.
I see no errors/exception occurred in this flow.
Why is it not running consistently for all 200k events? Am I missing something in the implementation?
The publish() function publishes the same message to another subscriber which is in the same service, there are some 10k-11k errors in that subscriber flow(null pointer error). Is error in this flow is main reason behind not executing all async call?
Related
I'm new to vert.x and would like to know if its possible to configure eventbus somehow to make it work consistently?
I mean need to send requests one by one using vert.x
At the moment I got this code which uses eventloop principle and waits until all handlers finished, but I don't need this done that fast, idea is to free server from lots of requests at the same time. Here eb_send() uses default EventBus.send() method. In other words I want to execute all requests with blocking, waiting for answers before requests.
List<Future> queue = new ArrayList<>();
files.forEach(fileObj -> {
Future<JsonObject> trashStatusHandler = Future.future();
queue.add(trashStatusHandler);
eb_send(segment, StorageType.getAddress(StorageType.getStorageType(fileInfo.getString("storageType"))) + ".getTrashStatus", fileInfo, reply -> {
Entity dummy = createDummySegment();
try {
if (reply.succeeded()) {
//succeded
}
} catch (Exception ex) {
log.error(ex);
}
trashStatusHandler.complete();
});
});
The basic idea is to extract this into a function, which you would invoke recursively.
public void sendFile(List<File> files, AtomicInteger c) {
eb_send(segment, StorageType.getAddress(StorageType.getStorageType(fileInfo.getString("storageType"))) + ".getTrashStatus", fileInfo, reply -> {
Entity dummy = createDummySegment();
try {
if (reply.succeeded()) {
//succeded
}
// Recursion
if (c.incrementAndGet() < files.size()) {
sendFile(files, c);
}
} catch (Exception ex) {
log.error(ex);
}
});
}
With the help of this link, I've managed to create a small Java application that pulls published messages for one minute. My implementation looks something like this.
public static void eventListener() throws InterruptedException {
MessageReceiver receiver = new MessageReceiver() {
#Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
System.out.println("Received message: " + message.getData().toStringUtf8());
consumer.ack();
}
};
//Subscriber subscriber = null;
try {
subscriber = Subscriber.newBuilder(subscription, receiver)
.setCredentialsProvider(FixedCredentialsProvider.create(creds)).build();
subscriber.addListener(new Subscriber.Listener() {
#Override
public void failed(Subscriber.State from, Throwable failure) {
// Handle failure. This is called when the Subscriber encountered a fatal error
// and is
// shutting down.
System.err.println(failure);
}
}, MoreExecutors.directExecutor());
subscriber.startAsync().awaitRunning();
// In this example, we will pull messages for one minute (60,000ms) then stop.
// In a real application, this sleep-then-stop is not necessary.
// Simply call stopAsync().awaitTerminated() when the server is shutting down,
// etc.
Thread.sleep(60000);
} finally {
if (subscriber != null) {
subscriber.stopAsync().awaitTerminated();
}
}
}
When I call this method in main
public static void main(String[] args) throws InterruptedException {
eventListener();
}
and upload an object to my Google cloud storage, the program prints a message sent by the publisher, like this
Received message: {
"kind": "storage#object",
"id": "roshanbucket/stones.jpg/1553765105996166",
"selfLink": "https://www.googleapis.com/storage/v1/b/roshanbucket/o/stones.jpg",
"name": "stones.jpg",
"bucket": "roshanbucket",
"generation": "1553765105996166",
"metageneration": "1",
"contentType": "image/jpeg",
"timeCreated": "2019-03-28T09:25:05.995Z",
"updated": "2019-03-28T09:25:05.995Z",
"storageClass": "STANDARD",
"timeStorageClassUpdated": "2019-03-28T09:25:05.995Z",
"size": "137256",
"md5Hash": "1GmpUnGeiW+/KU+0U8c8Wg==",
"mediaLink": "https://www.googleapis.com/download/storage/v1/b/roshanbucket/o/stones.jpg?generation=1553765105996166&alt=media",
"crc32c": "FMaEGg==",
"etag": "CIaj1InCpOECEAE="
}
For one minute since the program execution, it prints all the messages received on the account of the object upload, then it stops. To receive the event messages after a minute, I need to restart the application. Now, what I'd like to do is to run the listener continuously, So, I tried to run the method eventListener() inside an infinite loop inside the main method, like this
public static void main(String[] args) throws InterruptedException {
while(true) {
eventListener();
}
}
With this, I seem to be able to receive the event message right after every upload, regardless of when I upload the object. But then, every once in a while, it throws this stack trace.
Mar 28, 2019 12:56:34 PM io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=6, target=pubsub.googleapis.com:443} was not shutdown properly!!! ~*~*~*
Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
java.lang.RuntimeException: ManagedChannel allocation site
at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:103)
at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:53)
at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:44)
at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:440)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:223)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:164)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:156)
at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:157)
at com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:260)
at com.google.cloud.pubsub.v1.Subscriber.doStart(Subscriber.java:268)
at com.google.api.core.AbstractApiService$InnerService.doStart(AbstractApiService.java:148)
at com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:225)
at com.google.api.core.AbstractApiService.startAsync(AbstractApiService.java:120)
at com.google.cloud.pubsub.v1.Subscriber.startAsync(Subscriber.java:260)
at listener.AsynchronousPull.eventListener(AsynchronousPull.java:57)
at listener.AsynchronousPull.main(AsynchronousPull.java:74)
But, it still prints the message after every upload, while throwing the stack trace every now and then. I don't have much experience with threads and I'd really appreciate some help with fixing this issue.
Calling eventListener() in a tight loop is not what you want to do here. This is going to create many new instances of a subscriber that receive messages that each live for 60 seconds. What you want is for the single instance of subscriber you create to live until a time at which you want to shut it down. Generally, you would do this by creating the subscriber and awaiting its termination via awaitTerminated().
The code above would be altered to be like this:
public static void eventListener() throws InterruptedException {
MessageReceiver receiver = new MessageReceiver() {
#Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
System.out.println("Received message: " + message.getData().toStringUtf8());
consumer.ack();
}
};
Subscriber subscriber = null;
try {
subscriber = Subscriber.newBuilder(subscription, receiver)
.setCredentialsProvider(FixedCredentialsProvider.create(creds)).build();
subscriber.addListener(new Subscriber.Listener() {
#Override
public void failed(Subscriber.State from, Throwable failure) {
// Handle failure. This is called when the Subscriber encountered a fatal error
// and is
// shutting down.
System.err.println(failure);
}
}, MoreExecutors.directExecutor());
subscriber.startAsync().awaitRunning();
subscriber.awaitTerminated();
} finally {
if (subscriber != null) {
subscriber.stopAsync().awaitTerminated();
}
}
}
public static void main(String[] args) throws InterruptedException {
eventListener();
}
If you just want the subscriber to stop when the application terminates and don't want to bother with any additional cleanup, then the code above will work, allowing the subscriber to run and receive messages until an error occurs or the application is shut down. If you want to do some cleanup on clean termination of the application, e.g., you want to ensure that any messages already being handled by receiveMessage run to completion, then you can attach a shutdown hook to catch such terminations (though it will not run in all circumstances). In this hook, you would call stopAsync(). For example, you could insert the following before the try block:
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
subscriber.stopAsync().awaitTerminated();
}
});
Im using Akka 2.5.6 in Java 8 and I want to know the right way to finish de ActorSystem, part of the functionality of my code is to process some XML files and validate them, to achieve this I have created 3 actors:
Controller, Processor and Validator.
The Controller is responsible for initiating the process and sending file by file and other information to the Processor, then the Processor create a digital signature of the file and sends the response to the Validator that finally validates the status and sends an OK message to the Controller which is counting the number of files validated and compares them with the total files. Once the total of files with the total of validated files are equal, I call to finish the ActorSystem with the terminate () method.
The method to finish is as follows:
private void endActors()
{
ActorSystem actorSystem = getContext().system();
Future <Terminated> terminated = actorSystem.terminate();
do {
log.info ("Waiting to finish ...");
try {
Thread.sleep (30000L);
} catch (InterruptedException ex) {
log.error ("Error in Thread.");
}
} while (! ended.isCompleted ());
log.info ("Actors finished processing.");
}
The loop never ends because the future is never complete, I dont know if this is the right way, I hope you have understood me and can help me or give me some advice.
Try the following (the key here is the on complete) . I wrote a class along these lines to use in a setup and teardown for junit, to avoid issues from actor system not fully terminating in the teardown of one test before being created in another test. (that caused port already in use issues)
private static ActorSystem system = null;
private static Future<Terminated> terminatedFuture;
public static ActorSystem getFreshActorSystem() {
tearDownActorSystem();
while(system != null) {
try {
Thread.sleep(500L);
} catch (InterruptedException e) {
}
}
system = ActorSystem.create();
return system;
}
public static void tearDownActorSystem() {
if (system !=null && !isInMiddleOfTerminating()) {
terminatedFuture = system.terminate();
terminatedFuture.onComplete( new OnComplete(){
#Override
public void onComplete(Throwable failure, Object success) throws Throwable {
system = null;
terminatedFuture = null;
}
} , system.dispatcher());
}
}
private static boolean isInMiddleOfTerminating() {
return terminatedFuture !=null;
}
We have a data processing application that runs on Karaf 2.4.3 with Camel 2.15.3.
In this application, we have a bunch of routes that import data. We have a management view that lists these routes and where each route can be started. Those routes do not directly import data, but call other routes (some of them in other bundles, called via direct-vm), sometimes directly and sometimes in a splitter.
Is there a way to also completely stop a route/therefore stopping the entire exchange from being further processed?
When simply using the stopRoute function like this:
route.getRouteContext().getCamelContext().stopRoute(route.getId());
I eventually get a success message with Graceful shutdown of 1 routes completed in 10 seconds - the exchange is still being processed though...
So I tried to mimic the behaviour of the StopProcessor by setting the stop property, but that also didn't help:
public void stopRoute(Route route) {
try {
Collection<InflightExchange> browse = route.getRouteContext().getCamelContext().getInflightRepository()
.browse();
for (InflightExchange inflightExchange : browse) {
String exchangeRouteId = inflightExchange.getRouteId();
if ((exchangeRouteId != null) && exchangeRouteId.equals(route.getId())) {
this.stopExchange(inflightExchange.getExchange());
}
}
} catch (Exception e) {
Notification.show("Error while trying to stop route", Type.ERROR_MESSAGE);
LOGGER.error(e, e);
}
}
public void stopExchange(Exchange exchange) throws Exception {
AsyncProcessorHelper.process(new AsyncProcessor() {
#Override
public void process(Exchange exchange) throws Exception {
AsyncProcessorHelper.process(this, exchange);
}
#Override
public boolean process(Exchange exchange, AsyncCallback callback) {
exchange.setProperty(Exchange.ROUTE_STOP, Boolean.TRUE);
callback.done(true);
return true;
}
}, exchange);
}
Is there any way to completely stop an exchange from being processed from outside the route?
Can you get an exchange?
I use exchange.setProperty(Exchange.ROUTE_STOP, true);
Route stop flow and doesn't go to next route.
I execute my async function and before the result I reload the browser I get error - OnFailure(Throwable) is executed. Status error code is 0.
This problem is on FireFox and Chrome.
Could you tell me what this status code means.
do(new AsyncCallback<Boolean>() {
#Override
public void onSuccess(Boolean result) {}
#Override
public void onFailure(Throwable throw) {
do_sth();
}
});
Boolean do() { while(true); }
That also return status error 0
The 0 status code here means the request has been aborted (it could also denote a network error, or the request timed out).
See http://www.w3.org/TR/XMLHttpRequest/#the-status-attribute and http://www.w3.org/TR/XMLHttpRequest/#error-flag
You could always define your onFailure() like this (adapted from the GWT API) to be able to handle different kinds of failure nicely:
public void onFailure(Throwable caught) {
try {
throw caught;
} catch (IncompatibleRemoteServiceException e) {
// this client is not compatible with the server; cleanup and refresh the
// browser
} catch (InvocationException e) {
// the call didn't complete cleanly
} catch (YourOwnException e) {
// take appropriate action
} catch (Throwable e) {
// last resort -- a very unexpected exception
}
}