We Keep Coding

AWS SWF - Workflow passing activity results with signals driving the workflow

I have the requirement to use AWS Simple Workflow (SWF) for an orchestration type of system design. There is parent application that is start this child workflow then signal the workflow to work on activities. I have a workflow that starts up and waits for signals to happen before it can start doing activity work. Once one activity is done then it will report back to by closing out the activity on the parent workflow.
How do I wait for the signal and also use the results from another activity that was invoked by a signal?
Do I need to look into the execution history for the result of an activity and not rely on doing this work in the decide?
Thanks for the help
Code Example:
#SuppressWarnings("unused")
#Slf4j
public class ChildWorkflowImpl implements ChildWorkflow {
private final Settable<Message> firstStepReceived = new Settable<>();
private final Settable<Message> secondStepReceived = new Settable<>();
#Autowired
private FirstActivityClient firstActivityClient;
#Autowired
private SecondActivityClient secondActivityClient;
#Autowired
private AmazonSimpleWorkflow amazonSimpleWorkflow;
#Override
public void startWorkflow(SubsystemMessage subsystemMessage) {
//wait for signal to start
new Task(firstStepReceived) {
#Override
protected void doExecute() throws Throwable {
//Initiate Activity
startStage(firstStepReceived.get(););
}
};
//wait for second signal but pass in data from first activity
new Task(secondStepReceived) {
#Override
protected void doExecute() throws Throwable {
}
};
}
public void firstStep(Message message) {
Promise<FirstActivityResponse> firstActivity = firstActivityClient.execute();
//wait for signal for disable
new Task(firstActivity) {
public void doExecute() {
//report back status for stage by closing parent activity
}
};
}
public void secondStep(FirstActivityResponse response) {
Promise<SecondActivityResponse> secondActivityResponse = secondActivityClient.execute(response);
new Task(secondActivityResponse) {
public void doExecute() {
//report back status for stage
}
};
}
}

You add a signal method to the workflow interface and use Settable to notify the other part of the workflow code about the signal. See Settable documentation from this documentation page.
BTW. I recommend looking at temporal.io which is a greatly improved version of SWF which supports synchronous programming without all these pesky tasks.

Notifying on all completed schedules with sprring

I have a Java, Spring application where I schedule some report jobs. The component looks like this:
#Component
public class RegisterReportSchedules implements ApplicationListener<ContextRefreshedEvent> {
#Autowired
private ThreadPoolTaskScheduler ts;
private List<String> reportSchedules; //contains list of report schedules
#Autowired
private SomeTask sometask;
#Override
public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
reportSchedules.forEach((String schedule) -> {
ReportSchedule reportSchedule = new ReportSchedule(schedule,
propertiesUtil.getProperty(schedule + "." + Constants.CRON));
ts.schedule(new ReportTask(reportSchedule),
new CronTrigger(reportSchedule.getCronExpression()));
});
}
class ReportTask implements Runnable {
private ReportSchedule schedule;
public ReportTask(ReportSchedule schedule) {
this.schedule = schedule;
}
#Override
public void run() {
sometask.process(schedule);
}
}
}
Say I have 5 reportSchedules to process. After all 5 ReportTasks have been completed, I need to write into a db table one entry to say all report task completed.
But how can I track this information about each report schedule is completed in my application?
Do I need to write to database table for each schedule completed or is there a better alternative within Spring that will trigger a notification event of some sort which I can then use to write the ALL COMPLETED event to the table? Appreciate if some answers with examples are given.

Since you don't need to track the reportSchedules, I'd be tempted to do something like:
Move to use a Queue so that the Strings are removed when you poll.
Track the number of tasks that you submit. (*)
Add a custom ApplicationEvent type of ReportScheduleProcessedEvent (or similar) and publish one of these (to Spring's ApplicationEventPublisher) at the end of the ReportTask's run method.
Add a new ApplicationListener for this type, which waits until it has received as many events as you tracked in (*); and then publishes something to the database.
I'm afraid I haven't provided any code here, since you might or might not need to care about thread-safety at a bunch of points above, and handling this appropriately might not be trivial.
Edit; per comment asking for a sample. I'll assume you're using at least spring 4.3.
Edit Edit: per more comments.
abstract class ReportScheduleEvent extends ApplicationEvent { ... }
public class IncomingReportCompletionEvent
extends ReportScheduleEvent {
private final int eventsToExpect;
// ...
}
public class ReportCompletionEvent extends ReportSchedulingEvent {
// ...
}
public class YourListener
implements ApplicationListener<ReportSchedulingEvent> {
private final DatabaseWriter dbWriter;
private volatile int expectedEvents = 0;
public void onApplicationEvent(final ReportSchedulingEvent event) {
if (event instanceof IncomingReportCompletionEvent) {
this.expectedEvents =
((IncomingReportCompletionEvent) event)
.getExpectedEventCount();
} else {
this.expectedEvents--;
if (this.expectedEvents == 0) {
this.dbWriter.doYourThing();
}
}
}
}

Get int value from method, which executes on background thread

I'm using Google Architecture Components, especially Room.
In my Dao i have this method:
#Query("SELECT COUNT(*) FROM photos")
int getPersistedPhotosSize();
And i need to execute it in my Repository to check if persisted photos size is 0.
So i gotta execute this method on background and get the value from it.
Now i perform this operation like this:
public int getNumRowsFromFeed() {
final int[] rows = new int[1];
Completable.fromAction(() -> rows[0] = photosDao.getPersistedPhotosSize())
.subscribeOn(Schedulers.io())
.blockingAwait();
return rows[0];
}
But i guess it's not the best way to do it.
So how can i get the value the right way? Especially i want to do it without RX

In your DAO the function to get the photo count doesn't use either LiveData nor RX. So instead of wrapping the code afterwards in a Completable, you can basically use any Android Async technology, like AsyncTask.
public class LoadTask extends AsyncTask<Void, Void, Integer> {
public interface Callback {
void onPhotoCount(int count);
}
private final Callback callback;
public LoadTask(Callback callback) {
this.callback = callback;
}
protected Integer doInBackground(Void... params) {
return photosDao.getPersistedPhotosSize();
}
protected void onPostExecute(Integer result) {
callback.onPhotoCount(result);
}
}
...
new LoadTask(photoCount -> {
// Do stuff with value,e.g. update ui.
}).execute();
This is basically just a proposal, of course you can use Threads, Handler as well.
P.S: From my point of view, this example shows one advantage of the Rx development. You get the callback stuff for free, without defining anything. And you can cancel the Rx chain for example due to lifecycle events. This is not implemented in this example.

Managing threads accessing a database with Java

I am working on an app that accesses an SQLite database. The problem is the DB gets locked when there is a query to it. Most of the time this is not a problem because the flow of the app is quite linear.
However I have a very long calculation process which is triggered by the user. This process involves multiple calls to the database in between calculations.
I wanted the user to get some visual feedback so I have been using Javafx progressIndicator and a Service from the Javafx.Concurrency framework.
The problem is this leaves the user free to move around the app and potentially triggering other calls to the database.
This caused an exception that the database file is locked.
I would like a way to stop that thread from running when this case happens however I have not been able to find any clear examples online. Most of them are oversimplified and I would like a way which is scalable. I've tried using the cancel() method but this does not guarantee that the thread will be cancelled in time.
Because I am not able to check in all parts of the code for isCancelled sometimes there is a delay between the time the thread is canceled and the time it effectively stops.
So I thought of the following solution but I would like to know if there is a better way in terms of efficiency and avoiding race conditions and hanging.
// Start service
final CalculatorService calculatorService = new CalculatorService();
// Register service with thread manager
threadManager.registerService(CalculatorService);
// Show the progress indicator only when the service is running
progressIndicator.visibleProperty().bind(calculatorService.runningProperty());
calculatorService.setOnSucceeded(new EventHandler<WorkerStateEvent>() {
#Override
public void handle(WorkerStateEvent workerStateEvent) {
System.out.println("SUCCEEDED");
calculatorService.setStopped(true);
}
});
// If something goes wrong display message
calculatorService.setOnFailed(new EventHandler<WorkerStateEvent>() {
#Override
public void handle(WorkerStateEvent workerStateEvent) {
System.out.println("FAILED");
calculatorService.setStopped(true);
}
});
// Restart the service
calculatorService.restart();
This is my service class which I have subclassed to include methods that can be used to set the state of the service (stopped or not stopped)
public class CalculatorService extends Service implements CustomService {
private AtomicBoolean stopped;
private CalculatorService serviceInstance;
public FindBundleService() {
stopped = new AtomicBoolean(false);
instance = this;
}
#Override
protected Task<Results> createTask() {
return new Task<Result>() {
#Override
protected Result call() throws Exception {
try {
Result = calculationMethod(this, serviceInstance);
return Result;
} catch (Exception ex) {
// If the thread is interrupted return
setStopped(true);
return null;
}
}
};
}
#Override
public boolean isStopped() {
return stopped.get();
}
#Override
public void setStopped(boolean stopped) {
this.stopped.set(stopped);
}
}
The service implements this interface which I defined
public interface CustomService {
/**
* Method to check if a service has been stopped
*
* #return
*/
public boolean isStopped();
/**
* Method to set a service as stopped
*
* #param stopped
*/
public void setStopped(boolean stopped);
}
All services must register themselves with the thread manager which is a singleton class.
public class ThreadManager {
private ArrayList<CustomService> services;
/**
* Constructor
*/
public ThreadManager() {
services = new ArrayList<CustomService>();
}
/**
* Method to cancel running services
*/
public boolean cancelServices() {
for(CustomService service : services) {
if(service.isRunning()) {
((Service) service).cancel();
while(!service.isStopped()) {
// Wait for it to stop
}
}
}
return true;
}
/**
* Method to register a service
*/
public void registerService(CustomService service) {
services.add(service);
}
/**
* Method to remove a service
*/
public void removeService(CustomService service) {
services.remove(service);
}
}
In any place in the app if we want to stop the service we call cancelServices(). This will set the state to cancelled I'm checking for this in my calculationMethod() then setting the state to stopped just before returning (effectively ending the thread).
if(task.isCancelled()) {
service.setStopped(true);
return null;
}

(I will assume you are using JDBC for your database queries and that you have control over the code running the queries)
I would centralize all database accesses in a singleton class which would keep the last PreparedStatement running the current query in a single thread ExecutorService. You could then ask that singleton instance things like isQueryRunning(), runQuery(), cancelQuery() that would be synchronized so you can decide to show a message to the user whenever the computation should be canceled, cancel it and start a new one.
Something like (add null checks and catch (SQLException e) blocks):
public class DB {
private Connection cnx;
private PreparedStatement lastQuery = null;
private ExecutorService exec = Executors.newSingleThreadExecutor(); // So you execute only one query at a time
public synchronized boolean isQueryRunning() {
return lastQuery != null;
}
public synchronized Future<ResultSet> runQuery(String query) {
// You might want to throw an Exception here if lastQuery is not null (i.e. a query is running)
lastQuery = cnx.preparedStatement(query);
return exec.submit(new Callable<ResultSet>() {
public ResultSet call() {
try {
return lastQuery.executeQuery();
} finally { // Close the statement after the query has finished and return it to null, synchronizing
synchronized (DB.this) {
lastQuery.close();
lastQuery = null;
}
}
}
// Or wrap the above Future<ResultSet> so that Future.cancel() will actually cancel the query
}
public synchronized void cancelQuery() {
lastQuery.cancel(); // I hope SQLite supports this
lastQuery.close();
lastQuery = null;
}
}

A solution to your problem could be Thead.stop(), which has been deprecated centuries ago (you can find more on the topic here).
To implement the similar behavior it is suggested to use the Thread.interrupt(), which is (in the context of Task) the same as the the Task.cancel().
Solutions:
Fill your calculationMethod with isCancelled() checks.
Try to interrupt an underling operation through an other Thread.
The second solution is probably what you are looking for, but it depends on the actual code of the calculationMethod (which I guess you can't share).
Generic examples for killing long database operations (all of this are performed from another thread):
Kill the connection to the Database (assuming that the Database is smart enough to kill the operation on disconnect and then unlock the database).
Ask for the Database to kill an operation (eg. kill <SPID>).
EDIT:
I hadn't see that that you specified the database to SQLite when I wrote my answer. So to specify the solutions for SQLite:
Killing the connection will not help
Look for the equivalent of sqlite3_interrupt in your java SQLite interface

Maybe you can invoke thread instance t1, t1.interrupt() method, then in the run method of thread( Maybe calculationMethod), add a conditional statement.
public void run() {
while (!Thread.currentThread().isInterrupted()) {
try {
// my code goes here
} catch (IOException ex) {
log.error(ex,ex)
}
}
}

With WAL mode (write-ahead logging) you can do many queries in parallel to the sqlite database
WAL provides more concurrency as readers do not block writers and a
writer does not block readers. Reading and writing can proceed
concurrently.
https://sqlite.org/wal.html
Perhaps these links are of interest to you:
https://stackoverflow.com/a/6654908/1989579
https://groups.google.com/forum/#!topic/sqlcipher/4pE_XAE14TY
https://stackoverflow.com/a/16205732/1989579

Updating JavaFX ProgressIndicator multiple times from a Thread

I am working on the design of a multi-threading app in Javafx and would like to have a TableView with columns for Name and Progress of each Thread. After doing much research I found a similar example of what I am trying to accomplish here:
JavaFX Update progressbar in tableview from Task
(Which points to this: 'https://community.oracle.com/message/10999916')
The problem I am running into, however, is illustrated well in this example; how can you call a 'Task' object multiple times to update a ProgressIndicator?
My understanding from Oracle's documentation is that a Task object "is a one-shot class and cannot be reused". It would seem then that one can only invoke the call() method of a Task object once. I need to update the Task multiple times as it progresses through a Thread class, not call it once and arbitrarily increment through a For loop.
I have read about binding to Listeners and creating Service classes, but I am unsure if those are actual resolutions to this problem. I would therefore like to ask if this is even possible in Javafx, or if perhaps I am overlooking something. In the event someone has accomplished this in the past, it would be tremendously helpful if you might be able to illustrate how through the example provided previously.
Any direction on this would be appreciated, thank you.
-Drew
EDIT 1: I edited my wording as it was inaccurate.
EDIT 2: Here is an example with some pseudo code. Say I had a class with the following code:
public static class TaskEx extends Task<Void>{
#Override
protected Void call(){
updateProgress(.5, 1);
return null
}
public static void callThread() {
TableView<TaskEx> table = new TableView<TaskEx>();
//Some code for data in table.
TableColumn progressColumn = new TableColumn ("Progress");
progressColumn.setCellValueFactory(new PropertyValueFactor("progress");
table.setItems(<data>);
table.getColumns();addAll(progressColumn);
ExecutorService executor = Executors.newFixedThreadPool(<SomeNumber>);
for(TaskEx task : table.getItems(){
Threading.ThreadClass newThread = new Threading.ThreadClass(task);
executor.submit(newThread, <uniqueID>);
}
}
Then say I had a second class for Threading with this logic:
static class ThreadClass extends Thread{
Task progressTask;
public ThreadClass(Task task, Integer id){
progressTask = task;
}
public void run(){
ExecutorService executor = Executors.newFixedThreadPool(<someNumber>);
//This invokes the Task call for the correct progressIndicator in the Tableview.
//It will correctly set the progressIndicator to 50% done.
executor.submit(progressTask);
/* Main logic of the Threading class that involves the 'id' passed in. */
//This will do nothing because you cannot invoke the Task call more than once.
executor.submit(progressTask);
}
}
That is the sort of workflow I need, but I'm unsure how to accomplish this.

It seems like you don't get what we were talking about. You are trying to do your logic in the Thread.run(), and then each thread is creating a Task just to do the update of progress.
What you need is really to shift your logic from Thread.run() to Task.call(). Your thread is really just a thread, and all it does is to run a Runnable object (which is the Task).
public class TaskEx extends Task<Void> {
#Override
protected Void call() {
// Do whatever you need this thread to do
updateProgress(0.5, 1);
// Do the rest
updateProgress(1, 1);
}
}
public static void callThread() {
TableView<TaskEx> table = new TableView<TaskEx>();
ObservableList<TaskEx> data = FXCollections.observableArrayList<>();
data.add(new TaskEx()); // Add the data you need
TableColumn progressColumn = new TableColumn("Progress");
progressColumn.setCellValueFactory(new PropertyValueFactory("progress"));
progressColumn.setCellFactory(column -> {
return new TableCell<TaskEx, Double> {
private final ProgressBar bp = new ProgressBar();
#Override
public void updateItem(Double item, boolean empty) {
super.updateItem(item, empty);
if (empty || item == null) {
setText(null);
setGraphic(null);
}
else {
bp.setProgress(item.doubleValue());
setGraphic(bp);
}
}
}
});
table.setItems(data);
table.getColumns().add(progressColumn);
ExecutorService executor = Executors.newFixedThreadPool(data.size());
for (TaskEx task : table.getItems()) {
executor.submit(task);
}
}
This implement removes ThreadClass because there should not be any logic that must be done at a thread sub-class. If you really need to access the thread object as part of your logic, call Thread.getCurrentThread() from your TaskEx.call().
This implement also opens multiple threads doing exactly the same thing (which is quite meaningless). If you need to do a set of different logics, you can either make a set of different Task subclasses, or add a constructor taking in Runnable objects in TaskEx.
E.g.
public class TaskEx extends Task<Void> {
private final Runnable[] logics;
public TaskEx(Runnable[] logics) {
this.logics = logics;
}
#Override
protected Void call() {
for (int i = 0; i < logics.length; i++) {
logics[i].run();
updateProgress(i, logics.length);
}
}
}