I'm making a program that gets live price information from an API. I then want to display the price information on a JavaFX chart that live updates. When I try to pass the information to the JavaFX Thread it doesn't always pass over correctly and the JavaFX thread doesn't get the price information.
The API call is done on a separate thread and when it has the information it then calls the updateScene method. This is where I get an issue, the API Thread has all the information I try and set the variable for the JavaFX thread to use and it has none of the information.
private CandleStickFeed candleStickFeed;
public void updateScene(CandleStickFeed candleStickFeed){
this.candleStickFeed = candleStickFeed;
System.out.println("Feed Size This Thread = " + candleStickFeed.getSize());
Platform.runLater(new Runnable(){
#Override
public void run(){
System.out.println("Feed Size JavaFX Thread = " + candleStickFeed.getSize());
updateChart();
}
});
}
The program will sometimes output
Feed Size This Thread = 5
Feed Size JavaFX Thread = 5
Which is what I would expected. But it also sometimes outputs
Feed Size This Thread = 5
Feed Size JavaFX Thread = 0
Any help would be greatly appreciated. I new to using multiple threads so not sure what I'm doing really. I have looked for different answers but couldn't find any. Thank you
Try to extract the relevant information from the candleStickFeed, and pass that structure into a Runnable subclass.
public void updateScene(CandleStickFeed candleStickFeed) {
CandleStickData data = new CandleStickData(candleStickFeed);
Platform.runLater(new ChartUpdateRunnable (data));
}
private class CandleStickData {
private double[] numbers; // or whatever you need
CandleStickData (CandleStickFeed candleStickFeed) {
this.numbers = new double[candleStickFeed.getSize()];
// TODO: populate the data structure
}
}
private ChartUpdateRunnable implements Runnable {
private CandleStickData data;
ChartUpdateRunnable(CandleStickData data) {
this.data = data;
}
#Override
public void run(){
System.out.println("Feed Size JavaFX Thread = " + data);
updateChart();
}
}
The principle is to not pass around a feed class which might change state often, but extract an immutable state object, and pass that to a runnable class instance for update.
This is a supplement to #Simon's answer.
Your problem is not about "passing a variable." Your runLater(...) task apparently is getting the value of the variable. The problem is that the value is a reference to a mutable object, and that object sometimes is modified in between the time when the task is created and the time when the task is executed.
Simon's suggestion boils down to this: Give the new task its own private copy of whatever information it will need from the CandleStickFeed instance, and let it work exclusively from that private copy.
Related
Note: I understand the rules site, but I can't to put all code (complex/large code).
I put a DIFFERENT (all the real code is too much and you don't need here) code in Github but reproduces the Problem (the main class is joseluisbz.mock.support.TestOptimalDSP and switching class is joseluisbz.mock.support.runnable.ProcessorDSP) like the video.
Please don't recommend to me another jar or external library for this code.
I wish I was more specific, but I don't know what part to extract and show.
Before you close this question: Obviously, I am willing to refine my question if someone tells me where to look (technical detail).
I made a video in order to show my issue.
Even to formulate the question, I made a diagram to show the situation.
My program has a JTree, showing the relations between Worker.
I have a diagram interaction between threads controlling life with ExecutorService executorService = Executors.newCachedThreadPool(); and List<Future<?>> listFuture = Collections.synchronizedList(new ArrayList<>());
Each Runnable is started in this way listFuture().add(executorService().submit(this)); in its constructor. The lists are created like this: BlockingQueue<Custom> someBlockingQueue = new LinkedBlockingQueue<>();
My diagram shows who the Worker's father is if he has one.
It also shows, the writing relationships between the BlockingQueue.
RunnableStopper stops related runnables contained in Worker like property.
RunnableDecrementer, RunnableIncrementer, RunnableFilter operates with a cycle that runs each Custom that it receives for its BlockingQueue.
For which they always create a RunnableProcessor (it has no loop, but because of its long processing, once the task is finished it should be collected by the GC).
Internally the RunnableIncrementer has a Map Map<Integer, List<Custom>> mapListDelayedCustom = new HashMap<>();//Collections.synchronizedMap(new HashMap<>());
When arrives some Custom... I need to obtain the List of lastReceivedCustom List<Custom> listDelayedCustom = mapListDelayedCustom.putIfAbsent(custom.getCode(), new ArrayList<>());
I'm controlling the Size (is not growing indefinitely).
My code stops working when I add the following lines:
if (listDelayedCustom.size() > SomeValue) {
//No operation has yet been included in if sentence
}
But commenting the lines doesn't block
//if (listDelayedCustom.size() > SomeValue) {
// //No operation has yet been included in if sentence
//}
What could be blocking my Runnable?
It makes no sense that adding the lines indicated (Evaluate the size of a list: if sentence) above stops working.
Any advice to further specify my question?
First, the way you set thread names is wrong. You use this pattern:
public class Test
{
public static class Task implements Runnable
{
public Task()
{
Thread.currentThread().setName("Task");
}
#Override
public void run()
{
System.out.println("Task: "+Thread.currentThread().getName());
}
}
public static void main(String[] args)
{
new Thread(new Task()).start();
System.out.println("Main: "+Thread.currentThread().getName());
}
}
which gives the (undesired) result:
Main: Task
Task: Thread-0
It's incorrect because, in the Task constructor, the thread has not started yet, so you're changing the name of the calling thread, not the one of the spawned thread. You should set the name in the run() method.
As a result, the thread names in your screenshot are wrong.
Now the real issue. In WorkerDSPIncrement, you have this line:
List<ChunkDTO> listDelayedChunkDTO = mapListDelayedChunkDTO.putIfAbsent(chunkDTO.getPitch(), new ArrayList<>());
The documentation for putIfAbsent() says:
If the specified key is not already associated with a value (or is mapped to null) associates it with the given value and returns null, else returns the current value.
Since the map is initially empty, the first time you call putIfAbsent(), it returns null and assigns it to listDelayedChunkDTO.
Then you create a ProcessorDSP object:
ProcessorDSP processorDSP = new ProcessorDSP(controlDSP, upNodeDSP, null,
dHnCoefficients, chunkDTO, listDelayedChunkDTO, Arrays.asList(parent.getParentBlockingQueue()));
It means you pass null as the listDelayedChunkDTO parameter. So when this line executes in ProcessorDSP:
if (listDelayedChunkDTO.size() > 2) {
it throws a NullPointerException and the runnable stops.
I'm currently trying to implement a system list that would run in a few different threads:
1) First thread is listening to incoming requests and adds them to the list.
2) A new thread is created for each request to perform certain operations.
3) Another thread iterates through the list, checks the status of each request, and removes them from the list when they're complete.
Now, the way I have it in a very simplified pseudocode can be viewed below:
private List<Job> runningJobs = new ArrayList<>(); // our list of requests
private Thread monitorThread;
private Runnable monitor = new Runnable() { // this runnable is later called in a new thread to monitor the list and remove completed requests
#Override
public void run() {
boolean monitorRun = true;
while(monitorRun) {
try {
Thread.sleep(1000);
if (runningJobs.size()>0){
Iterator<Job> i = runningJobs.iterator();
while (i.hasNext()) {
try {
Job job = i.next();
if (job.jobStatus() == 1) { // if job is complete
i.remove();
}
}
catch (java.util.ConcurrentModificationException e){
e.printStackTrace();
}
}
}
if (Thread.currentThread().isInterrupted()){
monitorRun = false;
}
} catch (InterruptedException e) {
monitorRun = false;
}
}
}
};
private void addRequest(Job job){
this.runningJobs.add(newJob);
// etc
}
In short, the Runnable monitor is what runs continuously in the third thread; the first thread is calling addRequest() occasionally.
While my current implementation somewhat works, I'm concerned about the order of operations here and possible java.util.ConcurrentModificationException (and the system is anything but robust). I'm certain there is a much better way to organize this mess.
What's the proper or a better way to do this?
Your requirements would be met nicely with an ExecutorService. For each request, create Job, and submit it to the service. Internally, the service uses a BlockingQueue, which would address your question directly, but you don't have to worry about it with an ExecutorService.
Specifically, something like this:
/* At startup... */
ExecutorService workers = Executors.newCachedThreadPool();
/* For each request... */
Job job = ... ;
workers.submit(job); /* Assuming Job implements Runnable */
// workers.submit(job::jobEntryPoint); /* If Job has some other API */
/* At shutdown... */
workers.shutdown();
There are a few different ways.
You can synchronize the list. This is possibly the most brute-force and still wouldn't help prevent an insert while you are iterating over it.
There are a few synchronized* collections. These tend to be better but have ramifications. For instance CopyOnWriteArrayList will work but it creates a new array list each time (that you would assign back to the variable). This is good for occasionally updated collections.
There is a ConcurrentLinkedQueue--Since it's "Linked" you can't reference an item in the middle.
Look through the implementations of the "List" interface and pick the one that best suits your problem.
If your problem is a queue instead of a list, there are a few implementations of that as well and they will tend to be better suited for that type of problem.
In general my answer is that you should probably scan through the Javadocs every time java does a major release and examine (at least) the new collections. You might be surprised at the stuff that's in there.
Goal: To know, as I fork off a thread, which processor it's going to land on. Is that possible? Regardless of whether the underlying approach is valid, is there a good answer to that narrow question? Thanks.
(Right now I need to make a copy of one of our classes for each thread, write to it in that thread and merge them all later. Using a synchronized approach is not possible because my Java expert boss thinks it's a bad idea, and after a lot of discussion I agree. If I knew which processor each thread would land on, I would only need to make as many copies of that class as there are processors.)
We use Apache Spark to get our jobs spread across a cluster, but in our application is makes sense to run one big executor and then do some multi-threading of our own out on each machine in the cluster.
I could save a lot of deep copying if I could know which processor a thread is being sent to, is that possible? I threw in our code but it's probably more of a conceptual question:
When I get down to the "do task" part of compute(), can I know which processor it's running on?
public class TholdExecutor extends RecursiveTask<TholdDropEvaluation> {
final static Logger logger = LoggerFactory.getLogger(TholdExecutor.class);
private List<TholdDropResult> partitionOfN = new ArrayList<>();
private int coreCount;
private int desiredPartitionSize; // will be updated by whatever is passed into the constructor per-chromosome
private TholdDropEvaluation localDropEvaluation; // this DropEvaluation
private TholdDropResult mSubI_DR;
public TholdExecutor(List<TholdDropResult> subsetOfN, int cores, int partSize, TholdDropEvaluation passedDropEvaluation, TholdDropResult mDrCopy) {
partitionOfN = subsetOfN;
coreCount = cores;
desiredPartitionSize = partSize;
// the TholdDropEvaluation needs to be a copy for each thread? It can't be the same one passed to threads ... so ...
TholdDropEvaluation localDropEvaluation = makeDECopy(passedDropEvaluation); // THIS NEEDS TO BE A DEEP COPY OF THE DROP EVAL!!! NOT THE ORIGINAL!!
// we never modify the TholdDropResult that is passed in, we just need to read it all on the same JVM/worker, so
mSubI_DR = mDrCopy; // this is purely a reference and can point to the passed in value (by reference, right?)
}
// this makes a deep copy of the TholdDropEvaluation for each thread, we copy the SharingRun's startIndex and endIndex only,
// as LEG events will be calculated during the subsequent dropComparison. The constructor for TholdDropEvaluation must set
// LEG events to zero.
private void makeDECopy(TholdDropEvaluation passedDropEvaluation) {
TholdDropEvaluation tholdDropEvaluation = new TholdDropEvaluation();
// iterate through the SharingRuns in the SharingRunList from the TholdDropEval that was passed in
for (SharingRun sr : passedDropEvaluation.getSharingRunList()) {
SharingRun ourSharingRun = new SharingRun();
ourSharingRun.startIndex = sr.startIndex;
ourSharingRun.endIndex = sr.endIndex;
tholdDropEvaluation.addSharingRun(ourSharingRun);
}
return tholdDropEvaluation
}
#Override
protected TholdDropEvaluation compute() {
int simsToDo = partitionOfN.size();
UUID tag = UUID.randomUUID();
long computeStartTime = System.nanoTime();
if (simsToDo <= desiredPartitionSize) {
logger.debug("IN MULTI-THREAD compute() --- UUID {}:Evaluating partitionOfN sublist length", tag, simsToDo);
// job within size limit, do the task and return the completed TholdDropEvaluation
// iterate through each TholdDropResult in the sub-partition and do the dropComparison to the refernce mSubI_DR,
// writing to the copy of the DropEval in tholdDropEvaluation
for (TholdDropResult currentResult : partitionOfN) {
mSubI_DR.dropComparison(currentResult, localDropEvaluation);
}
} else {
// job too large, subdivide and call this recursively
int half = simsToDo / 2;
logger.info("Splitting UUID = {}, half is {} and simsToDo is {}", tag, half, simsToDo );
TholdExecutor nextExec = new TholdExecutor(partitionOfN.subList(0, half), coreCount, desiredPartitionSize, tholdDropEvaluation, mSubI_DR);
TholdExecutor futureExec = new TholdExecutor(partitionOfN.subList(half, simsToDo), coreCount, desiredPartitionSize, tholdDropEvaluation, mSubI_DR);
nextExec.fork();
TholdDropEvaluation futureEval = futureExec.compute();
TholdDropEvaluation nextEval = nextExec.join();
tholdDropEvaluation.merge(futureEval);
tholdDropEvaluation.merge(nextEval);
}
logger.info("{} Compute time is {} ns",tag, System.nanoTime() - computeStartTime);
// NOTE: this was inside the else block in Rob's example, but don't we want it outside the block so it's returned
// whether
return tholdDropEvaluation;
}
}
Even if you could figure out where a thread would run initially there's no reason to assume it would live on that processor/core for the rest of its life. In all probability for any task big enough to be worth the cost of spawning a thread it won't, so you'd need to control where it ran completely to offer that level of assurance.
As far as I know there's no standard mechanism for controlling mappings from threads to processor cores inside Java. Typically that's known as "thread affinity" or "processor affinity". On Windows and Linux for example you can control that using:
Windows: SetThreadAffinityMask
Linux: sched_setaffinity or pthread_setaffinity_np
so in theory you could write some C and JNI code that allowed you to abstract this enough on the Java hosts you cared about to make it work.
That feels like the wrong solution to the real problem you seem to be facing, because you end up withdrawing options from the OS scheduler, which potentially doesn't allow it to make the smartest scheduling decisions causing total runtime to increase. Unless you're pushing an unusual workload and modelling/querying processor information/topology down to the level of NUMA and shared caches it ought to do a better job of figuring out where to run threads for most workloads than you could. Your JVM typically runs a large number of additional threads besides just the ones you explicitly create from after main() gets called. Additionally I wouldn't like to promise anything about what the JVM you run today (or even tomorrow) might decide to do on its own about thread affinity.
Having said that it seems like the underlying problem is that you want to have one instance of an object per thread. Typically that's much easier than predicting where a thread will run and then manually figuring out a mapping between N processors and M threads at any point in time. Usually you'd use "thread local storage" (TLS) to solve this problem.
Most languages provide this concept in one form or another. In Java this is provided via the ThreadLocal class. There's an example in the linked document given:
public class ThreadId {
// Atomic integer containing the next thread ID to be assigned
private static final AtomicInteger nextId = new AtomicInteger(0);
// Thread local variable containing each thread's ID
private static final ThreadLocal<Integer> threadId =
new ThreadLocal<Integer>() {
#Override protected Integer initialValue() {
return nextId.getAndIncrement();
}
};
// Returns the current thread's unique ID, assigning it if necessary
public static int get() {
return threadId.get();
}
}
Essentially there are two things you care about:
When you call get() it returns the value (Object) belonging to the current thread
If you call get in a thread which currently has nothing it will call initialValue() you implement, which allows you to construct or obtain a new object.
So in your scenario you'd probably want to deep copy the initial version of some local state from a read-only global version.
One final point of note: if your goal is to divide and conquer; do some work on lots of threads and then merge all their results to one answer the merging part is often known as a reduction. In that case you might be looking for MapReduce which is probably the most well known form of parallelism using reductions.
just want to make clear an understanding on using for loops inside a SwingWorker doInbackground method.
For example, I have a list of files stored in Files ( File[] Files = ... ).
scanFiles = new SwingWorker<Object, Object>(){
public Object doInBackground(){
for( File f : Files ){
// process file f
}
}
}
....
scanFiles.execute();
In the above, is it alright to use a for loop inside the doInBackGround() method to go through a list of files , or is it better to bring the for loop outside the doInBackground() method, as in something like this:
for ( File f: Files ){
processFile(f);
}
private void processFile(File f){
scanFiles = new SwingWorker<Object, Object>(){
public Object doInBackground(){
// do something with f
}
}
}
The above are skeleton code and not actual working code. Just for illustration of what I want to do only. That is, I don't want my program to scan files one by one. I want to do something like parallel processing of files...
thanks
As mentioned in some of the comments: The appropriate solution heavily depends on how many files you want to process, and what processFile actually does.
The main difference between your approaches is (as MadProgrammer already said)
The first one creates one background thread that processes all the files
The second one creates many background threads, each processing one file
The border cases where either of the approaches is not appropriate are analogously:
The first one may be better when there many files, and processFile is a simple operation
The second one may be better when there are few files and processFile is a complex operation
But this is only a rough classification, and which one is the "best" approach still depends on other factors.
However, I'd like to propose another solution, that allows you to rather flexibly "shift" between the two extremes: You could create a List containing the File objects, and split this list into a specified number of "chunks" to let them be processed by the SwingWorker.
Sketched here, to show the basic idea: You create a method that processes a list of files with a SwingWorker:
private void processFiles(final List<File> files) {
SwingWorker<Object, Object> scanFiles = new SwingWorker<Object, Object>(){
#Override
public Object doInBackground(){
// do something with files
}
}
}
Then, at the call site, you can do the following:
// Obtain the list of files to process
File files[] = ...
List<File> fileList = Arrays.asList(files);
// Define the number of workers that should be used
int numWorkers = 10;
// Compute how many files each worker will process
int chunkSize = (int)Math.ceil((double)fileList.size() / numWorkers);
for (int i=0; i<numWorkers; i++) {
// Compute the part of the "fileList" that the worker will process
int minIndex = i * chunkSize;
int maxIndex = i * chunkSize + chunkSize;
maxIndex = Math.min(maxIndex, fileList.size());
List<File> chunk = fileList.sublist(minIndex, maxIndex);
// Start the worker
processFiles(chunk);
}
(This is only a sketch. There may be some index-hassle involved. If desired, I can post a more elaborate version of this. Until now, it only shows the basic idea)
Then, you can define how many worker threads you would like to use (maybe even depending on the number of Runtime.getRuntime().availableProcessors()).
If you want to process files parallely you must spawn some thread workers so the second sample should be your choice. You can inform the UI, or other components of your program, about the progress of processing files using following methods : protected void process(List<V> chunks), protected final void publish(V... chunks)
private void processFile(File f){
scanFiles = new SwingWorker<Object, Object>(){
public Object doInBackground(){
publish(V... chunks)
}
}
}
protected void process(List<V> chunks) {
//do something with intermediate data, for example show progress in the ui
}
I have a object obj that is read by many threads frequently , but updated by only one thread periodically.The update happens after long some interval (say 10 minutes).
The data is less transnational.Meaning if read threads gets stale data(old) for some time then its perfectly ok.
Now i thought of using following approach for synchronization:
final Object lock = new Object();
private MyObject obj = new MyObject(); //this is the data
public String getDataFieldName(){
synchronized(lock){
return this.obj.name;
}
}
/*the following code is wrong right?As its just synchronizes the code for getting reference.But after getting reference read thread R1 may try to get data while write Thread is modifying data.Will that give exception?How to solve this? */
public String getData(){
synchronized(lock){
return this.obj;
}
}
//only one thread can update.But how multipe threads can read at once?
public updateData(args ) {
synchronized(lock){
//do update
}
}
My questions are as follows:
I dont want only one thread to read the data.Reads should be parallel.
How can i synchronize read and write ?If write thread is updating and read thread is reading i dont what to get some exception.It is ok if read gets some old data
3)If read thread is reading while write thread is updating , will i get exception?Will there be any problem?
You don't need any synchronization in this scenario. All you have to do is the following:
Make sure MyObject is immutable, meaning that you never change any values in your object Instead you construct a new MyData object everytime you change it. This prevents anyone from seeing a half-changed object.
Declare obj as volatile, to make sure all threads see the updated value everytime.
You will never get an exception because of concurrent reads and writes if you follow these steps.
Use volatile keyword, that will not take the lock, unlike synchronization does, and will provide multiple access, reflecting the updates of one thread to the another..
But then its always better to have some sort of synchronization, as volatile will not ensure to prevent the Race Condition on the data. So if you dont want to use synchronization, then better go with Immutable object
Eg:
import java.util.Date;
/**
* Planet is an immutable class, since there is no way to change
* its state after construction.
*/
public final class Planet {
//Final primitive data is always immutable.
private final double fMass;
private final String fName;
private final Date fDateOfDiscovery;
public Planet (double aMass, String aName, Date aDateOfDiscovery) {
fMass = aMass;
fName = aName;
//make a private copy of aDateOfDiscovery
//this is the only way to keep the fDateOfDiscovery
//field private, and shields this class from any changes that
//the caller may make to the original aDateOfDiscovery object
fDateOfDiscovery = new Date(aDateOfDiscovery.getTime());
}
public double getMass() {
return fMass;
}
public String getName() {
return fName;
}
public Date getDateOfDiscovery() {
return new Date(fDateOfDiscovery.getTime());
}
}