I have a method that runs with Selenium to create user accounts on a website quickly. Currently it processes one after the other, but I'm thinking if I can process 10 at once that would be better.
I have a for loop currently, which is used to tell the code within, which line of my 2D array to read the user information from. I am struggling with the concept of how to make any stream or thread use the correct value and fetch the correct user information.
Currently I have something similar to the below simplified:
I need to load a new page and driver everytime this loops and need to send the value of the array to the web field. So basically I want this to go off and loop and not wait for the first loop to finish before starting the next loop, but probably limit to 10 or so running at once.
for(i=0,i<myarray.length, i++)
{
Webdriver.start();
WebElement.findby.(By.name("field1").sendkeys(myArray[i][2]);
Webdriver.end();
}
As I said code is not actual code it is just to get my question across.
Hope that is clear.
I think you're saying, I am iterating through myArray and running my test once for each element in that array, but instead of running one test and waiting for it to finish before running the next, I want to run a whole bunch at a time.
You can do this pretty trivially with the Java 8 ForkJoinPool.
ForkJoinTask[] tasks = new ForkJoinTask[myarray.length];
for(i=0,i<myarray.length, i++)
{
int j = i; // need an effectively final copy of i
tasks[i] = ForkJoinPool.commonPool().submit(() -> {
Webdriver.start();
WebElement.findby.(By.name("field1").sendkeys(myArray[j][2]);
Webdriver.end();
});
}
for (i = 0; i < my array.length; i++) {
tasks[i].join();
}
The tests will run in parallel using threads from the "common" ForkJoinPool. If you want to adjust the number of threads that are used, create your own ForkJoinPool. (See this question for more information.)
I would explicitly start separate Thread for every task as the most of time will be likely spent on waiting until the user account is created.
Please, see the code snippet with rough example below:
public void createAccounts() throws InterruptedException {
List<Thread> threadList = new ArrayList<>();
Object[][] myArray = new Object[1][1];
for(int i=0; i<myArray.length; i++) {
final int index = i;
//Add thread for user creation
threadList.add(new Thread(new Runnable() {
#Override
public void run() {
Webdriver.start();
WebElement.findby.(By.name("field1").sendkeys(myArray[index][2]);
Webdriver.end();
}
}));
}
//Start all threads
for (Thread thread : threadList) {
thread.start();
}
//Wait until all threads are finished
for (Thread thread : threadList) {
thread.join();
}
}
Related
I'm learning multithreaded counter and I'm wondering why no matter how many times I ran the code it produces the right result.
public class MainClass {
public static void main(String[] args) {
Counter counter = new Counter();
for (int i = 0; i < 3; i++) {
CounterThread thread = new CounterThread(counter);
thread.start();
}
}
}
public class CounterThread extends Thread {
private Counter counter;
public CounterThread(Counter counter) {
this.counter = counter;
}
public void run() {
for (int i = 0; i < 10; i++) {
this.counter.add();
}
this.counter.print();
}
}
public class Counter {
private int count = 0;
public void add() {
this.count = this.count + 1;
}
public void print() {
System.out.println(this.count);
}
}
And this is the result
10
20
30
Not sure if this is just a fluke or is this expected? I thought the result is going to be
10
10
10
Try increasing the loop count from 10 to 10000 and you'll likely see some differences in the output.
The most logical explanation is that with only 10 additions, a thread is too fast to finish before the next thread gets started and adds on top of the previous result.
I'm learning multithreaded counter and I'm wondering why no matter how many times I ran the code it produces the right result.
<ttdr> Check out #manouti's answer. </ttdr>
Even though you are sharing the same Counter object, which is unsynchronized, there are a couple of things that are causing your 3 threads to run (or look like they are running) serially with data synchronization. I had to work hard on my 8 proc Intel Linux box to get it to show any interleaving.
When threads start and when they finish, there are memory barriers that are crossed. According to the Java Memory Model, the guarantee is that the thread that does the thread.join() will see the results of the thread published to it but I suspect a central memory flush happens when the thread finishes. This means that if the threads run serially (and with such a small loop it's hard for them not to) they will act as if there is no concurrency because they will see each other's changes to the Counter.
Putting a Thread.sleep(100); at the front of the thread run() method causes it to not run serially. It also hopefully causes the threads to cache the Counter and not see the results published by other threads that have already finished. Still needed help though.
Starting the threads in a loop after they all have been instantiated helps concurrency.
Another thing that causes synchronization is:
System.out.println(this.count);
System.out is a Printstream which is a synchronized class. Every time a thread calls println(...) it is publishing its results to central memory. If you instead recorded the value and then displayed it later, it might show better interleaving.
I really wonder if some Java compiler inlining of the Counter class at some point is causing part of the artificial synchronization. For example, I'm really surprised that a Thread.sleep(1000) at the front and end of the thread.run() method doesn't show 10,10,10.
It should be noted that on a non-intel architecture, with different memory and/or thread models, this might be easier to reproduce.
Oh, as commentary and apropos of nothing, typically it is recommended to implement Runnable instead of extending Thread.
So the following is my tweaks to your test program.
public class CounterThread extends Thread {
private Counter counter;
int result;
...
public void run() {
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
Thread.currentThread().interrupt(); // good pattern
return;
}
for (int i = 0; i < 10; i++) {
counter.add();
}
result = counter.count;
// no print here
}
}
Then your main could do something like:
Counter counter = new Counter();
List<CounterThread> counterThreads = new ArrayList<>();
for (int i = 0; i < 3; i++) {
counterThread.add(new CounterThread(counter));
}
// start in a loop after constructing them all which improves the overlap chances
for (CounterThread counterThread : counterThreads) {
counterThread.start();
}
// wait for them to finish
for (CounterThread counterThread : counterThreads) {
counterThread.join();
}
// print the results
for (CounterThread counterThread : counterThreads) {
System.out.println(counterThread.result);
}
Even with this, I never see 10,10,10 output on my box and I often see 10,20,30. Closest I get is 12,12,12.
Shows you how hard it is to properly test a threaded program. Believe me, if this code was in production and you were expecting the "free" synchronization is when it would fail you. ;-)
This question already has answers here:
JavaFX periodic background task
(6 answers)
Closed 3 years ago.
I have Array String[] announcement = new String[20]; that has 20 value, I want to retrieve each of them every 5 seconds. I can't find any solution how to make increment every 5 Seconds without blocking my UI.
What you need is a Thread. Threads run alongside your program so that you can run lengthy tasks (such as downloading a file) while your program still runs without freezing up. An example of a program to set a value for your strings (which is what I assume you're doing from your explanation) every five seconds would look like this:
import java.util.concurrent.TimeUnit;
class Main {
public static void main(String[] args) {
// Thread runs alongside program without freezing
String[] retrievalArary = new String[20];
Thread thread = new Thread(new Runnable() {
public void run() {
for (int i = 0; i < retrievalArary.length; i++) { // Run for the same count as you have items in your array
try {
TimeUnit.SECONDS.sleep(5); // Sleeps the thread for five seconds
} catch (InterruptedException e) {
e.printStackTrace();
}
retrievalArary[i] = Integer.toString(i); // Add the integer run count to the array
System.out.println(i); // Show progress
}
}
});
thread.start();
}
}
I couldn't really tell exactly what you were trying to accomplish, but you can change that code pretty easily to fit your requirements.
As the question is tagged with JavaFX I assume you want to update some Node once your retrieve the value. If you use normal thread implementation, you have to wrap your code with Platform.runLater. But if you use javafx.animation.Timeline you dont need to do that extra work.
String[] announcement = new String[20];
AtomicInteger index = new AtomicInteger();
Timeline timeline= new Timeline(new KeyFrame(Duration.seconds(5), e->{
String msg = announcement[index.getAndIncrement()];
// Use this value to do some stuff on Nodes.
});
timeline.setCycleCount(announcement.length);
timeline.play();
My questions is, I have a data set of 1000 records. I want 3 threads that process the data like this,
thread1 from record 1 to 300, thread2 from 301 to 600 and so on. One thread can make a request and fetch 50 records at a time, create an object and put it in a queue.
Main thread will simultaneously read data from the queue.
Below is the code, the problem I am facing is that recordRead variable tells the starting point from where the thread should start reading the records.
But how can I set different value for each thread e.g for thread1 it should be 0 and recordsToRead should be 300 and for thread2, recordRead should be 300 and recordsToRead to be 300+300=600 and for last thread it should be 600 and upto the end.
pagesize=50
pagesize,recordRead and recordToRead are all variables that belong to main class and main thread.
ExecutorService service = Executors.newFixedThreadPool(nThreads);
while(nThreads > 0) {
nThreads--;
service.execute(new Runnable() {
#Override
public void run() {
// TODO Auto-generated method stub
do {
int respCode = 0;
int RecordsToRead = div;
JSONObject jsObj = new JSONObject();
jsObj.put("pagesize", pageSize);
jsObj.put("start", recordsRead);
jsObj.put("searchinternalid", searchInternalId);
try {
boolean status = req.invoke(jsObj);
respCode = req.getResponseCode();
} catch (Exception e) {
req.reset();
e.printStackTrace();
return true;
}
JSONObject jsResp = req.getResponseJson();
//here jsResp will be added to ArrayBlockingQueue.
req.reset();
}while(!isError && !isMaxLimit && recordsRead < RecordsToRead);
}
});
}
After this loop will be the code of main thread reading the queue.
how can I set recordsRead and recordToread for all threads.
And how to make main thread wait untill atleast one thread inserts an object in queue.
I see in you definition two problems. First problem is to perform parallel chunk computation and second one is to create a continous pipeline from this. Lets start from the first problem. To make parallel computations with predefined size the best option fmpv will be to use fork-join framework. Not only by performance (work stealing is really effective) but also due to simpler code. But since you are limited to 3 threads for me it also seems valid to use threads directly. Simply what you want can me implemented by this way:
final int chunkSize = 300;
//you can also use total amount of job
//int totalWork = 1000 and chunk size equals totalWork/threadsNumber
final int threadsNumber = 3;
Thread[] threads = new Thread[threadsNumber];
for (int ii = 0; ii < threadsNumber; ii++) {
final int i = ii;
threads[ii] = new Thread(() -> {
//count your variable according the volume
// for example you can do so
int chunkStart = i * chunkSize;
int chunkEnd = chunkStart + chunkSize;
for(int j = chunkStart; j < chunkEnd; j++) {
//object creation with necessary proprs
//offer to queue here
}
});
threads[ii].start();
}
//your code here
//take here
for (int ii = 0; ii < threadsNumber; ii++) {
try {
//this part is only as example
//you do not need it
//here if you want you can also w8 for completion of all threads
threads[ii].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Now about the second problem with consuming. For this puprose you can use for example ConcurrentLinkedBlockingQueue (http://www.jgroups.org/javadoc/org/jgroups/util/ConcurrentLinkedBlockingQueue.html). Make offer in producer threads and use take method in main.
But to be honest i still did not get the reason of your problem. Do you want to create continuous pipeline or it is just one-time computation?
Also i will recommend you to take this course: https://www.coursera.org/learn/parallel-programming-in-java/home/welcome.
This will help you exactly with your problem and provide various solutions. There are also concurrent and distributed computing courses.
I'm trying to write a class that can only run X(Let's say 3)threads at one time. I have 8 threads that need to execute but I only want to allow 3 to run at once, then wait. Once one of the currently running threads stops, then it will start another. I'm not quite sure how to do this. My code looks like this:
public class Main {
public void start() {
for(int i=0; i<=ALLTHREADS; i++) {
MyThreadClass thread = new MyThreadClass(someParam, someParam);
thread.run();
// Continue until we have 3 running threads, wait until a new spot opens up. This is what I'm having problems with
}
}
}
public class MyThreadClass implements Runnable {
public MyThreadClass(int param1, int param2) {
// Some logic unimportant to this post
}
public void run() {
// Generic code here, the point of this is to download a large file
}
}
As you can see above most of it is stubbed out pseudo-code. I can post it if anyone would like but it's unimportant to the main question.
you should use thread pooling mechanism here to run multiple threads.
to make it easy we can go for thread pool executor in java which is very easy
create a fixed pool of 3 threads using executors method.
write a for loop for 8 iteration and call execute on each thread and it will run only 3 threads at a time.
ExecutorService executor = Executors.newFixedThreadPool(3);
for (int i = 0; i < 8; i++) {
Task task = new Task(someParam, someParam);
executor.execute(task);
}
executor.shutdown();
Unless this is homework, you can use Executors.newFixedThreadPool(3) which returns an ExecutorService with a max of 3 threads to perform Runnable tasks.
I'm having a-bit of trouble with threads in java. Basically Im creating an array of threads and starting them. the point of the program is to simulate a race, total the time for each competitor ( i.e. each thread ) and pick the winner.
The competitor moves one space, waits ( i.e. thread sleeps for a random period of time between 5 and 6 seconds ) and then continues. The threads don't complete in the order that they started as expected.
Now for the problem. I can get the total time it takes for a thread to complete; what I want is to store all the times from the threads into a single array and be able to calculate the fastest time.
To do this should I place the array in the main.class file? Would I be right in assuming so because if it was placed in the Thread class it wouldn't work. Or should I create a third class?
I'm alittle confused :/
It's fine to declare it in the method where you invoke the threads, with a few notes:
each thread should know its index in the array. Perhaps you should pass this in constructor
then you have three options for filling the array
the array should be final, so that it can be used within anonymous classes
the array can be passed to each thread
the threads should notify a listener when they're done, which in turn will increment an array.
consider using Java 1.5 Executors framework for submitting Runnables, rather than working directly with threads.
EDIT: The solution below assumes you need the times only after all competitors have finished the race.
You can use a structure that looks like below, (inside your main class). Typically you want to add a lot of you own stuff; this is the main outline.
Note that concurrency is not an issue at all here because you get the value from the MyRunnable instance once its thread has finished running.
Note that using a separate thread for each competitor is probably not really necessary with a modified approach, but that would be a different issue.
public static void main(String[] args) {
MyRunnable[] runnables = new MyRunnable[NUM_THREADS];
Thread[] threads = new Thread[NUM_THREADS];
for (int i = 0; i < NUM_THREADS; i++) {
runnables[i] = new MyRunnable();
threads[i] = new Thread(runnables[i]);
}
// start threads
for (Thread thread : threads) {
thread.start();
}
// wait for threads
for (Thread thread : threads) {
try {
thread.join();
} catch (InterruptedException e) {
// ignored
}
}
// get the times you calculated for each thread
for (int i = 0; i < NUM_THREADS; i++) {
int timeSpent = runnables[i].getTimeSpent();
// do something with the time spent
}
}
static class MyRunnable implements Runnable {
private int timeSpent;
public MyRunnable(...) {
// initialize
}
public void run() {
// whatever the thread should do
// finally set the time
timeSpent = ...;
}
public int getTimeSpent() {
return timeSpent;
}
}