I have 25 batch jobs that are executed constantly, that is, when number 25 is finished, 1 is immediately started.
These batch jobs are started using an URL that contains the value 1 to 25. Basically, I use a for loop from 1 to 25 where I, in each round, call en URL with the current value of i, http://batchjobserver/1, http://batchjobserver/2 and so on.
The problem is that some of these batch jobs are a bit unstable and sometimes crashes which causes the for-loop to restart at 1. As a consequence, batch job 1 is run every time the loop is initiated while 25 runs much less frequently.
I like my current solution because it is so simple (in pseudo code)
for (i=1; i < 26; i++) {
getURL ("http://batchjob/" + Integer.toString(i));
}
However, I would like I to be a random number between 1 and 25 so that, in case something crashes, all the batch jobs, in the long run, are run approximately the same number of times.
Is there some nice hack/algorithm that allows me to achieve this?
Other requirements:
The number 25 changes frequently
This is not an absolut requirement but it would be nice one batch job wasn't run again until all other all other jobs have been attempted once. This doesn't mean that they have to "wait" 25 loops before they can run again, instead - if job 8 is executed in the 25th loop (the last loop of the first "set" of loops), the 26th loop (the first loop in the second set of loops) can be 8 as well.
Randomness has another advantage: it is desirable if the execution of these jobs looks a bit manual.
To handle errors, you should use a try-catch statement. It should look something like this:
for(int i = 1, i<26, i++){
try{
getURL();
}
catch (Exception e){
System.out.print(e);
}
}
This is a very basic example of what can be done. This will, however, only skip the failed attempts, print the error, and continue to the next iteration of the loop.
There are two parts of your requirement:
Randomness: For this, you can use Random#nextInt.
Skip the problematic call and continue with the remaining ones: For this, you can use a try-catch block.
Code:
Random random = new Random();
for (i = 1; i < 26; i++) {
try {
getURL ("http://batchjob/" + Integer.toString(random.nextInt(25) + 1));
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
Note: random.nextInt(25) returns an int value from 0 to 24 and thus, when 1 is added to it, the range becomes 1 to 25.
You could use a set and start randomizing numbers in the range of your batches, while doing this you will be tracking which batch you already passed by adding them to the set, something like this:
int numberOfBatches = 26;
Set<Integer> set = new HashSet<>();
List<Integer> failedBatches = new ArrayList<>();
Random random = new Random();
while(set.size() <= numberOfBatches)
{
int ran = random.nextInt(numberOfBatches) + 1;
if(set.contains(ran)) continue;
set.add(ran);
try
{
getURL ("http://batchjob/" + Integer.toString(ran));
} catch (Exception e)
{
failedBatches.add(ran);
}
}
As an extra, you can save which batches failed
The following is an example of a single-threaded, infinite looping (also colled Round-robin) scheduler with simple retry capabilities. I called "scrape" the routine that calls your batch job (scraping means indexing a website contents):
public static void main(String... args) throws Exception {
Runnable[] jobs = new Runnable[]{
() -> scrape("https://www.stackoverfow.com"),
() -> scrape("https://www.github.com"),
() -> scrape("https://www.facebook.com"),
() -> scrape("https://www.twitter.com"),
() -> scrape("https://www.wikipedia.org"),
};
for (int i = 0; true; i++) {
int remainingAttempts = 3;
while (remainingAttempts > 0) {
try {
jobs[i % jobs.length].run();
break;
} catch (Throwable err) {
err.printStackTrace();
remainingAttempts--;
}
}
}
}
private static void scrape(String website) {
System.out.printf("Doing my job against %s%n", website);
try {
Thread.sleep(100); // Simulate network work
} catch (InterruptedException e) {
throw new RuntimeException("Requested interruption");
}
if (Math.random() > 0.5) { // Simulate network failure
throw new RuntimeException("Ooops! I'm a random error");
}
}
You may want to add multi-thread capabilities (that is achieved by simply adding an ExecutorService guarded by a Semaphore) and some retry logic (for example only for certain type of errors and with a exponential backoff).
Related
I'm a student in Denmark trying to make a school project. What I'm working on at this moment is a reader class that takes in a string then prints out word by word and/or letter by letter.
I did some research and found out that Thread.sleep(time) did exactly what I needed it to do. But after I used it I found out it does not work properly! I tried to research some more and found something called a ThreadPoolExecutor but I can figure out how it works in my case.
My reader:
public class TextReader {
// Print method to print word by word from a string
public void wordByWord(String text) throws InterruptedException {
String[] words = text.split(" ");
for (int i = 0; i < words.length; i++) {
System.out.print(words[i] + " ");
Thread.sleep(250);
}
}
// Print method to print letter by letter from a string
public void letterByLetter(String text) throws InterruptedException {
String[] words = text.split(" ");
for (int i = 0; i < words.length; i++) {
String word = words[i] + " ";
char[] letters = (word.toCharArray());
for (int j = 0; j < letters.length; j++) {
System.out.print(letters[j]);
Thread.sleep(250); //so it does not print all the letters at once
}
}
}
}
The reason why Thread.sleep(time) not works in my case is because I need to print to the console and by using Thread.sleep(time) it does not print like a waterfall. It prints either the string I'm trying to break down (time lower than 250ms) or a few letters a once (250 ms) or is just so slow I can't look at it... (over 250ms). I need it to run fast and smooth! So it looks like someone is writing it.
I think I successfully recreated your problem. Every delay lower than about 205 ms seem to cause updating problems. Sometimes the words/letters don't appear but then at the next interval multiple words/letters appear at the same time.
This seems to be a limitation of the Console I/O performance (See this answer). There isn't really anything you can do about this. If you want to output text with a short, minimal delay like this, you need to program your own GUI (for example JavaFX). This will probably solve the performance issues.
Outputs at different delays
205 ms
190 ms
Thread's sleep method takes milliseconds to stop the execution of current thread for specified milliseconds. If it's slow, you can pass less MS and if it's fast then you can increase the timings. So you can tweak according to your need.
ExecutorFramework is a different thing.
It a way to submit your runnable task to the threads managed by ExecutorFramework.
What you are doing is putting a Thread to sleep for that time. That means the thread will become unblocked after that time, however you aren't accounting for the overhead of context switching from another thread. What you want is something more like this
Tried out the ScheduledExecutorService approach and seems to work fine. There's some optimization to be done and some hoops to jump through to wait for the scheduled printing to finish, but it doesn't seem to display the lag (in the two consoles I tried - Eclipse output and Windows Bash).
public class Output {
public static void main(String[] args) {
String toPrint = "Hello, my name is Voldemort, but few call me that.";
StringPrinter printer = new StringPrinter();
printer.print(toPrint, Output::byCharacter, 30);
System.out.println();
printer.print(toPrint, Output::byWord, 150);
}
private static List<String> byWord(String toSplit) {
Iterable<String> it = () -> new Scanner(toSplit);
return StreamSupport.stream(it.spliterator(), false).map(s -> s + " ").collect(Collectors.toList());
}
private static List<String> byCharacter(String toSplit) {
return toSplit.chars().mapToObj(i -> "" + (char) i).collect(Collectors.toList());
}
}
class StringPrinter implements Runnable {
// using an array to be most efficient
private String[] output;
private int currentIndex;
// the service providing the milliseconds delay
private ScheduledExecutorService printExecutor;
public void print(String toOutput, Function<String, List<String>> split, int delay) {
if (printExecutor != null) {
throw new IllegalStateException();
}
printExecutor = Executors.newSingleThreadScheduledExecutor();
List<String> list = split.apply(toOutput);
output = list.toArray(new String[list.size()]);
currentIndex = 0;
printExecutor.scheduleWithFixedDelay(this, 0, delay, TimeUnit.MILLISECONDS);
// wait until output has finished
synchronized (this) {
while (printExecutor != null)
try {
wait(); // wait for printing to be finished
} catch (InterruptedException e) {}
}
}
#Override
public void run() {
if (currentIndex < output.length) {
System.out.print(output[currentIndex++]);
} else {
// mark this print run as finished
printExecutor.shutdown();
printExecutor = null;
synchronized (this) { notifyAll(); }
}
}
}
I am developing a small game, (Java, LibGdx) where the player fills cloze-style functions with predefined lines of code. The game would then compile the code and run a small test suite to verify that the function does the stuff it is supposed to.
Compiling and running the code already works, but I am faced with the problem of detecting infinite loops. Consider the following function:
// should compute the sum of [1 .. n]
public int foo(int n) {
int i = 0;
while (n > 0) {
i += n;
// this is the place where the player inserts one of many predefined lines of code
// the right one would be: n--;
// but the player could also insert something silly like: i++;
}
return i;
}
Please note that the functions actually used may be more complex and in general it is not possible to make sure that there cannot be any infinite loops.
Currently I am running the small test suite (provided for every function) in a Thread using an ExecutorService, setting a timeout to abort waiting in case the thread is stuck. The problem with this is, that the threads stuck in an endless loop will run forever in the background, which of course will at some point have a considerable impact on game performance.
// TestClass is the compiled class containing the function above and the corresponding test suite
Callable<Boolean> task = new Callable<Boolean>() {
#Override
public Boolean call() throws Exception {
// call the test suite
return new TestClass().test();
}
};
Future<Boolean> future = executorService.submit(task);
try {
Boolean result = future.get(100, TimeUnit.MILLISECONDS);
System.out.println("result: " + (result == null ? "null" : result.toString()));
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
} catch (TimeoutException e) {
e.printStackTrace();
future.cancel(true);
}
My question is now: How can I gracefully end the threads that accidentally spin inside an endless loop?
*EDIT To clarify why in this case, preventing infinite loops is not possible/feasable: The functions, their test suite and the lines to fill the gaps are loaded from disk. There will be hundrets of functions with at least two lines of code that could be inserted. The player can drag any line into any gap. The effort needed to make sure no combination of function gap/code line produces something that loops infinitely or even runs longer than the timeout grows exponentially with the number of functions. This quickly gets to the point where nobody has the time to check all of these combinations manually. Also, in general, determining, whether a function will finish in time is pretty much impossible because of the halting problem.
There is no such thing as "graceful termination" of a thread inside the same process. The terminated thread can leave inconsistent shared-memory state behind it.
You can either organize things so that each task is started in its own JVM, or make do with forceful termination using the deprecated Thread.stop() method.
Another option is inserting a check into the generated code, but this would require much more effort to implement properly.
The right way is to change the design and avoids never ending loops.
For the time being, inside your loop you could check if the thread is interrupted some way by: isInterrupted() or even isAlive().
And if it is you just exit.
It is not normal to have a never ending loop if it not wanted.
To solve the problem You can add a counter in the loop and if you reach a limit you can exit.
int counter = 0;
while (n > 0) {
counter++;
if (counter > THRESHOLD) {
break;
}
i += n;
// this is the place where the player inserts one of many predefined lines of code
// the right one would be: n--;
// but the player could also insert something silly like: i++;
}
So I was wondering if there was anyway I could implement this code so that it pauses on each loop iteration. Currently, when I run the code, the program will stop for n seconds (n being the amount of loop iterations) and then display everything at once. However, I wish for it to display one item, wait one second and display the next item. I hope this is clear.
while(x > 0 || y > 0){
Try{
Thread.sleep(1000);
} catch (InterruptedException ie) {
ie.printStackTrace();
}
// Print x and y
//Change x and y vals
}
}
First, don't suppress InterruptedException. If there's nothing to be done about it, simply don't catch it at all, or convert it into a RuntimeException e.g.
throw new RuntimeException(ie);
Second, it sounds like you're describing a flushing problem, which can be addressed by adding calls to:
System.out.flush();
After you print your values. As mentioned in the linked question however, System.out and System.err auto-flush whenever a new line is printed; are you not printing to stdout, or not printing new lines?
You should have a Thread.sleep() call wherever you need the program to pause, so if you need to pause between printing x and y, add another Thread.sleep() between them.
Suggestion: separate the thread sleep in a method to do the job without rewriting all this code.
Suggestion 2: search aroung if thread sleep is the best solution for you
Try this:
public void waitSeconds(int seconds){
try{
Thread.sleep(seconds*1000l);
} catch(Exception e) {
e.printStackTrace(); //ugly but enough to explain
};
}
public void yourMethod(){
while(x > 0 || y > 0){
waitSeconds(1);
print x
waitSeconds(1);
print y
//Change x and y vals
}
}
It's really strange. Try this way:
while(x > 0 || y > 0) {
// Print old values
// Change your x and y
// Print new values
try {
TimeUnit.SECONDS.sleep(y);
} catch(InterruptedException ex) {}
}
I am new to java development, I have bit doubt on how to slow down the for loop. I have set of list counts, and i iterate those using for loop. i want to iterate the count for certain time limit. (ie) the iterating time between count 1 and two should be delayed for 2 sec. is this possible to do. Pl guide me on this
public static void main(String[] args) {
for (int i = 0; i < 5; i++) {
try {
System.out.println(i);
Thread.sleep(2000);
} //System.out.println(stem[0]);
catch (InterruptedException ex) {
Logger.getLogger(JavaApplication2.class.getName()).log(Level.SEVERE, null, ex);
}
}
The clearest way I can think of is to use
java.util.concurrent.TimeUnit.SECONDS.sleep(2);
Try using:
Thread.sleep(time); //time in milliseconds, in your case it is 2000
I wrote a simple code that uses multiple threads to calculate number of primes from 1 to N.
public static void main (String[] args) throws InterruptedException
{
Date start;
start = new Date();
long startms = start.getTime();
int number_primes = 0, number_threads =0;
number_primes = Integer.parseInt(args[0]);
number_threads = Integer.parseInt(args[1]);
MakeThread[] mt = new MakeThread[number_threads];
for(int i=1;i<=number_threads;i++)
{
mt[i-1] = new MakeThread(i,(i-1)*(number_primes/number_threads),i*(number_primes/number_threads));
mt[i-1].start();
}
for(int i=1;i<number_threads;i++)
{
mt[i-1].join();
}
Date end = new Date();
long endms = end.getTime();
System.out.println("Time taken = "+(endms-startms));
}
}
As show in above, I want the final time taken to be displayed (just to measure performance for different inputs). However I noticed that when I enter a really big value of N and assign only 1 or 2 threads, the scheduler seems to override the join functionality (i.e the last print statement is displayed before other threads end). Is the kernel allowed to do this? Or do I have some bug in my code?
P.S: I have only shown a part of my code. I have a similar System.out.println at the end of the function that the newly forked threads call.
Your loop is the problem.
for(int i=1;i<number_threads;i++)
{
mt[i-1].join();
}
Either you change the condition to <= or you make a less cryptic loop like this:
for(int i=0; i < number_threads;i++){
mt[i].join();
}
Or a for each loop:
for(MakeThread thread : mt)
thread.join();
Provided you correct your loop which calls join on all threads as shown below
for(int i=0;i<number_threads;i++)
{
mt[i].join();
}
there is no way that the last print line may get invoked before all threads ( as specified in the loop ) finish running and join the main thread. Scheduler cannot make any assumptions with this semantics. As pointed by Thomas , the bug is there in your code that does not call join on the last thread ( which therefore does not complete before the last print is called ).