We Keep Coding

Random for-loop in Java?

I have 25 batch jobs that are executed constantly, that is, when number 25 is finished, 1 is immediately started.
These batch jobs are started using an URL that contains the value 1 to 25. Basically, I use a for loop from 1 to 25 where I, in each round, call en URL with the current value of i, http://batchjobserver/1, http://batchjobserver/2 and so on.
The problem is that some of these batch jobs are a bit unstable and sometimes crashes which causes the for-loop to restart at 1. As a consequence, batch job 1 is run every time the loop is initiated while 25 runs much less frequently.
I like my current solution because it is so simple (in pseudo code)
for (i=1; i < 26; i++) {
getURL ("http://batchjob/" + Integer.toString(i));
}
However, I would like I to be a random number between 1 and 25 so that, in case something crashes, all the batch jobs, in the long run, are run approximately the same number of times.
Is there some nice hack/algorithm that allows me to achieve this?
Other requirements:
The number 25 changes frequently
This is not an absolut requirement but it would be nice one batch job wasn't run again until all other all other jobs have been attempted once. This doesn't mean that they have to "wait" 25 loops before they can run again, instead - if job 8 is executed in the 25th loop (the last loop of the first "set" of loops), the 26th loop (the first loop in the second set of loops) can be 8 as well.
Randomness has another advantage: it is desirable if the execution of these jobs looks a bit manual.

To handle errors, you should use a try-catch statement. It should look something like this:
for(int i = 1, i<26, i++){
try{
getURL();
}
catch (Exception e){
System.out.print(e);
}
}
This is a very basic example of what can be done. This will, however, only skip the failed attempts, print the error, and continue to the next iteration of the loop.

There are two parts of your requirement:
Randomness: For this, you can use Random#nextInt.
Skip the problematic call and continue with the remaining ones: For this, you can use a try-catch block.
Code:
Random random = new Random();
for (i = 1; i < 26; i++) {
try {
getURL ("http://batchjob/" + Integer.toString(random.nextInt(25) + 1));
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
Note: random.nextInt(25) returns an int value from 0 to 24 and thus, when 1 is added to it, the range becomes 1 to 25.

You could use a set and start randomizing numbers in the range of your batches, while doing this you will be tracking which batch you already passed by adding them to the set, something like this:
int numberOfBatches = 26;
Set<Integer> set = new HashSet<>();
List<Integer> failedBatches = new ArrayList<>();
Random random = new Random();
while(set.size() <= numberOfBatches)
{
int ran = random.nextInt(numberOfBatches) + 1;
if(set.contains(ran)) continue;
set.add(ran);
try
{
getURL ("http://batchjob/" + Integer.toString(ran));
} catch (Exception e)
{
failedBatches.add(ran);
}
}
As an extra, you can save which batches failed

The following is an example of a single-threaded, infinite looping (also colled Round-robin) scheduler with simple retry capabilities. I called "scrape" the routine that calls your batch job (scraping means indexing a website contents):
public static void main(String... args) throws Exception {
Runnable[] jobs = new Runnable[]{
() -> scrape("https://www.stackoverfow.com"),
() -> scrape("https://www.github.com"),
() -> scrape("https://www.facebook.com"),
() -> scrape("https://www.twitter.com"),
() -> scrape("https://www.wikipedia.org"),
};
for (int i = 0; true; i++) {
int remainingAttempts = 3;
while (remainingAttempts > 0) {
try {
jobs[i % jobs.length].run();
break;
} catch (Throwable err) {
err.printStackTrace();
remainingAttempts--;
}
}
}
}
private static void scrape(String website) {
System.out.printf("Doing my job against %s%n", website);
try {
Thread.sleep(100); // Simulate network work
} catch (InterruptedException e) {
throw new RuntimeException("Requested interruption");
}
if (Math.random() > 0.5) { // Simulate network failure
throw new RuntimeException("Ooops! I'm a random error");
}
}
You may want to add multi-thread capabilities (that is achieved by simply adding an ExecutorService guarded by a Semaphore) and some retry logic (for example only for certain type of errors and with a exponential backoff).

Java: Randomly generated numbers only appear in small part of given range

I'm working on a game and I want my enemies to spawn with a delay between 1-5 seconds. My code for that part looks like this:
#Override
public void run() {
try {
while (true) {
Random r = new Random();
int cooldownTime = r.nextInt((5000 - 1000) + 1) + 1000;
long lastSpawn = 0;
long time = System.currentTimeMillis();
if (time > lastSpawn + cooldownTime) {
System.out.println(cooldownTime);
addEnemies();
lastSpawn = time;
}
If I understand nextInt correctly this should spawn enemies 1000-5000 ms apart every time, but my results are really weird and I can't quite figure out why. This is an example of what it would look like if I print cooldownTime.
2523
1190
1095
1061
1168
1119
1052
1159
1071
1076
1000
1394
1249
1070
And so on... It seems that the first enemy is truly spawned randomly and the others are always in the low 1000's. This happens every time. Does anyone know why it's like that? I'm quite lost.

Calling Random r = new Random(); repeatedly is extremely pathological and ruins the statistical properties of the generator. (The results you get are most likely a strong function of your system clock time.)
Do that step once and your results will be far better. (There are also more efficient ways to implement a delay than this - consider sleep - Java compilers are not yet optimising out burn loops.)
Note also that your observations are further complicated by the fact that you are not printing every number drawn.

Not sure if this your exact issue, but you had some problems with the logic of System.currentTimeMillis() and were just looking at the total current time and not the difference between the time.
Additionally you only want to look for a new random value once the if actually executes so you do not generate a Random number each loop iteration, but rather each time an enemy spawns you generate a new one for the next enemy.
Here is the modified code that takes all of this into account:
Random r = new Random();
long time = System.currentTimeMillis();
int cooldownTime = r.nextInt((5000 - 1000) + 1) + 1000;
while (true) {
long timeDifference = System.currentTimeMillis() - time;
if (timeDifference > cooldownTime) {
//addEnemies();
System.out.println(timeDifference); //Prints the time taken for enemy to spawn
cooldownTime = r.nextInt((5000 - 1000) + 1) + 1000;
time = System.currentTimeMillis(); //set the initial time again
}
}
This will generate a random number between 1000 and 5000 and execute the if block each time after the delay, resetting the values in the if to do it forever.
However, there are most likely better ways to add a delay to your logic like some comments pointed out(sleep), but this is the corrected logic for your method.

Well, first of all, several of your variables are being created INSIDE the loop. You need to move the creation of the Random object, as well as the lastSpawn and cooldown variables outside the loop. This is because the lastSpawn variable is being overwritten each time the loop executes with 0, meaning you're always checking if the current time is greater than 0. You need to store it outside the loop so that it will retain the last value you assigned to it. For the same reason, cooldown needs to be outside the loop because you're generating a new cooldown every loop, and System.currentTimeMillis() is ALWAYS going to be larger than it because System.currentTimeMillis() gets you the system time offset from January 1, 1970. Finally, as System.currentTimeMillis() is represented by a long, you'll want any time-related variables to be long as well, otherwise you could end up overflowing your variable if the current time in milliseconds is too high for an integer to store.
Here is a better way to achieve what you're looking to do:
import java.util.Random;
public class RandomCooldown {
public static void main(String [] args) {
Random rand = new Random();
long start = System.currentTimeMillis();
long lastSpawn = start;
long cooldown = getCooldown(rand);
while(true) {
long time = System.currentTimeMillis();
long elapsed = (time - lastSpawn);
if(elapsed >= cooldown) {
System.out.println("Adding enemies!");
cooldown = getCooldown(rand); // only generate a new cooldown once the old cooldown has been surpassed
lastSpawn = time;
}
}
}
public static long getCooldown(Random rand) {
return (long)((rand.nextInt(4000) + 1) + 1000);
}
}
Hope this helps!

Here is a suggestion that works as I think you like. I has not one but two loops. The outer loop generates new enemies and the inner loop makes updates (not sure how much of that this enemy generating thread needs to update but included it just in case).
public void run() {
Random r = new Random();
// setup
while (true) {
int wait = r.nextInt((5000 - 1000) + 1) + 1000;
long time = System.currentTimeMillis();
System.out.println("Adding enemies at " + time
+ ", next add roughly in " + wait + " ms.");
while (wait + time > System.currentTimeMillis()) {
try {
Thread.sleep(30);
} catch (InterruptedException e) {
e.printStackTrace();
}
// update
}
}
}
The Random object is reused and the call Thread.sleep(30) makes the thread hand over control for 30 ms, a time during which the CPU can do more useful things than busy wait. (30 is BTW not a magic number that must be used but just a number I chose. You need to experiment and find what number works best in your game.)
The print out shows what happens.
Adding enemies at 1565096018610, next add roughly in 2890 ms.
Adding enemies at 1565096021530, next add roughly in 2301 ms.
Adding enemies at 1565096023863, next add roughly in 4944 ms.
Adding enemies at 1565096028813, next add roughly in 3042 ms.
Adding enemies at 1565096031879, next add roughly in 2661 ms.
... and so on. The actual numbers will not be the same of course when you run this code but similar.
Hope this helps and good luck with your game!

It looks fine to me. You should probably use the same Random() instance for each iteration. And remember: humans have no ability to perceive randomness. Alternatively, you could try seeding the Random-Generator (using the Random(long seed) constructor), just in case there's some weird stuff happening with your seed.

Probabilistically Selecting A Set Number of Items Over Time

I have an enterprise application running on a server that accepts files. Tens of thousands of files are submitted every day by users. The customer wants exactly 50 of these files to be automatically selected for audit each day.
The requirements are:
the files must be selected as they come in (we can't wait for all the files to come in and then choose 50 at the end of the day)
the files selected must meet some other criteria, which they haven't told me yet, but I am assured there will still be thousands of files that meet these criteria
the system must not be "game-able". That is - they don't want users who submit files to realise that if they wait until the afternoon or something, their files never get audited. This means we can't just choose the first 50 that come in, the selected files must be randomly spread out throughout the day.
we have to have EXACTLY 50 files. Or so they say. But I'm pretty sure if it just so happened that no user submitted a file that matched the criteria after midday one day, and we only got 25 files, they'd be ok with that. So I can assume that the types of files I'm interested in are submitted with a reasonably regular frequency throughout the day.
I figure then, that I need some function that calculates a probability that a file will be selected, that uses the number of currently chosen files and the time of day as inputs.
I've created a test harness. Please forgive the dodgy code. In this, the "pushTask" thread simulates files coming in by adding them to a stack. "Files" in this test are just Strings with a random number on the end.
The "pullTask" thread simulates files being pulled off the stack. It asks requirementsFunction() if the "file" meets the extra requirements needed (and in this test that's just - does it end in a zero), and it asks probabilityFunction() if it should select the file. If a file is selected, it is printed to System.out.
Really I need some help as to what to put in probabilityFunction(), because at the moment what's in there is garbage (I've left it in so you can see what I've tried). Or if someone knows of a mathematical probability function that uses items/time that would be great too.
package com.playground;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayDeque;
import java.util.Deque;
import java.util.Random;
public class ProbabilisticSelection {
private static int TOTAL_FILES = 1000;
private static int AUDIT_QUANTITY = 10;
private static int TIME_IN_SECONDS_FOR_ALL_FILES = 10;
private Random random = new Random();
private Deque<String> stack = new ArrayDeque<String>();
private boolean finished;
public static void main(String[] args) throws InterruptedException {
new ProbabilisticSelection().test();
}
private void test() throws InterruptedException {
Instant begin = Instant.now();
Runnable pushTask = () -> {
while (!finished) {
int next = random.nextInt(TOTAL_FILES);
String item = "File: " + next;
stack.push(item);
if (Duration.between(begin, Instant.now()).getSeconds() >= TIME_IN_SECONDS_FOR_ALL_FILES) {
finished = true;
}
try {
Thread.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
Runnable pullTask = () -> {
int itemNumber = 1;
while (itemNumber <= AUDIT_QUANTITY && !finished) {
String poll = stack.poll();
if (requirementsFunction(poll) &&
probabilityFunction(itemNumber, Duration.between(begin, Instant.now()))) {
System.out.println(itemNumber++ + ": "+ poll);
}
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
finished = true;
Duration delta = Duration.between(begin, Instant.now());
System.out.println();
System.out.println("Retrieved files: " + (itemNumber - 1) + ", should be, " + AUDIT_QUANTITY);
System.out.println("Time taken: " + delta.getSeconds() + ", should be, " + TIME_IN_SECONDS_FOR_ALL_FILES);
};
new Thread(pullTask).start();
new Thread(pushTask).start();
}
private boolean requirementsFunction(String item) {
return item != null && item.endsWith("0");
}
private boolean probabilityFunction(int itemNumber, Duration delta) {
double limit = ((double)(AUDIT_QUANTITY-itemNumber)/(double)AUDIT_QUANTITY + 1); // probability goes down as number of items goes up
double tension = (double)TIME_IN_SECONDS_FOR_ALL_FILES/((double)delta.getSeconds() + 1); // probablity goes up as time nears the end
if (tension == 1) {
return true;
}
double prob = limit * tension * 100;
int rand = random.nextInt(1000);
return prob > rand;
}
}

Algorithm is called Reservoir_sampling, which guarantees fair sampling of k items from some large and unknown N. Hereis Java code

Parallelize calculations

I need to calculate the mean and extract the root of some numbers from a huge file:
1, 2, 3, 4, 5,\n
6, 7, 8, 9, 10,\n
11, 12, 13, 14,15,\n
...
This is the code:
import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Scanner;
public class App1{
int res, c;
double mean, root;
ArrayList list = new ArrayList();
public App1() {
// einlesen
Scanner sc = null;
try {
sc = new Scanner(new File("file.txt")).useDelimiter("[,\\s]+");
} catch (FileNotFoundException ex) {
System.err.println(ex);
}
while (sc.hasNextInt()) {
list.add(sc.nextInt());
res += (int) list.get(c);
c++;
}
sc.close();
// Mean
mean = res / list.size();
// Root
root = Math.sqrt(mean);
System.out.println("Mean: " + mean);
System.out.println("Root: " + root);
}
public static void main(String[] args) {
App1 app = new App1();
}
}
Is there any way to parallelize it?
Before calculating the mean I need all the numbers, so one thread can't calculate while another is still fetching the numbers from the file.
The same thing with extracting the root: A thread can't extract it from the mean if the mean isn't calculated yet.
I thought about Future, would that be a solution?

There's something critical you will have to accept up front - you will not be able to process the data any faster than you can read it from the file. So first time how long it will take to read through the whole file and accept that you won't improve on that.
That said - have you considered a ForkJoinPool.

You can calculate the mean in parallel, because the mean is simply the sum divided by the count. There is no reason why you cannot sum up the values in parallel, and count them as well, then just do the division later.
Consider a class:
public class PartialSum() {
private final int partialcount;
private final int partialsum;
public PartialSum(int count, int sum) {
partialsum = sum;
partialcount = count;
public int getCount() {
return partialcount;
}
public int getSum() {
return partialsum;
}
}
Now, this could be the return type of a Future, as in Future<PartialSum>.
So, what you need to do is split the file in parts, and then send the parts to individual threads.
Each thread calculates a PartialSum. Then, as the threads complete, you can:
int sum = 0;
int count = 0;
for(Future<PartialSum> partial : futures) {
PartialSum ps = partial.get();
sum += ps.getSum();
count += ps.getCount();
}
double mean = (double)sum / count;
double root = ....

I think it's possible.
int offset = (filesize / Number of Threads)
Create n threads
Each thread starts reading from offset * thread number. Eg Thread 0 starts reading from byte 0, thread 1 starts reading from offset * 1, thread 2 starts reading from offset * 2
If thread num != 0, read ahead until you hit a newline character - start from there.
Add up an average per thread. Save to "thread_average" or something.
When all threads are finished, total average = average of all "thread_average" variables
Square root the total average variable.
It will need a bit of messing around to make sure the threads don't read too far into another threads block of the file, but should be do-able

No there is no way to parallelize this. Although you could do something that looks like you are using threading, the result will be overly complex but still run at about the same speed as before.
The reason for this is that file access is and has to be single-threaded, and beside reading from file all you do is two add operations. So in best case those add operations could be parallelized, however since those take almost no execution time, the gain would be like 5% - 10% at best. And that time is negated (or worse) by the thread creation and maintenance.
Once thing you can do to speed things up would be to remove the part where you put things into a list (assuming that you don't need those values later).
while (sc.hasNextInt()) {
res += sc.nextInt();
++c;
}
mean = res / c;

Implement a TimeWindowBuffer of limited size

I'd an interview yesterday. I couldn't figure out a solution to one programming problem and I'd like to get some ideas here. The problem is:
I need to implement a TimeWindowBuffer in Java, which stores the number a user continuously receives as time goes on. The buffer has a maxBufferSize. The user wants to know the average value of the past several seconds, a timeWindow passed in by user (so this is a sliding window). We could get the current time from the system (e.g. System.currentTimeMills() in Java). The TimeWindowBuffer class is like this:
public class TimeWindowBuffer {
private int maxBufferSize;
private int timeWindow;
public TimwWindowBuffer(int maxBufferSize, int timeWindow) {
this.maxBufferSize = maxBufferSize;
this.timeWindow = timeWindow;
}
public void addValue(long value) {
...
}
public double getAvg() {
...
return average;
}
// other auxiliary methods
}
Example:
Say, a user receive a number every second (the user may not receive a number at a certain rate) and wants to know the average value of the past 5 seconds.
Input:
maxBufferSize = 5, timeWindow = 5 (s)
numbers={-5 4 -8 -8 -8 1 6 1 8 5}
Output (I list the formula here for illustration but the user only needs the result)
:
-5 / 1 (t=1)
(-5 + 4) / 2 (t=2)
(-5 + 4 - 8) / 3 (t=3)
(-5 + 4 - 8 - 8) / 4 (t=4)
(-5 + 4 - 8 - 8 - 8) / 5 (t=5)
(4 - 8 - 8 - 8 + 1) / 5 (t=6)
(-8 - 8 - 8 + 1 + 6) / 5 (t=7)
(-8 - 8 + 1 + 6 + 1) / 5 (t=8)
(-8 + 1 + 6 + 1 + 8) / 5 (t=9)
(1 + 6 + 1 + 8 + 5) / 5 (t=10)
Since the data structure of the TimeWindowBuffer is not specified, I've been thinking about keeping a pair of value and its added time. So my declaration of underlying buffer is like this:
private ArrayList<Pair> buffer = new ArrayList<Pair>(maxBufferSize);
where
class Pair {
private long value;
private long time;
...
}
Since the Pair is added in time order, I could do a binary search on the list and calculate the average of the numbers that fall into the timeWindow. The problem is the buffer has a maxBufferSize (although ArrayList doesn't) and I have to remove the oldest value when the buffer is full. And that value could still satisfy the timeWindow but now it goes off the record and I will never know when it expires.
I'm stuck here for the current.
I don't need a direct answer but have some discussion or ideas here. Please let me now if there are any confusions about the problem and my description.

I enjoy little puzzles like this. I did not compile this code, nor did I take into account all the things you would have to for production usage. Like I did not design a way to set a missed value to 0 - i.e. if a value does not come in at every tick.
But this will give you another way to think of it....
public class TickTimer
{
private int tick = 0;
private java.util.Timer timer = new java.util.Timer();
public TickTimer(double timeWindow)
{
timer.scheduleAtFixedRate(new TickerTask(),
0, // initial delay
Math.round(1000/timeWindow)); // interval
}
private class TickerTask extends TimerTask
{
public void run ()
{
tick++;
}
}
public int getTicks()
{
return tick;
}
}
public class TimeWindowBuffer
{
int buffer[];
TickTimer timer;
final Object bufferSync = new Object();
public TimeWindowBuffer(int maxBufferSize, double timeWindow)
{
buffer = new int[maxBufferSize];
timer = TickTimer(timeWindow);
}
public boolean add(int value)
{
synchronize(bufferSync)
{
buffer[timer.getTicks() % maxBufferSize] = value;
}
}
public int averageValue()
{
int average = 0;
synchronize(bufferSync)
{
for (int i: buffer)
{
average += i;
}
}
return average/maxBufferSize;
}
}

Your question could be summarized as using constant memory to compute some statistics on a stream.
To me it's a heap (priority queue) with time as the key and value as the value, and least time on the top.
When you receive a new (time,value), add it to the heap. If the heap size is greater than the buffer size, just remove the root node in the heap, until the heap is small enough.
Also by using a heap you can get the minimum time in the buffer (i.e. the heap) in O(1) time, so just remove the root (the node with the minimum time) until all out-dated pairs are cleared.
For statistics, keep an integer sum. When you add a new pair to the heap, sum = sum + value of pair. When you remove the root from the heap, sum = sum - value of root.