We Keep Coding

Issue implementing a recursive shuffle array method in nlog(n) time

I'm trying to implement a recursive shuffle method that will take nlog(n) time but am having major difficulty, given I am restricted to a single parameter and can't include the size of n in my params when calling the recursive method. I looked at the structure of a similar shuffling problem with different parameters and got the idea for how to structure the shuffling using Random, but I've spent way too much time trying to figure out how to implement this.
If I could implement the method taking two parameters, a and n, I don't feel like I would be having so much trouble. My question is, should I have n be a global variable for the purpose of decrementing it? How would I go about either decrementing n for the purpose of managing the recursion, or modify a somehow to eventually end the recursion?
*Also, would declaring the objects in the indices like this cause a referential issue? If so, how would I fix this? I tried clone and couldn't get it to function within the requirements of the problem.
public static void shuffle(Object[] a) {
int n = a.length;
if (n == 0) {
return;
}
int d = (int) (Math.random() * (n-1)); //random index
Object c = a[n - 1]; //value at n-1
a[n-1]= a[d]; //a[n-1] index = val at rand index
a[d] = c; //val at rand index set to val at n-1
shuffle(a);
shuffle(a);
}
***What I've started to implement which looks a lot more like it would work in nlogn time complexity, but not sure if it's right or how it would need to be finished...
public static void shuffle(Object[] a){
if(a.length == 1) return; //return if length = 1: base case
Object[] b = new Object[a.length/2]();
Random rand = new Random(0,a.length); //random index to swap
for(int i = 0; i < b.length; i++){
b[i] = a[rand]; //how do I make sure no index of a is
//repeated?
}
shuffle(b); //recursively call shuffle on b, dividing size by 2
for(int i = 0; i < a.length; i++){
a[i] = b[i]; //copy values from b to a (I guess you could use arraycopy)
}
}

If I could implement the method taking two parameters, a and n, I don't feel like I would be having so much trouble. My question is, should I have n be a global variable for the purpose of decrementing it?
It does not have to be a global variable. You can create a class inside your method as follows. This should support any requirement that the implementation must be kept inside the publicly available method. Of course, you'll need to decide if this is in accordance with any other restrictions not mentioned in your question. I leave the actual algorithm up to you. But the array can be shuffled in-place in O(n).
public static void shuffle(Object[] a) {
class Shuffle {
private void recShuffle(Object[] ob, int n) {
// your shuffle algorithm here
}
}
Shuffle s = new Shuffle();
int n = ... // up to you
s.recShuffle(a, n);
}

So I was able to figure out a way to solve it this morning by implementing a helper method and moving most of my code there in case anyone ever comes across a problem like this:
//calls helper shuffle method
public static void shuffle(Object[] a) {
int n = a.length;
helperShuffle(a, n);
}
public static Object[] helperShuffle(Object[] a, int n) {
if (n <= 1)
return a; //base case: size of array is 1
Random rand = new Random(); //declare new random
int d = (rand.nextInt(n)); //random index < n
Object c = a[n - 1]; //value at n-1
a[n-1]= a[d]; //a[n-1] index = val at rand index
a[d] = c; //val at rand index set to val at n-1
return helperShuffle(a,n-1);
}

Finding the sum of common elements between n number of arrays in java

I have a program that sums the common elements of two arrays. For that I used two for loops and if I have three then I could use three for loops. But how to sum the common elements of n number of arrays where n is coming during run time.
I don't know how to change the number of loops during run time or is there any other relevant concept for this ?
Here is the code I've tried for summing twoarrays:
import java.util.Scanner;
public class Sample {
public static void main(String... args)
{
Scanner sc=new Scanner(System.in);
int arr1[]={1,2,3,4,5},arr2[]={4,5,6,7,8},sum=0;
for (int i=0;i<arr1.length;i++)
{
for (int j=0;j<arr2.length;j++)
{
if (arr1[i]==arr2[j])
{
sum+=(arr1[i]);
}
}
}
}
}

There can be different implementation for that. You can use the following approach. Here is the pseudo code
use a 2D array to store the array. if the number of array is n and size is m then the array will be input[n][m]
Use a ArrayList commonItems to store the common items of. Initiate it with the elements of input[0]
Now iterate through the array for i = 1 to n-1. compare with every input[i], store only the common items of commonItems and input[i] at each step. You can do it by converting the input[i] into a list and by using retainAll method.
At the end of the iteration the commonItem list will contains the common numbers only. Now sum the value of this list.

There is actually a more general method, that also answers the question "how to change the number of loops during run time?".
The general question
We are looking for a way to implement something equivalent to this:
for (i1 = 0; i1 < k1; i1++) {
for (i2 = 0; i2 < k2; i2++) {
for (i3 = 0; i3 < k3; i3++) {
...
for (in = 0; in < kn; in++) {
f(x1[i1], x2[i2], ... xn[in]);
}
...
}
}
}
where, n is given at runtime and f is a function taking a list of n parameters, processing the current n-tuple.
A general solution
There is a general solution, based on the concept of recursion.
This is one implementation that produces the desired behavior:
void process(int idx, int n, int[][] x, int[] k, Object[] ntuple) {
if (idx == n) {
// we have a complete n-tuple,
// with an element from each of the n arrays
f(ntuple);
return;
}
// this is the idx'th "for" statement
for (int i = 0; i < k[idx]; i++) {
ntuple[idx] = x[idx][i];
// with this recursive call we make sure that
// we also generate the rest of the for's
process(idx + 1, n, x, k, ntuple);
}
}
The function assumes that the n arrays are stored in a matrix x, and the first call should look like this:
process(0, n, x, k, new Object[n]);
Practical considerations
The solution above has a high complexity (it is O(k1⋅k2⋅..⋅kn)), but sometimes it is possible to avoid going until the deepest loop.
Indeed, in the specific problem mentioned in this post (which requires summing common elements across all arrays), we can skip generating some tuples e.g. if already x2[i2] ≠ x1[i1].
In the recursive solution, those situations can easily be pruned. The specific code for this problem would probably look like this:
void process(int idx, int n, int[][] x, int[] k, int value) {
if (idx == n) {
// all elements from the current tuple are equal to "value".
// add this to the global "sum" variable
sum += value;
return;
}
for (int i = 0; i < k[idx]; i++) {
if (idx == 0) {
// this is the outer "for", set the new value
value = x[0][i];
} else {
// check if the current element from the idx'th for
// has the same value as all previous elements
if (x[idx][i] == value) {
process(idx + 1, n, x, k, value);
}
}
}
}

Assuming that the index of the element is not important: a[1] = 2 and a[5] = 2, you only need two nested loops.
First you need to put n-1 arrays in a list of sets. Then loop over nth array and check if each element exists in all of the sets in the list. If it does exist then add to total.

Parallel sum of elements in a large Array

I have program that sums the elements in a very large array. I want to parallelize this sum.
#define N = some_very_large_no; // say 1e12
float x[N]; // read from a file
float sum=0.0;
main()
{
for (i=0, i<N, i++)
sum=sum+x[i];
}
How can I parallelize this sum using threads (c/c++/Java any code example is fine)? How many threads should I use if there is 8 cores in the machine for optimal performance?
EDIT: N may be really large ( larger than 1e6 actually) and varies based on the file size I read the data from. The file is in the order of GBs.
Edit: N is changed to a large value (1e12 to 1e16)

In Java you can write
int cpus = Runtime.getRuntime().availableProcessors();
// would keep this of other tasks as well.
ExecutorService service = Executors.newFixedThreadPool(cpus);
float[] floats = new float[N];
List<Future<Double>> tasks = new ArrayList<>();
int blockSize = (floats.length + cpus - 1) / cpus;
for (int i=0, i < floats.length, i++) {
final start = blockSize * i;
final end = Math.min(blockSize * (i+1), floats.length);
tasks.add(service.submit(new Callable<Double>() {
public Double call() {
double d= 0;
for(int j=start;j<end;j++)
d += floats[j];
return d;
}
});
}
double sum = 0;
for(Future<Double> task: tasks)
sum += task.get();
As WhozCraig mentions, it is likely that one million floats isn't enough to need multiple threads, or you could find that your bottle neck is how fast you can load the array from main memory (a single threaded resource) In any case, you can't assume it will be faster by the time you include the cost getting the data.

You say that the array comes from a file. If you time the different parts of the program, you'll find that summing up the elements takes a negligible amount of time compared to how long it takes to read the data from disk. From Amdahl's Law it follows that there is nothing to be gained by parallelising the summing up.
If you need to improve performance, you should focus on improving the I/O throughput.

you can use many threads(more than cores). But no of threads & its performance depends on ur algorithm as how they are working.
As array length is 100000 then create x thread & each will calculate arr[x] to arr[x+limit]. where u have to set limit so that no overlapping with other thread & no element should remain un-used.
thread creation:
pthread_t tid[COUNT];
int i = 0;
int err;
while (i < COUNT)
{
void *arg;
arg = x; //pass here a no which will tell from where this thread will use arr[x]
err = pthread_create(&(tid[i]), NULL, &doSomeThing, arg);
if (err != 0)
printf("\ncan't create thread :[%s]", strerror(err));
else
{
//printf("\n Thread created successfully\n");
}
i++;
}
// NOW CALCULATE....
for (int i = 0; i < COUNT; i++)
{
pthread_join(tid[i], NULL);
}
}
void* doSomeThing(void *arg)
{
int *x;
x = (int *) (arg);
// now use this x to start the array sum from arr[x] to ur limit which should not overlap to other thread
}

Use divide and conquer algorithm
Divide the array into 2 or more (keep dividing recursively until you get an array with manageable size)
Start computing the sum for the sub arrays (divided arrays) (using separate threads)
Finally add the sum generated (from all the threads) for all sub arrays together to produce final result

As others have said, the time-cost of reading the file is almost certainly going to be much larger than that of calculating the sum. Is it a text file or binary? If the numbers are stored as text, then the cost of reading them can be very high depending on your implementation.
You should also be careful adding a large number of floats. Because of their limited precision, small values late in the array may not contribute to the sum. Think about at least using a double to accumulate the values.

You can use pthreads in c to solve your problem
Here is my code for N=4 ( you can change it to suit your needs )
To run this code, apply the following command :
gcc -pthread test.c -o test
./test
#include<stdio.h>
#include<stdlib.h>
#include<pthread.h>
#define NUM_THREADS 5
pthread_t threads[NUM_THREADS];
pthread_mutex_t mutexsum;
int a[2500];
int sum = 0;
void *do_work(void* parms) {
long tid = (long)parms;
printf("I am thread # %ld\n ", tid);
int start, end, mysum;
start = (int)tid * 500;
end = start + 500;
int i = 0;
printf("Thread # %ld with start = %d and end = %d \n",tid,start,end);
for (int i = start; i < end; i++) {
mysum += a[i];
}
pthread_mutex_lock(&mutexsum);
printf("Thread # %ld lock and sum = %d\n",tid,sum);
sum += mysum;
pthread_mutex_unlock(&mutexsum);
pthread_exit(NULL);
}
void main(int argv, char* argc) {
int i = 0; int rc;
pthread_attr_t attr;
pthread_mutex_init(&mutexsum, NULL);
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
pthread_mutex_init(&mutexsum, NULL);
printf("Initializing array : \n");
for(i=0;i<2500;i++){
a[i]=1;
}
for (i = 0; i < NUM_THREADS; i++) {
printf("Creating thread # %d.\n", i);
rc = pthread_create(&threads[i], &attr, &do_work, (void *)i);
if (rc) {
printf("Error in thread %d with rc = %d. \n", i, rc);
exit(-1);
}
}
pthread_attr_destroy(&attr);
printf("Creating threads complete. start ruun " );
for (i = 0; i < NUM_THREADS; i++) {
pthread_join(threads[i], NULL);
}
printf("\n\tSum : %d", sum);
pthread_mutex_destroy(&mutexsum);
pthread_exit(NULL);
}

OpenMP supports built-in reduction. Add flag -fopenmp while compiling.
#include <omp.h>
#define N = some_very_large_no; // say 1e12
float x[N]; // read from a file
int main()
{
float sum = 0.0;
#pragma omp parallel for reduction(+:sum)
for (i=0, i<N, i++)
sum=sum+x[i];
return 0;
}

Java - Threaded Radix Sort

I have been working on different variations of the Radix Sort. At first I used chaining, which was really slow. Then I moved onto using a count sort while using val % (10 * pass), and most recently turning it into the respective bytes and count sorting those, which allows me to sort by negative values also.
I wanted to try it with multithreading, and can only get it to work about half the time. I was wondering if someone can help look at my code, and see where I'm going wrong with the threading. I have each thread count sort each byte. Thanks:
public class radixSort {
public int[] array;
public int arraySize, arrayRange;
public radixSort (int[] array, int size, int range) {
this.array = array;
this.arraySize = size;
this.arrayRange = range;
}
public int[] RadixSort() {
Thread[] threads = new Thread[4];
for (int i=0;i<4;i++)
threads[i] = new Thread(new Radix(arraySize, i));
for (int i=0;i<4;i++)
threads[i].start();
for (int i=0;i<4;i++)
try {
threads[i].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
return array;
}
class Radix implements Runnable {
private int pass, size;
private int[] tempArray, freqArray;
public Radix(int size, int pass) {
this.pass = pass;
this.size = size;
this.tempArray = new int[size];
this.freqArray = new int[256];
}
public void run() {
int temp, i, j;
synchronized(array) {
for (i=0;i<size;i++) {
if (array[i] <= 0) temp = array[i] ^ 0x80000000;
else temp = array[i] ^ ((array[i] >> 31) | 0x80000000);
j = temp >> (pass << 3) & 0xFF;
freqArray[j]++;
}
for (i=1;i<256;i++)
freqArray[i] += freqArray[i-1];
for (i=size-1;i>=0;i--) {
if (array[i] <= 0) temp = array[i] ^ 0x80000000;
else temp = array[i] ^ ((array[i] >> 31) | 0x80000000);
j = temp >> (pass << 3) & 0xFF;
tempArray[--freqArray[j]] = array[i];
}
for (i=0;i<size;i++)
array[i] = tempArray[i];
}
}
}
}

There is a basic problem with this approach. To get a benefit from multithreading, you need to give each thread a non-overlapping task compared to the other treads. By synchonizing on the array you have made it so only one thread does work at a time, meaning you get all the overhead of threads with none of the benefit.
Think of ways to partition the task so that threads work in parallel. For example, after the first pass, all the item with a 1 high bit will be in one part of the array, and those with a zero high-bit will be in the other. You could have one thread work on each part of the array without synchronizing.
Note that your runnable has to completely change so that it does one pass at a specified subset of the array then spawns threads for the next pass.

Besides wrong class and method names (class should start with capital letter, method shouldn't), I can see that you are synchronizing all thread works on the array. So it's in fact not parallel at all.

I am pretty sure that you can't really parallelize RadixSort, at least in the way you are trying to. Someone pointed out that you can do it by divide-and-conquer, as you first order by the highest bits, but in fact, RadixSort works by comparing the lower-order bits first, so you can't really divide-and-conquer. The array can basically be completely permuted after each pass.
Guys, prove me wrong, but i think it's inherently impossible to parallelize this algorithm like you try to. Maybe you can parallelize the (count) sorting that is done inside of each pass, but be aware that ++ is not an atomic operation.

Improving a prime sieve algorithm

I'm trying to make a decent Java program that generates the primes from 1 to N (mainly for Project Euler problems).
At the moment, my algorithm is as follows:
Initialise an array of booleans (or a bitarray if N is sufficiently large) so they're all false, and an array of ints to store the primes found.
Set an integer, s equal to the lowest prime, (ie 2)
While s is <= sqrt(N)
Set all multiples of s (starting at s^2) to true in the array/bitarray.
Find the next smallest index in the array/bitarray which is false, use that as the new value of s.
Endwhile.
Go through the array/bitarray, and for every value that is false, put the corresponding index in the primes array.
Now, I've tried skipping over numbers not of the form 6k + 1 or 6k + 5, but that only gives me a ~2x speed up, whilst I've seen programs run orders of magnitudes faster than mine (albeit with very convoluted code), such as the one here
What can I do to improve?
Edit: Okay, here's my actual code (for N of 1E7):
int l = 10000000, n = 2, sqrt = (int) Math.sqrt(l);
boolean[] nums = new boolean[l + 1];
int[] primes = new int[664579];
while(n <= sqrt){
for(int i = 2 * n; i <= l; nums[i] = true, i += n);
for(n++; nums[n]; n++);
}
for(int i = 2, k = 0; i < nums.length; i++) if(!nums[i]) primes[k++] = i;
Runs in about 350ms on my 2.0GHz machine.

While s is <= sqrt(N)
One mistake people often do in such algorithms is not precomputing square root.
while (s <= sqrt(N)) {
is much, much slower than
int limit = sqrt(N);
while (s <= limit) {
But generally speaking, Eiko is right in his comment. If you want people to offer low-level optimisations, you have to provide code.
update Ok, now about your code.
You may notice that number of iterations in your code is just little bigger than 'l'. (you may put counter inside first 'for' loop, it will be just 2-3 times bigger) And, obviously, complexity of your solution can't be less then O(l) (you can't have less than 'l' iterations).
What can make real difference is accessing memory effectively. Note that guy who wrote that article tries to reduce storage size not just because he's memory-greedy. Making compact arrays allows you to employ cache better and thus increase speed.
I just replaced boolean[] with int[] and achieved immediate x2 speed gain. (and 8x memory) And I didn't even try to do it efficiently.
update2
That's easy. You just replace every assignment a[i] = true with a[i/32] |= 1 << (i%32) and each read operation a[i] with (a[i/32] & (1 << (i%32))) != 0. And boolean[] a with int[] a, obviously.
From the first replacement it should be clear how it works: if f(i) is true, then there's a bit 1 in an integer number a[i/32], at position i%32 (int in Java has exactly 32 bits, as you know).
You can go further and replace i/32 with i >> 5, i%32 with i&31. You can also precompute all 1 << j for each j between 0 and 31 in array.
But sadly, I don't think in Java you could get close to C in this. Not to mention, that guy uses many other tricky optimizations and I agree that his could would've been worth a lot more if he made comments.

Using the BitSet will use less memory. The Sieve algorithm is rather trivial, so you can simply "set" the bit positions on the BitSet, and then iterate to determine the primes.

Did you also make the array smaller while skipping numbers not of the form 6k+1 and 6k+5?
I only tested with ignoring numbers of the form 2k and that gave me ~4x speed up (440 ms -> 120 ms):
int l = 10000000, n = 1, sqrt = (int) Math.sqrt(l);
int m = l/2;
boolean[] nums = new boolean[m + 1];
int[] primes = new int[664579];
int i, k;
while (n <= sqrt) {
int x = (n<<1)+1;
for (i = n+x; i <= m; nums[i] = true, i+=x);
for (n++; nums[n]; n++);
}
primes[0] = 2;
for (i = 1, k = 1; i < nums.length; i++) {
if (!nums[i])
primes[k++] = (i<<1)+1;
}

The following is from my Project Euler Library...Its a slight Variation of the Sieve of Eratosthenes...I'm not sure, but i think its called the Euler Sieve.
1) It uses a BitSet (so 1/8th the memory)
2) Only uses the bitset for Odd Numbers...(another 1/2th hence 1/16th)
Note: The Inner loop (for multiples) begins at "n*n" rather than "2*n" and also multiples of increment "2*n" are only crossed off....hence the speed up.
private void beginSieve(int mLimit)
{
primeList = new BitSet(mLimit>>1);
primeList.set(0,primeList.size(),true);
int sqroot = (int) Math.sqrt(mLimit);
primeList.clear(0);
for(int num = 3; num <= sqroot; num+=2)
{
if( primeList.get(num >> 1) )
{
int inc = num << 1;
for(int factor = num * num; factor < mLimit; factor += inc)
{
//if( ((factor) & 1) == 1)
//{
primeList.clear(factor >> 1);
//}
}
}
}
}
and here's the function to check if a number is prime...
public boolean isPrime(int num)
{
if( num < maxLimit)
{
if( (num & 1) == 0)
return ( num == 2);
else
return primeList.get(num>>1);
}
return false;
}

You could do the step of "putting the corresponding index in the primes array" while you are detecting them, taking out a run through the array, but that's about all I can think of right now.

I wrote a simple sieve implementation recently for the fun of it using BitSet (everyone says not to, but it's the best off the shelf way to store huge data efficiently). The performance seems to be pretty good to me, but I'm still working on improving it.
public class HelloWorld {
private static int LIMIT = 2140000000;//Integer.MAX_VALUE broke things.
private static BitSet marked;
public static void main(String[] args) {
long startTime = System.nanoTime();
init();
sieve();
long estimatedTime = System.nanoTime() - startTime;
System.out.println((float)estimatedTime/1000000000); //23.835363 seconds
System.out.println(marked.size()); //1070000000 ~= 127MB
}
private static void init()
{
double size = LIMIT * 0.5 - 1;
marked = new BitSet();
marked.set(0,(int)size, true);
}
private static void sieve()
{
int i = 0;
int cur = 0;
int add = 0;
int pos = 0;
while(((i<<1)+1)*((i<<1)+1) < LIMIT)
{
pos = i;
if(marked.get(pos++))
{
cur = pos;
add = (cur<<1);
pos += add*cur + cur - 1;
while(pos < marked.length() && pos > 0)
{
marked.clear(pos++);
pos += add;
}
}
i++;
}
}
private static void readPrimes()
{
int pos = 0;
while(pos < marked.length())
{
if(marked.get(pos++))
{
System.out.print((pos<<1)+1);
System.out.print("-");
}
}
}
}
With smaller LIMITs (say 10,000,000 which took 0.077479s) we get much faster results than the OP.

I bet java's performance is terrible when dealing with bits...
Algorithmically, the link you point out should be sufficient

Have you tried googling, e.g. for "java prime numbers". I did and dug up this simple improvement:
http://www.anyexample.com/programming/java/java_prime_number_check_%28primality_test%29.xml
Surely, you can find more at google.

Here is my code for Sieve of Erastothenes and this is actually the most efficient that I could do:
final int MAX = 1000000;
int p[]= new int[MAX];
p[0]=p[1]=1;
int prime[] = new int[MAX/10];
prime[0]=2;
void sieve()
{
int i,j,k=1;
for(i=3;i*i<=MAX;i+=2)
{
if(p[i])
continue;
for(j=i*i;j<MAX;j+=2*i)
p[j]=1;
}
for(i=3;i<MAX;i+=2)
{
if(p[i]==0)
prime[k++]=i;
}
return;
}