It seems that when using ordered Streams to process a short-circuiting operation on a difficult to bound numeric range, parallel() cannot be used.
E.g.:
public class InfiniteTest {
private static boolean isPrime(int x) {
if (x < 2) {
return false;
}
if (x % 2 == 0 && x > 2) {
return false;
}
// loop while i <= sqrt(x), using multiply for speedup
for (int i = 3; i * i <= x; i += 2) {
if (x % i == 0) {
return false;
}
}
return true;
}
private static int findNthPrime(final int n) {
// must not use infinite stream, causes OOME
// but even big size causes huge slowdown
IntStream.range(1, 1000_000_000)
// .parallel()
.filter(InfiniteTest::isPrime)
.skip(n - 1)
.findFirst()
.getAsInt();
}
public static void main(String[] args) {
int n = 1000; // find the nth prime number
System.out.println(findNthPrime(n));
}
}
This sequential stream works fine. But when I add parallel(), it seems to run forever (or very long at last). I assume it's because the stream threads work on arbitrary numbers instead of starting with the first numbers in the stream. I cannot usefully bound the range of integers to scan for prime numbers.
So is there any simple trick to run this problem in parallel with streams without that trap, such as forcing the splititerator to serve chunks of work from the beginning of the stream? Or building the stream from substreams that cover increasing number ranges?
Or somehow setting up the multithreading as producer/consumer pattern but with streams?
Similar questions all just seem to try to discourage use of parallel:
Generate infinite parallel stream
Java 8, using .parallel in a stream causes OOM error
Java 8's streams: why parallel stream is slower?
Apart from 2 and 3, all prime numbers are of the form 6n-1 or 6n+1. You already treat 2 as a special case in your code. You might want to try also treating 3 as special:
if (x % 3 == 0) {
return x == 3;
}
And then run two parallel streams, one testing numbers of the form 6n-1, starting at 5, and the other testing numbers of the form 6n+1, starting at 7. Each stream can skip six numbers at a time.
You can use the Prime Number theorem to estimate the value of the nth prime and set the limit of your search slightly above that estimate for safety.
TL/DR: It is not possible.
It seems processing unbounded streams in parallel with a short-circuit method to find the earliest occurrences(in stream order) of anything is not possible in a useful way ("useful" meaning better than sequential in terms of time to find the result).
Explanation
I tried a custom implementation of AbstractIntSpliterator that splits the stream not in partitions (1-100, 101-200, ...) but instead splits them interleavingly ([0, 2, 4, 6, 8, ...], [1, 3, 5, 6 ...]). This works correctly in the sequential case:
/**
* Provides numbers starting at n, on split splits such that child iterator and
* this take provide interleaving numbers
*/
public class InterleaveSplitIntSplitIterator extends Spliterators.AbstractIntSpliterator {
private int current;
private int increment;
protected InterleaveSplitIntSplitIterator(int start, int increment) {
super(Integer.MAX_VALUE,
Spliterator.DISTINCT
// splitting is interleaved, not prefixing
// | Spliterator.ORDERED
| Spliterator.NONNULL
| Spliterator.IMMUTABLE
// SORTED must imply ORDERED
// | Spliterator.SORTED
);
if (increment == 0) {
throw new IllegalArgumentException("Increment must be non-zero");
}
this.current = start;
this.increment = increment;
}
#Override
public boolean tryAdvance(IntConsumer action) {
// Don't benchmark with this on
// System.out.println(Thread.currentThread() + " " + current);
action.accept(current);
current += increment;
return true;
}
// this is required for ORDERED even if sorted() is never called
#Override
public Comparator<? super Integer> getComparator() {
if (increment > 0) {
return null;
}
return Comparator.<Integer>naturalOrder().reversed();
}
#Override
public OfInt trySplit() {
if (increment >= 2) {
return null;
}
int newIncrement = this.increment * 2;
int oldIncrement = this.increment;
this.increment = newIncrement;
return new InterleaveSplitIntSplitIterator(current + oldIncrement, newIncrement);
}
// for convenience
public static IntStream asIntStream(int start, int increment) {
return StreamSupport.intStream(
new InterleaveSplitIntSplitIterator(start, increment),
/* no, never set parallel here */ false);
}
}
However, such streams cannot have the Spliterator.ORDERED characteristics, because
If so, this Spliterator guarantees that method
{#link #trySplit} splits a strict prefix of elements
and this also means such a stream cannot keep it's SORTED characteristics, because
A Spliterator that reports {#code SORTED} must also report {#code ORDERED}
So my splititerator in parallel ends up having (somewhat) jumbled numbers, which would have to be fixed by sorting before applying a limit, which does not work well with infinite streams (in the general case).
So all solutions to this must use a splititerator that splits in chunks or prefix data, which then are consumed in ~arbitrary order, which causes many number ranges beyond the actual result to be processed, becoming (much) slower in general than a sequential solution.
So other than bounding the number range to test, it seems there cannot be a solution using a parallel stream. The problem is in the specification requiring ORDERED characteristics to split a Stream by prefixing, instead of providing a different means of reassembling ordered stream results from multiple splititerators.
However a solution using a sequential stream with parallelly processed (buffered) inputs may still be possible (but not as simple as calling parallel()).
Related
class Generate // to print all numbers from 1000 to 9999 whose digits are in ascending order
{
private boolean order(int n, int i) // checks if the digits of given number are in ascending order
{
if (n == 0) return true;
if (n % 10 < i) return (order(n / 10, n % 10));
return false;
}
void show(int n) // recursive function to generate numbers from 1000 to 9999
{
if (n > 9999) // base case for recursor
System.out.print("");
else
{
if (order(n, 10)) // if digits are in ascending order, prints the number
System.out.println(n);
show(n + 1); // recursive call
}
}
}
The above code was supposed to print all numbers from 1000 to 9999. Code compiled and run but received a runtime exception: java.lang.StackOverflowError: null. Would a try-catch block fix my problem? This is my first time posting here hence I am not familiar with the question etiquette, please correct me if I'm wrong.
Java (and most of the programming languages) provides finite number of memory slot to be used for stack-frame, when your program/code reaches that limit it throws the StackOverFlow error.
In your case, your program reaches that limit before coming to base condition.
My take on this -
Do not use Recursive approach until and unless you are splitting your problem into acceptable sub-problems.
this shows you how you can increase stack-frame-size to be used by the JVM for a particular thread.
In my opinion StackOverFlow is due to double recursion i.e both show(), and order() are recursive... I think show() does not need to be recursive. modified your code as under:
public static void main(String[] args) {
/
new Generate().show(); // added main function to get output or exception.. you can omit it and call Generate().show() at the point u require.
}
}
class Generate//to print all numbers from 1000 to 9999 whose digits are in ascending order
{
private boolean order(int n, int i)//checks if the digits of given number are in ascending order
{
if(n==0)
return true;
if(n%10<i)
return(order(n/10,n%10));
return false;
}
void show()//function to generate numbers from 1000 to 9999 - WITHOUT Recursion
{
int n = 1000;
while(n<=9999)
{ if(order(n,10))//if digits are in ascending order, prints the number
System.out.println(n);
n++;
}
}
}
and here is output: (showing first 7 iterations only)
1234
1235
1236
1237
1238
1239
1245 .......
This is a problem I'm trying to solve on my own to be a bit better at recursion(not homework). I believe I found a solution, but I'm not sure about the time complexity (I'm aware that DP would give me better results).
Find all the ways you can go up an n step staircase if you can take k steps at a time such that k <= n
For example, if my step sizes are [1,2,3] and the size of the stair case is 10, I could take 10 steps of size 1 [1,1,1,1,1,1,1,1,1,1]=10 or I could take 3 steps of size 3 and 1 step of size 1 [3,3,3,1]=10
Here is my solution:
static List<List<Integer>> problem1Ans = new ArrayList<List<Integer>>();
public static void problem1(int numSteps){
int [] steps = {1,2,3};
problem1_rec(new ArrayList<Integer>(), numSteps, steps);
}
public static void problem1_rec(List<Integer> sequence, int numSteps, int [] steps){
if(problem1_sum_seq(sequence) > numSteps){
return;
}
if(problem1_sum_seq(sequence) == numSteps){
problem1Ans.add(new ArrayList<Integer>(sequence));
return;
}
for(int stepSize : steps){
sequence.add(stepSize);
problem1_rec(sequence, numSteps, steps);
sequence.remove(sequence.size()-1);
}
}
public static int problem1_sum_seq(List<Integer> sequence){
int sum = 0;
for(int i : sequence){
sum += i;
}
return sum;
}
public static void main(String [] args){
problem1(10);
System.out.println(problem1Ans.size());
}
My guess is that this runtime is k^n where k is the numbers of step sizes, and n is the number of steps (3 and 10 in this case).
I came to this answer because each step size has a loop that calls k number of step sizes. However, the depth of this is not the same for all step sizes. For instance, the sequence [1,1,1,1,1,1,1,1,1,1] has more recursive calls than [3,3,3,1] so this makes me doubt my answer.
What is the runtime? Is k^n correct?
TL;DR: Your algorithm is O(2n), which is a tighter bound than O(kn), but because of some easily corrected inefficiencies the implementation runs in O(k2 × 2n).
In effect, your solution enumerates all of the step-sequences with sum n by successively enumerating all of the viable prefixes of those step-sequences. So the number of operations is proportional to the number of step sequences whose sum is less than or equal to n. [See Notes 1 and 2].
Now, let's consider how many possible prefix sequences there are for a given value of n. The precise computation will depend on the steps allowed in the vector of step sizes, but we can easily come up with a maximum, because any step sequence is a subset of the set of integers from 1 to n, and we know that there are precisely 2n such subsets.
Of course, not all subsets qualify. For example, if the set of step-sizes is [1, 2], then you are enumerating Fibonacci sequences, and there are O(φn) such sequences. As k increases, you will get closer and closer to O(2n). [Note 3]
Because of the inefficiencies in your coded, as noted, your algorithm is actually O(k2 αn) where α is some number between φ and 2, approaching 2 as k approaches infinity. (φ is 1.618..., or (1+sqrt(5))/2)).
There are a number of improvements that could be made to your implementation, particularly if your intent was to count rather than enumerate the step sizes. But that was not your question, as I understand it.
Notes
That's not quite exact, because you actually enumerate a few extra sequences which you then reject; the cost of these rejections is a multiplier by the size of the vector of possible step sizes. However, you could easily eliminate the rejections by terminating the for loop as soon as a rejection is noticed.
The cost of an enumeration is O(k) rather than O(1) because you compute the sum of the sequence arguments for each enumeration (often twice). That produces an additional factor of k. You could easily eliminate this cost by passing the current sum into the recursive call (which would also eliminate the multiple evaluations). It is trickier to avoid the O(k) cost of copying the sequence into the output list, but that can be done using a better (structure-sharing) data-structure.
The question in your title (as opposed to the problem solved by the code in the body of your question) does actually require enumerating all possible subsets of {1…n}, in which case the number of possible sequences would be exactly 2n.
If you want to solve this recursively, you should use a different pattern that allows caching of previous values, like the one used when calculating Fibonacci numbers. The code for Fibonacci function is basically about the same as what do you seek, it adds previous and pred-previous numbers by index and returns the output as current number. You can use the same technique in your recursive function , but add not f(k-1) and f(k-2), but gather sum of f(k-steps[i]). Something like this (I don't have a Java syntax checker, so bear with syntax errors please):
static List<Integer> cache = new ArrayList<Integer>;
static List<Integer> storedSteps=null; // if used with same value of steps, don't clear cache
public static Integer problem1(Integer numSteps, List<Integer> steps) {
if (!ArrayList::equal(steps, storedSteps)) { // check equality data wise, not link wise
storedSteps=steps; // or copy with whatever method there is
cache.clear(); // remove all data - now invalid
// TODO make cache+storedSteps a single structure
}
return problem1_rec(numSteps,steps);
}
private static Integer problem1_rec(Integer numSteps, List<Integer> steps) {
if (0>numSteps) { return 0; }
if (0==numSteps) { return 1; }
if (cache.length()>=numSteps+1) { return cache[numSteps] } // cache hit
Integer acc=0;
for (Integer i : steps) { acc+=problem1_rec(numSteps-i,steps); }
cache[numSteps]=acc; // cache miss. Make sure ArrayList supports inserting by index, otherwise use correct type
return acc;
}
I got a nightmare on threads on solving Factorial using multithreading I got a simple idea like why dont we use quicksort technique to solve this i am talking about divide and conquer...
for example my explanation in code format...
For suppose if i got to find factorial for 8 let us confine a parition to only 2 parts
thread1 solving factorial n to n/2
8*fact(7)
8*7*fact(6)
8*7*6*fact(5)
8*7*6*5*fact(4)
thread2 calculates fact(4)
now my problem i dont know how to solve is how to remove fact(4) from thread1 so that i just finish off my answer as
result=thread1output*thread2output
Please provide code with comments please Hoping to get positive response from you..Thanks for going through till end
If you want to attack factorial with multiple threads, you need to partition the series into units of work. With 2 threads, a simplistic approach is to divide the series in 2.
For example 8!
thread 1 8x7x6x5
thread 2 4x3x2x1
One way is to allocate the work in an outer method, passing the range into each thread then multiplying the result.
Actually, you can do this with the stream API:
long factorial = LongStream.rangeClosed(1, n).parallel().reduce((a, b) -> a * b).get();
This can process the calculation in parallel.
For n larger than 20, the result will exceed the largest value a long can hold; use BigInteger for the arithmetic instead:
BigInteger factorial = IntStream.rangeClosed(1, n)
.parallel()
.mapToObj(String::valueOf)
.map(BigInteger::new)
.reduce((a, b) -> a.multiply(b))
.get();
A generally applicable optimization for when one computation depends on the results of a previous computation is a memoization. Rather then simply recursing, check if the value has already been computed; and if so return the previously computed value. Something like,
private static Map<Integer, BigInteger> memo = new HashMap<>();
static {
memo.put(1, BigInteger.ONE);
}
public static BigInteger fact(int i) {
if (memo.containsKey(i)) {
return memo.get(i);
}
BigInteger result = BigInteger.valueOf(i).multiply(fact(i - 1));
memo.put(i, result);
return result;
}
Elliott is correct in his comment that multi-threading is not a good way to optimize the calculation of factorials. Certainly, it won't be effective (compared with memoization for example) until you calculate really large factorials.
In simple recursive divide and conquer strategy, the pseudo-code is
factorial(N) =
return factorialHelper(set_of(1, N))
factorialHelper(set) =
if set.size == 1
return set.first
else
let set1, set2 = split_into_subsets(set, 2)
return factorialHelper(set1) * factorialHelper(set2)
Now we can naively use threads like this:
factorialHelper(set) =
if set.size == 1
return set.first
else
let set1, set2 = split_into_subsets(set, 2)
return fork(lambda factorialHelper(set1)).join() *
fork(lambda factorialHelper(set2)).join()
(Explanation: fork(lambda factorialHelper(set1)).join() is supposed to mean that we are:
- creating a new thread to calculate factorialHelper(set1),
- starting the thread,
- waiting for it to deliver a result.)
Here's the problem. Each thread is doing a bunch of housekeeping (e.g. splitting the set) and then a single multiplication. And if you look at the big picture, that means N! requires roughly N threads.
We can do better. Rather than waiting for two subthreads to complete. The "current" thread could do the work of one of them; e.g.
factorialHelper(set) =
if set.size == 1
return set.first
else
let set1, set2 = split_into_subsets(set, 2)
let child = fork(lanbda factorialHelper(set1))
return factorialHelper(set2) * child.join()
But that still requires roughly N / 2 threads. And there are bigger problems:
Creating threads is very expensive.
Thread switching / waiting for another thread to finish is relatively expensive.
At any given time, your computer will only run as many threads as it has physical (or hyperthread) cores.
So if you were to code the above literally in Java, it would run like a dog. Too many threads. Too much thread creating / switching overhead and too little useful work for each thread.
If you use the Java Fork / Join Pool, you can do a lot better, but you also need to do something about the split_into_subsets(set, 2) step. That is going to do a lot of superfluous work copying set elements.
The classic approach would be to have a single shared array (of 1 to N) and pass "low" and "high" indexes. But in this case we don't even need an array 'cos we know that array[i] is the same as i. (Ignore off-by-one. I'm still talking pseudo-code!)
But then we get to the final problem. Balancing the workload. The work of multiplying two large (e.g. BigInteger) numbers is not a constant. It actually depends on the magnitude of the numbers. (I think that for M * N it is O(log2M * log2N). If you use naive divide and conquer, multiplications at the "left end" are much quicker than those at the "right end". Solving this is tricky, and I suspect that divide and conquer is the wrong approach for dealing with it.
I would actually consider a hybrid approach:
Create T worker threads, where T corresponds to the number of available cores.
Distribute the numbers 1 through N into T subsets, in round robin fashion.
Get each thread to (serially) calculate the product of all numbers in its subset.
Do the final multiplies of the subset products using as many threads as possible.
At the end of the calculation, no matter how you do it, there is going to be a final multiplication that is done using one thread. And the two calculations before that can be done by at most 2 thread. So at the end, some thread "starvation" will be inevitable.
Finally, there is the multiplication itself. Clearly, if we are calculating factorial(N) for big enough N for multiple threads to be worth while, the result will be larger than Long.MAX_VALUE. Using BigInteger would be an obvious choice. However, there is a bit of a question mark over BigInteger.multiply(BigInteger). It might be worthwhile to look at high performance N-digit multiplication algorithms, and multiplication algorithms that can be parallelised. Start here:
Computational complexity of mathematical operations
For the record, the complexity of BigInteger.multiply(BigInteger) for two n-digit numbers is as follows:
Java 6 uses the "school boy" algorithm; i.e the complexity is O(n*n) for n digits
Java 8 uses the "school boy" algorithm, the Karatsuba algorithm (O(n*1.585)) or 3-way Toom–Cook multiplication (O(n*1.465)) depending on the size of n. So ultimately, the complexity is O(n*1.465).
How about this code :-
1. Get input number and create pairs(start,end).
2. Create a Task which multiply given number and return results;
3. Submit the task.
4. Get the results using future.
public class TestFactorial {
public static void main(String[] args) throws InterruptedException,ExecutionException {
Scanner scanner = new Scanner(new InputStreamReader(System.in));
System.out.println(" Please enter the number for which you want to calculate Factorial");
String input = scanner.nextLine();
int number =0;
int answer =1;
scanner.close();
if(input!=null)
number = Integer.valueOf(input);
if(number==0 || number ==1){
answer = 1;
}
calcuateFactorial(number, answer);
}
private static void calcuateFactorial(int number, int answer) throws InterruptedException, ExecutionException {
ExecutorService executor = Executors.newFixedThreadPool(10);
List<Future<Integer>> totalTaskResults = new ArrayList<Future<Integer>>();
for(int start =1;start<=number ;start++){
int end = start+1;
Future<Integer> taskResult;
if(end>start)
taskResult = executor.submit(new Multiply(start, 1));
else
taskResult= executor.submit(new Multiply (start, end));
totalTaskResults.add(taskResult);
}
for (Future<Integer> future :totalTaskResults){
answer = answer* future.get();
}
executor.shutdown();
System.out.println("Answer "+ answer);
}
}
Here is the callable class (tasks) :-
public class Multiply implements Callable<Integer>{
private int start ;
private int end;
public Multiply (int start, int end){
this.start= start;
this.end = end;
}
#Override
public Multiply call() throws Exception {
return start*end;
}
}
I have an array of operations and a target number.
The operations could be
+ 3
- 3
* 4
/ 2
I want to find out how close I can get to the target number by using those operations.
I start from 0 and I need to iterate through the operations in that order, and I can choose to either use the operation or not use it.
So if the target number is 13, I can use + 3 and * 4 to get 12 which is the closest I can get to the target number 13.
I guess I need to compute all possible combinations (I guess the number of calculations is thus 2^n where n is the number of operations).
I have tried to do this in java with
import java.util.*;
public class Instruction {
public static void main(String[] args) {
// create scanner
Scanner sc = new Scanner(System.in);
// number of instructions
int N = sc.nextInt();
// target number
int K = sc.nextInt();
//
String[] instructions = new String[N];
// N instructions follow
for (int i=0; i<N; i++) {
//
instructions[i] = sc.nextLine();
}
//
System.out.println(search(instructions, 0, N, 0, K, 0, K));
}
public static int search(String[] instructions, int index, int length, int progressSoFar, int targetNumber, int bestTarget, int bestDistance) {
//
for (int i=index; i<length; i++) {
// get operator
char operator = instructions[i].charAt(0);
// get number
int number = Integer.parseInt(instructions[i].split("\\s+")[1]);
//
if (operator == '+') {
progressSoFar += number;
} else if (operator == '*') {
progressSoFar *= number;
} else if (operator == '-') {
progressSoFar -= number;
} else if (operator == '/') {
progressSoFar /= number;
}
//
int distance = Math.abs(targetNumber - progressSoFar);
// if the absolute distance between progress so far
// and the target number is less than what we have
// previously accomplished, we update best distance
if (distance < bestDistance) {
bestTarget = progressSoFar;
bestDistance = distance;
}
//
if (true) {
return bestTarget;
} else {
return search(instructions, index + 1, length, progressSoFar, targetNumber, bestTarget, bestDistance);
}
}
}
}
It doesn't work yet, but I guess I'm a little closer to solving my problem. I just don't know how to end my recursion.
But maybe I don't use recursion, but should instead just list all combinations. I just don't know how to do this.
If I, for instance, have 3 operations and I want to compute all combinations, I get the 2^3 combinations
111
110
101
011
000
001
010
100
where 1 indicates that the operation is used and 0 indicates that it is not used.
It should be rather simple to do this and then choose which combination gave the best result (the number closest to the target number), but I don't know how to do this in java.
In pseudocode, you could try brute-force back-tracking, as in:
// ops: list of ops that have not yet been tried out
// target: goal result
// currentOps: list of ops used so far
// best: reference to the best result achieved so far (can be altered; use
// an int[1], for example)
// opsForBest: list of ops used to achieve best result so far
test(ops, target, currentOps, best, opsForBest)
if ops is now empty,
current = evaluate(currentOps)
if current is closer to target than best,
best = current
opsForBest = a copy of currentOps
otherwise,
// try including next op
with the next operator in ops,
test(opsAfterNext, target,
currentOps concatenated with next, best, opsForBest)
// try *not* including next op
test(opsAfterNext, target, currentOps, best, opsForBest)
This is guaranteed to find the best answer. However, it will repeat many operations once and again. You can save some time by avoiding repeat calculations, which can be achieved using a cache of "how does this subexpression evaluate". When you include the cache, you enter the realm of "dynamic programming" (= reusing earlier results in later computation).
Edit: adding a more OO-ish variant
Variant returning the best result, and avoiding the use of that best[] array-of-one. Requires the use of an auxiliary class Answer with fields ops and result.
// ops: list of ops that have not yet been tried out
// target: goal result
// currentOps: list of ops used so far
Answer test(ops, target, currentOps, opsForBest)
if ops is now empty,
return new Answer(currentOps, evaluate(currentOps))
otherwise,
// try including next op
with the next operator in ops,
Answer withOp = test(opsAfterNext, target,
currentOps concatenated with next, best, opsForBest)
// try *not* including next op
Answer withoutOp = test(opsAfterNext, target,
currentOps, best, opsForBest)
if withOp.result closer to target than withoutOp.target,
return withOp
else
return withoutOp
Dynamic programming
If the target value is t, and there are n operations in the list, and the largest absolute value you can create by combining some subsequence of them is k, and the absolute value of the product of all values that appear as an operand of a division operation is d, then there's a simple O(dkn)-time and -space dynamic programming algorithm that determines whether it's possible to compute the value i using some subset of the first j operations and stores this answer (a single bit) in dp[i][j]:
dp[i][j] = dp[i][j-1] || dp[invOp(i, j)][j-1]
where invOp(i, j) computes the inverse of the jth operation on the value i. Note that if the jth operation is a multiplication by, say, x, and i is not divisible by x, then the operation is considered to have no inverse, and the term dp[invOp(i, j)][j-1] is deemed to evaluate to false. All other operations have unique inverses.
To avoid loss-of-precision problems with floating point code, first multiply the original target value t, as well as all operands to addition and subtraction operations, by d. This ensures that any division operation / x we encounter will only ever be applied to a value that is known to be divisible by x. We will essentially be working throughout with integer multiples of 1/d.
Because some operations (namely subtractions and divisions) require solving subproblems for higher target values, we cannot in general calculate dp[i][j] in a bottom-up way. Instead we can use memoisation of the top-down recursion, starting at the (scaled) target value t*d and working outwards in steps of 1 in each direction.
C++ implementation
I've implemented this in C++ at https://ideone.com/hU1Rpq. The "interesting" part is canReach(i, j); the functions preceding this are just plumbing to handle the memoisation table. Specify the inputs on stdin with the target value first, then a space-separated list of operations in which operators immediately preceed their operand values, e.g.
10 +8 +11 /2
or
10 +4000 +5500 /1000
The second example, which should give the same answer (9.5) as the first, seems to be around the ideone (and my) memory limits, although this could be extended somewhat by using long long int instead of int and a 2-bit table for _m[][][] instead of wasting a full byte on each entry.
Exponential worst-case time and space complexity
Note that in general, dk or even just k by itself could be exponential in the size of the input: e.g. if there is an addition, followed by n-1 multiplication operations, each of which involves a number larger than 1. It's not too difficult to compute k exactly via a different DP that simply looks for the largest and smallest numbers reachable using the first i operations for all 1 <= i <= n, but all we really need is an upper bound, and it's easy enough to get a (somewhat loose) one: simply discard the signs of all multiplication operands, convert all - operations to + operations, and then perform all multiplication and addition operations (i.e., ignoring divisions).
There are other optimisations that could be applied, for example dividing through by any common factor.
Here's a Java 8 example, using memoization. I wonder if annealing can be applied...
public class Tester {
public static interface Operation {
public int doOperation(int cur);
}
static Operation ops[] = { // lambdas for the opertions
(x -> x + 3),
(x -> x - 3),
(x -> x * 4),
(x -> x / 2),
};
private static int getTarget(){
return 2;
}
public static void main (String args[]){
int map[];
int val = 0;
int MAX_BITMASK = (1 << ops.length) - 1;//means ops.length < 31 [int overflow]
map = new int[MAX_BITMASK];
map[0] = val;
final int target = getTarget();// To get rid of dead code warning
int closest = val, delta = target < 0? -target: target;
int bestSeq = 0;
if (0 == target) {
System.out.println("Winning sequence: Do nothing");
}
int lastBitMask = 0, opIndex = 0;
int i = 0;
for (i = 1; i < MAX_BITMASK; i++){// brute force algo
val = map[i & lastBitMask]; // get prev memoized value
val = ops[opIndex].doOperation(val); // compute
map[i] = val; //add new memo
//the rest just logic to find the closest
// except the last part
int d = val - target;
d = d < 0? -d: d;
if (d < delta) {
bestSeq = i;
closest = val;
delta = d;
}
if (val == target){ // no point to continue
break;
}
//advance memo mask 0b001 to 0b011 to 0b111, etc.
// as well as the computing operation.
if ((i & (i + 1)) == 0){ // check for 2^n -1
lastBitMask = (lastBitMask << 1) + 1;
opIndex++;
}
}
System.out.println("Winning sequence: " + bestSeq);
System.out.println("Closest to \'" + target + "\' is: " + closest);
}
}
Worth noting, the "winning sequence" is the bit representation (displayed as decimal) of what was used and what wasn't, as the OP has done in the question.
For Those of you coming from Java 7, this is what I was referencing for lambdas: Lambda Expressionsin GUI Applications. So if you're constrained to 7, you can still make this work quite easily.
This question is quite a long shot. It could take quite long, so if you haven't the time I understand.
Let me start by explaining what I want to achieve:
Me and some friends play this math game where we get 6 random numbers out of a pool of possible numbers: 1 to 10, 25, 50, 75 and 100. 6 numbers are chosen out of these and no duplicates are allowed. Then a goal number will be chosen in the range of [100, 999]. With the 6 aforementioned numbers, we can use only basic operations (addition, subtraction, multiplication and division) to reach the goal. Only integers are allowed and not all 6 integers are required to reach the solution.
An example: We start with the numbers 4,8,6,9,25,100 and need to find 328.
A possible solution would be: ((4 x 100) - (9 x 8)) = 400 - 72 = 328. With this, I have only used 4 out of the 6 initial numbers and none of the numbers have been used twice. This is a valid solution.
We don't always find a solution on our own, that's why I figured a program would be useful. I have written a program (in Java) which has been tested a few times throughout and it had worked. It did not always give all the possible solutions, but it worked within its own limitations. Now I've tried to expand it so all the solutions would show.
On to the main problem:
The program that I am trying to execute is running incredibly long. As in, I would let it run for 15 minutes and it doesn't look like it's anywhere near completion. So I thought about it and the options are indeed quite endless. I start with 6 numbers, I compare the first with the other 5, then the second with the other 5 and so on until I've done this 6 times (and each comparison I compare with every operator, so 4 times again). Out of the original one single state of 6 numbers, I now have 5 times 6 times 4 = 120 states (with 5 numbers each). All of these have to undergo the same ritual, so it's no wonder it's taking so long.
The program is actually too big to list here, so I will upload it for those interested:
http://www.speedyshare.com/ksT43/MathGame3.jar
(Click on the MathGame3.jar title right next to download)
Here's the general rundown on what happens:
-6 integers + goal number are initialized
-I use the class StateNumbers that are acting as game states
-> in this class the remaining numbers (initially the 6 starting numbers)
are kept as well as the evaluated expressions, for printing purposes
This method is where the main operations happen:
StateNumbers stateInProcess = getStates().remove(0);
ArrayList<Integer> remainingNumbers = stateInProcess.getRemainingNumbers();
for(int j = 0; j < remainingNumbers.size(); j++){
for(int i = 0; i < remainingNumbers.size(); i++){
for(Operator op : Operator.values()){ // Looping over different operators
if(i == j) continue;
...
}
}
}
I evaluate for the first element all the possible operations with all the remaining numbers for that state. I then check with a self written equals to see if it's already in the arraylist of states (which acts as a queue, but the order is not of importance). If it's not there, then the state will be added to the list and then I do the same for the other elements. After that I discard the state and pick another out of the growing list.
The list grows in size to 80k states in 10 minutes and grows slower and slower. That's because there is an increasing amount of states to compare to when I want to add a new state. It's making me wonder if comparing with other states to prevent duplicates is such a good idea.
The completion of this program is not really that important, but I'd like to see it as a learning experience. I'm not asking anyone to write the code for me, but a friendly suggestion on what I could have handled better would be very much appreciated. This means if you have something you'd like to mention about another aspect of the program, please do. I'm unsure if this is too much to ask for on this forum as most topics handle a specific part of a program. While my question is specific as well, the causes could be many.
EDIT: I'm not trying to find the fastest single solution, but every solution. So if I find a solution, my program will not stop. It will however try to ignore doubles like:
((4+5)7) and (7(5+4)). Only one of the two is accepted because the equals method in addition and multiplication do not care about the positioning of the operands.
It would probably be easier to write this using recursion, i.e. a depth-first search, as this would simplify the bookkeeping for intermediary states.
If you want to keep a breath-first approach, make sure that the list of states supports efficient removal of the first element, i.e. use a java.util.Queue such as java.util.ArrayDeque. I mention this because the most frequently used List implementation (i.e. java.util.ArrayList) needs to copy its entire contents to remove the first element, which makes removing the first element very expensive if the list is large.
120 states (with 5 numbers each). All of these have to undergo the same ritual, so it's no wonder it's taking so long.
Actually, it is quite surprising that it would. After all, a 2GHz CPU performs 2 billion clock cycles per second. Even if checking a state were to take as many as 100 clock cycles, that would still mean 20 million states per second!
On the other hand, if I understand the rules of the game correctly, the set of candidate solutions is given by all orderings of the 6 numbers (of which there are 6! = 720), with one of 4 operators in the 5 spaces in between, and a defined evaluation order of the operators. That is, we have a total of 6! * 4^5 * 5! = 88 473 600 candidate solutions, so processing should complete in a couple of seconds.
PS: A full solution would probably not be very time-consuming to write, so if you wish, I can also postcode - I just didn't want to spoil your learning experience.
Update: I have written the code. It was harder than I thought, as the requirement to find all solutions implies that we need to print a solution without unwinding the stack. I, therefore, kept the history for each state on the heap. After testing, I wasn't quite happy with the performance (about 10 seconds), so I added memoization, i.e. each set of numbers is only processed once. With that, the runtime dropped to about 3 seconds.
As Stackoverflow doesn't have a spoiler tag, I increased the indentation so you have to scroll right to see anything :-)
package katas.countdown;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;
enum Operator {
plus("+", true),
minus("-", false),
multiply("*", true),
divide("/", false);
final String sign;
final boolean commutes;
Operator(String sign, boolean commutes) {
this.sign = sign;
this.commutes = commutes;
}
int apply(int left, int right) {
switch (this) {
case plus:
return left + right;
case minus:
return left - right;
case multiply:
return left * right;
case divide:
int mod = left % right;
if (mod == 0) {
return left / right;
} else {
throw new ArithmeticException();
}
}
throw new AssertionError(this);
}
#Override
public String toString() {
return sign;
}
}
class Expression implements Comparable<Expression> {
final int value;
Expression(int value) {
this.value = value;
}
#Override
public int compareTo(Expression o) {
return value - o.value;
}
#Override
public int hashCode() {
return value;
}
#Override
public boolean equals(Object obj) {
return value == ((Expression) obj).value;
}
#Override
public String toString() {
return Integer.toString(value);
}
}
class OperationExpression extends Expression {
final Expression left;
final Operator operator;
final Expression right;
OperationExpression(Expression left, Operator operator, Expression right) {
super(operator.apply(left.value, right.value));
this.left = left;
this.operator = operator;
this.right = right;
}
#Override
public String toString() {
return "(" + left + " " + operator + " " + right + ")";
}
}
class State {
final Expression[] expressions;
State(int... numbers) {
expressions = new Expression[numbers.length];
for (int i = 0; i < numbers.length; i++) {
expressions[i] = new Expression(numbers[i]);
}
}
private State(Expression[] expressions) {
this.expressions = expressions;
}
/**
* #return a new state constructed by removing indices i and j, and adding expr instead
*/
State replace(int i, int j, Expression expr) {
Expression[] exprs = Arrays.copyOf(expressions, expressions.length - 1);
if (i < exprs.length) {
exprs[i] = expr;
if (j < exprs.length) {
exprs[j] = expressions[exprs.length];
}
} else {
exprs[j] = expr;
}
Arrays.sort(exprs);
return new State(exprs);
}
#Override
public boolean equals(Object obj) {
return Arrays.equals(expressions, ((State) obj).expressions);
}
public int hashCode() {
return Arrays.hashCode(expressions);
}
}
public class Solver {
final int goal;
Set<State> visited = new HashSet<>();
public Solver(int goal) {
this.goal = goal;
}
public void solve(State s) {
if (s.expressions.length > 1 && !visited.contains(s)) {
visited.add(s);
for (int i = 0; i < s.expressions.length; i++) {
for (int j = 0; j < s.expressions.length; j++) {
if (i != j) {
Expression left = s.expressions[i];
Expression right = s.expressions[j];
for (Operator op : Operator.values()) {
if (op.commutes && i > j) {
// no need to evaluate the same branch twice
continue;
}
try {
Expression expr = new OperationExpression(left, op, right);
if (expr.value == goal) {
System.out.println(expr);
} else {
solve(s.replace(i, j, expr));
}
} catch (ArithmeticException e) {
continue;
}
}
}
}
}
}
}
public static void main(String[] args) {
new Solver(812).solve(new State(75, 50, 2, 3, 8, 7));
}
}
}
As requested, each solution is reported only once (where two solutions are considered equal if their set of intermediary results is). Per Wikipedia description, not all numbers need to be used. However, there is a small bug left in that such solutions may be reported more than once.
What you're doing is basically a breadth-first search for a solution. This was also my initial idea when I saw the problem, but I would add a few things.
First, the main thing you're doing with your ArrayList is to remove elements from it and test if elements are already present. Since your range is small, I would use a separate HashSet, or BitSet for the second operation.
Second, and more to the point of your question, you could also add the final state to your initial points, and search backward as well. Since all your operations have inverses (addition and subtraction, multiplication and division), you can do this. With the Set idea above, you would effectively halve the number of states you need to visit (this trick is known as meet-in-the-middle).
Other small things would be:
Don't divide unless your resulting number is an integer
Don't add a number outside the range (so >999) into your set/queue
The total number of states is 999 (the number of integers between 1 and 999 inclusive), so you shouldn't really run into performance issues here. I'm thinking your biggest drain is that you're testing inclusion in an ArrayList which is O(n).
Hope this helps!
EDIT: Just noticed this. You say you check whether a number is already in the list, but then remove it. If you remove it, there's a good chance you're going to add it back again. Use a separate data structure (a Set works perfectly here) to store your visited states, and you should be fine.
EDIT 2: As per other answers and comments (thanks #kutschkem and #meriton), a proper Queue is better for popping elements (constant versus linear for ArrayList). In this case, you have too few states for it to be noticeable, but use either a LinkedList or ArrayDeque when you do a BFS.
Updated answer to solve Countdown
Sorry for my misunderstandings before. To solve countdown, you can do something like this:
Suppose your 6 initial numbers are a1, a2, ..., a6, and your target number is T. You want to check whether there is a way to assign operators o1, o2, ..., o5 such that
a1 o1 a2 ... o5 a6 = T
There are 5 operators, each can take one of 4 values, so there are 4 ^ 5 = 2 ^ 10 possibilities. You can use less than the entire 6, but if you build your solution recursively, you will have checked all of them at the end (more on this later). The 6 initial numbers can also be permuted in 6! = 720 ways, which leads to a total number of solutions of 2 ^ 10 * 6! which is roughly 720,000.
Since this is small, what I would do is loop through every permutation of the initial 6 numbers, and try to assign the operators recursively. For that, define a function
void solve(int result, int index, List<Integer> permutation)
where result is the value of the computation so far, and index is the index in the permutation list. You then loop over every operator and call
solve(result op permutation.get(index), index + 1, permutation)
If at any point you find a solution, check to see if you haven't found it before, and add it if not.
Apologies for being so dense before. I hope this is more to the point.
Your problem is analogous to a Coin Change Problem. First do all of the combinations of subtractions so that you can have your 'unit denomination coins' which should be all of the subtractions and additions, as well as the normal numbers you are given. Then use a change making algorithm to get to the number you want. Since we did subtractions beforehand, the result may not be exactly what you want but it should be close and a lot faster than what you are doing.
Say we are given the 6 numbers as the set S = {1, 5, 10, 25, 50, 75, 100}. We then do all the combinations of subtractions and additions and add them to S i.e. {-99, -95, -90,..., 1, 5, 10,..., 101, 105,...}. Now we use a coin change algorithm with the elements of S as the denominations. If we do not get a solution then it is not solvable.
There are many ways to solve the coin change problem, a few are discussed here:
AlgorithmBasics-examples.pdf