Preparing for exams I came across this question in an old exam:
What is the worst case/big-O complexity of this function:
float foo(float[] A) {
int n = A.length;
if (n == 1) return A[0];
float[] A1 = new float[n/2];
float[] A2 = new float[n/2];
float[] A3 = new float[n/2];
float[] A4 = new float[n/2];
for (i = 0; i <= (n/2)-1; i++) {
for (j = 0; j <= (n/2)-1; j++) {
A1[i] = A[i];
A2[i] = A[i+j];
A3[i] = A[n/2+j];
A4[i] = A[j];
}
}
return foo(A1)
+ foo(A2)
+ foo(A3)
+ foo(A4);
}
(Yes, the code makes no sense, but this is exactly the way it was written).
What's tripping me up is that the total size of n doubles for each recursive level, but the suggested answer(with the end result O(log n * n^2)) ignores that part. Am I misunderstanding things?
Edit: Replaced the semi-pseudocode with syntactically correct(but still nonsensical) code.
If you solve this recursive relation, you'd be able to determine the complexity.
T(n) = 4T(n/2) + O(n²)
With
T(1) = c
Okay, I finally figured it out.
Every time we recurse we do 4 times as many function calls as last time, so if we define the recursion level as m the number of function calls per level is
Every time we recurse we also halve the size of the array, so the amount of work per function call is
At each recursive level the work done in total then is:
In fact 4^m is the same as (2^m)^2:
Thus the amount of work can be written as just n^2:
There are log n recursive levels.
Thus the total amount of work is O(n^2 * log n), and that is because there are 4 recursive calls.
If there were just 2 recursive calls the amount of work at each level would be
which we can't reduce nicely(but turns out to be in O(n^2) if my math is correct).
Related
I am trying to calculate the time complexity of these two algorithms. The book I am referring to specifies these time complexities of each.
A) Algorithm A: O(nlogn)
int i = n;
while (i > 0)
{
for (int j = 0; j < n; j++)
System.out.println("*");
i = i / 2;
}
B) Algorithm B: O(n)
while (n > 0)
{
for (int j = 0; j < n; j++)
System.out.println("*");
n = n / 2;
}
I can see how algo. A is O(nlogn). The for loop is O(n) and while loop is O(logn). However I am failing to see how AlgoB has a time complexity of O(n). I was expecting it to be O(nlogn) also. Any help would be appreciated.
Let's look at Algorithm B from a mathematical standpoint.
The number of * printed in the first loop is n. The number of * printed in each subsequent loop is at most n/2. That recurrence relation leads to the sequence:
n + n/2 + n/4 + n/8 + ...
If this were an infinite sequence, then the sum could be represented by the formula n/(1 - r), where r is the factor between terms. In this case, r is 1/2, so the infinite sequence has a sum of 2(n).
Your algorithm certainly won't go on forever, and it each time it loops, it may be printing less than half the stars of the previous loop. The number of stars printed is therefore less than or equal to 2(n).
Because constant factors drop out of time complexity, the algorithm is O(n).
This concept is called amortized complexity, which averages the cost of operations in a loop, even if some of the operations might be relatively expensive. Please see this question and the Wikipedia page for a more thorough explanation of amortized complexity.
Algorithm B is printing half the starts at every iteration. Assume n=10, then:
n=10 -> 10*
n=5 -> 5*
n=2 -> 2*
n=1 -> 1*
In total 18* are printed. You will print n + n/2 + n/4 + ... + n/(2^i) stars. How much does i value? It is equal to the number of steps required for n to become 0. In other terms, it is the exponent to which 2 must be raised to produce n: log_2(n). You get the sum in picture:
Which can be approximated to O(n).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I've been researching a lot about time complexity for my Data Structures class. And I've been tasked to report about Shell sort algorithm and explain its time complexity (best/worst/average case). I found this site https://stackabuse.com/shell-sort-in-java/ that shows that the time complexity of this Shell sort algorithm:
void shellSort(int array[], int n){
//n = array.length
for (int gap = n/2; gap > 0; gap /= 2){
for (int i = gap; i < n; i += 1) {
int temp = array[i];
int j;
for (j = i; j >= gap && array[j - gap] > temp; j -= gap){
array[j] = array[j - gap];
}
array[j] = temp;
}
}
}
is O(n log n). But the problem is that I'm still confused about makes logn a logn or what does nlogn means.
I also tried step count method but again, I don't know where to start so I just copied from the site above and did this.
void shellSort(int array[], int n){
//n = array.length
for (int gap = n/2; gap > 0; gap /= 2){ //step 1 = runs logn times
for (int i = gap; i < n; i += 1) { //step 2 = runs n-gap times
int temp = array[i]; //step 3 = 1
int j; //step 4 = 1
for (j = i; j >= gap && array[j - gap] > temp; j -= gap){ //step 5 = i/gap times
array[j] = array[j - gap]; //step 6 = 1
}
array[j] = temp; //step 7 = 1
}
}
}
But I don't know if this is correct, I just based it off on this site. https://stackabuse.com/shell-sort-in-java/.
I've also tried comparing the total number of moves between Insertion Sort and Shell Sort since Shell Sort is a generalization of Insertion and Bubble Sort. I'll attach the pics below. I also used an online number generator that will give me 100 random numbers, copied it and applied it to both the Insertion Sort and Shell sort and assigned it as the array to sort.
And this was what came up,
Total number of moves of Insertion Sort = 4731
Total number of moves of Shell Sort = 1954
Shell Sort implementation that tells me the total number of moves it does
Insertion Sort implementation that tells me the total number of moves it does
What I've understood from all of this is that despite Shell sort being a generalization of Insertion sort, when it comes to sorting large arrays such as 100 elements Shell Sort is 2x faster than Insertion Sort.
But the ultimate question is, is there a beginner way to calculate the time complexity like this Shell Sort algorithm?
You have to take a look at the big O or big Theta analysis of your function. Your outer loop is being divided by half on every iteration so the overall time that it runs is log n. Now when you look at your inner loop it runs initially from n/2 to n all the way to 1 to n or 2 to n depending on the initial size of n so its execution time will be n/2 + n/4 + .... n /2^k which its a 'Harmonic series' (You can search geometric series as well, if you factor n -> n(1/2 + 1/4 + ... + 1/2^k) which equals nlogn. Now the best case where every list may be sorted to some extent will be Ω(nlogn) as the in the middle of the outer loop we will find optimal solution so we can say that nlogn is its lower bound - Meaning it is definitely equal or bigger than that - therefor we can say that the average case is Θ(nlog^2 n) meaning that it is in the tight bound of that - Please note for average case I use Big theta. Now if we assume that the list is completely reverse the outer loop will run all the way to the end meaning log n. The inter loop will run nlogn so the total time will be nlog^2(n) which we can say it will be O(nlog^2(n)) (we can also use Big O but theta is better you can search that up that how theta provides tight bound and big O only provides upper bound). Therefore, we can also say the worst case is O(n^2) which is relatively correct in some context.
I suggest you take a look at Big-O and Big-Theta as well as Big-Omega which can also be useful in this case.
However, the most precise mathematical representation for shell algorithm will be O(n^3/2). However, there are still arguments and analyzation taking place.
I hope this helps.
First, I'll show that the algorithm will never be slower than O(n^2), and then I'll show that the worst-case run time is at least O(n^2).
Assume n is a power of two. We know the worst case for insertion sort is O(n^2). When h-sorting the array, we're performing h insertion sorts, each on an array of size n / h. So the complexity for a h-sort pass is O(h * (n / h)^2) = O(n^2 / h). The complexity of the whole algorithm is now the sum of n^2 / h where h is each power of two up to n / 2. This is a geometric series with first term n^2, common ratio 1 / 2, and log2(n) terms. Using the geometric series sum formula gives n^2*((1 / 2)^log2(n) - 1) / (1 / 2 - 1) = n^2*(1 / n - 1) / (-1 / 2) = n^2*(-2 / n + 2) = 2n^2 - 2n = O(n^2).
Consider an array created by interweaving two increasing sequences, where all elements in one sequence is greater than all elements in the other sequence, such as [1, 5, 2, 6, 3, 7, 4, 8]. Since this array is two-sorted, all passes except the last one does nothing. In the last pass, an element at index i where i is even has to be moved to index i / 2, which uses O(i / 2) operations. So we have 1 + 2 + 3 + ... + n / 2 = (n / 2) * (n / 2 + 1) / 2 = O(n^2).
Let pi(x) denote the number of primes <= x. For example, pi(100) = 25. I would like to create a table which stores values of pi(x) for all x <= L. I figured the quickest way would be to use the sieve of Eratosthenes. First I mark all primes, and then I use dynamic programming, summing the count of primes and increasing each time a new prime appears. This is implemented in the Java code below:
public static int [] piTableSimple (int L)
{
int sqrtl = (int) Math.sqrt(L);
int [] piTable = new int [L + 1];
Arrays.fill(piTable, 1);
piTable[0] = 0;
piTable[1] = 0;
for (int i = 2 ; i <= sqrtl ; i++)
if (piTable[i] == 1)
for (int j = i * i ; j <= L ; j += i)
piTable[j] = 0;
for (int i = 1 ; i < piTable.length ; i++)
piTable[i] += piTable[i - 1];
return piTable;
}
There are 2 problems with this implementation:
It uses large amounts of memory, as the space complexity is O(n)
Because Java arrays are "int"-indexed, the bound for L is 2^31 - 1
I can "cheat" a little though. Because for even values of x, pi(x) = pi(x - 1), enabling me to both reduce memory usage by a factor of 2, and increase the bound for L by a factor of 2 (Lmax <= 2^32).
This is implemented with a simple modification to the above code:
public static long [] piTableSmart (long L)
{
long sqrtl = (long) Math.sqrt(L);
long [] piTable = new long [(int) (L/2 + 1)];
Arrays.fill(piTable, 1);
piTable[0] = 0;
piTable[1] = 0;
piTable[2] = 1;
for (int i = 3 ; i <= sqrtl ; i += 2)
if (piTable[(i + 1) / 2] == 1)
{
long li = (long) i;
long inc = li * 2L;
for (long j = li * li ; j <= L ; j += inc)
piTable[(int) ((j + 1) / 2)] = 0;
}
piTable[2] = 2;
for (int i = 1 ; i < piTable.length ; i++)
piTable[i] += piTable[i - 1];
return piTable;
}
Note that the value of pi(2) = 1 is not directly represnted in this array. But this has simple workarounds and checks that solve it. This implementation comes with a small cost, that the actual value of pi(x) is not accessed in a straight-forward way, but rather to access the value of pi(x), one has to use
piTable[(x + 1) / 2]
And this works for both even and odd values of x of course. The latter completes creating a pi(x) table for x <= L = 10^9 in 10s on my rather slowish laptop.
I would like to further reduce the space required and also increase the bound for L for my purposes, without severly deteriorating performance (for example, the cost of slightly more arithmetic operations to access the value of pi(x) as in the latter code barely deteriorates performance). Can it be done in an efficient and smart way?
You should use a segmented Sieve of Eratosthenes, which reduces the memory requirement from O(n) to O(sqrt(n)). Here is an implementation.
Do you need to store all the pi? Here is a function that computes pi(x). It's reasonably quick up to 10**12.
If you find this useful, please upvote this answer and also the two linked answers.
Now that I understand better what you want to do, I can give a better answer.
The normal way to compute pi(x) starts with pre-computed tables arranged at intervals, then uses a segmented sieve to interpolate between the pre-computed points; the pre-computations may be done by sieving or by any of several other methods. Those tables get big, as you have pointed out. If you want to be able to compute pi(x) up to 1020, and you are willing to sieve a range up to 1012 each time someone calls your function, you will need a table with 108 64-bit integers, which will take nearly a gigabyte of space; calls to your function will take about half-a-minute each for the sieving, assuming a recent-vintage personal computer. Of course, you can choose where you want to be on the time/space trade-off curve by choosing how many pre-computed points you will have.
You are talking about computing pi(x) for x > 1024, which will take lots more space, or lots more time, or both. Lots. Recent projects that have computed huge values of pi(x), for values of x like 1024 or 1025, have taken months to compute.
You might want to look at Kim Walisch's primesieve program, which has a very fast segmented sieve. You might also look at the website of Tomás Oliveira e Silva, where you will find tables of pi(x) up to 1022.
Having said all that, what you want to do probably isn't feasible.
for (int i = 1; i <= Math.pow(2, n); i = i * 2) {
for (int j = 0; j <= Math.log(i); j++) {
sum = i + j;
System.out.println(sum); // we would like to print the sum..
}
}
How can i count the number of primitive operation my code has?
Analyzing the first loop, you can see that the limit is 2^n but you can see that the increment step is i = i x 2, so how many multiplications until you reach the limit? The answer is obviously n.
The inner loop, in the worst case how many iterations will perform? Since it depends on the maximum value that the first loop variable (i) will ever take, then it is the natural logarithm of that value, in other words, log(2^n).
Summarizing, the total complexity of the algorithm is O(n * log(2^n)) which simplifies to O(n*n) by taking out the exponent (as promptly suggested by #Andreas).
I just need some clarification or help on this Big O problem. I don't know if I'm explaining this correctly, but I noticed that the for loop has a false condition, so that means it won't loop at all. And my professor said it's possible to still determine the run time of the loops. So what I'm thinking is this:
1 + (n - 1 - n) * (n) = 1 + 1 * n = 1 + n = O(n)
Explanation: 1 is for the operation outside of the loop. (n - 1 - n) is the iteration of the outer loop and n is the iteration of the inner loop.
Note: I'm still learning Big O, so please correct me if any of my logic is wrong.
int total = 0;
for (int i = n; i < n - 1; i++) {
for (int j = 0; j < n; j++) {
total = total + 1
}
}
There shouldn't be any negative number in Big O analysis. It doesn't make sense for negative running time. Also, (n - 1 - n) is not just in order O(1). Your outer loop doesn't even go into one iteration. Thus, the time complexity for whatever statement in your loop doesn't matter.
To conclude, the running time is 1 + 1 = O(1).
Big O notation to describe the asymptotic behavior of functions. Basically, it tells you how fast a function grows or
declines
For example, when analyzing some algorithm, one might find that the time (or the number of steps) it takes to complete a problem of size n is given by
T(n) = 4 n^2 - 2 n + 2
If we ignore constants (which makes sense because those depend on the particular hardware the program is run on) and slower growing terms, we could say "T(n)" grows at the order of n^2 " and write:T(n) = O(n^2)
For the formal definition, suppose f(x) and g(x) are two functions defined on some subset of the real numbers. We write
f(x) = O(g(x))
(or f(x) = O(g(x)) for x -> infinity to be more precise) if and only if there exist constants N and C such that
|f(x)| <= C|g(x)| for all x>N
Intuitively, this means that f does not grow faster than g
If a is some real number, we write
f(x) = O(g(x)) for x->a
if and only if there exist constants d > 0 and C such that
|f(x)| <= C|g(x)| for all x with |x-a| < d
So for your case it would be O(n^2) as |f(x)| > C|g(x)|
Reference from http://web.mit.edu/16.070/www/lecture/big_o.pdf
int total = 0;
for (int i = n; i < n - 1; i++) { // --> n loop
for (int j = 0; j < n; j++) { // --> n loop
total = total + 1; // -- 1 time
}
}
}
Big O Notation gives an assumption when value is very big outer loop will run n times and inner loop is running n times
Assume n -> 100
than total n^2 10000 run times