How do I calculate the time complexity of the following program?
int[] vars = { 2, 4, 5, 6 };
int len = vars.length;
int[] result = new int[len];
for (int i = 0; i < len; i++) {
int value = 1;
for (int k = 0; k < i; k++) {
value = value * vars[k];
}
for (int j = i + 1; j < len; j++) {
value = value * vars[j];
}
result[i] = value;
}
and how is the above one same as below?
for (int i = 0; i < len; i++) {
int value = 1;
for (int j = 0; j < len; j++) {
if(j != i) {
value = value * vars[j];
}
}
result[i] = value;
}
The i for loop is of time complexity O(n), because it performs one iteration for every element of the array. For every element in the array, you are looping through it once more -- half on average in the k for loop, and half on average in the j for loop. Each of these is O(n) as well. If there are 4 elements in the array, the number of operations is proportional to n*(n - 1), but in time-complexity, constants such as the 1 are ignored.
The number of operations your method will perform is proportional to the number of elements in it multiplied by itself, therefore, overall, the method is O(n2).
For the first fragment:
For the second fragment:
A general approach in determining the complexity is counting the iterations.
In your example, you have an outer for loop with two loops nested in it. Note: Instead of len, I'll write n.
The outer loop
for (int i = 0; i < n; i++)
iterates n-times.
The number of iterations of the two next loops are actually more easy to count, than it looks like:
The second loop iterates i-times and the third n-i-times. If you add them together you get n-many iterations within the outer loop.
Finally, if the outer loop does n iterations and within each of these iteration the code loops another n times you get the result of n^2 iterations. In the traditional notation of complexity-theory you'd write, that the algorithm has an upper-bound of n^2 or is in O(n).
Related
int sum = 0;
for(int i = 1; i < n; i++) {
for(int j = 1; j < i * i; j++) {
if(j % i == 0) {
for(int k = 0; k < j; k++) {
sum++;
}
}
}
}
I don't understand how when j = i, 2i, 3i... the last for loop runs n times. I guess I just don't understand how we came to that conclusion based on the if statement.
Edit: I know how to compute the complexity for all the loops except for why the last loop executes i times based on the mod operator... I just don't see how it's i. Basically, why can't j % i go up to i * i rather than i?
Let's label the loops A, B and C:
int sum = 0;
// loop A
for(int i = 1; i < n; i++) {
// loop B
for(int j = 1; j < i * i; j++) {
if(j % i == 0) {
// loop C
for(int k = 0; k < j; k++) {
sum++;
}
}
}
}
Loop A iterates O(n) times.
Loop B iterates O(i2) times per iteration of A. For each of these iterations:
j % i == 0 is evaluated, which takes O(1) time.
On 1/i of these iterations, loop C iterates j times, doing O(1) work per iteration. Since j is O(i2) on average, and this is only done for 1/i iterations of loop B, the average cost is O(i2 / i) = O(i).
Multiplying all of this together, we get O(n × i2 × (1 + i)) = O(n × i3). Since i is on average O(n), this is O(n4).
The tricky part of this is saying that the if condition is only true 1/i of the time:
Basically, why can't j % i go up to i * i rather than i?
In fact, j does go up to j < i * i, not just up to j < i. But the condition j % i == 0 is true if and only if j is a multiple of i.
The multiples of i within the range are i, 2*i, 3*i, ..., (i-1) * i. There are i - 1 of these, so loop C is reached i - 1 times despite loop B iterating i * i - 1 times.
The first loop consumes n iterations.
The second loop consumes n*n iterations. Imagine the case when i=n, then j=n*n.
The third loop consumes n iterations because it's executed only i times, where i is bounded to n in the worst case.
Thus, the code complexity is O(n×n×n×n).
I hope this helps you understand.
All the other answers are correct, I just want to amend the following.
I wanted to see, if the reduction of executions of the inner k-loop was sufficient to reduce the actual complexity below O(n⁴). So I wrote the following:
for (int n = 1; n < 363; ++n) {
int sum = 0;
for(int i = 1; i < n; ++i) {
for(int j = 1; j < i * i; ++j) {
if(j % i == 0) {
for(int k = 0; k < j; ++k) {
sum++;
}
}
}
}
long cubic = (long) Math.pow(n, 3);
long hypCubic = (long) Math.pow(n, 4);
double relative = (double) (sum / (double) hypCubic);
System.out.println("n = " + n + ": iterations = " + sum +
", n³ = " + cubic + ", n⁴ = " + hypCubic + ", rel = " + relative);
}
After executing this, it becomes obvious, that the complexity is in fact n⁴. The last lines of output look like this:
n = 356: iterations = 1989000035, n³ = 45118016, n⁴ = 16062013696, rel = 0.12383254507467704
n = 357: iterations = 2011495675, n³ = 45499293, n⁴ = 16243247601, rel = 0.12383580700180696
n = 358: iterations = 2034181597, n³ = 45882712, n⁴ = 16426010896, rel = 0.12383905075183874
n = 359: iterations = 2057058871, n³ = 46268279, n⁴ = 16610312161, rel = 0.12384227647628734
n = 360: iterations = 2080128570, n³ = 46656000, n⁴ = 16796160000, rel = 0.12384548432498857
n = 361: iterations = 2103391770, n³ = 47045881, n⁴ = 16983563041, rel = 0.12384867444612208
n = 362: iterations = 2126849550, n³ = 47437928, n⁴ = 17172529936, rel = 0.1238518469862343
What this shows is, that the actual relative difference between actual n⁴ and the complexity of this code segment is a factor asymptotic towards a value around 0.124... (actually 0.125). While it does not give us the exact value, we can deduce, the following:
Time complexity is n⁴/8 ~ f(n) where f is your function/method.
The wikipedia-page on Big O notation states in the tables of 'Family of Bachmann–Landau notations' that the ~ defines the limit of the two operand sides is equal. Or:
f is equal to g asymptotically
(I chose 363 as excluded upper bound, because n = 362 is the last value for which we get a sensible result. After that, we exceed the long-space and the relative value becomes negative.)
User kaya3 figured out the following:
The asymptotic constant is exactly 1/8 = 0.125, by the way; here's the exact formula via Wolfram Alpha.
Remove if and modulo without changing the complexity
Here's the original method:
public static long f(int n) {
int sum = 0;
for (int i = 1; i < n; i++) {
for (int j = 1; j < i * i; j++) {
if (j % i == 0) {
for (int k = 0; k < j; k++) {
sum++;
}
}
}
}
return sum;
}
If you're confused by the if and modulo, you can just refactor them away, with j jumping directly from i to 2*i to 3*i ... :
public static long f2(int n) {
int sum = 0;
for (int i = 1; i < n; i++) {
for (int j = i; j < i * i; j = j + i) {
for (int k = 0; k < j; k++) {
sum++;
}
}
}
return sum;
}
To make it even easier to calculate the complexity, you can introduce an intermediary j2 variable, so that every loop variable is incremented by 1 at each iteration:
public static long f3(int n) {
int sum = 0;
for (int i = 1; i < n; i++) {
for (int j2 = 1; j2 < i; j2++) {
int j = j2 * i;
for (int k = 0; k < j; k++) {
sum++;
}
}
}
return sum;
}
You can use debugging or old-school System.out.println in order to check that i, j, k triplet is always the same in each method.
Closed form expression
As mentioned by others, you can use the fact that the sum of the first n integers is equal to n * (n+1) / 2 (see triangular numbers). If you use this simplification for every loop, you get :
public static long f4(int n) {
return (n - 1) * n * (n - 2) * (3 * n - 1) / 24;
}
It is obviously not the same complexity as the original code but it does return the same values.
If you google the first terms, you can notice that 0 0 0 2 11 35 85 175 322 546 870 1320 1925 2717 3731 appear in "Stirling numbers of the first kind: s(n+2, n).", with two 0s added at the beginning. It means that sum is the Stirling number of the first kind s(n, n-2).
Let's have a look at the first two loops.
The first one is simple, it's looping from 1 to n. The second one is more interesting. It goes from 1 to i squared. Let's see some examples:
e.g. n = 4
i = 1
j loops from 1 to 1^2
i = 2
j loops from 1 to 2^2
i = 3
j loops from 1 to 3^2
In total, the i and j loops combined have 1^2 + 2^2 + 3^2.
There is a formula for the sum of first n squares, n * (n+1) * (2n + 1) / 6, which is roughly O(n^3).
You have one last k loop which loops from 0 to j if and only if j % i == 0. Since j goes from 1 to i^2, j % i == 0 is true for i times. Since the i loop iterates over n, you have one extra O(n).
So you have O(n^3) from i and j loops and another O(n) from k loop for a grand total of O(n^4)
For the algorithm below:
int x = 0;
for (int i = 0; i < n; i++)
for (j = 0; j < n; j++) {
if (j < i) j = j + n;
else x = x + 1;
}
So for this algorithm, my thought process goes like this:
The inner-loop performs n iterations for j when i=0. However, for every value of i=0,1..n-1, j will only perform one iteration because the if-statement will evaluate to true and end the inner-loop.
Here is my source of confusion:
Since the outer-loop will perform n iterations no matter what, and since the inner loop performs n iterations when i=0 (very first iteration), how come the big-Oh time complexity isn't O(n²) and is instead, O(n) if the loops are nested and both perform n iterations in the very first iteration?
You have a line that says if (j < i) j = j + n; which essentially breaks out of the loop (when j < i), and since the inner loop starts at 0, this will trigger on the first iteration every time (except the first time), making it run in constant time.
You essentially only have one loop here. The code can be rewritten as
int x = 0;
for (int i = 0; i < n; i++) {
x = x + 1;
}
which makes it clear why it is O(n).
You could just print the values of i and j from the inner loop:
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
System.out.print(i + "" + j + " ");
if (j < i) j = j + n;
else x = x + 1;
}
System.out.println();
}
Output:
00 01 02 03 04 ..
10
20
30
40
..
Which represents only the first row and the first column of the matrix, thus the complexity is:
O(2n) => O(n).
You properly notice that only for i=0 inner loop will iterate n times. For i>0 inner loop runs once only. We can sum all these iterations: i0+i1+...+n-1, where i0=n, and for other indexes ix=1. So, sum of inner iterations will be n + n - 1 => 2n - 1. Which gives us: O(2n - 1) => O(2n) - O(1) => 2*O(n) - O(1) => O(n).
For every n outer iteration, there are n inner iterations. So you will do n*n=n^2 computations and hence the complexity is O(n^2). Does this make sense or it is more confusing?
I read such a solution to rotate an array
the question:
public class Solution {
public void rotate(int[] nums, int k) {
int temp, previous;
for (int i = 0; i < k; i++) {
previous = nums[nums.length - 1];
for (int j = 0; j < nums.length; j++) {
temp = nums[j];
nums[j] = previous;
previous = temp;
}
}
}
}
I am confused about previous = nums[num.lengh -1],
is it a range as nums[0:10] or a single element as nums[0]?
It's a single element, he is taking the element in the num.length -1 position, and the value inside is being swapped with nums[j]:
for j=0 you have:
temp = num[0];
num[0] = num[num.length-1]
num[num.length-1] = temp;
And so on.
It is a single element, because num.lengh - 1 is an int and just gives you the last accessible index of the array.
If you check wether the length of the array is > 0, then you can safely use the length of the array to determine the last accessible index.
The length of an array is very often used in loops like yours:
Here the length is used to make sure you don't access unavailable indexes:
for (int j = 0; j < nums.length; j++)
You can write slightly a different condition without changing the functionality
for (int j = 0; j <= nums.length - 1; j++)
But if you do the following, you will get an IndexOutOfBoundsException:
for (int j = 0; j <= nums.length; j++)
The last iteration would try to access nums[nums.length], which isn't there...
nums[nums.length - 1]; gives you the last position of the array. It is -1 cause the positions of an array starts with 0 not with 1.
If you don't write -1 you would get an Out of bounds exception.
single, this is due to you're performing a mathematical expression and then getting a valeu from that. meaning you're taking the length of the array list then subtracting 1 and then getting that value or in other words you're getting the last value.
to get all values you must loop like a for loop etc.
for (i = 0; i < array.length; i++) {
System.out.print(array[i]);
}
I want to analyze the execution time complexity of the below program.
Please answer with the explanation.
private static void printSecondLargest(int[] arr) {
int length = arr.length, temp;
for (int i = 0; i < 2; i++) {
for (int j = i+1; j < length; j++) {
if(arr[i]<arr[j]) {
temp = arr[i];
arr[i] = arr[j];
arr[j] = temp;
}
}
}
System.out.println("Second largest is: "+arr[1]);
}
It's O(n) where n represents the length of the array.
The body of the inner most loop:
if(arr[i]<arr[j]) {
temp = arr[i];
arr[i] = arr[j];
arr[j] = temp;
}
runs in constant time.
This code will be executed first arr.length-1 times, then arr.length-2 times. That is 2 * arr.length - 3. Thus the execution time is proportional to 2n-3 which is O(n).
It is clearly O(n). The outer loop is running only 2 times and inner loop N times. So, overall complexity is O(2*n).
The outer loop will run two times and inner loop runs (length-1) and second time (length-2)
suppose length is N
so it will be 2*((N-1)/2+(N-2/)2)==2*(2n-3)/2
Which is final (2N-3) and in O notation it is O(N)
private static void printSecondLargest(int[] arr) {
int length = arr.length, temp; // **it takes contant time**
for (int i = 0; i < 2; i++) { // as loop goes only two step it also takes constant time
for (int j = i+1; j < length; j++) { // this loop takes n time if we consider arr length of size n
if(arr[i]<arr[j]) {
temp = arr[i]; // it also takes constant time
arr[i] = arr[j];
arr[j] = temp;
}
}
}
System.out.println("Second largest is: "+arr[1]);
}
So as per above calculation we neglect constant time and calculate all varying time constraint and as per code complexity will be O(n).
Here's a little exercise i'm working on about dynamic programming. I have the following function :
I have to program this function with two approaches (top-down with memoization and bottom-up).
Here's what I currently do for bottom up:
public static int functionBottomUp (int n){
int [] array = new int[n+1];
array[0] = 1;
for(int i = 1; i < array.length; i++){
if(i == 1)
array[i] = array[i - 1];
else {
for(int p = 0; p < i; p ++)
array[i] += array[p];
}
}
return array[n];
}
And for memoization :
public static int functionMemoization(int n){
int[] array = new int[n+1];
for(int i = 0; i < n; i++)
array[i] = 0;
return compute(array, n);
}
private static int compute(int[] array, int n){
int ans = 0;
if(array[n] > 0)
return array[n];
if(n == 0 || n == 1)
ans = 1;
else
for(int i = 0; i < n; i++)
ans += compute(array, i);
array[n] = ans;
return array[n];
}
I get correct outputs for both but now i'm struggling myself to calculate the complexities of both.
First the complexity of f(n) is 2^n because f(3) make 7 calls to f(0) and f(4) make 15 calls to f(0) (I know this is not a formal proof but this is just to give me an idea).
But now i'm stuck for calculating the complexity of both functions.
Bottom-Up : I would say that the complexity is O(n) (because of the for(int i = 1; i < array.length; i++)) but there is this inner loop for(int p = 0; p < i; p ++) and I don't know if this modifies the complexity.
Memoization : Clearly this is a most O(n) because of the first loop which initialize the array. But I don't know how the compute function could modify this complexity.
Could someone clarify this for me ?
Let's take a look at your functions. Here's the bottom-up DP version:
public static int functionBottomUp (int n){
int [] array = new int[n+1];
array[0] = 1;
for(int i = 1; i < array.length; i++){
if(i == 1)
array[i] = array[i - 1];
else {
for(int p = 0; p < i; p ++)
array[i] += array[p];
}
}
return array[n];
}
To count up the work that's being done, we can look at how much work is required to complete loop iteration i for some arbitrary i. Notice that if i = 1, the work done is O(1). Otherwise, the loop runtime is taken up by this part here:
for(int p = 0; p < i; p ++)
array[i] += array[p];
The time complexity of this loop is proportional to i. This means that loop iteration i does (more or less) i work. Therefore, the total work done is (approximately)
1 + 2 + 3 + ... + n = Θ(n2)
So the runtime here is Θ(n2) rather than O(n) as you conjectured in your question.
Now, let's look at the top-down version:
public static int functionMemoization(int n){
int[] array = new int[n+1];
for(int i = 0; i < n; i++)
array[i] = 0;
return compute(array, n);
}
private static int compute(int[] array, int n){
int ans = 0;
if(array[n] > 0)
return array[n];
if(n == 0 || n == 1)
ans = 1;
else
for(int i = 0; i < n; i++)
ans += compute(array, i);
array[n] = ans;
return array[n];
}
You initially do Θ(n) work to zero out the array, then call compute to compute all the values. You're eventually going to fill in all of array with values and will do so exactly once per array element, so one way to determine the time complexity is to determine, for each array entry, how much work is required to fill it. In this case, the work done is determined by this part:
for(int i = 0; i < n; i++)
ans += compute(array, i);
Since you're memoizing values, when determining the work required to evaluate the function on value n, we can pretend each recursive call takes time O(1); the actual work will be accounted for when we sum up across all n. As before, the work here done is proportional to n. Therefore, since n ranges from 1 to n, the work done is roughly
1 + 2 + 3 + ... + n = Θ(n2)
Which again is more work than your estimated O(n).
However, there is a much faster way to evaluate this recurrence. Look at the first few values of f(n):
f(0) = 1
f(1) = 1
f(2) = 2
f(3) = 4
f(4) = 8
f(5) = 16
...
f(n) = 2n-1
Therefore, we get that
f(0) = 1
f(n) = 2n-1 if n > 0
Therefore, the following function evaluates f time:
int f(int n) {
return n == 0? 1 : 1 << (n - 1);
}
Assuming that you're working with fixed-sized integers (say, 32-bit or 64-bit integers), this takes time O(1). If you're working with arbitrary-precision integers, this will take time Θ(n) because you can't express 2n-1 without writing out Θ(n) bits, but if we're operating under this assumption the runtimes for the original code would also need to be adjusted to factor in the costs of the additions. For simplicity, I'm going to ignore it or leave it as an exercise to the reader. ^_^
Hope this helps!