We Keep Coding

Time complexity for all subsets using backtracking

I am trying to understand the time complexity while using backtracking. The problem is
Given a set of unique integers, return all possible subsets.
Eg. Input [1,2,3] would return [[],[1],[2],[1,2],[3],[1,3],[2,3],[1,2,3]]
I am solving it using backtracking as this:
private List<List<Integer>> result = new ArrayList<>();
public List<List<Integer>> getSubsets(int[] nums) {
for (int length = 1; length <= nums.length; length++) { //O(n)
backtrack(nums, 0, new ArrayList<>(), length);
}
result.add(new ArrayList<>());
return result;
}
private void backtrack(int[] nums, int index, List<Integer> listSoFar, int length) {
if (length == 0) {
result.add(listSoFar);
return;
}
for (int i = index; i < nums.length; i++) { // O(n)
List<Integer> temp = new ArrayList<>();
temp.addAll(listSoFar); // O(2^n)
temp.add(nums[i]);
backtrack(nums, i + 1, temp, length - 1);
}
}
The code works fine, but I am having trouble understanding the time/space complexity.
What I am thinking is here the recursive method is called n times. In each call, it generates the sublist that may contain max 2^n elements. So time and space, both will be O(n x 2^n), is that right?
Is that right? If not, can any one elaborate?
Note that I saw some answers here, like this but unable to understand. When recursion comes into the picture, I am finding it a bit hard to wrap my head around it.

You're exactly right about space complexity. The total space of the final output is O(n*2^n), and this dominates the total space used by the program. The analysis of the time complexity is slightly off though. Optimally, time complexity would, in this case, be the same as the space complexity, but there are a couple inefficiencies here (one of which is that you're not actually backtracking) such that the time complexity is actually O(n^2*2^n) at best.
It can definitely be useful to analyze a recursive algorithm's time complexity in terms of how many times the recursive method is called times how much work each call does. But be careful about saying backtrack is only called n times: it is called n times at the top level, but this is ignoring all the subsequent recursive calls. Also every call at the top level, backtrack(nums, 0, new ArrayList<>(), length); is responsible for generating all subsets sized length, of which there are n Choose length. That is, no single top-level call will ever produce 2^n subsets; it's instead that the sum of n Choose length for lengths from 0 to n is 2^n:
Knowing that across all recursive calls, you generate 2^n subsets, you might then want to ask how much work is done in generating each subset in order to determine the overall complexity. Optimally, this would be O(n), because each subset varies in length from 0 to n, with the average length being n/2, so the overall algorithm might be O(n/2*2^n) = O(n*2^n), but you can't just assume the subsets are generated optimally and that no significant extra work is done.
In your case, you're building subsets through the listSoFar variable until it reaches the appropriate length, at which point it is appended to the result. However, listSoFar gets copied to a temp list in O(n) time for each of its O(n) characters, so the complexity of generating each subset is O(n^2), which brings the overall complexity to O(n^2*2^n). Also, some listSoFar subsets are created which never figure into the final output (you never check to see that there are enough numbers remaining in nums to fill listSoFar out to the desired length before recursing), so you end up doing unnecessary work in building subsets and making recursive calls which will never reach the base case to get appended to result, which might also worsen the asymptotic complexity. You can address the first of these inefficiencies with back-tracking, and the second with a simple break statement. I wrote these changes into a JavaScript program, leaving most of the logic the same but re-naming/re-organizing a little bit:
function getSubsets(nums) {
let subsets = [];
for (let length = 0; length <= nums.length; length++) {
// refactored "backtrack" function:
genSubsetsByLength(length); // O(length*(n Choose length))
}
return subsets;
function genSubsetsByLength(length, i=0, partialSubset=[]) {
if (length === 0) {
subsets.push(partialSubset.slice()); // O(n): copy partial and push to result
return;
}
while (i < nums.length) {
if (nums.length - i < length) break; // don't build partial results that can't finish
partialSubset.push(nums[i]); // O(1)
genSubsetsByLength(length - 1, ++i, partialSubset);
partialSubset.pop(); // O(1): this is the back-tracking part
}
}
}
for (let subset of getSubsets([1, 2, 3])) console.log(`[`, ...subset, ']');
The key difference is using back-tracking to avoid making copies of the partial subset every time you add a new element to it, such that each is built in O(length) = O(n) time rather than O(n^2) time, because there is now only O(1) work done per element added. Popping off the last character added to the partial result after each recursive call allows you to re-use the same array across recursive calls, thus avoiding the O(n) overhead of making temp copies for each call. This, along with the fact that only subsets which appear in the final output are built, allows you to analyze the total time complexity in terms of the total number of elements across all subsets in the output: O(n*2^n).

Your code works not efficiently.
Like first solution in the link, you only think about the number will be included or not. (like getting combination)
It means, you don't have to iterate in getSubsets and backtrack function.
"backtrack" function could iterate "nums" array with parameter
private List<List<Integer>> result = new ArrayList<>();
public List<List<Integer>> getSubsets(int[] nums) {
backtrack(nums, 0, new ArrayList<>(), new ArrayList<>());
return result;
}
private void backtrack(int[] nums, int index, List<Integer> listSoFar)
// This function time complexity 2^N, because will search all cases when the number included or not
{
if (index == nums.length) {
result.add(listSoFar);
return;
}
// exclude num[index] in the subset
backtrack(nums, index+1, listSoFar)
// include num[index] in the subset
backtrack(nums, index+1, listSoFar.add(nums[index]))
}

Arrays Left Rotation { A baby coder's struggle with time complexities}

I am working on a Hackerrank problem where I am given an array and must rotate elements to the left n amount of times. I was able to solve the problem but I am still stuck on how to calculate the time complexity of my solution since I am traversing through the array n amount of times and initializing i at 0 until the while loop condition is false, I wanted to say O(n), but I'm leaning towards O(n^2) or I'm just way off... Can someone please help me understand the approach to calculating time complexities? I know that my space complexity is O(1) since I am not using an extra data structure. Lastly, depending on the actual time complexity is there a need to make my code more efficient? P.S. please be kind with your responses, I am an undergrad still trying to master basic Computer Science principles. If you have any good books or website suggestions specifically for learning time complexities I'd be so grateful if you'd provide a link in the comments.
Here is the question:
A left rotation operation on an array shifts each of the array's elements unit to the left. For example, if 2 left rotations are performed on the array [1,2,3,4,5], then the array would become [3,4,5,1,2].
Given an array of integers n and a number, d, perform d left rotations on the array. Return the updated array to be printed as a single line of space-separated integers.
Here is my input:
5 4
1 2 3 4 5
n = 5 d = 4
Here is my output:
5, 1, 2, 3, 4
My Solution/Code :
static int[] rotLeft(int[] a, int d) {
while(d != 0)
{
int k = a[0]; int i = 0;
while(i < a.length-1)
{
a[i] = a[i+1];
i++;
}
a[a.length-1] = k;
d--;
}
return a;
}

A left rotation performs n assignments on the array, so it's an O(n) operation.
You're performing d of those operations. Since d is unrelated to n, you could say that the general form of the time complexity is O(n*d).
If you were given any additional information about the relative sizes of d and n, you could further refine this:
If d is much larger than n, n is negligible and you could say the time complexity of the entire operation is O(d).
If d is much smaller than n, d is negligible and you could say the time complexity of the entire operation is O(n).
If d is the same order of magnitude as n, you could say the time complexity of the entire operation is O(n2).

Complexity of BigO notation

I've been doing some questions but answers not provided so I was wondering if my answers are correct
a) given that a[i....j] is an integer array with n elements and x is an integer
int front, back;
while(i <= j) {
front = (i + j) / 3;
back = 2 * (i + j) / 3;
if(a[front] == x)
return front;
if (a[back] ==x)
return back;
if(x < a[front])
j = front - 1;
else if(x > a[back])
i = back+1;
else {
j = back-1;
i = front + 1;
}
}
My answer would be O(1) but I have a feeling I'm wrong.
B)
public static void whatIs(int n) {
if (n > 0)
System.out.print(n+" "+whatIs(n/2)+" "+whatIs(n/2));
}
ans: I'm not sure whether is it log4n or logn since recursion happens twice.

A) Yes. O(1) is wrong. You are going around the loop a number of times that depends on i, j, x ... and the contents of the array. Work out how many times you go around the loop in the best and worst cases.
B) Simplify log(4*n) using log(a*b) -> log(a) + log(b) (basic high-school mathematics) and then apply the definition of big O.
But that isn't the right answer either. Once again, you should go back to first principles and count the number of times that the method gets called for a given value of the parameter n. And do a proof by induction.

Both answers are incorrect.
In the first example on each iteration either you find the number or you shrink the length of the interval by 1/3. I.e. if the length used to be n you make it (2/3)*n. Worst case you find x on the last iteration - when the length of the interval is 1. So just like with binary search the complexity is calculated via a log: the complexity is O(log3/2(n)) and this in fact is simply O(log(n))
In the second example for a given number n you perform twice the number of operations needed for n/2. Start from n = 0 and n = 1 and use induction to prove the complexity is in fact O(n).
Hope this helps.

A) This algorithm seems similar to the Golden section search. When analyzing complexity, it's sometimes easier to imagine what would happen if we would extend the data structure, rather than contracting it. Think of it like this: Every loop removes a third from the search interval. That means, that if we know exactly how long time it takes for a certain length, we could add 50% more if we're allowed to loop once more – an exponential growth. Thus, the search algorithm must have complexity O(log n).
B) Every time we add a "layer" of function calls, we need to double the number of them (since the function always calls itself twice). In other words, given a certain length and time consumption, doubling n also requires twice as many function calls in the last layer. The algorithm is O(n).

How does this implementation of Fibonacci(n) work? [recursion]

Theres an exercise from a Java book I'm reading that has me confused:
A Fibonacci sequence is the sequence of numbers 1, 1, 2, 3, 5, 8,
13, 21, 34, etc., where each number (from the third on) is the sum
of the previous two. Create a method that takes an integer as an
argument and displays that many Fibonacci numbers starting from the
beginning. If, e.g., you run java Fibonacci 5 (where Fibonacci is
the name of the class) the output will be: 1, 1, 2, 3, 5.
I could have swore that it would need an array or some way of storing the previous numbers but when I saw the answer it wasn't the case:
import java.util.*;
public class Fibonacci {
static int fib(int n) {
if (n <= 2)
return 1;
return fib(n-1) + fib(n-2);
}
public static void main(String[] args) {
// Get the max value from the command line:
int n = Integer.parseInt(args[0]);
if(n < 0) {
System.out.println("Cannot use negative numbers"); return;
}
for(int i = 1; i <= n; i++)
System.out.print(fib(i) + ", ");
}
}
Would someone be able to explain how using a function within itself produces this?

The code you gave is an example of a recursive solution. When the function is called for the first time, it executes until it calls itself. Its state is stored on the stack, and the code begins executing again with new input data. This process repeats until the input is less than 2, at which point 1 is returned, and the answer returns to the previous caller.
For example, calling fib(5) results in the following execution:
fib(5):
fib(4):
fib(3):
fib(2): 1
fib(1): 1
fib(2): 1
fib(3):
fib(2): 1
fib(1): 1
Note that you are partially correct. Intermediate results are stored on the stack in this solution. That is one of the reasons why the recursive solution is so expensive. The other is its O(2^n) complexity. However, if is possible to compute Fibonacci(n) iteratively and without storing all previous results. All you really need is to store the last to results and count from 1 up to n.

This is a recursive solution. A recursive function calls itself until a given stop condition is met. Then each call exits until only the first call remains. That first call outputs the result.
In your solution, the stop condition is :
if (n <= 2)
return 1;
until this condition is met, the function will make successive calls to itself. Each call reduces the int passed as a parameter. When it reaches 2, the stop condition dictates that the function returns with value 1 (the result of n=1 and n=2 in fibonacci(n) ).
Because the fibonacci is the sum of the last two numbers, the recursive part of the function,
return fib(n-1) + fib(n-2);
does a sum of n-1 and n-2 (as I said, the two last numbers of the sequence). When the n equals 2, these calls to function fib will finally have a value, and will be returned.
Example, if n = 3, the recursive part will call fib(2) and fib(1), both are equal or less than 2, so both calls will return 1. So the printf will print 1, 1, 2.
2 is the sum of 1 + 1 (I know it's obvious, but sometimes stating the obvious helps).

F(n)
/ \
F(n-1) F(n-2)
/ \ / \
F(n-2) F(n-3) F(n-3) F(n-4)
/ \
F(n-3) F(n-4)
Important point to note is this algorithm is exponential because it does not store the result of previous calculated numbers. eg F(n-3) is called 3 times.
For more details refer algorithm by dasgupta chapter 0.2

This is the standard example for an recursive function.
The Fibonacci Numbers are declared recursive, too.
Both fib(1) and fib(2) are declared as 1. All others are calculated by adding the last two fibonacci numbers, so fib(3)=fib(2)+fib(1).
Compare this to the calculation method which does exactly this:
static int fib(int n) {
if (n <= 2)
return 1;
return fib(n-1) + fib(n-2);
}
By the way, this is a very slow way to calculate the fibonacci numbers, using two variables (you don't need all of them for the last one, only the last two!) and a loop is in O(n), the recursive function above has O(2^n) operations.

How does the recursion here work?

Code 1:
public static int fibonacci (int n){
if (n == 0 || n == 1) {
return 1;
} else {
return fibonacci (n-1) + fibonacci (n-2);
}
}
How can you use fibonacci if you haven't gotten done explaining what it is yet? I've been able to understand using recursion in other cases like this:
Code 2:
class two
{
public static void two (int n)
{
if (n>0)
{
System.out.println (n) ;
two (n-1) ;
}
else
{
return ;
}
}
public static void main (String[] arg)
{
two (12) ;
}
}
In the case of code 2, though, n will eventually reach a point at which it doesn't satisfy n>0 and the method will stop calling itself recursively. In the case of code 2, though, I don't see how it would be able to get itself from 1 if n=1 was the starting point to 2 and 3 and 5 and so on. Also, I don't see how the line return fibonacci (n-1) + fibonacci (n-2) would work since fibonacci (n-2) has to contain in some sense fibonacci (n-1) in order to work, but it isn't there yet.
The book I'm looking at says it will work. How does it work?

Well, putting aside what a compiler actually does to your code (it's horrible, yet beautiful) and what how a CPU actually interprets your code (likewise), there's a fairly simple solution.
Consider these text instructions:
To sort numbered blocks:
pick a random block.
if it is the only block, stop.
move the blocks
with lower numbers to the left side,
higher numbers to the right.
sort the lower-numbered blocks.
sort the higher-numbered blocks.
When you get to instructions 4 and 5, you are being asked to start the whole process over again. However, this isn't a problem, because you still know how to start the process, and when it all works out in the end, you've got a bunch of sorted blocks. You could cover the instructions with slips of paper and they wouldn't be any harder to follow.

In the case of code 2 though n will eventualy reach a point at which it doesnt satisfy n>0 and the method will stop calling itself recursivly
to make it look similar you can replace condition if (n == 0 || n == 1) with if (n < 2)
Also i don't see how the line `return fibonacci (n-1) + fibonacci (n-2) would work since fibbonacci n-2 has to contain in some sense fibonacci n-1 in order to wrok but it isn't there yet.
I suspect you wanted to write: "since fibbonacci n-1 has to contain in some sense fibonacci n-2"
If I'm right, then you will see from the example below, that actually fibonacci (n-2) will be called twice for every recursion level (fibonacci(1) in the example):
1. when executing fibonacci (n-2) on the current step
2. when executing fibonacci ((n-1)-1) on the next step
(Also take a closer look at the Spike's comment)
Suppose you call fibonacci(3), then call stack for fibonacci will be like this:
(Veer provided more detailed explanation)
n=3. fibonacci(3)
n=3. fibonacci(2) // call to fibonacci(n-1)
n=2. fibonacci(1) // call to fibonacci(n-1)
n=1. returns 1
n=2. fibonacci(0) // call to fibonacci(n-2)
n=0. returns 1
n=2. add up, returns 2
n=3. fibonacci(1) //call to fibonacci(n-2)
n=1. returns 1
n=3. add up, returns 2 + 1
Note, that adding up in fibonacci(n) takes place only after all functions for smaller args return (i.e. fibonacci(n-1), fibonacci(n-2)... fibonacci(2), fibonacci(1), fibonacci(0))
To see what is going on with call stack for bigger numbers you could run this code.
public static String doIndent( int tabCount ){
String one_tab = new String(" ");
String result = new String("");
for( int i=0; i < tabCount; ++i )
result += one_tab;
return result;
}
public static int fibonacci( int n, int recursion_level )
{
String prefix = doIndent(recursion_level) + "n=" + n + ". ";
if (n == 0 || n == 1){
System.out.println( prefix + "bottommost level, returning 1" );
return 1;
}
else{
System.out.println( prefix + "left fibonacci(" + (n-1) + ")" );
int n_1 = fibonacci( n-1, recursion_level + 1 );
System.out.println( prefix + "right fibonacci(" + (n-2) + ")" );
int n_2 = fibonacci( n-2, recursion_level + 1 );
System.out.println( prefix + "returning " + (n_1 + n_2) );
return n_1 + n_2;
}
}
public static void main( String[] args )
{
fibonacci(5, 0);
}

The trick is that the first call to fibonacci() doesn't return until its calls to fibonacci() have returned.
You end up with call after call to fibonacci() on the stack, none of which return, until you get to the base case of n == 0 || n == 1. At this point the (potentially huge) stack of fibonacci() calls starts to unwind back towards the first call.
Once you get your mind around it, it's kind of beautiful, until your stack overflows.

"How can you use Fibonacci if you haven't gotten done explaining what it is yet?"
This is an interesting way to question recursion. Here's part of an answer: While you're defining Fibonacci, it hasn't been defined yet, but it has been declared. The compiler knows that there is a thing called Fibonacci, and that it will be a function of type int -> int and that it will be defined whenever the program runs.
In fact, this is how all identifiers in C programs work, not just recursive ones. The compiler determines what things have been declared, and then goes through the program pointing uses of those things to where the things actually are (gross oversimplification).

Let me walkthrough the execution considering n=3. Hope it helps.
When n=3 => if condition fails and else executes
return fibonacci (2) + fibonacci (1);
Split the statement:
Find the value of fibonacci(2)
Find the value of fibonacci(1)
// Note that it is not fib(n-2) and it is not going to require fib(n-1) for its execution. It is independent. This applies to step 1 also.
Add both values
return the summed up value
The way it gets executed(Expanding the above four steps):
Find the value of fibonacci(2)
if fails, else executes
fibonacci(1)
if executes
value '1' is returned to step 1.2. and the control goes to step 1.3.
fibonacci(0)
if executes
value '1' is returned to step 1.3. and the control goes to step 1.4.
Add both
sum=1+1=2 //from steps 1.2.2. and 1.3.2.
return sum // value '2' is returned to step 1. and the control goes to step 2
Find the value of fibonacci(1)
if executes
value '1' is returned
Add both values
sum=2+1 //from steps 1.5. and 2.2.
return the summed up value //sum=3

Try to draw an illustration yourself, you will eventually see how it works. Just be clear that when a function call is made, it will fetch its return value first. Simple.

Try debugging and use watches to know the state of the variable

Understanding recursion requires also knowing how the call stack works i.e. how functions call each other.
If the function didn't have the condition to stop if n==0 or n==1, then the function would call itself recursively forever.
It works because eventually, the function is going to petter out and return 1. at that point, the return fibonacci (n-1) + fibonacci (n-2) will also return with a value, and the call stack gets cleaned up really quickly.

I'll explain what your PC is doing when executing that piece of code with an example:
Imagine you're standing in a very big room. In the room next to this room you have massive amounts of paper, pens and tables. Now we're going to calculate fibonacci(3):
We take a table and put it somewhere in the room. On the table we place a paper and we write "n=3" on it. We then ask ourselves "hmm, is 3 equal to 0 or 1?". The answer is no, so we will do "return fibonacci (n-1) + fibonacci (n-2);".
There's a problem however, we have no idea what "fibonacci (n-1)" and "fibonacci (n-2)" actually do. Hence, we take two more tables and place them to the left and right of our original table with a paper on both of them, saying "n=2" and "n=1".
We start with the left table, and wonder "is 2 equal to 0 or 1?". Of course, the answer is no, so we will once again place two tables next to this table, with "n=1" and "n=0" on them.
Still following? This is what the room looks like:
n=1
n=2 n=3 n=1
n=0
We start with the table with "n=1", and hey, 1 is equal to 1, so we can actually return something useful! We write "1" on another paper and go back to the table with "n=2" on it. We place the paper on the table and go to the other table, because we still don't know what we're going to do with that other table.
"n=0" of course returns 1 as well, so we write that on a paper, go back to the n=2 table and put the paper there. At this point, there are two papers on this table with the return values of the tables with "n=1" and "n=0" on them, so we can compute that the result of this method call is actually 2, so we write it on a paper and put it on the table with "n=3" on it.
We then go to the table with "n=1" on it all the way to the right, and we can immediately write 1 on a paper and put it back on the table with "n=3" on it. After that, we finally have enough information to say that fibonacci(3) returns 3.
It's important to know that the code you are writing is nothing more than a recipe. All the compiler does is transform that recipe in another recipe your PC can understand. If the code is completely bogus, like this:
public static int NotUseful()
{
return NotUseful();
}
will simply loop endlessly, or as in my example, you'll keep on placing more and more tables without actually getting anywhere useful. Your compiler doesn't care what fibonacci(n-1) or fibonacci(n-2) actually do.