We Keep Coding

Trying to understand the reason for this time complexity

I am trying to calculate the time complexity of these two algorithms. The book I am referring to specifies these time complexities of each.
A) Algorithm A: O(nlogn)
int i = n;
while (i > 0)
{
for (int j = 0; j < n; j++)
System.out.println("*");
i = i / 2;
}
B) Algorithm B: O(n)
while (n > 0)
{
for (int j = 0; j < n; j++)
System.out.println("*");
n = n / 2;
}
I can see how algo. A is O(nlogn). The for loop is O(n) and while loop is O(logn). However I am failing to see how AlgoB has a time complexity of O(n). I was expecting it to be O(nlogn) also. Any help would be appreciated.

Let's look at Algorithm B from a mathematical standpoint.
The number of * printed in the first loop is n. The number of * printed in each subsequent loop is at most n/2. That recurrence relation leads to the sequence:
n + n/2 + n/4 + n/8 + ...
If this were an infinite sequence, then the sum could be represented by the formula n/(1 - r), where r is the factor between terms. In this case, r is 1/2, so the infinite sequence has a sum of 2(n).
Your algorithm certainly won't go on forever, and it each time it loops, it may be printing less than half the stars of the previous loop. The number of stars printed is therefore less than or equal to 2(n).
Because constant factors drop out of time complexity, the algorithm is O(n).
This concept is called amortized complexity, which averages the cost of operations in a loop, even if some of the operations might be relatively expensive. Please see this question and the Wikipedia page for a more thorough explanation of amortized complexity.

Algorithm B is printing half the starts at every iteration. Assume n=10, then:
n=10 -> 10*
n=5 -> 5*
n=2 -> 2*
n=1 -> 1*
In total 18* are printed. You will print n + n/2 + n/4 + ... + n/(2^i) stars. How much does i value? It is equal to the number of steps required for n to become 0. In other terms, it is the exponent to which 2 must be raised to produce n: log_2(n). You get the sum in picture:
Which can be approximated to O(n).

A beginner way to understand how time complexity works [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I've been researching a lot about time complexity for my Data Structures class. And I've been tasked to report about Shell sort algorithm and explain its time complexity (best/worst/average case). I found this site https://stackabuse.com/shell-sort-in-java/ that shows that the time complexity of this Shell sort algorithm:
void shellSort(int array[], int n){
//n = array.length
for (int gap = n/2; gap > 0; gap /= 2){
for (int i = gap; i < n; i += 1) {
int temp = array[i];
int j;
for (j = i; j >= gap && array[j - gap] > temp; j -= gap){
array[j] = array[j - gap];
}
array[j] = temp;
}
}
}
is O(n log n). But the problem is that I'm still confused about makes logn a logn or what does nlogn means.
I also tried step count method but again, I don't know where to start so I just copied from the site above and did this.
void shellSort(int array[], int n){
//n = array.length
for (int gap = n/2; gap > 0; gap /= 2){ //step 1 = runs logn times
for (int i = gap; i < n; i += 1) { //step 2 = runs n-gap times
int temp = array[i]; //step 3 = 1
int j; //step 4 = 1
for (j = i; j >= gap && array[j - gap] > temp; j -= gap){ //step 5 = i/gap times
array[j] = array[j - gap]; //step 6 = 1
}
array[j] = temp; //step 7 = 1
}
}
}
But I don't know if this is correct, I just based it off on this site. https://stackabuse.com/shell-sort-in-java/.
I've also tried comparing the total number of moves between Insertion Sort and Shell Sort since Shell Sort is a generalization of Insertion and Bubble Sort. I'll attach the pics below. I also used an online number generator that will give me 100 random numbers, copied it and applied it to both the Insertion Sort and Shell sort and assigned it as the array to sort.
And this was what came up,
Total number of moves of Insertion Sort = 4731
Total number of moves of Shell Sort = 1954
Shell Sort implementation that tells me the total number of moves it does
Insertion Sort implementation that tells me the total number of moves it does
What I've understood from all of this is that despite Shell sort being a generalization of Insertion sort, when it comes to sorting large arrays such as 100 elements Shell Sort is 2x faster than Insertion Sort.
But the ultimate question is, is there a beginner way to calculate the time complexity like this Shell Sort algorithm?

You have to take a look at the big O or big Theta analysis of your function. Your outer loop is being divided by half on every iteration so the overall time that it runs is log n. Now when you look at your inner loop it runs initially from n/2 to n all the way to 1 to n or 2 to n depending on the initial size of n so its execution time will be n/2 + n/4 + .... n /2^k which its a 'Harmonic series' (You can search geometric series as well, if you factor n -> n(1/2 + 1/4 + ... + 1/2^k) which equals nlogn. Now the best case where every list may be sorted to some extent will be Ω(nlogn) as the in the middle of the outer loop we will find optimal solution so we can say that nlogn is its lower bound - Meaning it is definitely equal or bigger than that - therefor we can say that the average case is Θ(nlog^2 n) meaning that it is in the tight bound of that - Please note for average case I use Big theta. Now if we assume that the list is completely reverse the outer loop will run all the way to the end meaning log n. The inter loop will run nlogn so the total time will be nlog^2(n) which we can say it will be O(nlog^2(n)) (we can also use Big O but theta is better you can search that up that how theta provides tight bound and big O only provides upper bound). Therefore, we can also say the worst case is O(n^2) which is relatively correct in some context.
I suggest you take a look at Big-O and Big-Theta as well as Big-Omega which can also be useful in this case.
However, the most precise mathematical representation for shell algorithm will be O(n^3/2). However, there are still arguments and analyzation taking place.
I hope this helps.

First, I'll show that the algorithm will never be slower than O(n^2), and then I'll show that the worst-case run time is at least O(n^2).
Assume n is a power of two. We know the worst case for insertion sort is O(n^2). When h-sorting the array, we're performing h insertion sorts, each on an array of size n / h. So the complexity for a h-sort pass is O(h * (n / h)^2) = O(n^2 / h). The complexity of the whole algorithm is now the sum of n^2 / h where h is each power of two up to n / 2. This is a geometric series with first term n^2, common ratio 1 / 2, and log2(n) terms. Using the geometric series sum formula gives n^2*((1 / 2)^log2(n) - 1) / (1 / 2 - 1) = n^2*(1 / n - 1) / (-1 / 2) = n^2*(-2 / n + 2) = 2n^2 - 2n = O(n^2).
Consider an array created by interweaving two increasing sequences, where all elements in one sequence is greater than all elements in the other sequence, such as [1, 5, 2, 6, 3, 7, 4, 8]. Since this array is two-sorted, all passes except the last one does nothing. In the last pass, an element at index i where i is even has to be moved to index i / 2, which uses O(i / 2) operations. So we have 1 + 2 + 3 + ... + n / 2 = (n / 2) * (n / 2 + 1) / 2 = O(n^2).

Big-O complexity of recursion with nested loops

Preparing for exams I came across this question in an old exam:
What is the worst case/big-O complexity of this function:
float foo(float[] A) {
int n = A.length;
if (n == 1) return A[0];
float[] A1 = new float[n/2];
float[] A2 = new float[n/2];
float[] A3 = new float[n/2];
float[] A4 = new float[n/2];
for (i = 0; i <= (n/2)-1; i++) {
for (j = 0; j <= (n/2)-1; j++) {
A1[i] = A[i];
A2[i] = A[i+j];
A3[i] = A[n/2+j];
A4[i] = A[j];
}
}
return foo(A1)
+ foo(A2)
+ foo(A3)
+ foo(A4);
}
(Yes, the code makes no sense, but this is exactly the way it was written).
What's tripping me up is that the total size of n doubles for each recursive level, but the suggested answer(with the end result O(log n * n^2)) ignores that part. Am I misunderstanding things?
Edit: Replaced the semi-pseudocode with syntactically correct(but still nonsensical) code.

If you solve this recursive relation, you'd be able to determine the complexity.
T(n) = 4T(n/2) + O(n²)
With
T(1) = c

Okay, I finally figured it out.
Every time we recurse we do 4 times as many function calls as last time, so if we define the recursion level as m the number of function calls per level is
Every time we recurse we also halve the size of the array, so the amount of work per function call is
At each recursive level the work done in total then is:
In fact 4^m is the same as (2^m)^2:
Thus the amount of work can be written as just n^2:
There are log n recursive levels.
Thus the total amount of work is O(n^2 * log n), and that is because there are 4 recursive calls.
If there were just 2 recursive calls the amount of work at each level would be
which we can't reduce nicely(but turns out to be in O(n^2) if my math is correct).

Writing a method for factorial

I'm studying the code below in my textbook. It uses the combinations method and the factorial method to calculate the possible outcomes given n and k. My question is with the factorial method specifically the content in the for loop.
I understand everything else about the program but I don't understand the code i <=n in the for loop in the factorial method. What other part of the program is n referred to? I'm just not sure about the rationale behind i <= n or how the programmer comes up with that.
import acm.program.*;
public class combinations extends ConsoleProgram {
public void run(){
int n = readInt("Enter the number of objects in the set (n): ");
int k = readInt("Enter the number to be chosen (k): ");
println("C (" + n + ", " + k + ") = " + combinations (n, k) );
}
private int combinations (int n, int k){
return factorial (n) / (factorial (k) * factorial (n-k));
}
private int factorial (int n){
int result = 1;
for (int i = 1; i <= n; i++){
result *= i;
}
return result;
}
}

n is a parameter of the method: because the method is declared as int factorial(int n), you invoke it as (for example) factorial(5) to get the local variable n set to 5. (In formal terms, n is the parameter and 5 is the argument, though usually people don't bother to distinguish those two terms.)

So, a bit of math. Typically, when one deals with a mathematical expression, math conventions are frequently used. n is typically referred to some upper ceiling value that the method should refer to.
Essentially, the function definition for a factorial is this.
factorial(n) = { 1 if n = 0, n*factorial(n-1) otherwise.
The loop includes the final value of n, so you get the full expression of the function (if you didn't, your answer would be off by a factor of n every time.

You need i <= n because when you calculate factorial of 3! for example, you will have
3! = 3 * 2 * 1 <=> 1 * 2 * 3
So, you have your n, that is 3 and the i is 1, then 2, then 3 (n).

If you look closely you see that in loop for i take a values from 1 to n, so in i=n point for loop terminated. Loop for is created in this form to assure that factorial(0)=1. However, you can redesign this function in recursive style.

Bounding this program to determine the sum of reciprocal integers not containing zero

Let A denote the set of positive integers whose decimal representation does not contain the digit 0. The sum of the reciprocals of the elements in A is known to be 23.10345.
Ex. 1,2,3,4,5,6,7,8,9,11-19,21-29,31-39,41-49,51-59,61-69,71-79,81-89,91-99,111-119, ...
Then take the reciprocal of each number, and sum the total.
How can this be verified numerically?
Write a computer program to verify this number.
Here is what I have written so far, I need help bounding this problem as this currently takes too long to complete:
Code in Java
import java.util.*;
public class recip
{
public static void main(String[] args)
{
int current = 0; double total = 0;
while(total < 23.10245)
{
if(Integer.toString(current).contains("0"))
{
current++;
}
else
{
total = total + (1/(double)current);
current++;
}
System.out.println("Total: " + total);
}
}
}

This is not that hard when approached properly.
Assume for example that you want to find the sum of reciprocals of all integers starting (i.e. the left-most digits) with 123 and ending with k non-zero digits. Obviously there are 9k such integers and the reciprocal of each of these integers is in the range 1/(124*10k) .. 1/(123*10k). Hence the sum of reciprocals of all these integers is bounded by (9/10)k/124 and (9/10)k/123.
To find bounds for sum of all reciprocals starting with 123 one has to add up the bounds above for every k>=0. This is a geometric serie, hence it can be derived that the sum of reciprocals of integers starting with 123 is bounded by 10*(9/10)k/124 and 10*(9/10)k/123.
The same method can of course be applied for any combination of left-most digits.
The more digits we examine on the left, the more accurate the result becomes.
Here is an implementation of this approach in python:
def approx(t,k):
"""Returns a lower bound and an upper bound on the sum of reciprocals of
positive integers starting with t not containing 0 in its decimal
representation.
k is the recursion depth of the search, i.e. we append k more digits
to t, before approximating the sum. A larger k gives more accurate
results, but takes longer."""
if k == 0:
return 10.0/(t+1), 10.0/t
else:
if t > 0:
low, up = 1.0/t, 1.0/t
else:
low, up = 0, 0
for i in range(10*t+1, 10*t+10):
l,u = approx(i, k-1)
low += l
up += u
return low, up
Calling approx(0, 8) for example gives the lower and upper bound:
23.103447707... and 23.103448107....
which is close to the claim 23.10345 given by the OP.
There are methods that converge faster to the sum in question, but they require more math.
A much better approximation of the sum can be found here. A generalization of the problem are the Kempner series.

For all values of current greater than some threshold N, 1.0/(double)current will be sufficiently small that total does not increase as a result of adding 1.0/(double)current. Thus, the termination criterion should be something like
while(total != total + (1.0/(double)current))
instead of testing against the limit that is known a priori. Your loop will stop when current reaches this special value of N.

I suspect that casting to string and then checking for the character '0' is the step that takes too long. If you want to avoid all zeroes, might help to increase current thus:
(Edited -- thanks to Aaron McSmooth)
current++;
for( int i = 10000000; i >= 10; i = i / 10 )
{
if ( current % i ) == 0
{
current = current + ( i / 10 );
}
}
This is untested, but the concept should be clear: whenever you hit a multiple of a power of ten (e.g. 300 or 20000), you add the next lower power of 10 (in our examples 10 + 1 and 1000 + 100 + 10 + 1, respectively) until there are no more zeroes in your number.
Change your while loop accordingly and see if this doesn't help performance to the point were your problem becomes manageable.
Oh, and you might want to restrict the System.out output a bit as well. Would every tenth, one hundreth or 10000th iteration be enough?
Edit the second:
After some sleep, I suspect my answer might be a little short-sighted (blame the late hour, if you will). I simply hoped that, oh, one million iterations of current would get you to the solution and left it at that, instead of calculating the correction cases using log( current ) etc.
On second thought, I see two problems with this whole problem. One is that your target number of 23.10345 is a leeeeettle to round for my tastes. After all, you are adding thousands of items like "1/17", "1/11111" and so on, with infinite decimal representations, and it is highly unlikely that they add up to exactly 23.10345. If some specialist for numerical mathematics says so, fine -- but then I'd like to see the algorithm by which they arrived at this conclusion.
The other problem is related to the first and concerns the limited in-memory binary representation of your rational numbers. You might get by using BigDecimals, but I have my doubts.
So, basically, I suggest you reprogram the numerical algorithm instead of going for the brute force solution. Sorry.
Edit the third:
Out of curiosity, I wrote this in C++ to test my theories. It's run for 6 minutes now and is at about 14.5 (roughly 550 mio. iterations). We'll see.
Current version is
double total = 0;
long long current = 0, currPowerCeiling = 10, iteration = 0;
while( total < 23.01245 )
{
current++;
iteration++;
if( current >= currPowerCeiling )
currPowerCeiling *= 10;
for( long long power = currPowerCeiling; power >= 10; power = power / 10 )
{
if( ( current % power ) == 0 )
{
current = current + ( power / 10 );
}
}
total += ( 1.0 / current );
if( ! ( iteration % 1000000 ) )
std::cout << iteration / 1000000 << " Mio iterations: " << current << "\t -> " << total << std::endl;
}
std::cout << current << "\t" << total << std::endl;
Calculating currPowerCeiling (or however one might call this) by hand saves some log10 and pow calculations each iteration. Every little bit helps -- but it still takes forever...
Edit the fourth:
Status is around 66,000 mio iterations, total is up to 16.2583, runtime is at around 13 hours. Not looking good, Bobby S. -- I suggest a more mathematical approach.

How about storing the current number as a byte array where each array element is a digit 0-9? That way, you can detect zeroes very quickly (comparing bytes using == instead of String.contains).
The downside would be that you'll need to implement the incrementing yourself instead of using ++. You'll also need to devise a way to mark "nonexistent" digits so that you don't detect them as zeroes. Storing -1 for nonexistent digits sounds like a reasonable solution.

For a signed 32-bit integer, this program will never stop. It will actually converge towards -2097156. Since the maximum harmonic number (the sum of integral reciprocals from 1 to N) of a signed 32-bit integer is ~14.66, this loop will never terminate, even when current wraps around from 2^31 - 1 to -2^31. Since the reciprocal of the largest negative 32-bit integer is ~-4.6566e-10, every time current returns to 0, the sum will be negative. Given that the largest number representable by a double such that number + + 1/2^31 == number is 2^52/2^31, you get roughly -2097156 as the converging value.
Having said that, and assuming you don't have a direct way of calculating the harmonic number of an arbitrary integer, there are a few things you can do to speed up your inner loop. First, the most expensive operation is going to be System.out.println; that has to interact with the console in which case your program will eventually have to flush the buffer to the console (if any). There are cases where that may not actually happen, but since you are using that for debugging they are not relevant to this question.
However, you also spend a lot of time determining whether a number has a zero. You can flip that test around to generate ranges of integers such that within that range you are guaranteed not to have an integer with a zero digit. That is really simple to do incrementally (in C++, but trivial enough to convert to Java):
class c_advance_to_next_non_zero_decimal
{
public:
c_advance_to_next_non_zero_decimal(): next(0), max_set_digit_index(0)
{
std::fill_n(digits, digit_count, 0);
return;
}
int advance_to_next_non_zero_decimal()
{
assert((next % 10) == 0);
int offset= 1;
digits[0]+= 1;
for (int digit_index= 1, digit_value= 10; digit_index<=max_set_digit_index; ++digit_index, digit_value*= 10)
{
if (digits[digit_index]==0)
{
digits[digit_index]= 1;
offset+= digit_value;
}
}
next+= offset;
return next;
}
int advance_to_next_zero_decimal()
{
assert((next % 10)!=0);
assert(digits[0]==(next % 10));
int offset= 10 - digits[0];
digits[0]+= offset;
assert(digits[0]==10);
// propagate carries forward
for (int digit_index= 0; digits[digit_index]==10 && digit_index<digit_count; ++digit_index)
{
digits[digit_index]= 0;
digits[digit_index + 1]+= 1;
max_set_digit_index= max(digit_index + 1, max_set_digit_index);
}
next+= offset;
return next;
}
private:
int next;
static const size_t digit_count= 10; // log10(2**31)
int max_set_digit_index;
int digits[digit_count];
};
What the code above does is to iterate over every range of numbers such that the range only contains numbers without zeroes. It works by determining how to go from N000... to N111... and from N111... to (N+1)000..., carrying (N+1) into 1(0)000... if necessary.
On my laptop, I can generate the harmonic number of 2^31 - 1 in 8.73226 seconds.

public class SumOfReciprocalWithoutZero {
public static void main(String[] args) {
int maxSize=Integer.MAX_VALUE/10;
long time=-System.currentTimeMillis();
BitSet b=new BitSet(maxSize);
setNumbersWithZeros(10,maxSize,b);
double sum=0.0;
for(int i=1;i<maxSize;i++)
{
if(!b.get(i))
{
sum+=1.0d/(double)i;
}
}
time+=System.currentTimeMillis();
System.out.println("Total: "+sum+"\nTimeTaken : "+time+" ms");
}
static void setNumbersWithZeros(int srt,int end,BitSet b)
{
for(int j=srt;j<end;j*=10)
{
for(int i=1;i<=10;i++)
{
int num=j*i;
b.set(num);
}
if(j>=100)
setInbetween(j, b);
}
}
static void setInbetween(int strt,BitSet b)
{
int bitToSet;
bitToSet=strt;
for(int i=1;i<=10;i++)
{
int nxtInt=-1;
while((nxtInt=b.nextSetBit(nxtInt+1))!=strt)
{
b.set(bitToSet+nxtInt);
}
nxtInt=-1;
int lim=strt/10;
while((nxtInt=b.nextClearBit(nxtInt+1))<lim)
{
b.set(bitToSet+nxtInt);
}
bitToSet=strt*i;
}
}
}
This is an implementation using BitSet.I calculated the sum of reciprocal's for all integer's in range (1-Integer.MAX_VALUE/10).The sum comes upto 13.722766931560747.This is the maximum I could calculate using BitSet since the maximum range for BitSet is Integer.MAX_VALUE.I need to divide it by 10 and limit the range to avoid overflow.But there is significant improvement in speed.I'm just posting this code in-case it might give you some new idea to improve your code.(Increase your memory using the VM argument -Xmx[Size>350]m)
Output:
Total: 13.722766931560747
TimeTaken : 60382 ms
UPDATE:
Java Porting of a previous , deleted answer :
public static void main(String[] args) {
long current =11;
double tot=1 + 1.0/2 + 1.0/3 + 1.0/4 + 1.0/5 + 1.0/6 + 1.0/7 + 1.0/8 + 1.0/9;
long i=0;
while(true)
{
current=next_current(current);
if(i%10000!=0)
System.out.println(i+" "+current+" "+tot);
for(int j=0;j<9;j++)
{
tot+=(1.0/current + 1.0/(current + 1) + 1.0/(current + 2) + 1.0/(current + 3) + 1.0/(current + 4) +
1.0/(current + 5) + 1.0/(current + 6) + 1.0/(current + 7) + 1.0/(current + 8));
current += 10;
}
i++;
}
}
static long next_current(long n){
long m=(long)Math.pow(10,(int)Math.log10(n));
boolean found_zero=false;
while(m>=1)
{
if(found_zero)
n+=m;
else if((n/m)%10==0)
{
n=n-(n%m)+m;
found_zero=true;
}
m=m/10;
}
return n;
}