We Keep Coding

Why is a Java BitSet packed in 6 bytes?

I was looking into BitSet and the following are not clear to me:
When we pass the number of bits there is a division by 6. Why is 6 being used and not some power of 2?
When initializing the underlying array why is there first a subtraction by 1 before the division by 6 followed by an addition of 1?

I'm assuming you are asking about this bit of code in the JDK:
private static int wordIndex(int bitIndex) {
return bitIndex >> ADDRESS_BITS_PER_WORD; // ADDRESS_BITS_PER_WORD is 6, question 1
}
public BitSet(int nbits) {
// nbits can't be negative; size 0 is OK
if (nbits < 0)
throw new NegativeArraySizeException("nbits < 0: " + nbits);
initWords(nbits);
sizeIsSticky = true;
}
private void initWords(int nbits) {
words = new long[wordIndex(nbits-1) + 1]; // question 2
}
initWords initialises a long[] to back the bits, essentially storing the bits into "words" of 64 bits. Note that this seems to be an implementation detail. How long should this long[] be? Well, it should be the word index of the last word + 1, because indices are zero-based.
What is the index of the last word? Well, the wordIndex method can tell us the word index of a bit, so if we give it the index of the last bit, nbits - 1 (again because indices are zero-based), it will give us what we want. This should answer your second question.
How does wordIndex find the word index? Well, there are 64 bits in a long, so we just need to divide the bitIndex by 64. What's another way of dividing by 64? Shift left 6 times, since 64 = 2 to the power 6. See this post for more info.

I suppose you talk about the implementation of this class?
It says so in a comment in the source file:
/*
* BitSets are packed into arrays of "words." Currently a word is
* a long, which consists of 64 bits, requiring 6 address bits.
* The choice of word size is determined purely by performance concerns.
*/
So from a given bit number, the lower 6 bits are used for addressing a bit withing a 64 bit word, and the remaining bits are for addressing the word.
For the point 2, I suppose you talk about
wordIndex(nbits-1) + 1
which is
bitIndex >> ADDRESS_BITS_PER_WORD
Suppose you want to initialize a BitSet with initially 0 entries. Then you need an array size of 0.
Suppose you want to initialize a BitSet with initially 1 to 64 entries. Then you need an array size of 1.
Suppose you want to initialize a BitSet with initially 65 to 128 entries. Then you need an array size of 2.
And so on.
This means, you map the original range (1-64, 65-128) to "one less" (0-63, 64-127), divide by 64 (0, 1) and increase the result again (1, 2) to get the number of needed words in the array.
To demonstrate both:
Suppose you want a BitSet with 128 entries. You initialize it and you get an array with 2 64 bit entries. Why?
That's because wach word can hold 64 bits, so in order to hold 128 bits, you need 2 array entries:
(128-1)/64 + 1 = 127/64 + 1 = 1 + 1 = 2. (Remember that integer divisions go towards the lower value.)
Now, you want to set the bis 5, 13 and 66.
Bit 5 and 13 are fine - you just set bits 5 and 13 in the word at index 0.
But what do you do with the 66? Each word has only 64 bits! (0..63)
Well, in this situation, you substract 64 for each step you make in the array. So you go to word at index 1 and for "compensation", you go from 66 to 2.
That's exactly what happens with these bit manipulations: from each of these bit indexes, the lower 6 bits are taken and used as bit address in the respective word.
5 = 0 000101 = 0/5
13 = 0 001101 = 0/13
66 = 1 000001 = 1/2
So
5 is set by setting bit 5 in the word 0.
13 is set by setting bit 13 in the word 0.
66 is set by setting bit 2 in the word 1.

Competitive Coding - Clearing all levels with minimum cost : Not passing all test cases

I was solving problems on a competitive coding website when I came across this. The problem states that:
In this game there are N levels and M types of available weapons. The levels are numbered from 0 to N-1 and the weapons are numbered from 0 to M-1 . You can clear these levels in any order. In each level, some subset of these M weapons is required to clear this level. If in a particular level, you need to buy x new weapons, you will pay x^2 coins for it. Also note that you can carry all the weapons you have currently to the next level . Initially, you have no weapons. Can you find out the minimum coins required such that you can clear all the levels?
Input Format
The first line of input contains 2 space separated integers:
N = the number of levels in the game
M = the number of types of weapons
N lines follows. The ith of these lines contains a binary string of length M. If the jth character of
this string is 1 , it means we need a weapon of type j to clear the ith level.
Constraints
1 <= N <=20
1<= M <= 20
Output Format
Print a single integer which is the answer to the problem.
Sample TestCase 1
Input
1 4
0101
Output
4
Explanation
There is only one level in this game. We need 2 types of weapons - 1 and 3. Since, initially Ben
has no weapons he will have to buy these, which will cost him 2^2 = 4 coins.
Sample TestCase 2
Input
3 3
111
001
010
Output
3
Explanation
There are 3 levels in this game. The 0th level (111) requires all 3 types of weapons. The 1st level (001) requires only weapon of type 2. The 2nd level requires only weapon of type 1. If we clear the levels in the given order(0-1-2), total cost = 3^2 + 0^2 + 0^2 = 9 coins. If we clear the levels in the order 1-2-0, it will cost = 1^2 + 1^2 + 1^2 = 3 coins which is the optimal way.
Approach
I was able to figure out that we can calculate the minimum cost by traversing the Binary Strings in a way that we purchase minimum possible weapons at each level.
One possible way could be traversing the array of binary strings and calculating the cost for each level while the array is already arranged in the correct order. The correct order should be when the Strings are already sorted i.e. 001, 010, 111 as in case of the above test case. Traversing the arrays in this order and summing up the cost for each level gives the correct answer.
Also, the sort method in java works fine to sort these Binary Strings before running a loop on the array to sum up cost for each level.
Arrays.sort(weapons);
This approach work fine for some of the test cases, however more than half of the test cases are still failing and I can't understand whats wrong with my logic. I am using bitwise operators to calculate the number of weapons needed at each level and returning their square.
Unfortunately, I cannot see the test cases that are failing. Any help is greatly appreciated.

This can be solved by dynamic programming.
The state will be the bit mask of weapons we currently own.
The transitions will be to try clearing each of the n possible levels in turn from the current state, acquiring the additional weapons we need and paying for them.
In each of the n resulting states, we take the minimum cost of the current way to achieve it and all previously observed ways.
When we already have some weapons, some levels will actually require no additional weapons to be bought; such transitions will automatically be disregarded since in such case, we arrive at the same state having paid the same cost.
We start at the state of m zeroes, having paid 0.
The end state is the bitwise OR of all the given levels, and the minimum cost to get there is the answer.
In pseudocode:
let mask[1], mask[2], ..., mask[n] be the given bit masks of the n levels
p2m = 2 to the power of m
f[0] = 0
all f[1], f[2], ..., f[p2m-1] = infinity
for state = 0, 1, 2, ..., p2m-1:
current_cost = f[state]
current_ones = popcount(state) // popcount is the number of 1 bits
for level = 1, 2, ..., n:
new_state = state | mask[level] // the operation is bitwise OR
new_cost = current_cost + square (popcount(new_state) - current_ones)
f[new_state] = min (f[new_state], new_cost)
mask_total = mask[1] | mask[2] | ... | mask[n]
the answer is f[mask_total]
The complexity is O(2^m * n) time and O(2^m) memory, which should be fine for m <= 20 and n <= 20 in most online judges.

The dynamic optimization idea by #Gassa could be extended by using A* by estimating min and max of the remaining cost, where
minRemaining(s)=bitCount(maxState-s)
maxRemaining(s)=bitCount(maxState-s)^2
Start with a priority queue - and base it on cost+minRemaining - with the just the empty state, and then replace a state from this queue that has not reached maxState with at most n new states based the n levels:
Keep track bound=min(cost(s)+maxRemaining(s)) in queue,
and initialize all costs with bitCount(maxState)^2+1
extract state with lowest cost
if state!=maxState
remove state from queue
for j in 1..n
if (state|level[j]!=state)
cost(state|level[j])=min(cost(state|level[j]),
cost(state)+bitCount(state|level[j]-state)^2
if cost(state|level[j])+minRemaining(state|level[j])<=bound
add/replace state|level[j] in queue
else break
The idea is to skip dead-ends. So consider an example from a comment
11100 cost 9 min 2 max 4
11110 cost 16 min 1 max 1
11111 cost 25 min 0 max 0
00011 cost 4 min 3 max 9
bound 13
remove 00011 and replace with 11111 (skipping 00011 since no change)
11111 cost 13 min 0 max 0
11100 cost 9 min 2 max 4
11110 cost 16 min 1 max 1
remove 11100 and replace with 11110 11111 (skipping 11100 since no change):
11111 cost 13 min 0 max 0
11110 cost 10 min 1 max 1
bound 11
remove 11110 and replace with 11111 (skipping 11110 since no change)
11111 cost 11 min 0 max 0
bound 11
Number of operations should be similar to dynamic optimization in the worst case, but in many cases it will be better - and I don't know if the worst case can occur.

The logic behind this problem is that every time you have to find the minimum count of set bits corresponding to a binary string which will contain the weapons so far got in the level.
For ex :
we have data as
4 3
101-2 bits
010-1 bits
110-2 bits
101-2 bits
now as 010 has min bits we compute cost for it first then update the current pattern (by using bitwise OR) so current pattern is 010
next we find the next min set bits wrt to current pattern
i have used the logic by first using XOR for current pattern and the given number then using AND with the current number(A^B)&A
so the bits become like this after the operation
(101^010)&101->101-2 bit
(110^010)&110->100-1 bit
now we know the min bit is 110 we pick it and compute the cost ,update the pattern and so on..
This method returns the cost of a string with respect to current pattern
private static int computeCost(String currPattern, String costString) {
int a = currPattern.isEmpty()?0:Integer.parseInt(currPattern, 2);
int b = Integer.parseInt(costString, 2);
int cost = 0;
int c = (a ^ b) & b;
cost = (int) Math.pow(countSetBits(c), 2);
return cost;
}

Parse CSV file to give values in ranges Java

I have a CSV file which has data on it such as:
File % Diff
testfile0.xml 44.948
testfile1.xml 22.232
testfile2.xml 45.343
testfile3.xml 2.345
testfile4.xml -3.948
testfile5.xml 22.232
testfile6.xml 45.343
testfile7.xml 2.345
testfile8.xml -3.948
testfile9.xml 90.948
I'd like to summarize this data and categorize the %diff results into 10% increments
%Grouping No files
neg-00 2
00-10 2
10-20 0
20-30 2
30-40 0
40-50 3
50-60 0
60-70 0
70-80 0
80-90 1
90-100 1
So essentially, how can I group doubles together in ranges of 10% increments using Java. Any help of assistance would be much appreciated.

If your groups are -1 to 9 then:
groupNumber = (diff<=0) ? -1 : (diff.intValue() / 10);

Store the counts of the different increments in an array with 11 locations. Initialize the 11 counts to 0.
Read the double value on each line.
If the double value is negative, increment the count in the 0th index. Otherwise, truncate the double value to an integer, and then integer divide the result by 10 and add 1 to get the index of the count to increment.

Strange Numbers

Here are the properties of "strange numbers" in the problem I'm doing:
1) They have an even number of decimal digits (no leading zeros).
2) Define left half to be the number represented by the most significant half of digits of the original number, and right half to be the one represented by the least significant half. The right half may have leading zeros. The strange number is the square of the sum of its halves: 81 = (8 + 1)^2
Here are some other examples: 998001 = (998 + 001)^2, 3025 = (30 + 25)^2
How can I write a program that lists all the strange numbers in increasing order that have no more than 18 decimal digits?
I understand how to do this by looking at all the possibilities (numbers with 2 digits, 4 digits, 6 digits, ... , 18 digits), but that would take days to run. Are there any patterns to this, so I can output all the strange numbers in a matter of seconds? I would prefer answers in Java, but pseudo code is okay also.

All these 'strange' numbers are perfect squares. So you can start by going through all the numbers and squaring them (until the square has more than 18 digits). And for each square, check to see if it is 'strange'.
Edit
I'll also add that the reason this speeds things up so much is that it changes the solution from O(n) to O(√n)

Besides #spatulamania's speed-up, you can use modulo arithmetic to further speed up the checks.
To check every perfect square, you'll have to split the number into the two parts, add them, square the sum and compare it with the original number. (I'll name this as "full-check")
Instead, you can first check only the last digits of the two parts (and square their sum). For example, for number 99980001, take digits 8 and 1, take the square of (8+1)^2 = 9^2 = 81 and test that the last digit (1 in this case), is same as the last digit of 99980001 (I'll name this as "small-check"). If yes, then proceed with the full-check.
Since there are only 10x10=100 such combinations, this just needs to be done once. You'll create an array of acceptable combinations, that you can use:
0 0
0 1
8 1
4 4
8 4
0 5
0 6
8 6
4 9
8 9
Using this, you'll need to do only the "small-check" for about 82% of the perfect squares (those that fail the small-check) and both checks for the rest 18% (that pass the small-check, so "full-check" will be needed too). Therefore, if the "small-check" can be done fast enough, you'll gain some speed.
You may find even faster to expand this table for the last 2 digits of the two parts and use it (when n is large enough).

class strange_number
{
int number(int n)
{
int x = n;
String a = Integer.toString(n);
int d = a.length();
if(((int)(Math.pow(((x%(int)(Math.pow(10,d/2)))+(x/(int)(Math.pow(10,d/2)))),2))) == x)
return 1;
else
return 0;
}
}
can try this way. This may help u.

Extracting rightmost N bits of an integer

In the yester Code Jam Qualification round http://code.google.com/codejam/contest/dashboard?c=433101#s=a&a=0 , there was a problem called Snapper Chain. From the contest analysis I came to know the problem requires bit twiddling stuff like extracting the rightmost N bits of an integer and checking if they all are 1. I saw a contestant's(Eireksten) code which performed the said operation like below:
(((K&(1<<N)-1))==(1<<N)-1)
I couldn't understand how this works. What is the use of -1 there in the comparison?. If somebody can explain this, it would be very much useful for us rookies. Also, Any tips on identifying this sort of problems would be much appreciated. I used a naive algorithm to solve this problem and ended up solving only the smaller data set.(It took heck of a time to compile the larger data set which is required to be submitted within 8 minutes.). Thanks in advance.

Let's use N=3 as an example. In binary, 1<<3 == 0b1000. So 1<<3 - 1 == 0b111.
In general, 1<<N - 1 creates a number with N ones in binary form.
Let R = 1<<N-1. Then the expression becomes (K&R) == R. The K&R will extract the last N bits, for example:
101001010
& 111
————————————
000000010
(Recall that the bitwise-AND will return 1 in a digit, if and only if both operands have a 1 in that digit.)
The equality holds if and only if the last N bits are all 1. Thus the expression checks if K ends with N ones.

For example: N=3, K=101010
1. (1<<N) = 001000 (shift function)
2. (1<<N)-1 = 000111 (http://en.wikipedia.org/wiki/Two's_complement)
3. K&(1<<N)-1 = 0000010 (Bitmask)
4. K&(1<<N)-1 == (1<<N)-1) = (0000010 == 000111) = FALSE

I was working through the Snapper Chain problem and came here looking for an explanation on how the bit twiddling algorithm I came across in the solutions worked. I found some good info but it still took me a good while to figure it out for myself, being a bitwise noob.
Here's my attempt at explaining the algorithm and how to come up with it. If we enumerate all the possible power and ON/OFF states for each snapper in a chain, we see a pattern. Given the test case N=3, K=7 (3 snappers, 7 snaps), we show the power and ON/OFF states for each snapper for every kth snap:
1 2 3
0b:1 1 1.1 1.0 0.0 -> ON for n=1
0b:10 2 1.0 0.1 0.0
0b:11 3 1.1 1.1 1.0 -> ON for n=1, n=2
0b:100 4 1.0 0.0 1.0
0b:101 5 1.1 1.0 1.0 -> ON for n=1
0b:110 6 1.0 0.1 0.1
0b:111 7 1.1 1.1 1.1 -> ON for n=2, n=3
The lightbulb is on when all snappers are on and receiving power, or when we have a kth snap resulting in n 1s. Even more simply, the bulb is on when all of the snappers are ON, since they all must be receiving power to be ON (and hence the bulb). This means for every k snaps, we need n 1s.
Further, you can note that k is all binary 1s not only for k=7 that satisfies n=3, but for k=3 that satisifes n=2 and k=1 that satisifes n=1. Further, for n = 1 or 2 we see that every number of snaps that turns the bulb on, the last n digits of k are always 1. We can attempt to generalize that all ks that satisfy n snappers will be a binary number ending in n digits of 1.
We can use the expression noted by an earlier poster than 1 << n - 1 always gives us n binary digits of 1, or in this case, 1 << 3 - 1 = 0b111. If we treat our chain of n snappers as a binary number where each digit represents on/off, and we want n digits of one, this gives us our representation.
Now we want to find those cases where 1 << n - 1 is equal to some k that ends in n binary digits of 1, which we do by performing a bitwise-and: k & (1 << n - 1) to get the last n digits of k, and then comparing that to 1 << n - 1.
I suppose this type of thinking comes more naturally after working with these types of problems a lot, but it's still intimidating to me and I doubt I would ever have come up with such a solution by myself!
Here's my solution in perl:
$tests = <>;
for (1..$tests) {
($n, $k) = split / /, <>;
$m = 1 << $n - 1;
printf "Case #%d: %s\n", $_, (($k & $m) == $m) ? 'ON' : 'OFF';
}

I think we can recognize this kind of problem by calculating the answer by hand first, for some series of N (for example 1,2,3,..). After that, we will recognize the state change and then write a function to automate the process (first function). Run the program for some inputs, and notice the pattern.
When we get the pattern, write the function representing the pattern (second function), and compare the output of the first function and the second function.
For the Code Jam case, we can run both function against the small dataset, and verify the output. If it is identical, we have a high probability that the second function can solve the large dataset in time.