We Keep Coding

How to fill high-end bits in a Java byte with '1' without knowing the last 1 in advance? (FAST FIX Negative Integer decoder)

I am writing a FIX/FAST decoder for negative numbers as described below:
My question is:
How to fill the high-end bits of a Java byte with 1s as it is described above? I am probably unaware of some bit manipulation magic I need to in this conversion.
So I need to go from 01000110 00111010 01011101 to 11110001 10011101 01011101.
I know how to shift by 7 to drop the 8th bit. What I don't know is how to fill the high-end bits with 1s.

It seems like the question you're asking doesn't really match up with the problem you're trying to solve. You're not trying to fill in the high bits with 1; you're trying to decode a stop-bit-encoded integer from a buffer, which involves discarding the sign bits while combining the payload bits. And, of course, you want to stop after you find a byte with a 1 in the stop bit position. The method below should decode the value correctly:
private static final byte SIGN_BIT = (byte)0x40;
private static final byte STOP_BIT = (byte)0x80;
private static final byte PAYLOAD_MASK = 0x7F;
public static int decodeInt(final ByteBuffer buffer) {
int value = 0;
int currentByte = buffer.get();
if ((currentByte & SIGN_BIT) > 0)
value = -1;
value = (value << 7) | (currentByte & PAYLOAD_MASK);
if ((currentByte & STOP_BIT) != 0)
return value;
currentByte = buffer.get();
value = (value << 7) | (currentByte & PAYLOAD_MASK);
if ((currentByte & STOP_BIT) != 0)
return value;
currentByte = buffer.get();
value = (value << 7) | (currentByte & PAYLOAD_MASK);
if ((currentByte & STOP_BIT) != 0)
return value;
currentByte = buffer.get();
value = (value << 7) | (currentByte & PAYLOAD_MASK);
if ((currentByte & STOP_BIT) != 0)
return value;
currentByte = buffer.get();
value = (value << 7) | (currentByte & PAYLOAD_MASK);
return value;
}
A loop would be cleaner, but I unrolled it manually since messaging protocols tend to be hot code paths, and there's a fixed maximum byte length (5 bytes). For simplicity's sake, I read the bytes from a ByteBuffer, so you may need to adjust the logic based on how you're reading the encoded data.

Fillig the high bits might go as:
int fillHighBits(int b) { // 0001abcd
int n = Integer.highestOneBit(b); // 00010000
n = ~n; // 11101111
++n; // 11110000
return (n | b) 0xFF; // 1111abcd
}
As expression
(~Integer.highestOneBit(b) + 1) | b
Though the examples you gave lets me doubt this is what you want.

This can be done very simply using a simple accumulator where you shift in 7 bits at a time. You need to keep track of how many bits you have in the accumulator.
Sign extension can be performed by simple logical shift left follwed by arithmetic shift right (by the same distance) to copy the topmost bit to all unused positions.
byte[] input = new byte[] { 0x46, 0x3A, (byte) 0xDD };
int accumulator = 0;
int bitCount = 0;
for (byte b : input) {
accumulator = (accumulator << 7) | (b & 0x7F);
bitCount += 7;
}
// now sign extend the bits in accumulator
accumulator <<= (32 - bitCount);
accumulator >>= (32 - bitCount);
System.out.println(Integer.toHexString(accumulator));
The whole trick is that >>N operator replicates the top bit N times.

do logical OR (|) with a number which has highend bits set to 1 and rest are 0
for example:
1010101010101010
OR 1111111100000000
--------------------
11111111101010101

something like this:
int x = ...;
x = x | 0xF000;

how to get absolute value of a number in java using bit manipulation

I want to implement a function to get the absolute value of a number in java: do nothing if it is positive, if it is negative, convert to positive.
I want to do this only using bit manipulations and no number comparators.
Please help

Well a negation:
-n
Is the same as the two's complement:
~n + 1
The problem is here you only want to negate if the value is < 0. You can find that out by using a logical shift to see if the MSB is set:
n >>> 31
A complement would be the same as an XOR with all 1's, something like (for a 4-bit integer):
~1010 == 1010 ^ 1111
And we can get a mask with the arithmetic right shift:
n >> 31
Absolute value says:
if n is < 0, negate it (take the complement and add 1 to it)
else, do nothing to it
So putting it together we can do the following:
static int abs(int n) {
return (n ^ (n >> 31)) + (n >>> 31);
}
Which computes:
if n is < 0, XOR it with all 1's and add 1 to it
else, XOR it with all 0's and add 0 to it
I'm not sure there's an easy way to do it without the addition. Addition involves any number of carries, even for a simple increment.
For example 2 + 1 has no carry:
10 + 1 == 11
But 47 + 1 has 4 carries:
101111 + 1 == 110000
Doing the add and carry with bitwise/bit shifts would basically just be a loop unroll and pointless.
(Edit!)
Just to be silly, here is an increment and carry:
static int abs(int n) {
int s = n >>> 31;
n ^= n >> 31;
int c;
do {
c = (n & s) << 1;
n ^= s;
} while((s = c) != 0);
return n;
}
The way it works is it flips the first bit, then keeps flipping until it finds a 0. So then the job is just to unroll the loop. The loop body can be represented by a somewhat ridiculous compound one-liner.
static int abs(int n) {
int s = n >>> 31;
n ^= n >> 31;
int c = (n & s) << 1;
c = ((n ^= s) & (s = c)) << 1; // repeat this line 30 more times
n ^= s;
return n;
}
So there's an abs using only bitwise and bit shifts.
These aren't faster than Math.abs. Math.abs just returns n < 0 ? -n : n which is trivial. And actually the loop unroll totally sucks in comparison. Just a curiosity I guess. Here's my benchmark:
Math.abs: 4.627323150634766ns
shift/xor/add abs: 6.729459762573242ns
loop abs: 12.028789520263672ns
unrolled abs: 32.47122764587402ns
bit hacks abs: 6.380939483642578ns
(The bit hacks abs is the non-patented one shown here which is basically the same idea as mine except a little harder to understand.)

you can turn a two's-compliment number positive or negative by taking it's logical negation
i = ~i; // i equals not i
You can use the Math.max() function to always get the positive
public static int abs(int i) {
return Math.max(i,~i);
}

This depends on what type of number you are using. For an int, use
int sign = i >> 31;
This gets the sign bit, which is 0 for positive numbers, and 1 for negative numbers. For other primitive types, replace 31 with the number of bits used for the primitive minus 1.
You can then use that sign in your if statement.
if (sign == 1)
i = ~i + 1;

I think you'll find that this little ditty is what you're looking for:
int abs(int v) {
int mask = v >> Integer.SIZE - 1;
return v + mask ^ mask;
}
It's based on Bit Twiddling Hacks absolute value equation and uses no comparison operations. If you aren't allowed to use addition, then (v ^ mask) - mask is an alternative. The value of this function is fairly questionable; since it's nearly as clear as the implementation of Math.abs and it's only marginally faster (at least on a i7):
v + mask ^ mask: 2.0844380704220384 abs/ns
(v ^ mask) - mask: 2.0819764093030244 abs/ns
Math.abs: 2.2636355843860656 abs/ns
Here's a test that proves that it works over the entire range of integers (the test runs in less than 2 minutes on an i7 processor under Java 7 update 51):
package test;
import org.hamcrest.core.Is;
import org.junit.Assert;
import org.junit.Test;
public class AbsTest {
#Test
public void test() {
long processedCount = 0L;
long numberOfIntegers = 1L << Integer.SIZE; //4294967296L
int value;
for (value = Integer.MIN_VALUE; processedCount < numberOfIntegers; value++) {
Assert.assertEquals((long) abs(value), (long) StrictMath.abs(value));
if (processedCount % 1_000_000L == 0L) {
System.out.print(".");
}
processedCount++;
}
System.out.println();
Assert.assertThat(processedCount, Is.is(numberOfIntegers));
Assert.assertThat(value - 1, Is.is(Integer.MAX_VALUE));
}
private static int abs(int v) {
int mask = v >> Integer.SIZE - 1;
return v + mask ^ mask;
}
}

This problem can be broken down into 2 simple steps:
1.
If >= 0 then just return the number.
2.
If smaller than 0 (ie. negative), then flip the first bit that indicates that the number is negative. This can easily be done with an XOR operation with -1 and the number; Then simply add +1 to deal with the offset (signed integers start at -1 not 0).
public static int absolute(int a) {
if (a >= 0) {
return a;
} else {
return (a ^ -1) + 1;
}
}

Getting/Setting a bit value from image

Question is based on this site.
Could someone explain the meaning of these lines:
private int getBitValue(int n, int location) {
int v = n & (int) Math.round(Math.pow(2, location));
return v==0?0:1;
}
and
private int setBitValue(int n, int location, int bit) {
int toggle = (int) Math.pow(2, location), bv = getBitValue(n, location);
if(bv == bit)
return n;
if(bv == 0 && bit == 1)
n |= toggle;
else if(bv == 1 && bit == 0)
n ^= toggle;
return n;
}

int v = n & (int) Math.round(Math.pow(2, location));
Math.pow(2, location) raises 2 to the given power. This is rounded and converted to an integer. In binary, this will be 00000001 if location==0, 00000010 if location==1, 00000100 if location==2, etc. (Much better would be 1 << location which shifts a "1" by a certain number of bits, filling in 0 bits at the right. Using Math.pow will probably try to compute the logarithm of 2 every time it's called.)
n & ... is a bitwise AND. Since the item on the right has just one bit set, the effect is to zero out every bit in n except for that one bit, and put the result in v. This means that v will be 0 if that one bit is 0 in n, and something other than 0 if that bit is `, which means
return v==0?0:1;
returns 0 if the bit is clear and 1 if it's set.
int toggle = (int) Math.pow(2, location), bv = getBitValue(n, location);
toggle is set to that Math.pow thing I already described. bv is set to the bit that's already in n, which is 0 or 1. If this equals the thing you're setting it to, then we don't need to do anything to n:
if(bv == bit)
return n;
Otherwise, either we need to set it to 1 (remember that toggle will have just one bit set). n |= toggle is the same as n = n | toggle. | is a bit-wise OR, so that one bit will be set in n and all other bits in n will remain the same"
if(bv == 0 && bit == 1)
n |= toggle;
Or we need to set the bit to 0. n ^= toggle is the same as n = n ^ toggle. n is an exclusive OR. If we get here, then the bit in n is 1, and the bit in toggle is 1, and we want to set the bit in n to 0, so exclusive OR will change that bit to 0 while leaving every other bit the same:
else if(bv == 1 && bit == 0)
n ^= toggle;

The getBitValue just gets the value of a specified bit (on a certain location)
The setBitValue sets the value of a bit on the matched specific location.
These getter/setter methods are usually used for image processing, i.e. if you have a musk and you want to change a specific bit value.
Nothing more or less.

Java: Check binary value with a mask then get integer

Here is my problem:
I have a binary value
101001
and a mask
011100
I would like to compare them and get the result as an integer. In this case that would give:
1 **010** 01
0 **111** 00
= 010 => 2
My first idea consists of dealing with a character array. But I would like to know if there is a better way to achieve this aim in Java?

I would like to compare them and get the result as an integer
Assuming you meant 'mask' rather than 'compare':
int result = 0B011100 & 0B011100;
No char arrays required.
This is rather trivial.

I just improved the algorithm to be able to have mask where one bits can be splitted such as:
00111011011
Here is my function to get a value from a mask and a masked value
private static long getMaskedValue(long maskedValue, long mask){
long definitiveMaskedValue = 0;
int count=0;
maskedValue = mask & maskedValue;
while (mask != 0){
while ((mask & 1) == 0){
mask = mask >>> 1;
maskedValue = maskedValue >>> 1;
}
while ((mask & 1) == 1){
definitiveMaskedValue = definitiveMaskedValue + ((maskedValue & 1) << count);
count++;
mask = mask >>> 1;
maskedValue = maskedValue >>> 1;
}
}
return definitiveMaskedValue;
}
Here is my function to store a value in a variable thanks to a bitmask, it returns the old variable with the value stored inside.
I had to use BigInteger because of the shift operator that can not shift more than 32bits left in Java.
private static long setMaskedValue (long maskedValue, long mask, long valueToAdd) {
int nbZero=0;
int nbLeastSignificantBit=0;
long tmpMask=mask;
maskedValue = maskedValue & ~mask;
while (tmpMask != 0){
while ((tmpMask & 1) == 0){
tmpMask = tmpMask >>> 1;
nbLeastSignificantBit++;
nbZero ++;
}
while ((tmpMask & 1) == 1){
tmpMask = tmpMask >>> 1;
BigInteger bigValueToAdd = BigInteger.valueOf(valueToAdd).shiftLeft(nbZero);
long tmpValueToAdd = bigValueToAdd.longValue();
BigInteger bigMaskOneBit = BigInteger.valueOf(1).shiftLeft(nbLeastSignificantBit);
long maskOneBit = bigMaskOneBit.longValue();
long bitValueToSet = getMaskedValue(tmpValueToAdd, maskOneBit);
maskedValue = maskedValue | bitValueToSet << nbLeastSignificantBit;
nbLeastSignificantBit++;
}
}
return maskedValue;
}

Of course.
You need first AND your bits.
Shift right to avoid those zeros at right of mask.
You need value as integer already.
Then do the AND: int masked = value & mask;
Then shift right until the first 1 in the mask.
while (mask % 2 == 0) {
mask = mask >>> 1;
masked = masked >>> 1;
}
You can use while (mask & 1 == 0) { if you prefer :)
& is bitwise AND.
| is bitwise OR.
^ is bitwise XOR (if my memory doesn't fail :).
>>> is shifting right (unsigned integer)

represent an integer using binary in java language

Here is the problem:
You're given 2 32-bit numbers, N & M, and two bit positions, i & j. write a method to set all bits between i and j in N equal to M (e.g. M becomes a substring of N at locating at i
and starting at j)
For example:
input:
int N = 10000000000, M = 10101, i = 2, j = 6;
output:
int N = 10001010100
My solution:
step 1: compose one mask to clear sets from i to j in N
mask= ( ( ( ((1<<(31-j))-1) << (j-i+1) ) + 1 ) << i ) - 1
for the example, we have
mask= 11...10000011
step 2:
(N & mask) | (M<<i)
Question:
what is the convenient data type to implement the algorithm? for example
we have int n = 0x100000 in C, so that we can apply bitwise operators on n.
in Java, we have BitSet class, it has clear, set method, but doesnt support
left/right shift operator; if we use int, it supports left/right shift, but
doesnt have binary representation (I am not talking about binary string representation)
what is the best way to implement this?
Code in java (after reading all comments):
int x = Integer.parseInt("10000000000",2);
int x = Integer.parseInt("10101",2);
int i = 2, j = 6;
public static int F(int x, int y, int i, int j){
int mask = (-1<<(j+1)) | (-1>>>(32-i));
return (mask & x ) | (y<<i);
}

the bit-wise operators |, &, ^ and ~ and the hex literal (0x1010) are all available in java
32 bit numbers are ints if that constraint remains int will be a valid data type
btw
mask = (-1<<j)|(-1>>>(32-i));
is a slightly clearer construction of the mask

Java's int has all the operations you need. I did not totally understand your question (too tired now), so I'll not give you a complete answer, just some hints. (I'll revise it later, if needed.)
Here are j ones in a row: (1 << j)-1.
Here are j ones in a row, followed by i zeros: ((1 << j) - 1) << i.
Here is a bitmask which masks out j positions in the middle of x: x & ~(((1 << j) - 1) << i).
Try these with Integer.toBinaryString() to see the results. (They might also give strange results for negative or too big values.)

I think you're misunderstanding how Java works. All values are represented as 'a series of bits' under the hood, ints and longs are included in that.
Based on your question, a rough solution is:
public static int applyBits(int N, int M, int i, int j) {
M = M << i; // Will truncate left-most bits if too big
// Assuming j > i
for(int loopVar = i; loopVar < j; loopVar++) {
int bitToApply = 1 << loopVar;
// Set the bit in N to 0
N = N & ~bitToApply;
// Apply the bit if M has it set.
N = (M & bitToApply) | N;
}
return N;
}
My assumptions are:
i is the right-most (least-significant) bit that is being set in N.
M's right-most bit maps to N's ith bit from the right.
That premature optimization is the root of all evil - this is O(j-i). If you used a complicated mask like you did in the question you can do it in O(1), but it won't be as readable, and readable code is 97% of the time more important than efficient code.