We Keep Coding

Should multiplication/bitshift optimization be visible in java bytecode

I keep reading that bitshifting is not needed, as compiler optimisations will translate a multiplication to a bitshift. Such as Should I bit-shift to divide by 2 in Java? and Is shifting bits faster than multiplying and dividing in Java? .NET?
I am not inquiring here about the performance difference, I could test that out myself. But what I think is curious, is that several people mention that it will be "compiled to the same thing". Which seems to be not true. I have written a small piece of code.
private static void multi()
{
int a = 3;
int b = a * 2;
System.out.println(b);
}
private static void shift()
{
int a = 3;
int b = a << 1L;
System.out.println(b);
}
Which gives the same result, and just prints it out.
When I look at the generated Java Bytecode, the following is shown.
private static void multi();
Code:
0: iconst_3
1: istore_0
2: iload_0
3: iconst_2
4: imul
5: istore_1
6: getstatic #4 // Field java/lang/System.out:Ljava/io/PrintStream;
9: iload_1
10: invokevirtual #5 // Method java/io/PrintStream.println:(I)V
13: return
private static void shift();
Code:
0: iconst_3
1: istore_0
2: iload_0
3: iconst_1
4: ishl
5: istore_1
6: getstatic #4 // Field java/lang/System.out:Ljava/io/PrintStream;
9: iload_1
10: invokevirtual #5 // Method java/io/PrintStream.println:(I)V
13: return
Now we can see the difference between "imul" and "ishl".
My question being: clearly the spoken off optimization is not visible in the java bytecode. I am still assuming the optimization does happen, so does it just happen at a lower level? Or, alternatively because it is Java, does the JVM when encountering the imul statement somehow know that it should be translated to something else. If so, any resources on how this is handled would be greatly appreciated.
(as a sidenote, I am not trying to justify the need for bitshifting. I think it decreases readability, at least to people used to Java, for C++ that might be different. I am just trying to see where the optimization happens).

The question in the title sounds a bit different than the question in the text. The quoted statement that the shift and the multiplication would be "compiled to the same thing" is true. But it does not yet apply to the bytecode.
In general, the Java bytecode is rather un-optimized. There are very few optimizations done at all - mainly inlining of constants. Beyond that, the Java bytecode is just an intermediate representation of the original program. And the translation from Java to Java bytecode is done rather "literally".
(Which, I think, is a good thing. The bytecode still closely resembles the original Java code. All the nitty-gritty (platform specific!) optimizations that may be possible are left to the virtual machine, which has far more options here.
All further optimizations, like arithmetic optimizations, dead code elimination or method inlining, are done by the JIT (Just-In-Time-Compiler), at runtime. The Just-In-Time compiler also applies the optimization of replacing the multiplication by a bit shift.
The example that you gave makes it a bit difficult to show the effect, for several reasons. The fact that the System.out.println was included in the method tends to make the actual machine code large, due to inlining and the general prerequisites for calling this method. But more importantly, the shift by 1, which corresponds to a multiplication with 2, also corresponds to an addition of the value to itself. So instead of observing a shl (left-shift) assembler instruction in the resulting machine code for the multi method, you'd likely see a disguised add instruction in the multi- and the shift method.
However, here is a very pragmatic example that does a left-shift by 8, corresponding to a multiplication with 256:
class BitShiftOptimization
{
public static void main(String args[])
{
int blackHole = 0;
for (int i=0; i<1000000; i++)
{
blackHole += testMulti(i);
blackHole += testShift(i);
}
System.out.println(blackHole);
}
public static int testMulti(int a)
{
int b = a * 256;
return b;
}
public static int testShift(int a)
{
int b = a << 8L;
return b;
}
}
(It receives the value to be shifted as an argument, to prevent it from being optimized away into a constant. It calls the methods several times, to trigger the JIT. And it returns and collects the values from both methods to prevent the method calls to be optimized away. Again, this is very pragmatic, but sufficient to show the effect)
Running this in a Hotspot Disassembler VM with
java -server -XX:+UnlockDiagnosticVMOptions -XX:+TraceClassLoading -XX:+LogCompilation -XX:+PrintInlining -XX:+PrintAssembly BitShiftOptimization
will produce the following assembler code for the testMulti method:
Decoding compiled method 0x000000000286fbd0:
Code:
[Entry Point]
[Verified Entry Point]
[Constants]
# {method} {0x000000001c0003b0} &apos;testMulti&apos; &apos;(I)I&apos; in &apos;BitShiftOptimization&apos;
# parm0: rdx = int
# [sp+0x40] (sp of caller)
0x000000000286fd20: mov %eax,-0x6000(%rsp)
0x000000000286fd27: push %rbp
0x000000000286fd28: sub $0x30,%rsp
0x000000000286fd2c: movabs $0x1c0005a8,%rax ; {metadata(method data for {method} {0x000000001c0003b0} &apos;testMulti&apos; &apos;(I)I&apos; in &apos;BitShiftOptimization&apos;)}
0x000000000286fd36: mov 0xdc(%rax),%esi
0x000000000286fd3c: add $0x8,%esi
0x000000000286fd3f: mov %esi,0xdc(%rax)
0x000000000286fd45: movabs $0x1c0003a8,%rax ; {metadata({method} {0x000000001c0003b0} &apos;testMulti&apos; &apos;(I)I&apos; in &apos;BitShiftOptimization&apos;)}
0x000000000286fd4f: and $0x1ff8,%esi
0x000000000286fd55: cmp $0x0,%esi
0x000000000286fd58: je 0x000000000286fd70 ;*iload_0
; - BitShiftOptimization::testMulti#0 (line 17)
0x000000000286fd5e: shl $0x8,%edx
0x000000000286fd61: mov %rdx,%rax
0x000000000286fd64: add $0x30,%rsp
0x000000000286fd68: pop %rbp
0x000000000286fd69: test %eax,-0x273fc6f(%rip) # 0x0000000000130100
; {poll_return}
0x000000000286fd6f: retq
0x000000000286fd70: mov %rax,0x8(%rsp)
0x000000000286fd75: movq $0xffffffffffffffff,(%rsp)
0x000000000286fd7d: callq 0x000000000285f160 ; OopMap{off=98}
;*synchronization entry
; - BitShiftOptimization::testMulti#-1 (line 17)
; {runtime_call}
0x000000000286fd82: jmp 0x000000000286fd5e
0x000000000286fd84: nop
0x000000000286fd85: nop
0x000000000286fd86: mov 0x2a8(%r15),%rax
0x000000000286fd8d: movabs $0x0,%r10
0x000000000286fd97: mov %r10,0x2a8(%r15)
0x000000000286fd9e: movabs $0x0,%r10
0x000000000286fda8: mov %r10,0x2b0(%r15)
0x000000000286fdaf: add $0x30,%rsp
0x000000000286fdb3: pop %rbp
0x000000000286fdb4: jmpq 0x0000000002859420 ; {runtime_call}
0x000000000286fdb9: hlt
0x000000000286fdba: hlt
0x000000000286fdbb: hlt
0x000000000286fdbc: hlt
0x000000000286fdbd: hlt
0x000000000286fdbe: hlt
(the code for the testShift method has the same instructions, by the way).
The relevant line here is
0x000000000286fd5e: shl $0x8,%edx
which corresponds to the left-shift by 8.

Is binary operation more efficient than modulo?

There are two ways to check if the number is divisible by 2:
x % 2 == 1
(x & 1) == 1
Which of the two is more efficient?

The bit operation is almost certainly faster.
Division/modulus is a generalized operation which must work for any divisor you provide, not just 2. It must also check for underflow, range errors and division by zero, and maintain a remainder, all of which takes time.
The bit operation just does a bit "and" operation, which in this case just so happens to correspond to division by two. It might actually use just a single processor operation to execute.

Either the & expression will be faster or they will be the same speed. Last time I tried, they were the same speed when I used a literal 2 (because the compiler could optimise it) but % was slower if the 2 was in a variable.
The expression x % 2 == 1 as a test for odd numbers does not work for negative x.
So there's at least one reason to prefer &.

There will hardly be a noticable difference in practice. Particularly, it's hard to imagine a case where such an instruction will be the actual bottleneck.
(Some nitpicking: The "binary" operation should rather be called bitwise operation, and the "modulo" operation actually is a remainder operation)
From a more theoretical point of view, one could assume that the binary operation is more efficient than the remainder operation, for reasons that already have been pointed out in other answers.
However, back to the practical point of view again: The JIT will almost certainly come for the rescue. Considering the following (very simple) test:
class BitwiseVersusMod
{
public static void main(String args[])
{
for (int i=0; i<10; i++)
{
for (int n=100000; n<=100000000; n*=10)
{
long s0 = runTestBitwise(n);
System.out.println("Bitwise sum "+s0);
long s1 = runTestMod(n);
System.out.println("Mod sum "+s1);
}
}
}
private static long runTestMod(int n)
{
long sum = 0;
for (int i=0; i<n; i++)
{
if (i % 2 == 1)
{
sum += i;
}
}
return sum;
}
private static long runTestBitwise(int n)
{
long sum = 0;
for (int i=0; i<n; i++)
{
if ((i & 1) == 1)
{
sum += i;
}
}
return sum;
}
}
Running it with a Hotspot Disassembler VM using
java -server -XX:+UnlockDiagnosticVMOptions -XX:+TraceClassLoading -XX:+LogCompilation -XX:+PrintAssembly BitwiseVersusMod
creates the JIT disassembly log.
Indeed, for the first invocations of the modulo version, it creates the following disassembly:
...
0x00000000027dcae6: cmp $0xffffffff,%ecx
0x00000000027dcae9: je 0x00000000027dcaf2
0x00000000027dcaef: cltd
0x00000000027dcaf0: idiv %ecx ;*irem
; - BitwiseVersusMod::runTestMod#11 (line 26)
; implicit exception: dispatches to 0x00000000027dcc18
0x00000000027dcaf2: cmp $0x1,%edx
0x00000000027dcaf5: movabs $0x54fa0888,%rax ; {metadata(method data for {method} {0x0000000054fa04b0} &apos;runTestMod&apos; &apos;(I)J&apos; in &apos;BitwiseVersusMod&apos;)}
0x00000000027dcaff: movabs $0xb0,%rdx
....
where the irem instruction is translated into an idiv, which is considered to be rather expensive.
In contrast to that, the binary version uses an and instruction for the decision, as expected:
....
0x00000000027dc58c: nopl 0x0(%rax)
0x00000000027dc590: mov %rsi,%rax
0x00000000027dc593: and $0x1,%eax
0x00000000027dc596: cmp $0x1,%eax
0x00000000027dc599: movabs $0x54fa0768,%rax ; {metadata(method data for {method} {0x0000000054fa0578} &apos;runTestBitwise&apos; &apos;(I)J&apos; in &apos;BitwiseVersusMod&apos;)}
0x00000000027dc5a3: movabs $0xb0,%rbx
....
However, for the final, optimized version, the generated code is more similar for both versions. In both cases, the compiler does a lot of loop unrolling, but the core of the methods can still be identified: For the bitwise version, it generates an unrolled loop containing the following instructions:
...
0x00000000027de2c7: mov %r10,%rax
0x00000000027de2ca: mov %r9d,%r11d
0x00000000027de2cd: add $0x4,%r11d ;*iinc
; - BitwiseVersusMod::runTestBitwise#21 (line 37)
0x00000000027de2d1: mov %r11d,%r8d
0x00000000027de2d4: and $0x1,%r8d
0x00000000027de2d8: cmp $0x1,%r8d
0x00000000027de2dc: jne 0x00000000027de2e7 ;*if_icmpne
; - BitwiseVersusMod::runTestBitwise#13 (line 39)
0x00000000027de2de: movslq %r11d,%r10
0x00000000027de2e1: add %rax,%r10 ;*ladd
; - BitwiseVersusMod::runTestBitwise#19 (line 41)
...
There is still the and instruction for testing the lowest bit. But for the modulo version, the core of the unrolled loop is
...
0x00000000027e3a0a: mov %r11,%r10
0x00000000027e3a0d: mov %ebx,%r8d
0x00000000027e3a10: add $0x2,%r8d ;*iinc
; - BitwiseVersusMod::runTestMod#21 (line 24)
0x00000000027e3a14: test %r8d,%r8d
0x00000000027e3a17: jl 0x00000000027e3a2e ;*irem
; - BitwiseVersusMod::runTestMod#11 (line 26)
0x00000000027e3a19: mov %r8d,%r11d
0x00000000027e3a1c: and $0x1,%r11d
0x00000000027e3a20: cmp $0x1,%r11d
0x00000000027e3a24: jne 0x00000000027e3a2e ;*if_icmpne
; - BitwiseVersusMod::runTestMod#13 (line 26)
...
I have to admit that I can not fully understand (at least, not in reasonable time) what exactly it is doing there. But in any case: The irem bytecode instruction is also implemented with an and assembly instruction, and there is no longer any idiv instruction in the resulting machine code.
So to repeat the first statement from this answer: There will hardly be a noticable difference in practice. Not only because the cost of a single instruction will hardly ever be the bottleneck, but also because you never know which instructions actually will be executed, and in this particular case, you have to assume that they will basically be equal.

Actually neither of those expressions test divisibility by two (other than in the negative). They actually both resolve to true if x is odd.
There are many other ways of testing even/oddness (e.g. ((x / 2) * 2) == x)) but none of them have the optimal properties of x & 1 solely because no compiler could possibly get it wrong and use a divide.
Most modern compilers would compile x % 2 to the same code as x & 1 but a particularly stupid one could implement x % 2 using a divide operation so it could be less efficient.
The argument as to which is better is a different story. A rookie/tired programmer may not recognize x & 1 as a test for odd numbers but x % 2 would be clearer so there is an argument that x % 2 would be the better option.
Me - I'd go for if ( Maths.isEven(x) ) making it absolutely clear what I mean. IMHO Efficiency comes way down the list, well past clarity and readability.
public class Maths {
// Method is final to encourage compiler to inline if it is bright enough.
public static final boolean isEven(long n) {
/* All even numbers have their lowest-order bit set to 0.
* This `should` therefore be the most efficient way to recognise
* even numbers.
* Also works for negative numbers.
*/
return (n & 1) == 0;
}
}

The binary operation is faster.
The mod operation has to calculate a division in order to get the remainder.

Strange JIT pessimization of a loop idiom

While analyzing the results of a recent question here, I encountered a quite peculiar phenomenon: apparently an extra layer of HotSpot's JIT-optimization actually slows down execution on my machine.
Here is the code I have used for the measurement:
#OutputTimeUnit(TimeUnit.NANOSECONDS)
#BenchmarkMode(Mode.AverageTime)
#OperationsPerInvocation(Measure.ARRAY_SIZE)
#Warmup(iterations = 2, time = 1)
#Measurement(iterations = 5, time = 1)
#State(Scope.Thread)
#Threads(1)
#Fork(2)
public class Measure
{
public static final int ARRAY_SIZE = 1024;
private final int[] array = new int[ARRAY_SIZE];
#Setup public void setup() {
final Random random = new Random();
for (int i = 0; i < ARRAY_SIZE; ++i) {
final int x = random.nextInt();
array[i] = x == 0? 1 : x;
}
}
#GenerateMicroBenchmark public int normalIndex() {
final int[] array = this.array;
int result = 0;
for (int i = 0; i < array.length; i++) {
final int j = i & array.length-1;
final int entry = array[i];
result ^= entry + j;
}
return result;
}
#GenerateMicroBenchmark public int maskedIndex() {
final int[] array = this.array;
int result = 0;
for (int i = 0; i < array.length; i++) {
final int j = i & array.length-1;
final int entry = array[j];
result ^= entry + i;
}
return result;
}
#GenerateMicroBenchmark public int normalWithExitPoint() {
final int[] array = this.array;
int result = 0;
for (int i = 0; i < array.length; i++) {
final int j = i & array.length-1;
final int entry = array[i];
result ^= entry + j;
if (entry == 0) break;
}
return result;
}
#GenerateMicroBenchmark public int maskedWithExitPoint() {
final int[] array = this.array;
int result = 0;
for (int i = 0; i < array.length; i++) {
final int j = i & array.length-1;
final int entry = array[j];
result ^= entry + i;
if (entry == 0) break;
}
return result;
}
}
The code is quite subtle so let me point out the important bits:
the "normal index" variants use the straight variable i for array index. HotSpot can easily determine the range of i throughout the loop and eliminate the array bounds check;
the "masked index" variants index by j, which is actually equal to i, but that fact is "hidden" from HotSpot through the AND-masking operation;
the "with exit point" variants introduce an explict loop exit point. The importance of this will be explained below.
Loop unrolling and reordering
Observe that the bounds check figures in two important ways:
it has runtime overhead associated with it (a comparison followed by a conditional branch);
it constitutes a loop exit point which can interrupt the loop on any step. This turns out to have important consequences for the applicable JIT optimizations.
By inspecting the machine code emitted for the above four methods, I have noticed the following:
in all cases the loop is unrolled;
in the case of normalIndex, which is distinguished as the only one having no premature loop exit points, the operations of all the unrolled steps are reordered such that first all the array fetches are performed, followed by XOR-ing all the values into the accumulator.
Expected and actual measurement results
Now we can classify the four methods according to the discussed features:
normalIndex has no bounds checks and no loop exit points;
normalWithExitPoint has no bounds checks and 1 exit point;
maskedIndex has 1 bounds check and 1 exit point;
maskedWithExitPoint has 1 bounds check and 2 exit points.
The obvious expectation is that the above list should present the methods in descending order of performance; however, these are my actual results:
Benchmark Mode Samples Mean Mean error Units
normalIndex avgt 20 0.946 0.010 ns/op
normalWithExitPoint avgt 20 0.807 0.010 ns/op
maskedIndex avgt 20 0.803 0.007 ns/op
maskedWithExitPoint avgt 20 1.007 0.009 ns/op
normalWithExitPoint and maskedIndex are identical modulo measurement error, even though only the latter has the bounds check;
the greatest anomaly is observed on normalIndex which should have been the fastest, but is significantly slower than normalWithExitPoint, identical to it in every respect save for having one more line of code, the one which introduces the exit point.
Since normalIndex is the only method which has the extra reordering "optimization" applied to it, the conclusion is that this is the cause of the slowdown.
I am testing on:
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode) (Java 7 Update 40)
OS X Version 10.9.1
2.66 GHz Intel Core i7
I have successfully reproduced the results on Java 8 EA b118 as well.
My question:
Is the above phenomenon reproducible on other, similar machines? From the question mentioned at the beginning I already have a hint that at least some machines do not reproduce it, so another result from the same CPU would be very interesting.
Update 1: more measurements inspired by maaartinus's findings
I have gathered the following table which correlates execution times with the -XX:LoopUnrollLimit command-line argument. Here I focus only on two variants, with and without the if (entry == 0) break; line:
LoopUnrollLimit: 14 15 18 19 22 23 60
withExitPoint: 96 95 95 79 80 80 69 1/100 ns
withoutExitPoint: 94 64 64 63 64 77 75 1/100 ns
The following sudden changes can be observed:
on the transition from 14 to 15, the withoutExitPoint variant receives a beneficial LCM1 transformation and is significantly sped up. Due to the loop-unrolling limit, all loaded values fit into registers;
on 18->19, the withExitPoint variant receives a speedup, lesser than the above;
on 22->23, the withoutExitPoint variant is slowed down. At this point I see the spill-over into stack locations, as described in maaartinus's answer, starting to happen.
The default loopUnrollLimit for my setup is 60, so I present its results in the last column.
1 LCM = Local Code Motion. It is the transformation which causes all array access to happen on the top, followed by the processing of the loaded values.
Update 2: this is actually a known, reported problem
https://bugs.openjdk.java.net/browse/JDK-7101232
Appendix: The unrolled and reordered loop of normalIndex in machine code
0x00000001044a37c0: mov ecx,eax
0x00000001044a37c2: and ecx,esi ;*iand
; - org.sample.Measure::normalIndex#20 (line 44)
0x00000001044a37c4: mov rbp,QWORD PTR [rsp+0x28] ;*iload_3
; - org.sample.Measure::normalIndex#15 (line 44)
0x00000001044a37c9: add ecx,DWORD PTR [rbp+rsi*4+0x10]
0x00000001044a37cd: xor ecx,r8d
0x00000001044a37d0: mov DWORD PTR [rsp],ecx
0x00000001044a37d3: mov r10d,esi
0x00000001044a37d6: add r10d,0xf
0x00000001044a37da: and r10d,eax
0x00000001044a37dd: mov r8d,esi
0x00000001044a37e0: add r8d,0x7
0x00000001044a37e4: and r8d,eax
0x00000001044a37e7: mov DWORD PTR [rsp+0x4],r8d
0x00000001044a37ec: mov r11d,esi
0x00000001044a37ef: add r11d,0x6
0x00000001044a37f3: and r11d,eax
0x00000001044a37f6: mov DWORD PTR [rsp+0x8],r11d
0x00000001044a37fb: mov r8d,esi
0x00000001044a37fe: add r8d,0x5
0x00000001044a3802: and r8d,eax
0x00000001044a3805: mov DWORD PTR [rsp+0xc],r8d
0x00000001044a380a: mov r11d,esi
0x00000001044a380d: inc r11d
0x00000001044a3810: and r11d,eax
0x00000001044a3813: mov DWORD PTR [rsp+0x10],r11d
0x00000001044a3818: mov r8d,esi
0x00000001044a381b: add r8d,0x2
0x00000001044a381f: and r8d,eax
0x00000001044a3822: mov DWORD PTR [rsp+0x14],r8d
0x00000001044a3827: mov r11d,esi
0x00000001044a382a: add r11d,0x3
0x00000001044a382e: and r11d,eax
0x00000001044a3831: mov r9d,esi
0x00000001044a3834: add r9d,0x4
0x00000001044a3838: and r9d,eax
0x00000001044a383b: mov r8d,esi
0x00000001044a383e: add r8d,0x8
0x00000001044a3842: and r8d,eax
0x00000001044a3845: mov DWORD PTR [rsp+0x18],r8d
0x00000001044a384a: mov r8d,esi
0x00000001044a384d: add r8d,0x9
0x00000001044a3851: and r8d,eax
0x00000001044a3854: mov ebx,esi
0x00000001044a3856: add ebx,0xa
0x00000001044a3859: and ebx,eax
0x00000001044a385b: mov ecx,esi
0x00000001044a385d: add ecx,0xb
0x00000001044a3860: and ecx,eax
0x00000001044a3862: mov edx,esi
0x00000001044a3864: add edx,0xc
0x00000001044a3867: and edx,eax
0x00000001044a3869: mov edi,esi
0x00000001044a386b: add edi,0xd
0x00000001044a386e: and edi,eax
0x00000001044a3870: mov r13d,esi
0x00000001044a3873: add r13d,0xe
0x00000001044a3877: and r13d,eax
0x00000001044a387a: movsxd r14,esi
0x00000001044a387d: add r10d,DWORD PTR [rbp+r14*4+0x4c]
0x00000001044a3882: mov DWORD PTR [rsp+0x24],r10d
0x00000001044a3887: mov QWORD PTR [rsp+0x28],rbp
0x00000001044a388c: mov ebp,DWORD PTR [rsp+0x4]
0x00000001044a3890: mov r10,QWORD PTR [rsp+0x28]
0x00000001044a3895: add ebp,DWORD PTR [r10+r14*4+0x2c]
0x00000001044a389a: mov DWORD PTR [rsp+0x4],ebp
0x00000001044a389e: mov r10d,DWORD PTR [rsp+0x8]
0x00000001044a38a3: mov rbp,QWORD PTR [rsp+0x28]
0x00000001044a38a8: add r10d,DWORD PTR [rbp+r14*4+0x28]
0x00000001044a38ad: mov DWORD PTR [rsp+0x8],r10d
0x00000001044a38b2: mov r10d,DWORD PTR [rsp+0xc]
0x00000001044a38b7: add r10d,DWORD PTR [rbp+r14*4+0x24]
0x00000001044a38bc: mov DWORD PTR [rsp+0xc],r10d
0x00000001044a38c1: mov r10d,DWORD PTR [rsp+0x10]
0x00000001044a38c6: add r10d,DWORD PTR [rbp+r14*4+0x14]
0x00000001044a38cb: mov DWORD PTR [rsp+0x10],r10d
0x00000001044a38d0: mov r10d,DWORD PTR [rsp+0x14]
0x00000001044a38d5: add r10d,DWORD PTR [rbp+r14*4+0x18]
0x00000001044a38da: mov DWORD PTR [rsp+0x14],r10d
0x00000001044a38df: add r13d,DWORD PTR [rbp+r14*4+0x48]
0x00000001044a38e4: add r11d,DWORD PTR [rbp+r14*4+0x1c]
0x00000001044a38e9: add r9d,DWORD PTR [rbp+r14*4+0x20]
0x00000001044a38ee: mov r10d,DWORD PTR [rsp+0x18]
0x00000001044a38f3: add r10d,DWORD PTR [rbp+r14*4+0x30]
0x00000001044a38f8: mov DWORD PTR [rsp+0x18],r10d
0x00000001044a38fd: add r8d,DWORD PTR [rbp+r14*4+0x34]
0x00000001044a3902: add ebx,DWORD PTR [rbp+r14*4+0x38]
0x00000001044a3907: add ecx,DWORD PTR [rbp+r14*4+0x3c]
0x00000001044a390c: add edx,DWORD PTR [rbp+r14*4+0x40]
0x00000001044a3911: add edi,DWORD PTR [rbp+r14*4+0x44]
0x00000001044a3916: mov r10d,DWORD PTR [rsp+0x10]
0x00000001044a391b: xor r10d,DWORD PTR [rsp]
0x00000001044a391f: mov ebp,DWORD PTR [rsp+0x14]
0x00000001044a3923: xor ebp,r10d
0x00000001044a3926: xor r11d,ebp
0x00000001044a3929: xor r9d,r11d
0x00000001044a392c: xor r9d,DWORD PTR [rsp+0xc]
0x00000001044a3931: xor r9d,DWORD PTR [rsp+0x8]
0x00000001044a3936: xor r9d,DWORD PTR [rsp+0x4]
0x00000001044a393b: mov r10d,DWORD PTR [rsp+0x18]
0x00000001044a3940: xor r10d,r9d
0x00000001044a3943: xor r8d,r10d
0x00000001044a3946: xor ebx,r8d
0x00000001044a3949: xor ecx,ebx
0x00000001044a394b: xor edx,ecx
0x00000001044a394d: xor edi,edx
0x00000001044a394f: xor r13d,edi
0x00000001044a3952: mov r8d,DWORD PTR [rsp+0x24]
0x00000001044a3957: xor r8d,r13d ;*ixor
; - org.sample.Measure::normalIndex#34 (line 46)
0x00000001044a395a: add esi,0x10 ;*iinc
; - org.sample.Measure::normalIndex#36 (line 43)
0x00000001044a395d: cmp esi,DWORD PTR [rsp+0x20]
0x00000001044a3961: jl 0x00000001044a37c0 ;*if_icmpge
; - org.sample.Measure::normalIndex#12 (line 43)

The reason why the JITC tries to group everything together is rather unclear to me. AFAIK there are (were?) architectures for which grouping of two loads leads to a better performance (some early Pentium, I think).
As JITC knows the hot spots, it can inline much more aggressively than an ahead-of-time compiler, so it does it 16 times in this case. I can't see any clear advantage thereof here, except for making the looping relatively cheaper. I also doubt that there's any architecture profiting from grouping 16 loads together.
The code computes 16 temporary values, one per iteration of
int j = i & array.length-1;
int entry = array[i];
int tmp = entry + j;
result ^= tmp;
Each computation is pretty trivial, one AND, one LOAD, and one ADD. The values are to be mapped to registers, but there aren't enough of them. So the values have to be stored and loaded later.
This happens for 7 out of the 16 registers and increases the costs significantly.
Update
I'm not very sure about verifying this by using -XX:LoopUnrollLimit:
LoopUnrollLimit Benchmark Mean Mean error Units
8 ..normalIndex 0.902 0.004 ns/op
8 ..normalWithExitPoint 0.913 0.005 ns/op
8 ..maskedIndex 0.918 0.006 ns/op
8 ..maskedWithExitPoint 0.996 0.008 ns/op
16 ..normalIndex 0.769 0.003 ns/op
16 ..normalWithExitPoint 0.930 0.004 ns/op
16 ..maskedIndex 0.937 0.004 ns/op
16 ..maskedWithExitPoint 1.012 0.003 ns/op
32 ..normalIndex 0.814 0.003 ns/op
32 ..normalWithExitPoint 0.816 0.005 ns/op
32 ..maskedIndex 0.838 0.003 ns/op
32 ..maskedWithExitPoint 0.978 0.002 ns/op
- ..normalIndex 0.830 0.002 ns/op
- ..normalWithExitPoint 0.683 0.002 ns/op
- ..maskedIndex 0.791 0.005 ns/op
- ..maskedWithExitPoint 0.908 0.003 ns/op
The limit of 16 makes normalIndex be the fastest variant, which indicates that I was right with the "overallocation penalty". Bur according to Marko, the generated assembly changes with the unroll limit also in other aspects, so things are more complicated.

Vector Distance Calculation in Java - Optimization

As part of image processing function I need to calculate a sum of squares between
two lines in images.
This part of the code takes 96% of the run time:
for(int dx=0;dx<size;dx++) {
int left = a[pa+dx];
int right = b[pb+dx];
int diff = (left & 0xFF) - (right & 0xFF);
sum += diff*diff;
}
where:
a, b are of type byte[]
sum is long
size is int and generally has big values (around 400)
Running Java 7 64 bit. I've tried to replace a[pa+dx] with stuff like a[pa++]
the performance is not better.
Exactly the same code written in C++ that does exactly the save runs
overall twice faster(!) and
as far as I see there should be no significant reasons why wouldn't this Java
code be as fast especially when boundary checks can be moved out of the loop by
the compiler.
How can this stuff be optimized to perform as well as C++ code - as finally it is
integer arithmetic it shouldn't be much slower in Java
Edit: C++ sample looks like:
unsigned char const *srcptr=&a[pa];
unsigned char const *tgtptr=&b[pb];
for(int dx=0;dx < size;dx++) {
int p1=*srcptr++;
int p2=*tgtptr++;
int diff = p1 - p2;
sum += diff * diff;
}
I'd like to find out how to make HotSpot optimizer to create a code that
as fast as the C++ code shown above, finally it is quite simple and easy
to optimize lines.

It's only small, but you don't need the & 0xFF to calculate the difference: The difference will be the same signed or unsigned.
100 - -1 = 101 // signed
228 - 127 = 101 // unsigned
Then it would be tighter loop body:
for (int dx = 0; dx < size; dx++) {
int diff = a[pa+dx] - b[pb+dx];
sum += diff*diff;
}
Edit:
There seems to be some confusion regarding signed vs unsigned byte arithmetic. If you doubt they are the same, execute this:
byte a = -128;
byte b = 127;
int diff = a - b;
System.out.println(diff); // -255
a = 127;
b = -128;
diff = a - b;
System.out.println(diff); // 255
The reason the range of diff values is greater than byte (-128..127), is that java automatically widens byte to int before the calculation because the destination variable is an int.

Move the & 0xFF's outside the loop.
Do this by calculating an int[]-version of both a and b and rewrite your loop using these.

After I tested the same algorithm using different C++ compilers and different Java versions I've got to conclusion that GCC is very good compiler it optimizes the code better than intel and clang!
These are run-times of the same algorithm implemented in C++ and Java (when the line above is 96% of the run time:
Intel 12.1 1:58
GCC 4.6 0:43
GCC 4.4 0:43
Clang 1:20
Java 7 1:20
Java 6 1:23
Which shows that Java runs as fast as clang and Intel compiler for some reason does very bad job however gcc gives the best result, so I can't really expect from Java to run faster than
most of C++ compilers do.
Note this is the assembly generated by gcc:
.L225:
movzbl (%rcx), %r8d
movzbl (%rsi), %r10d
addl $1, %edx
addq $1, %rcx
addq $1, %rsi
subl %r10d, %r8d
imull %r8d, %r8d
movslq %r8d, %r8
addq %r8, %rax
cmpl %edx, %ebp
ja .L225
And this one generated by clang:
.LBB0_26:
movzbl (%r11), %r13d
movzbl (%r14), %esi
subl %r13d, %esi
imull %esi, %esi
movslq %esi, %rsi
addq %rsi, %rcx
incq %r11
incq %r14
decq %r12
jne .LBB0_26
What is the difference? GCC rearranges the instructions such that they can
run in parallel in the pipe-line, for example:
movzbl (%rcx), %r8d
movzbl (%rsi), %r10d
addl $1, %edx
addq $1, %rcx
addq $1, %rsi
Bottom line, Java run times are fine.
Edit: after providing -xHost option to Intel compiler (optimize for current CPU), the run time improved to 56s (using mmx instructions) but still not as fast as gcc, but little better than Java

If "size" if size of array a or b you can avoid the for-condition:
try{
for (int dx = 0; ; dx++) {
...
...
}
}catch(ArrayIndexOutOfBoundException e){}
The two lines are straight or curved? May you post a grafical representation of the problem, or a numeric example of the arrays? maybe there is a better geometrical solution?

Difference between declaring variables before or in loop?

I have always wondered if, in general, declaring a throw-away variable before a loop, as opposed to repeatedly inside the loop, makes any (performance) difference?
A (quite pointless) example in Java:
a) declaration before loop:
double intermediateResult;
for(int i=0; i < 1000; i++){
intermediateResult = i;
System.out.println(intermediateResult);
}
b) declaration (repeatedly) inside loop:
for(int i=0; i < 1000; i++){
double intermediateResult = i;
System.out.println(intermediateResult);
}
Which one is better, a or b?
I suspect that repeated variable declaration (example b) creates more overhead in theory, but that compilers are smart enough so that it doesn't matter. Example b has the advantage of being more compact and limiting the scope of the variable to where it is used. Still, I tend to code according example a.
Edit: I am especially interested in the Java case.

Which is better, a or b?
From a performance perspective, you'd have to measure it. (And in my opinion, if you can measure a difference, the compiler isn't very good).
From a maintenance perspective, b is better. Declare and initialize variables in the same place, in the narrowest scope possible. Don't leave a gaping hole between the declaration and the initialization, and don't pollute namespaces you don't need to.

Well I ran your A and B examples 20 times each, looping 100 million times.(JVM - 1.5.0)
A: average execution time: .074 sec
B: average execution time : .067 sec
To my surprise B was slightly faster.
As fast as computers are now its hard to say if you could accurately measure this.
I would code it the A way as well but I would say it doesn't really matter.

It depends on the language and the exact use. For instance, in C# 1 it made no difference. In C# 2, if the local variable is captured by an anonymous method (or lambda expression in C# 3) it can make a very signficant difference.
Example:
using System;
using System.Collections.Generic;
class Test
{
static void Main()
{
List<Action> actions = new List<Action>();
int outer;
for (int i=0; i < 10; i++)
{
outer = i;
int inner = i;
actions.Add(() => Console.WriteLine("Inner={0}, Outer={1}", inner, outer));
}
foreach (Action action in actions)
{
action();
}
}
}
Output:
Inner=0, Outer=9
Inner=1, Outer=9
Inner=2, Outer=9
Inner=3, Outer=9
Inner=4, Outer=9
Inner=5, Outer=9
Inner=6, Outer=9
Inner=7, Outer=9
Inner=8, Outer=9
Inner=9, Outer=9
The difference is that all of the actions capture the same outer variable, but each has its own separate inner variable.

The following is what I wrote and compiled in .NET.
double r0;
for (int i = 0; i < 1000; i++) {
r0 = i*i;
Console.WriteLine(r0);
}
for (int j = 0; j < 1000; j++) {
double r1 = j*j;
Console.WriteLine(r1);
}
This is what I get from .NET Reflector when CIL is rendered back into code.
for (int i = 0; i < 0x3e8; i++)
{
double r0 = i * i;
Console.WriteLine(r0);
}
for (int j = 0; j < 0x3e8; j++)
{
double r1 = j * j;
Console.WriteLine(r1);
}
So both look exactly same after compilation. In managed languages code is converted into CL/byte code and at time of execution it's converted into machine language. So in machine language a double may not even be created on the stack. It may just be a register as code reflect that it is a temporary variable for WriteLine function. There are a whole set optimization rules just for loops. So the average guy shouldn't be worried about it, especially in managed languages. There are cases when you can optimize manage code, for example, if you have to concatenate a large number of strings using just string a; a+=anotherstring[i] vs using StringBuilder. There is very big difference in performance between both. There are a lot of such cases where the compiler cannot optimize your code, because it cannot figure out what is intended in a bigger scope. But it can pretty much optimize basic things for you.

This is a gotcha in VB.NET. The Visual Basic result won't reinitialize the variable in this example:
For i as Integer = 1 to 100
Dim j as Integer
Console.WriteLine(j)
j = i
Next
' Output: 0 1 2 3 4...
This will print 0 the first time (Visual Basic variables have default values when declared!) but i each time after that.
If you add a = 0, though, you get what you might expect:
For i as Integer = 1 to 100
Dim j as Integer = 0
Console.WriteLine(j)
j = i
Next
'Output: 0 0 0 0 0...

I made a simple test:
int b;
for (int i = 0; i < 10; i++) {
b = i;
}
vs
for (int i = 0; i < 10; i++) {
int b = i;
}
I compiled these codes with gcc - 5.2.0. And then I disassembled the main ()
of these two codes and that's the result:
1º:
0x00000000004004b6 <+0>: push rbp
0x00000000004004b7 <+1>: mov rbp,rsp
0x00000000004004ba <+4>: mov DWORD PTR [rbp-0x4],0x0
0x00000000004004c1 <+11>: jmp 0x4004cd <main+23>
0x00000000004004c3 <+13>: mov eax,DWORD PTR [rbp-0x4]
0x00000000004004c6 <+16>: mov DWORD PTR [rbp-0x8],eax
0x00000000004004c9 <+19>: add DWORD PTR [rbp-0x4],0x1
0x00000000004004cd <+23>: cmp DWORD PTR [rbp-0x4],0x9
0x00000000004004d1 <+27>: jle 0x4004c3 <main+13>
0x00000000004004d3 <+29>: mov eax,0x0
0x00000000004004d8 <+34>: pop rbp
0x00000000004004d9 <+35>: ret
vs
2º
0x00000000004004b6 <+0>: push rbp
0x00000000004004b7 <+1>: mov rbp,rsp
0x00000000004004ba <+4>: mov DWORD PTR [rbp-0x4],0x0
0x00000000004004c1 <+11>: jmp 0x4004cd <main+23>
0x00000000004004c3 <+13>: mov eax,DWORD PTR [rbp-0x4]
0x00000000004004c6 <+16>: mov DWORD PTR [rbp-0x8],eax
0x00000000004004c9 <+19>: add DWORD PTR [rbp-0x4],0x1
0x00000000004004cd <+23>: cmp DWORD PTR [rbp-0x4],0x9
0x00000000004004d1 <+27>: jle 0x4004c3 <main+13>
0x00000000004004d3 <+29>: mov eax,0x0
0x00000000004004d8 <+34>: pop rbp
0x00000000004004d9 <+35>: ret
Which are exaclty the same asm result. isn't a proof that the two codes produce the same thing?

It is language dependent - IIRC C# optimises this, so there isn't any difference, but JavaScript (for example) will do the whole memory allocation shebang each time.

I would always use A (rather than relying on the compiler) and might also rewrite to:
for(int i=0, double intermediateResult=0; i<1000; i++){
intermediateResult = i;
System.out.println(intermediateResult);
}
This still restricts intermediateResult to the loop's scope, but doesn't redeclare during each iteration.

In my opinion, b is the better structure. In a, the last value of intermediateResult sticks around after your loop is finished.
Edit:
This doesn't make a lot of difference with value types, but reference types can be somewhat weighty. Personally, I like variables to be dereferenced as soon as possible for cleanup, and b does that for you,

I suspect a few compilers could optimize both to be the same code, but certainly not all. So I'd say you're better off with the former. The only reason for the latter is if you want to ensure that the declared variable is used only within your loop.

As a general rule, I declare my variables in the inner-most possible scope. So, if you're not using intermediateResult outside of the loop, then I'd go with B.

A co-worker prefers the first form, telling it is an optimization, preferring to re-use a declaration.
I prefer the second one (and try to persuade my co-worker! ;-)), having read that:
It reduces scope of variables to where they are needed, which is a good thing.
Java optimizes enough to make no significant difference in performance. IIRC, perhaps the second form is even faster.
Anyway, it falls in the category of premature optimization that rely in quality of compiler and/or JVM.

There is a difference in C# if you are using the variable in a lambda, etc. But in general the compiler will basically do the same thing, assuming the variable is only used within the loop.
Given that they are basically the same: Note that version b makes it much more obvious to readers that the variable isn't, and can't, be used after the loop. Additionally, version b is much more easily refactored. It is more difficult to extract the loop body into its own method in version a. Moreover, version b assures you that there is no side effect to such a refactoring.
Hence, version a annoys me to no end, because there's no benefit to it and it makes it much more difficult to reason about the code...

Well, you could always make a scope for that:
{ //Or if(true) if the language doesn't support making scopes like this
double intermediateResult;
for (int i=0; i<1000; i++) {
intermediateResult = i;
System.out.println(intermediateResult);
}
}
This way you only declare the variable once, and it'll die when you leave the loop.

I think it depends on the compiler and is hard to give a general answer.

I've always thought that if you declare your variables inside of your loop then you're wasting memory. If you have something like this:
for(;;) {
Object o = new Object();
}
Then not only does the object need to be created for each iteration, but there needs to be a new reference allocated for each object. It seems that if the garbage collector is slow then you'll have a bunch of dangling references that need to be cleaned up.
However, if you have this:
Object o;
for(;;) {
o = new Object();
}
Then you're only creating a single reference and assigning a new object to it each time. Sure, it might take a bit longer for it to go out of scope, but then there's only one dangling reference to deal with.

My practice is following:
if type of variable is simple (int, double, ...) I prefer variant b (inside).
Reason: reducing scope of variable.
if type of variable is not simple (some kind of class or struct) I prefer variant a (outside).
Reason: reducing number of ctor-dtor calls.

I had this very same question for a long time. So I tested an even simpler piece of code.
Conclusion: For such cases there is NO performance difference.
Outside loop case
int intermediateResult;
for(int i=0; i < 1000; i++){
intermediateResult = i+2;
System.out.println(intermediateResult);
}
Inside loop case
for(int i=0; i < 1000; i++){
int intermediateResult = i+2;
System.out.println(intermediateResult);
}
I checked the compiled file on IntelliJ's decompiler and for both cases, I got the same Test.class
for(int i = 0; i < 1000; ++i) {
int intermediateResult = i + 2;
System.out.println(intermediateResult);
}
I also disassembled code for both the case using the method given in this answer. I'll show only the parts relevant to the answer
Outside loop case
Code:
stack=2, locals=3, args_size=1
0: iconst_0
1: istore_2
2: iload_2
3: sipush 1000
6: if_icmpge 26
9: iload_2
10: iconst_2
11: iadd
12: istore_1
13: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
16: iload_1
17: invokevirtual #3 // Method java/io/PrintStream.println:(I)V
20: iinc 2, 1
23: goto 2
26: return
LocalVariableTable:
Start Length Slot Name Signature
13 13 1 intermediateResult I
2 24 2 i I
0 27 0 args [Ljava/lang/String;
Inside loop case
Code:
stack=2, locals=3, args_size=1
0: iconst_0
1: istore_1
2: iload_1
3: sipush 1000
6: if_icmpge 26
9: iload_1
10: iconst_2
11: iadd
12: istore_2
13: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
16: iload_2
17: invokevirtual #3 // Method java/io/PrintStream.println:(I)V
20: iinc 1, 1
23: goto 2
26: return
LocalVariableTable:
Start Length Slot Name Signature
13 7 2 intermediateResult I
2 24 1 i I
0 27 0 args [Ljava/lang/String;
If you pay close attention, only the Slot assigned to i and intermediateResult in LocalVariableTable is swapped as a product of their order of appearance. The same difference in slot is reflected in other lines of code.
No extra operation is being performed
intermediateResult is still a local variable in both cases, so there is no difference access time.
BONUS
Compilers do a ton of optimization, take a look at what happens in this case.
Zero work case
for(int i=0; i < 1000; i++){
int intermediateResult = i;
System.out.println(intermediateResult);
}
Zero work decompiled
for(int i = 0; i < 1000; ++i) {
System.out.println(i);
}

From a performance perspective, outside is (much) better.
public static void outside() {
double intermediateResult;
for(int i=0; i < Integer.MAX_VALUE; i++){
intermediateResult = i;
}
}
public static void inside() {
for(int i=0; i < Integer.MAX_VALUE; i++){
double intermediateResult = i;
}
}
I executed both functions 1 billion times each.
outside() took 65 milliseconds. inside() took 1.5 seconds.

I tested for JS with Node 4.0.0 if anyone is interested. Declaring outside the loop resulted in a ~.5 ms performance improvement on average over 1000 trials with 100 million loop iterations per trial. So I'm gonna say go ahead and write it in the most readable / maintainable way which is B, imo. I would put my code in a fiddle, but I used the performance-now Node module. Here's the code:
var now = require("../node_modules/performance-now")
// declare vars inside loop
function varInside(){
for(var i = 0; i < 100000000; i++){
var temp = i;
var temp2 = i + 1;
var temp3 = i + 2;
}
}
// declare vars outside loop
function varOutside(){
var temp;
var temp2;
var temp3;
for(var i = 0; i < 100000000; i++){
temp = i
temp2 = i + 1
temp3 = i + 2
}
}
// for computing average execution times
var insideAvg = 0;
var outsideAvg = 0;
// run varInside a million times and average execution times
for(var i = 0; i < 1000; i++){
var start = now()
varInside()
var end = now()
insideAvg = (insideAvg + (end-start)) / 2
}
// run varOutside a million times and average execution times
for(var i = 0; i < 1000; i++){
var start = now()
varOutside()
var end = now()
outsideAvg = (outsideAvg + (end-start)) / 2
}
console.log('declared inside loop', insideAvg)
console.log('declared outside loop', outsideAvg)

A) is a safe bet than B).........Imagine if you are initializing structure in loop rather than 'int' or 'float' then what?
like
typedef struct loop_example{
JXTZ hi; // where JXTZ could be another type...say closed source lib
// you include in Makefile
}loop_example_struct;
//then....
int j = 0; // declare here or face c99 error if in loop - depends on compiler setting
for ( ;j++; )
{
loop_example loop_object; // guess the result in memory heap?
}
You are certainly bound to face problems with memory leaks!. Hence I believe 'A' is safer bet while 'B' is vulnerable to memory accumulation esp working close source libraries.You can check usinng 'Valgrind' Tool on Linux specifically sub tool 'Helgrind'.

It's an interesting question. From my experience there is an ultimate question to consider when you debate this matter for a code:
Is there any reason why the variable would need to be global?
It makes sense to only declare the variable once, globally, as opposed to many times locally, because it is better for organizing the code and requires less lines of code. However, if it only needs to be declared locally within one method, I would initialize it in that method so it is clear that the variable is exclusively relevant to that method. Be careful not to call this variable outside the method in which it is initialized if you choose the latter option--your code won't know what you're talking about and will report an error.
Also, as a side note, don't duplicate local variable names between different methods even if their purposes are near-identical; it just gets confusing.

this is the better form
double intermediateResult;
int i = byte.MinValue;
for(; i < 1000; i++)
{
intermediateResult = i;
System.out.println(intermediateResult);
}
1) in this way declared once time both variable, and not each for cycle.
2) the assignment it's fatser thean all other option.
3) So the bestpractice rule is any declaration outside the iteration for.

Tried the same thing in Go, and compared the compiler output using go tool compile -S with go 1.9.4
Zero difference, as per the assembler output.

I use (A) when I want to see the contents of the variable after exiting the loop. It only matters for debugging. I use (B) when I want the code more compact, since it saves one line of code.

Even if I know my compiler is smart enough, I won't like to rely on it, and will use the a) variant.
The b) variant makes sense to me only if you desperately need to make the intermediateResult unavailable after the loop body. But I can't imagine such desperate situation, anyway....
EDIT: Jon Skeet made a very good point, showing that variable declaration inside a loop can make an actual semantic difference.