We Keep Coding

Java Optimization with Threads

I'm using a custom class Foo in Java as the key type in a HashMap. All the fields of Foo instances are immutable (they are declared final and private and are assigned values only in the constructor). Thus, the hashCode() of a given Foo object is also fixed, and for optimization purposes, I am calculating it in the constructor and simply returning that value in the hashCode() method.
Instances of Foo also have a value() method which returns a similar fixed value once the object has been instantiated. Currently I am also calculating it in the constructor and returning it in the method, but there is a difference between hashCode() and value(): hashCode() is called for the first time almost instantly after the object is created, but value() is called much later. I understand that having a separate Thread to calculate the hash-code would simply increase the run-time because of synchronization issues, but:
is this a good way to calculate value()? Would it improve run-time at all?
are simple Threads enough, or do I need to use pools etc.?
Note: this may seem like I'm optimizing the wrong parts of my program, but I've already worked on the 'correct' parts and brought the average run-time down from ~17 seconds to ~2 seconds. Edit: there will be upwards of 5000 Foo objects, and that's a conservative estimate.

It definitely sounds like deferred calculation is a good approach here - and yes, if you create a lot of these objects, a thread pool is the way to go.
As for value()'s return value until it's ready, I would stay away from returning invalid values, and instead either make it blocking (and add some isValueReady() helper) or make it instantly return a "future" - some object that offers those same isReady and a blocking get methods.
Also, never rely on "much later" - always make sure the value there is ready before using it.

I recommend creating a Future for value - create a static fixedTheadPool and submit the value calculations on it. This way there's no risk that value will be accessed before it's available - the worst case is that whatever is accessing value will block on a Future.get call (or use the version with a timeout if e.g. deadlock is a concern)
Because Future.get throws checked exceptions which can be a nuisance, you can wrap the get call in your class's getter method and wrap the checked exceptions in a RuntimeException
class MyClass {
private static final ExecutorService executor = Executors.newFixedThreadPool(/* some value that makes sense */);
private final Future<Value> future;
public MyClass() {
future = executor.submit(/* Callable */);
}
public boolean isValueDone() {
return future.isDone();
}
public Value value() {
try {
return future.get();
} catch(InterruptedException|ExecutionException e) {
throw new RuntimeException(e);
}
}
}

Assigning fields in an equals() method

Suppose you have a written a class and have used lazy initialization to assign one of its fields. Suppose that the computation for that field only involves the other fields and is guaranteed to produce the same result every time. When two equal instances of the class encounter one another, it makes sense for them to share the value of the lazily initialized field (if either knows it). You could do this in the equals() method. Here is a class showing what I mean.
final class MyClass {
private final int number;
private String string;
MyClass(int number) {
this.number = number;
}
String getString() {
if (string == null) {
string = OtherClass.expensiveCalculation(number);
}
return string;
}
#Override
public boolean equals(Object object) {
if (object == this) { return true; }
if (!(object instanceof MyClass)) { return false; }
MyClass that = (MyClass) object;
if (that.number != number) { return false; }
String thatString = that.string;
if (string == null && thatString != null) {
string = thatString;
} else if (thatString == null && string != null) {
that.string = string;
}
return true;
}
#Override
public int hashCode() { return number; }
}
To me, this information-sharing seems the logical thing to do if you are going to go to the effort of lazily initializing a field, yet I have never seen an example of anyone using the equals() method in this way.
Is it a common or standard technique? If so, what is it called? If it is not a common technique, can I ask (at the risk of having the question put on hold as primarily opinion-based) what people think about it? Is it a good idea to use the equals() method to do anything other than check for equality?

This looks dangerous to me: the use of a side affect of a public method of Object to set an object's state. This will break if you subclass this class, and then override the subclass's equals method, a common thing to do. Just don't do this.

"Suppose that the computation for that field only involves the other fields and is guaranteed to produce the same result every time."
Given this supposition, you can assert that the value of the lazily initialized field does not matter because if the values of the other fields are the same, the calculated value will also be the same.
Edit
I guess I sidestepped the original question, so I'll answer that too. In the scenario you've created, there is nothing inherently wrong with what you're proposing.
The argument I would make is simply from a pragmatic standpoint: what happens when someone else is changing the definition of getString() (or more likely - changing the definition of the long running calculation that results in that value) and it starts relying on something that's not part of the object's equality considerations?
The reason conventional wisdom says that equals() should be side effect free is that most developers expect it to be side effect free.

I would not do this, for three reasons:
General software-engineering principles, such as cohesion, loose coupling, and "don't repeat yourself", militate against it: your equals(...) method will be doing something not very "equals"-y, that overlaps with the logic of your getString() method. Someone updating the logic of getString() might well fail to notice if they also need to update the logic of equals(...). (You might think that the logic of equals(...) will continue to be correct no matter how getString() is changed — after all, you're just having equals(...) copy the reference from one object to an equivalent one, so presumably that should always stay the same? — but the problem is that complex systems evolve in ways that you can't always predict in advance. When a requirement changes, you don't want to have make random changes in parts of the code that aren't obviously related to the requirement.)
Thread-safety. Your string field currently isn't volatile, and your getString() method currently isn't synchronized, so there's no attempt at thread-safety here anyway; but if you were to make the rest of the class thread-safe, it would not be perfectly straightforward to change equals(...) to be thread-safe without risking deadlocks. (This overlaps a bit with point #1, but I'm listing it separately because #1 is solely about the difficulty of knowing that you have to change equals(...), whereas this issue is a bit tricky to address even given that knowledge.)
Unlikelihood of usefulness. There's not much reason to expect it to happen very often that two instances get equals(...)-compared when one has already been lazy-initialized and the other has not; so the extra code complexity, and downsides mentioned above, are not likely to be worth it. (Remember: code is not free. In order to pass cost–benefit analysis, the benefits of a piece of code must exceed the costs of testing, understanding, maintaining, and supporting it in the future.) If it's worthwhile to share these lazy-initialized values between equivalent instances, then that should be done in a clearer and more-organized fashion that does not rely on happenstance. (For example, you might make the class's constructor private, and have a static factory-method that checks a static WeakHashMap for an existing instance before creating and returning a new one.)

The approach you describe is sometimes a good one, especially in situations where it is likely that many large immutable objects, despite being independently constructed, will end up being identical. Because it is much faster to compare equal references than to compare large objects which happen to be equal, it may be advantageous to have code which compares two large-objects and finds them to be identical replace one of the references with a reference to the other. For this to be workable, one should attempt to establish some sort of ordering among the objects in question to ensure that repeated comparisons will eventually yield the same canonical value. This could be accomplished by having objects include a long sequence number and consistently replacing references to newer values with references to older-but-equal values, or by comparing the identityHashCode value of the equal references and discarding whichever one, if any, has the lower value (if two references which identify distinct but identical instances, happen to report the same identityHashCode, both should be kept).
A nasty but unfortunate wrinkle in this is that Java has very poor multi-threading support for effectively-immutable objects. For an effectively-immutable object to be thread-safe, any access to an array or non-final field must go through a final field. The cheapest way of accomplishing that is probably to have the object contain a final field into which it stores a reference to itself, and have all methods which access non-final fields do so through that final field, but that's a bit ugly. Still, changing references distinct-but-identical references with references to the same object could offer some significant performance advantages despite the silly redundant final field accesses (since the target of the final field would be guaranteed to be in-cache, dereferencing it would be much cheaper than a normal dereference).
BTW, it would in many cases be possible to include an "equivalence-relation" mechanism such that once some objects were compared and found to be equal, discovering that any of them is equal to another object would cause all of them to be quickly recognizable as such. I haven't figured out how to avoid the possibility of a deliberately-nasty-but-legitimate usage pattern causing a memory leak, however.

How to "safely publish" lazily-generated effectively-immutable array

Java's present memory model guarantees that if the only reference to an object "George" is stored into a final field of some other object "Joe", and neither George nor Joe have never been seen by any other thread, all operations upon George which were performed before the store will be seen by all threads as having been performed before the store. This works out very nicely in cases where it makes sense to store into a final field a reference to an object which will never be mutated after that.
Is there any efficient way of achieving such semantics in cases where an object of mutable type is supposed to be lazily created (sometime after the owning object's constructor has finished execution)? Consider the fairly simple class ArrayThing which encapsulates an immutable array, but it offers a method (three versions with the same nominal purpose) to return the sum of all elements prior to a specified one. For purposes of this example, assume that many instances will be constructed without ever using that method, but on instances where that method is used, it will be used a lot; consequently, it's not worthwhile to precompute the sums when every instance of ArrayThing is constructed, but it is worthwhile to cache them.
class ArrayThing {
final int[] mainArray;
ArrayThing(int[] initialContents) {
mainArray = (int[])initialContents.clone();
}
public int getElementAt(int index) {
return mainArray[index];
}
int[] makeNewSumsArray() {
int[] temp = new int[mainArray.length+1];
int sum=0;
for (int i=0; i<mainArray.length; i++) {
temp[i] = sum;
sum += mainArray[i];
}
temp[i] = sum;
return temp;
}
// Unsafe version (a thread could be seen as setting sumOfPrevElements1
// before it's seen as populating array).
int[] sumOfPrevElements1;
public int getSumOfElementsBefore_v1(int index) {
int[] localElements = sumOfPrevElements1;
if (localElements == null) {
localElements = makeNewSumsArray();
sumOfPrevElements1 = localElements;
}
return localElements[index];
}
static class Holder {
public final int[] it;
public Holder(int[] dat) { it = dat; }
}
// Safe version, but slower to read (adds another level of indirection
// but no thread can possibly see a write to sumOfPreviousElements2
// before the final field and the underlying array have been written.
Holder sumOfPrevElements2;
public int getSumOfElementsBefore_v2(int index) {
Holder localElements = sumOfPrevElements2;
if (localElements == null) {
localElements = new Holder(makeNewSumsArray());
sumOfPrevElements2 = localElements;
}
return localElements.it[index];
}
// Safe version, I think; but no penalty on reading speed.
// Before storing the reference to the new array, however, it
// creates a temporary object which is almost immediately
// discarded; that seems rather hokey.
int[] sumOfPrevElements3;
public int getSumOfElementsBefore_v3(int index) {
int[] localElements = sumOfPrevElements3;
if (localElements == null) {
localElements = (new Holder(makeNewSumsArray())).it;
sumOfPrevElements3 = localElements;
}
return localElements[index];
}
}
As with the String#hashCode() method, it is possible that two or more threads might see that a computation hasn't been performed, decide to perform it, and store the result. Since all threads would end up producing identical results, that wouldn't be an issue. With getSumOfElementsBefore_v1(), however, there is a different problem: Java could re-order program execution so the array reference gets written to sumOfPrevElements1 before all the elements of the array have been written. Another thread which called getSumOfElementsBefore() at that moment could see that the array wasn't null, and then proceed to read an array element which hadn't yet been written. Oops.
From what I understand, getSumOfElementsBefore_v2() fixes that problem, since storing a reference to the array in final field Holder#it would establish a "happens-after" relationship with regard to the array element writes. Unfortunately, that version of the code would need to create and maintain an extra heap object, and would require that every attempt to access the sum-of-elements array go through an extra level of indirection.
I think getSumOfElementsBefore_v3() would be cheaper but still safe. The JVM guarantees that all actions which were done to a new object before a reference is stored into a final field will be visible to all threads by the time any thread can see that reference. Thus, even if other threads don't use Holder#it directly, the fact that they are using a reference which was copied from that field would establish that they can't see the reference until after everything that was done before the store has actually happened.
Even though the latter method limits the overhead (versus the unsafe method) to the times when the new array is created (rather than adding overhead to every read), it still seems rather ugly to create a new object purely for the purpose of writing and reading back a final field. Making the array field volatile would achieve legitimate semantics, but would add memory-system overhead every time the field is read (a volatile qualifier would require that the code notice if the field has been written in another thread, but that's overkill for this application; what's necessary is merely that any thread which does see that the field has been written also see all writes which occurred to the array identify thereby before the reference was stored). Is there any way to achieve similar semantics without having to either create and abandon a superfluous temporary object, or add additional overhead every time the field is read??

Your third version does not work. The guarantees made for a properly constructed object stored in a final instance field apply to reads of that final field only. Since the other threads don’t read that final variable, there is no guaranty made.
Most notably, the fact that the initialization of the array has to be completed before the array reference is stored in the final Holder.it variable does not say anything about when the sumOfPrevElements3 variable will be written (as seen by other threads). In practice, a JVM might optimize away the entire Holder instance creation as it has no side-effects, thus the resulting code behaves like an ordinary unsafe publication of an int[] array.
For using the final field publication guaranty you have to publish the Holder instance containing the final field, there is no way around it.
But if that additional instance annoys you, you should really consider using a simple volatile variable. After all, you are making only assumptions about the cost of that volatile variable, in other words, thinking about premature optimization.
After all, detecting a change made by another thread doesn’t have to be expensive, e.g. on x86 it doesn’t even need an access to the main memory as it has cache coherence. It’s also possible that an optimizer detects that you never write to the variable again once it became non-null, then enabling almost all optimizations possible for ordinary fields once a non-null reference has been read.
So the conclusion is as always: measure, don’t guess. And start optimizing only once you found a real bottleneck.

I think your second and third examples do work (sort of, as you say the reference itself might not be noticed by another thread, which might re-assign the array. That's a lot of extra work!).
But those examples are based on a faulty premise: it is not true that a volatile field requires the reader to "notice" the change. In fact, volatile and final fields perform exactly the same operation. The read operation of a volatile or a final has no overhead on most CPU architectures. I believe on a write volatile has a tiny amount of extra overhead.
So I would just use volatile here, and not worry about your supposed "optimizations". The difference in speed, if any, is going to be extremely slight, and I'm talking like an extra 4 bytes written with a bus-lock, if that. And your "optimized" code is pretty god-awful to read.
As a minor pendant, it is not true that final fields require you to have the sole reference to an object to make it immutable and thread safe. The spec only requires you to prevent changes to the object. Having the sole reference to an object is one way to prevent changes, sure. But objects that are already immutable (like java.lang.String for example) can be shared without problems.
In summary: Premature Optimization is the Root of All Evil.. Loose the tricky nonsense and just write a simple array update with assignment to a volatile.
volatile int[] sumOfPrevElements;
public int getSumOfElementsBefore(int index) {
if( sumOfPrevElements != null ) return sumOfPrevElements[index];
sumOfPrevElements = makeNewSumsArray();
return sumOfPrevElements[index];
}

Why is boxing a primitive value-type in .NET uncached, unlike Java?

Consider:
int a = 42;
// Reference equality on two boxed ints with the same value
Console.WriteLine( (object)a == (object)a ); // False
// Same thing - listed only for clarity
Console.WriteLine(ReferenceEquals(a, a)); // False
Clearly, each boxing instruction allocates a separate instance of a boxed Int32, which is why reference-equality between them fails. This page appears to indicate that this is specified behaviour:
The box instruction converts the 'raw'
(unboxed) value type into an object
reference (type O). This is
accomplished by creating a new object
and copying the data from the value
type into the newly allocated object.
But why does this have to be the case?
Is there any compelling reason why the CLR does not choose to hold a "cache" of boxed Int32s, or even stronger, common values for all primitive value-types (which are all immutable)? I know Java has something like this.
In the days of no-generics, wouldn't it have helped out a lot with reducing the memory requirements as well as GC workload for a large ArrayListconsisting mainly of small integers? I'm also sure that there exist several modern .NET applications that do use generics, but for whatever reason (reflection, interface assignments etc.), run up large boxing-allocations that could be massively reduced with (what appears to be) a simple optimization.
So what's the reason? Some performance implication I haven't considered (I doubt if testing that the item is in the cache etc. will result in a net performance loss, but what do I know)? Implementation difficulties? Issues with unsafe code? Breaking backwards compatibility (I can't think of any good reason why a well-written program should rely on the existing behaviour)? Or something else?
EDIT: What I was really suggesting was a static cache of "commonly-occurring" primitives, much like what Java does. For an example implementation, see Jon Skeet's answer. I understand that doing this for arbitrary, possibly mutable, value-types or dynamically "memoizing" instances at run-time is a completely different matter.
EDIT: Changed title for clarity.

One reason which I find compelling is consistency. As you say, Java does cache boxed values in a certain range... which means it's all too easy to write code which works for a while:
// Passes in all my tests. Shame it fails if they're > 127...
if (value1 == value2) {
// Do something
}
I've been bitten by this - admittedly in a test rather than production code, fortunately, but it's still nasty to have something which changes behaviour significantly outside a given range.
Don't forget that any conditional behaviour also incurs a cost on all boxing operations - so in cases where it wouldn't use the cache, you'd actually find that it was slower (because it would first have to check whether or not to use the cache).
If you really want to write your own caching box operation, of course, you can do so:
public static class Int32Extensions
{
private static readonly object[] BoxedIntegers = CreateCache();
private static object[] CreateCache()
{
object[] ret = new object[256];
for (int i = -128; i < 128; i++)
{
ret[i + 128] = i;
}
}
public object Box(this int i)
{
return (i >= -128 && i < 128) ? BoxedIntegers[i + 128] : (object) i;
}
}
Then use it like this:
object y = 100.Box();
object z = 100.Box();
if (y == z)
{
// Cache is working
}

I can't claim to be able to read minds, but here's a couple factors:
1) caching the value types can make for unpredictability - comparing two boxed values that are equal could be true or false depending on cache hits and implementation. Ouch!
2) The lifetime of a boxed value type is most likely short - so how long do you hold the value in cache? Now you either have a lot of cached values that will no longer be used, or you need to make the GC implementation more complicated to track the lifetime of cached value types.
With these downsides, what is the potential win? Smaller memory footprint in an application that does a lot of long-lived boxing of equal value types. Since this win is something that is going to affect a small number of applications and can be worked around by changing code, I'm going to agree with the c# spec writer's decisions here.

Boxed value objects are not necessarily immutable. It is possible to change the value in a boxed value type, such as through an interface.
So if boxing a value type always returned the same instance based on the same original value, it would create references which may not be appropriate (for example, two different value type instances which happen to have the same value end up with the same reference even though they should not).
public interface IBoxed
{
int X { get; set; }
int Y { get; set; }
}
public struct BoxMe : IBoxed
{
public int X { get; set; }
public int Y { get; set; }
}
public static void Test()
{
BoxMe original = new BoxMe()
{
X = 1,
Y = 2
};
object boxed1 = (object) original;
object boxed2 = (object) original;
((IBoxed) boxed1).X = 3;
((IBoxed) boxed1).Y = 4;
Console.WriteLine("original.X = " + original.X);
Console.WriteLine("original.Y = " + original.Y);
Console.WriteLine("boxed1.X = " + ((IBoxed)boxed1).X);
Console.WriteLine("boxed1.Y = " + ((IBoxed)boxed1).Y);
Console.WriteLine("boxed2.X = " + ((IBoxed)boxed2).X);
Console.WriteLine("boxed2.Y = " + ((IBoxed)boxed2).Y);
}
Produces this output:
original.X = 1
original.Y = 2
boxed1.X = 3
boxed1.Y = 4
boxed2.X = 1
boxed2.Y = 2
If boxing didn't create a new instance, then boxed1 and boxed2 would have the same values, which would be inappropriate if they were created from different original value type instance.

There's an easy explanation for this: un/boxing is fast. It needed to be back in the .NET 1.x days. After the JIT compiler generates the machine code for it, there's but a handful of CPU instructions generated for it, all inline without method calls. Not counting corner cases like nullable types and large structs.
The effort of looking up a cached value would greatly diminish the speed of this code.

I wouldn't think a run-time-filled cache would be a good idea, but I would think it might be reasonable on 64-bit systems, to define ~8 billion of the 64 quintillion possible objects-reference values as being integer or float literals, and on any system pre-box all primitive literals. Testing whether the upper 31 bits of a reference type hold some value should probably be cheaper than a memory reference.

Adding to the answers already listed is the fact that in .net, at least with the normal garbage collector, object references are internally stored as direct pointers. This means that when a garbage collection is performed the system has to update every single reference to every object that gets moved, but it also means that "main-line" operation can be very fast. If object references were sometimes direct pointers and sometimes something else, this would require extra code every time an object is dereferenced. Since object dereferencing is one of the most common operations during the execution of a .net program, even a 5% slowdown here would be devastating unless it was matched by an awesome speedup. It's possible, for example, a "64-bit compact" model, in which each object reference was a 32-bit index into an object table, might offer better performance than the existing model in which each reference is a 64-bit direct pointer. Deferencing operations would require an extra table lookup, which would be bad, but object references would be smaller, thus allowing more of them to be stored in the cache at once. In some circumstances, that could be a major performance win (maybe often enough to be worthwhile--maybe not). It's unclear, though, that allowing an object reference to sometimes be a direct memory pointer and sometimes be something else would really offer much advantage.

Premature optimization in Java: when to use "x = foo.getX()" vs simply "foo.getX()"

When I find myself calling the same getter method multiple times, should this be considered a problem? Is it better to [always] assign to a local variable and call only once?
I'm sure the answer of course is "it depends".
I'm more concerned about the simpler case where the getter is simply a "pass-along-the-value-of-a-private-variable" type method. i.e. there's no expensive computation involved, no database connections being consumed, etc.
My question of "is it better" pertains to both code readability (style) and also performance. i.e. is it that much of a performance hit to have:
SomeMethod1(a, b, foo.getX(), c);
SomeMethod2(b, foo.getX(), c);
SomeMethod3(foo.getX());
vs:
X x = foo.getX();
SomeMethod1(a, b, x, c);
SomeMethod2(b, x, c);
SomeMethod3(x);
I realize this question is a bit nit-picky and gray. But I just realized, I have no consistent way of evaluating these trade-offs, at all. Am fishing for some criteria that are more than just completely whimsical.
Thanks.

The choice shouldn't really be about performance hit but about code readability.
When you create a variable you can give it the name it deserves in the current context. When you use a same value more than one time it has surely a real meaning, more than a method name (or worse a chain of methods).
And it's really better to read:
String username = user.getName();
SomeMethod1(a, b, username, c);
SomeMethod2(b, username, c);
SomeMethod3(username);
than
SomeMethod1(a, b, user.getName(), c);
SomeMethod2(b, user.getName(), c);
SomeMethod3(user.getName());

For plain getters - those that just returns a value - HotSpot inlines it in the calling code, so it will be as fast as it can be.
I, however, have a principle about keeping a statement on a single line, which very often results in expressions like "foo.getBar()" being too long to fit. Then it is more readable - to me - to extract it to a local variable ("Bar bar = foo.getBar()").

They could be 2 different things.
If GetX is non-deterministic then the 1st one will give different results than the 2nd
Personally, I'd use the 2nd one. It's more obvious and less unnecessarily verbose.

I use the second style if it makes my code more readable or if I have to use the assigned value again. I never consider performance (on trivial things) unless I have to.

That depends on what getX() actually does. Consider this class:
public class Foo {
private X x;
public X getX() { return x; }
}
In this case, when you make a call to foo.getX(), JVM will optimize it all the way down to foo.x (as in direct reference to foo's private field, basically a memory pointer). However, if the class looks like this:
public class Foo {
private X x;
public X getX() { return cleanUpValue(x); }
private X cleanUpValue(X x) {
/* some modifications/sanitization to x such as null safety checks */
}
}
the JVM can't actually inline it as efficiently anymore since by Foo's constructional contract, it has to sanitize x before handing it out.
To summarize, if getX() doesn't really do anything beyond returning a field, then there's no difference after initial optimization runs to the bytecode in whether you call the method just once or multiple times.

Most of the time I would use getX if it was only once, and create a var for it for all other cases. Often just to save typing.
With regards to performance, the compiler would probably be able to optimize away most of the overhead, but the possibility of side-effects could force the compiler into more work when doing multiple method-calls.

I generally store it locally if:
I'm will use it in a loop and I don't want or expect the value to change during the loop.
I'm about to use it in a long line of code or the function & parameters are very long.
I want to rename the variable to better correspond to the task at hand.
Testing indicates a significant performance boost.
Otherwise I like the ability to get current values and lower level of abstraction of method calls.

Two things have to be considered:
Does the call to getX() have any side effects? Following established coding patterns, a getter should not alter the object on which it is called, the in most cases, there is no side effect. Therefore, it is semantically equivalent to call the getter once and store the value locally vs. calling the getter multiple times. (This concept is called idempotency - it does not matter whether you call a method once or multiple times; the effect on the data is exactly the same.)
If the getter has no side effect, the compiler can safely remove subsequent calls to the getter and create the temporary local storage on its own - thus, the code remains ultra-readable and you have all the speed advantage from calling the getter only once. This is all the more important if the getter does not simply return a value but has to fetch/compute the value or runs some validations.
Assuming your getter does not change the object on which it operates it is probably more readable to have multiple calls to getX() - and thanks to the compiler you do not have to trade performance for readability and maintainability.