We Keep Coding

thread safe map operation

I came across the following piece of code ,and noted a few inconsistencies - for a multi-thread safe code.
Map<String,Map<String,Set<String>> clusters = new HashMap<.........>;
Map<String,Set<String>> servers = clusters.get(clusterkey);
if(servers==null){
synchronized(clusterkey){
servers = clusters.get(clusterkey);
if(servers==null){....initialize new hashmap and put...}
}
}
Set<String> users=servers.get(serverkey);
if(users==null){
synchronized(serverkey){
users=servers.get(serverkey);
if(users==null){ ... initialize new hashset and put...}
}
}
users.add(userid);
Why would map be synchronized on clusterkey- shouldnt it be on the map as monitor itself?
Shouldnt the last users.add... be synchronized too?
This seems to be a lot of code to add a single user in a thread-safe manner.What would be a smarter implementation?

Here just some observations:
Synchronizing on a String is a very bad idea -> sync on clusterKey and serverKey will probably not work the way intended.
Better would be to use ConcurrentHashMaps and ConcurrentHashSets.
Though without more context it is not really possible to answer this question. It seems the code-author wanted to safely create just 1 mapping per clusterKey and serverKey so the user can be added just once.
A (probably better) way would be to just synchronize on the clusters map itself and then you're safe as only one thread can read and/or write to said map.
Another way would be to use custom Locks, maybe one for reading, and another one for writing, though this may lead again to inconsistencies if one thread is writing to the Map while another is reading that exact value from it.

The code looks like a non-thought through version of the Double checked locking idiom that sometimes is used for lazy initialisation. Read the provided link for why this is a really bad implementation of it.
The problem with the given code is that it fails intermittently. There is a race condition when there are several threads trying to work on the map using the same key (or keys with the same hashcode) which means that the map created first might be replaced by the second hashmap.

1 -The synch is trying to avoid that two threads, at the same time, create a new Entry in that Map. The second one must wait so his (servers==null) doesn't also return true.
2 - That users list seems to be out of scope, but seems like it doesn't need a synch. Maybe the programmer knows there is no duplicated userIds, or maybe he doesn't care about resetting the same user again and again.
3- ConcurrentHashMap maybe?

A function providing single object to recursive method

The code is posted for review on review board. My intention is not asking to review the code.
[Maze] : https://codereview.stackexchange.com/questions/33155/maze-code-review
In the above code the function solve does nothing but provide a stack object (reference to object of type stack) which would be used by a code executing recursively.
Since there are so many patterns is there a name for such a function which only assists / or does setup for recursive calls ?
If so any do's / dont's / alternatives ?

I think you did just fine. Every recursive algorithm needs some initial values for it's first step. It's common practice to encapsulate this initial call in another method so the caller doesn't have to bother with those values.
If your initial values would be more complicated to set up, you could encapsulate that in additional methods too. Say your stack needs to have some content instead of being empty. You could do something like this:
public List<Coordinate> solve() {
return getMazePath(0, 0, getInitialStack());
}
This way the solve method stays clear and easy as the entry point of your recursion.

Why does Java toString() loop infinitely on indirect cycles?

This is more a gotcha I wanted to share than a question: when printing with toString(), Java will detect direct cycles in a Collection (where the Collection refers to itself), but not indirect cycles (where a Collection refers to another Collection which refers to the first one - or with more steps).
import java.util.*;
public class ShonkyCycle {
static public void main(String[] args) {
List a = new LinkedList();
a.add(a); // direct cycle
System.out.println(a); // works: [(this Collection)]
List b = new LinkedList();
a.add(b);
b.add(a); // indirect cycle
System.out.println(a); // shonky: causes infinite loop!
}
}
This was a real gotcha for me, because it occurred in debugging code to print out the Collection (I was surprised when it caught a direct cycle, so I assumed incorrectly that they had implemented the check in general). There is a question: why?
The explanation I can think of is that it is very inexpensive to check for a collection that refers to itself, as you only need to store the collection (which you have already), but for longer cycles, you need to store all the collections you encounter, starting from the root. Additionally, you might not be able to tell for sure what the root is, and so you'd have to store every collection in the system - which you do anyway - but you'd also have to do a hash lookup on every collection element. It's very expensive for the relatively rare case of cycles (in most programming). (I think) the only reason it checks for direct cycles is because it so cheap (one reference comparison).
OK... I've kinda answered my own question - but have I missed anything important? Anyone want to add anything?
Clarification: I now realize the problem I saw is specific to printing a Collection (i.e. the toString() method). There's no problem with cycles per se (I use them myself and need to have them); the problem is that Java can't print them. Edit Andrzej Doyle points out it's not just collections, but any object whose toString is called.
Given that it's constrained to this method, here's an algorithm to check for it:
the root is the object that the first toString() is invoked on (to determine this, you need to maintain state on whether a toString is currently in progress or not; so this is inconvenient).
as you traverse each object, you add it to an IdentityHashMap, along with a unique identifier (e.g. an incremented index).
but if this object is already in the Map, write out its identifier instead.
This approach also correctly renders multirefs (a node that is referred to more than once).
The memory cost is the IdentityHashMap (one reference and index per object); the complexity cost is a hash lookup for every node in the directed graph (i.e. each object that is printed).

I think fundamentally it's because while the language tries to stop you from shooting yourself in the foot, it shouldn't really do so in a way that's expensive. So while it's almost free to compare object pointers (e.g. does obj == this) anything beyond that involves invoking methods on the object you're passing in.
And at this point the library code doesn't know anything about the objects you're passing in. For one, the generics implementation doesn't know if they're instances of Collection (or Iterable) themselves, and while it could find this out via instanceof, who's to say whether it's a "collection-like" object that isn't actually a collection, but still contains a deferred circular reference? Secondly, even if it is a collection there's no telling what it's actual implementation and thus behaviour is like. Theoretically one could have a collection containing all the Longs which is going to be used lazily; but since the library doesn't know this it would be hideously expensive to iterate over every entry. Or in fact one could even design a collection with an Iterator that never terminated (though this would be difficult to use in practice because so many constructs/library classes assume that hasNext will eventually return false).
So it basically comes down to an unknown, possibly infinite cost in order to stop you from doing something that might not actually be an issue anyway.

I'd just like to point out that this statement:
when printing with toString(), Java will detect direct cycles in a collection
is misleading.
Java (the JVM, the language itself, etc) is not detecting the self-reference. Rather this is a property of the toString() method/override of java.util.AbstractCollection.
If you were to create your own Collection implementation, the language/platform wouldn't automatically safe you from a self-reference like this - unless you extend AbstractCollection, you would have to make sure you cover this logic yourself.
I might be splitting hairs here but I think this is an important distinction to make. Just because one of the foundation classes in the JDK does something doesn't mean that "Java" as an overall umbrella does it.
Here is the relevant source code in AbstractCollection.toString(), with the key line commented:
public String toString() {
Iterator<E> i = iterator();
if (! i.hasNext())
return "[]";
StringBuilder sb = new StringBuilder();
sb.append('[');
for (;;) {
E e = i.next();
// self-reference check:
sb.append(e == this ? "(this Collection)" : e);
if (! i.hasNext())
return sb.append(']').toString();
sb.append(", ");
}
}

The problem with the algorithm that you propose is that you need to pass the IdentityHashMap to all Collections involved. This is not possible using the published Collection APIs. The Collection interface does not define a toString(IdentityHashMap) method.
I imagine that whoever at Sun put the self reference check into the AbstractCollection.toString() method thought of all of this, and (in conjunction with his colleagues) decided that a "total solution" is over the top. I think that the current design / implementation is correct.
It is not a requirement that Object.toString implementations be bomb-proof.

You are right, you already answered your own question. Checking for longer cycles (especially really long ones like period length 1000) would be too much overhead and is not needed in most cases. If someone wants it, he has to check it himself.
The direct cycle case, however, is easy to check and will occur more often, so it's done by Java.

You can't really detect indirect cycles; it's a typical example of the halting problem.

strange HashMap.put() behaviour

I am attempting to troubleshoot an intermittent failure that appears to be related to removing an object from a HashMap and then putting that same object back using a new key. My HashMap is created as follows:
transactions = new HashMap<Short, TransactionBase>();
The code that does the re-assignment is as follows:
transactions.remove(transaction.tran_no);
transaction.tran_no = generate_transaction_id();
transactions.put(transaction.tran_no, transaction);
The intermittent behaviour that I am seeing is that code that executes immediately after this that depends upon the transaction object being locatable does not appear to find the transaction object using the new transaction id. However, at some future point, the transaction can be located. So pulling at straws, is there any sort of asynchronous effect to put() or remove that can cause this sort of behaviour?
I should mention that, to the best of my knowledge, the container is being accessed by only one thread. I have already read in he documentation that class HashMap is not "synchronised".

There is a slight difference between remove/get and put (although my guess is that you have a threading issue).
The parameter for remove/get is of type Object; for put it is of type K. The reason for that has been stated many times before. This means it has problems with boxing. I'm not even going to guess what the rules are. If a value gets boxed as a Byte in one place and a Short in another, then those two objects cannot be equal.
There is a similar issue with List.remove(int) and List.remove(Object).

I presume that every time you check for the presence of the item you're definitely using a short or Short argument to Map.get() or Map.contains()?
These methods take Object arguments so if you pass them an int it will be converted to an Integer and will never match any item in your Map because they will all have Short keys.

There are no "asynchronous" effects in the HashMap class. As soon as you put something in there, it's there. You should double- and triple- check to make sure that there are no threading issues.
The only other thing I can think of is that you're making a copy of the HashMap somewhere. The copy obviously won't be affected by you adding stuff into the original, or vice versa.

Just one suggestion...you're focusing on accesses to the HashMap but I wonder if you should also see if your generate_transaction_id() is thread safe or if it is behaving in an unexpected way.

Have you overridden equals() but not hashCode() in your model objects? How about compareTo() ? If you get these wrong, Collections will behave strangely indeed.
Check Java Practices on equals() and compareTo().

What does this generate_transaction_id() function do? If it is generating a 16-bit GUID-like thing, you could easily get hash collisions. Combined with threading, you could get:
T1: transaction1.tran_no = generate_transaction_id(); => 1729
T2: transaction2.tran_no = generate_transaction_id(); => 1729
T1: transactions.put(transaction1.tran_no, transaction1); => map.put(1729, transaction1)
T2: transactions.put(transaction2.tran_no, transaction2); => map.put(1729, transaction2)
T1: int tran_no = transactions.get(1729); => transaction2
T1: transactions.remove(transaction.tran_no); => map.remove(1729)
T2: int tran_no = transactions.get(1729); => null
Of course, this could only be a solution if that 'to the best of your knowledge' part is not true.

So you know HashMap's not thread safe. Are you sure it's being accessed by only one thread? After all, intermittent failures a frequently threading related. If not, you can wrap it with Collections.synchronizedMap(), like so:
Collections.synchronizedMap(transactions);
You could always just try so you can eliminate that possibility.
It should be pointed out that this just wraps the original map with one with all the methods synchronized. You may want to consider using a synchronized block if the access is localized.

Threading has been mentioned in a few responses already, but have you considered the visibility issue for objects used by multiple threads? It's possible (and quite common) that if you put an object into a collection in one thread, it will not be "published" to other threads unless you have properly synchronized on the collection.
Threads and Locks
Synchronization and thread safety in Java

As others have observed you HAVE to know whether the HashMap is accessed by just one Thread, or not. CollectionSpy is a new profiler that lets you find out instantly, for all containers, how many threads perform any accesses. See www.collectionspy.com for more details.

Named blocks to limit variable scope: good idea?

For years, I've been using named blocks to limit the scope of temporary variables. I've never seen this done anywhere else, which makes me wonder if this is a bad idea. Especially since the Eclipse IDE flags these as warnings by default.
I've used this to good effect, I think, in my own code. But since it is un-idiomatic to the point where good programmers will distrust it when they see it, I really have two ways to go from here:
avoid doing it, or
promote it, with the hope that it will become an idiom.
Example (within a larger method):
final Date nextTuesday;
initNextTuesday: {
GregorianCalendar cal = new GregorianCalendar();
... // About 5-10 lines of setting the calendar fields
nextTuesday = cal.getTime();
}
Here I'm using a GregorianCalendar just to initialize a date, and I want to make sure that I don't accidentally reuse it.
Some people have commented that you don't actually need to name the block. While that's true, a raw block looks even more like a bug, as the intent is unclear. Furthermore, naming something encourages you to think about the intention of the block. The goal here is to identify distinct sections of code, not to give every temporary variable its own scope.
Many people have commented that it's best to go straight to small methods. I agree that this should be your first instinct. However, there may be several mitigating factors:
To even consider a named block, the code should be short, one-off code that will never be called elsewhere.
A named block is a quick way to organize an oversized method without creating a one-off method with a dozen parameters. This is especially true when a class is in flux, and the inputs are likely to change from version to version.
Creating a new method encourages its reuse, which may be ill-advised if the use cases aren't well-established. A named block is easier (psychologically, at least) to throw away.
Especially for unit tests, you may need to define a dozen different objects for one-off assertions, and they are just different enough that you can't (yet) find a way to consolidate them into a small number of methods, nor can you think of a way to distinguish them with names that aren't a mile long.
Advantages of using the named scope:
Can't accidentally reuse temporary variables
Limited scope gives garbage collector and JIT compiler more information about programmer intent
Block name provides a comment on a block of code, which I find more readable than open-ended comments
Makes it easier to refactor code out of a big method into little methods, or vice versa, since the named block is easier to separate than unstructured code.
Disadvantages:
Not idiomatic: programmers who haven't seen this use of named blocks (i.e. everyone but me) assume it's buggy, since they can't find references to the block name. (Just like Eclipse does.) And getting something to become idiomatic is an uphill battle.
It can be used as an excuse for bad programming habits, such as:
Making huge, monolithic methods where several small methods would be more legible.
Layers of indentation too deep to read easily.
Note: I've edited this question extensively, based on some thoughtful responses. Thanks!

I'd just go straight for refactoring into smaller methods. If a method is big enough that it needs breaking up like this, it really needs breaking up into multiple methods if at all possible.
While limiting scope is nice, this isn't really what named blocks are for. It's unidiomatic, which is very rarely a good thing.

If this was bad, then why is this a feature in the language! It's got a purpose, and you've found it.
I often write code exactly as in your example. When you want to initialize a variable, and there's a little calculation that needs doing to work out what that should be, and that involves a couple of variables... then you don't want those variables hanging around for the entire scope of your function, then a little scope to contain the initialization works great.
Mini scopes are an easy way to break code into 'paragraphs'. If you split into methods you can make the code harder to navigate when those methods don't get called from anywhere else and have a serial kind of order in which they need to be executed.
It's always a balance, but if you think it's going to be easiest to maintain and it actually adds value to a future reader of your code if its all inline, then go for it.
There are no hard and fast rules. I get a little fed up sometimes with co-workers who excessively put everything into its own method or class or file, and this becomes a nightmare to navigate. There's a nice balance somewhere!

Sometimes I use unnamed blocks to isolate mutable things needed to prepare some immutable thing. Instead of having a label I put the Block under the immutable variable declaration.
final String example;
{
final StringBuilder sb = new StringBuilder();
for(int i = 0; i < 100; i++)
sb.append(i);
example = sb.toString();
}
When I find some other use for the block, or just think that it's in the way, I turn it into a method.

Using blocks to limit scope is a good technique in my book.
But since you're using the label to do the work of a comment, why not just use an actual comment instead? This would remove the confusion about the unreferenced label.

This is the 1st time I am seeing someone else using blocks. whew! I thought I was the only one. I know that I didn't invent it -- remembered reading it somewhere -- possibly from my previous C++ world.
I don't use the labels, though and just comment what I'm doing.
I don't agree with all the guys that are asking you extract it into a method. Most of the things we don in such blocks aren't really reusable blocks. It makes sense in a big initialization AND YES, I've used blocks to prevent COPY/PASTE errors.
BR,
~A

If you have 5-10 lines of code that can safely be put into a block like that, the same code could just as well be extracted into a method.
This might seem like it's only a semantic difference, but at least with extracting into a method then you would gain the benefit of the ability of re-use.

Just because they exist doesn't mean they should be used. Most of the advantages gained from using named blocks are better gained by using a new private method.
You won't be able to use the temporary variables declared in the new method
The GC and JIT Compiler will glean the same info by using a new method
Using a descriptive name for the new method (using "private Date initNextTuesday()" in your case) will allow for the self commenting code advantage
No need to refactor code when you have already "pre-factored" it
In addition to these benefits, you also get code reuse benefits and it will shorten your long methods.

I'd use a block with a comment rather adding a label there.
When I see a label, I can't assume that nothing else is referencing the block.
If I change the behavior of the block, then the label name may not be appropriate any more. But I can't just reach out and change it: I'll have to look through the rest of the method to determine what label is calling out to the block. At which point I'll figure out that it's an unreferenced label.
Using a comment is clearer in this instance, because it describes the behavior of the block without imposing any extra work on the part of the maintainer.

It's a good technique in my book. Managing large numbers of throwaway methods is evil and the reasons you're providing for naming the blocks are good.
What does the generated bytecode look like? That'd be my only hesitation. I suspect it strips away the block name and might even benefit from greater optimizations. But you'd have to check.

Sorry for resurrecting this, but I didn't see anyone mention what I consider to be a very important point. Let's look at your example:
final Date nextTuesday;
initNextTuesday: {
GregorianCalendar cal = new GregorianCalendar();
... // About 5-10 lines of setting the calendar fields
nextTuesday = cal.getTime();
}
Including this initialization logic here makes it easier to understand if you're reading the file from top to bottom and care about every line. But think about how you read code. Do you start reading from the top of a file and continue to the bottom? Of course not! The only time you would ever do that is during a code review. Instead, you probably have a starting point based on previous knowledge, a stack trace, etc. Then you drill further down/up through the execution path until you find what you're looking for. Optimize for reading based on execution path, not code reviews.
Does the person reading the code that uses nextTuesday really want to read about how it's initialized? I would argue that the only information that they need is that there's a Date corresponding to next Tuesday. All of this information is contained in its declaration. This is a perfect example of code that should be broken into a private method, because it isn't necessary to understand the logic that the reader cares about.
final Date nextTuesday;
initNextTuesday: {
GregorianCalendar cal = new GregorianCalendar();
//1
//2
//3
//4
//5
nextTuesday = cal.getTime();
}
vs:
final Date nextTuesday = getNextTuesday();
Which would you rather read on your way through a module?

Name Blocks helps: Using break as a Form of Goto
Using break as a civilized form of goto.
class Break {
public static void main(String args[]) {
boolean t = true;
first: {
second: {
third: {
System.out.println("Before the break.");
if (t)
break second; // break out of second block
System.out.println("This won't execute");
}
System.out.println("This won't execute");
}
System.out.println("This is after second block.");
}
}
}
Using break to exit from nested loops
class BreakLoop4 {
public static void main(String args[]) {
outer: for (int i = 0; i < 3; i++) {
System.out.print("Pass " + i + ": ");
for (int j = 0; j < 100; j++) {
if (j == 10)
break outer; // exit both loops
System.out.print(j + " ");
}
System.out.println("This will not print");
}
System.out.println("Loops complete.");
}
}
Source Link

I have done this in some of my c#. I didn't know you could name the blocks though, I'll have to try that see if it works in c# too.
I think the scope block can be a nice idea, because you can encapsulate code specific to something within a block of code, where you might not want to split it out into its own function.
As for the disadvantage of nesting them, I see that as more of a fault of a programmer not of scope blocks themselves.

Named scopes are technically ok here, it's just they aren't used in this way very often. Therefore, when someone else comes to maintain your code in the future it may not be immediately obvious why they are there. IMHO a private helper method would be a better choice...

I love the idea of using block to limit var scope.
So many times I was confused by short-lived vars given large scope which should go away immediately after use. Long method + many non-final vars make it difficult to reason about the coder's intention, especially when comments are rare. Considering much of the logic I see in a method were like below
Type foo(args..){
declare ret
...
make temp vars to add information on ret
...
make some more temp vars to add info on ret. not much related to above code. but previously declared vars are still alive
...
return ret
}
if vars can have smaller scope than the entire method body, I can quickly forget most of them (good thing).
Also I agree that too many or too few private things leads to spaghetti code.
Actually what I was looking for was something like nested method in functional languages, and seems its cousin in Java is a {BLOCK} (inner class and labmda expression are not for this..).
However, I would prefer to use a unnamed block since this may be misleading to people trying to find the reference to the label, plus I can explain better with commented block.
For using a private method, I would consider it as the next step of using blocks.