I have a class that needs to provide a fast classification service. For example, I want to write code like "classify("Ac Kd Kh 3c 3s")" that quickly returns TWO_PAIR. (This isn't the application but you get the jist)
Because I need the classification to be quick I want to precompute, and then store, a look-up table that lists the classification output for all possible inputs. In the interest of time I want to parallelize this precomputation. HOWEVER, attempting to use "classifySlowly" from a 2nd thread creates a deadlock.
public class Classifcation Service {
enum CLASS {TYPE_A, TYPE_B, ...};
static CLASS[] preComputedClassLookUpTable;
static {
preComputedClassLookUpTable = constructLookUpTableInParallel();
}
//Note: using this method from with constructLookUpTableInParallel causes deadlock
private static CLASS classifySlowly(Object classifyMe) {
//do time intensive work to classify the input
// -- uses no other methods from this class
return classification;
}
public static CLASS classify(Object classifyMe) {
//use the lookup table to do a quick classification
return classification;
}
}
So my question is: Is there a way to precompute this lookup table IN PARALLEL from within the static initalizer block?
The only (poor) alternative I see is to switch from:
preComputedClassLookUpTable = constructLookUpTableInParallel();
To:
preComputeClassLookUpTable = loadLookUpTableFromFile();
if(preComputedClassLookUpTable == null) {
System.out.println("WARNING: Construction incomplete, Must call computeAndSaveLookUpTableFile();}
}
I thought this would be too much but here is the implementation of constructLookUpTableInParallel
private static CLASS[] constructLookUpTableInParallel() {
//build a collection of Runnables by iterating over all possible input Objects
//wrap each possible input in an anonymous Runnable that calls classifySlowly.
//submit the collection of Runnables to a new ExecutorService
//process runnables...
//shutdown executor service
}
////////END OF POORLY WORDED ORIGINAL QUESTION ///////////
The solution that works somewhat cleanly is splitting the classifySlowly(Object classifyMe) and classify(Object classifyMe) methods into two different classes.
This will allow the (first) class that contains "public static CLASS classifySlowly(Object classifyMe)" to be fully loaded by the time the (second) class that contains "public static CLASS classifyQuickly(Object classifyMe)" needs to use the classifySlowly method. Now, that the 2nd static inialization block don't require any of its own static methods it can be fully parallelized.
"So my question is: Is there a way to precompute this lookup table IN PARALLEL from within the static initalizer block?"
Yes, it's practically certain there is a way. Just new the array and launch a Runnable for each array element. Give each Runnable reference to the array, and index it is computing, then have it do the computing without locking, then lock when assigning result to the array element.
Note/disclaimer: this answer is based on the rather incomplete information given in the question...
Related
I'm trying to automate the testing process for customly written programs designed to solve competitive programming challenges. Below is a dummy sample implementation of Solution:
public class Solution {
private static String dummyField = "initial";
public static int initialize(InputStream in) {
//competitive programmer custom code
System.out.println(dummyField);
dummyField = "changed";
return subCaseCount;
}
public void processSingleSubCase(InputStream in) {
//competitive programmer custom code
}
}
Prewritten test code for solution regardless of its implementation:
public void testSolution() throws FileNotFoundException {
for(File testResource : testResources) {
InputStream in = new FileInputStream(testResource);
int subCaseCount = Foo.initialize(in);
for (int subCase = 0; subCase < subCaseCount; subCase++) {
new Foo().processSingleSubCase(in);
}
//magic call to re-init all static fields without knowing their number/names in advance goes here
}
//console current output:
//initial
//changed
//changed
//...
//desired:
//initial
//initial
//initial
//....
}
The static fields can be mutable, so caching the initial values and mapping them to field names using reflection as a first setup, then reassigning them in between iterations won't do.
I did manage to come up with a working solution which basically reloads the class using a different class loader in between iterations, it did work but was slow: it took about 50 seconds just to reload classes 300 times (test resources are auto generated and I'd like to have the flexibility to auto generate as many as tolerable).
Is there a faster alternative?
My two thoughts for how to do this are:
Use instances rather than statics, since that way the new instance for each test is fresh.
If you need (or want) to stick with statics: Prior to the first test, cache the static values, then reassign them from the cache between tests. If the static values are object references referring to mutable objects, you'll need to make deep copies.
So I have been having a go with using the method reference in Java 8 (Object::Method). What I am attempting to do, which I have done before but have forgotten (last time I used this method reference was about 4 months ago), is find the amount of players that != online using the Method Reference.
public static Set<Friend> getOnlineFriends(UUID playerUUID)
{
Set<Friend> friends = new HashSet<>(Arrays.asList(ZMFriends.getFriends(playerUUID)));
return friends.stream().filter(Friend::isOnline).collect(Collectors.toSet());
}
public static Set<Friend> getOfflineFriends(UUID playerUUID)
{
Set<Friend> friends = new HashSet<>(Arrays.asList(ZMFriends.getFriends(playerUUID)));
return friends.stream().filter(Friend::isOnline).collect(Collectors.toSet());
As you can see I managed to so it when the player (friend) is online but I cannot figure out how to filter though the Set and collect the offline players. I'm missing something obvious, but what is it?!?!
Thanks,
Duke.
In you code
public static Set<Friend> getOnlineFriends(UUID playerUUID)
{
Set<Friend> friends = new HashSet<>(Arrays.asList(ZMFriends.getFriends(playerUUID)));
return friends.stream().filter(Friend::isOnline).collect(Collectors.toSet());
}
you are creating a List view to the array returned by ZMFriends.getFriends(playerUUID), copy its contents to a HashSet, just to call stream() on it.
That’s a waste of resources, as the source type is irrelevant to the subsequent stream operation. You don’t need to have a Set source to get a Set result. So you can implement your operation simply as
public static Set<Friend> getOnlineFriends(UUID playerUUID)
{
return Arrays.stream(ZMFriends.getFriends(playerUUID))
.filter(Friend::isOnline).collect(Collectors.toSet());
}
Further, you should consider whether you really need both, getOnlineFriends and getOfflineFriends in your actual implementation. Creating utility methods in advance, just because you might need them, rarely pays off. See also “You aren’t gonna need it”.
But if you really need both operations, it’s still an unnecessary code duplication. Just consider:
public static Set<Friend> getFriends(UUID playerUUID, boolean online)
{
return Arrays.stream(ZMFriends.getFriends(playerUUID))
.filter(f -> f.isOnline()==online).collect(Collectors.toSet());
}
solving both tasks. It still wastes resource, if the application really needs both Sets, as the application still has to perform the same operation twice to get both Sets. Consider:
public static Map<Boolean,Set<Friend>> getOnlineFriends(UUID playerUUID)
{
return Arrays.stream(ZMFriends.getFriends(playerUUID))
.collect(Collectors.partitioningBy(Friend::isOnline, Collectors.toSet()));
}
This provides you both Sets at once, the online friends being associated to true, the offline friends being associated to false.
There are 2 ways I can think of:
friends.stream().filter(i -> !i.isOnline()).collect(Collectors.toSet());
But I guess that's not what you want, since it's not using a method reference. So maybe something like this:
public static <T> Predicate<T> negation(Predicate<T> predicate) {
return predicate.negate();
}
...
friends.stream().filter(negation(Friend::isOnline)).collect(Collectors.toSet());
I have an enum as follows:
public enum ServerTask {
HOOK_BEFORE_ALL_TASKS("Execute"),
COPY_MASTER_AND_SNAPSHOT_TO_HISTORY("Copy master db"),
PROCESS_CHECKIN_QUEUE("Process Check-In Queue"),
...
}
I also have a string (lets say string = "Execute") which I would like to make into an instance of the ServerTask enum based on which string in the enum that it matches with. Is there a better way to do this than doing equality checks between the string I want to match and every item in the enum? seems like this would be a lot of if statements since my enum is fairly large
At some level you're going to have to iterate over the entire set of enumerations that you have, and you'll have to compare them to equal - either via a mapping structure (initial population) or through a rudimentary loop.
It's fairly easy to accomplish with a rudimentary loop, so I don't see any reason why you wouldn't want to go this route. The code snippet below assumes the field is named friendlyTask.
public static ServerTask forTaskName(String friendlyTask) {
for (ServerTask serverTask : ServerTask.values()) {
if(serverTask.friendlyTask.equals(friendlyTask)) {
return serverTask;
}
}
return null;
}
The caveat to this approach is that the data won't be stored internally, and depending on how many enums you actually have and how many times you want to invoke this method, it would perform slightly worse than initializing with a map.
However, this approach is the most straightforward. If you find yourself in a position where you have several hundred enums (even more than 20 is a smell to me), consider what it is those enumerations represent and what one should do to break it out a bit more.
Create static reverse lookup map.
public enum ServerTask {
HOOK_BEFORE_ALL_TASKS("Execute"),
COPY_MASTER_AND_SNAPSHOT_TO_HISTORY("Copy master db"),
PROCESS_CHECKIN_QUEUE("Process Check-In Queue"),
...
FINAL_ITEM("Final item");
// For static data always prefer to use Guava's Immutable library
// http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/ImmutableMap.html
static ImmutableMap< String, ServerTask > REVERSE_MAP;
static
{
ImmutableMap.Builder< String, ServerTask > reverseMapBuilder =
ImmutableMap.builder( );
// Build the reverse map by iterating all the values of your enum
for ( ServerTask cur : values() )
{
reverseMapBuilder.put( cur.taskName, cur );
}
REVERSE_MAP = reverseMapBuilder.build( );
}
// Now is the lookup method
public static ServerTask fromTaskName( String friendlyName )
{
// Will return ENUM if friendlyName matches what you stored
// with enum
return REVERSE_MAP.get( friendlyName );
}
}
If you have to get the enum from the String often, then creating a reverse map like Alexander suggests might be worth it.
If you only have to do it once or twice, looping over the values with a single if statement might be your best bet (like Nizil's comment insinuates)
for (ServerTask task : ServerTask.values())
{
//Check here if strings match
}
However there is a way to not iterate over the values at all. If you can ensure that the name of the enum instance and its String value are identical, then you can use:
ServerTask.valueOf("EXECUTE")
which will give you ServerTask.EXECUTE.
Refer this answer for more info.
Having said that, I would not recommend this approach unless you're OK with having instances have the same String representations as their identifiers and yours is a performance critical application which is most often not the case.
You could write a method like this:
static ServerTask getServerTask(String name)
{
switch(name)
{
case "Execute": return HOOK_BEFORE_ALL_TASKS;
case "Copy master db": return COPY_MASTER_AND_SNAPSHOT_TO_HISTORY;
case "Process Check-In Queue": return PROCESS_CHECKIN_QUEUE;
}
}
It's smaller, but not automatic like #Alexander_Pogrebnyak's solution. If the enum changes, you would have to update the switch.
I'm new to java rmi and I have issues with synchronization.
The server handles a small data base throw files (7 files,each one represents an university),
the client when connected gives the name of the university then chooses an option :
add a student
remove a student
update a student
search for a student
Everything works fine, but I have a problem with synchronization.It doesn't work the way I expected.
Say we have 3 files, I created 3 private static Integers to use like this
public class CarnetImpl extends UnicastRemoteObject implements Carnet {
private String fac;
private static Integer univ1=1;
private static Integer univ2=1;
private static Integer univ3=1;
CarnetImpl(String fac) throws RemoteException {
this.fac=fac;
}
public void add(Student e) throws RemoteException {
Integer lock=1
switch (fac){
case "univ1":
lock=univ1;
break;
case "univ2":
lock=univ2;
break;
case "univ3":
lock=univ3;
break;
}
synchronized(lock){
//creating a file named "fac.txt" (fac can be univ1,2 or3) and adding a student
}
}
}
I did the same thing for the other methods.
What I expected is that for a given university only one client can use a method,while more than one client can use the same method for different universities at the same time.
But after testing it seems that even for different universities a client have to wait for an other to finish to use the method.
Example :
Client 1 asks the server to add student1 to univ1 (i added a 5' sleep and a println to detect threads behavior ).
Before the end of the 5 seconds client 2 asks the server to add(or any other method) student2 to univ2.
As client2 asked an add on univ2,I expected that lock will take univ2 so the thread wouldn't wait because univ2 is not locked unlike univ1.
Can anyone help me understand ?
Any suggestions to get the expected behavior would be most welcome.
Thanks.
private static Integer univ1=1;
private static Integer univ2=1;
private static Integer univ3=1;
These are three references to the one and the same object, the Integer instance from that class's internal cache of all values between -128 and 128.
You would have avoided this if you kept to the recommended practice to use plain Objects for locks.
I should also comment that your whole design is needlessly roundabout: since fac is fixed at instantiation time, you would be better off assigning the proper lock object to an instance variable at that point instead of going through the decision cascade on each method call.
I've written a few multithreaded hobby programs and some in my previous(engineering/physics) studies as well, so I consider myself to have an above-beginner knowledge in the area of synchronization/thread safety and primitives, what the average user finds to be challanging with the JMM and multiple threads etc.
What I find that I need and there is no proper method of marking instance or static members of classes as shared by different threads. Think about it, we have access rules such as private/protected/public and conventions on how to name getters/setters and a lot of things.
But what about threading? What if I want to mark a variable as thread shared and have it follow certain rules? Volatile/Atomic refs might do the job, but sometimes you just do need to use mutexes. And when you manually have to remember to use something...you will forget about it :) - At some point.
So I had an idea, and I see I am not the first, I also checked out http://checkthread.org/example-threadsafe.html - They seem to have a pretty decent code analyzer which I might try later which sort of lets me do some of the things I want.
But coming back to the initial problem. Let's say we need something a little more low level than a message passing framework and we need something a little more high level than primitive mutexes... What do we have...wel...nothing?
So basically, what I've made is a sort of pure java super-simple framework for threading that lets you declare class members as shared or non-shared...well sort of :).
Below is an example of how it could be used:
public class SimClient extends AbstractLooper {
private static final int DEFAULT_HEARTBEAT_TIMEOUT_MILLIS = 2000;
// Accessed by single threads only
private final SocketAddress socketAddress;
private final Parser parser;
private final Callback cb;
private final Heart heart;
private boolean lookingForFirstMsg = true;
private BufferedInputStream is;
// May be accessed by several threads (T*)
private final Shared<AllThreadsVars> shared = new Shared<>(new AllThreadsVars());
.
.
.
.
static class AllThreadsVars {
public boolean connected = false;
public Socket socket = new Socket();
public BufferedOutputStream os = null;
public long lastMessageAt = 0;
}
And to access the variables marked as thread shared you must send a runnable-like functor to the Shared object:
public final void transmit(final byte[] data) {
shared.run(new SharedRunnable<AllThreadsVars, Object, Object>() {
#Override
public Object run(final AllThreadsVars sharedVariable, final Object input) {
try {
if (sharedVariable.socket.isConnected() && sharedVariable.os != null) {
sharedVariable.os.write(data);
sharedVariable.os.flush();
}
} catch (final Exception e) { // Disconnected
setLastMessageAt(0);
}
return null;
}
}, null);
}
Where a shared runnable is defined like:
public interface SharedRunnable<SHARED_TYPE, INPUT, OUTPUT> {
OUTPUT run(final SHARED_TYPE s, final INPUT input);
}
Where is this going?
Well this gives me the help (yes you can leak out and break it but far less likely) that I can mark variable sets (not just variables) as thread shared, and once that is done, have it guaranteed in compile time ( I cannot forget to synchronize some method). It also allows me to standardize and perform tests to look for possible deadlocks also in compile time (Though atm I only implemented it in runtime cause doing it in compile time with the above framework will probably require more than just the java compiler).
Basically this is extremely useful to me and I'm wondering if I'm just reinventing the wheel here or of this could be some anti-pattern I don't know of. And I really don't know who to ask. (Oh yeah and Shared.run(SharedRunnable r, INPUT input) works just like
private final <OUTPUT, INPUT> OUTPUT run(final SharedRunnable<SHARED_TYPE, INPUT, OUTPUT> r, final INPUT input) {
try {
lock.lock();
return r.run(sharedVariable, input);
} finally {
lock.unlock();
}
}
This is just my own experimentation so it's not really finished by any means, but I have one decent project using it right now and it's really helping out a lot.
You mean something like this? (which can be enforced by tools like findbugs.)
If you have values which should be shared, the best approach is encapsulate this within the class. This way the caller does need to know what thread model you are using. If you want to know what model is used internally, you can read the source, however the caller cannot forget to access a ConcurrentMap (for example) correctly because all its method are thread safe.