Java: Framework for thread shared data - java

I've written a few multithreaded hobby programs and some in my previous(engineering/physics) studies as well, so I consider myself to have an above-beginner knowledge in the area of synchronization/thread safety and primitives, what the average user finds to be challanging with the JMM and multiple threads etc.
What I find that I need and there is no proper method of marking instance or static members of classes as shared by different threads. Think about it, we have access rules such as private/protected/public and conventions on how to name getters/setters and a lot of things.
But what about threading? What if I want to mark a variable as thread shared and have it follow certain rules? Volatile/Atomic refs might do the job, but sometimes you just do need to use mutexes. And when you manually have to remember to use something...you will forget about it :) - At some point.
So I had an idea, and I see I am not the first, I also checked out http://checkthread.org/example-threadsafe.html - They seem to have a pretty decent code analyzer which I might try later which sort of lets me do some of the things I want.
But coming back to the initial problem. Let's say we need something a little more low level than a message passing framework and we need something a little more high level than primitive mutexes... What do we have...wel...nothing?
So basically, what I've made is a sort of pure java super-simple framework for threading that lets you declare class members as shared or non-shared...well sort of :).
Below is an example of how it could be used:
public class SimClient extends AbstractLooper {
private static final int DEFAULT_HEARTBEAT_TIMEOUT_MILLIS = 2000;
// Accessed by single threads only
private final SocketAddress socketAddress;
private final Parser parser;
private final Callback cb;
private final Heart heart;
private boolean lookingForFirstMsg = true;
private BufferedInputStream is;
// May be accessed by several threads (T*)
private final Shared<AllThreadsVars> shared = new Shared<>(new AllThreadsVars());
.
.
.
.
static class AllThreadsVars {
public boolean connected = false;
public Socket socket = new Socket();
public BufferedOutputStream os = null;
public long lastMessageAt = 0;
}
And to access the variables marked as thread shared you must send a runnable-like functor to the Shared object:
public final void transmit(final byte[] data) {
shared.run(new SharedRunnable<AllThreadsVars, Object, Object>() {
#Override
public Object run(final AllThreadsVars sharedVariable, final Object input) {
try {
if (sharedVariable.socket.isConnected() && sharedVariable.os != null) {
sharedVariable.os.write(data);
sharedVariable.os.flush();
}
} catch (final Exception e) { // Disconnected
setLastMessageAt(0);
}
return null;
}
}, null);
}
Where a shared runnable is defined like:
public interface SharedRunnable<SHARED_TYPE, INPUT, OUTPUT> {
OUTPUT run(final SHARED_TYPE s, final INPUT input);
}
Where is this going?
Well this gives me the help (yes you can leak out and break it but far less likely) that I can mark variable sets (not just variables) as thread shared, and once that is done, have it guaranteed in compile time ( I cannot forget to synchronize some method). It also allows me to standardize and perform tests to look for possible deadlocks also in compile time (Though atm I only implemented it in runtime cause doing it in compile time with the above framework will probably require more than just the java compiler).
Basically this is extremely useful to me and I'm wondering if I'm just reinventing the wheel here or of this could be some anti-pattern I don't know of. And I really don't know who to ask. (Oh yeah and Shared.run(SharedRunnable r, INPUT input) works just like
private final <OUTPUT, INPUT> OUTPUT run(final SharedRunnable<SHARED_TYPE, INPUT, OUTPUT> r, final INPUT input) {
try {
lock.lock();
return r.run(sharedVariable, input);
} finally {
lock.unlock();
}
}
This is just my own experimentation so it's not really finished by any means, but I have one decent project using it right now and it's really helping out a lot.

You mean something like this? (which can be enforced by tools like findbugs.)

If you have values which should be shared, the best approach is encapsulate this within the class. This way the caller does need to know what thread model you are using. If you want to know what model is used internally, you can read the source, however the caller cannot forget to access a ConcurrentMap (for example) correctly because all its method are thread safe.

Related

Is it possible to reinitialize static mutable fields in a class?

I'm trying to automate the testing process for customly written programs designed to solve competitive programming challenges. Below is a dummy sample implementation of Solution:
public class Solution {
private static String dummyField = "initial";
public static int initialize(InputStream in) {
//competitive programmer custom code
System.out.println(dummyField);
dummyField = "changed";
return subCaseCount;
}
public void processSingleSubCase(InputStream in) {
//competitive programmer custom code
}
}
Prewritten test code for solution regardless of its implementation:
public void testSolution() throws FileNotFoundException {
for(File testResource : testResources) {
InputStream in = new FileInputStream(testResource);
int subCaseCount = Foo.initialize(in);
for (int subCase = 0; subCase < subCaseCount; subCase++) {
new Foo().processSingleSubCase(in);
}
//magic call to re-init all static fields without knowing their number/names in advance goes here
}
//console current output:
//initial
//changed
//changed
//...
//desired:
//initial
//initial
//initial
//....
}
The static fields can be mutable, so caching the initial values and mapping them to field names using reflection as a first setup, then reassigning them in between iterations won't do.
I did manage to come up with a working solution which basically reloads the class using a different class loader in between iterations, it did work but was slow: it took about 50 seconds just to reload classes 300 times (test resources are auto generated and I'd like to have the flexibility to auto generate as many as tolerable).
Is there a faster alternative?
My two thoughts for how to do this are:
Use instances rather than statics, since that way the new instance for each test is fresh.
If you need (or want) to stick with statics: Prior to the first test, cache the static values, then reassign them from the cache between tests. If the static values are object references referring to mutable objects, you'll need to make deep copies.

Method reference and boolean

So I have been having a go with using the method reference in Java 8 (Object::Method). What I am attempting to do, which I have done before but have forgotten (last time I used this method reference was about 4 months ago), is find the amount of players that != online using the Method Reference.
public static Set<Friend> getOnlineFriends(UUID playerUUID)
{
Set<Friend> friends = new HashSet<>(Arrays.asList(ZMFriends.getFriends(playerUUID)));
return friends.stream().filter(Friend::isOnline).collect(Collectors.toSet());
}
public static Set<Friend> getOfflineFriends(UUID playerUUID)
{
Set<Friend> friends = new HashSet<>(Arrays.asList(ZMFriends.getFriends(playerUUID)));
return friends.stream().filter(Friend::isOnline).collect(Collectors.toSet());
As you can see I managed to so it when the player (friend) is online but I cannot figure out how to filter though the Set and collect the offline players. I'm missing something obvious, but what is it?!?!
Thanks,
Duke.
In you code
public static Set<Friend> getOnlineFriends(UUID playerUUID)
{
Set<Friend> friends = new HashSet<>(Arrays.asList(ZMFriends.getFriends(playerUUID)));
return friends.stream().filter(Friend::isOnline).collect(Collectors.toSet());
}
you are creating a List view to the array returned by ZMFriends.getFriends(playerUUID), copy its contents to a HashSet, just to call stream() on it.
That’s a waste of resources, as the source type is irrelevant to the subsequent stream operation. You don’t need to have a Set source to get a Set result. So you can implement your operation simply as
public static Set<Friend> getOnlineFriends(UUID playerUUID)
{
return Arrays.stream(ZMFriends.getFriends(playerUUID))
.filter(Friend::isOnline).collect(Collectors.toSet());
}
Further, you should consider whether you really need both, getOnlineFriends and getOfflineFriends in your actual implementation. Creating utility methods in advance, just because you might need them, rarely pays off. See also “You aren’t gonna need it”.
But if you really need both operations, it’s still an unnecessary code duplication. Just consider:
public static Set<Friend> getFriends(UUID playerUUID, boolean online)
{
return Arrays.stream(ZMFriends.getFriends(playerUUID))
.filter(f -> f.isOnline()==online).collect(Collectors.toSet());
}
solving both tasks. It still wastes resource, if the application really needs both Sets, as the application still has to perform the same operation twice to get both Sets. Consider:
public static Map<Boolean,Set<Friend>> getOnlineFriends(UUID playerUUID)
{
return Arrays.stream(ZMFriends.getFriends(playerUUID))
.collect(Collectors.partitioningBy(Friend::isOnline, Collectors.toSet()));
}
This provides you both Sets at once, the online friends being associated to true, the offline friends being associated to false.
There are 2 ways I can think of:
friends.stream().filter(i -> !i.isOnline()).collect(Collectors.toSet());
But I guess that's not what you want, since it's not using a method reference. So maybe something like this:
public static <T> Predicate<T> negation(Predicate<T> predicate) {
return predicate.negate();
}
...
friends.stream().filter(negation(Friend::isOnline)).collect(Collectors.toSet());

designing classes for other developers to use in java

class CSVReader {
private List<String> output;
private InputStream input;
public CSVReader(InputStream input) {
this.input = input;
}
public void read() throws Exception{
//do something with the inputstream
// create output list.
}
public List<String> getOutput() {
return Collections.unmodifiableList(output);
}
I am trying to create a simple class which will be part of a library. I would like to create code that satisfies the following conditions:
handles all potential errors or wraps them into library errors and
throws them.
creates meaningful and complete object states (no incomplete object structures).
easy to utilize by developers using the library
Now, when I evaluated the code above, against the goals, I realized that I failed badly. A developer using this code would have to write something like this -
CSVReader reader = new CVSReader(new FileInputStream("test.csv");
reader.read();
read.getOutput();
I see the following issues straight away -
- developer has to call read first before getOutput. There is no way for him to know this intuitively and this is probably bad design.
So, I decided to fix the code and write something like this
public List<String> getOutput() throws IOException{
if(output==null)
read();
return Collections.unmodifiableList(output);
}
OR this
public List<String> getOutput() {
if(output==null)
throw new IncompleteStateException("invoke read before getoutput()");
return Collections.unmodifiableList(output);
}
OR this
public CSVReader(InputStream input) {
read(); //throw runtime exception
}
OR this
public List<String> read() throws IOException {
//read and create output list.
// return list
}
What is a good way to achieve my goals? Should the object state be always well defined? - there is never a state where "output" is not defined, so I should create the output as part of constructor? Or should the class ensure that a created instance is always valid, by calling "read" whenever it finds that "output" is not defined and just throw a runtime exception? What is a good approach/ best practice here?
I would make read() private and have getOutput() call it as an implementation detail. If the point of exposing read() is to lazy-load the file, you can do that with exposing getOutput only
public List<String> getOutput() {
if (output == null) {
try {
output = read();
} catch (IOException) {
//here you either wrap into your own exception and then declare it in the signature of getOutput, or just not catch it and make getOutput `throws IOException`
}
}
return Collections.unmodifiableList(output);
}
The advantage of this is that the interface of your class is very trivial: you give me an input (via constructor) I give you an output (via getOutput), no magic order of calls while preserving lazy-loading which is nice if the file is big.
Another advantage of removing read from the public API is that you can go from lazy-loading to eager-loading and viceversa without affecting your clients. If you expose read you have to account for it being called in all possible states of your object (before it's loaded, while it's already running, after it already loaded). In short, always expose the least possible
So to address your specific questions:
Yes, the object state should always be well-defined. Your point of not knowing that an external call on read by the client class is needed is indeed a design smell
Yes, you could call read in the constructor and eagerly load everything upfront. Deciding to lazy-load or not is an implementation detail dependent on your context, it should not matter to a client of your class
Throwing an exception if read has not been called puts again the burden to calling things in the right, implicit order on the client, which is unnecessary due to your comment that output is never really undefined so the implementation itself can make the risk-free decision of when to call read
I would suggest you make your class as small as possible, dropping the getOutput() method all together.
The idea is to have a class that reads a CSV file and returns a list, representing the result. To achieve this, you can expose a single read() method, that will return a List<String>.
Something like:
public class CSVReader {
private final InputStream input;
public CSVReader(String filename) {
this.input = new FileInputStream(filename);
}
public List<String> read() {
// perform the actual reading here
}
}
You have a well defined class, a small interface to maintain and the instances of CSVReader are immutable.
Have getOutput check if it is null (or out of date) and load it in automatically if it is. This allows for a user of your class to not have to care about internal state of the class's file management.
However, you may also want to expose a read function so that the user can chose to load in the file when it is convenient. If you make the class for a concurrent environment, I would recommend doing so.
The first approach takes away some flexibility from the API: before the change the user could call read() in a context where an exception is expected, and then call getOutput() exception-free as many times as he pleases. Your change forces the user to catch a checked exception in contexts where it wasn't necessary before.
The second approach is how it should have been done in the first place: since calling read() is a prerequisite of calling getOutput(), it is a responsibility of your class to "catch" your users when they "forget" to make a call to read().
The third approach hides IOException, which may be a legitimate exception to catch. There is no way to let the user know if the exception is going to be thrown or not, which is a bad practice when designing runtime exceptions.
The root cause of your problem is that the class has two orthogonal responsibilities:
Reading a CSV, and
Storing the result of a read for later use.
If you separate these two responsibilities from each other, you would end up with a cleaner design, in which the users would have no confusion over what they must call, and in what order:
interface CSVData {
List<String> getOutput();
}
class CSVReader {
public static CSVData read(InputStream input) throws IOException {
...
}
}
You could combine the two into a single class with a factory method:
class CSVData {
private CSVData() { // No user instantiation
}
// Getting data is exception-free
public List<String> getOutput() {
...
}
// Creating instances requires a factory call
public static CSVData read(InputStream input) throws IOException {
...
}
}

JAVA api validation/Exception handling

How to handle error conditions when writing a Java API/Utility
This is my Implementation for my API interface
public void bin2zip(InputStream[] is,OuputStream os, String[] names)
{
//if number of streams and number of names do not match do something
}
What I am trying to do is handling a case when the length of the is != length of name.
How do i handle this. I dont want my API to do some work until ArrayOutOfBound exception to be thrown. I want to catch this early.
One solution is something like this:
if it does not match I throw
if(is.length==names.length)
throws new Exception("ParemeterValidationException: The inputstream array and name array length should match");
if(containsInvalidFileName(names))
throws new Exception("ParemeterValidationException: The names array length should contain valid filenames");
Also, can this be done compile time using DataDependency (I can make ValidationClass for the API and make sure the developer get hold of this object to pass on to this conversion API) or the runtime exception is the best way?
I believe doing a ValidationClass will make API use complicated
I did go through some materials (if anyone interested), but need some directions.
http://lcsd05.cs.tamu.edu/slides/keynote.pdf
Java: checked vs unchecked exception explanation
http://docs.oracle.com/javase/tutorial/collections/interoperability/api-design.html
Wherever possible, don't let end users screw it up.
public final class Bin2Zipper {
private final List<InputStream> inputStreams = ...;
private final List<String> names = ...;
public BinZipper() {
}
public void add(final InputStream is, final String name) {
this.inputStreams.add(is);
this.names.add(name);
}
public void bin2zip(final OutputStream os) {
// ...
}
}
A fluent interface might even be better. Then your code would look like:
Bin2Zipper.add(is1, name1).add(is2, name2).add(is3, name3).toZip(os);
public final class Bin2Zipper {
private final List<InputStream> inputStreams = ...;
private final List<String> names = ...;
private Bin2Zipper(final InputStream is, final String name) {
this.inputStreams.add(is);
this.names.add(name);
}
public static Bin2Zipper add(final InputStream is, final String name) {
return new Bin2Zipper(is, name);
}
public Bin2Zipper add(final InputStream is, final String name) {
this.inputStreams.add(is);
this.names.add(name);
return this;
}
public void zip(final OutputStream os) {
...
}
}
Where these fall down is when the client starts off with the two arrays. In that case, it can be annoying for them to have to loop over all the entries themselves. I think it's still worth it. If you don't, then you'll have to compare the sizes of the inputs right away. You almost certainly want to throw an unchecked exception, probably an IllegalArgumentException like Vince said.
I think your solution of comparing the array lengths is perfectly appropriate. I think in this case you should throw an IllegalArgumentException; this exception is defined in the standard and used by most standard functions doing this kind of checking.
Many standard libraries use this kind of interface it is easily understood.
That said I think you should prefer an interface that simply doesn't facilitate such misuse such as that suggested by #Eric - the library everybody likes to use is the one that works first time every time because it's too simple to mess up.

Saving on Instance Variables

Our server recently has been going down a lot and I was tasked to improve the memory usage of a set of classes that was identified to be the culprit.
I have code which initializes an instance of an object and goes like this:
boolean var1;
boolean var2;
.
.
.
boolean var100;
void setup() {
var1 = map.hasFlag("var1");
var2 = map.hasFlag("var2);
.
.
.
if (map.hasFlag("some flag") {
doSomething();
}
if (var1) {
increment something
}
if (var2) {
increment something
}
}
The setup code takes about 1300 lines. My question is if it is possible for this method to be more efficient in terms of using too many instance variables.
The instance variables by the way are used in a "main" method handleRow() where for example:
handleRow(){
if (var1) {
doSomething();
}
.
.
.
if (var100) {
doSomething();
}
}
One solution I am thinking is to change the implementation by removing the instance variables in the setup method and just calling it directly from the map when I need it:
handleRow(){
if (map.hasFlag("var1") {
doSomething();
}
.
.
.
if (map.hasFlag("var100") {
doSomething();
}
}
That's one solution I am considering but I would like to hear the inputs of the community. :)
If these are really all boolean variables, consider using a BitSet instead. You may find that reduces the memory footprint by a factor of 8 or possibly even 32 depending on padding.
100 boolean variables will take 1.6k of memory when every boolean with overhead takes 16 bytes (which is a bit much imho) I do not think this will be the source of the problem.
Replacing these flags with calls into the map will negatively impact performance, so your change will probably make things worse.
Before you go redesigning your code (a command pattern looks like a good candidate) you should look further into where the memory leak is that you are asked to solve.
Look for maps that the classes keep adding to, collections that are static variables etc. Once you find out where the reason for the memory growth lies you can decide which part of your classes to refactor.
You could save memory at the cost of time (but if your memory use is a real problem, then it's probably a nett gain in time) by storing the values in a bitset.
If the class is immutable (once you create it, you never change it) then you can perhaps gain by using a variant on Flyweight pattern. Here you have a store of in-use objects in a weak hashmap, and create your objects in a factory. If you create an object that is identical to an existing object, then your factory returns this previous object instead. The saving in memory can be negliable or massive depending on how many repeated objects there are.
If the class is not immutable, but there is such repetition, you can still use the Flyweight pattern, but you will have to do a sort of copy-on-write where altering an object makes it change from using a shared internal representation to one of its own (or a new one from the flyweight store). This is yet more complicated and yet more expensive in terms of time, but again if its appropriate, the savings can be great.
You can use command pattern:
public enum Command {
SAMPLE_FLAG1("FLAG1") {
public abstract void call( ){
//Do you increment here
}
},
SAMPLE_FLAG2("FLAG2") {
public abstract void call( ){
//Do you increment here
}
};
private Map<String, Command> commands = new HashMap<String, Command>( );
static {
for ( Command cmd : Command.values( )) {
commands.put( cmd.name, cmd);
}
};
private String name;
private Command( String name) {
this.name = name;
}
public Command fromString( String cmd) {
return commands.get( cmd);
}
public abstract void call( );
}
and then:
for( String $ : flagMap.keySet( )) {
Command.fromString( $).call( );
}

Categories