We Keep Coding

Java inter-thread communication and feasibility

First, to shortly describe my problem. Based on the simbad simulator ( http://simbad.sourceforge.net/doc.php - not important for my question ), I want to build a system that deploys rovers which will explore the environment. The idea is that these rovers will avoid obstacles in the environment as well as other rovers. Let's call this a simulation.
The main elements in this simulation are of course the rovers, the environment, and a central station which will control the rovers and also send commands to it. This will run on a thread.
What I would like to have, is on another thread/process, to have a listener. This will listen to commands inputted from the keyboard and translate them into commands that will be applied in my simulation by the central station.
For example, each rover might have an ID, and I might want to remove a remover based on its id. Then I'd like to write something like: remove rover 1, the listener that is running on another thread maps this to a command and for example calls the function centralStation.removeRobot(id_of_robot).
What is the best way of implementing this ? Basically I will have 2 threads, one running the simulation, one listening to commands, and the centralStation should be a shared resource ? How do I make it a shared resource (make a main, initiate the central station, then call the other 2 threads to start doing their job? ) ?
I was wondering what the best practices for this is, and how to make it as simple as possible.
Thank you :)

A simple solution is to simply put an appropriate data structure "between" your components.
For example an instance of ConcurrentLinkedQueue. The idea here: your "input" thread writes "command" objects into that queue; and the other thread looks into that queue, and when it finds a new command, that is "applied" to the simulation.
The important aspect is: you really do not want that two threads are operating on the same data somehow.

Well how about Java Exchanger, where String is the id of rover/command that your listener would transfer to central station
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Exchanger.html
If I am understanding it correct then you want to start the system and at runtime pass the rover id/command, after processing it via a Listener(which would be in a separate thread), to the central station(which would be in a separate thread).
So how I might have proceeded with this would be:
In main thread, start the simulator, Create an Exchanger, and start two threads, one for central station and another for listener.
// pseudocode for main
main() {
// start simulator ( I am not sure what this thing is)
Exchanger<String> exc = new Exchanger<String>();
new CentralStationThread(exc);
new CommandListenerThread(exc);
}
Now in CentralStationThread one of the first thing that you might wanna do is register with the listener
//pseudocode for run method of central station
public void run(){
String roverIdToStop = exc.exchange(new String);
// some code to trigger the rover stop
// send in replacement rover
}
And something similar in CommandListenerThread thread, but not at start
//pseudocode for run method of listener
public void run(){
// Listen to keyboard
// String roverIdOrCommand = Parse the command & make something out out it
// when the command is ready to be sent to central station do following
String roverIdToStop = exc.exchange(roverIdOrCommand);
// keep looking for further commands
}
I agree, There might me several ways to achieve the same but this is what came to my mind. Hope it helps !

Ensuring that threads get (approximately) equal CPU time in Java

I'm writing a game in which players write AI agents that compete against one another, on the JVM. Right now the architecture looks like this:
A core server module that handles the physics simulations, and takes messages from the players as input to alter the world. The core also determines what the world looks like from the perspective of each of the players, based on various rules (think fog of war).
Player modules receive updated versions of the world from the core, process them, and stream messages to the core as inputs based on that processing.
The idea is that the core is compiled along with two player modules, and then the simulation is run producing an output stream that can be played back to generate visualization of the match.
My question is, if each of the players runs on a single Java thread, is it possible to ensure that the two player threads get equal amounts of resources (CPU time, primarily, I think)? Because I don't control the nature of the processing that each AI is doing, it's possible that one of the players might be extremely inefficient but written in such a way that its thread consumes so many resources the other player's AI is resource starved and can't compete fairly.
I get the feeling that this isn't possible without a hard realtime OS, which the JVM isn't even close to being, but if there's even a way to get reasonably close I'd love to explore it.

"Player modules receive updated versions of the world from the core, process them, and stream messages to the
core as inputs based on that processing". This means that player module has a loop inside it which receives update message and sends result messages to the core. Then I would use lightweight actor model, each player being an actor, and all actors use the same ExecutorService. Since activated actors go through the same executor task queue, they got roughly the same access to CPU.

Your intuition is right that this isn't really possible in Java. Even if you had a real-time OS, someone could still write a very resource intensive AI thread.
There are a couple of approaches you could take to at least help here. First be sure to give the two player module threads the same priority. If you are running on a machine that has more than 2 processors, and you set each of the player module threads to have the highest priority, then theoretically they should both run whenever they have something to do. But if there's nothing to stop the player modules from spawning new threads themselves, then you can't guarantee a player won't do that.
So short answer is no, you can't make these guarantees in java.
Depending on how your simulation works, maybe you can have a concept of "turns". So the simulation instructs player 1 to make a move, then player 2 makes its move, and back and forth ,so they can each only make one "move" at a time. Not sure if this will work in your situation though.

If you have any knobs to turn regarding how much work the threads have to do (or just set their priority), you can set up another thread that periodically monitors threads using ThreadMXBeans and find their CPU usage using ThreadInfo.getThreadCpuTime. You can then compare each players CPU time and react accordingly.
Not sure if this is timely and accurate enough for you, but over time you could balance the CPU usage.
However, splitting the work in packets and using Executors like suggested before should be the better way and more java-like.

How to manage the game state in face of the EDT?

I'm developing a real time strategy game clone on the Java platform and I have some conceptional questions about where to put and how to manage the game state. The game uses Swing/Java2D as rendering. In the current development phase, no simulation and no AI is present and only the user is able to change the state of the game (for example, build/demolish a building, add-remove production lines, assemble fleets and equipment). Therefore, the game state manipulation can be performed in the event dispatch thread without any rendering lookup. The game state is also used to display various aggregated information to the user.
However, as I need to introduce simulation (for example, building progress, population changes, fleet movements, manufacturing process, etc.), changing the game state in a Timer and EDT will surely slow down the rendering.
Lets say the simulation/AI operation is performed in every 500ms and I use SwingWorker for the computation of about 250ms in length. How can I ensure, that there is no race condition regarding the game state reads between the simulation and the possible user interaction?
I know that the result of the simulation (which is small amount of data) can be efficiently moved back to the EDT via the SwingUtilities.invokeLater() call.
The game state model seems to be too complex to be infeasible for just using immutable value classes everywhere.
Is there a relatively correct approach to eliminate this read race condition? Perhaps doing a full/partial game state cloning on every timer tick or change the living space of the game state from EDT into some other thread?
Update: (from the comments I gave)
The game operates with 13 AI controlled players, 1 human player and has about 10000 game objects (planets, buildings, equipment, research, etc.). A game object for example has the following attributes:
World (Planets, Players, Fleets, ...)
Planet (location, owner, population, type,
map, buildings, taxation, allocation, ...)
Building (location, enabled, energy, worker, health, ...)
In a scenario, the user builds a new building onto this planet. This is performed in EDT as the map and buildings collection needs to be changed. Parallel to this, a simulation is run on every 500ms to compute the energy allocation to the buildings on all game planets, which needs to traverse the buildings collection for statistics gathering. If the allocation is computed, it is submitted to the EDT and each building's energy field gets assigned.
Only human player interactions have this property, because the results of the AI computation are applied to the structures in EDT anyway.
In general, 75% of the object attributes are static and used only for rendering. The rest of it is changeable either via user interaction or simulation/AI decision. It is also ensured, that no new simulation/AI step is started until the previous one has written back all changes.
My objectives are:
Avoid delaying the user interaction, e.g. user places the building onto the planet and only after 0.5s gets the visual feedback
Avoid blocking the EDT with computation, lock wait, etc.
Avoid concurrency issues with collection traversal and modification, attribute changes
Options:
Fine grained object locking
Immutable collections
Volatile fields
Partial snapshot
All of these have advantages, disadvantages and causes to the model and the game.
Update 2: I'm talking about this game. My clone is here. The screenshots might help to imagine the rendering and data model interactions.
Update 3:
I'll try to give a small code sample for clarify my problem as it seems from the comments it is misunderstood:
List<GameObject> largeListOfGameObjects = ...
List<Building> preFilteredListOfBuildings = ...
// In EDT
public void onAddBuildingClicked() {
Building b = new Building(100 /* kW */);
largeListOfGameObjects.add(b);
preFilteredListOfBuildings.add(b);
}
// In EDT
public void paint(Graphics g) {
int y = 0;
for (Building b : preFilteredListOfBuildings) {
g.drawString(Integer.toString(b.powerAssigned), 0, y);
y += 20;
}
}
// In EDT
public void assignPowerTo(Building b, int amount) {
b.powerAssigned = amount;
}
// In simulation thread
public void distributePower() {
int sum = 0;
for (Building b : preFilteredListOfBuildings) {
sum += b.powerRequired;
}
final int alloc = sum / (preFilteredListOfBuildings.size() + 1);
for (final Building b : preFilteredListOfBuildings) {
SwingUtilities.invokeLater(=> assignPowerTo(b, alloc));
}
}
So the overlapping is between the onAddBuildingClicked() and distributePower(). Now imagine the case where you have 50 of these kind of overlappings between various parts of the game model.

This sounds like it could benefit from a client/server approach:
The player is a client - interactivity and rendering happen on that end. So the player presses a button, the request goes to the server. The reply from the server comes back, and the player's state is updated. At any point between these things happening, the screen can be re-painted, and it reflects the state of the game as the client currently knows it.
The AI is likewise a client - it's the equivalent of a bot.
The simulation is the server. It gets updates from its clients at various times and updates the state of the world, then sends out these updates to everyone as appropriate. Here's where it ties in with your situation: The simulation/AI requires a static world, and many things are happening at once. The server can simply queue up change requests and apply them before sending the updates back to the client(s). So as far as the server's concerned, the game world isn't actually changing in real time, it's changing whenever the server darn well decides it is.
Finally, on the client side, you can prevent the delay between pressing the button and seeing a result by doing some quick approximate calculations and displaying a result (so the immediate need is met) and then displaying the more correct result when the server gets around to talking to you.
Note that this does not actually have to be implemented in a TCP/IP over-the-internet sort of way, just that it helps to think of it in those terms.
Alternately, you can place the responsibility for keeping the data coherent during the simulation on a database, as they're already built with locking and coherency in mind. Something like sqlite could work as part of a non-networked solution.

Not sure I fully understand the behavior you are looking for, but it sounds like you need something like a state change thread/queue so all state changes are handled by a single thread.
Create an api, maybe like SwingUtilities.invokeLater() and/or SwingUtilities.invokeAndWait() for your state change queue to handle your state change requests.
How that is reflected in the gui I think depends on the behavior you are looking for. i.e. Can't withdraw money because current state is $0, or pop back to the user that the account was empty when the withdraw request was processed. (probably not with that terminology ;-) )

The easiest approach is to make the simulation fast enough to run in the EDT. Prefer programs that work!
For the two-thread model, what I suggest is synchronise the domain model with a rendering model. The render model should keep data on what came from the domain model.
For an update: In the simulation thread lock the render model. Traverse the render model updating where things are different from what is expected update the render model. When finished traversing, unlock the render model and schedule a repaint. Note that in this approach you don't need a bazillion listeners.
The render model can have different depths. At one extreme it might be an image and the update operation is just to replace a single reference with the new image object (this wont handle, for instance, resizing or other superficial interaction very well). You might not bother checking whether an item has change and just update eveything.

If changing the game state is fast (once you know what to change it to) you can treat the game state like other Swing models and only change or view the state in the EDT. If changing the game state is not fast, then you can either synchronize state change and do it in swing worker/timer (but not the EDT) or you can do it in separate thread that you treat similarly to the EDT (at which point you look at using a BlockingQueue to handle change requests). The last is more useful if the UI never has to retrieve information from the game state but instead has the rendering changes sent via listeners or observers.

Is it possible to incrementally update the game state and still have a model that is consistent? For example recalculate for a subset of planet/player/fleet objects in between renders/user updates.
If so, you could run incremental updates in the EDT that only calculate a small part of the state before allowing the EDT to process user inputs and render.
Following each incremental update in the EDT you would need to remember how much of the model remains to be updated and schedule a new SwingWorker on the EDT to continue this processing after any pending user inputs and rendering has been performed.
This should allow you to avoid copying or locking the game model while still keeping the user interactions responsive.

I think you shouldn't have World store any data or make changes to any objects itself, it should only be used to maintain a reference to an object and when that object needs to be changed, have the Player making the change change it directly. In this event, the only thing you need to do is synchronize each object in the game world so that when a Player is making a change, no other Player can do so. Here's an example of what I'm thinking:
Player A needs to know about a Planet, so it asks World for that Planet (how is dependent upon your implementation). World returns a reference to the Planet object Player A asked for. Player A decides to make a change, so it does so. Let's say it adds a building. The method to add a building to the Planet is synchronized so only one player can do so at a time. The building will keep track of its own construction time (if any) so the Planet's add building method would be freed up almost immediately. This way multiple players can ask for information on the same planet at the same time without affecting each other and players can add buildings almost simultaneously without much appearance of lag. If two players are looking for a place to put the building (if that is part of your game), then checking the suitability of a location will be a query not a change.
I'm sorry if this doesn't answer you're question, I'm not sure if I understood it correctly.

How about implementing a pipes and filters architecture. Pipes connect filters together and queue requests if the filter is not fast enough. Processing happens inside filters. The first filter is the AI engine while the rendering engine is implemented by a set of subsequent filters.
On every timer tick, the new dynamic world state is computed based on all the inputs (Time is also an input) and a copy inserted into the first pipe.
In the simplest case your rendering engine is implemented as a single filter. It just takes the state snapshots from the input pipe and renders it together with the static state. In a live game, the rendering engine may want to skip states if there are more than one in the pipe while if you're doing a benchmark or outputting a video you'll want to render every one.
The more filters you can decompose your rendering engine into, the better the parallelism will be. Maybe it is even possible to decompose the AI engine, e.g. you may want to separate dynamic state into fast changing and slow changing state.
This architecture gives you good parallelism without a lot of synchronization.
A problem with this architecture is that garbage collection is going to run frequently freezing all the threads every time, possible killing any advantage gained from multi-threading.

It looks like you need a priorityqueue to put the updates to the model on, in which updates frmo the user have priority over the updates from the simulation and other inputs. What I hear you saying is that the user always needs immediate feedback over his actions wheras the other inputs (simulation, otherwise) could have workers that may take longer than one simulation step.
Then synchronize on the priorityqueue.

Limiting Thread Execution Processor Cycles in Java

I'm writing an AI-testing Framework for a competition. Participants submit a Bot class which matches a given Interface. Then all the bots play a turn-based game. On every turn, I want to do the following:
For every bot B:
start a thread that runs at most N cycles and does B.getNextMove()
wait for all threads to complete
Make all moves (from each bot).
My difficulty comes in saying "at most N cycles". I can limit all the bots by time (say half a second per turn) but this means that some can get more processor cycles than others and doesn't permit a strict "your bot should be able to make its decision re: a turn in X time" requirement in the competition.
As stated, this is in Java. Any ideas? I've been looking at concurrency and locking but this doesn't feel like the right direction. Also, it's possible to not run the bots in Parralel and then use time for the restriction (given that the computer isn't running anything else at the time), but this would be undesirable as it would significantly slow down the speed at which we could have results from the games.

I'd make an interface with the bot to have them do 1 iteration of their algorithm,and do a simple count.
If you need hard time/cpu limits there arn't that many(easy) ways to manage that in java.
You can't measure cpu cycles with java, but you can measure CPU time - which is a vast improvement over using just wall clock time.
To get the cpu time of the current thread you'd use (from the standard java.lang.management package)
ThreadMXBean tm = ManagementFactory.getThreadMXBean();
long cpuTime = tm.getCurrentThreadCpuTime();

Since you're controlling the bot execution and explicitly calling next yourself, why not just count the iterations? e.g.
public class Botcaller extends Thread
{
private Bot bot;
int cycles_completed;
public static final int MAX_ALLOWED_CYCLES=...;
public void run()
{
while (cycles_completed <MAX_ALLOWED_CYCLES)
{
bot.move;
cycles_completed++;
yield()
}
}
}

I think it will be very difficult to control threads at this low a level from Java. I no of know way to stop a thread safely that is running.
Another option is to let each bot run in whatever time it wants. Have one thread per bot but poll the location in a master thread every second. If they haven't updated location in that "round" then they miss out and have to wait for the next one. Make sure that the world state that each bot sees is the one controlled by the master thread.
The nice thing about this approach is that it lets bots sacrifice rounds if they want to do something more complicated.

I think Robocode does something similar... you may want to look there.

Calling function when program exits in java

I would like to save the programs settings every time the user exits the program. So I need a way to call a function when the user quits the program. How do I do that?
I am using Java 1.5.

You can add a shutdown hook to your application by doing the following:
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
public void run() {
// what you want to do
}
}));
This is basically equivalent to having a try {} finally {} block around your entire program, and basically encompasses what's in the finally block.
Please note the caveats though!

Adding a shutdown hook addShutdownHook(java.lang.Thread) is probably what you look for. There are problems with that approach, though:
you will lose the changes if the program aborts in an uncontrolled way (i.e. if it is killed)
you will lose the changes if there are errors (permission denied, disk full, network errors)
So it might be better to save settings immediately (possibly in an extra thread, to avoid waiting times).

Are you creating a stand alone GUI app (i.e. Swing)?
If so, you should consider how you are providing options to your users how to exit the application.
Namely, if there is going to be a File menu, I would expect that there will be an "Exit" menu item.
Also, if the user closes the last window in the app, I would also expect it to exit the application.
In both cases, it should call code that handles saving the user's preferences.

Using Runtime.getRuntime().addShutdownHook() is certainly a way to do this - but if you are writing Swing applications, I strongly recommend that you take a look at JSR 296 (Swing Application Framework)
Here's a good article on the basics: http://java.sun.com/developer/technicalArticles/javase/swingappfr/.
The JSR reference implementation provides the kind of features that you are looking for at a higher level of abstraction than adding shutdown hooks.
Here is the reference implementation: https://appframework.dev.java.net/