So I am new to Java, I have done a bit of c programming.
I am trying to make a virtual network of nodes, each node would need to be a thread.
The nodes are only allowed to talk to their neighbor nodes.
there will be a master node that can talk to any node but the nodes would have to talk to each other to get back to the master node. the master nodes neighbors can talk to the master node.
I was originally going to keep an array list of the nodes, but then I realized all the nodes needed to be there own thread.
My question is how do I pass information back in forward between threads in Java.
I need to be able to have the master node give all the regular nodes position information.
and I need the regular nodes to be able to pass messages to their neighbor regular nodes.?
here are my git repos if you would like to look at the code I got going now.
https://github.com/fieldju/cs372_project
in C I made a program that used pipes for the children to talk to each other and a server connected the clients, but in this problem the nodes to to have p2p communication as most of them can not directly communicate to the master node / server
Just an Update for anyone who looks at this and wants to see the results. I got the nodes up and running and communicating you can check out the code at
https://github.com/fieldju/cs372_project
I am still working on the distance vector stuff and a few other things but by the end of next week the entire thing should be done.
I was originally going to keep an
array list of the nodes, but then I
realized all the nodes needed to be
there own thread.
You can keep an array of threads, it would still maintain a thread-per-node with the same logic structure.
how do I pass information back in
forward between threads in Java.
If threads reside in the same process then definitely sockets are an overkill. I would use one or several ConcurrentLinkedQueue instances to push/pop messages.
One or several really depends on the kind of communication that you are implementing. Maybe one ConcurrentLinkedQueue per node, so nodes push messages to queues and every node knows where from to pop the message.
Few hints for implementation
Wrap up all the logic to en-route messages in a class - let's call this class VirtualNetwork. VirtualNetwork deals with all the instances of ConcurrentLinkedQueue and offers an API of methods to all threads for sending/receiving messages. Make one instance of the class VirtualNetwork accessible to all nodes - by passing a reference to it on the Thread constructor.
This is an sketch of how your class NodeThread would be. Notice that the classes VirtualNetwork and Message are classes that you have to implement your self.
class NodeThread extends Thread {
private int nodeId;
private VirtualNetwork network;
public NodeThread(int nodeId,VirtualNetwork network) {
this.network = network;
this.nodeId = nodeId;
}
public void run() {
/* when you have to send */
int nodeReceptor = this.nodeId -1; /* neighbor in the array of threads */
Message m = new Message(this.nodeId,nodeReceptor);
m.setContent(10);
network.send(m);
/* when you have to receive */
Message m = network.receive(this.nodeId);
/* it's your decision to implement this in a blocking way or not */
}
}
An easy (if not optimal) way to start is to create a BlockingQueue for each pair of threads that need to pass values in one direction (if it's bidirectional you need twice as many.)
You could definitely use Sockets. There are a couple of tutorials/descriptions here and here. Your description of the project suggests a client/server setup would work well since you have a central "master node". The master will be the server and the other threads will be able to connect to the master/server to receive their updates.
Related
I've got an assignment to find out the all possible initiator nodes for a state recording algorithm in the distributed system.
The question that has been given exactly is
"Write a program to find out the all possible initiator nodes for a state recording algorithm in a distributed system.".
I want to mentioned that we have studied Chandy - Lamport's global state recording algorithm on our course of distributed operating system. I wrote a code for Chandy - Lamport's global state recording algorithm for the another assignment.
What does this initiator node signifies? I thought that those nodes who have recorded their corresponding states. Am I right? I've to write the code in java. Please suggest me the approach or an algorithm to follow.
According to the Wikipedia page on the Chandy-Lamport algorithm:
The assumptions of the algorithm are as follows:
There are no failures and all messages arrive intact and only once
The communication channels are unidirectional and FIFO ordered
There is a communication path between any two processes in the system
Any process may initiate the snapshot algorithm
The snapshot algorithm does not interfere with the normal execution of the processes
Each process in the system records its local state and the state of its incoming channels
The algorithm works using marker messages. Each process that wants to
initiate a snapshot records its local state and sends a marker on each
of its outgoing channels. All the other processes, upon receiving a
marker, record their local state, the state of the channel from which
the marker just came as empty, and send marker messages on all of
their outgoing channels. If a process receives a marker after having
recorded its local state, it records the state of the incoming channel
from which the marker came as carrying all the messages received since
it first recorded its local state.
You are using slightly different terminology to the Wikipedia description, but I assume that your "nodes" correspond to the "processes" in the above. Thus an "initiator node" is a simply a node that initiates (requests) a snapshot.
If that is what your terminology means, then with the Chandy-Lamport algorithm, any node could be an initiator node. Hence the answer to the question is "all of them".
But, given the trivial nature of the answer / solution, I suspect that is not what your assignment really means. Either you have left out some context, or the assignment is misstated. I suggest that you ask your instructor.
(Or ... maybe it is a "trick question".)
I have a project which I watch several nodes in several different threads. Now, I noticed that when I watch a node, and it changed, and an event is raised, the watch on a certain node (called A for example) blocks all the other watchers. So only after the watcher on A is finished , the other watcher will return to watch the nodes changes. Meaning, if a node is changed (called B for example) while its watcher is blocked, only after the watcher on A is finished, the watcher on node B will raise the event.
This problem causes the application be slower.
So, in order to fix this, I wanted to use a different client connnection for each thread, (using curator), but I have read that one connection should be enough, and if I need more than one, there is something worng with my implementation.
1) I dont understand what is the problem with multiple connection to zookeeper server
2) Is there another solution for my problem?
Edit - more specific about the problem
I have a master which gets requests from clients ( and each clients can save files on my server and we do some process on this file, it is more complex than it sounds, i wont elaborate it), and the master creates a node in /tasks/ for a worker to process the file (without its data of course, the data is in a db). When the worker watches his node, he processes the file, and when he finishes, he creates a node in /status (which has all the files that their process were finished).
The master watches the node /status , and when something was changed , it gets the children, and creates a thread (in order to make all faster, because zookeeper watchers and callbacks are single threaded ), which will release those files (remove some meta from db, return a response to client, remove some variable etc.).
That is one of the main flows, but I have another important parts of the code which listen on nodes, and process their children when there are changes.
Because this thing is in a thread, I created a list of the nodes that were finished already, so I wont do the final process more than once, but it was more complex than that, and that solution caused other problems, some concurrency bugs.
So as i asked
1) What is the problem with multiple connection , for each important flow, so i wont have to create threades inside watches and callback?
2) Is there another solution i can use here?
It's not well documented, but ZooKeeper has a single thread for handling watchers and async callbacks. We wrote a tech note for Curator about it. https://cwiki.apache.org/confluence/display/CURATOR/TN1
I'm developing a VOIP server on UDP using Netty. When a call is placed, I store a "call" object on a global list of calls, like this:
public final List<Call> calls = new ArrayList<Call>();
Every time i receive a response from a Call, i have to iterate through the list to find the right "call" then, use this call object to make decisions, maybe route the call to another place, etc.
My currently lazy solution to this threading problem is to use the synchronized key word over the whole list, every time I need to access the list OR the individual "call" objects. I know this is terrible but it's OK for a POC.
Now I need the do the right way. To the access the list, using a ConcurrentHashMap seems to be a good option, but my question is:
What is the proper way to lock and access individual "call" objects ?
I can have up to 4k simultaneous calls (500 packets/sec), could it be a problem to lock a lot of objects? What is the best solution to this?
Thanks in advance!
Do you need to lock the call objects from the outside? I am assuming that the call object is essentially 2 FIFO queues - one processing audio packets from caller1 to caller2, the second processing packets from caller2 to caller1. You can have methods that lock these queues internally (and individually) as they add/remove the "next" packet. Other than that, it seems like the data in the call object would be rather static through the life of the call. This also allows concurrent data processing for each direction of the call since any operation on a packet in one direction won't impact the other direction.
I have created a number of threads. I know each threads name(suppose through an alien mechanism I set name of thread.) Now I am inside a thread and want to send a message to another thread.
I am trying to code a simulator of Pastry and Chord protocol. I can not have a number of distributed nodes, so I have created a number of threads. Now I want each thread send and receive messages from one another. I have set each nodes name as its IP(a randomly generated number). Now I do not know how to send a message from one node to another. Please tell me how to send a message from one thread to another if you know another threads name.
I would suggest some kind of a message system. The easiest way would be to create a thread-safe FIFO and pass it into each thread. If you want to send messages directly to each different thread, make a "Topic" for each thread.
Don't try to hack something in using the thread name, it'll just constrain you later.
Pasted from comment so I can parse it:
private static BlockingQueue[] queue;
private static int queueNum = 0;
public static void newQueue(String ip)
{
queue[queueNum] = new ArrayBlockingQueue(1024);
try{ queue[queueNum].put(ip); }
catch (InterruptedException e){e.printStackTrace(); }
queueNum++;
}
Oh, I see your problem. You never assign BlockingQueue a value. Try changing that line to:
private static BlockingQueue[] queue=new BlockingQueue[10];
That will allow you 10 queues.
I'd also suggest that instead of an array you use a HashMap so you can name, add and delete queues at will. Instead of being queue[2] you'll be addressing queue.get("Thread1Queue") or something more descriptive.
Note response to comments:
A HashMap can generally replace an array, it's lookup is nearly as quick but it uses anything for an index instead of numbers--Strings, enums, Objects, whatever you want (as long as it has the hash and equals methods overriden), but usually strings.
So if you are storing a bunch of queues, and you want to name them specifically, you can say:
HashMap queues=new HashMap();
queues.put("queue1", new ArrayBlockingQueue(1024));
queues.put("queue2",new ArrayBlockingQueue(1024));
...
Then whenever you want to access one you can use:
queues.get("queue1").put(new ThingToAddToArrayBlockingQueue())...
to put a "Thing to add" to queue1.
If you just want a "Bunch" of them and don't need to know which is which (Just a collection of threads that can be fed generic taskss) there are specific collections/patterns for that in the concurrent package.
The usual way to communicate between threads is by passing an object to each thread which then allows to communicate between them. Keep in mind that all fields and methods of that object which are accessed by more than one thread should be synchronized.
But when you want to simulate a network protocol, then why not go all the way and use network sockets for interprocess communication? Just make each thread listen to a different socket on 127.0.0.1.
If you want to send messages and then have them processed by other threads you need a shared object (queue, map etc.) into which threads can pump messages. Receiving threads must check for incoming messages, pull them and do the necessary processing.
I'm programming mobile ad hoc network routing protocol in JAVA (using UDP). That routing protocol consists of ring topology (each node as one predecessor node and one successor node).
First, I've combined one transmitter (one thread) and one receiver (one thread) to form one node. But, I'm facing some problems like:
I'd that a third node could listen transmission from one node to another node. Per example,
node A sends a packet to node B, and if node C is in the range of node A then it might listen that transmission too.
I'd set one channel per ring to reduce interference. But, I don't know which java network API mechanism I should use.
I'd have your guidance.
Thank you in advance (sorry for my poor english)!
Per example, node A sends a packet to node B, and if node C is in the range of node A then it might listen that transmission too.
This is expected behavior for wireless ad-hoc network. If C is not destination (according to MAC-address) you can drop received message.
I'd set one channel per ring to reduce interference.
One channel per ring would oppositely increase interference, especially if you expect high load and many messages being routed around. But it is much easier to manage single channel.
You need to think more what is your environment and requirements.
Are you using 802.11 at MAC level?
Do you want reliable guaranteed delivery?