I have a Java project where I am required to both send and receive network packets with only the Ethernet header present. The header contains custom MAC addresses as well, which do not match the physical hardware address of the receiving/sending interface.
I have selected jNetPcap 1.3 to accomplish this task, but I am concerned about the thread safety of my Java application and am in need of some help with the particularities of libpcap.
I have two threads, where the first thread executes a
org.jnetpcap.Pcap.loop(Pcap.LOOP_INFINITE, handler, outputQueue)
loop to capture packets from a previously opened org.jnetpcap.Pcap Object (representing a pcap_t) passed to the thread by the caller.
The second thread is supposed to pick payload/header pairs from an input queue and send them using
org.jnetpcap.Pcap.sendPacket(packetByteBuffer)
using the SAME org.jnetpcap.Pcap Object as the thread executing the reception loop.
Problem:
From using google I concluded that this approach is not working because libpcap is not threadsafe when accessing the same pcap_t object from different threads.
Theoretical Solution:
I think the solution to my problem is to create two separate instances of org.jnetpcap.Pcap, open them separately using org.jnetpcap.Pcap.openLive() and passing one instance to the transmission thread and one to the reception thread.
Before I run off changing a lot of code, I hope someone can confirm that this is indeed the right approach to solving this problem.
Thanks in advance for your answers.
You must in some way synchronize access between the threads, e.g. you could use
Pcap.breakloop() and break the loop that receives packages to send some and continue the loop afterwards.
Pcap.dispatch() and a short timeout for Pcap.openLive() and switch between queued packages to be send and receiving packages.
From the jNetPcap Documentation: It is however safe to interact with various Pcap objects from multiple threads, as long as access is externally synchronized.
Related
I have a problem caused by multi-threading and Android Open Accessory.
I need to communicate with a USB Accessory, but I need to do it from 2 threads. One thread generates and sends data the other one reads data.
Why I don't use a single thread? Because there can be 1 or more writes before a read and reads are blocking, so that is not an option.
If using multiple threads, I do run into "I/O Error (No such device)" sooner or later, because I will have a collision between read & write being executed at the same time.
Locking will more or less put me back in single-thread situation, so not good.
.available() method on the input-stream returns is not supported, so I cannot check if anything is available before doing a read
Since it's not a socket-based stream I cannot set timeout either.
I have tried getting the FileDescriptor from the USBAccessory and passing to JNI to handle it there, but after the first read/write the device becomes inaccessible.
Question/Suggestion needed:
What will be a suggested/best-practice approach to this? I do not expect written code, I just need some guidance on how to approach this problem.
To clarify:
The software at the other end might or might NOT respond with any data. There are some so called silent sends were the data sent it's just received but there is no ACK. Since the app I'm working on is only a proxy, I do not have a clear picture if the data will or will not produce an answer. That will require analysis of the data as well, which isn't on the books at the moment.
As you want to do read and write in parallel, writing will always lead to a pause to read if the read is on the same part as write.
May be you can follow similar approach as ConcurrentHashMap and use different locks for different segments and lock read only if write is on the same segment else allow the read to happen.
This will
Avoid blocking read during write in most scenarios
Avoid collision and
Definitely wont be a single thread approach.
Hope that helps.
If using multiple threads, I do run into I/O Error (No such device)
sooner or later, because I will have a collision between read & write
being executed at the same time.
This says it all. Since you are doing read and write on the same channel that does not support concurrent access, you are required to have your thread wait until the other thread is done doing read/write.
Your two-thread approach is what I would do, more or less. Good luck and trust in yourself.
I have a standard client/server setup.
The program I'd like to build acts a lot like a mail office(which is my Server). Multiple people (client with ObjectOutputStream) hand the office (server with the single ObjectInputStream) mail with an attached address and the office sends the mail where it is supposed to go. If possible, I'd like to have one ObjectInputStream in the server that blocks, waiting for "mail" to come in from any ObjectOutputStream, then sends the "mail" where it's supposed to go. This way I can just have one thread that is completely dedicated to receiving data and sending it.
I will have a thread for each person's client with their ObjectOutputStream, but would like to not also need a matching thread in the server to communicate with each person. I am interested in this idea because I find it excessive to build tons of threads to separately handle connections, when it's possible that a single thread will only send data once in my case.
Is this feasible? or just silly?
Use a JMS queue of Java Message Service, is the design pattern for this case.
http://en.wikipedia.org/wiki/Java_Message_Service
If you have in the server app just one instance of ObjectInputStream and you have many clients then this instance needs to be shared by all threads thus you need to synchronize the access to it.
You can read more here. Hope this helps.
OR
You can have a pool of ObjectInputStream instances and using a assignment algorithm like Round Robin (doc) you can return the same instance for each x order thread for example ... this will make the flow in the server app to be more paralleled
Your question doesn't make sense. You need a separate pair of ObjectInputStream and ObjectOutputStream per Socket. You also need a Thread per Socket, unless you are prepared to put up with the manifest limitations of polling via InputStream.available(), which won't prevent your reads from blocking. If you are using Object Serialization you are already committed to blocking I/O and therefore to a thread per Socket.
I am working on creating a chat client based on UDP. The main structure is that there is a server where clients register and where clients can also request to form a connection with another client that is registered with the server. The clients are structures as follows using pseudo code:
public UDPClient() {
// Create datagram socket
// Execute RECEIVE thread using datagram socket above
// Execute SEND thread using datagram socket above
}
The idea is to have the send and receive executing on separate threads so I don't get blocked I/O on the receive. Both of these threads have loops within their run methods that allow you to continually send and receive messages. The problem I have is this. If a message comes in on the RECEIVE thread that changes how my SEND should be executing, how do I communicate this to the SEND thread? Do I have to shoot a datagram off to myself or can I communicate this in the code somehow?
Assuming boths threads have no reference to each other, create a third singleton class, which both read/send threads (classes) reference, that has a volatile member field to store the state data you want shared and which has synchronized access.
The volatile keyword, combined with synchronized access, guarantees that a change made to the field by one thread will be seen by another thread. Without this, changes may not be visible due to the java memory model specification.
Edited:
Following "separation of concerns" design guideline, it would be better to not have the read/send threads know about each other and to use a third class to orchestrate their activities/behaviour. Add methods to your read/send classes to stop(), start() etc and call these from the other class.
Using a separate class would also allow:
Behaviour control by other means, for example a "stop sending" button on an admin web page
Allowing multiple threads of each type, yet still having proper control through a central point, perhaps using a pool of such threads (without a separate class, you would have a many-to-many nightmare and lots of code that has nothing to do with the job at hand: ie ending and receiving)
Easier testing of your worker classes, because they do less and are more focused
porting/embedding them stand-alone for other uses
your SEND thread should have public (accesible) method (synchronized if possible) that you should be able to access from your RECEIVE thread. You could use this method to create a boolean flag, string message, etc. that you should always read before you .send(yourPacket); from your SEND thread.
Have a member variable in your READ method that your code can read from and change the SEND method based on that variable.
is there any way two threads within the same process can communicate without knowing anything about each other's interface ?
basically, one thread is a STOMP server, the other is a client. they're supposed to communicate in a direct manner (not via a socket) and it should be independent of the implementation so i can't assume either of the threads is waiting for messages on some common message queue.
what i'm looking for is some kind of a built-in mechanism in java that allows threads within the same process to communicate.
is there such a mechanism ? and if not, is there any other way to approach this ?
You can use a concurrent message queue where threads can post and receive messages. Instead of knowing the other's thread interface, now each thread must be able to create own messages and understand the messages of other threads.
By using a distinct interface for these messages, this is rather easy. And as a bonus, there is a wide range of queues for concurrent access available, so you can pick the queue that fits most to your scenario.
Well, you can have a third thread to act as a message board. But then you'd have to hope that the two threads can agree on a protocol before hand. It would also be rather slow.
Can you provide more details/examples? What do you mean by "communicate" exactly?
There are a few ways I can think of for doing this, shared (global) state, PipedInputStream/PipedOutputStream etc. But the details will depend on what you're trying to do.
I'm writing a Java application that will instantiate objects of a class to represent clients that have connected and registered with an external system on the other side of my application.
Each client object has two nested classes within it, representing front-end and back-end. the front-end class will continuously receive data from the actual client, and send indications and data to the back-end class, which will take that data from the front-end and send it to the external system in using the proper format and protocol that system requires.
In the design, we're looking to have each instantiation of a client object be a thread. Then, within each thread will naturally be two sockets [EDIT]with their own NIO channels each[/EDIT], one client-side, one system-side residing in the front- and back-end respectively. However, this now introduces the need for nonblocking sockets. I have been reading the tutorial here that explains how to safely use a Selector in your main thread for handling all threads with connections.
But, what I need are multiple selectors--each operating in their own thread. From reading the aforementioned tutorial, I've learned that the key sets in a Selector are not threadsafe. Does this mean that separate Selectors instantiated in their own repsective threads may create conflicting keys if I try to give them each their own pair of sockets and channels? Moving the selector up to the main thread is a slight possibility, but far from ideal based on the software requirements I've been given. Thank you for your help.
Using multiple selectors would be fine as long as you do not register the same channel with the same interests (OP_READ / OP_WRITE etc) with both the selector instances. Registering the same channel with multiple selector instances could cause a problem where selector1.select() could consume an event that selector2.select() could be interested in.
The default selectors on most of the platforms are poll() [or epoll()] based.
Selector.select internally calls the int poll( ListPointer, Nfdsmsgs, Timeout) method.
where the ListPointer structure can then be initialized as follows:
list.fds[0].fd = file_descriptorA;
list.fds[0].events = requested_events;
list.msgs[0].msgid = message_id;
list.msgs[0].events = requested_events;
That said, I would recommend the usage of a single selecting thread as mentioned in the ROX RPC nio tutorial. NIO implementations are platform dependant, and it is quite possible that what works on one platform may not work on another. I have seen problems across minor versions too.
For instance, AIX JDK 1.6 SR2 used a poll() based selector - PollSelectorImpl and the corresponding selector provider as PollSelectorProvider, our server ran fine. When I moved to AIX JDK 1.6 SR5, which used a pollset interface based optimized selector (PollSetSelectorImpl), we encountered frequent hangs in our server in the select() and socketchannel.close(). One reason I see is that we open multiple selectors in our application (as opposed to the ideal one Selecting Thread model) and the implementation of the PollSetSelectorImpl as described here.
If you have to use this single socket connection, you have to separate the process of receiving and writing data from and to the channel from the data processing itself. You do not must delegate the channel. The channel is like a bus. The bus (the single thread, that manages the channel) has to read the data and to write it into a (thread-safe) input queue including the information required, so your client thread(s) can pick up the correct datagram package from the queue. If the client thread likes to write data, that data is written to an output queue which is then read by the channels thread to write the data to the channel.
So from a concept of sharing a connection between actors using this connection with their unpredictable processing time (which is the main reason for blocks), you move to a concept of asynchronous data read, data processing and data writing. So, it's not the processing time which is unpredictable anymore, but the time, your data is read or written. Non-blocking means, that the stream of data is as constant as possible, despite what time is required to process that data.