I am having trouble developing a chat application including file sharing. It's a server and client based desktop application. I am able to develop chat and also file sharing individually.
Now I want merge two project and having problem. Is it possible to use multiple socket in single thread like main thread? Or do I have to use one for chatting and another for file sharing?
I think that is possible but in the same thread one operation with one socket have to wait all the operations from annother finish. if you want the two operations execute at the same time do it in different threads.
Hope that helps :)
Yes it is possible for a single thread to use multiple sockets. See the Selector API.
The basic idea is that the code in your thread needs to know about all of the selectable channels (sockets, open files, etc) that it needs to read from or write to. It registers the channels with the selector. Then you make a select call which will (for example) block until one of the registered channels is ready; e.g. a socket that has data that can be read now ... without blocking.
It is complicated, but rather than explaining it all here, I suggest that you read Baeldung's Introduction to the Java NIO Selector which includes an example.
But the bottom line is that using Selector will probably entail significant restructuring of your existing code. And if your code is using (for example) a 3rd-party library to do the I/O, then this approach probably won't work. Using multiple threads may well be a simpler option.
Related
I have the following situation:
I have 2 JVM processes (really 2 java processes running separately, not 2 threads) running on a local machine. Let's call them ProcessA an ProcessB.
I want them to communicate (exchange data) with one another (e.g. ProcessA sends a message to ProcessB to do something).
Now, I work around this issue by writing a temporary file and these process periodically scan this file to get message. I think this solution is not so good.
What would be a better alternative to achieve what I want?
Multiple options for IPC:
Socket-Based (Bare-Bones) Networking
not necessarily hard, but:
might be verbose for not much,
might offer more surface for bugs, as you write more code.
you could rely on existing frameworks, like Netty
RMI
Technically, that's also network communication, but that's transparent for you.
Fully-fledged Message Passing Architectures
usually built on either RMI or network communications as well, but with support for complicated conversations and workflows
might be too heavy-weight for something simple
frameworks like ActiveMQ or JBoss Messaging
Java Management Extensions (JMX)
more meant for JVM management and monitoring, but could help to implement what you want if you mostly want to have one process query another for data, or send it some request for an action, if they aren't too complex
also works over RMI (amongst other possible protocols)
not so simple to wrap your head around at first, but actually rather simple to use
File-sharing / File-locking
that's what you're doing right now
it's doable, but comes with a lot of problems to handle
Signals
You can simply send signals to your other project
However, it's fairly limited and requires you to implement a translation layer (it is doable, though, but a rather crazy idea to toy with than anything serious.
Without more details, a bare-bone network-based IPC approach seems the best, as it's the:
most extensible (in terms of adding new features and workflows to your
most lightweight (in terms of memory footprint for your app)
most simple (in terms of design)
most educative (in terms of learning how to implement IPC). (as you mentioned "socket is hard" in a comment, and it really is not and should be something you work on)
That being said, based on your example (simply requesting the other process to do an action), JMX could also be good enough for you.
I've added a library on github called Mappedbus (http://github.com/caplogic/mappedbus) which enable two (or many more) Java processes/JVMs to communicate by exchanging messages. The library uses a memory mapped file and makes use of fetch-and-add and volatile read/writes to synchronize the different readers and writers. I've measured the throughput between two processes using this library to 40 million messages/s with an average latency of 25 ns for reading/writing a single message.
What you are looking for is inter-process communication. Java provides a simple IPC framework in the form of Java RMI API. There are several other mechanisms for inter-process communication such as pipes, sockets, message queues (these are all concepts, obviously, so there are frameworks that implement these).
I think in your case Java RMI or a simple custom socket implementation should suffice.
Sockets with DataInput(Output)Stream, to send java objects back and forth. This is easier than using disk file, and much easier than Netty.
I tend to use jGroup to form local clusters between processes. It works for nodes (aka processes) on the same machine, within the same JVM or even across different servers.
Once you understand the basics it is easy working with it and having the options to actually run two or more processes in the same JVM makes it easy to test those processes easily.
The overhead and latency is minimal if both are on the same machine (usually only a TCP rountrip of about >100ns per action).
socket may be a better choice, I think.
Back in 2004 I implement code which do the job with sockets. Until then, many times I search for a better solution, because socket approach triggers firewall and my clients worry. There is no better solution until now. Client must serialize your data, send and server must receive and unserialize.
It is easy.
I'm often need to organize something like critical section where multiple thread might access simultaneously but not more than X threads with the same key.
Simplified example:
thread pool to process the device commands from queue;
queue contains commands for different devices;
while one thread process device command other threads that process commands for the same device should wait their turn.
I know this could be achieved in different ways and you could say that there are architectural refactoring required and so on but this is the simplest example I could think.
Edit: I don't need the solution for this particular example, I need some library which allows to restrict access based on some key for example device IP/MAC string. For example: getLock().lock(deviceIP)
I have my own code to do this but I wounder did I miss something? Is there some standard Java SE 8 components or third-party libraries to do this?
I'm used to C++/Qt's concept of signals (emit/listen) and now I'm doing a project in Java which requires some sort of data sending/receiving mechanism.
My needs are:
Emit an event (with some data) and let all threads listen/catch it.
Obviously, given the previous requirement, being able to listen/catch signals with attached data.
Is this possible in Java, and how? (I'll appreciate a small compilable example/link)
Java by default doesn't have a simple event handling mechanism such as .Net's events or Qt's Signals and Slots. It does have the notion of Listeners in various java GUI frameworks but I don't think that's what you're looking for.
You should consider a pub-sub library like Google Guava's EventBus framework.
If you don't want to use a third party lib then I suggest you start looking into using one of the sub-classes of BlockingQueue. See the FileCrawler example from page 62 of Java Concurrency in Practice to see how to use a BlockingQueue to send events/data to worker threads.
If you're looking for a more complicated solution for message/event notifications across the process boundary or the local machine boundary then you may want to look into:
RabbitMq
Redis
JMS
not sure if this will match your exact query but have you tried CountDownLatch or CyclicBarrier?
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CountDownLatch.html
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CyclicBarrier.html
I'm building a GUI application which needs to have a TCP Server running. The GUI is built in Swing. The trouble I'm having is running the server. I want to have one desktop application people can install, that will in turn start the server for me. The server is blocking while it runs ie:
while (true) {
Socket client = serverSocket.accept();
System.out.println("S: Receiving...");
....
I tried to call the server class, but then it blocks the GUI. What do you think is the best way to separate the Server from the GUI, while easily being able to package both together? Thanks for any help!
There are two basic approaches to socket programming in Java:
Use one thread per Socket
Use Non-Blocking IO in one or a small number of threads
You probably want the first one. If you don't know which one you want, then you almost certainly want the first one. This is subjective, but I think most people would agree #1 is easier to get correct.
Here's an answer that discusses the difference.
Assuming you decide to go with threads, use a thread pool! It's easy and tidy.
I think you're past this stage, but if you want some basic "101" type material, the old Sun tutorial on Sockets and Threads is dusty but useful.
Threads.
Note that once you introduce multi-threaded programming, there are a lot of gotchas. One important one is that access to any Swing or GUI element must be done in the Swing Event Thread. You can use SwingWorkers to make that task easier.
You can also use Java NIO.
The easiest solution is generally to run your ServerSocket in a different thread. You may also try things like asynchronous IO, but it's a real pain.
Well, from your description, perhaps the problem is that you need to put the GUI and the server part (with that while(true) ), into separate Threads, so that you can handle user generated events.
Simplest thing is building your server in separate thread.
I can create multiple threads for supporting multi-client feature in socket programming; that's working fine. But if 10,000 clients want to be connected, my server cannot create so many threads.
How can I manage the threads so that I can listen to all these clients simultaneously?
Also, if in this case the server wants to send something to a particular client, then how is it possible?
You should investigate Java's NIO ("New I/O") library for non-blocking network programming. NIO was designed to solve precisely the server scalability problem you are facing!
Introductory article about NIO: Building Highly Scalable Servers with Java NIO
Excerpts from O'Reilly's Java NIO book
Highly scalable socket programming in Java requires the selectable channels provided in the "New I/O", or NIO packages. By using non-blocking IO, a single thread can service many sockets, tending only to those sockets that are ready.
One of the more scalable open-source NIO applications is the Grizzly component of the Glassfish application server. Jean-Francois Arcand has written a number of informative, in-depth blog posts about his work on the project, and covers many subtle pitfalls in writing this kind of software with NIO.
If the concept of non-blocking IO is new to you, using existing software like Grizzly, or at least using it as a starting point for your adaptation, might be very helpful.
The benefits of NIO are debatable. See Paul Tyma's blog entries here and here.
A thread-per-connection threading model (Blocking Socket I/O) will not scale too well. Here's an introduction to Java NIO which will allow you to use non-blocking socket calls in java:
http://today.java.net/cs/user/print/a/350
As the article states, there are plenty of frameworks available so you don't have to roll your own.
As previously mentioned, 10.000 clients is not easy. For java, NIO (possibly augmented with a separate threadpool to handle each request without blocking the NIO thread) is usual way to handle a large amount of clients.
As mentioned, depending on implementation, threads might actually scale, but it depends a lot on how much interaction there is between client connections. Massive threads are more likely to work if there is little synchronization between the threads.
That said, NIO is notoriously difficult to get 100% right the first time you implement it.
I'd recommend either trying out, or at least looking at the source for the Naga NIO lib at naga.googlecode.com. The codebase for the lib is small compared to most other NIO frameworks. You should be able to quickly implement a test to see if you can get 10.000 clients up and running.
(The Naga source also happens to be free to modify or copy without attributing the original author)
This is not a simple question, but for a very in depth (sorry, not in java though) answer see this: http://www.kegel.com/c10k.html
EDIT
Even with nio, this is still a difficult problem. 10000 connections is a tremendous resource burden on the machine, even if you are using non-blocking sockets. This is why large web sites have server farms and load balancers.
Why don't you process only a certain amount of requests at a time.
Let's say you want to process a maximum of 50 requests at a time (for not creating too many threads)
You create a threadpool of 50 threads.
You put all the requests in a Queue (accept connections, keep sockets open), and each thread, when it is done, gets the next request then process it.
This should scale more easily.
Also, if the need arise, it will be easier to do load balancing, since you could share your queues for multiple servers
Personally I would rather use create a custom I/O non blocking setup, for example using one thread to accept clients and using one other thread to process them (checking if any input is available and writing data to the output if necessary).
You'll have to figure out why your application is failing at 10,000 threads.
Is there a hard limit to the number of threads in the JVM or the OS? If so, can it be lifted?
Are you running out of memory? Try configuring a smaller stack size per thread, and/or add more memory to the server.
Something else? Fix it.
Only once you have determined the source of the problem will you be able to fix it. In theory 10,000 threads should be OK but at that level of concurrency it requires some extra tuning of the JVM and operating system if you want it to work out.
You can also consider NIO but I think it can work fine with threads as well.