We Keep Coding

running two threads simultaneously inside a stateless agent and guaranting communication between them

I am developing a stateless Agent in Java that takes informations from one Server and transfer it to another client. It means that the agent is located between a client and a server. So I am thinking to run two threads simultaneously on the agent: one thread (thread1) runs a serverSocket and get request from client while another threads (thread2)is runnning and makes communication with the server. The problem consists in synchronizing between the two threads. I am thinking in making thread 1 asking whole the time thread 2 about a new Information. If thread 2 has nothing new, he will not answer it. What is the best way to synchronize between them. Should I use a global variable (a flag) to synchronize between them? Can I save Information when I have a stateless agent?

I think you should modify your app into async model.
Your app needs:
- an entry point to accept incoming connections -> a good example is an async servlet (or one dedicated thread).
- a ThreadPoolExecutor that provides fixed numbers of workers and a blocking queue (use this constructor).
The workflow:
Accept incomming request.
Wrapp incoming request into (Runnable) task.
Put task into blocking queue.
If ThreadPoolExecutor has a free worker starts processing the task
An advantage of such a model is that you are able to handle one request using one thread. So there is no need to manually synchronize anything.

Must a listening socket runs in thread?

I have some problems understanding how a socket should be handled. I get that server socket must runs in its own thread, because it must check if there are new connections. Now, i'm not sure if every socket opened by a new connection should runs in a thread.
What i have in mind is checking every x time the socket states. If it has something to be read, then read. If not, check the next socket. I see some examples where this process is done in a thread, but i dont want a socket to do stuff, just want to read if it has some data, and process them.

The answer is no, you don't need to listen in a separate thread. But, just realize that while you are "listening" your entire program will be waiting for that to complete before moving onward.
So unless you are fine with your entire program waiting, I would suggest a separate thread.

You can also have one thread which communicates with all sockets in a round-robin manner. It checks each socket if it has new data, and when it hasn't it checks the next.
Another alternative is to use NIO (New Input/Output).
The idea behind NIO is that you have a thread with one Selector which owns multiple Channels (a channel can be a network socket or any other IO interface). You then call selector.select() in a loop. This method blocks until one or more channels have data, and then returns a set of these channels. You can then process the data the channels delivered.
Here is a tutorial.

The problems with round-robin using available() are many.
It assumes that available() actually works, which isn't guaranteed.
It assumes that all clients need the same amount of service.
N-1 clients wait while one client is serviced.
A non-responsive client can block not only your application but all the other clients.
I'm sure there are more.
Don't do this. Use threads or NIO.

How to write an UDP server that will service n concurrent requests from different clients?

I am connecting 10 devices to a LAN, all of them have a udp server that goes like:
while(true){
serverSocket.receive(receivePacket);
dostuff(receivePacket);
}
serverSocket.close();
Now lets assume 9 of the devices try to initiate connection to the 10th device simultaenously. How can I accept all 9 instead of just the first which will then block the socket untill the server completes computation? Should I start a thread which will take care of dostuf() ? Will this let me get request from all of the simultaneous requests I got?

A basic design would have on thread responsible for handling incoming requests (with your desired limit) and then handing them off to worker/request handler threads. When each of these worker threads is finished, you'd want to update a shared/global counter to let the main thread know that it can establish a new connection. This will require a degree of synchronization, but it can be pretty fun.
Here's the idea:
serverThread:
while true:
serverLock.acquire()
if numberOfRequests < MAX_REQUESTS:
packet = socket.receive()
numberOfRequests++
requestThread(packet).run()
else
serverMonitor.wait(serverLock);
serverLock.release()
requestThread:
//handle packet
serverLock.acquire()
if numberOfRequests == MAX_REQUESTS:
numberOfRequests--
serverMonitor.pulse();
serverLock.release()
You'll want to make sure the synchronization is all correct, this is just to give you an idea of what you can start out with. But when you get the hang of it, you'll be able to make optimizations and enhancements. One particular enhancement, which also lends itself to limited number of requests, is something called a ThreadPool.
Regardless the basic structure is very much the same with most servers: a main thread responsible for handing off requests to worker threads. It's a neat and simple abstraction.

You can use threads in order to solve that problem. Since java already has an API that handles threads you can just create instance of runnable executors, take a look at the Executor Interface. Here is another useful link that could potentially help: blocking queue

Use a relatively larger size threadpool since udp doesn't require response.
main method will run as a listener and a threadpool will be doing rest of the heavy lifting

implementing keepalives with Java

I am building a client-server application where I have to implement a keepalive mechanism in order to detect that the client has crashed or not. I have separate threads on both client and server side. the client thread sends a "ping" then sleeps for 3 seconds, while the server reads the BufferedInputStream and checks whether ping is received, if so it makes the ping counter equals zero, else it increments the counter by +1, the server thread then sleeps for 3 seconds, if the ping counter reaches 3, it declares the client as dead.
The problem is that when the server reads the input stream, its a blocking call, and it blocks until the next ping is received, irrespective of how delayed it is, so the server never detects a missed ping.
any suggestions, so that I can read the current value of the stream and it doesn't block if there is nothing on the incoming stream.
Thanks,

Java 1.4 introduced the idea of non-blocking I/O, represented by the java.nio package. This is probably what you need.
See this tutorial for how to use non-blocking I/O.
Also, assuming this isn't homework or a learning exercise, then I recommend using a more robust protocol framework such as Apache Mina or JBoss Netty, rather than building this stuff from scratch. See this comparison between them, and why you'd want to use them.

You can have a separate monitoring thread which monitors all the blocking connections. When a connection receives anything it can reset a counter. (I would treat any packet as good as a heartbeat) Your monitoring thread can increment this counter each times it runs and when it reaches a limit (i.e. because it wasn't reset back to zero) you can close the connection. You only need one such thread. The thread which is blocking on the connection you just closed with throw an IOException, waking the thread.
On the other side, a heartbeat can be triggered whenever a packet has not been sent for some period of time. This mean a busy connection doesn't send any heartbeats, it shouldn't need to.

Servers and threading models

I am troubled with the following concept:
Most books/docs describe how robust servers are multithreaded and that the most common approach is to start a new thread to serve each new client. E.g. a thread is dedicated to each new connection. But how is this actually implemented in big systems? If we have a server that accepts requests from 100000 clients, it has started 100000 threads? Is this realistic? Aren't there limits on how many threads can run in a server? Additionally the overhead of context switching and synchronization, doesn't it degrade performance? Is it implemented as a mix of queues and threads? In this case is the number of queues fixed? Can anybody enlighten me on this, and perhaps give me a good reference that describes these?
Thanks!

The common method is to use thread pools. A thread pool is a collection of already created threads. When a new request gets to the server it is assigned a spare thread from the pool. When the request is handled, the thread is returned to the pool.
The number of threads in a pool is configured depending on the characteristics of the application. For example, if you have an application that is CPU bound you will not want too many threads since context switches will decrease performance. On the other hand, if you have a DB or IO bound application you want more threads since much time is spent waiting. Hence, more threads will utilize the CPU better.
Google "thread pools" and you will for sure find much to read about the concept.

Also Read up on the SEDA pattern link , link

In addition to the answers above I should notice, that really high-performance servers with many incoming connections attempt not to spawn a thread per each connection but use IO Completion Ports, select() and other asynchronous techniques for working with multiple sockets in one thread. And of course special attention must be paid to ensure that problems with one request or one socket won't block other sockets in the same thread.
Also thread management consumes CPU time, so threads should not be spawned for each connection or each client request.

In most systems a thread pool is used. This is a pool of available threads that wait for incoming requests. The number of threads can grow to a configured maximum number, depending on the number of simultaneous requests that come in and the characteristics of the application.
If a requests arrives, an unoccupied thread is requested from the thread pool. This thread is then dedicated to handling the request until the request finishes. When that happens, the thread is returned to the thread pool to handle another request.
Since there is only a limited number of threads, in most server systems one should attempt to make the lifetime of requests as short as possible. The less time a request needs to execute, the sooner a thread can be reused for a new request.
If requests come in while all threads are occupied, most servers implement a queueing mechanism for requests. Of course the size of the queue is also limited, so when more requests arrive than can be queued, new requests will be denied.
One other reason for having a thread pool instead of starting threads for each request is that starting a new thread is an expensive operation. It's better to have a number of threads started beforehand and reusing them then starting new threads all the time.

To get network servers to handle lots of concurrent connections there are several approaches (mostly divided up in "one thread per connection" and "several connections per thread" categories), take a look at the C10K page, which is a great resource on this topic, discussing and comparing a lot of approaches and linking to further resources on them.

Creating 10k threads is not likely to be efficient in most cases, but can be done and would work.
If you needed to serve 10k clients at once, doing so on a single machine would be unlikely but possible.
Depending on the client side implementation, it may be that the 10,000 clients do not need to maintain an open TCP connection - depending on the purpose, the protocol design can greatly improve the efficiency of implementation.
I think the appropriate solution for high scale systems is probably extremely domain-specific, and if you wanted a suggestion you'd have to explain more about your problem domain.