I am trying to implement a HTTP server using netty & i wanted to know few thing which i could not understand from the netty api. I read many other netty related stackoverflow question but still i couldn't udnertand.
1.If i want the connection from client to be opened for a certain period of time, what should i use CONNECT_TIMEOUT_MILLIS or add a read timeout handler & add a timeout in it. Basically i want to understand the difference between these two. & what is the default value of CONNECT_TIMEOUT_MILLIS.
what is the default value of SO_BACKLOG,i read it in one of the that it is equal to SOMAXCONN in io.netty.netUtils.But what it the value of it. Also, i want to be sure that so_backlog limits the number of worker thread ri8?. I mean if i set it to say 1000 it means netty won't allow more than 1000 open connection at a time.
can somebody explain how netty responds to a HTTP request as in internally in terms of writing & reading from a channel?
Thanks in advance!!!
CONNECT_TIMEOUT_MILLIS is the timeout for connection attempt. Once the connection is established, it has no effect. What you are interested in is ReadTimeoutHandler.
The default SO_BACKLOG is NetUtils.SOMAXCONN. It does not limit the number of worker threads. For more information about SO_BACKLOG, please refer to this question. To limit the number of worker threads, you must specify it when you construct an NioEventLoop. SO_BACKLOG is unrelated to the maximum number of concurrent connections, either.
Re: How HTTP works in Netty - The question is too broad to give a simple answer. Please use your debugger to step into the Netty internals to find our how it works.
One way of limiting the number of concurrent connections is by limiting the number of open files a process can have. This property can be set in Linux using ulimit command or using limits.conf file
Related
Ok, so let´s clarify the questions...
I'm studing Sockets in Java, from my understood until now, related to this subject are:
To make multiple clients to connect to only one address in the server (port), then it is necessary to assign each client connection to another thread
Based on that I got confused about somethings AND could not find any acceptable answer here or at Google until now.
If Socket is synchronous, what happens if 2 clients try to connect AT THE SAME TIME and how the server decides who will connect first?
How the server process multiple messages from one client? I mean, does it process in order? Return ordered?
Same question above BUT with multiple messages from multiple clients?
If the messages are not ordered, how to achieve that? (in java)
Sorry about all those questions but for me all of them are related...
Edit:
As the comment said, I misunderstood the concept of synchronization, so changed that part.
Guys we ask here to LEARN not to get judged by other SO think about that before giving -1 vote ok.
what happens if 2 clients try to connect AT THE SAME TIME
It is impossible for 2 clients to connect at exactly the same time: networking infrastructure guarantees it. Two requests happening at the exact same time is called a collision (wikipedia), and the network handles it in some way: it can be through detection or through avoidance.
How the server process multiple messages from one client? I mean, does it process in order?
Yes. The Socket class API uses the TCP/IP protocol, which includes sequence numbers in every segment, and re-orders segments so that they are processed in the order they are sent, which may be different from the order they are received.
If you used DatagramSocket instead, that would use UDP, which does not guarantee ordering.
Same question above BUT with multiple messages from multiple clients?
There are no guarantees of the relative ordering of segments sent from multiple sources.
I am working on socket programming on Java recently and something is confusing me. I have three questions about it.
First one is;
There is a ServerSocket method in Java. And this method can take up to 3 parameters such as port, backlog and ip address. Backlog means # of clients that can connect as a form of queue into a server. Now lets think about this situation.
What happens if 10 clients try to connect this server at the same
time?
Does Server drop last 5 clients which tried to connect? Lets increase the number of clients up to 1 million per hour. How can I handle all of them?
Second question is;
Can a client send messages concurrently without waiting server's response? What happens if a client sends 5 messages into server that has 5 backlog size?
The last one is not a question actually. I have a plan to manage load balancing in my mind. Lets assume we have 3 servers running on a machine.
Let the servers names are A, B and C and both of them are running smoothly. According to my plan, if I gave them a priority according to incoming messages then smallest priority means the most available server. For example;
Initial priorities -> A(0), B(0), C(0) and respond time is at the end of 5. time unit.
1.Message -> A (1), B(0), C(0)
2.Message -> A (1), B(1), C(0)
3.Message -> A (1), B(1), C(1)
4.Message -> A (2), B(1), C(1)
5.Message -> A (2), B(2), C(1)
6.Message -> A (1), B(2), C(2)
.
.
.
Is this logic good? I bet there is a far better logic. What do I do to handle more or less a few million requests in a day?
PS: All this logic is going to be implemented into Java Spring-Boot project.
Thanks
What happens if 10 clients try to connect this server at the same time?
The javadoc explains it:
The backlog argument is the requested maximum number of pending connections on the socket. Its exact semantics are implementation specific. In particular, an implementation may impose a maximum length or may choose to ignore the parameter altogther.
.
Lets increase the number of clients up to 1 million per hour. How can I handle all of them?
By accepting them fast enough to handle them all in one hour. Either the conversations are so quick that you can just handle them one after another. Or, more realistically, you will handle the various messages in several threads, or use non-blocking IO.
Can a client send messages concurrently without waiting server's response?
Yes.
What happens if a client sends 5 messages into server that has 5 backlog size?
Sending messages has nothing to do with the backlog size. The backlog is for pending connections. Messages can only be sent once you're connected.
All this logic is going to be implemented into Java Spring-Boot project.
Spring Boot is, most of the time, not used for low-level socket communication, but to expose web services. You should probably do that, and let standard solutions (a reverse proxy, software or hardware) do the load-balancing for you. Especially given that you don't seem to understand how sockets, non-blocking IO, threads, etc. work yet.
So for your first question, the backlog queue is something where the clients will be held in wait if you are busy with handling other stuff (IO with already connected client e.g.). If the list grows beyond backlog, the those news clients will get a connection refused. You should be ok with 10 clients connect at the same time. It's long discussion, but keep a thread pool, as soon you get a connected socket from accept, hand it to your thread pool and go back to wait in accept. You can't support millions of client "practically" on one single server period! You'll need to load balance.
Your second question is not clear, clients can't send messages, as long as they are on the queue, they will be taken off the queue, once you accept them & then it's not relevant how long the queue is.
And lastly your question about load balancing, I'd suggest if you are going to have to serve millions of clients, invest in some good dedicated load-balancer :), that can do round robin as well as you mentioned.
With all that said, don't reinvent the wheel :), there are some open source java servers, my favorite: https://netty.io/
This question already has answers here:
What is the difference between connection and read timeout for sockets?
(2 answers)
Closed 8 years ago.
Just curiosity. Is there a good reason why the class URLConnection needs to have two different timeouts?
The connectTimeout is the maximum time in milliseconds to wait while connecting. Connecting to a server will fail with a SocketTimeoutException if the timeout elapses before a connection is established.
The readTimeout is the maximum time to wait for an input stream read to complete before giving up. Reading will fail with a SocketTimeoutException if the timeout elapses before data becomes available.
Can you give me a good reason why these two values should be different? Why a call would need more time for performing the connection rather than receiving some data (or viceversa)?
I am asking this because I have to configure these values and my idea is to set the same value for both.
Let's say server is busy and is configured to accept 'N' connection and all the connections are long runner and all of sudden you send in request, What should happen? Should you wait indefinitely or should you time out? That's connectTimeout.
While let's say your server turns brain dead service just accepting connection and doing nothing with it (or say server synchronously goes to db and does some time taking activity and server ends up with deadlock for e.g.) and on the other hand client keeps on waiting for the response, in this case what should client do? Should it wait indefinitely for response or should it timeout? That's read timeout.
The connection timeout is how long you're prepared to wait to get some sort of response from the server. It's not particularly related to what it is that you're trying to achieve.
But suppose you had a service that would allow you to give it a large number, and have it return its prime factors. The server might take quite a while to generate the answer and send it to you.
You might well have clear expectations that the server would quickly respond to the connection: maybe even a delay of 5 seconds here tells you that the server is likely to be down. But the read timeout might need to be much higher: it might be a few minutes before you get to be able to read the server's answer to your query.
The connect time-out is the time-out in which you want a (in normal situations TCP) connection to be established. The default time-outs as specified in the internet RFCs and implemented by the various OSes are normally in the minute(s) range. But we know that if a server is available and reachable, it will respond in a matter of milli-seconds and otherwise not at all. A normal value would be a couple of seconds at a maximum.
The read timeout is the time in which the server is expected to respond after it received the incoming request. Read time-outs therefore depend on time within you expect the server to deliver the result. These are depending on the type of the request you are making and should be larger if the processing requires some time or the server may be very busy in some situations. Especially if you do a retry after a read time-out, it is best to put the read time-outs not too low, normally a factor 3-4 times the expected time.
Just start scaling APNS provider program unfortunately I am really new to networking protocol implementation.
The provider now only runs on one thread and it's just handling a tiny amount of notifications. Now I want to increase its capability to send significantly more than before.
My questions are:
According to Apple doc I can maintain multiple connections to gateways. So my understanding is that I run multithreads in the provider program and maintain a separate connection in each. Is this right?
It first one is right the real difficulty for me comes: my program polls a queue database every 5 seconds to check new message that's to be sent. I do not think it's a good idea for all the threads to poll this same database because there should be duplicate message same to users. How to solve this problem?
I have seen the connections pooling but I do not really understand what that is. Is that the thing I need to study and use? If it is can someone offer an brief explanation regarding what it is and how to use it?
Thanks guys!
Your first assumption is reasonable. Each thread should have its own connection.
As for the second point, the access to the DB that contains the new messages should be synchronized. For example, you can access that DB by a synchronized method that fetches a message or several messages that haven't been processed yet, and marks them as being processed. Two threads can't access that method at the same time, and therefore won't get the same messages.
Another option is to put the messages in memory in a blocking quoue (with the DB only serving for backup in case of a crash). The threads can request an item from the queue, which would block them until an item is available.
I've built a simple Java program that works as a server locally.
At the moment it does a few things, such as previews directories, forwards to index.html if directory contains it, sends Last-Modified header and responds properly to a client's If-Modifed-Since request.
What I need to do now is make my program accept persistent connections. It's threaded at the moment so that each connection has it's own thread. I want to put my entire thread code within a loop that continues until either Connection: close, or a specified timeout.
Does anybody have any ideas where to start?
Edit: This is a university project, and has to be done without the use of Frameworks.
I have a main method, which loops indefinitely, each time it loops it creates a Socket object, a HTTPThread object is then created (A class of my own creation) - that processes the single request.
I want to allow multiple requests to work within a single connection making use of the Connection: keep-alive request header. I expect to use a loop in my HTTPThread class, I'm just not sure how to pass multiple requests.
Thanks in advance :)
I assume that you are implementing the HTTP protocol code yourself starting with the Socket APIs. And that you are implementing the persistent connections part of the HTTP spec.
You can put the code in the loop as you propose, and use Socket.setSoTimeout to set the timeout on blocking operations, and hence your HTTP timeouts. You don't need to do anything to reuse the streams for your connection ... apart from not closing them.
I would point out that there are much easier ways to implement a web server. There are many existing Java web server frameworks and application servers, or you could repurpose the Apache HTTP protocol stacks.
If it should act like a web service: Open 2 sockets from the client side, one for requests, one for
responses. Keep the sockets and streams open.
You need to define a separator to notify the other side that a
transfer is over. A special bit string for a binary, a special
character (usually newline) for a text-based protocol (like XML).
If you really try to implement an own http-server, you should rather make use of a library that already implements the HTTP 1.1 connection-keepalive standard.
Some ideas to get you started:
This wikipedia article describes HTTP 1.1 persistent connections:
http://en.wikipedia.org/wiki/HTTP_persistent_connection
You want to not close the socket, but after some inactive time period (apache 2.2 uses 5 seconds) you want to close it.
You have two ways to implement:
in your thread do not close the socket and do not exit the thread, but instead put a read timeout on the socket (whatever you want to support). When you call read it will block and if the timeout expires then you close the socket, else you read next request. The downside of this is that each persistent connection holds both a thread and a socket for whatever your max wait period is. Meaning that your solution doesn't scale because you're holding threads for too long (but may be fine for the purposes of a school project)!
You can get around the limitation of (1) by maintaining a list of tuples {socket,timestamp}, having a background thread monitor and close connections that timeout, and using NIO to detect a new read on an existing open socket. So after you finish reading the initial request you just exit the thread (returning it to the thread pool). Obviously this is much more complicated but it has the benefit of freeing up request threads.