Well, the title is quite self expleneationry.. I'm tryin to improve mq client connection time since it takes to much time in my app.
(look here for code)
MQ previously had the UDP protocol available on AIX only. It was removed a few releases ago. TCP/IP is the transport of choice now. If you are having long connection times, you should get your network guys involved.
IBM MQ clients connect to queue manager over TCP sockets as TCP is connection oriented and reliable. Messaging applications require this kind of QoS from underlying transport. UDP is not connection oriented and reliable when compared to TCP.
You say your application takes lot of time to connect, have you looked at if there are any issues in your network? Are you connecting over slow network? Is your application the only one that takes time to connect or other applications have similar problem?
Related
Recently I started using ActiveMQ to act as a Message Middleware in my new project, this is the first time I try use ActiveMQ, the projects I had participated before used our previous company's inner message framework like Swallow. So before I begin implementing the system, I need to clear some design points.
Cases in our system will use ActiveMQ include sending mail, sending tasks to queue and doing tasks from queue, asynchronous request/response, so what kind of protocol and network connection is the right choice for our cases? I list some protocols and network connection options here:
ActiveMQ protocols:
MQTT
WS
Openwire
Stomp
Stomp
ActiveMQ Network connections:
VM
TCP
UDP
HTTP
Failover
Discovery
I will also consider the aspects of HA and cluster for my system, so can anybody gives me some ideas to decide how to choose the protocol and network connection?
Thanks a lot.
Openwire has historically been the default protocol the NIO transport can give performance improvements over TCP so if you use ActiveMQ as your only broker use one of these. However using AMQP means in the future you could possibly use RabbitMQ, another popular Message Broker. There are others, STOMP or MQTT are lightweight, VM is designed to be used when the application resides on the same machine as the broker so gets very high throughput.
As ActiveMQ can enable all protocols by default do some quick tests to gain an idea of throughput on the specific application you are building. Then consider the above points in making a decison.
Regarding UDP, TCP, HTTP I would choose TCP. UDP is unreliable and TCP is more than adequate in sending 1000's per second. HTTP could be useful if your company has awkward firewall rules.
I would wrap this in a failover transport. I have never used discovery but would argue this is more advanced and not required initially as it requires a discovery agent. Its only purpose is too discover the ActiveMQ broker dynamically (although you still have to know where the discover agent is).
Basically I am new to server and client programming in java , I google all the necessary resources to learn from this particular topic however I did not understand the difference between them.
What I Understand so far for these two is that Both of them can Handle Client Request, but I need to further know the benefits of each Class and what particular scenario or specific case where when can I used it efficiently.
Like for instance , I have a Server Client Program which is a subset of team-viewer in which The client program must send Screenshot to the server in every millisecond while the server is going to publish it from another connected client. The code is working but I found out ServerSocket consumes so much Heap although it delivers successfully to the servers and client as well. I also read a blog (The link is missing) that is related to my problem suggested that DatagramSocket is the solution because it does not execute Handshakes.
I am really concern of the Benefits and Disadvantage of these classes.
A ServerSocket is for accepting incoming network connections on some stream protocol; e.g. TCP/IP.
A DatagramSocket is for sending and receiving datagrams on some connectionless datagram / message protocol; e.g. UDP/IP
Supplementary questions:
Basically what is a datagram
A datagram is bunch of information sent in a single logical packet. For example, a UDP packet.
and does this mean datagram = lightweight packets ?
It depends on your definition of lightweight!
UDP datagrams are sent as IP packets. If a UDP datagram is too big for an IP packet, it is broken into multiple IP packets by the sender and reassembled by the receiver.
and what does connectionless [mean],
It means that no logical connection exists between the 2 parties. If a component IP packet of a UDP datagram is lost, the UDP datagram is lost. The receiver never knows (at the application level). There is no reporting of data loss and no retrying in UDP. This is typical "connectionless" behavior.
does it mean Data might get lost during transmission?
Basically, yes. If you want reliable / lossless data transmissin the event that a datagram or on you should use ServerSocket and Socket; e.g. TCP/IP streams.
However, be aware that even with a (bare) TCP/IP stream, data delivery is not guaranteed:
If there is a network failure, or if either the sender or receiver has a failure, then a connection can be broken while data is in transit. That will result in data loss ... for that connection. (Sockets do not support reconnecting.) If the sender and/or receiver are still alive they will typically be informed that the connection has been broken, but they won't know why, or how much data was lost in transit.
It is possible for data to be corrupted in transit in ways that TCP/IP's error detection cannot spot. The receiver won't know this has happened.
Both of these issues can be addressed at the application protocol level; e.g. using message queues for the first and strong encryption and strong checksumming for the second.
Concerning your attempt to use ServerSocket.
The code is working but I found out ServerSocket consumes so much Heap although it delivers successfully to the servers and client as well.
You are doing something wrong. If you use the API appropriately the memory overheads should be insignificant.
My guess is that you are doing one or more of the following:
Opening a new connection for each client / server interaction
On the server side, creating a new thread for each connection
Not closing the connections.
I also read a blog (The link is missing) that is related to my problem suggested that DatagramSocket is the solution because it does not execute Handshakes.
Handshakes won't cause significant memory consumption.
TCP/IP stacks don't typically do handshakes by default anyway.
You say you have looked on google, but there are several pages on google that address your question directly. There are several that have the same title as your question. You have even indicated that you understand some of the difference between them by using the [tcp] and [udp] tags on your question.
The difference is one uses TCP communication protocol and one uses the UDP communication protocol. Perhaps your question is not one about Java but about how the internet, computer networking, and the communication protocols work?
TCP is a connection oriented reliable delivery protocol.
UDP is a connectionless unreliable delivery protocol.
What this means is you have to decide which is important, speed or reliability.
Is the data such that it must not be corrupted in transit? If it is then you must use TCP or a serversocket.
If the data must arrive by the fastest method, even at risk of getting lost, then you must use UDP or a datagramsocket.
If you need more explanation to understand you should take a course on computer networking.
The point of my question is to ask if it is accepted to use both TCP and UDP to communicate between client and server.
I am making a real-time client server game with parts of the communication that need to be guaranteed (logging in, etc..), but other parts will be ok to lose packets (state updates, etc). So, I would like to use UDP for most of the data communication but I do not want to have to implement my own framework to insure that my control communication (logging in) is guaranteed.
So, would it be reasonable to initially use TCP to manage a connection, and then on a separate port send data communication pack and forth?
You should absolutely do it that way (use TCP and UDP to accomplish different communication tasks.) And you don't even have to use two different ports. One will suffice. You can listen to the two different protocols on the same port.
It is quite reasonable and already used in mainstream. Even when browsing the Web, DNS operations are UDP-based and HTTP connections are TCP-based.
Keep in mind that you should either consider the two connection types to be completely independent or employ additional measures to properly handle any inter-dependencies. TCP connections can have timing issues at the OS and network levels and UDP connections have packet loss issues. You should take specific measures to avoid deadlocks and performance problems when the TCP part of your application stalls or a UDP packet is lost.
It is not only accepted but is widely used. As a good example, BATS Exchange is using this approach in their market data distribution system, to implement a recovery mechanisms.
I am using Jersey 1.4, the ApacheHttpClient, and the Apache MultiThreadedHttpConnectionManager class to manage connections. For the HttpConnectionManager, I set staleCheckingEnabled to true, maxConnectionsPerHost to 1000 and maxTotalConnections to 1000. Everything else is default. We are running in Tomcat and making connections out to multiple external hosts using the Jersey client.
I have noticed that after after a short period of time I will begin to see sockets in a CLOSE_WAIT state that are associated with the Tomcat process. Some monitoring with tcpdump shows that the external hosts appear to be closing the connection after some time but it's not getting closed on our end. Usually there is some data in the socket read queue, often 24 bytes. The connections are using https and the data seems to be encrypted so I'm not sure what it is.
I have checked to be sure that the ClientRequest objects that get created are closed. The sockets in CLOSE_WAIT do seem to get recycled and we're not running out of any resources, at least at this time. I'm not sure what's happening on the external servers.
My question is, is this normal and should I be concerned?
Thanks,
John
This is likely to be a device such as the firewall or the remote server timing out the TCP session. You can analyze packet captures of HTTPS using Wireshark as described on their SSL page:
http://wiki.wireshark.org/SSL
The staleCheckingEnabled flag only issues the check when you go to actually use the connection so you aren't using network resources (TCP sessions) when they aren't needed.
I'm writting a java application, and I need to quickly discover any other running clients on any wired or wireless local networks in order to establish a TCP connection.
what's the best way of doing this? Are there libraries or code snippets that would do this?
Multicast UDP is a good way of doing this. It's used in a couple of technologies that support automatic discovery of networked devices over local IP networks (UPnP and ZeroConf).
Multicast UDP is not TCP, but it is still based on IP and, so, uses the same addressing mechanism i.e. IP addresses. Quite often it is compared to radio broadcasting i.e. a multicast sender only needs to send 1 message (i.e. it is like a broadcast) but only clients that are "tuned-in" to the multicast channel will receive it.
You can do a quick search on google or wikipedia for these as a starter, but the basic idea is as follows:
when a client starts, it sends out a multicast UDP "hello" message to some pre-specified multicast address and port (e.g. UPnP uses 239.255.255.250:1900)
existing clients are listening for incoming multicast "hello" messages on the specified address and port - when a client receives one, it sends a response to the sender
the client sending the "hello" message receives a response from each existing client on the network and is now aware of the IP address of each client
If you are looking for libraries to use, UPnP libraries can tend to be a bit heavyweight and a lot of folk generally don't like them, so ZeroConf might be a little more suitable. I don't know of any java implementations of such things but I'm sure you can find some with a little digging around.
A network scan can be very long, even longer on wireless networks. If you need them quickly thru Java you may implement a "meeting point" server on your network. This server listen to a predefined port, clients register on the server on startup and the server can distribute information about the clients on request.
HTH.
I guess you need to do a scan on your application's port on all IPs in your subnet.
Just what are the available IPs - or what is your subnet for that matter?
I'm afraid determining that could turn out to be impossible as the network is designed to be transparent to your application.
So, i'd use brute force: pick your IP and change the last byte. Might be too much, might be not enough though.
Or you send a broadcast (which usually would be targeted at x.x.x.255) and see who answers.
See Datagram Broadcasting and Multicasts. But i think that's not TCP/IP anymore.
There is a JGroups toolkit for reliable multicast communications. It allows automatic discovery of additional clients using Multicast techniques as described in other answers.
It also provides communication APIs on top of multicast sockets.
It is used in a number of projects such as JBoss, Tomcat and more to provide an infrastructure for distributed cache. See more here.