I am looking for a good TCP connection library from Java with the following facilities:
1. Retry on failed publishes
2. Multiple connections
Which library have you sucessfully used.
EDIT: Based on the comment changed the question to reflect which type of connection library.
May be Apache MINA will help you.Have a look .
I'm not sure this really makes sense. You're talking about retrying on failed publishes, yet TCP doesn't have a concept of publishing. Merely message transfer. So you could be publishing, or you could be requesting info.
e.g. HTTP over TCP has the verbs GET/PUT/POST (amongst others). All of these run over TCP. Only two actually write something (PUT/POST). And only PUT is supposed to be idempotent (that is to say, you should be able to the same operation again and again with the same result). If you POSTed repeatedly, I'd expect to republish something and create a new version on the server for every POST.
And the above are only recommendations for how PUT/POST are implemented. I wouldn't want an HTTP library to assume this on my behalf.
So the concept of retrying messages at the TCP layer is mistaken (note that TCP will resend packets etc. making up a message). This is a higher-level function, which may use TCP at a lower level. e.g. I've written my own wrappers around HTTPClient to retry PUTting when my remote server becomes temporarily unavailable or reports an error (I'm not sure a retrying HTTP library exists)
Maybe this help others, Try this library called socketal, Pure Java uses ServerSocket and Socket, it's pretty simple and doesn't have any unnecessary feature.
This library is capable of:
Autoreconnect on disconnection
Capable of handling Connecting/Disconnected/Connected
Pretty simple to send String, Object or File
Set your own Authentication code and Verification just like Login Password
It's seems like the Netty but these doesn't have a lot of complicated setup and features.
It's compatible for Android/Java.
Related
I have a multithreaded java program that runs on Amazon's EC2. It queries and fetches data items from a vendor via HttpPost and HttpGet, using a org.apache.http.impl.client.DefaultHttpClient. Concurrently, it pushes the retrieved data items into S3 using AWS's Java SDK.
After a few days of running, I get the symptoms that normally come with http connection leaks:
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection
at org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByRoute.java:417)
at org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRoute.java:300)
at org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection(ThreadSafeClientConnManager.java:224)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:391)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
Since both AWS and my requests to the data vendor use Http connections, I am not quite sure where exactly I forget to HttpEntity.consume(), or S3ObjectInputStream.close() (unless it is yet something else...).
So here is my question: are there ways to monitor org.apache.http.impl.conn.tsccm.ConnPoolByRoute so that at least I can detect when I am starting to leak connections/entities not properly consumed/http streams not closed? (I have a feeling it happens only under certain conditions, e.g. when certain exceptions are being thrown, by-passing the logic in my code that consumes HttpEntities, closes streams, etc.) Any idea on how to diagnose what eventually causes all my http connections to fail with that ConnectionPoolTimeoutException would be most welcome. I don't feel like waiting 4+ days between attempts to fix the root cause of the problem.
If you're using the PoolingClientConnectionManager note there are the methods getTotalStats() and getStats(final HttpRoute route) which will give you a PoolStats object with the data you're looking to monitor.
Just fetch the ConnectionManager from your httpclient:
PoolingClientConnectionManager poolManager = (PoolingClientConnectionManager) httpClient.getConnectionManager();
If you can access the org.apache.http.impl.conn.tsccm.ConnPoolByRoute then set it's connTTL to a low enough value so that it's WaitingThreadAborter will eventually terminate a connection. It will show a nice stacktrace there. The other option is to use CGLIB or some other bytecode manipulating framework to create a proxy class wrapping org.apache.http.impl.conn.tsccm.ConnPoolByRoute. Depending on your environment it might not be that easy to set it up, but it's a rather valuable tool to debug issues like yours. (And yes, if you happen to use spring or just plain Aspects the setup will be supereasy :) )
I have not worked with HTTP post/get before, my up coming project in my office is based on http post/get in java. Its basically client - server based application. the client will post some info and I need to get that info and process the string and vice-verse. this project has to be developed on J2SE. You can assume this some thing like a JMS queue message processing stuff. I googled for the info but most of the information was for web application, mine should work like a message queue. Can someone explain me how to do this or point me where I can get some useful info.
Thanks
Arun
Well, if you don't need to specifically use strict HTTP, and you need to just use Java SE (and not Java EE, which rules out Servlets, JSPs, JMS, etc), then you need to probably investigate ServerSocket and Socket classes.
Server
Your server would need to listen on a TCP port (say, port 8080) - usually you would pick a port number between 1025 and 65,535, however if you are attempting to use an already defined service that has a default port, then use that. Note however, that on unix, in order to listen on any port below 1024, I believe you need to be root. Traditionally, port 80 is used for HTTP.
To listen on this port, you would need something like this in your code:
ServerSocket srvSocket = new ServerSocket(8080);
Socket socket = srvSocket.accept();
This pretty much the most basic code that would cause your application to wait until something connected to port 8080. Once connected, you could obtain both an InputStream and OutputStream for your connected client, by interrogating the returned socket object, allowing you to read content from the client, and inserting these requests in a queue. This queue could be then processed by some other Thread.
Client
In order for your client to connect to the server, you would need to use something based on the following example:
Socket connection = new Socket("server.domain.com", 8080);
OutputStream output = connection.getOutputStream();
You would then write your request to the server into the OutputStream (and read from the InputStream returned from getInputStream() if you expected a response)
The code supplied is pretty basic, but it should give you a rough idea of how to proceed. You can even use this method if you wanted to use real HTTP, however it might be a better idea to use some premade library if that was the case (although its probable that you're not going to require all functionality defined in the HTTP spec itself).
Anyway, I hope that provides you a good starting point from which to build.
Jetty is a popular web server, designed to easily be embedded in an application.
Its HTTP server component can run inside your application and respond to requests by dispatching to your custom code.
Jetty also features an HTTP client that you can use on the client side to send requests.
This is a rather big topic and I won't be able to post a complete guide, but Jetty's documentation is generally of very high quality and should be a good starting point.
I suggest you start with learning the basics of HTTP protocol. This article is a good starter. After you understood the basics follow the this article on how to programatically communicate (read/write) with HTTP servers. After that Google is your friend.
If you weren't restricted to J2SE, you could use Servlets for managing the POST/GET methods of HTTP. Evaluate if it is possible, otherwise you'd be reinventing the wheel
I also have a mainly SE background. On the client side, writing get/post is pretty easy. Or you can Google to find source code. I found that using REST was straightforward and understandable. On the server side, there are many options and I have very limited experience. I wrote the server using standard JEE6 and it wasn't too painful, but sounds like that is not an option for you.
I'm writing a multiplayer/multiroom game (Hearts) in java, using RMI and a centralized server.
But there's a problem: RMI Callbacks will not work beacause clients are Natted and Firewalled. I basically need the server to push data updates to clients, possibly without using polling and without using sockets (I would code at an higher level)
In your opinion, what's the best solution for realizing this kind of architecture? Is an ajax application the only solution?
You say that you don't want polling, but AJAX is exactly that. You can look at Comet but it's hard to escape polling anyway (e.g. Comet itself uses polling underneath).
You could use a peer to peer framework such as JXTA.
I can suggest two main techniques.
The server has a method getUpdates, callable by clients. The method returns the control to the client when there is an update to show.
When Clients perform the registration, they give the server a callback remote object
Since this object is not registered in any rmi registry, there should no be any issue with natted clients.
I'm not sure how(if) ajax works for a non-browser-based app. You could just maintain your own pool of SocketConnections open for the duration of the application with a Thread per connection.
If you need to scale to a lot of concurrent connections, look to a non-blocking I/O framework like Apache Mina or Netty (related SO post: Netty vs Apache MINA).
I'm working on a WSDL-based web service and using Apache Axis 2. I'm not an expert on web services, and the person I'm working with claims that in order for this particular web service to work two calls have to be made on the same connection, i.e. using http keep-alive (There's basically a "commit transaction" method that needs to be called after the "save" method). This seems like it would be a pretty common issue, but I haven't found anything on Google.
I'm wondering if there's a way to explicitly tell Axis to do this. Also, how could I verify whether or not two calls are indeed being made on the same connection. I imagine some HTTP monitoring software like wireshark might be able to tell me this, but I haven't installed it yet.
The person you are working with is wrong. Even if HTTP can be optimized by using keep-alive to process multiple requests over a single TCP connection, this optimization should be transparent to the caller or callee, e.g. it should not matter if a client make two requests after each other on a keep-alive connection or if it's using two separate connections.
Java libraries (HttpURLConnection on the client side and the servlet API on the server side) do not even offer access to this information, so that the using software cannot know how the HTTP requests are actually performed.
You can use nmaplink text to see what is actually running on each port.
But if you are making 2 calls at same time, axis2 will throw port is already binded error. Any port can't handle 2 requests at the same time (my opinion). Maybe you can queue it and do it consecutively. But just confirm with other sources as well.
I am writing a web server in Java and I want it to support HTTP 1.1 Keep-Alive connections. But how can I tell when the client is done sending requests for a given connection? (like a double end-of-line or something).
Lets see how stackoverflow handles this very obscure question -- answers for which, on Google, are mired in technical specifications and obscure language. I want a plain-english answer for a non-C programmer :)
I see. that confirms my suspicion of having to rely on the SocketTimeoutException. But i wasn't sure if there was something i could rely on from the client that indicates it is done with the connection--which would allow me to close the connections sooner in most cases--instead of waiting for the timeout. Thanks
If you're building your server to meet the standard, then you've got a lot of information to guide you here already.
Simple spoken, it should be based on a time since a connection was used, and not so much at the level of request data.
In a longer-winded way, the practical considerations section of the HTTP/1.1 document has some guidance for you:
"Servers will usually have some
time-out value beyond which they will
no longer maintain an inactive
connection. Proxy servers might make
this a higher value since it is likely
that the client will be making more
connections through the same server.
The use of persistent connections
places no requirements on the length
(or existence) of this time-out for
either the client or the server."
or
"When a client or server wishes to
time-out it SHOULD issue a graceful
close on the transport connection.
Clients and servers SHOULD both
constantly watch for the other side of
the transport close, and respond to it
as appropriate. If a client or server
does not detect the other side's close
promptly it could cause unnecessary
resource drain on the network."
Lets see how stackoverflow handles this very obscure question -- answers for which, on Google, are mired in technical specifications and obscure language.
I just put When should I close an HTTP 1.1 connection? into Google, and the third hit was HTTP Made Really Easy. In the table of contents, there is a link to a section entitled Persistent Connections and the "Connection: close" Header. This section is three paragraphs long, uses very simple language, and tells you exactly what you want to know.
I want a plain-english answer for a non-C programmer :)
With all due respect, programming is a technical endeavour where the details matter a great deal. Reading technical documentation is an absolutely essential skill. Relying on "plain English" third-party interpretations of the specifications will only result in you doing a poor job.
You close it whenever you'd like. The header indicates that the client would prefer you to leave the connection open, but that doesn't require the server to comply. Most servers leave it open for about 5-10 seconds, some don't pay attention to it at all.
You should read the RFCs dealing with the Keep-Alive feature. Otherwise you might end up with a server that doesn't work as expected.
As #[Stephen] has already pointed out, the server is free to close the connection anytime it wishes (ok, not in the middle of a request/response pair though). Ditto for the client. Any other solution would allow the server or the client to perform a DoS on the other party.
EDIT: Have a look at the Connection header. The client (and the server) can request a graceful connection closure using the header. For example, Connection: close inside the request is a request to the server to close the connection after it sends the response.