We Keep Coding

Use XNIO or Netty for writing a client side (JDBC driver) application?

Assume the following scenario:
I want to write (or rewrite) a JDBC driver (client side) using NIO, using pure NIO seems to be overly complex, and two of the recommended choices are Netty and XNIO, there are others like Apache MINA that can be considered too.
So the question is, for a client side application, which fits best for this based on:
Easy of use to implement a codec (protocol) on it.
Allow easy shaded jar (don't have awful dependency spaghetti)
First-class TLS support.
Have good documentation.
Don't break things often.
Or is using just plain java.net more easy, efficient and enought for this job?

Or is using just plain java.net more easy, efficient and enought for this job?
Yes. The overheads are in the network and at the server. The benefits of NIO in a client are minimal. Specifically, you don't get to conserve threads, because you're not a server, and the 'throughput benefits of non-blocking I/O' you allege in comments are non-existent.

Which NIO library (Netty, Grizzly, kryonet, ...) for simple backend server implementation in Java?

Our frontend is simple Jetty (might be replaced with Tomcat later on) server. Through servlets, we are providing a public HTTP API (more or less RESTful) to expose our product functionality.
In the backend, we have a Java process which does several kind of maintenance tasks. While the backend process usually does it own tasks when it's time, now and then, the frontend needs to wake-up the backend to execute a certain task in the background.
Which (N)IO library would be ideal for this task? I found Netty, Grizzly, kryonet and plain RMI. For now, I am inclined to say Netty, it seems simple to use and it is probably very reliable.
Does any of you have experience in this kind of setups? What would your choice be?
thanks!

Try to translate this document which answer to your question.
http://blog.xebia.fr/2011/11/09/java-nio-et-framework-web-haute-performance/
This society, as french famous Java EE experts, did a lot of poc of NIO servers in the context of a french challenge sponsored by VmWare (USI2011). It was about building a simple quizz app that can handle a load of 1 million connected users.
They won that challenge with great results.
Their implementation was Netty + Gemfire and they only replaced the CachedThreadPool by a MemoryAwareThreadPool.
Netty seems to offer great performances, and is well documented.
They also considered Deft, inspired by Tornado (python/facebook) but it's still a bit immature for them
Edit: here's the translated link provided in the comments

My preference is Netty. It's simple yet flexible. Very fast and the community around Netty is awesome.

The company I work for is currently evaluating CoralReactor. It is a commercial software but it has the easiest API I have ever seen for Java NIO. My personal opinion is that Netty makes things too complicated, especially if you want to go garbage-free and single-threaded, which are a requirement for many companies from the finance, advertisement and game industry.

I would decouple them by using JMS, just have some (set of) control queues your backend sits there listening on and you're done. No need to write a custom nio api here.
One sample provider is hornetq. This can be run as an in process jms broker as well, it uses Netty under the covers.

Implementing Server Push

Read about Server push here.
I want to push data to client from my web application in real time.
I was looking at TCP sockets as one of the options.
For HTTP I found a variety of frameworks for Java, PHP, Python and others over here.
However I don't know whether any of these support Push.
What options and frameworks would you
suggest for implementing Server push?
What language would you advocate for implementing the same and why?

I'm using Orbited right now, it's great!
If you are doing chat or subscription type stuff use stompservice and orbited.
If you are doing 1 to 1 client mapping use TCPSocket.
I can give you some code examples if you want.

How about Orbited, it's very good and being used by Echowaves

Comet is the protocol you want. What Comet implementation is best, is a harder call.
If you're OK with Java (or, I guess, Jython), or .NET (where IronPython's a possibility), I suspect (not having extensively tried them all!-) that stream hub must be a major contender. It'a typical "freemium" product -- you can get a free ("as in free beer";-) version, or you can try the pricey Web Edition, or the even-pricier Enterprise Edition; feature comparison is here (e.g., free edition: no https, no more than 10 concurrent users, no .NET).

Ok, I'm using ASP.NET with PokeIn comet ajax library on my project. Also, I tried Atmosphere under JAVA.. My last choice was PokeIn.. Because, only server push support is not solving the problems. You will need some kind of client to server object serialization and object life time management. PokeIn covered all these needs for me.

What about Ajax Push Engine?

I'm personally biased, but I like WebSync, for IIS/.NET. It integrates with IIS, so no other server software necessary, just a dll to add to your project.

I believe xmpp implementation is one which is being use by a lot of big companies but the common thing is to use a comet server as well.
a lot of implementation in python for thoses you can google around.

Have you tried StreamHub Push Server?

PHP Communicating w/ Java program

We're come across a problem here at my company and I'm trying to find the best solution.
Software was recently purchased that utilizes a Java program to get the tax for a certain shipment. The site that needs this was written in PHP4. How can I communicate between the two?
It was suggested to use files to communicate but that was horribly slow since the Java program needed to be recompiled every time. So, what is the best solutions to this:
Create a mutli-threaded Java server and use PHP to send/receive the info.
Some other type of file-writing method
Something cool that I dont even know about.
Thanks in advance!
Edit:
I understand the importance of web services but why would this be more efficient that using a mutli-threaded socket-based java server? The only thing connecting to this web services will be my PHP program, no one else. It seems like it might be overkill for my simple task. Am I mistaken? If so, why? Thanks.

Wrap the Java program in a Web Service, and invoke it from PHP. You can even use caching in the Web Service, to optimize performance.

Why not dump the info into a database and have some sort of schedualed job read from it once and a while?
You can always use Quercus which allows you to run PHP in a Tomcat Servlet container.

Web Services is the elegant solution. But in many cases I found much practical to go for a quick-and-dirty solution: start a Java server that communicates using a lightweight communication protocol (none of the heavyweight stuff like XML from Web Services) - example: Apache Thrift. The write a very light client, that takes parameters from command line and writes the output to the console. The client can be in Java or even in other languages, like C++ (Apache Thrift supports that). Then you call the client with system() or with exec() from PHP.
This is not a solution I would ever recommend for production, but it's great for prototyping. Quick and dirty and flexible and extremely modest learning curve (if you already use light-weight communication between your Java processes).

Since you are using PHP4, you may want to just set up a tomcat server that is on a closed network, or just local on the machine of interest, and have it communicate with a servlet, that way you don't have to write a multi-threaded server and deal with creating a communication interface.
If you can upgrade, this page has two other options that may of interest:
http://us3.php.net/manual/en/intro.java.php

Give a look at Quercus
Quercus is Caucho Technology's fast, open-source, 100% Java implementation of the PHP language
I never used it though,

Web Services is the answer. Here's a nice intro link. Your problem is the very reason web services came to the forefront - communication between systems that couldn't ordinarily communicate.
What a web service is essentially going to do is send XML between the PHP and the Java systems. You're going to have to establish an interface for the two, which might be more difficult at the upstart, but you'll reap the benefits later on. In either case, it will be much faster than reading and writing files on the server. Disk I/O are the major bottlenecks on any server.

I may miss something, but if your java program output the needed values, can't you just start the java program from php using exec (http://dk.php.net/manual/en/function.exec.php)

Use the PHP/Java Bridge from sourceforge.net. It is mature, fast and easy to install.

Netty vs Apache MINA

They both provide roughly the same functionality. Which one should I choose to develop my high-performance TCP server? What are the pros & cons?
Reference links:
Apache MINA (source)
Netty (source)

While MINA and Netty have similar ambitions, they are quite different in practice and you should consider your choice carefully. We were lucky in that we had lots of experience with MINA and had the time to play around with Netty. We especially liked the cleaner API and much better documentation. Performance seemed better on paper too. More importantly we knew that Trustin Lee would be on hand to answer any questions we had, and he certainly did that.
We found everything easier in Netty. Period. While we were trying to reimplement the same functionality we already had on MINA, we did so from scratch. By following the excellent documentation and examples we ended up with more functionality in much, much less code.
The Netty Pipeline worked better for us. It is somehow simpler than MINAs, where everything is a handler and it is up to you to decide whether to handle upstream events, downstream events, both or consume more low-level stuff. Gobbling bytes in "replaying" decoders was almost a pleasure. It was also very nice to be able to reconfigure the pipeline on-the-fly so easily.
But the star attraction of Netty, imho, is the ability to create pipeline handlers with a "coverage of one". You've probably read about this coverage annotation already in the documentation, but essentially it gives you state in a single line of code. With no messing around, no session maps, synchronization and stuff like that, we were simply able to declare regular variables (say, "username") and use them.
But then we hit a roadblock. We already had a multi-protocol server under MINA, in which our application protocol ran over TCP/IP, HTTP and UDP. When we switched to Netty we added SSL and HTTPS to the list in a matter of minutes! So far so good, but when it came to UDP we realised that we had slipped up. MINA was very nice to us in that we could treat UDP as a "connected" protocol. Under Netty there is no such abstraction. UDP is connectionless and Netty treats it as such. Netty exposes more of the connectionless nature of UDP at a lower level than MINA does. There are things you can do with UDP under Netty than you can't under the higher-level abstraction that MINA provides, but on which we relied.
It is not so simple to add a "connected UDP" wrapper or something. Given time constraints and on Trustin's advice that the best way to proceed was to implement our own transport provider in Netty, which would not be quick, we had to abandon Netty in the end.
So, look hard at the differences between them and quickly get to a stage where you can test any tricky functionality is working as expected. If you're satisfied that Netty will do the job, then I wouldn't hesitate to go with it over MINA. If you're moving from MINA to Netty then the same applies, but it is worth noting that the two APIs really are significantly different and you should consider a virtual rewrite for Netty - you won't regret it!

Update: Just use Netty. It is now a mature project with all the bells and whistles required for building protocol clients and servers. It has strong community with several active contributors backed by enterprises. It also has a book, 'Netty in Action'. It has been adopted by many many well-known companies and projects already. Unlike Netty, Apache MINA has been under maintenance mode since I left the project.
MINA has more out-of-the-box features at the cost of complexity and relatively poor performance. Some of those features were integrated into the core too tightly to be removed even if they are not needed by a user. In Netty, I tried to address such design issues while retaining the known strengths of MINA.
Currently, most features available in MINA are also available in Netty. In my opinion, Netty has cleaner and more documented API since Netty is the result of trying to rebuild MINA from scratch and address the known issues. If you find that an essential feature is missing, please feel free to post your suggestion to the forum. I'd be glad to address your concern.
It is also important to note that Netty has faster development cycle. Simply, check out the release date of the recent releases. Also, you should consider that MINA team will proceed to a major rewrite, MINA 3, which means they will break the API compatibility completely.

I performance tested 2 "Google Protobuffer RPC" implementations where one was based on Netty (netty-protobuf-rpc) and the other based on mina (protobuf-mina-rpc). Netty ended up being consistently faster ( +- 10% ) for all message sizes - which backs up the overall performance claim on the Netty web site. Since you want to squeeze every bit of efficiency out of your code when you use such an RPC library, i ended up writing protobuf-rpc-pro based on Netty. I have used MINA in the past, but find their documentation of the 2.0 stuff has big holes, and the breaking of API backward compatibility a big minus.

MINA and Netty were initially designed and build by the same author. That's why they are so similiar to each other.
MINA is designed at a slightly higher level with a little more features, while Netty is a little faster.
I think that there's not much difference at all, the basic concepts are the same.

In Netty site you can find some performance reports. As expected :-) they point out Netty as the framework with the best performance.
I never used Netty, but I already used MINA to implement a TCP protocol. The implementation of encoding and decoding was easy, but the implementation of the state machine was not so easy. MINA provides some classes to aid you when implementing the state machine, but I found them kind of hard to use. In the end we decided to ditch MINA and implement the protocol from scratch, and surprisingly we ended with a faster server.

I use Netty.
Twitter also chose Netty to build its new Search System and made it 3x faster.
Ref: Twitter Search is Now 3x Faster
We chose Netty over some of its other competitors, like Mina and Jetty, because it has a cleaner API, better documentation and, more importantly, because several other projects at Twitter are using this framework.

I've only ever used MINA to build a small http like server. The biggest problems I've run into with it so far:
It will hold your "request" and "response" in memory. This is only an issue because the protocol I choose to use is http. You could use your own protocol however to get around this.
No option to provide a stream off disk in case you want to serve up large files. Again can be worked around by implementing your own protocol
Nice things about it:
Can handle a lot of connections
If you choose to implement some sort of distributed work system then knowing when one of your nodes goes down and loses connection is useful for restarting the work on another node.