I'm a networking newbie and have explored a few frameworks such as Jersey and CXF for building RESTful web services in Java.
I have a new application that requires sending large amounts of binary data over the internet (between client and servers), and it needs to be extremely fast. I'm wondering if there is such a thing as a pure "TCP/IP (non-)web service", and if there are any open source Java libraries for building such things.
If all network services have to sit on top of TCP/IP, then I guess I'm looking for something that still uses binary data, but that introduces extremely little overhead for speedy service.
I always associated REST with XML or JSON; if it can be configured to be super-fast and work with binary data, I'd even be into that since I'm already somewhat familiar with Jersey.
I thought RMI might be a good choice, but not sure if it's not appropriate for this use case.
I need speed and I need a binary protocol, and not sure where to start. Any ideas? Thanks in advance!
Can´t you go with Java Sockets?
That is how low level you can go.
http://docs.oracle.com/javase/tutorial/networking/sockets/
Yes RMI is way faster than any high level transport abstraction like CFX. You have two options, raw sockets or RMI, RMI is easier to work with.
REST is an architectural pattern whose distinctive trait is using the HTTP verbs for what they were originally intended to. Even if the majority of deployed REST services use JSON to serialize objects, the entire HTTP transaction looks like this:
PUT /user/foo HTTP/1.1
Host: example.com
X-Auth-Token: foobarbazanythingyouwanthere
{"fist": "Jeff", "last": "Atwood"}
And the response is:
HTTP/1.1 200 OK
So, there is no real "overhead". Instead, you exploit the capabilities of existing libraries to deal with routing, authentication, serialization, and so on. Since HTTP uses MIME-like envelops, it has no problem with binary content, and even if this adds some overhead, it's very difficult to come up with a super-efficient, yet effective protcol, not to mention that you need to design it, implement all the libraries and -well- nobody else will ever be interested in your work.
Since you state you are a "network newbie", and since you seem to use terms like REST and RMI in a random manner, my strong suggestion is not thinking about performance and just go with standard technologies. You can use HTTP, there are lots of pre-built framework, servlet containers, server stubs, you name it.
Come back when you measure real performance bottlenecks, so that we'll have the figures to better assist you.
Related
I need to interact with a server over TCP/IP with a basic message/response protocol (so for each request I shall receive a defined response).
In the JVM ecosystem, I think Java Socket was the tool to use 15 years ago, but I'm wondering if there is anything more suitable nowadays in the JDK? For example, with Java Sockets one still needs to manually timeout a request if no answer is received, which feels really old fashioned.
So is there anything new in the JDK or JVM universe?
No, there are much better option nowadays which allow you to implement your client/server asynchronously without additional threading.
Look at SocketChannel from nio or even better AsynchronousSocketChannel from nio2. Check this tutorial
Especially the latter option will allow you to just start the connection listener and register callback which will be called whenever new connection is requested, data arrived, data was written etc.
Also consider looking at some high level solutions like Netty. It will take care of the network core, distribute load evenly to executors. Additionally it provides clears separation of the network layer and processing layer. For the processing layer it will provide you with lot of reusable codecs for common protocols.
You can try RMI which works on top of TCP/IP but hides all the hardwork with a convenient set of APIs.
Follow this tutorial post
Well, there are really a lot of other technologies to use, for example JMS has various implementations which work out of the box.
Sockets are low-level building blocks of network communications, like wires in the electricity network of your house. Yes, they're old fashioned, yes, we likely don't want to see them, but they're there and they will stay there for a good reason.
On top of Sockets, you can e.g. pick the HTTPUrlConnection, which implements most of the HTTP protocol. Still, setting timeout policies are in your hands, which I find quite useful, and extremelly painful at the same time.
http://www.mkyong.com/java/how-to-send-http-request-getpost-in-java/
You are free to move one abstraction level above, and use a ready-made REST library, such as this: http://unirest.io/java.html
The example above connects to a server, configures a HTTP query string, perform the request (timeout, encodings, all the mess under the hood), and finally get the response in Json format in a few lines:
Unirest.post("http://httpbin.org/post")
.queryString("name", "Mark")
.field("last", "Polo")
.asJson();
Nowadays a vast amount of web services are available using REST protocol, which is a simple command-response over HTTP. If you have a chance, I'd suggest using REST, as you can easily find available client and server side implementations, and you don't need to reinvent the wheel on the command-protocol layer either.
On client side, unirest is quite convenient. On the server side, we have had really great experience in the 1.2.xx series of Play! framework. But there are thousands of these things out there, just search for "REST".
Take a look to Netty Project, "Netty is a NIO client server framework which enables quick and easy development of network applications such as protocol servers and clients. It greatly simplifies and streamlines network programming such as TCP and UDP socket server."
This framework give us a lot of capabilities that simplify the programming process, allowing a big scalability.
Is used by Twitter and a lot of big companies in the tecnology industry.
This is a nice presentation from Norman Maurer.
I have done some searching but haven't come up with anything on this topic. I was wondering if anyone has ever compared (to some degree) the performance difference between an RPC over a socket and a REST web service. If both do the same thing, which would have a tendency to be the better performer? I've already started building some socket code and would like to know if REST would give better performance before I progress much further. Any input would be really appreciated. Thanks indeed
RMI
Feels like a local API, much like
XMLRPC
Can provide some fairly nice remote
exception data
Java specific means this causes lock
in and limits your options
Has horrible versioning problems
between different versions of clients
Skeleton files must be compiled in
like CORBA, which is not very flexible
REST:
easy to route around firewalls
useful for uploading files as it can
be rather lightweight
very simple if you just want to shove
simple things at something and get
back an integer (like for uploaders)
easy to proxy security behind Apache
and let it take the heat
does not define any standard format
for the way the data is being
exchanged (could be JSON, YAML 1.0,
YAML 2.0, arbitrary XML format, etc)
does not define any convention about
having remote faults sent back to the
caller, integer codes are frequently
used, but method of sending back data
is not defined. Ideally this would be
standardized.
may require a lot of work on the
client side caller of the library to
make use of data (custom serialization
and so forth)
In short from here
web services do allow a loosely
coupled architecture. With RMI, you
have to make sure that the objects
stay in sync in all applications
RMI works best for smaller
applications, that are not
internet-related and thus not scalable
Its hard to imagine that REST is faster than a simple socket connection given it also goes over a Socket.
However REST may be performant enough, standard and easier to use. I would test whether REST is fast enough and meets your requirements first (or one of the many other existing solutions) before attempting your own Socket solution.
I have a background in protocol stack - for 3g handsets. Now I need to communicate from an Android slate PC to a server, and I will code both sides of the interfacet.
Update: I ought to have said, from Android (multiple) slate to local server (multiple), then over satellite to a single central master server.
And now I think that I might not be implementing, just designing so looking for something easy for junior engineers to handle
SOAP looks ok, but are their any good IDEs to develop something for SOAP (or GSOAP) for Android (not sure yet which o/s the server will run; with luck I will get to choose).
Or should I just roll my own and use TCP/IP? (I have feeling, which I can't justify, SOAP might be quicker to develop and easier for others to maintain).
If I roll my own, I can just use C or C++ at both ends. IF SOAP, can I use C/C++ on Android (I know that I can if it's non-SOAP), or do I go with Java? And, if I do, should I then go with Java at the server, for maintainability?
Final note: I guess that SOAP will add an overhead and I will be doing this over a satellite link, where every byte costs.
Does any of that make sense, or do I need to explain it better?
Don't use SOAP if you control both ends. Think about JSON (or JSON-RPC) over HTTP or roll your own using prtocol buffers
SOAP is OK for B2B (remember that term, eh?) communications, but is very heavy and not pleasant to deal with. It might make sense to expose a SOAP interface to the outside world, especially in a Microsoft environment, but it is not ideal for internal use.
UPDATE:
If using SOAP both ends do not have to be the same language/library. There are some quirks, I'm sure, but generally interoperability while not great, exists. So you can use Java on the device and C++ on the server, or the other way around, doesn't matter. Regarding IDE -- for Java Eclipse has some relatively bad but usable plugins.
Now on to the satellite links... ARE YOU KIDDING ME? Seriously, depends on the size of your objects. If your requests and responses are going to be big, it might be tolerable... maybe. But if you are going to do a lot of small requests with short responses you are looking at 90% overhead. Check out these sample SOAP messages: http://www.w3schools.com/soap/soap_example.asp
Does anyone have a real-world experience in building such a project? I'd like to move away questions about "is it good idea or not", but focus on possible solutions. I see one simple way - HTTP GET/POST + xml/json - and one more elegant - AJAX/DWR. As for the first one - I understand that it is possible, but needs quite a lot coding. As for second way - is it possible to use Java DWR engine with PHP front-end? Is DWR language-independent for client side (as it uses just JavaScript)?
Would it be a problem, that client page was generated by one web server (for example, apache+php) and served on server-side by another (for example, tomcat)? I suspect, that Tomcat will complain about sessions. Can this problem be fixed with allowing cross-domain AJAX?
Thank you in advance.
Denis.
If what you want to do is (as I suspect) to use PHP to assemble your web pages while the "business logic" is written in Java, I would suggest using PHP/Java Bridge (LGPL and MIT licenses)
Both Java and PHP are server-side technologies. Your "front-end" will be written using HTML, CSS, and JavaScript - although you could certainly use PHP (or JSP) templates to render portions of the front-end.
If you are using PHP as the "front-end", then you would need it to act as a proxy, passing requests back to the Java web server.
I've worked on a project that uses a Java 'backend' and a mod_perl 'frontend'. For the naysayers, this is because the Java is providing service/API facilities, it's not and shouldn't be involved in dealing with UI, be they HTML, WAP, SMTP, SOAP, etc.
For historical reasons the mod_perl talks XML-RPC. It's not a route I'd recommend at this stage. Java, Perl and PHP can quite happily handle far more JSON type transactions due to lower encoding/decoding overhead. Also, in a mod_perl (though not PHP) environment it's possible to run JSON-RPC easily over a persistent connection, reducing the overhead even further.
There are plenty of benefits to this approach, including separate upgrades to the various UIs, stability of the service layer, and distinct responsibilities for each layer.
Downsides include delays getting service improvements live, more complicated development, staging and test environments, a taller barrier to entry for new developers, more documentation and management.
For the "Java front to back" guys, this is a similar type approach to using an OSGi container, only using more domain suitable languages; Java for heavy lifting, scripts for more fluid, text based interfaces.
I have a bunch of Java code which was written using the Hibernate framework, originally destined to have a front end written using JSPs. However, the requirements for the front end have changed, and we've decided that a desktop client (which will be written in .NET) is a better match for our users.
I don't really want to waste the code that's already been written - can anybody suggest a good set of tools for writing a document-based web services interface that we will be able to access from .NET?
Thanks,
Jim
If you truly want a document based service interface (rather than an RPC style web service architecture), your best bet is going to be creating a SOAP based web service interface.
A quick glance at the Java site shows that the Metro stack might help a bit:
Java Web Services at a Glance
We're developing an application with the exact architecture you describe for a finance application. We reviewed several different options, and have finally landed on using compressed CSV over HTTP.
CSV was chosen since the vast majority of data was going to be displayed in a grid on the front end, we had very large result sets >250k rows on a regular basis, and it compresses really really well.
We also looked at using:
ICE, but declined on that due to licensing costs and the need to reinvent so much.
Google's protocol buffers via servlets, but declined on that due to lack of C# support (as of last fall).
Compressed XML using WOX, but declined on that due to lock-in to a small thesis project for support and XML being too verbose.
The industry supports a couple of different options as well:
SOAP, but that has its own well documented issues.
IIOP, J-Integra has a product called Espresso which will allow you to do RMI from a front end.
I'd personally use some lightweight RPC protocol, be it XML-RPC or a homegrown one. SOAP, IMO, is way too fat and is not as interoperable as it's supposed to be. The simpler the better.
We have a quite large application using a Java RMI server and IIOP.NET for interoperability. We have used IIOP.NET with the Sun RMI and the Bea Weblogic (now Oracle) without major issues.