I need to communicate between server/client. I saw that CORBA is used for different languages to work like RMI, is it?
In my application I will have to transfer objects between client/server, transfer binary files (which I saw that I can do with RMI) and also play live streaming from one client to another.
I was thinking about CORBA because it also can be used with C++ if I need, isnt it?
So can I play streaming with CORBA?
RMI and CORBA are technologies for distributed objects. You then invoke methods on a remote object the same way as on a local object.
Sure, you can send and receive bytes if you implement methods that do so (e.g. void sendChunk(byte[] data)). But I wouldn't consider them appropriate for streaming. Also for streaming, you should pick something to address the quality of service of the stream -- which RMI or CORBA definitively don't do. For that I would maybe have a look at UDP sockets, or something like that, which just drops packets if the channel is saturated.
CORBA provides you a lot of services and it possibly is not the best of the options for streaming media. Two reasons I can think of (though one can find more reasons against too)
The object payload is more than just the data (marshalling and unmarshalling)
CORBA (specifically the implementations) generally strive for a good QoS aka there will be retries for the same call
That said, it has been demonstrated that ORBs can work with real-time communication too. So, CORBA as a framework is not completely off the table.
I am not sure of the multicast communication capabilities of CORBA though.
If you're hell bent on using CORBA to address this, have a look at RT-CORBA (Real-time corba). I believe TAO has an implementation of it, however I've never used RT-CORBA so I can't speak first-hand if it'll give you the performance you'll require for streaming.
Related
I have the following situation:
I have 2 JVM processes (really 2 java processes running separately, not 2 threads) running on a local machine. Let's call them ProcessA an ProcessB.
I want them to communicate (exchange data) with one another (e.g. ProcessA sends a message to ProcessB to do something).
Now, I work around this issue by writing a temporary file and these process periodically scan this file to get message. I think this solution is not so good.
What would be a better alternative to achieve what I want?
Multiple options for IPC:
Socket-Based (Bare-Bones) Networking
not necessarily hard, but:
might be verbose for not much,
might offer more surface for bugs, as you write more code.
you could rely on existing frameworks, like Netty
RMI
Technically, that's also network communication, but that's transparent for you.
Fully-fledged Message Passing Architectures
usually built on either RMI or network communications as well, but with support for complicated conversations and workflows
might be too heavy-weight for something simple
frameworks like ActiveMQ or JBoss Messaging
Java Management Extensions (JMX)
more meant for JVM management and monitoring, but could help to implement what you want if you mostly want to have one process query another for data, or send it some request for an action, if they aren't too complex
also works over RMI (amongst other possible protocols)
not so simple to wrap your head around at first, but actually rather simple to use
File-sharing / File-locking
that's what you're doing right now
it's doable, but comes with a lot of problems to handle
Signals
You can simply send signals to your other project
However, it's fairly limited and requires you to implement a translation layer (it is doable, though, but a rather crazy idea to toy with than anything serious.
Without more details, a bare-bone network-based IPC approach seems the best, as it's the:
most extensible (in terms of adding new features and workflows to your
most lightweight (in terms of memory footprint for your app)
most simple (in terms of design)
most educative (in terms of learning how to implement IPC). (as you mentioned "socket is hard" in a comment, and it really is not and should be something you work on)
That being said, based on your example (simply requesting the other process to do an action), JMX could also be good enough for you.
I've added a library on github called Mappedbus (http://github.com/caplogic/mappedbus) which enable two (or many more) Java processes/JVMs to communicate by exchanging messages. The library uses a memory mapped file and makes use of fetch-and-add and volatile read/writes to synchronize the different readers and writers. I've measured the throughput between two processes using this library to 40 million messages/s with an average latency of 25 ns for reading/writing a single message.
What you are looking for is inter-process communication. Java provides a simple IPC framework in the form of Java RMI API. There are several other mechanisms for inter-process communication such as pipes, sockets, message queues (these are all concepts, obviously, so there are frameworks that implement these).
I think in your case Java RMI or a simple custom socket implementation should suffice.
Sockets with DataInput(Output)Stream, to send java objects back and forth. This is easier than using disk file, and much easier than Netty.
I tend to use jGroup to form local clusters between processes. It works for nodes (aka processes) on the same machine, within the same JVM or even across different servers.
Once you understand the basics it is easy working with it and having the options to actually run two or more processes in the same JVM makes it easy to test those processes easily.
The overhead and latency is minimal if both are on the same machine (usually only a TCP rountrip of about >100ns per action).
socket may be a better choice, I think.
Back in 2004 I implement code which do the job with sockets. Until then, many times I search for a better solution, because socket approach triggers firewall and my clients worry. There is no better solution until now. Client must serialize your data, send and server must receive and unserialize.
It is easy.
I'm planning on building a Java server that will handle real time game communications between clients. What is the best type of Java implementation out there that could efficiently and, hopefully, accurately communicate between a client and server at high speeds (say 5-15 packets per second)? I know there are many types of Java networking APIs (ie. ObjectInputStream and ObjectOutputStream, DatagramPacket, KyroNet, etc.), but I'm not sure what is the most effective and/or commonly used implementation for such a scenario. I would assume that most real time games use UDP communication methods, but I understand the reliability issues that come with it. Are there UDP implementations that have some form of flow control? Anyway, thanks in advance!
A few things to consider:
Java NIO is really good, and can handle the kind of throughput/latency you are looking for. Don't use any of the older networking / serialization frameworks and APIs
Latency is really important. You basically want a minimal layer over NIO that allows you to send very fast, small, inidividual messages with minimal overhead.
Depending on the game, you may want TCP or UDP or both. Use TCP for important messages, UDP for messages that aren't strictly necessary for the game to proceed or will be subsumed by a future update (e.g. position updates in a FPS)
Some people implement their own TCP-like messaging protocol over UDP for real time games. This is probably more hassle than it's worth, but be aware of it as an option if you really need to optimise for a specific type of communication
For real time games, you are nearly always doing custom serialisation (e.g. only sending deltas rather than full updates of object positions) - so make sure your framework allows this
Given this, I'd recommend one of the following
Kryonet - lightwieght, customisable, designed for this kind of purpose
Netty - slightly more enterprise-oriented, but very capable, robust and scalable
Roll-your-own based on NIO - tricky but possible if you want really fine grained control. I've done this before, but in retrospect I probably should have picked Kryonet or Netty
Good luck!
Immidiately forget ObjectOutputStream and ObjectInputStream. These are the standard output-input mechanisms of the old standard java serialization, which is slow and produces bloat objects. Some resources to start with:
http://code.google.com/p/kryonet/
http://code.google.com/p/pyronet/
I have done some searching but haven't come up with anything on this topic. I was wondering if anyone has ever compared (to some degree) the performance difference between an RPC over a socket and a REST web service. If both do the same thing, which would have a tendency to be the better performer? I've already started building some socket code and would like to know if REST would give better performance before I progress much further. Any input would be really appreciated. Thanks indeed
RMI
Feels like a local API, much like
XMLRPC
Can provide some fairly nice remote
exception data
Java specific means this causes lock
in and limits your options
Has horrible versioning problems
between different versions of clients
Skeleton files must be compiled in
like CORBA, which is not very flexible
REST:
easy to route around firewalls
useful for uploading files as it can
be rather lightweight
very simple if you just want to shove
simple things at something and get
back an integer (like for uploaders)
easy to proxy security behind Apache
and let it take the heat
does not define any standard format
for the way the data is being
exchanged (could be JSON, YAML 1.0,
YAML 2.0, arbitrary XML format, etc)
does not define any convention about
having remote faults sent back to the
caller, integer codes are frequently
used, but method of sending back data
is not defined. Ideally this would be
standardized.
may require a lot of work on the
client side caller of the library to
make use of data (custom serialization
and so forth)
In short from here
web services do allow a loosely
coupled architecture. With RMI, you
have to make sure that the objects
stay in sync in all applications
RMI works best for smaller
applications, that are not
internet-related and thus not scalable
Its hard to imagine that REST is faster than a simple socket connection given it also goes over a Socket.
However REST may be performant enough, standard and easier to use. I would test whether REST is fast enough and meets your requirements first (or one of the many other existing solutions) before attempting your own Socket solution.
I need to transfer files fast over the Internet from a Java server to C++ clients, where often many clients would need the same files. I was looking at say transferTo() in Java which sounds like it would be a decently optimized function to send files. However, I'm not sure when I use transferTo() how to best receive that in C++ (i.e. is it just a raw data transfer, how do I determine when the file is over on the client side, etc.). I need this to work on both Windows and Linux. Also, other than transferTo(), would there be some way to be more efficient, especially by taking advantage of the fact that many clients will usually need the same files? I'm not sure how to do say multicast etc. Also, I'm using application-level security rather than a VPN, and on the Java server, encrypting with AES and using MAC digital signing, so I'm also looking for a cross-platform library recommendation to deal with the crypto on the C++ side with minimal pain.
I'm very proficient in C++ but have no previous experience with network programming, so please consider than in any suggestions.
Thanks.
An embedded webserver? http-transfers are efficient enough for you?
The simplest embeddable Java webserver I remember seeing is http://acme.com/java/software/Acme.Serve.Serve.html. We use embedded Jetty 6 in production at work, but that takes more elbow grease.
If your clients doesn't know where to find your webserver in the first place, consider announcing using Zeroconf. http://jmdns.sourceforge.net/
For scalability reasons, Thorbjørns suggestion of using http seems like a very good idea as it would allow you to easily set up http proxies for caching, use standard load balancing tools and so forth.
If you are looking to transfer more than just a blob of data, you might want to have a look at googles protocol buffers. They allow for very easy and fast encoding/decoding on the java and c++ end.
Consider chunking the file and sending via UDP datagram. C++ can re-compile as it receives it. Have you considered implementing/embedding an existing P2P protocol implementation?
If you need effecient transfer to many clients then your bottleneck is the server.
For this please look at the bit-torrent protocol as it distributes the transfer between the clients.
I know that the Java can use the Socket Programming to send an Object. Apart from socket programming, anything other way to do it?
Java's Remote Method Invocation (RMI) is probably the easiest and most widely supported way.
Java's Advanced Socket Programming describes marshalling objects over a socket.
Control a high-speed robot hand to type it in.
Pretty well every other thing you can do will be a layer built on sockets.
RFC1149!
Attach rubber band to computer A
Put Object in rubber band.
Pull back, aim at Computer B.
Let go.
Via web service for example. But its build on top on sockets again.
SOAP (Simple Object Access Protocol).
Serialize the object, write to a file, copy the file if the computers are network connected if not use a removable disk and deserialize it.
Objects can be transferred via a shared database.
I work near a production system that's been doing this for 10 years.
Ironically, except in rare cases, DB connections are also implemented via sockets.
No. Sockets are how computers communicate. Other than writing the data to some media and physically transporting it between computers, you will have to use sockets.
At the lowest level, data is transferred over sockets as bytes. So first you need to serialize your object to bytes, then you can send it, then on the other side you need to deserialize the object from the bytes.
Approaching your question less literally, there are Java libraries that handle the serialization automatically and hide the nastiness of dealing directly with sockets. I recommend KryoNet. KryoNet can do remote method invocations, and a lot simpler and more efficiently than Java's built-in RMI support.