Theory:
Let's say I have an application A, written in Java, that uses a TCP stream for client/server communication (it's on the client end in the relationship). Now, purely as an experiment, I am trying to create an application B, written in VB.NET, that would serve as a proxy for application A's network stream, allowing app B to read and write to the stream.
Is it, at all, possible to access such a network stream from another application, also taking the language boundary into account?
Your question is pretty vague, but if you're asking about the possibility of making a proxy server, then yes, it's possible. The language doesn't matter, but the interface does (the way that the content in the stream is encoded). For instance, Java typically serializes things into a stream using big endian (most significant byte of each byte sequence sent first), whereas .NET uses little endian (least significant byte of each byte sequence sent first). Again though, as long as you're aware of how the data is actually encoded into those streams, you can write a decent proxy server. If all that your proxy server will be doing is passing along data without caring what the data is, then you can just read a byte from one stream and write it to the other. But if you're actually reading values (integers, strings, pictures, etc.), then you will be dealing with the endianness issues, because Java and VB.NET's default stream readers will read and write integers differently, etc.
There will be some complications if you want to actually edit the data instead of simply passing it along. You'll have to deal with the client's and server's reactions to strange network behavior. For instance, if Client A is a video game, and Proxy B injects a message to the server to "join the game", then you'll have to deal with the fact that the server is going to send "ok, you've joined the game". When the client receives that message, it will most likely ignore it, because it had no knowledge that the proxy tried to join the game on its behalf, and will just assume the server made a mistake.
Related
I am studying about the Interprocess Communication Methods in the course Operating System Concept.
I don't really understand the mechanism in transferring data. In the case of pipe method, a conduit will be created between 2 process to transfer byte streams , right?
And how about Serialization?
I know Serialization is the method to convert an object into byte stream to transfer and we can rebuild the object when it reached the destination.
So in which case we use Serialzation or Pipe to transfer data?
What is the advantages and the disadvantages between them?
Can anyone explain to me a very deep mechanism in transferring data of these methods? And are these mechanisms different between Java and C? , or it is the same?
Thanks in advanced.
There are two basic types of pipe in UNIX/Linux: a named pipe and an anonymous one.
An anonymous pipe is created by the "pipe()" system call, which returns 2 file descriptors associated with a newly created pipe, one for writing data, the other for reading from it. The shell uses anonymous pipes to connect the standard output of one process to the standard input of another when you connect two process with the "|" operator.
A named pipe appears as a file in the file system, and can be opened with the normal "open()" system call.
In blocking mode (the default), the process that reads from the pipe will block until data appears there; the writer can then send data which will appear as a byte stream to the reader.
The important fact here is that the data that is transferred is a byte stream. The sender and receiver of the data must agree on a protocol to determine how to interpret the bytes. One typical method for this is serialization. Consider a 32 bit integer ... 4 bytes. Some systems store those bytes with the most significant bit in the first byte (known as big-endian), some store the least significant bit in the first byte (little-endian system, such as x86). When transmitting such data across a network, serialization of such data is important, since it is entirely possible that each end stores the data in a different order.
But even when transmitting data between two processes on the same host, serialization helps. It can be used to encapsulate objects so that the receiver knows when it has received everything. For example, with our 32 bit integer, if the receiver doesn't know it is expecting an integer, and gets 3 bytes (the 4th having been delayed by some scheduling), it must know that it needs to wait before continuing.
None of this is particular language specific, save that some languages have built in support for serialization. Java is one such language (see ObjectInputStream and ObjectOutputStream). If you are trying to move data between Java and C programs, and on the Java side you want to use these classes, then you'll need to understand the serialization protocol used by them.
Another common serialization technique is JSON (JavaScript Object Notation), for which there exists several good libraries in C and Java.
I don't really understand the mechanism in transferring data. In the case of pipe method, a conduit will be created between 2 process to transfer byte streams , right?
A named or anonymous pipe is a stream rather like a socket connection over loop back. In fact in some OSes, it is implemented by the same drivers/library.
And how about Serialization?
How serialization is done is not a language specific and you can serialize data in a manner which can be shared between C and Java.
What is the advantages and the disadvantages between them?
There is many forms of serialization and this is too broad a topic to cover in one answer. You could do an entire thesis on it.
Can explain one explain to me a very deep mechanism in transferring data of these methods?
There isn't much to it. A block of data is copied to memory managed by the OS and this buffered data can be read by another program (or the same one)
And are these mechanisms different between Java and C? , or it is the same?
They both use the same OS calls to do the real work. The Java API hides this fact from you and makes it more Java friendly, but they are the same.
I wrote a client-server pair for a 2D top down RPG game using sockets. However, I'm not clear on how exactly to transmit what the user wants to do.
Would I simply transmit a string like "Move north" and then use switch statements to decide what action to take or is there a more elegant solution?
If both client and server are java programs, then I would go with Command pattern where each command class does a specific job (like "Move north" in your case) and the command objects are exchanged between the client and the server.
We used to use this approach in our enterprise application (though not a game), while implementing multi-master replication and it was quite extendible without any if/else chains or switch statements anywhere, just receive command and call command.execute() without worrying about what the command is.
Yeah, basically. You can transmit strings, integers, whatever you want. You basically write the commands to the OutputStream you can get from the network socket on the client side, and read them in on the InputStream on the server side.
You can wrap the Input/OutputStream objects that come from a Socket with other stream types, such as ObjectOutputStream / ObjectInputStream, which allow you to read/write full objects and primitive types directly to the stream, the same way you would read/write data to a file on your file system using these same stream types.
Reading/writing to the network is really no different than read/writing from/to any other place, except that in networking you have to first establish a connection via a socket. Once you've got that socket and a proper stream type setup you just read/write data the same way you would anywhere else in your app.
Instead of raw parsing from stream, I think you should choose a Serialize library to do this.
I suggest Protocol Buffers or FlatBuffers.
Just transmit bytes array and they will parse to Object for you.
I'm currently trying to find out whether it's a good idea to transfer rather large byte arrays (<50MB) over RMI.
I read that it is slow and the data needs to be hold in memory both on the client and the server. This could result into a problem when there are multiple calls.
Are there any (simple) alternatives to this?
RMIIO lets you stream objects in chunked fashion.
EDIT : you can also use KRYO to serialize and compress the object to send across the wire.
RMI is intended to transfer objects. If you have a byte array object on the server and want it on the client you must have it both places until it has been delivered successfully (then you can let the original go away).
A more reasonable approach might be repeated calls populating a remote object transferring only a small chunk at a time. This will then in turn require multiple trips making it slower.
What is the actual (non-technical) problem you want to solve?
Consider java streams which support compression to send/receive large amounts of data.
For instance GZipOutputStream to send data and GZipInputStream to receive sent data block.
It's a very bad idea. The byte array has to be formed in memory before calling the remote method; then it has to be transmitted in the call; then it has to be read by the server; then it has to exist in the server; then it can be processed by the server. You never want to deal with data items this large in a single chunk. It wastes both time and space. Consider a streaming API where you can use moderate sized buffers at both ends; send the data in chunks that are convenient to the sender; and receive it in chunks that are convenient to the receiver.
Let me present my situation.
I have a lot of data in bytes stored in files on server. I am writing and reading this files using AIO that is coming in JDK7. Thus, I am using ByteBuffer(s) for read and write operations.
The question is once I have performed a read on AsynchronousFileChannel I want to transfer the content of the ByteByffer that was used in read operation to the client. Thus I actually want to send the bytes.
What would be the best way to go from here. I don't want to send the ByteBuffer, because I have a pool of them that I reuse, thus this is not an option. I want to be able also to even maybe combine several reads and send the content of several ByteBuffer(s) combined at once.
So what do I send. Just a byte[] array? Or do I need some stream? What be the best solution regarding performance here.
I am using RMI for communication.
Thanx in advance.
You can simulate streams over rmi using the RMIIO library, which will allow you to stream arbitrary amounts of bytes via RMI without causing memory problems on either end.
(disclaimer, i wrote the library)
Unless there is a very good reason not to, then just send the byte array along with sufficient meta data that you can provide reliable service.
The less of the underlying implementation you need to transfer back and forth over RMI, the better. Especially when you work with Java 7 which is not yet generally available.
To use RMI you have to retrieve the contents of the buffer as a byte[], then write it to an ObjectOutputStream (the write happens under the covers). Assuming that you're currently using direct buffers, this means CPU time to create the array in the Java heap, and CPU time to garbage-collect that array once it's been written, and the possibility that the stream will hold onto the reference too long, causing an out-of-memory error.
A better approach, in my opinion, is to open a SocketChannel to the destination and use it to write the buffer's contents. Of course, to make this work you'll need to write additional data describing the size of the buffer, and this will probably evolve into a communication protocol.
Here's my scenario. I have an application written in C++ but not the complete source but the "meat" of it is there. I also have a compiled exe of this application. It communicates to a server somewhere here on our network. I am trying to replicate the C++ code in java, however it uses dwords and memory references, sizeof etc, all things that don't exist in java since it manages it's own memory. It builds this large complex message and then fires it over the network. So I am basically sniffing the traffic and inspecting the packet and trying to hardcode the data it's sending over to see if I can get a response from the server this way. However I can't seem to replicate the message perfectly. Some of it, such as the license code it sends is in "clear hex", that is, hex that translates into ascii, where-as some other portions of the data are not "clear hex" such as "aa" which does not translate into ascii (or at least a common character set?? if that makes any sense I'm not sure).
Ideally I'd like to not do it like this, but it's a stepping stone to see if can get the server to respond to me. One of the functions is for the application to register itself and that's the one I am trying to replicate.
Some of my assumptions above may be wrong, so I apologize in advance. Thanks for your time.
In Java, all "character" data is encoded as Unicode (and not ASCII). So when you talk to something outside, you need to map the internal strings to the outside world. There are several ways to do it:
Use a ByteArrayOutputStream. This is basically a growing buffer of bytes to which you can append. This allows you to build the message using bytes.
Use getBytes(encoding) where encoding is the encoding the other side understands. In your case, that would be "ASCII" for the text parts.
In your case, you probably need both. Create a byte buffer and then append strings and bytes to it and then send the final result (getByteArray()) via the socket API.