I am trying to implement a client-server in java
and i made connection between in sockets
and sending JSON objects as strings on streams
if i have big object is there's a way to handle it
so i don't have to regroup it because the limit size of tcp packet (cant know when the single object is fully transferred to me as client or not yet)
note :am using G-son to convert objects to JSON objects
If I have big object is there's a way to handle it so I don't have to regroup it because the limit size of tcp packet. (I cant know when the single object is fully transferred to me as client or not yet)
Actually, the client can know when it has received a complete JSON object. When your client sees the } that matches the opening {, you have the complete object. Of course, this means that that your client needs to understand JSON syntax, but you can use an off-the-shelf JSON parser to do that.
So the best way to do this is for the server to generate and send the JSON, and the client to parse the socket input stream using a normal JSON parser. If you do it that way, then you don't need to know whether the TCP/IP stack has broken the data stream into multiple packets. By the time the JSON parser sees them, they will have been reassembled into a stream of bytes.
If this doesn't answer your question, we need to see what your code is currently doing to generate and send the JSON on the server side.
is there's a way to handle it so i don't have to regroup it because the limit size of tcp packet
You don't have to care about the size of TCP packets. Just write the data. TCP will segmentize and packetize it for you.
(cant know when the single object is fully transferred to me as client or not yet)
Yes you can. You reach the closing '}', as #StephenC mentions. Your JSON parser should take of that for you in any case.
Your question is founded on false assumptions.
Related
I'm using Netty to implement a client/server application, I'm also using Gson to send data from/to the client in json format and convert it from/to a java POJO.
The problem is that if the data exceeds a certain size the message will be truncated and will not be used in the program. So I'm trying to find a compressed format (better than the json provided by the Gson library) or maybe a way to compress the json string and avoid having the messages truncated.
Any help will be appreciated..
If protocol you are using is TCP/IP, you don't have guarantee, that message you send will came in one part. You should put some date to your message, which will allow client to determine if it got whole message (e.g. you can put message length in the begining of the message or some delimiter on the end of the message).
On the client side you should check if whole message came, and if not you should wait for the rest of the message. If you are using netty on the client side, you should put frame decoder in the begining of channel pipeline (e.g. DelimiterBasedFrameDecoder in case of delimiter, LengthFieldBasedFrameDecoder in case of length field).
So, for the past few weeks I've been learning very simple network programming and protocol buffers. Right now, I have a Java client and a C# server that are communicating back and forth using the protocol buffers. It's all working fine, but to make it work on the client (Java) side I had to create my byte array with the exact size of the incoming message or else the parser would throw an error of "Protocol message contained an invalid tag (zero)"
After doing some research, I came to find out that the array I had created (1024bytes) for my DatagramPacket had tons of trailing zeros (since my incoming data from the server was 27bytes long), and that's why I now, as previously mentioned, have to create the array with the exact size of the incoming data.
As for the question, is there any way to find out the size of all of my proto "messages" in my .proto files? If there isn't some sort of static getSize(), is there a way I can calculate that just by the types of fields I have within the "message"?
My message I'm using right now contains 3 doubles, and now that I'm thinking about it, but I want a for sure answer from someone who knows what's going on, is it 27 because 8bytes per double and the 1byte per "tag" on each message field?
The root object in protobuf data is not self-terminating; it is designed to be appendable (with append===merge), so normally the library simply reads until it runs out of data. If you have spare zeros, it will fail to parse the next field header. There are two ways of addressing this:
if you only want to send one message, simply close the outbound socket at the end of you message; the client should detect the end of the socket and compensate accordingly (note, you still don't want to use an oversized buffer unless you are using a length-limited stream wrapper)
use some kind of "framing" protocol; the simplest of which is simply to prefix each message with the number of bytes in that message (note that in the general case this size is not fixed, but in the case off 3 doubles, each with a field-header of a field-number no-greater-than 16, then yes: it will be 27 bytes); you would then either create the buffer the right size (noting that repeated array allocations can be expensive), or more typically: use a length-limited stream wrapper, or a memory-backed in-memory stream
I believe your problem lies in your socket receive code. Having an array with trailing zeroes is not a problem, but when receiving you should check the number of bytes received (it is the return value of a receive call) and only consider the bytes of the buffer array from the beginning up to "bytes received".
I wrote a client-server pair for a 2D top down RPG game using sockets. However, I'm not clear on how exactly to transmit what the user wants to do.
Would I simply transmit a string like "Move north" and then use switch statements to decide what action to take or is there a more elegant solution?
If both client and server are java programs, then I would go with Command pattern where each command class does a specific job (like "Move north" in your case) and the command objects are exchanged between the client and the server.
We used to use this approach in our enterprise application (though not a game), while implementing multi-master replication and it was quite extendible without any if/else chains or switch statements anywhere, just receive command and call command.execute() without worrying about what the command is.
Yeah, basically. You can transmit strings, integers, whatever you want. You basically write the commands to the OutputStream you can get from the network socket on the client side, and read them in on the InputStream on the server side.
You can wrap the Input/OutputStream objects that come from a Socket with other stream types, such as ObjectOutputStream / ObjectInputStream, which allow you to read/write full objects and primitive types directly to the stream, the same way you would read/write data to a file on your file system using these same stream types.
Reading/writing to the network is really no different than read/writing from/to any other place, except that in networking you have to first establish a connection via a socket. Once you've got that socket and a proper stream type setup you just read/write data the same way you would anywhere else in your app.
Instead of raw parsing from stream, I think you should choose a Serialize library to do this.
I suggest Protocol Buffers or FlatBuffers.
Just transmit bytes array and they will parse to Object for you.
I'm currently trying to find out whether it's a good idea to transfer rather large byte arrays (<50MB) over RMI.
I read that it is slow and the data needs to be hold in memory both on the client and the server. This could result into a problem when there are multiple calls.
Are there any (simple) alternatives to this?
RMIIO lets you stream objects in chunked fashion.
EDIT : you can also use KRYO to serialize and compress the object to send across the wire.
RMI is intended to transfer objects. If you have a byte array object on the server and want it on the client you must have it both places until it has been delivered successfully (then you can let the original go away).
A more reasonable approach might be repeated calls populating a remote object transferring only a small chunk at a time. This will then in turn require multiple trips making it slower.
What is the actual (non-technical) problem you want to solve?
Consider java streams which support compression to send/receive large amounts of data.
For instance GZipOutputStream to send data and GZipInputStream to receive sent data block.
It's a very bad idea. The byte array has to be formed in memory before calling the remote method; then it has to be transmitted in the call; then it has to be read by the server; then it has to exist in the server; then it can be processed by the server. You never want to deal with data items this large in a single chunk. It wastes both time and space. Consider a streaming API where you can use moderate sized buffers at both ends; send the data in chunks that are convenient to the sender; and receive it in chunks that are convenient to the receiver.
hi every body could you please help me . I write java code for sending string msg between client and server using udp socket . but I want to to send real time voice so could you please give some notes to do it
I can point you a little of the way, you probably would want to use the Real-time Transport Protocol (RTP), which is more or less the standard for sending audio or video real time over the net. However the implementation is not straight forward, and you should use a helper library like jlibrtp for the implementation. There is also a RTP packetizer in Java Media Framework (JMF), but you don't wanna go there....
UDP has no quality of service guarantee, so when sending your packets of data you will need to add some sort of order number to your data to detremine how to put the data back together. For example you could send 3 datagram packets in order from the server, yet the client may get them in a different order (2,1,3). Or it may not get one of them at all, in which case you either want it resent (doubtful) or simply ignore it and move on at some timeout.
Look into using Real Time Protocol RFC3550 (http://en.wikipedia.org/wiki/Real-time_Transport_Protocol)
as the transport over UDP. RTCP as the control over TCP.