I have developed a java/scala XMPP client app that sends data asynchronously using (say) a write method and receives data using a listener method. The listener method receives data as discrete XMPP message packets and processes them using a processPacket method (which I can modify based on what I want to do with the received data)
I want to hook up a 3rd party library that reads data from an inputstream and writes to an outputstream. Specifically, I want the inputstream of the 3rd party library to be emulated using the data received via my listener method and the outputstream to be emulated by my write method.
What would be the easiest way to do this? I know that this requires conversion from a stream to chunks of strings and vice-versa. Some hints would be appreciated.
The XMPP message packet structure is as follows (though this can be changed if needed):
<message to = ... from = ...><body>data</body></message>
Use a ByteArrayInputStream to create an input stream for a given String. You have to think about encoding, because your sending bytes instead of characters.
String text = message.getBody(); // that's what you need?
InputStream is = new ByteArrayInputStream(text.getBytes("UTF-8"));
For the other way round, you write to an ByteArrayOutputStream and create a new String from it's bytes:
String text = new String( baos.toByteArray(), "UTF-8" );
Again - don't forget to think about character encoding.
Related
I'm sending a string over the socket I previously sent a file to, but the recipient reads it as part of the file itself, is there a way to send a sort of EOF before sending the string?
To send the file I'm using
byte[] buffer = new byte[1024];
int count;
while ((count = fis.read(buffer)) >= 0) os.write(buffer, 0, count);
os.flush();
(and almost the same to receive it)
To send the string I'm using OutputStreamWriter
(Here you are my code: hatebin)
I've also read here that I should send a SOH character, but which one should I send and how?
Thanks in advance.
No there's no way to send an "eof" and then send something afterwards.
If you don't want to open a new connection, there are basically two ways to solve this.
You can modify the client so it recognizes some special byte sequence as a "delimiter", and stops writing to the file when it reads the delimiter from the socket. In this case you need to have some strategy to deal with the possibility that the file actually contains the delimiter.
You can send the size of the file in bytes before sending the file, and modify the client so it counts the number of bytes it reads from the socket. When the client has read enough, it should stop writing to the file.
In my server-side code I need to be able to listen to a socket to exchange JSON 'packets' with a Java 7 test application on the same machine. The connection is made and a JSON string is constructed and written to the socket by the Java test application. It is received by the Dart server-side application and passed to a callback method, handleJson, which attempts to decode it. The process dies on 'JSON.decode'.
I think it dies because the string is prepended, by the Java 'writeUTF' method with a short int that contains the number of bytes in the JSON UTF-8 uncluding the leading short and the leading byte is 0.
Is there a Dart method to handle this, in each direction, or must I write the code? (I had thought that JSON work easily between languages.)
The JSON string before writing to the socket in my Java test application:
{"target":"DOOR","command":"OPEN"} // 34 characters
A Java snippet:
// in a try-catch
Socket client = new Socket(serverName, port);
OutputStream outToServer = client.getOutputStream();
DataOutputStream out = new DataOutputStream(outToServer);
out.writeUTF(json);
client.close();
The Java documentation states that the out.writeUTF method converts the json string to UTF-8 with the string length prepended as a short int containing the total number of bytes written.
In main:
ServerSocket.bind('127.0.0.1', 4041)
.then((serverSocket) {
print('connected');
// prints: 'connected'
serverSocket.listen((socket) {
socket.transform(UTF8.decoder).listen(handleJson);
});
});
handleJson method:
handleJson(String stringAsJson){
print('string length is ' + (stringAsJson.length).toString());
// prints: 'string length is 36'
print('received json $stringAsJson');
// prints: 'received json '
String json = JSON.decode(stringAsJson);
// dies on decode
print('Sever Socket received: $json');
}
This will give you some troubles, since Socket is raw TCP, and TCP is streaming. That means that the text (bytes) you send can be split and merged in any way the network may find suitable.
In your case, you need a way to mark the end of each JSON message. An example could be to accumulate all bytes received, until the byte 0 is seen (invalid in JSON). Those bytes could then be converted to UTF8 and then again converted to JSON. Note that the peer needs to send this 0 byte in between messages, for this to work.
Now, you also consider using WebSockets as a way to sent messages. After the initial HTTP handshake, it's actually just a raw TCP socket with some extra header information, to make it package oriented - exactly what you need. dart:io already includes a WebSocket implementation.
My problem is that C sockets look to act differently than Java sockets. I have a C proxy and I tested it between a workload generator (oltp benchmark client written in Java) and the JDBC connector of the Postgres DB.
This works great and forwards data from one to other, as it should. We need to make this proxy work in Java, so I used plain ServerSocket and Socket classes from java.net and I cannot make it work. The Postgres returns an authentication error message, assuming that the client did not send the correct password.
Here is how the authentication at the JDBC protocol works:
-client sends a requests to connect to a database specifying the database name and the username
-server responds back with a one time challenge message (13 byte message with random content)
-client concatenates this message with the user password and performs a md5 hash
-server compares the hash got from the client with the hash he computes
[This procedure is performed in order to avoid replay attacks (if client would send only the md5 hash of its password then an attacker could replay this message, pretending he is the client)]
So I inspected the packets with tcpdump and they look correct! The size is exactly as it should, so maybe the content is corrupted (??)
Sometimes though the DB server responds ok for the authentication (depending on the value of the challenge message)!! And then the oltp client sends a couple of queries, but it crashes in a while…
I guess that maybe it has to do with the encoding, so I tried with the encoding that C uses (US-ANSII), but still the same.
I send the data using fixed size character or byte arrays both in C and in Java!
I really don't have any more ideas, as I tried so many cases...
What is your guess of what would be the problem?
Here is a representative code that may help you have a more clear view:
byte [] msgBuf;
char [] msgBufChars;
while(fromInputReader.ready()){
msgBuf = new byte[1024];
msgBufChars = new char[1024];
// read data from one party
int read = fromInputReader.read(msgBufChars, 0, 1024);
System.out.println("Read returned : " + read);
for(int i=0; i<1024; i++)
msgBuf[i] = (byte) msgBufChars[i];
String messageRead = new String(msgBufChars);
String messageToWrite = new String(msgBuf);
System.out.println("message read : "+messageRead);
System.out.println("message to write : "+new String(messageToWrite));
// immediatelly write data to other party (write the amount of data we read (read value) )
// there is no write method that takes a char [] as a parameter, so pass a byte []
toDataOutputStream.write(msgBuf, 0, read);
toDataOutputStream.flush();
}
There are a couple of message exchanges in the beginning and then Postgres responds with an authentication failure message.
Thanks for your time!
What is your guess of what would be the problem?
It is nothing to do with C versus Java sockets. It is everything to do with bad Java code.
I can see some problems:
You are using a Reader in what should be a binary stream. This is going to result in the data being converted from bytes (from the JDBC client) to characters and then back to bytes. Depending on the character set used by the reader, this is likely to be destructive.
You should use plain, unadorned1 input streams for both reading and writing, and you should read / write to / from a preallocated byte[].
This is terrible:
for(int i=0; i<1024; i++)
msgBuf[i] = (byte) msgBufChars[i];
If the characters you read are not in the range 0 ... 255 you are mangling them when you stuff them into msgBuf.
You are assuming that you actually got 1024 characters.
You are using the ready() method to decide when to stop reading stuff. This is almost certainly wrong. Read the javadoc for that method (and think about it) and you should understand why it is wrong. (Hint: what happens if the proxy can read faster than the client can deliver?)
You should use a while(true), and then break out of the loop if read tells you it has reached the end of stream; i.e. if it returns -1 ...
1 - Just use the stream objects that the Socket API provides. DataXxxStream is unnecessary because the read and write methods are simply call-throughs. I wouldn't even use BufferedXxxStream wrappers in this case, because you are already doing your own buffering using the byte array.
Here's how I'd write that code:
byte [] buffer = new byte[1024]; // or bigger
while(true) {
int nosRead = inputStream.read(buffer);
if (nosRead < 0) {
break;
}
// Note that this is a bit dodgy, given that the data you are converting is
// binary. However, if the purpose is to see what embedded character data
// looks like, and if the proxy's charset matches the text charset used by
// the client-side JDBC driver for encoding data, this should achieve that.
System.out.println("Read returned : " + nosRead);
System.out.println("message read : " + new String(buffer, 0, nosRead));
outputStream.write(buffer, 0, nosRead);
outputStream.flush();
}
C sockets look to act differently than Java sockets.
Impossible. Java sockets are just a very thin layer over C sockets. You're on the wrong track with this line of thinking.
byte [] msgBuf;
char [] msgBufChars;
Why are you reading chars when you want to write bytes? Don't use Readers unless you know that the input is text.
And don't call ready(). There are very few correct uses, and this isn't one of them. Just block.
I'm trying to write a WebSocket Server in both, java and C++ but I'm stuck right now.
Using java and java.net.ServerSocket/java.net.Socket I managed to get a connection and succesfully do the handshake but the data sent by the WebSocket to the Java Server is not quite what I expected.
When sending messages from javascript like this:
var count = 0;
function loop(){
websocket.send("loop: " + count + "\n");
count++;
setTimeout(loop, 100);
}
The Java server receives this, with line feeds every now and then but not for every websocket.send() that has been invoked.
?‡½÷"˜Ñ˜Mè‡×?‡AÎ3-¡C{îN?‡ŒÍ[Uà¢4%¶íi?‡$ÍåøH¢ŠˆíÖ?‡·†ÞžÛé±î?¦ê?‡'½Ø…KÒ·õ?í?‡dÒÛ‘½´á^òí?‡+ù?YG–â)Ùº?‡›?
Ë÷àb»¡¯5?‡mÉŒQ¦ã!Wéµ?ˆ:J FV%f6
The Java server retrieves values from the socket using BufferedReader.readLine()
BufferedReader socketReader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String line =socketReader.readLine();
This works fine for the handshake and all handshake data is readable but it does not work after the handshake is done.
Is the data after the handshake somehow encrypted? How can I read it?
EDIT :
The program files:
SocketConnectTest.html
ServerTest.java
ClientSessionTest.java
ResponseGenerator.java
output
Just run ServerTest.java and then open SocketConnectTest.html.
ClientSessionTest.initClientListener() handles the messages from the client.
SOLUTION :
For the solution see pimvdbs post below and his answer at How to (de)construct data frames in WebSockets hybi 08+?
The data coming across web sockets is raw, not string encoded data.
I'd suggest not wrapping a BufferedReader around the incoming data as packets are framed with 0x00 bytes. The crazy characters you are seeing are a result of Java not understanding the encoding that the data is in.
You will need to be responsible for splitting up the data into character and control parts. Once you've split the data up into the appropriate areas, then you can decode the data as a string.
I'm writing a protocol on top of sockets, so I've decided to implement headers then send the information. So there is one thread per connection on the server which sits there reading in headers, then delegates off to methods to read in the rest of the information when it arrives.
So essentially it looks like this:
while ((length = inStream.read(buffer)) != -1)
{
dispatch(buffer, length);
}
So the dispatch method then decrypts the headers and delegates the method depending what is found in the header. It looks similar to:
byte[] clearText = decrypt(message,length);
if (cleartext == foo) sendFooToSocket();
So then sendFooToSocket() would then sit there and read from the instream or send to the outstream.
This is where I seem to run into some problems, in the client I'm sending the header then flushing, then sending the rest of the data, but it appears it's all coming as one and not being split up into header then data. Also is there a best way to force out of the sendFooToSocket method?
public void sendFooToSocket()
{
byte[] buffer = new byte[1024];
int length = 0;
while ((length = inStream.read(buffer) >0)
{
message = decrypt(buffer, length);
}
}
I would assume flush would allow me to break out of this method as it closes then opens the stream?
So I have 2 problems, flush doesn't seem to be breaking up my messages and flush doesn't seem to be allowing to drop out of methods such as sendFooToSocket(), any suggestions?
For clarity sake, the client just does this:
byte[] header = "MESG".getBytes();
cipher = encrypt(header);
outStream.write(cipher,0,cipher.length);
outStream.flush();
byte[] message = "Hi server".getBytes();
cipher = encrypt(message);
outStream.write(cipher,0,cipher.length);
outStream.flush();
But this is received by the server as 1 message even though it's been flushed after every write. Sending just the header works, and we get stuck in the sendFooToSocket() method, but if I send the data after the flush it comes all at once.
The client uses OutputStream and InputStreams just from the socket.get. The client also uses OutputStream and InputStream. Not sure if this matters?
What you seem to want is "record boundaries". With streams in general there are no implicit record boundaries. If you want that kind of functionality you will need to implement it yourself, by buffering the input and looking for, say, newlines to indicate the end of a record.
Look at BufferedInputStream.
inStream.read() may not be returning on a message boundary. You can't assume that it'll return at any particular boundary (such as a blank line separating headers and content if that's how you're doing it.) You'll have to manually parse the content and ignore the fact that it could come from multiple read()s or maybe one read() contains both the headers and content.
Unless you actually want control at the level you have implemented, you could consider Object streams (see ObjectInputStream and ObjectOutputStream). Such streams will allow you to send Java Objects over sockets and read them at the other end with out having to deal with headers and boundaries etc. See ObjectOutputStream for more details, but it's pretty much:
Sender:
writeObject(objectX)
Receiver:
myCopyOfObjectx = readObject()
and you can send any objects you like (as long as they are Serializable).