We Keep Coding

Retrieve data from via UART

I am using UART to get the input to my system from PIC Development Board. and I use the following code to get the values from the board.
public SerialReader( InputStream in ) {
this.in = in;
}
public void run() {
byte[] buffer = new byte[ 1024 ];
int len = -1;
try {
/*len = this.in.read(buffer);
int x = buffer[0] & 0xff;
System.out.println(buffer[0]);
System.out.println(x);
System.out.println((char)x);*/
while( ( len = this.in.read( buffer ) ) > -1 ) {
System.out.print( new String( buffer, 0, len ) );
System.out.println(len);
/*String s = new String(buffer); //buffer.toString(); 1
System.out.println(s);
for (int i=0; i<=buffer.length; i++)
System.out.println(buffer[i]);
//System.out.print( new String( buffer, 0, len ) );
*/ }
} catch( IOException e ) {
e.printStackTrace();
}
}
}
Output: ààà
Expected Output: #C01=0155,INT=16:11,OUT=05:11
How do I retrive the expected output.

ObjectInputStream being corrupted during byte array read

I appear to be getting corruption when reading an ObjectInputStream. The attached snippet throws an exception prior to completion. I fixed the example to call oos.writeObject( p1 ) as suggested.
The Exception stack is as follows:
java.lang.OutOfMemoryError: Java heap space
at test.POJO.readExternal(STest.java:82)
at java.io.ObjectInputStream.readExternalData(Unknown Source)
at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.readObject(Unknown Source)
at test.STest.test(STest.java:37)
I believe this OutOfMemoryError exception to be misleading. I added a print statement showing the readExternal(..) behavior and am seeing a large value being pulled from ObjectInputStream, this does not correlate to what was written. If DIM is set to 5 it works if set to 15 I get the above exception. If I lower the number of bytes written per array element I get more successful iterations.
package test;
import static org.junit.Assert.*;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectInputStream;
import java.io.ObjectOutput;
import java.io.ObjectOutputStream;
import org.junit.Test;
public class STest
{
#Test
public void test() throws Exception
{
POJO p1 = new POJO();
POJO p2 = new POJO();
// Initialize and serialize POJO 1
// --------------------------------
p1.hydrate();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream( baos );
oos.writeObject( p1 );
oos.flush();
oos.close();
byte [] baSerialized = baos.toByteArray();
// Parse POJO 2
// -------------
ObjectInputStream ois = new ObjectInputStream( new ByteArrayInputStream( baSerialized ) );
p2 = (POJO)ois.readObject();
// Test Result
// ------------
byte [][] baa1 = p1._baa;
byte [][] baa2 = p2._baa;
for ( int i=0; i < baa1.length; i++ )
{
String str1 = new String( baa1[ i ] );
String str2 = new String( baa2[ i ] );
assertTrue( str1.equals( str2 ) );
}
}
}
class POJO implements Externalizable
{
protected static final int DIM = 5;
protected byte [][] _baa = null;
public POJO()
{
}
public void hydrate()
{
_baa = new byte[ DIM ][];
for ( int i = 0; i < _baa.length; i++ )
{
_baa[ i ] = ("This is a serialize and parse test, it will be interesting to see if it completes without exception, I suspect not as there appears be a bug in the JRE - " + i).getBytes();
}
}
#Override
public void readExternal( ObjectInput oi ) throws IOException, ClassNotFoundException
{
int iDim = oi.readInt();
_baa = new byte[ iDim ][];
for ( int i=0; i < iDim; i++ )
{
int iSize = oi.readInt();
System.out.println( iSize );
byte [] ba = new byte[ iSize ];
oi.read( ba );
_baa[ i ] = ba;
}
}
#Override
public void writeExternal( ObjectOutput oo ) throws IOException
{
oo.writeInt( _baa.length );
for ( int i=0; i < _baa.length; i++ )
{
oo.writeInt( _baa[ i ].length );
oo.write( _baa[ i ] );
}
}
}

p1.writeExternal(o);
That should be:
oo.writeObject(p1);
You aren't supposed to call your own writeExternal() method directly. The ObjectOutputStream does that.

GZIP outputs and sizes between C#/dot Net and Java

I am testing the feasibility of compressing some messaging between Java and C#.
The messaging used ranges from small strings (40bytes) to larger strings (4K).
I have found differences in the output of Java GZIP implementation to the dot Net GZIP implementation.
I'm guessing that dot Net has a larger header that is causing the large overhead.
I prefer the Java implementation as it works better on small strings, and would like the dot Net to achieve similar results.
Output, Java version 1.6.0_10
Text:EncodeDecode
Bytes:(12 bytes)RW5jb2RlRGVjb2Rl <- Base64
Compressed:(29)H4sIAAAAAAAAAHPNS85PSXVJBZEAd9jYdgwAAAA=
Decompressed:(12)RW5jb2RlRGVjb2Rl
Converted:EncodeDecode
Text:EncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecode
Bytes:(120)RW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2Rl
Compressed:(33)H4sIAAAAAAAAAHPNS85PSXVJBZGudGQDAOcKnrd4AAAA
Decompressed:(120)RW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2Rl
Converted:EncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecode
Output, dot Net 2.0.50727
Text:EncodeDecode
Bytes:(12)RW5jb2RlRGVjb2Rl
Compressed:(128)H4sIAAAAAAAEAO29B2AcSZYlJi9tynt/SvVK1+B0oQiAYBMk2JBAEOzBiM3mkuwdaUcjKasqgcplVmVdZhZAzO2dvPfee++999577733ujudTif33/8/XGZkAWz2zkrayZ4hgKrIHz9+fB8/Ik6X02qWP83x7/8Dd9jYdgwAAAA=
Decompressed:(12)RW5jb2RlRGVjb2Rl
Text:EncodeDecode
Text:EncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecode
Bytes:(120)RW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2Rl
Compressed:(131)H4sIAAAAAAAEAO29B2AcSZYlJi9tynt/SvVK1+B0oQiAYBMk2JBAEOzBiM3mkuwdaUcjKasqgcplVmVdZhZAzO2dvPfee++999577733ujudTif33/8/XGZkAWz2zkrayZ4hgKrIHz9+fB8/Ik6X02qWP83x7w/z9/8H5wqet3gAAAA=
Decompressed:(120)RW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2RlRW5jb2RlRGVjb2Rl
Text:EncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecodeEncodeDecode
How can I achieve the smaller sized encoding on the dot Net side?
Note,
Java implementation can decode dot Net implementation and
dot Net implementation can decode Java implementation.
Java Code
#Test
public void testEncodeDecode()
{
final String strTitle = "EncodeDecode";
try
{
debug( "Text:" + strTitle );
byte[] ba = strTitle.getBytes( "UTF-8" );
debug( "Bytes:" + toString( ba ) );
byte[] eba = encode_GZIP( ba );
debug( "Encoded:" + toString( eba ) );
byte[] ba2 = decode_GZIP( eba );
debug( "Decoded:" + toString( ba2 ) );
debug( "Converted:" + new String( ba2, "UTF-8" ) );
}
catch( Exception ex ) { fail( ex ); }
}
#Test
public void testEncodeDecode2()
{
final String strTitle = "EncodeDecode";
try
{
StringBuilder sb = new StringBuilder();
for( int i = 0 ; i < 10 ; i++ ) sb.append( strTitle );
debug( "Text:" + sb.toString() );
byte[] ba = sb.toString().getBytes( ENCODING );
debug( "Bytes:" + toString( ba ) );
byte[] eba = encode_GZIP( ba );
debug( "Encoded:" + toString( eba ) );
byte[] ba2 = decode_GZIP( eba );
debug( "Decoded:" + toString( ba2 ) );
debug( "Converted:" + new String( ba2, ENCODING ) );
}
catch( Exception ex ) { fail( ex ); }
}
private String toString( byte[] ba )
{
return "("+ba.length+")"+Base64.byteArrayToBase64( ba );
}
protected static byte[] encode_GZIP( byte[] baData ) throws IOException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ByteArrayInputStream bais = new ByteArrayInputStream( baData );
GZIPOutputStream zos = new GZIPOutputStream( baos );
byte[] baBuf = new byte[ 1024 ];
int nSize;
while( -1 != ( nSize = bais.read( baBuf ) ) )
{
zos.write( baBuf, 0, nSize );
zos.flush();
}
Utilities.closeQuietly( zos );
Utilities.closeQuietly( bais );
return baos.toByteArray();
}
protected static byte[] decode_GZIP( byte[] baData ) throws IOException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ByteArrayInputStream bais = new ByteArrayInputStream( baData );
GZIPInputStream zis = new GZIPInputStream( bais );
byte[] baBuf = new byte[ 1024 ];
int nSize;
while( -1 != ( nSize = zis.read( baBuf ) ) )
{
baos.write( baBuf, 0, nSize );
baos.flush();
}
Utilities.closeQuietly( zis );
Utilities.closeQuietly( bais );
return baos.toByteArray();
}
private void debug( Object o ) { System.out.println( o ); }
private void fail( Exception ex )
{
ex.printStackTrace();
Assert.fail( ex.getMessage() );
}
dot Net Code
[Test]
public void TestJava6()
{
string strData = "EncodeDecode";
Console.WriteLine("Text:" + strData);
byte[] baData = Encoding.UTF8.GetBytes(strData);
Console.WriteLine("Bytes:" + toString(baData));
byte[] ebaData2 = encode_GZIP(baData);
Console.WriteLine("Encoded:" + toString(ebaData2));
byte[] baData2 = decode_GZIP(ebaData2);
Console.WriteLine("Decoded:" + toString(baData2));
Console.WriteLine("Text:" + Encoding.UTF8.GetString(baData2));
}
[Test]
public void TestJava7()
{
string strData = "EncodeDecode";
StringBuilder sb = new StringBuilder();
for (int i = 0; i < 10; i++) sb.Append(strData);
Console.WriteLine("Text:" + sb.ToString());
byte[] baData = Encoding.UTF8.GetBytes(sb.ToString());
Console.WriteLine("Bytes:" + toString(baData));
byte[] ebaData2 = encode_GZIP(baData);
Console.WriteLine("Encoded:" + toString(ebaData2));
byte[] baData2 = decode_GZIP(ebaData2);
Console.WriteLine("Decoded:" + toString(baData2));
Console.WriteLine("Text:" + Encoding.UTF8.GetString(baData2));
}
public string toString(byte[] ba)
{
return "(" + ba.Length + ")" + Convert.ToBase64String(ba);
}
protected static byte[] decode_GZIP(byte[] ba)
{
MemoryStream writer = new MemoryStream();
using (GZipStream zis = new GZipStream(new MemoryStream(ba), CompressionMode.Decompress))
{
Utilities.CopyStream(zis, writer);
}
return writer.ToArray();
}
protected static byte[] encode_GZIP(byte[] ba)
{
using (MemoryStream reader = new MemoryStream(ba))
{
MemoryStream writer = new MemoryStream();
using (GZipStream zos = new GZipStream(writer, CompressionMode.Compress))
{
Utilities.CopyStream(reader, zos);
}
return writer.ToArray();
}
}

This is one of several bugs in the .NET gzip code. That code should be avoided. Use DotNetZip instead. See answer here: Why does my C# gzip produce a larger file than Fiddler or PHP? .

Why does my server never receive WebSocket send?

I'm implementing a WebSocket server (for learning purposes) and I have it correctly handling the handshake (websocket.onopen is called so I assume this means handshake was successful), however, when the client (browser) sends a message after the handshake, the server never receives it.
Using Chrome's developer tools, I'm able to see that all the headers were correctly received and no errors are thrown. It also says that it sent the "hello" despite the readLine() never firing in Java.
What's wrong in my code?
EDIT 1: I discovered that if I refresh the web page, then (and only then) the ServerSocket receives the data from the last connection (that the refresh just killed)! Why is this the only way it receives it?
EDIT 2: I also found that I can send a message to the client after the handshake and the client receieves it but STILL the server never receives the client's message! I sent the message to the client like this:
byte[] message = new byte[ 7 ];
message[ 0 ] = new Integer(129).byteValue();
message[ 1 ] = new Integer(5).byteValue();
byte[] raw = "hello".getBytes();
message[ 2 ] = raw[ 0 ];
message[ 3 ] = raw[ 1 ];
message[ 4 ] = raw[ 2 ];
message[ 5 ] = raw[ 3 ];
message[ 6 ] = raw[ 4 ];
outStream.write( message);
out.println();
HTML PAGE
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>WebSocket Test</title></head>
<body>
<script>
try
{
function writeToScreen(message)
{
var p = document.createElement( "p" );
p.innerHTML = message;
document.getElementsByTagName( "body" )[ 0 ].appendChild( p );
}
function onOpen(evt)
{
writeToScreen( "opened" );
doSend( "hello" );
//We reach here but the server never recieves the message! (and bufferedAmount == 0)
writeToScreen( "sent: " + websocket.bufferedAmount );
}
function onClose(evt)
{
alert( "closed" );
websocket.close();
}
function onMessage(evt)
{
alert( "Message: " + evt.data );
}
function onError(evt)
{
alert( "Error: " + evt );
}
function doSend (message)
{
websocket.send( message );
}
//PUT IN YOUR OWN LOCAL IP ADDRESS HERE TO GET IT TO WORK
var websocket = new WebSocket( "ws://192.168.1.19:4444/" );
websocket.onopen = onOpen;
websocket.onclose = onClose;
websocket.onmessage = onMessage;
websocket.onerror = onError;
}
catch(e)
{
}
</script>
</body>
</html>
JAVA CODE
import java.net.*;
import java.io.*;
import java.security.*;
public class WebListener
{
public static void main(String[] args) throws Exception
{
ServerSocket serverSocket = null;
boolean listening = true;
try {
serverSocket = new ServerSocket(4444);
} catch (IOException e) {
System.err.println("Could not listen on port: 4444.");
System.exit(-1);
}
while (listening) new ServerThread(serverSocket.accept()).start();
serverSocket.close();
}
}
class ServerThread extends Thread {
private Socket socket = null;
public ServerThread(Socket socket) {
super("ServerThread");
this.socket = socket;
}
public void run() {
try {
OutputStream outStream = null;
PrintWriter out = new PrintWriter( outStream = socket.getOutputStream(), true);
BufferedReader in = new BufferedReader( new InputStreamReader( socket.getInputStream()));
String inputLine, outputLine;
//Handle the headers first
doHeaders( out, in );
//Now read anything they have to send
while ( ( inputLine = in.readLine() ) != null )
{
//WE NEVER REACH HERE!
System.out.println( inputLine );
}
out.close();
in.close();
socket.close();
} catch (Exception e) {
e.printStackTrace();
}
}
public void doHeaders(PrintWriter out, BufferedReader in) throws Exception
{
String inputLine = null;
String key = null;
//Read the headers
while ( ( inputLine = in.readLine() ) != null )
{
//Get the key
if ( inputLine.startsWith( "Sec-WebSocket-Key" ) ) key = inputLine.substring( "Sec-WebSocket-Key: ".length() );
//They're done
if ( inputLine.equals( "" ) ) break;
}
//We need a key to continue
if ( key == null ) throw new Exception( "No Sec-WebSocket-Key was passed!" );
//Send our headers
out.println( "HTTP/1.1 101 Web Socket Protocol Handshake\r" );
out.println( "Upgrade: websocket\r" );
out.println( "Connection: Upgrade\r" );
out.println( "Sec-WebSocket-Accept: " + createOK( key ) + "\r" );
out.println( "\r" );
}
public String createOK(String key) throws NoSuchAlgorithmException, UnsupportedEncodingException, Exception
{
String uid = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
String text = key + uid;
MessageDigest md = MessageDigest.getInstance( "SHA-1" );
byte[] sha1hash = new byte[40];
md.update( text.getBytes("iso-8859-1"), 0, text.length());
sha1hash = md.digest();
return new String( base64( sha1hash ) );
}
public byte[] base64(byte[] bytes) throws Exception
{
ByteArrayOutputStream out_bytes = new ByteArrayOutputStream();
OutputStream out = new Base64.OutputStream(out_bytes); //Using http://iharder.net/base64
out.write(bytes);
out.close();
return out_bytes.toByteArray();
}
private String convertToHex(byte[] data) {
StringBuffer buf = new StringBuffer();
for (int i = 0; i < data.length; i++) {
int halfbyte = (data[i] >>> 4) & 0x0F;
int two_halfs = 0;
do {
if ((0 <= halfbyte) && (halfbyte <= 9))
buf.append((char) ('0' + halfbyte));
else
buf.append((char) ('a' + (halfbyte - 10)));
halfbyte = data[i] & 0x0F;
} while(two_halfs++ < 1);
}
return buf.toString();
}
}

WebSocket messages are not terminated by \r\n so you can't use in.readline() to read them. See the data framing section of the spec for how to messages are constructed.
For text messages from client (browser) to server, messages will have the form:
(byte)0x81
1, 3 or 9 byte structure indicating message length and whether the message body is masked. (Messages from a browser should always be masked.)
4 byte mask
Message (utf-8 encoded)
There is no end-of-message marker you can search for. You just need to read the first few bytes of a client request to figure out the length of its payload.

your code WebListener must running on windows, and you will find out line.separator is CRLF
byte[] lineSeperator=System.getProperty("line.separator").getBytes();
System.out.println("line seperator: "+Arrays.toString(lineSeperator));
In your response header
out.println( "Header xxxx"+ "\r" );
so header is ended with \r\r\n
Per HTTP rfc2616
Response = Status-Line ; Section 6.1
*(( general-header ; Section 4.5
| response-header ; Section 6.2
| entity-header ) CRLF) ; Section 7.1
CRLF
[ message-body ] ; Section 7.2
Client don't can not decode your header with \r\r\n.

Why do I get two connections from WebSocket client? One is "empty"

When I open a websocket connection to my websocket server application from Java, the server sees two connections. The first one never sends any data and the second one sends all the proper headers, etc. Anyone know what the reason for this is?
Client side connection is:
var websocket = new WebSocket( "ws://192.168.1.19:3333/websession" );
On the server side, in a while loop I call "serverSocket.accept()" and this gets called twice. But one of them never sends any data (the in.read() simply times out eventually without returning anything).
JAVA SERVER CODE
import java.net.*;
import java.io.*;
import java.security.*;
public class WebListener {
public static void main(String[] args) throws Exception {
ServerSocket serverSocket = null;
boolean listening = true;
try {
serverSocket = new ServerSocket(4444);
} catch (IOException e) {
System.err.println("Could not listen on port: 4444.");
System.exit(-1);
}
while (listening) new ServerThread(serverSocket.accept()).start();
serverSocket.close();
}
}
class ServerThread extends Thread {
private Socket socket = null;
public ServerThread(Socket socket) {
super("ServerThread");
this.socket = socket;
}
public void run() {
try {
OutputStream outStream = null;
PrintWriter out = new PrintWriter( outStream = socket.getOutputStream(), true);
BufferedReader in = new BufferedReader( new InputStreamReader( socket.getInputStream()));
String inputLine, outputLine;
//Handle the headers first
doHeaders( out, in );
// ..elided..
out.close();
in.close();
socket.close();
} catch (Exception e) {
e.printStackTrace();
}
}
public void doHeaders(PrintWriter out, BufferedReader in) throws Exception {
String inputLine = null;
String key = null;
//Read the headers
while ( ( inputLine = in.readLine() ) != null ) {
//Get the key
if ( inputLine.startsWith( "Sec-WebSocket-Key" ) )
key = inputLine.substring( "Sec-WebSocket-Key: ".length() );
//They're done
if ( inputLine.equals( "" ) ) break;
}
//We need a key to continue
if ( key == null ) throw new Exception( "No Sec-WebSocket-Key was passed!" );
//Send our headers
out.println( "HTTP/1.1 101 Web Socket Protocol Handshake\r" );
out.println( "Upgrade: websocket\r" );
out.println( "Connection: Upgrade\r" );
out.println( "Sec-WebSocket-Accept: " + createOK( key ) + "\r" );
out.println( "\r" );
}
public String createOK(String key) throws NoSuchAlgorithmException, UnsupportedEncodingException, Exception {
String uid = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
String text = key + uid;
MessageDigest md = MessageDigest.getInstance( "SHA-1" );
byte[] sha1hash = new byte[40];
md.update( text.getBytes("iso-8859-1"), 0, text.length());
sha1hash = md.digest();
return new String( base64( sha1hash ) );
}
public byte[] base64(byte[] bytes) throws Exception {
ByteArrayOutputStream out_bytes = new ByteArrayOutputStream();
OutputStream out = new Base64.OutputStream(out_bytes); //Using http://iharder.net/base64
out.write(bytes);
out.close();
return out_bytes.toByteArray();
}
private String convertToHex(byte[] data) {
StringBuffer buf = new StringBuffer();
for (int i = 0; i < data.length; i++) {
int halfbyte = (data[i] >>> 4) & 0x0F;
int two_halfs = 0;
do {
if ((0 <= halfbyte) && (halfbyte <= 9))
buf.append((char) ('0' + halfbyte));
else
buf.append((char) ('a' + (halfbyte - 10)));
halfbyte = data[i] & 0x0F;
} while(two_halfs++ < 1);
}
return buf.toString();
}
}

This looks to be a bug with Firefox. In Chrome it only opens one connection, while the same page in Firefox 15 opens two connections.