Send objects with MPJ express - java

I'm new to parallel programming and I want to do it in java.
I am wondering if it is possible to send and receive more complex ojects via MPI. I'm using MPJ express. However whenever I want to send a object I get a ClassCastException.
MPI.Init(args);
myrank = MPI.COMM_WORLD.Rank();
numprocs = MPI.COMM_WORLD.Size();
Vector<CustomClass> chr = new Vector<CustomClass>();
if (myrank == 0 ) { //am I the master?
for (int i = 1; i < numprocs; i++) {
MPI.COMM_WORLD.Send(chr, 0, chr.size(), MPI.OBJECT, i, 99); //Here's where the
exception occurs
}
}
else {
Vector<BasicRegion> chr_received = new Vector<BasicRegion>();
MPI.COMM_WORLD.Recv(chr_received, 0, 1, MPI.OBJECT, 0, 99 );
}
Exception:
mpi.MPIException: mpi.MPIException: java.lang.ClassCastException: java.util.Vector cannot be cast to [Ljava.lang.Object;
So my questions are:
- is it possible to send/receive more complex objects with MPJ Express?
- if so: what am I doing wrong?

I am new to MPJ express as well, but seems the enclosing object needs to be primitive type - array of something. (like as you do with the C/C++ implementation in OpenMPI).
This kind of code worked for me well:
Node t[] = new Node[4];
...
count[0] = t.length;
MPI.COMM_WORLD.Send(count, 0, 1, MPI.INT, 1, 98);
MPI.COMM_WORLD.Send(t, 0, t.length, MPI.OBJECT, 1, 99);
} else if( myRank == 1 ) {
int count[] = new int[1];
MPI.COMM_WORLD.Recv( count, 0, 1, MPI.INT, 0, 98);
Status mps = MPI.COMM_WORLD.Recv( t, 0, count[0], MPI.OBJECT, 0, 99 );
...
And of course, you have to have that custom class implementing Serializable interface.

You want to serialize it prior to sending.
import mpi.*;
import java.io.*;
import java.nio.ByteBuffer;
public class MPITest
{
public static void main(String[] args)
{
MPI.Init(args);
int me = MPI.COMM_WORLD.Rank();
int tasks = MPI.COMM_WORLD.Size();
MPI.COMM_WORLD.Barrier();
if(me == 0)
{
Cat cat = new Cat("Tom", 15);
cat.Speak();
ByteBuffer byteBuff = ByteBuffer.allocateDirect(2000 + MPI.SEND_OVERHEAD);
MPI.Buffer_attach(byteBuff);
try
{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput out = null;
out = new ObjectOutputStream(bos);
out.writeObject(cat);
byte[] bytes = bos.toByteArray();
System.out.println("Serialized to " + bytes.length);
MPI.COMM_WORLD.Isend(bytes, 0, bytes.length, MPI.BYTE, 1, 0);
}
catch(IOException ex)
{
}
}
else
{
byte[] bytes = new byte[2000];
Cat recv = null;
MPI.COMM_WORLD.Recv(bytes, 0, 2000, MPI.BYTE, MPI.ANY_SOURCE, 0);
ByteArrayInputStream bis = new ByteArrayInputStream(bytes);
ObjectInput in = null;
try
{
in = new ObjectInputStream(bis);
Object obj = in.readObject();
recv = (Cat)obj;
recv.Speak();
}
catch(IOException ex)
{
}
catch(ClassNotFoundException cnf)
{
}
}
MPI.COMM_WORLD.Barrier();
MPI.Finalize();
}
}
This works however you may want to use Externalization and do it manually to avoid some of the extra garbage that the serialize routine will send.
HTH
Brian

import mpi.*;
/**
* Compile: javac -cp $MPJ_HOME/lib/mpj.jar:. ObjSend.java
* Execute: mpjrun.sh -np 2 -dport 11000 ObjSend
*/
public class ObjSend {
public static void main(String[] args) throws Exception {
int peer ;
MPI.Init(args);
int rank = MPI.COMM_WORLD.Rank() ;
int size = MPI.COMM_WORLD.Size() ;
int tag = 100 ;
if(rank == 0) {
String [] smsg = new String[1] ;
smsg[0] = "Hi from proc 0" ;
peer = 1 ;
MPI.COMM_WORLD.Send(smsg, 0, smsg.length, MPI.OBJECT,
peer, tag);
System.out.println("proc <"+rank+"> sent a msg to "+
"proc <"+peer+">") ;
} else if(rank == 1) {
String[] rmsg = new String[1] ;
peer = 0 ;
MPI.COMM_WORLD.Recv(rmsg, 0, rmsg.length , MPI.OBJECT,
peer, tag);
System.out.println("proc <"+rank+"> received a msg from "+
"proc <"+peer+">") ;
System.out.println("proc <"+rank+"> received the following "+
"message: \""+rmsg[0]+"\"") ;
}
MPI.Finalize();
}
}

Related

No data found in socket but definitely received a response

I currently am writing a program which communicates to a server hosted on a separate device via TCP packets. Writing to the server works fine, but I never seem to be able to read the response. I am positive there is a response because when sniffing network packets, I see the incoming packet from the device, though the program never detects it.
public class SocketTest
{
private Socket socket;
private int currentSequence = 0;
public static void main(String[] args)
{
new SocketTest();
}
public SocketTest()
{
try
{
System.out.println("Connecting");
socket = new Socket("192.168.1.8", 8000);
System.out.println("Connected!");
System.out.println("Pinging...");
sendPacket(0, 3, null, 0);
System.out.println("Pinged!");
System.out.println("Reading..");
while (socket.isConnected())
{
byte[] res = new byte[84];
socket.getInputStream().read(res);
System.out.println("Read!");
}
}
catch (IOException e)
{
e.printStackTrace();
}
}
public void sendPacket(int type, int command, int[] args, int length)
{
int t = 0;
currentSequence += 1000;
byte[] resultBuffer = new byte[84];
byte[] payload1 = getBytes(0x12345678);
System.arraycopy(payload1, 0, resultBuffer, t, payload1.length);
t += 4;
byte[] payload2 = getBytes(currentSequence);
System.arraycopy(payload2, 0, resultBuffer, t, payload2.length);
t += 4;
byte[] payload3 = getBytes(type);
System.arraycopy(payload3, 0, resultBuffer, t, payload3.length);
t += 4;
byte[] payload4 = getBytes(command);
System.arraycopy(payload4, 0, resultBuffer, t, payload4.length);
for (int i = 0; i < 16; i++)
{
t += 4;
int arg = 0;
if (args != null)
{
arg = args[i];
}
byte[] payloadArg = getBytes(arg);
System.arraycopy(payloadArg, 0, resultBuffer, t, payloadArg.length);
}
t += 4;
byte[] payload5 = getBytes(length);
System.arraycopy(payload5, 0, resultBuffer, t, payload5.length);
write(resultBuffer, 0, resultBuffer.length);
}
private void write(byte[] buffer, int startOffset, int length)
{
DataOutputStream out;
try
{
out = new DataOutputStream(socket.getOutputStream());
out.write(buffer, startOffset, length);
out.flush();
}
catch (IOException e)
{
e.printStackTrace();
}
}
private byte[] getBytes(int value)
{
ByteBuffer buffer = ByteBuffer.allocate(4).order(ByteOrder.nativeOrder());
buffer.putInt(value);
return buffer.array();
}
}
I've tried a bunch of different methods of reading.. Multithreading, heartbeat, etc.. But everytime I use InputStream's read() method, it blocks because there is no data to be read, and the socket is definitely open because the result isn't -1.
This is the actual read method i'm using in my program (The above code is an shortened xample)
private int read(byte[] buffer, int offset, int length)
{
int res = 0;
DataInputStream bis;
try
{
bis = new DataInputStream(socket.getInputStream());
if (bis.available() > 0)
{
System.out.println("Waiting bytes: " + bis.available());
System.out.println("buffer = [" + Arrays.toString(buffer) + "], offset = [" + offset + "], length = [" + length + "]");
res = bis.read(buffer, offset, length);
}
else
{
NTR.getLogger().debug("Available: " + bis.available());
}
}
catch (IOException e)
{
e.printStackTrace();
}
return res;
}
Interestingly, the C# version of the code works just fine.
namespace ntrclient
{
public class NtrClient
{
public delegate void LogHandler(string msg);
private readonly object _syncLock = new object();
private uint _currentSeq;
private int _heartbeatSendable;
public string Host;
private string _lastReadMemFileName;
private uint _lastReadMemSeq;
public NetworkStream NetStream;
public Thread PacketRecvThread;
public int Port;
public volatile int progress = -1;
public TcpClient Tcp;
public event LogHandler OnLogArrival;
private int ReadNetworkStream(Stream stream, byte [] buf, int length)
{
var index = 0;
var useProgress = length > 100000;
do
{
if (useProgress)
{
progress = (int) ((double) index / length * 100);
}
var len = stream.Read(buf, index, length - index);
if (len == 0)
{
Console.WriteLine("No data to be read");
return 0;
}
Console.WriteLine("Read " + len + " datas");
index += len;
} while (index < length);
progress = -1;
Console.WriteLine("Length: " + length + ", Buffer: " + buf);
return length;
}
private void PacketRecvThreadStart()
{
var buf = new byte [84];
var args = new uint [16];
var stream = NetStream;
while (true)
{
try
{
var ret = ReadNetworkStream(stream, buf, buf.Length);
if (ret == 0)
{
break;
}
var t = 0;
var magic = BitConverter.ToUInt32(buf, t);
t += 4;
var seq = BitConverter.ToUInt32(buf, t);
t += 4;
var type = BitConverter.ToUInt32(buf, t);
t += 4;
var cmd = BitConverter.ToUInt32(buf, t);
for (var i = 0; i < args.Length; i++)
{
t += 4;
args [i] = BitConverter.ToUInt32(buf, t);
}
t += 4;
var dataLen = BitConverter.ToUInt32(buf, t);
if (cmd != 0)
{
Log($"packet: cmd = {cmd}, dataLen = {dataLen}");
}
if (magic != 0x12345678)
{
Log($"broken protocol: magic = {magic}, seq = {seq}");
break;
}
if (cmd == 0)
{
if (dataLen != 0)
{
var dataBuf = new byte [dataLen];
ReadNetworkStream(stream, dataBuf, dataBuf.Length);
var logMsg = Encoding.UTF8.GetString(dataBuf);
Log(logMsg);
}
lock (_syncLock)
{
_heartbeatSendable = 1;
}
continue;
}
if (dataLen != 0)
{
var dataBuf = new byte [dataLen];
ReadNetworkStream(stream, dataBuf, dataBuf.Length);
HandlePacket(cmd, seq, dataBuf);
}
}
catch (Exception e)
{
Log(e.Message);
break;
}
}
Log("Server disconnected.");
Disconnect(false);
}
public void ConnectToServer()
{
if (Tcp != null)
{
Disconnect();
Log("Disconnected from previous server, connecting to new one");
}
new Thread(() =>
{
//Thread.CurrentThread.IsBackground = true;
Tcp = new TcpClient { NoDelay = true };
Tcp.Connect(Host, Port);
_currentSeq = 0;
NetStream = Tcp.GetStream();
_heartbeatSendable = 1;
PacketRecvThread = new Thread(PacketRecvThreadStart);
PacketRecvThread.Start();
Program.getMain().Connected = true;
Log("Server connected.");
}).Start();
}
public void Disconnect(bool waitPacketThread = true)
{
try
{
Tcp?.Close();
if (waitPacketThread)
{
PacketRecvThread?.Join();
}
// Not connected anymore
Program.getMain().Connected = false;
Log("Server disconnected.");
}
catch (Exception ex)
{
Log(ex.Message);
}
Tcp = null;
}
public void SendPacket(uint type, uint cmd, uint [] args, uint dataLen)
{
var t = 0;
_currentSeq += 1000;
var buf = new byte [84];
BitConverter.GetBytes(0x12345678).CopyTo(buf, t);
t += 4;
BitConverter.GetBytes(_currentSeq).CopyTo(buf, t);
t += 4;
BitConverter.GetBytes(type).CopyTo(buf, t);
t += 4;
BitConverter.GetBytes(cmd).CopyTo(buf, t);
for (var i = 0; i < 16; i++)
{
t += 4;
uint arg = 0;
if (args != null)
{
arg = args [i];
}
BitConverter.GetBytes(arg).CopyTo(buf, t);
}
t += 4;
BitConverter.GetBytes(dataLen).CopyTo(buf, t);
NetStream.Write(buf, 0, buf.Length);
}
public void SendHeartbeatPacket()
{
if (Tcp == null)
return;
lock (_syncLock)
{
if (_heartbeatSendable != 1)
return;
_heartbeatSendable = 0;
SendPacket(0, 0, null, 0);
}
}
}
}
I'm honestly stumped. I noticed that C#'s NetworkStream read and write methods are different from java, but I don't exactly know how different they are, or if it makes any difference.
Thanks in advance!

Successfully implemented threading, but I cannot figure out how to write results to file

So, I have the program doing what I want. Using threading to encrypt the passwords has turned a 15 minute task into a 2 minute task. However, I cannot seem to figure out how to write the encrypted passwords to a file. I figured I would have every thread store its result in an array and then I planned on writing the contents of the array out to a file to a file. This doesn't seem to be working at all and I'm not sure why.
I know the code if very sloppy, but I'm just trying to get a working solution before I try to pretty things up.
Thanks!
import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.lang.Thread;
import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
class encryptThread extends Thread {
private Thread t;
private String threadName;
private long[] password_aes;
private String uh = "";
private static final char[] CA = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/".toCharArray();
private static final int[] IA = new int[256];
private int j = 0;
static {
Arrays.fill(IA, -1);
for (int i = 0, iS = CA.length; i < iS; i++)
IA[CA[i]] = i;
IA['='] = 0;
}
encryptThread( String name, int i){
threadName = name;
j = i;
}
public void run() {
String finalString = "";
String[] parts = threadName.split(":");
password_aes = prepare_key_pw(parts[1]);
uh = stringhash(parts[0], password_aes);
finalString = (parts[0] + ":" + parts[1] + ":" + uh + "\n");
//System.out.println(finalString);
PassArray.passwordArray[j] = finalString;
if (j == 176) {
for (int x = 0; x < 500; x++) {
System.out.println(PassArray.passwordArray[x]);
}
}
}
public void start ()
{
System.out.println("Starting " + threadName );
if (t == null)
{
t = new Thread (this, threadName);
t.start ();
}
}
public static long[] str_to_a32(String string) {
if (string.length() % 4 != 0) {
string += new String(new char[4 - string.length() % 4]);
}
long[] data = new long[string.length() / 4];
byte[] part = new byte[8];
for (int k = 0, i = 0; i < string.length(); i += 4, k++) {
String sequence = string.substring(i, i + 4);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
baos.write(sequence.getBytes("ISO-8859-1"));
System.arraycopy(baos.toByteArray(), 0, part, 4, 4);
ByteBuffer bb = ByteBuffer.wrap(part);
data[k] = bb.getLong();
} catch (IOException e) {
data[k] = 0;
}
}
return data;
}
public static String a32_to_str(long[] data) {
byte[] part = null;
StringBuilder builder = new StringBuilder();
ByteBuffer bb = ByteBuffer.allocate(8);
for (int i = 0; i < data.length; i++) {
bb.putLong(data[i]);
part = Arrays.copyOfRange(bb.array(), 4, 8);
bb.clear();
ByteArrayInputStream bais = new ByteArrayInputStream(part);
while (bais.available() > 0) {
builder.append((char) bais.read());
}
}
return builder.toString();
}
public static byte[] aes_cbc_encrypt(byte[] data, byte[] key) {
String iv = "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0";
IvParameterSpec ivSpec = new IvParameterSpec(iv.getBytes());
byte[] output = null;
try {
SecretKeySpec keySpec = new SecretKeySpec(key, "AES");
Cipher cipher = Cipher.getInstance("AES/CBC/NOPADDING");
cipher.init(Cipher.ENCRYPT_MODE, keySpec, ivSpec);
output = cipher.doFinal(data);
} catch (Exception e) {
e.printStackTrace();
}
return output;
}
public static long[] aes_cbc_encrypt_a32(long[] idata, long[] ikey) {
try {
byte[] data = a32_to_str(idata).getBytes("ISO-8859-1");
byte[] key = a32_to_str(ikey).getBytes("ISO-8859-1");
byte[] encrypt = aes_cbc_encrypt(data, key);
return str_to_a32(new String(encrypt, "ISO-8859-1"));
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
return new long[0];
}
public static String base64_url_encode(String data) {
try {
data = new String(base64_url_encode_byte((data.getBytes("ISO-8859-1")),true), "ISO-8859-1");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
data = data.replaceAll("\\+", "-");
data = data.replaceAll("/", "_");
data = data.replaceAll("=", "");
return data;
}
public static String a32_to_base64(long[] a) {
return base64_url_encode(a32_to_str(a));
}
public static String stringhash(String email, long[] aeskey) {
long[] s32 = str_to_a32(email);
long[] h32 = {0, 0, 0, 0};
for (int i = 0; i < s32.length; i++) {
h32[i % 4] ^= s32[i];
}
for (int r = 0; r < 0x4000; r++) {
h32 = aes_cbc_encrypt_a32(h32, aeskey);
}
long[] h32Part = new long[2];
h32Part[0] = h32[0];
h32Part[1] = h32[2];
return a32_to_base64(h32Part);
}
public static long[] prepare_key(long[] password) {
long[] pkey = {0x93C467E3, 0x7DB0C7A4, 0xD1BE3F81, 0x0152CB56};
for (int r = 0; r < 0x10000; r++) {
for (int j = 0; j < password.length; j += 4) {
long[] key = {0, 0, 0, 0};
for (int i = 0; i < 4; i++) {
if (i + j < password.length) {
key[i] = password[i + j];
}
}
pkey = aes_cbc_encrypt_a32(pkey, key);
}
}
return pkey;}
public static long[] prepare_key_pw(String password) {
return prepare_key(str_to_a32(password));
}
public final static byte[] base64_url_encode_byte(byte[] sArr, boolean lineSep){
// Check special case
int sLen = sArr != null ? sArr.length : 0;
if (sLen == 0)
return new byte[0];
int eLen = (sLen / 3) * 3; // Length of even 24-bits.
int cCnt = ((sLen - 1) / 3 + 1) << 2; // Returned character count
int dLen = cCnt + (lineSep ? (cCnt - 1) / 76 << 1 : 0); // Length of returned array
byte[] dArr = new byte[dLen];
// Encode even 24-bits
for (int s = 0, d = 0, cc = 0; s < eLen;) {
// Copy next three bytes into lower 24 bits of int, paying attension to sign.
int i = (sArr[s++] & 0xff) << 16 | (sArr[s++] & 0xff) << 8 | (sArr[s++] & 0xff);
// Encode the int into four chars
dArr[d++] = (byte) CA[(i >>> 18) & 0x3f];
dArr[d++] = (byte) CA[(i >>> 12) & 0x3f];
dArr[d++] = (byte) CA[(i >>> 6) & 0x3f];
dArr[d++] = (byte) CA[i & 0x3f];
// Add optional line separator
if (lineSep && ++cc == 19 && d < dLen - 2) {
dArr[d++] = '\r';
dArr[d++] = '\n';
cc = 0;
}
}
// Pad and encode last bits if source isn't an even 24 bits.
int left = sLen - eLen; // 0 - 2.
if (left > 0) {
// Prepare the int
int i = ((sArr[eLen] & 0xff) << 10) | (left == 2 ? ((sArr[sLen - 1] & 0xff) << 2) : 0);
// Set last four chars
dArr[dLen - 4] = (byte) CA[i >> 12];
dArr[dLen - 3] = (byte) CA[(i >>> 6) & 0x3f];
dArr[dLen - 2] = left == 2 ? (byte) CA[i & 0x3f] : (byte) '=';
dArr[dLen - 1] = '=';
}
return dArr;
}
}
public class TestThread {
final static String OUTPUT_FILE_NAME = "C:\\combo_encrypted.txt";
public static void main(String args[]) throws IOException, Throwable {
for (int f = 0; f < 500; f++){
PassArray.passwordArray[f] = "haddy ma'am";
}
File file1 = new File("File1.txt");
File file2 = new File("File2.txt");
File file3 = new File("File3.txt");
File file4 = new File("File4.txt");
FileInputStream fis1 = null;
FileInputStream fis2 = null;
FileInputStream fis3 = null;
FileInputStream fis4 = null;
BufferedInputStream bis1 = null;
BufferedInputStream bis2 = null;
BufferedInputStream bis3 = null;
BufferedInputStream bis4 = null;
DataInputStream dis1 = null;
DataInputStream dis2 = null;
DataInputStream dis3 = null;
DataInputStream dis4 = null;
fis1 = new FileInputStream(file1);
fis2 = new FileInputStream(file2);
fis3 = new FileInputStream(file3);
fis4 = new FileInputStream(file4);
bis1 = new BufferedInputStream(fis1);
bis2 = new BufferedInputStream(fis2);
bis3 = new BufferedInputStream(fis3);
bis4 = new BufferedInputStream(fis4);
dis1 = new DataInputStream(bis1);
dis2 = new DataInputStream(bis2);
dis3 = new DataInputStream(bis3);
dis4 = new DataInputStream(bis4);
int i = 0;
while ( (dis4.available() != 0) ) {
encryptThread[] threadList = new encryptThread[4];
String combo1 = dis1.readLine();
String combo2 = dis2.readLine();
String combo3 = dis3.readLine();
String combo4 = dis4.readLine();
threadList[0] = new encryptThread(combo1, i);
threadList[1] = new encryptThread(combo2, i);
threadList[2] = new encryptThread(combo3, i);
threadList[3] = new encryptThread(combo4, i);
threadList[0].start();
threadList[1].start();
threadList[2].start();
threadList[3].start();
/*
RunnableDemo R1 = new RunnableDemo(combo1, array1, i);
RunnableDemo R2 = new RunnableDemo(combo2, array2, i);
RunnableDemo R3 = new RunnableDemo(combo3, array3, i);
RunnableDemo R4 = new RunnableDemo(combo4, array4, i);
R1.start();
R2.start();
R3.start();
R4.start();
*/
i++;
}
fis1.close();
fis2.close();
fis3.close();
fis4.close();
bis1.close();
bis2.close();
bis3.close();
bis4.close();
dis1.close();
dis2.close();
dis3.close();
dis4.close();
System.out.println(PassArray.passwordArray[5]);
}
}
class PassArray {
public static String[] passwordArray = new String[500];
}
there's a problem here where you are passing this into new Thread(). Maybe the Thread API supports it, but I don't think it's very standard.
public void start ()
{
System.out.println("Starting " + threadName );
if (t == null)
{
t = new Thread (this, threadName);
t.start ();
}
}
as one of the commenters pointed out:
class encryptThread extends Thread
should become
class EncryptThread implements Runnable
then you do
new Thread(new EncryptThread()).start();
when you pass an instance of a class that implements Runnable, or in shorter terms, an instance of Runnable (they are the same) into the Thread constructor, then when you call .start() it actually kicks off in the .run() method of the Runnable instance.
Try to add the following codes:
PrintWriter file = new PrintWriter(new BufferedWriter(new FileWriter(OUTPUT_FILE_NAME, true)));
file.println(finalString);file.close();
...right before the line of code PassArray.passwordArray[j] = finalString; in your thread.
Edit: Remove the previous codes from the main thread.

Retrieve data from via UART

I am using UART to get the input to my system from PIC Development Board. and I use the following code to get the values from the board.
public SerialReader( InputStream in ) {
this.in = in;
}
public void run() {
byte[] buffer = new byte[ 1024 ];
int len = -1;
try {
/*len = this.in.read(buffer);
int x = buffer[0] & 0xff;
System.out.println(buffer[0]);
System.out.println(x);
System.out.println((char)x);*/
while( ( len = this.in.read( buffer ) ) > -1 ) {
System.out.print( new String( buffer, 0, len ) );
System.out.println(len);
/*String s = new String(buffer); //buffer.toString(); 1
System.out.println(s);
for (int i=0; i<=buffer.length; i++)
System.out.println(buffer[i]);
//System.out.print( new String( buffer, 0, len ) );
*/ }
} catch( IOException e ) {
e.printStackTrace();
}
}
}
Output: ààà
Expected Output: #C01=0155,INT=16:11,OUT=05:11
How do I retrive the expected output.

Java Client to C# server. Not receiving negative numbers

I have a Java app sending coordinates to a c# game in Unity. My data checks out with the exception of the negative values. I instead only get 63 or 250 something. Depending on the encoding.
Here is my Java client app:
#Override
public void run() {
//Keep in a loop as long as the running variable is true
while(running)
{
//Try catch block to catch exceptions for the networking code
try{
data1[0]=currentPlayer; // These are integers
.....
data1[8]=sendPause;
for(int i=0; i<=9;i++){
buf1[i]=(byte)data1[i]; //Casting to byte array
//System.out.println(buf1[i]);
}
//socket is created in "onCreate" earlier
socket.setTcpNoDelay(true);
out = socket.getOutputStream();
dos = new DataOutputStream(out);
dos.write(buf1, 0, buf1.length);
dos.flush();
synchronized(this){ this.wait(30); } //This is to minimize the frequency
}catch(Exception e)
{
Log.e("TCP", "Error",e);
}
}
}
This is my method for receiving in the C# server.
private void HandleClient(TcpClient client){
try{
byte[] bytes = new byte[256];
String data = null;
while(true)
{
data = null;
NetworkStream stream = client.GetStream();
int i;
while((i = stream.Read(bytes, 0, bytes.Length))!=0)
{
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
byte[] msg = System.Text.Encoding.UTF8.GetBytes(data);
/*sbyte[] sbytes = new sbyte[msg.Length];
for (int j = 0; j < msg.Length; j++)
sbytes[j] = (sbyte)msg[j];
Debug.Log(sbytes[2]+" "+sbytes[3]);*/
packetID=msg[0];
switch(packetID){
case 1:
p1.joyLx = msg[1];
....
p1.pause = msg[8];
p1.active=true;
break;
....
case 4:
p4.joyLx = msg[1];
....
p4.active=true;
break;
}
}
client.Close();
}
}
catch(SocketException e)
{
Debug.Log("SocketException:"+e);
}
}
I've been looking into the whole signed/unsigned byte ordeal but I'm worried that upon saving it into a byte array beforehand is erasing this data.
P.S. this is my first visit here and I'm not really a programmer so I've probably missed something obvious. So sorry if I'm not accustomed...
Got it working! Thanks for the help and pointing out my oddities!
[...]byte[] bytes = new byte[256];
sbyte[] sbytes;
while(true)
{
NetworkStream stream = client.GetStream();
int i;
while((i = stream.Read(bytes, 0, bytes.Length))!=0)
{
sbytes = new sbyte[bytes.Length];
for (int j = 0; j < bytes.Length; j++)
sbytes[j] = (sbyte)bytes[j];
packetID=bytes[0]; [...]
I think your problem is that you are using a byte array in your C# code. A byte in java has a range of -128 to 127. A byte in C# ranges from 0 to 255. Try copying the received byte array into an sbyte array:
sbyte[] signedB = new sbyte[bytes.Length];
Buffer.BlockCopy(bytes, 0, signedB, 0, bytes.Length);
Check the values of signedB after this.

Byte array not fully consumed

I am writing an FLV parser in Java and have come up against an issue. The program successfully parses and groups together tags into packets and correctly identifies and assigns a byte array for each tag's body based upon the BodyLength flag in the header. However in my test files it successfully completes this but stops before the last 4 bytes.
The byte sequence left out in the first file is :
00 00 14 C3
And in the second:
00 00 01 46
Clearly it is an issue with the final 4 bytes of both files however I cannot spot the error in my logic. I suspect it might be:
while (in.available() != 0)
However I also doubt this is the case as the program is successfully entering the loop for the final tag however it is just stopping 4 bytes short. Any help is greatly appreciated. (I know proper exception handling is as yet not taking place)
Parser.java
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.lang.reflect.Array;
import java.net.URI;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.InputMismatchException;
/**
*
* #author A
*
* Parser class for FLV files
*/
public class Parser {
private static final int HEAD_SIZE = 9;
private static final int TAG_HEAD_SIZE = 15;
private static final byte[] FLVHEAD = { 0x46, 0x4C, 0x56 };
private static final byte AUDIO = 0x08;
private static final byte VIDEO = 0x09;
private static final byte DATA = 0x12;
private static final int TYPE_INDEX = 4;
private File file;
private FileInputStream in;
private ArrayList<Packet> packets;
private byte[] header = new byte[HEAD_SIZE];
Parser() throws FileNotFoundException {
throw new FileNotFoundException();
}
Parser(URI uri) {
file = new File(uri);
init();
}
Parser(File file) {
this.file = file;
init();
}
private void init() {
packets = new ArrayList<Packet>();
}
public void parse() {
boolean test = false;
try {
test = parseHeader();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
if (test) {
System.out.println("Header Verified");
// Add header packet to beginning of list & then null packet
Packet p = new Packet(PTYPE.P_HEAD);
p.setSize(header.length);
p.setByteArr(header);
packets.add(p);
p = null;
try {
parseTags();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
} else {
try {
in.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// throw FileNotFoundException because incorrect file
}
}
private boolean parseHeader() throws FileNotFoundException, IOException {
if (file == null)
throw new FileNotFoundException();
in = new FileInputStream(file);
in.read(header, 0, 9);
return Arrays.equals(FLVHEAD, Arrays.copyOf(header, FLVHEAD.length));
}
private void parseTags() throws IOException {
if (file == null)
throw new FileNotFoundException();
byte[] tagHeader = new byte[TAG_HEAD_SIZE];
Arrays.fill(tagHeader, (byte) 0x00);
byte[] body;
byte[] buf;
PTYPE pt;
int OFFSET = 0;
while (in.available() != 0) {
// Read first 5 - bytes, previous tag size + tag type
in.read(tagHeader, 0, 5);
if (tagHeader[TYPE_INDEX] == AUDIO) {
pt = PTYPE.P_AUD;
} else if (tagHeader[TYPE_INDEX] == VIDEO) {
pt = PTYPE.P_VID;
} else if (tagHeader[TYPE_INDEX] == DATA) {
pt = PTYPE.P_DAT;
} else {
// Header should've been dealt with - if previous data types not
// found then throw exception
System.out.println("Unexpected header format: ");
System.out.print(String.format("%02x\n", tagHeader[TYPE_INDEX]));
System.out.println("Last Tag");
packets.get(packets.size()-1).diag();
System.out.println("Number of tags found: " + packets.size());
throw new InputMismatchException();
}
OFFSET = TYPE_INDEX;
// Read body size - 3 bytes
in.read(tagHeader, OFFSET + 1, 3);
// Body size buffer array - padding for 1 0x00 bytes
buf = new byte[4];
Arrays.fill(buf, (byte) 0x00);
// Fill size bytes
buf[1] = tagHeader[++OFFSET];
buf[2] = tagHeader[++OFFSET];
buf[3] = tagHeader[++OFFSET];
// Calculate body size
int bSize = ByteBuffer.wrap(buf).order(ByteOrder.BIG_ENDIAN)
.getInt();
// Initialise Array
body = new byte[bSize];
// Timestamp
in.read(tagHeader, ++OFFSET, 3);
Arrays.fill(buf, (byte) 0x00);
// Fill size bytes
buf[1] = tagHeader[OFFSET++];
buf[2] = tagHeader[OFFSET++];
buf[3] = tagHeader[OFFSET++];
int milliseconds = ByteBuffer.wrap(buf).order(ByteOrder.BIG_ENDIAN)
.getInt();
// Read padding
in.read(tagHeader, OFFSET, 4);
// Read body
in.read(body, 0, bSize);
// Diagnostics
//printBytes(body);
Packet p = new Packet(pt);
p.setSize(tagHeader.length + body.length);
p.setByteArr(concat(tagHeader, body));
p.setMilli(milliseconds);
packets.add(p);
p = null;
// Zero out for next iteration
body = null;
Arrays.fill(buf, (byte)0x00);
Arrays.fill(tagHeader, (byte)0x00);
milliseconds = 0;
bSize = 0;
OFFSET = 0;
}
in.close();
}
private byte[] concat(byte[] tagHeader, byte[] body) {
int aLen = tagHeader.length;
int bLen = body.length;
byte[] C = (byte[]) Array.newInstance(tagHeader.getClass()
.getComponentType(), aLen + bLen);
System.arraycopy(tagHeader, 0, C, 0, aLen);
System.arraycopy(body, 0, C, aLen, bLen);
return C;
}
private void printBytes(byte[] b) {
System.out.println("\n--------------------");
for (int i = 0; i < b.length; i++) {
System.out.print(String.format("%02x ", b[i]));
if (((i % 8) == 0 ) && i != 0)
System.out.println();
}
}
}
Packet.java
public class Packet {
private PTYPE type = null;
byte[] buf;
int milliseconds;
Packet(PTYPE t) {
this.setType(t);
}
public void setSize(int s) {
buf = new byte[s];
}
public PTYPE getType() {
return type;
}
public void setType(PTYPE type) {
if (this.type == null)
this.type = type;
}
public void setByteArr(byte[] b) {
this.buf = b;
}
public void setMilli(int milliseconds) {
this.milliseconds = milliseconds;
}
public void diag(){
System.out.println("|-- Tag Type: " + type);
System.out.println("|-- Milliseconds: " + milliseconds);
System.out.println("|-- Size: " + buf.length);
System.out.println("|-- Bytes: ");
for(int i = 0; i < buf.length; i++){
System.out.print(String.format("%02x ", buf[i]));
if (((i % 8) == 0 ) && i != 0)
System.out.println();
}
System.out.println();
}
}
jFLV.java
import java.net.URISyntaxException;
public class jFLV {
/**
* #param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
Parser p = null;
try {
p = new Parser(jFLV.class.getResource("sample.flv").toURI());
} catch (URISyntaxException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
p.parse();
}
}
PTYPE.java
public enum PTYPE {
P_HEAD,P_VID,P_AUD,P_DAT
};
Both your use of available() and your call to read are broken. Admittedly I would have somewhat expected this to be okay for a FileInputStream (until you reach the end of the stream, at which point ignoring the return value for read could still be disastrous) but I personally assume that streams can always return partial data.
available() only tells you whether there's any data available right now. It's very rarely useful - just ignore it. If you want to read until the end of the stream, you should usually keep calling read until it returns -1. It's slightly tricky to combine that with "I'm trying to read the next block", admittedly. (It would be nice if InputStream had a peek() method, but it doesn't. You can wrap it in a BufferedInputStream and use mark/reset to test that at the start of each loop... ugly, but it should work.)
Next, you're ignoring the result of InputStream.read (in multiple places). You should always use the result of this, rather than assuming it has read the amount of data you've asked for. You might want a couple of helper methods, e.g.
static byte[] readExactly(InputStream input, int size) throws IOException {
byte[] data = new byte[size];
readExactly(input, data);
return data;
}
static void readExactly(InputStream input, byte[] data) throws IOException {
int index = 0;
while (index < data.length) {
int bytesRead = input.read(data, index, data.length - index);
if (bytesRead < 0) {
throw new EOFException("Expected more data");
}
}
}
You should use one of the read methods instead of available, as available() "Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream."
It is not designed to check how long you can read.

Categories