We Keep Coding

Python byte object vs java

Good morning, I have a quick question regarding the differences between a byte object in python (denoted b'') and how to replicate it in java.
The project I am working on is some personal work on an emulation server for a dead game to better my reversing skills. I have a working rendition of the project in python, but would like to switch over to java as I am better with the language and it comes with many additional tools included that are useful for a project like this.
I am using a ServerSocket to capture TCP data in the java project.
When data comes over the network from the Python project it looks a little something like this:
When I capture the same data over the java ServerSocket I get something like this:
My question is how can I reformat this ASCII text to get the proper data as seen in the python version of the software.
Currently I am able to get an output like this:
By converting the byte[] data from the ServerSocket as such
while(true) {
try {
Socket socket = serverSocket.accept();
onConnection(socket);
byte[] incomingData = new byte[0];
byte[] temp = new byte[1024];
int k = -1;
//this is due to the client of said game not sending EOL (readLine() does not work here)
while((k = socket.getInputStream().read(temp, 0, temp.length)) > -1) {
byte[] tbuff = new byte[incomingData.length + k];
System.arraycopy(incomingData, 0, tbuff, 0, incomingData.length);
System.arraycopy(temp, 0, tbuff, incomingData.length, k);
incomingData = tbuff;
receiveData(socket, incomingData); <--- this is the important bit
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void receiveData(Socket socket, byte[] data) {
int lenLo = (int) (data[0]);
int lenHi = (int) (data[1]);
int length = lenHi * 256 + lenLo;
if(lenHi < 0) {
System.out.println("Invalid Packet Length");
}
if(data.length != length) {
System.out.println("Incomplete Packet Received");
}
try {
String test = new String(data, "UTF-8");
serverGUI.serverDebug(test); //produces the string in a jframe (pic 2)
serverGUI.debugByteArray(test.getBytes(StandardCharsets.UTF_8)); //produces the byte[] in jframe (pic 3 -- all bytes in this array are & 0xff prior to being printed out)
} catch (UnsupportedEncodingException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
}
However this has obviously not produced the desired outcome. Any advice is appreciated or any resources that can be put forth are also appreciated.
Thanks in advance!

Vertx NetServer control read flow

I am trying to mimic a TCP server for tests with Vertx based on existing infrastructure that I have to work with.
The server I am mimicking works completely async and knows the length of the incoming buffer based on a pre-header in the buffer that indicates the length of the request.
I need to read the first 6 characters of the incoming request on each client socket that connect to my mock TCP server. from this pre-header I read the actual length of the request (e.g. for xx3018, i know the full length of the request is 3018).
Then I need to read the rest of the buffer according to the length, match it to a map of responses and return the right response for the request.
Example for a working mock server with plain java (fast implementation so other development won't be blocked :) )
public void run(String... args) throws Exception {
log.info("Starting TCP Server");
ServerSocket serverSocket = new ServerSocket(1750);
while (true) {
try {
Socket socket = serverSocket.accept();
CompletableFuture.runAsync(() -> {
Exception e = null;
while (e == null) {
try {
InputStream inputStream = socket.getInputStream();
OutputStream outputStream = socket.getOutputStream();
byte[] preHeader = new byte[6];
inputStream.read(preHeader);
String preHeaderValue = new String(preHeader);
log.info("Pre header: {}", preHeaderValue);
int length = Integer.valueOf(preHeaderValue.substring(2));
log.info("Request full length: {}", length);
byte[] request = new byte[length - 6];
inputStream.read(request);
String requestValue = new String(request);
log.info("Request: {}", requestValue);
String response = this.requestResponseProvider.getResponse(preHeaderValue + requestValue);
log.info("Response: {}", response);
outputStream.write(response.getBytes());
} catch (Exception ex) {
log.error("Encountered a problem: {}", e.getMessage());
e = ex;
}
}
});
} catch (Exception e) {
log.error("Encountered a problem: {}", e.getMessage());
}
}
}
I can't seem to find a way to control the input stream the same way I control it with plain java.

After a very long time of leaving this issue aside, I decided to play with it a bit.
I remembered using the following module for a different project: https://github.com/vert-x3/vertx-tcp-eventbus-bridge
I also remembered that in the tcp bridge's internal protocol, it appends the length of the payload to the buffer that is being sent via the tcp bridge, I looked into the source code to find out how it handles chunks (aka frames)
I found the following: https://github.com/vert-x3/vertx-tcp-eventbus-bridge/blob/master/src/main/java/io/vertx/ext/eventbus/bridge/tcp/impl/protocol/FrameParser.java which does exactly what I wanted to achieve :)
I modified it a bit, converted to Kotlin, and made it so I can control the header size and the way it extracts the payload length.
The following is a rough quick and dirty example of controlling the read flow with Vert.x NetServer:
suspend fun main() {
val vertx = Vertx.vertx()
initServer(vertx)
initClient(vertx)
}
suspend fun initServer(vertx: Vertx) {
val server = vertx.createNetServer(netServerOptionsOf(port = 8888, host = "localhost"))
server
.connectHandler { socket ->
val parser = FrameParser(
headerSize = 4,
headerHandler = {
it.getInt(0)
},
handler = {
println(it.toString())
println("---")
}
)
socket.handler(parser)
socket.exceptionHandler {
it.printStackTrace()
socket.close()
}
}
.listenAwait()
}
suspend fun initClient(vertx: Vertx) {
val client = vertx.createNetClient()
val socket = client.connectAwait(port = 8888, host = "localhost")
val message = "START|${"foobarfoobar".repeat(100)}|END"
val length = message.length
repeat(5) {
repeat(100) {
vertx.setPeriodic(10) {
socket.write(
Buffer.buffer()
.appendInt(length)
.appendString(message)
)
}
}
delay(1000)
}
}
/**
* Based on: https://github.com/vert-x3/vertx-tcp-eventbus-bridge/blob/master/src/main/java/io/vertx/ext/eventbus/bridge/tcp/impl/protocol/FrameParser.java
*/
class FrameParser(
private val headerSize: Int,
private val headerHandler: (Buffer) -> Int,
private val handler: (Buffer) -> Unit
) : Handler<Buffer?> {
private var _buffer: Buffer? = null
private var _offset = 0
override fun handle(buffer: Buffer?) {
append(buffer)
var offset: Int
while (true) {
// set a rewind point. if a failure occurs,
// wait for the next handle()/append() and try again
offset = _offset
// how many bytes are in the buffer
val remainingBytes = bytesRemaining()
// at least expected header size
if (remainingBytes < headerSize) {
break
}
// what is the length of the message
val length: Int = headerHandler(_buffer!!.getBuffer(_offset, _offset + headerSize))
_offset += headerSize
if (remainingBytes - headerSize >= length) {
// we have a complete message
handler(_buffer!!.getBuffer(_offset, _offset + length))
_offset += length
} else {
// not enough data: rewind, and wait
// for the next packet to appear
_offset = offset
break
}
}
}
private fun append(newBuffer: Buffer?) {
if (newBuffer == null) {
return
}
// first run
if (_buffer == null) {
_buffer = newBuffer
return
}
// out of data
if (_offset >= _buffer!!.length()) {
_buffer = newBuffer
_offset = 0
return
}
// very large packet
if (_offset > 0) {
_buffer = _buffer!!.getBuffer(_offset, _buffer!!.length())
}
_buffer!!.appendBuffer(newBuffer)
_offset = 0
}
private fun bytesRemaining(): Int {
return if (_buffer!!.length() - _offset < 0) {
0
} else {
_buffer!!.length() - _offset
}
}
}

Is there any way to send an array of Point with DataOutputStream over a Socket?

I want to send an array of point (Point points[] = new point[20]) with DataOutputStream over a socket and be able to correctly read it with DataInputStream on the other side. I CANNOT send each element separately, it must be sent as a whole array and be able to be interpreted as a whole array.

See e.g. the section "Transporting Custom Objects" in Advanced Socket Programming:
Put your data into a Serializable
class, say MyData. (You don't really have to do this in your case, because arrays implement Serializable, but there is a chance that later you will want to send other data along with your point array...)
On the sender side, create a MyData
object and fill it with data. Cast socket.getOutputStream() to an ObjectOutputStream, and
call its writeObject method to send the MyData object.
On the receiver
side, cast socket.getInputStream() to an ObjectInputStream, and call
its readObject method to receive the object (you will have to cast
it to MyData).
Anyway, I would also consider using RMI. For example, you could create a PointServer, register it in the RMI registry, and access it through simple function calls in your client application. RMI is much easier to work with than sockets. For example, you don't have to design a protocol, you only need to define methods for your distributed objects.

Why not use an ObjectInputStream/ObjectOutputStream instead?
Here's a piece of code for something I wrote long ago, it will allow you to send any kind of object through the wire so long as it's serializable, obviously:
ObjectInputStream input = new ObjectInputStream(communicationSocket.getInputStream());
if (input != null) {
try {
Object o = input.readObject();
if (o != null && o instanceof DataObject) {
DataObject response = (DataObject) o;
{
//Do something cool with DataObject
}
}
}//etc...
DataObject is just a "wrapper"/custom class that you can define as having as one of it's properties a Point[]; in other words, you could define DataObject as:
import java.io.Serializable;
public class DataObject implements Serializable {
private Point[] points; //etc - define getter setter for points
}
And you can write it as:
output = new ObjectOutputStream(communicationSocket
.getOutputStream());
output.writeObject(yourDataObject);
Now, you can read this as so:
while (listening) {
try {
Object currentObject = input.readObject();
if (currentObject != null
&& currentObject instanceof DataObject) {
Point[] points = ((DataObject) currentObject).getPoints();
//Do something with points
}
}
catch (IOException e) {
e.printStackTrace();
}
catch (ClassNotFoundException e) {
e.printStackTrace();
}
finally {
listening = false;
}
}

Well, there are some points that you should be aware of, for example, the size of an int 32 or 64 bits, the byte order of the sending and the receiving machines, etc.
Also, there should be a protocol to communicate both sockets, I mean, there should be a header in every packets send so that you know how much data you are going to received, I won't deal with that, I assume that you want to send and receive an array of 20 points.
In this example it's assumed that both machines are 32 bits, i.e. an int is 4 bytes in size and that both machines use LITTLE_ENDIAN byte order.
// To send an array of points use these two methods:
public byte[] pointsToBytes(Point[] p, int numPts)
{
// 2 is for x and y and 4 is the size of an int in bytes
ByteBuffer buff = ByteBuffer.allocateDirect(numPts * 2 * 4);
buff.order(ByteOrder.LITTLE_ENDIAN);
for (int i = 0; i < numPts; i++)
{
buff.putInt(p[i].x);
buff.putInt(p[i].y);
}
return buff.array();
}
public boolean sendData(Point[] pts, int size)
{
try
{
byte[] buffer = pointsToBytes(pts, size);
mainSocket.getOutputStream().write(buffer, 0, buffer.length);
return true;
}
catch (Exception e)
{
e.printStackTrace();
}
return false;
}
// To receive an array of point use these two methods:
public Point[] bytesToPoints(byte[] byteBuff, int numBytes)
{
ByteBuffer buff = ByteBuffer.wrap(byteBuff);
buff.order(ByteOrder.LITTLE_ENDIAN);
// 2 is for x and y and 4 is the size of an int in bytes
int ptsInBuffer = numBytes / 2 / 4;
Point[] pts = new Point[ptsInBuffer];
for (int i = 0; i < ptsInBuffer; i++)
{
pts[i].x=buff.getInt();
pts[i].y=buff.getInt();
}
return pts;
}
public Point[] getData(int pointsToRead)
{
try
{
byte[] buffer = new byte[pointsToRead * 4 * 2];
int bytesRead = mainSocket.getInputStream().read(buffer, 0, pointsToRead*4*2);
return bytesToPoints(buffer, bytesRead);
}
catch (Exception e)
{
e.printStackTrace();
}
return null;
}

How to get sound from java-based services

I have a Java-based server side and a flex client side using Spring BlazeDS Integration. It works fine, but I want to get sound from server side recently.
I followed this BlazeDS mapping doc, it says when Java return a Byte[], it will be converted to ByteArray which I want. So I handle the MP3 file by ByteArrayOutputStream, convert it to Byte[] and return it back to front-end, but the value that Actionscript gets turns to be null value.
public Byte[] sayHello() {
Byte[] ba = null;
try {
FileInputStream fis = new FileInputStream(
"D:/e/Ryan Adams - I Wish You Were Here.mp3");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = fis.read(buffer)) > 0) {
baos.write(buffer, 0, bytesRead);
}
byte[] byteArray = baos.toByteArray();
ba = new Byte[byteArray.length];
for (int i = 0; i < byteArray.length; i++) {
ba[i] = Byte.valueOf(byteArray[i]);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return ba;
}
The ActionScript code:
<s:RemoteObject id="ro" destination="helloWorldService" fault="handleFault(event)">
<s:channelSet>
<s:ChannelSet>
<s:AMFChannel uri="/flexspring/messagebroker/amf"/>
</s:ChannelSet>
</s:channelSet>
</s:RemoteObject>
...
private function loaded():void {
var bArr:ByteArray = ro.sayHello() as ByteArray;
l.text = "" + (bArr == null);
}
...
<s:Label id="l" text=""/>
And it says "true". Does anyone have any idea what's the problem.

The problem with your code is that all flex calls over a BlazeDS are async. So, ro.SomeMethod() doesn't return immediately, it queues it up and then does callbacks as necessary.
Here's an example of something that works Note that I've never sent byte[] over a BlazeDS connection, but I don't see why it wouldn't work --- as J_A_X suggests, you probably want to stream the sound, rather than sending the whole thing at once.
Anyway - here's the example:
public function loaded():void
{
var token:AsyncToken = ro.sayHello();
token.addResponder(new mx.rpc.Responder(result, fault));
// ...Code continues to execute...
}
public function result(event:ResultEvent):void
{
// The byte[] is in event.result
var bArr:ByteArray = event.result as ByteArray;
}
public function fault(event:FaultEvent):void
{
// Something went wrong (maybe the server on the other side went AWOL)
}

You can return the sound bytes through a web service. After you got the bytes, you can add it to a Sound object and play that. The only problem is that since it's a web service, the client will have to load all the bytes before it can play. If you want to stream the sound, you'll need a streaming server like FMS or Wowza (I recommend the latter).

Sending an int from Java to C using sockets

I was just wondering how to send an int from a Java application to a C application using sockets. I have got different C programs communicating with each other and have got the Java application retrieving data from the C application, but I can't work out sending.
The C application is acting as database, the Java application then sends a user id (a 4 digit number) to the C application, if it exists it returns that record's details.
In Java I have tried using a printWriter and DataOutputStream to send the data, printWriter produces weird symbols and DataOutputStream produces "prof_agent.so".
Any help would be appreciated as I don't have a good grasp of sockets at the moment.

You can use DataOutputStream.writeInt. It writes an int already in network byte order by contract.
On a C side you can call recv, or read to fill in the 4-byte buffer, and then you can use ntohl ( Network-TO-Host-Long ) to convert the value you've just read to your platform int representation.

You can send the textual representation. So the number 123 would be sent as 3 bytes '1' '2' '3'.

It's a bit too late but let this answer be here. Using UDP sockets:
Java code:
public void runJavaSocket() {
System.out.println("Java Sockets Program has started."); int i=0;
try {
DatagramSocket socket = new DatagramSocket();
System.out.println("Sending the udp socket...");
// Send the Message "HI"
socket.send(toDatagram("HI",InetAddress.getByName("127.0.0.1"),3800));
while (true)
{
System.out.println("Sending hi " + i);
Thread.currentThread();
Thread.sleep(1000);
socket.send(toDatagram("HI " + String.valueOf(i),InetAddress.getByName("127.0.0.1"),3800));
i++;
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
public DatagramPacket toDatagram(
String s, InetAddress destIA, int destPort) {
// Deprecated in Java 1.1, but it works:
byte[] buf = new byte[s.length() + 1];
s.getBytes(0, s.length(), buf, 0);
// The correct Java 1.1 approach, but it's
// Broken (it truncates the String):
// byte[] buf = s.getBytes();
return new DatagramPacket(buf, buf.length,
destIA, destPort);
}
C# code:
string returnData;
byte[] receiveBytes;
//ConsoleKeyInfo cki = new ConsoleKeyInfo();
using (UdpClient udpClient = new UdpClient(new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3800)))
{
IPEndPoint remoteIpEndPoint = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3800);
while (true)
{
receiveBytes = udpClient.Receive(ref remoteIpEndPoint);
returnData = Encoding.ASCII.GetString(receiveBytes);
Console.WriteLine(returnData);
}
}

Try this:
Socket s = ...;
DataOutputStream out = null;
try {
out = new DataOutputStream( s.getOutputStream() );
out.writeInt( 123456 );
} catch ( IOException e ) {
// TODO Handle exception
} finally {
if ( out != null ) {
try {
out.close();
} catch ( IOException e ) {
// TODO Handle exception
}
}
}
It whould help if you could explain a little more what your problem is.