We Keep Coding

Vertx NetServer control read flow

I am trying to mimic a TCP server for tests with Vertx based on existing infrastructure that I have to work with.
The server I am mimicking works completely async and knows the length of the incoming buffer based on a pre-header in the buffer that indicates the length of the request.
I need to read the first 6 characters of the incoming request on each client socket that connect to my mock TCP server. from this pre-header I read the actual length of the request (e.g. for xx3018, i know the full length of the request is 3018).
Then I need to read the rest of the buffer according to the length, match it to a map of responses and return the right response for the request.
Example for a working mock server with plain java (fast implementation so other development won't be blocked :) )
public void run(String... args) throws Exception {
log.info("Starting TCP Server");
ServerSocket serverSocket = new ServerSocket(1750);
while (true) {
try {
Socket socket = serverSocket.accept();
CompletableFuture.runAsync(() -> {
Exception e = null;
while (e == null) {
try {
InputStream inputStream = socket.getInputStream();
OutputStream outputStream = socket.getOutputStream();
byte[] preHeader = new byte[6];
inputStream.read(preHeader);
String preHeaderValue = new String(preHeader);
log.info("Pre header: {}", preHeaderValue);
int length = Integer.valueOf(preHeaderValue.substring(2));
log.info("Request full length: {}", length);
byte[] request = new byte[length - 6];
inputStream.read(request);
String requestValue = new String(request);
log.info("Request: {}", requestValue);
String response = this.requestResponseProvider.getResponse(preHeaderValue + requestValue);
log.info("Response: {}", response);
outputStream.write(response.getBytes());
} catch (Exception ex) {
log.error("Encountered a problem: {}", e.getMessage());
e = ex;
}
}
});
} catch (Exception e) {
log.error("Encountered a problem: {}", e.getMessage());
}
}
}
I can't seem to find a way to control the input stream the same way I control it with plain java.

After a very long time of leaving this issue aside, I decided to play with it a bit.
I remembered using the following module for a different project: https://github.com/vert-x3/vertx-tcp-eventbus-bridge
I also remembered that in the tcp bridge's internal protocol, it appends the length of the payload to the buffer that is being sent via the tcp bridge, I looked into the source code to find out how it handles chunks (aka frames)
I found the following: https://github.com/vert-x3/vertx-tcp-eventbus-bridge/blob/master/src/main/java/io/vertx/ext/eventbus/bridge/tcp/impl/protocol/FrameParser.java which does exactly what I wanted to achieve :)
I modified it a bit, converted to Kotlin, and made it so I can control the header size and the way it extracts the payload length.
The following is a rough quick and dirty example of controlling the read flow with Vert.x NetServer:
suspend fun main() {
val vertx = Vertx.vertx()
initServer(vertx)
initClient(vertx)
}
suspend fun initServer(vertx: Vertx) {
val server = vertx.createNetServer(netServerOptionsOf(port = 8888, host = "localhost"))
server
.connectHandler { socket ->
val parser = FrameParser(
headerSize = 4,
headerHandler = {
it.getInt(0)
},
handler = {
println(it.toString())
println("---")
}
)
socket.handler(parser)
socket.exceptionHandler {
it.printStackTrace()
socket.close()
}
}
.listenAwait()
}
suspend fun initClient(vertx: Vertx) {
val client = vertx.createNetClient()
val socket = client.connectAwait(port = 8888, host = "localhost")
val message = "START|${"foobarfoobar".repeat(100)}|END"
val length = message.length
repeat(5) {
repeat(100) {
vertx.setPeriodic(10) {
socket.write(
Buffer.buffer()
.appendInt(length)
.appendString(message)
)
}
}
delay(1000)
}
}
/**
* Based on: https://github.com/vert-x3/vertx-tcp-eventbus-bridge/blob/master/src/main/java/io/vertx/ext/eventbus/bridge/tcp/impl/protocol/FrameParser.java
*/
class FrameParser(
private val headerSize: Int,
private val headerHandler: (Buffer) -> Int,
private val handler: (Buffer) -> Unit
) : Handler<Buffer?> {
private var _buffer: Buffer? = null
private var _offset = 0
override fun handle(buffer: Buffer?) {
append(buffer)
var offset: Int
while (true) {
// set a rewind point. if a failure occurs,
// wait for the next handle()/append() and try again
offset = _offset
// how many bytes are in the buffer
val remainingBytes = bytesRemaining()
// at least expected header size
if (remainingBytes < headerSize) {
break
}
// what is the length of the message
val length: Int = headerHandler(_buffer!!.getBuffer(_offset, _offset + headerSize))
_offset += headerSize
if (remainingBytes - headerSize >= length) {
// we have a complete message
handler(_buffer!!.getBuffer(_offset, _offset + length))
_offset += length
} else {
// not enough data: rewind, and wait
// for the next packet to appear
_offset = offset
break
}
}
}
private fun append(newBuffer: Buffer?) {
if (newBuffer == null) {
return
}
// first run
if (_buffer == null) {
_buffer = newBuffer
return
}
// out of data
if (_offset >= _buffer!!.length()) {
_buffer = newBuffer
_offset = 0
return
}
// very large packet
if (_offset > 0) {
_buffer = _buffer!!.getBuffer(_offset, _buffer!!.length())
}
_buffer!!.appendBuffer(newBuffer)
_offset = 0
}
private fun bytesRemaining(): Int {
return if (_buffer!!.length() - _offset < 0) {
0
} else {
_buffer!!.length() - _offset
}
}
}

send an integer from a C client to a Java server

I use this code to send an integer from my Java Client to my Java Server
int n = rand.nextInt(50) + 1;
DataOutputStream dos = new DataOutputStream(_socket.getOutputStream());
dos.writeInt(n);
And i read it in the server with this code
DataInputStream din = new DataInputStream(socket.getInputStream());
int ClientNumber= din.readInt();
System.out.println(ClientNumber);
ClientNumber++;
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
dos.writeInt(ClientNumber);
String randomString= getRandomValue(10,20);
dos.writeUTF(randomString);
It work perfectly but now i want to write a C client
I tried this code
#include <stdio.h>
#include <errno.h>
#include <signal.h>
#include <netdb.h>
#include <netinet/in.h>
#include <sys/socket.h>
#define SERVEURNAME "localhost" // adresse IP de mon serveur
int to_server_socket = -1;
void main ( void )
{
char *server_name = SERVEURNAME;
struct sockaddr_in serverSockAddr;
struct hostent *serverHostEnt;
long hostAddr;
long status;
char buffer[512];
bzero(&serverSockAddr,sizeof(serverSockAddr));
hostAddr = inet_addr(SERVEURNAME);
if ( (long)hostAddr != (long)-1)
bcopy(&hostAddr,&serverSockAddr.sin_addr,sizeof(hostAddr));
else
{
serverHostEnt = gethostbyname(SERVEURNAME);
if (serverHostEnt == NULL)
{
printf("gethost rate\n");
exit(0);
}
bcopy(serverHostEnt->h_addr,&serverSockAddr.sin_addr,serverHostEnt->h_length);
}
serverSockAddr.sin_port = htons(8071);
serverSockAddr.sin_family = AF_INET;
/* creation de la socket */
if ( (to_server_socket = socket(AF_INET,SOCK_STREAM,0)) < 0)
{
printf("creation socket client ratee\n");
exit(0);
}
/* requete de connexion */
if(connect( to_server_socket,
(struct sockaddr *)&serverSockAddr,
sizeof(serverSockAddr)) < 0 )
{
printf("demande de connection ratee\n");
exit(0);
}
/* envoie de donne et reception */
int value = htons( 4 );
write( to_server_socket, &value, sizeof( value ) );
printf(buffer);
}
but it don't work. I use Eclipse for to compile the java code and running the Server and xcode for the C code ( the Client ) but i don't think that the problem is there
Edit:
I got an error on the server
java.net.SocketException: Broken pipe at
java.net.SocketOutputStream.socketWrite0(Native Method) at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:115) at
java.io.DataOutputStream.writeInt(DataOutputStream.java:182) at
ServiceRequest.run(ServiceRequest.java:36) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
I think it's because i the server wait for an integer but it isn't ...?

Two things that jump out at me in this statement...
write(to_server_socket,"4",4);
1) "4" is not an integer, it's a null terminated string (well, okay, it is an integer, but it's not what you "meant" to do me thinks)
2) You are sending the value in host-byte-order which may or may not be the same as network-byte-order
int value = htons( 4 );
write( to_server_socket, &value, sizeof( value ) );
Beyond that, however, the "broken pipe" error from the java socketWrite()would tend to indicate that your sending side (the C application) has closed the socket and your java side is still trying to write to it.
Your C client code is opening the socket, writing to it then immediately printing a buffer you never filled with anything and exiting the program. As soon as the program exits, the socket you created for it is closed, thus the "broken pipe" error in your Java server. You need to read a reply from the server...
int value = htonl( 4 );
int reply = 0;
if( send( to_server_socket, &value, sizeof( value ), 0 ) != sizeof( value ) )
{
printf( "socket write failed: %s", strerror( errno ) );
exit( -1 );
}
if( recv( to_server_socket, &reply, sizeof( reply ), MSG_WAITALL ) != sizeof( reply ) )
{
printf( "socket read failed: %s", streror( errno ) );
exit( -1 )
}
printf( "got reply: %d\n", ntohl( reply ) );
On a separate note... you indicate that you receive 262144 on the server... is that before or after the changes I suggested? 262144 is 0x00040000 -- so... you did get 4, just not where you expected to receive it. So, you're using 32 bit ints (I should have realized that) which means you want to use htonl() and ntohl() instead of the htons() and ntohs() which are short integer conversions.

Stack Smashing in Java Interposer

I am writing a Java interposer to modify network communication related system calls. Basically, I want to modify the IP and port of the intended recipient.
The code works correctly on my laptop, but on university PC, it gives a stack smashing error as:
*** stack smashing detected ***: java terminated
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(__fortify_fail+0x45)[0xb7702dd5]
/lib/i386-linux-gnu/libc.so.6(+0xffd8a)[0xb7702d8a]
/home/mwaqar/vibe/ldinterposer_2.so(+0x28e4)[0xb77c98e4]
/home/mwaqar/vibe/ldinterposer_2.so(connect+0x9c5)[0xb77c9093]
/usr/lib/jvm/java-7-openjdk-i386/jre/lib/i386/libnet.so(+0xceff)[0x8b226eff]
/usr/lib/jvm/java-7-openjdk-i386/jre/lib/i386/libnet.so(Java_java_net_PlainSocketImpl_socketConnect+0x4c1)[0x8b227c51]
The relevant code (interposition of connect system call) is as follows:
int connect(int fd, const struct sockaddr *sk, socklen_t sl)
{
struct sockaddr_in *lsk_in = (struct sockaddr_in *) sk;
struct sockaddr_in6 *lsk_in6 = (struct sockaddr_in6 *) sk;
struct sockaddr_in addr4;
unsigned int len;
int nbytes, oport, tport, ret, i;
char ip_address[30];
char buffer[1024];
char tempBuffer[1024];
if((lsk_in->sin_family == AF_INET) || (lsk_in->sin_family == AF_INET6))
{
if(lsk_in->sin_family == AF_INET)
{
oport = ntohs(lsk_in->sin_port);
memcpy(&addr4.sin_addr.s_addr, &lsk_in->sin_addr.s_addr, sizeof(addr4.sin_addr.s_addr));
}
else if(lsk_in->sin_family == AF_INET6)
{
oport = ntohs(lsk_in6->sin6_port);
memcpy(&addr4.sin_addr.s_addr, lsk_in6->sin6_addr.s6_addr+12, sizeof(addr4.sin_addr.s_addr));
}
memset(buffer, '\0', sizeof(buffer));
sprintf(buffer, "%s%c%s%c%i", NAT_VM_CONNECT_RULE, NAT_VM_DELIMITER, (char *)inet_ntoa(addr4.sin_addr), NAT_VM_DELIMITER, oport);
nbytes = send(sock, buffer, strlen(buffer), 0);
if(DEBUG_MODE)
fprintf(stdout, "[LD_INTERPOSER] Sent[%s]\n", buffer);
memset(buffer, '\0', sizeof(buffer));
nbytes = recv(sock, buffer, sizeof(buffer), 0);
fprintf(stderr, "[LD_INTERPOSER] Received CONNECT [%s]\n", buffer);
memset(ip_address, '\0', sizeof(ip_address));
int pos = strrchr(buffer, NAT_VM_DELIMITER) - buffer;
strncpy(ip_address, buffer, pos);
ip_address[pos] = '\0';
tport = atoi(buffer + pos + 1);
if(lsk_in->sin_family == AF_INET)
{
lsk_in->sin_addr.s_addr = inet_addr(ip_address + 7);
lsk_in->sin_port = htons(tport);
}
else if(lsk_in->sin_family == AF_INET6)
{
inet_pton(AF_INET6, ip_address, &(lsk_in6->sin6_addr));
lsk_in6->sin6_port = htons(tport);
}
fprintf(stderr, "[LD_INTERPOSER] IP[%s], Port[%d] for VM[%s]\n", ip_address, tport, vm_ip);
}
int my_ret = real_connect(fd, sk, sl);
fprintf(stderr, "Done\n");
return my_ret;
}
Here, sock is a socket that I have initialized in "constructor" of the shared library.
The program works correctly and prints Done. On the last (return) line, it gives the stack smashing error. I have no idea what is causing this.

I suspect that strrcr returns NULL in the line
int pos = strrchr(buffer, NAT_VM_DELIMITER) - buffer;
Then pos will be huge, and the following lines will read and write invalid addresses.
Always check the return value of functions (especially when they're run on data received from outside your program).
Also, as I wrote in my comment, never use sprintf. I can't tell if it fails, because I don't know what's NAT_VM_CONNECT_RULE. Even if you counted the bytes and know you're OK, you should still be careful and use snprintf instead.

Socket communication between Java and C++

I'm trying to have a connection between a Java server and a C++ client. But when I read the data in my client I always have the same strange character (’). I tried to change the encoding in both side but nothing work.
Here is my Java code :
public class Serveur
{
public static void main(String[] args) throws Exception
{
final int PORT = 13370;
try
{
ServerSocket service= new ServerSocket(PORT);
Socket connection = service.accept();
PrintWriter pw = new PrintWriter(connection.getOutputStream());
String s = Integer.toString(5);
while(true)
{
pw.print(s.getBytes("UTF-8"));
pw.flush();
pw.close();
}
connection.close();
}
}
I also tried to use an OutputStream, a DataOutputStream and a BufferedOutputStream.
And here is the C++ code :
int main(int argc, char* argv[])
{
WSADATA WSAData;
WSAStartup(MAKEWORD(2,0), &WSAData);
SOCKET sock;
SOCKADDR_IN sin;
char buffer[512];
sin.sin_addr.s_addr = inet_addr("127.0.0.1");
sin.sin_family = AF_INET;
sin.sin_port = htons(13370);
sock = socket(AF_INET,SOCK_STREAM,0);
if(connect(sock, (SOCKADDR*)&sin, sizeof(sin)) != SOCKET_ERROR)
{
cout<<"connection"<<endl;
if(recv(sock, buffer, sizeof(buffer), 0) != SOCKET_ERROR)
{
string s = buffer;
wchar_t *pwchello = L"Hi";
wchar_t *pwc = (wchar_t *)malloc( sizeof( wchar_t ));
char *pmbhello = buffer;
int i = mbstowcs(pwc,pmbhello, MB_CUR_MAX);
cout << i << endl;
cout<<"cout : "<<pwc<<endl;
cout <<buffer<<endl;
printf("printf : %s\n", buffer);
cout << "wsagetlasterror() : "<<WSAGetLastError();
closesocket(sock);
WSACleanup();
free(m_pBuffer);
}
return 0;
}
As you can see, I tried different solution but without success.
Thanks in advance, and sorry for my english it may be not very good

You are mixing up lots of different encoding conversions and I/O strategies. You should try out the following simplified version:
if(connect(sock, (SOCKADDR*)&sin, sizeof(sin)) != SOCKET_ERROR)
{
cout << "connection" << endl;
// the result of 'recv()' is either SOCKET_ERROR or
// the number of bytes received. don't though away
// the return value.
const int result = recv(sock, buffer, sizeof(buffer), 0);
if(result != SOCKET_ERROR)
{
// use length (in bytes) returned by 'recv()'
// since buffer is not null terminated.
string s(buffer,result);
// 's' is in UTF-8 no converstion to wide strings
// should be necessary.
cout << "message: '" << s << "'." << endl;
}
closesocket(sock);
}
WSACleanup();
However, note that the standard output is in the current code page and usually UTF-8 is not the default code page. Outputing Unicode data to the console in windows requires a few other library calls to configure.

recv does not turn its destination buffer into null-terminated string. It fills in a number of bytes in the buffer, but does not append a 0.
You need top do this (with error checking, of course):
ssize_t bytesRead = recv(buffer, ...);
string str(buffer, bytesRead);
Also, be aware that recv does not guarantee that something sent in one call gets received in one call (unless you're doing UDP).

You're only allocating room for a single wchar_t here:
wchar_t *pwc = (wchar_t *)malloc( sizeof( wchar_t ));
You also assign buffer to string s, but never seem to use s

I have been having the same problem since last night. Finally figured out that encoding is not recognized by my server (written in C). Therefore, I changed in my client
someOutputStream.writeUTF(someSillyString);
to
someOutputStream.write(someSillyString.getBytes());
This way, I did not even need to typecast on the server side.

Sending an int from Java to C using sockets

I was just wondering how to send an int from a Java application to a C application using sockets. I have got different C programs communicating with each other and have got the Java application retrieving data from the C application, but I can't work out sending.
The C application is acting as database, the Java application then sends a user id (a 4 digit number) to the C application, if it exists it returns that record's details.
In Java I have tried using a printWriter and DataOutputStream to send the data, printWriter produces weird symbols and DataOutputStream produces "prof_agent.so".
Any help would be appreciated as I don't have a good grasp of sockets at the moment.

You can use DataOutputStream.writeInt. It writes an int already in network byte order by contract.
On a C side you can call recv, or read to fill in the 4-byte buffer, and then you can use ntohl ( Network-TO-Host-Long ) to convert the value you've just read to your platform int representation.

You can send the textual representation. So the number 123 would be sent as 3 bytes '1' '2' '3'.

It's a bit too late but let this answer be here. Using UDP sockets:
Java code:
public void runJavaSocket() {
System.out.println("Java Sockets Program has started."); int i=0;
try {
DatagramSocket socket = new DatagramSocket();
System.out.println("Sending the udp socket...");
// Send the Message "HI"
socket.send(toDatagram("HI",InetAddress.getByName("127.0.0.1"),3800));
while (true)
{
System.out.println("Sending hi " + i);
Thread.currentThread();
Thread.sleep(1000);
socket.send(toDatagram("HI " + String.valueOf(i),InetAddress.getByName("127.0.0.1"),3800));
i++;
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
public DatagramPacket toDatagram(
String s, InetAddress destIA, int destPort) {
// Deprecated in Java 1.1, but it works:
byte[] buf = new byte[s.length() + 1];
s.getBytes(0, s.length(), buf, 0);
// The correct Java 1.1 approach, but it's
// Broken (it truncates the String):
// byte[] buf = s.getBytes();
return new DatagramPacket(buf, buf.length,
destIA, destPort);
}
C# code:
string returnData;
byte[] receiveBytes;
//ConsoleKeyInfo cki = new ConsoleKeyInfo();
using (UdpClient udpClient = new UdpClient(new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3800)))
{
IPEndPoint remoteIpEndPoint = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3800);
while (true)
{
receiveBytes = udpClient.Receive(ref remoteIpEndPoint);
returnData = Encoding.ASCII.GetString(receiveBytes);
Console.WriteLine(returnData);
}
}

Try this:
Socket s = ...;
DataOutputStream out = null;
try {
out = new DataOutputStream( s.getOutputStream() );
out.writeInt( 123456 );
} catch ( IOException e ) {
// TODO Handle exception
} finally {
if ( out != null ) {
try {
out.close();
} catch ( IOException e ) {
// TODO Handle exception
}
}
}
It whould help if you could explain a little more what your problem is.