We Keep Coding

File transferred via TFTP has different size than on host

For quite long time now I'm struggling with handling TFTP protocol in my Android app. Its main feature is downloading files from custom designed device which hosts TFTP server.
I was browsing internet hoping to find some good, already written, implementation. First I've tried with TFTP library which is part of Apache Commons. Unfortunately no luck - constant timeouts or even complete freeze. After some further research I found some code on github - please take a look. I've adopted code to Android and after some tweaking I managed to finally receive some files.
Creator of the device stated, that block size should be exactly 1015 bytes. So I increased package size to 1015 and updated creating read request packet method:
DatagramPacket createReadRequestPacket(String strFileName) {
byte[] filename = strFileName.getBytes();
byte[] mode = currentMode.getBytes();
int len = rOpCode.length + filename.length + mode.length + 2;
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(len);
try {
outputStream.write(rOpCode);
outputStream.write(filename);
byte term = 0;
outputStream.write(term);
outputStream.write(mode); // "octet"
outputStream.write(term);
outputStream.write("blksize".getBytes());
outputStream.write(term);
outputStream.write("1015".getBytes());
outputStream.write(term);
} catch (IOException e) {
e.printStackTrace();
}
byte[] readPacketArray = outputStream.toByteArray();
return new DatagramPacket(readPacketArray, readPacketArray.length, serverAddr, port);
}
Chunks are being downloaded, but there is one major issue - files I'm downloading are in parts, 512kB each (except last one), and each part I receive on Android device is around 0,5kB larger. It seems like there is one byte more each time or one whole append more. Apparently I don't understand it completely and I'm missing something.
This is my method for file receiving:
byte previousBlockNumber = (byte) -1;
try {
PktFactory pktFactory;
DatagramSocket clientSocket;
byte[] buf;
DatagramPacket sendingPkt;
DatagramPacket receivedPkt;
System.out.print(ftpHandle);
if (isConnected) {
System.out.println("You're already connected to " + hostname.getCanonicalHostName());
}
try {
hostname = InetAddress.getByName(host);
if (!hostname.isReachable(4000)) {
System.out.println("Hostname you provided is not responding. Try again.");
return false;
}
} catch (UnknownHostException e) {
System.out.println("tftp: nodename nor servname provided, or not known");
return false;
}
clientSocket = new DatagramSocket();
pktFactory = new PktFactory(PKT_LENGTH + 4, hostname, TFTP_PORT);
System.out.println("Connecting " +
hostname.getCanonicalHostName() + " at the port number " + TFTP_PORT);
isConnected = true;
ftpHandle = "tftp#" + hostname.getCanonicalHostName() + "> ";
System.out.println("mode " + PktFactory.currentMode);
if (!isConnected) {
System.out.println("You must be connected first!");
}
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
buf = new byte[PKT_LENGTH + 4];
/* Sending the reading request with the filename to the server. **/
try {
/* Sending a RRQ with the filename. **/
System.out.println("Sending request to server.");
sendingPkt = pktFactory.createReadRequestPacket(filename);
clientSocket.setSoTimeout(4500);
clientSocket.send(sendingPkt);
} catch (Exception e) {
e.printStackTrace();
System.out.println("Connection with server failed");
}
boolean receivingMessage = true;
while (true) {
try {
receivedPkt = new DatagramPacket(buf, buf.length);
clientSocket.setSoTimeout(10000);
clientSocket.receive(receivedPkt);
byte[] dPkt = receivedPkt.getData();
byte[] ropCode = pktFactory.getOpCode(dPkt);
/* rPkt either a DATA or an ERROR pkt. If an error then print the error message and
* terminate the program finish get command. **/
if (ropCode[1] == 5) {
String errorMsg = pktFactory.getErrorMessage(dPkt);
System.out.println(errorMsg);
return false;
}
if (receivedPkt.getLength() < PKT_LENGTH + 4 && ropCode[1] == 3) {
byte[] fileDataBytes = pktFactory.getDataBytes(dPkt);
outputStream.write(fileDataBytes);
if (isListFile) {
listBytes = outputStream.toByteArray();
} else {
FileOutputStream fstream = new FileOutputStream(Constants.EEG_DATA_PATH.concat("file.bin"), true);
// Let's get the last data pkt for the current transfering file.
fstream.write(outputStream.toByteArray());
fstream.close();
}
// It's time to send the last ACK message before Normal termination.
byte[] bNum = pktFactory.getBlockNum(dPkt);
DatagramPacket sPkt = pktFactory.createAckPacket(bNum, receivedPkt.getPort());
clientSocket.send(sPkt);
disconnect();
return true;
}
if (ropCode[1] == 3) {
if (receivingMessage) {
System.out.println("Receiving the file now..");
receivingMessage = false;
}
byte[] bNum = pktFactory.getBlockNum(dPkt);
//I've added this if and it reduces file size a little (it was more than 0,5kB bigger)
if (previousBlockNumber != bNum[1]) {
byte[] fileDataBytes = pktFactory.getDataBytes(dPkt);
previousBlockNumber = bNum[1];
outputStream.write(fileDataBytes);
}
/* For each received DATA pkt we need to send ACK pkt back. **/
DatagramPacket sPkt = pktFactory.createAckPacket(bNum, receivedPkt.getPort());
clientSocket.send(sPkt);
}
} catch (SocketTimeoutException e) {
disconnect();
System.out.println("Server didn't respond and timeout occured.");
return false;
}
}
} catch (Exception e) {
System.out.println(e.getMessage());
return false;
}

I know what was wrong. That strange behavior was result of this line when last packet was received:
byte[] fileDataBytes = pktFactory.getDataBytes(dPkt);
Returned array size was always equal to specified packet length, even if received data was smaller. In my case last packet was 0 bytes (+4 bytes for tftp), but even then extra 512 bytes was added to output stream.
To resolve this I overload mentioned method with extra parameter - actual size of received packet when received data size is higher than 4 bytes and lower than specified packet size (512 bytes). This change resulted with getting correct size of array for last packet, so received file has correct size at the end of the operation.

How can I send a string from C++ winsock to Android via TCP socket?

I want an android app and Windows C++ winsock to communicate using TCP sockets and I successfully sent a string from android to the C++ server but I cannot send string the other way around (from C++ server to Android client).
Here is the important C++ server part:
recvbuf = "Back At u \0";
cout << " \n " << recvbuf << "\n";
int iResult= send(ClientSocket, recvbuf, (int) strlen(recvbuf), 0);
if (iResult == SOCKET_ERROR) {
wprintf(L"send failed with error: %d\n", WSAGetLastError());
closesocket(ClientSocket);
WSACleanup();
return 1;
}
printf("Bytes Sent: %d\n", iResult);
And here is the android client recieving part:
class TextRcv extends AsyncTask<Void, Void, String>
{
#Override
protected String doInBackground(Void... params) {
//TO SEND A STRING
Socket clientSocket = null;
try {
clientSocket= new Socket("192.168.1.5",8889);
DataOutputStream oos= new DataOutputStream(clientSocket.getOutputStream());
oos.writeBytes(String.valueOf(mystr.length()));
oos.flush();
byte[] bufferout=mystr.getBytes();
oos.write(bufferout, 0, bufferout.length);
oos.close();
} catch (IOException e) {
e.printStackTrace();
}
//to recieve a string
String input =null;
char[] buffin=new char[128];
try {
BufferedReader in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
in.read(buffin, 0, 128);
input=String.valueOf(buffin);
clientSocket.close();
} catch (IOException e) {
e.printStackTrace();
}
return input;
}
#Override
protected void onPostExecute(String input) {
super.onPostExecute(input);
Toast toast=Toast.makeText(getApplicationContext(),input,Toast.LENGTH_LONG);
toast.show();
}
}
The C++ output says that there is no error and that 11 bytes (length of the recvbuff string) are sent. But on the android the 'input' string is always null.
Here is the c++ server output:
Start Receving
length of string recieved in bytes =14
AndroidID - Hello World...
Done
Back At u
Bytes Sent: 11
Press any key to continue . . .

String input =null;
At this point input is null.
char[] buffin=new char[128];
try {
BufferedReader in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
in.read(buffin, 0, 128);
input=String.valueOf(buffin);
This code is not correct, but if it executes at all, input cannot possibly be null. The correct code is as follows:
int count = in.read(buffin);
if (count > 0)
{
input = new String(buffin, 0, count);
}
Back to your code:
clientSocket.close();
} catch (IOException e) {
e.printStackTrace();
}
return input;
If input is still null at this point, there must have been an IOException which you haven't disclosed.

Broken pipe error communicating with Java server & C# client in multi thread env

I hope to find any help on my old annoying problem.
I have a TCP sever program with java and client program with c#
packet protocol between those two is simply consist of 4byte length & body ASCII data.
The Problem is that C# client faces FormatException which is from parsing fail on length byte. If I look into an error from client side, then client is trying to parse somewhere in the body which is not length header.
But apparently, Server does not send broken packet.
meanwhile, at the server, I could find an Broken pipe error whenever this kind of problem happens.
Unfortunately this error does not always happen and was not able to recreate the problem situation. it makes me difficult to find exact cause of this problem
Please see below codes for server side
public class SimplifiedServer {
private Map<InetAddress, DataOutputStream> outMap;
private Map<InetAddress,DataInputStream> inMap;
protected void onAcceptNewClient(Socket client) {
DataOutputStream out = null;
DataInputStream in = null;
try {
out = new DataOutputStream(client.getOutputStream());
in = new DataInputStream(client.getInputStream());
} catch (IOException e) {
e.printStackTrace();
}
outMap.put(client.getInetAddress(), out);
inMap.put(client.getInetAddress(), in);
}
public void writeToAll(String packet) {
outMap.forEach((key, out) -> {
try {
byte[] body = packet.getBytes("UTF-8");
int len = body.length;
if (len > 9999) {
throw new IllegalArgumentException("packet length is longer than 10000, this try will be neglected");
}
String lenStr = String.format("%04d%s", len, packet);
byte[] obuf = lenStr.getBytes();
synchronized (out) {
out.write(obuf);
out.flush();
}
} catch (IOException e) {
e.printStackTrace();
}
});
}
public void listenClient(Socket client) {
try {
DataOutputStream out = outMap.get(client.getInetAddress());
DataInputStream in = inMap.get(client.getInetAddress());
while (true) {
byte[] received = SimplePacketHandler.receiveLpControlerData(in);
byte[] lenBytes = new byte[4];
for( int i = 0 ; i < 4 ; i ++){
lenBytes[i] = in.readByte();
}
String lenString = new String(lenBytes);
int length = Integer.parseInt(lenString);
byte[] data = new byte[length];
for ( int i = 0 ; i < length ; i ++){
data[i] = in.readByte();
}
if ( data == null ){
System.out.println("NetWork error, closing socket :" + client.getInetAddress());
in.close();
out.close();
outMap.remove(client.getInetAddress());
inMap.remove(client.getInetAddress());
return;
}
doSomethingWithData(out, data);
}
} catch (NumberFormatException e) {
e.printStackTrace();
} catch ( Exception e ) {
e.printStackTrace();
} finally {
try {
System.out.println(client.getRemoteSocketAddress().toString() + " closing !!! ");
// remove stream handler from map
outMap.remove(client.getInetAddress());
inMap.remove(client.getInetAddress());
//close socket.
client.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
And here is client side code
public class ClientSide
{
public TcpClient client;
public String ip;
public int port;
public NetworkStream ns;
public BinaryWriter writer;
public BinaryReader reader;
public Boolean isConnected = false;
public System.Timers.Timer t;
public String lastPacketSucceeded = String.Empty;
public ClientSide(String ip, int port)
{
this.ip = ip;
this.port = port;
client = new TcpClient();
}
public bool connect()
{
try
{
client.Connect(ip, port);
}
catch (SocketException e)
{
Console.WriteLine(e.ToString());
return false;
}
Console.WriteLine("Connection Established");
reader = new BinaryReader(client.GetStream());
writer = new BinaryWriter(client.GetStream());
isConnected = true;
return true;
}
public void startListen()
{
Thread t = new Thread(new ThreadStart(listen));
t.Start();
}
public void listen()
{
byte[] buffer = new byte[4];
while (true)
{
try
{
reader.Read(buffer, 0, 4);
String len = Encoding.UTF8.GetString(buffer);
int length = Int32.Parse(len);
byte[] bodyBuf = new byte[length];
reader.Read(bodyBuf, 0, length);
String body = Encoding.UTF8.GetString(bodyBuf);
doSomethingWithBody(body);
}
catch (FormatException e)
{
Console.WriteLine(e.Message);
}
}
}
public void writeToServer(String bodyStr)
{
byte[] body = Encoding.UTF8.GetBytes(bodyStr);
int len = body.Length;
if (len > 10000)
{
Console.WriteLine("Send Abort:" + bodyStr);
}
len = len + 10000;
String lenStr = Convert.ToString(len);
lenStr = lenStr.Substring(1);
byte[] lengthHeader = Encoding.UTF8.GetBytes(lenStr);
String fullPacket = lenStr + bodyStr;
byte[] full = Encoding.UTF8.GetBytes(fullPacket);
try
{
writer.Write(full);
}
catch (Exception)
{
reader.Close();
writer.Close();
client.Close();
reader = null;
writer = null;
client = null;
Console.WriteLine("Send Fail" + fullPacket);
}
Console.WriteLine("Send complete " + fullPacket);
}
}
Considering it is impossible to recreate problem, I would guess this problem is from multithread issue. but I could not find any further clue to fix this problem.
Please let me know if you guys need any more information to solve this out.
Any help will be great appreciated, thanks in advance.

A broken pipe exception is caused by closing the connection on the other side. Most likely the C# client has a bug, causing the format exception which causes it to close the connection and therefore the broken pipe on the server side. See what is the meaning of Broken pipe Exception?.
Check the return value of this read:
byte[] bodyBuf = new byte[length];
reader.Read(bodyBuf, 0, length);
According to Microsoft documentation for BinaryReader.Read https://msdn.microsoft.com/en-us/library/ms143295%28v=vs.110%29.aspx
[The return value is ] The number of bytes read into buffer. This might be less than the number of bytes requested if that many bytes are not available, or it might be zero if the end of the stream is reached.
If it reads less than the length bytes then next time it will be parsing the length using data somewhere in the middle of the last message.

These broke pipe exceptions happen when the client (browser) has closed the connection, but the server (your tag) continues to try to write to the stream.
This usually happens when someone clicks Back, Stop, etc. in the browser and it disconnects from the server before the request is finished. Sometimes, it can happen because, for example, the Content-Length header is incorrect (and the browser takes its value as true).
Usually, this is a non-event, and nothing to worry about. But if you are seeing them in your dev environment when you know you have not interrupted your browser, you might dig a bit more to find out why.
WLS server will try to filter these exceptions from the web container out of the log, since it is due to client (browser) action and we can't do anything about it. But the server doesn't catch all of them.
refer from :: https://community.oracle.com/thread/806884

Android UDP needs to recover from back packets

So I have a listener thread for UDP in order for it to bring in packets for h264 and KLV. The problem I have is that it runs great, but if it happens to miss a few packets (I repro by playing a video online), it never recovers and instead is just jumbled frame after frame. Here is the code handling the input data:
private void listenAndWaitAndThrowIntent(InetAddress broadcastIP, Integer port) throws Exception
{
if (m_socket == null || m_socket.isClosed())
{
m_socket = new DatagramSocket(port);
m_socket.setBroadcast(true);
}
m_socket.setSoTimeout(1000);
DatagramPacket packet = new DatagramPacket(m_recvPacket, ARSTools.m_packetSize);
//Log.e("UDP", "Waiting for UDP broadcast");
try
{
m_socket.receive(packet);
}
catch (Exception e)
{
Log.i("UDP", "Socket Read Error: " + e);
return;
}
//Log.e("UDP", "UDP packet obtained: " + packet.getLength());
int count = packet.getLength();
byte [] newPacket = new byte[count];
for(int i = 0; i < count; ++i)
newPacket[i] = packet.getData()[i];
ARSDemuxer.GetInstance().m_inPackets.addLast(newPacket);
}
Obviously my ignorance to this is the cause, but I am not sure what the solution is. Once 3 or 4 packets are lost the count starts being almost every frame and what you see after is just junk.

Sending an int from Java to C using sockets

I was just wondering how to send an int from a Java application to a C application using sockets. I have got different C programs communicating with each other and have got the Java application retrieving data from the C application, but I can't work out sending.
The C application is acting as database, the Java application then sends a user id (a 4 digit number) to the C application, if it exists it returns that record's details.
In Java I have tried using a printWriter and DataOutputStream to send the data, printWriter produces weird symbols and DataOutputStream produces "prof_agent.so".
Any help would be appreciated as I don't have a good grasp of sockets at the moment.

You can use DataOutputStream.writeInt. It writes an int already in network byte order by contract.
On a C side you can call recv, or read to fill in the 4-byte buffer, and then you can use ntohl ( Network-TO-Host-Long ) to convert the value you've just read to your platform int representation.

You can send the textual representation. So the number 123 would be sent as 3 bytes '1' '2' '3'.

It's a bit too late but let this answer be here. Using UDP sockets:
Java code:
public void runJavaSocket() {
System.out.println("Java Sockets Program has started."); int i=0;
try {
DatagramSocket socket = new DatagramSocket();
System.out.println("Sending the udp socket...");
// Send the Message "HI"
socket.send(toDatagram("HI",InetAddress.getByName("127.0.0.1"),3800));
while (true)
{
System.out.println("Sending hi " + i);
Thread.currentThread();
Thread.sleep(1000);
socket.send(toDatagram("HI " + String.valueOf(i),InetAddress.getByName("127.0.0.1"),3800));
i++;
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
public DatagramPacket toDatagram(
String s, InetAddress destIA, int destPort) {
// Deprecated in Java 1.1, but it works:
byte[] buf = new byte[s.length() + 1];
s.getBytes(0, s.length(), buf, 0);
// The correct Java 1.1 approach, but it's
// Broken (it truncates the String):
// byte[] buf = s.getBytes();
return new DatagramPacket(buf, buf.length,
destIA, destPort);
}
C# code:
string returnData;
byte[] receiveBytes;
//ConsoleKeyInfo cki = new ConsoleKeyInfo();
using (UdpClient udpClient = new UdpClient(new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3800)))
{
IPEndPoint remoteIpEndPoint = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 3800);
while (true)
{
receiveBytes = udpClient.Receive(ref remoteIpEndPoint);
returnData = Encoding.ASCII.GetString(receiveBytes);
Console.WriteLine(returnData);
}
}

Try this:
Socket s = ...;
DataOutputStream out = null;
try {
out = new DataOutputStream( s.getOutputStream() );
out.writeInt( 123456 );
} catch ( IOException e ) {
// TODO Handle exception
} finally {
if ( out != null ) {
try {
out.close();
} catch ( IOException e ) {
// TODO Handle exception
}
}
}
It whould help if you could explain a little more what your problem is.