I am trying to send an image from java TCP Client to Node.js TCP Server i have tried to: send binary data, send base64 encoded binary data,send utf-8 encoded strings and at the moment nothing has worked, i always got the data and it creates the image(using fs), but its always corrupted.
Here is my code:
JAVA CLIENT:
import java.util.*;
import java.io.*;
import java.net.Socket;
public class client{
public static void main(String[] args) throws Exception {
Socket client = new Socket("localhost",8080);
Scanner scanner = new Scanner(System.in);
String path= scanner.nextLine();
FileInputStream fis = new FileInputStream(path);
byte[] buffer = new byte[fis.available()];
fis.read(buffer);
buffer= Base64.getEncoder().encode(buffer);
System.out.println(buffer.length);
ObjectOutputStream oos= new ObjectOutputStream(client.getOutputStream());
oos.writeObject(buffer);
client.close();
}
}
Node.js Server:
var fs = require('fs');
var net = require('net');
var sockets = [];
var server=net.createServer().listen(8080,"localhost");
server.on('connection', function(socket){
console.log('Someone connected.');
sockets.push(socket);
var imageData;
socket.on('data',function(data){
imageData+= Buffer.from(data, 'base64');
});
socket.on('end',function(){
console.log(imageData.length)
fs.writeFileSync(new Date()+".png", imageData, function (err) {
if (err) throw err;
else console.log('Saved!');
});
});
socket.on('close',function(){
console.log("Someone disconnected.");
});
});
please i need help, its the fourth time i made this question and nobody answers me, or they say something that doesn't help and give me -1.
Here is the image output
UPDATE:
I tried working with strings and i first detected that the data length when received is not equal of the data length sent, so i removed the data in excess and still didn't work, i am thinking that fs library is bugged, can someone suggest me a better library that works with file in Node.js?
There are a couple of errors that are causing the problem.
First of all, in your sender code, you're using an ObjectOutputStream which, as the name suggests, is made for sending objects. What you need is instead the DataOutputStream, which is made for primitive data types. It's also a lot faster than ObjectOutputStream since it has to handle fewer situations.
So the first change will be something like this:
DataOutputStream oos= new DataOutputStream(client.getOutputStream());
oos.write(buffer);
The next problem is on the receiving side. The socket.on('data') event happens many times during transmission, even if we're receiving the same string! This means that you'll have to wait for the transmission to finish before decoding from base64 to raw data. Another mistake you've made is not initializing the imageData var when creating it.
Let's see the code to make everything more clear. First of all, declaring imageData:
var imageData = "";
Now we know it is for sure an empty string. Next, receiving data:
socket.on('data',function(data){
imageData += data;
});
As you can see I'm not decoding data here anymore, because it must be done after receiving everything. Last thing, decoding:
socket.on('end',function(){
console.log(imageData.length)
var decoded = Buffer.from(imageData, 'base64');
fs.writeFileSync(new Date()+".png", decoded, function (err) {
if (err) throw err;
else console.log('Saved!');
});
});
The decoding now is done during the end event, when we have the entire string.
I've tried this and it's working fine now.
I think you are convert image data to base64 in java client, then convert base64 string to base64 again in server.js .
socket.on('data',function(data){
let imageData=Buffer.from(data, 'base64');
fs.writeFileSync(new Date()+".png", imageData, function (err) {
if (err) throw err;
else console.log('Saved!');
});
});
Related
I was trying to make an app that receives telemetry of F1 2020 via UDP.
For some reason, no matter how I try to turn the received bytes into a string, it just returns random characters.
I'm sure its a noob mistake somewhere but I just can't figure out where
This is the class that receives and logs the packets:
class ClientListen implements Runnable {
private Thread t;
public void run() {
boolean run = true;
try {
DatagramSocket udpSocket = new DatagramSocket(20777);
byte[] buffer = new byte[2048];
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
while (run) {
udpSocket.receive(packet);
String received = new String(packet.getData(), 0, packet.getLength());
packet.setLength(buffer.length);
Log.i("Received data", received);
try{
Thread.sleep(50);
} catch (InterruptedException e) {}
}
udpSocket.close();
}catch (IOException e) {
Log.e("UDP client has IOException", "error: ", e);
run = false;
}
}
public void start() {
if (t == null) {
t = new Thread();
t.start();
}
}
}
The thread beggins when I click a button on the app, and I can see it logs data and stops when the game is paused, as it should
The data output is something like I/Received data: �����R<�<li�)C�������GndBƨS#3Z��N��9��y�}�Ժ�~��g�r��~�ө;��J�qӼ��?�RQ=�k�<h���p�<#84�C�������...
I thought it was beacause I was logging bytes and not a string, but I tried many different ways to turn in into a string and it always has this result.
For some reason, no matter how I try to turn the received bytes into a string, it just returns random characters.
When you do new String(bytes, ...) you are attempting to decode text encoded as bytes into Unicode characters ... using your platform's default character set:
If the data represented by the bytes is text encoded in a different character set, the characters will not be decoded correctly. You need to use the correct character set.
But if the data in the bytes is binary rather than text, it is unlikely that you will be able to turn it into anything other than "random characters" by decoding this way, no matter what character set you are using.
If is common for data sent in UDP packets to be binary. It looks like that is the case here.
I'm sure its a noob mistake somewhere but I just can't figure out where
I think that your mistake is assuming that all data is fundamentally text.
When there is a specification for the format of those UDP packets, you should start by reading it. The format will determine how you should decode the packets.
A Google search found this for me:
https://forums.codemasters.com/topic/50942-f1-2020-udp-specification/
If there was no specification, you would have to reverse engineer the format from example packets. That is tedious and time-consuming, and it isn't something we can teach you to do.
I am writing a program in android studio that communicates with a python server. I tried to send a long message (mp3 file encoded in base64 - about 10k bytes). The problem is that when I check what I received in the server, I get a lot less than 10k bytes.
Anyone knows how to fix it?
Thanks in advance.
Recording message and putting it in base64:
// Record audio from user in mp3 format
MediaRecorder recorder = new MediaRecorder();
recorder.setAudioSource(MediaRecorder.AudioSource.MIC);
recorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
recorder.setOutputFile(Environment.getExternalStorageDirectory()
.getAbsolutePath() + "/messageRecord.mp3");
recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
recorder.prepare();
recorder.start();
TimeUnit.SECONDS.sleep(5);
recorder.stop();
File file = new File(Environment.getExternalStorageDirectory()
.getAbsolutePath() + "/messageRecord.mp3");
int size = (int) file.length();
byte[] bytes = new byte[size];
BufferedInputStream buf = new BufferedInputStream(new FileInputStream(file));
buf.read(bytes, 0, bytes.length);
buf.close();
String content = Base64.encodeToString(bytes, Base64.DEFAULT);
// Prepare audio message request
JSONObject sendRecordReq = new JSONObject();
sendRecordReq.put("code", Codes.SPEECH_TO_TEXT_CODE);
sendRecordReq.put("src_phone", ChatScreen.this.srcPhone);
sendRecordReq.put("dst_phone", ChatScreen.this.dstPhone);
sendRecordReq.put("content", content);
// Send message request
ChatScreen.this.client.send(sendRecordReq);
How I send it:
//In class client
public void send(JSONObject request) {
this.outgoingMessages.addConversationFlow(request); //Send request
}
//In class OutgoingMessages
public void run() {
try {
while (true) {
while(!this.conversationFlow.isEmpty()) {
JSONObject msgToSend = this.conversationFlow.remove();
String strRequest = msgToSend.toString();
this.out.write(Integer.toString(strRequest.length()).getBytes()); //Sends message size
this.out.write(strRequest.getBytes()); //Sends message
}
}
}
catch (Exception e) {
System.out.println(e);
}
}
Server:
while True:
# Receiving data size from client
message_size = int(client_socket.recv(MAX_SIZE_LEN))
# Receiving data from the client
client_message = client_socket.recv(message_size)
print client_message
# Add message to messages queue with client's socket
MESSAGES_QUEUE.append((client_socket, client_message))
EDIT:
the "message_size" value is right (14806 - the size of the message that should b e received in the next line) but it still doesn't receive it all.
EDIT2:
I figured it out, ill post the solution in the answers
Your java code is sending data in a protocol that couldn't possibly work.
If the input JSON object is, say, the string 'a', then you'd send this:
3"a"
as in, 4 bytes: 51, 34, 97, 34.
The python side has no idea when the 'this is how long the data' is part ends.
I assume what you intended to send is something along the lines of:
00, 00, 00, 03, 34, 97, 34.
In other words: 4 bytes containing a network-endian sent integer value with the length, and then that many bytes.
Separately, don't call .getBytes(); when converting strings to bytes, you should always explicitly specify encoding. JSON is more or less by definition UTF-8, so, call .getBytes("UTF-8") instead.
NB: To replace your 'send the length' code, see Convert integer into byte array (Java)
So the problem wasn't with the length of the file or anything else. The problem was that the recv function in the python would get some of the bytes but the code continued for some reason, so it didn't get the whole message.
The solution is to add a function that doesn't continue till the whole length that is specified is received. I found the solution in the next post:
Python Socket Receive Large Amount of Data
I'll add the code as well:
Replace the recv function with recvall(socket, size). This is not my code, it was posted by Adam Rosenfield and edited by Hedde van der Heide.
def recvall(sock, n):
# Helper function to recv n bytes or return None if EOF is hit
data = b''
while len(data) < n:
packet = sock.recv(n - len(data))
if not packet:
return None
data += packet
return data
I'm having an issue with my android client where I click a button to send data to a python server through a socket, which processes it, then sends it back upper cased. My android client's datainputstream always seems to be "one step behind": the first time I click the button the python server clearly receives it(as shown by the print function) but the TextView in my android doesn't show anything. When I click again with different data the python server receives the new data but my android's dis.readUTF() seems to read the old data.
Here is my android client onClick function:
public void onClick(View arg0) {
Thread t = new Thread(){
#Override
public void run() {
try {
Socket s = new Socket("192.168.4.1", 9999);
//OutputStreamWriter osw= new OutputStreamWriter(s.getOutputStream(), "UTF-8");
DataOutputStream dos = new DataOutputStream(s.getOutputStream());
//byte[] bufO=message.getText().toString().getBytes("UTF-8");
dos.writeUTF(message.getText().toString());
DataInputStream dis = new DataInputStream(s.getInputStream());
serverResponse= dis.readUTF();
s.close();
}
catch (IOException e) {
e.printStackTrace();
}
}
};
if(message.getText().toString().equals("")){
Toast.makeText(this, "Empty command", Toast.LENGTH_SHORT).show();
}
else{
message_received.setText("");
t.start();
Toast.makeText(this, "Command sent", Toast.LENGTH_SHORT).show();
message_received.append(serverResponse + "\n");
Log.d(TAG, serverResponse);
}
}
Here is my Python server:
import socketserver
import socket
class MyHandler(socketserver.BaseRequestHandler):
def handle(self):
self.sentence= self.request.recv(1024).strip()
print(self.sentence)
self.request.sendall(self.sentence.upper())
def main():
print ("Opening socket")
host='192.168.4.1'
port=9999
server1=socketserver.TCPServer((host,port),MyHandler)
print ("Running server")
server1.serve_forever()
main()
I've tried different things including adding a thread.sleep between the writeutf() and readutf() but nothing seems to fix this.
This is because readUTF (and therefore writeUTF) may not do what you think it does; from DataInput documentation, referenced by the DataInputStream:
First, two bytes are read and used to construct an unsigned 16-bit integer in exactly the manner of the readUnsignedShort method . This integer value is called the UTF length and specifies the number of additional bytes to be read.
In general it is a good idea to include the length of the string into the string encoding. So maybe you should alter the python server code to receive and send the length encoding as well.
I'm trying to write an upload system for a fairly complex java server. I have reproduced the error in the two small programs listed below. Basically, I am using an ObjectOutputStream/ObjectInputStream to communicate via the client/server. This is a requirement; I have thousands of lines of code working perfectly fine around this ObjectOutputStream/ObjectInputStream setup, so I must be able to still use these streams after an upload is complete.
To access the files(the one being read on the client and the one being written on the server), FileInputStream and FileOutputStream is used. My client appears to be functioning perfectly; it reads in the file and sends a different byte array each iteration(it reads in 1MB at a time, so large files can be handled without overflowing the heap). However, on the server it appears as though the byte array is ALWAYS just the first array sent(the first 1MB of the file). This does not conform to my understanding of ObjectInputStream/ObjectOutputStream. I am seeking either a working solution to this issue or enough education on the matter to form my own solution.
Below is the client code:
import java.net.*;
import java.io.*;
public class stupidClient
{
public static void main(String[] args)
{
new stupidClient();
}
public stupidClient()
{
try
{
Socket s = new Socket("127.0.0.1",2013);//connect
ObjectOutputStream output = new ObjectOutputStream(s.getOutputStream());//init stream
//file to be uploaded
File file = new File("C:\\Work\\radio\\upload\\(Op. 9) Nocturne No. 1 in Bb Minor.mp3");
long fileSize = file.length();
output.writeObject(file.getName() + "|" + fileSize);//send name and size to server
FileInputStream fis = new FileInputStream(file);//open file
byte[] buffer = new byte[1024*1024];//prepare 1MB buffer
int retVal = fis.read(buffer);//grab first MB of file
int counter = 0;//used to track progress through upload
while (retVal!=-1)//until EOF is reached
{
System.out.println(Math.round(100*counter/fileSize)+"%");//show current progress to system.out
counter += retVal;//track progress
output.writeObject("UPACK "+retVal);//alert server upload packet is incoming, with size of packet read
System.out.println(""+buffer[0]+" "+buffer[1]+" "+buffer[2]);//preview first 3 bytes being sent
output.writeObject(buffer);//send bytes
output.flush();//make sure all bytes read are gone
retVal = fis.read(buffer);//get next MB of file
}
System.out.println(Math.round(100*counter/fileSize)+"%");//show progress at end of file
output.writeObject("UPLOAD_COMPLETE");//let server know protocol is finished
output.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
The following is my server code:
import java.net.*;
import java.io.*;
public class stupidServer
{
Socket s;
ServerSocket server;
public static void main(String[] args)
{
new stupidServer();
}
public stupidServer()
{
try
{
//establish connection and stream
server = new ServerSocket(2013);
s = server.accept();
ObjectInputStream input = new ObjectInputStream(s.getInputStream());
String[] args = ((String)input.readObject()).split("\\|");//args[0] will be file name, args[1] will be file size
String fileName = args[0];
long filesize = Long.parseLong(args[1]);
String upack = (String)input.readObject();//get upload packet(string reading UPACK [bytes read])
FileOutputStream outStream = new FileOutputStream("C:\\"+fileName.trim());
while (!upack.equalsIgnoreCase("UPLOAD_COMPLETE"))//until protocol is complete
{
int bytes = Integer.parseInt(upack.split(" ")[1]);//get number of bytes being written
byte[] buffer = new byte[bytes];
buffer = (byte[])input.readObject();//get bytes sent from client
outStream.write(buffer,0,bytes);//go ahead and write them bad boys to file
System.out.println(buffer[0]+" "+buffer[1]+" "+buffer[2]);//peek at first 3 bytes received
upack = (String)input.readObject();//get next 'packet' - either another UPACK or a UPLOAD_COMPLETE
}
outStream.flush();
outStream.close();//make sure all bytes are in file
input.close();//sign off
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
As always, many thanks for your time!
Your immediate problem is that ObjectOutputStream uses an ID mechanism to avoid sending the same object over the stream multiple times. The client will send this ID for the second and subsequent writes of buffer, and the server will use its cached value.
The solution to this immediate problem is to add a call to reset():
output.writeObject(buffer);//send bytes
output.reset(); // force buffer to be fully written on next pass through loop
That aside, you're misusing object streams by layering your own protocol on top of them. For example, writing the filename and filesize as a single string delimited by "|"; just write them as two separate values. Ditto for the number of bytes on each write.
The Bluetooth chat example for Android is very useful to learn how to pass strings between phones - is it possible to use the same code to pass objects between phones? I have the same classes defined in both phones, I just want to pass the instance of one class from one phone to another. Is there any sample code available? I tried using serialization and replacing outputstream and inputstream in the chat example with objectoutputstream and objectinputstream but it didn't seem to work
The best way I found to handle this was the following:
I set up my objects as implementing Serializable that I wanted to send.
I set up the following code to manage the messages:
public byte[] serialize() throws IOException {
ByteArrayOutputStream b = new ByteArrayOutputStream();
ObjectOutputStream o = new ObjectOutputStream(b);
o.writeObject(this);
return b.toByteArray();
}
//AbstractMessage was actually the message type I used, but feel free to choose your own type
public static AbstractMessage deserialize(byte[] bytes) throws IOException, ClassNotFoundException {
ByteArrayInputStream b = new ByteArrayInputStream(bytes);
ObjectInputStream o = new ObjectInputStream(b);
return (AbstractMessage) o.readObject();
I changed the write statements to accept a Serializable, and then make the final write:
/**
* Write to the connected OutStream.
* #param buffer The bytes to write
*/
public void write(AbstractMessage buffer) {
try {
Log.v(TAG,"Writing \""+(buffer.serialize())+"\"");
mmOutStream.write(buffer.serialize());
// Share the sent message back to the UI Activity
mHandler.obtainMessage(AbstractMessageManager.MESSAGE_WRITE, -1, -1, buffer)
.sendToTarget();
} catch (IOException e) {
Log.e(TAG, "Exception during write", e);
}
}
The Bluetooth Chat example is a demonstration of using the Serial Port Profile (SPP) which is based upon RFCOMM. You can serially send across any data you like once the connection is established; you simply need to be able to represent your objects into a serial stream of bytes, i.e. serialize them.
Therefore the use of serialization would certainly be a way of getting your objects sent over the link. The Bluetooth API's send and receive functions deal with arrays of bytes, but you could easily adapt the Bluetooth Chat example to use streams, e.g. the send function would read bytes out of a stream and put them into an array buffer, then you send that buffer, etc. Then the application code would simply talk via input and output stream pipes - that's one way I've done it in the past.
So there's nothing wrong with your actual idea. The bigger problem is that the way you've implemented it is not right, and more problematic still is that the way you've asked your question is quite poor, too. You need to be more descriptive about exactly what didn't work, explain what debugging you've already tried, and post code samples and Logcat outputs so we can help you properly.
Finally, I did find what I think is a bug in the Bluetooth Chat code example: The data receive function passes a reference of the receive byte array to the ArrayList that's used to show each line of text received. This is alright when small amounts of slow text are being transmitted across, but when you try to send large amounts of data, you start to see the data being corrupted, presumably because the ArrayList adapter is still reading bytes out of that same array when the array is being filled with even newer data.
The answer is yes. A String is an Object. Remember? But how exactly to do it, I am still searching for a solution and that's what brought me here...
Trev16v,
First of all, thanks for your initial feedback.
In order to serialise my object, I used the classes serializeObject and deserializeObject from
http://www.jondev.net/articles/Android_Serialization_Example_(Java)
They seem to work well: if I serialise an object (created out of a class that implements Serializable) from a phone/activity and deserialize it from the same phone i manage to get an object out of the generated byte[].
I then tried to use the same code in the class BluetoothChatServices in the bluetooth chat example in oder to send the serialised object to the other phone (in that example there is
public ConnectedThread(BluetoothSocket socket) {
Log.d(TAG, "create ConnectedThread");
mmSocket = socket;
InputStream tmpIn = null;
OutputStream tmpOut = null;
// Get the BluetoothSocket input and output streams
try {
tmpIn = socket.getInputStream();
tmpOut = socket.getOutputStream();
} catch (IOException e) {
Log.e(TAG, "temp sockets not created", e);
}
mmInStream = tmpIn;
mmOutStream = tmpOut;
}
and the bytes are passed using
public void write(byte[] buffer) {
try {
mmOutStream.write(buffer);
// Share the sent message back to the UI Activity
mHandler.obtainMessage(BluetoothChat.MESSAGE_WRITE, -1, -1, buffer)
.sendToTarget();
} catch (IOException e) {
Log.e(TAG, "Exception during write", e);
}
}
and read using
public void run() {
Log.i(TAG, "BEGIN mConnectedThread");
byte[] buffer = new byte[10240];
int bytes;
// Keep listening to the InputStream while connected
while (true) {
try {
// Read from the InputStream
bytes = mmInStream.read(buffer);
// Send the obtained bytes to the UI Activity
mHandler.obtainMessage(BluetoothManageActivity.MESSAGE_READ, bytes, -1, buffer)
.sendToTarget();
} catch (IOException e) {
Log.e(TAG, "disconnected", e);
connectionLost();
break;
}
}
}
The problem with using BluetoothChatServices as it is is that the array of bytes received on the other phone is different from the one sent when it comes to serialised objects. For example, to give an idea element [0] of the seriealized object is =-84 when i send it, the one i receive from the other phone has element [0] to [4] =0, then [5]=4 and all the other elements are also not aligned. I tried in the methods write and run above to change Inputstream and Outputstream with ObjectInputStream and ObjectOutputstream but without success (if this was supposed to be the way to implement it, I can post the code I tried to use)
Again, thanks a lot for your help, I am new to all these concepts so if I am talking nonsense I will be also happy to be addressed to a tutorial
thanks
Facing same problem ... When i am sending a series of objects from one Android device, data sends properly ... But in receiving end all objects does not construct from received byte[].
Error occurs randomly for any received object but the same code works properly in Java ... I think the some bytes misses when transferring data from one device to another ...
Serializable object to byte[] and byte[] to object conversion can be done with the following code
public static byte[] toByteArray(Object obj)
{
byte[] bytes = null;
ObjectOutputStream oos = null;
try
{
oos = new ObjectOutputStream(new ByteArrayOutputStream());
oos.writeObject(obj);
oos.flush();
return bos.toByteArray();
}
catch(Exception e)
{
Log.e("Bluetooth", "Cast exception at sending end ...");
}
return bytes;
}
public static Object toObject(byte[] bytes)
{
Object obj = null;
ObjectInputStream ois = null;
try
{
ois = new ObjectInputStream(new ByteArrayInputStream(bytes));
return ois.readObject();
}
catch(Exception ex)
{
Log.e("Bluetooth", "Cast exception at receiving end ...");
}
return obj;
}
I actually found the problem - when the bytes are loaded using
try {
// Read from the InputStream
bytes = mmInStream.read(buffer);
they are actually loaded in 2 steps.. While debugging and stepping into the code I found that if first loads 990 bytes and then the remaining bytes.. so when i am back to the UI handler i see only the bytes loaded in the second step..
i wonder if there is a way to force to load all bytes at once