We Keep Coding

Receive DJI Tello's video stream using Java

I'm trying to write a java application for controlling the DJI Tello drone.
I'm already able to send simple commands to the drone by using java.net.DatagramSocket as client.
The Tello SDK says:
It says I have to use java.net.DatagramSocket as the server to receive the video stream.
This is my try to receive the video stream:
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketException;
public class TelloCamera extends Thread {
private boolean isStreamOn;
private DatagramSocket serverSocket;
private byte[] receiveData = new byte[1470];
public TelloCamera() {
isStreamOn = true;
}
public void run() {
try {
serverSocket = new DatagramSocket(11111);
} catch (SocketException e) {
e.printStackTrace();
return;
}
while (isStreamOn) {
receiveData = new byte[1470];
try {
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
serverSocket.receive(receivePacket);
String z = new String(receivePacket.getData());
System.out.println(z);
} catch (IOException e) {
e.printStackTrace();
}
isStreamOn = false;
}
serverSocket.close();
}
public boolean isStreamOn() {
return isStreamOn;
}
public void setStreamOn(boolean streamOn) {
isStreamOn = streamOn;
}
}
Main:
package tellotest;
public class maintellotest {
public static void main(String[] args) {
TelloCommander tello = new TelloCommander();
tello.sendCommand("command");
tello.sendCommand("streamon");
TelloCamera camera = new TelloCamera();
camera.start();
}
}
Does anybody know why I get no string printed to console?

You have to send "streamon" command at 8889 port to Tello first and then listen at 11111 port. But in your code code, there is no code for sending the command. Try adding the code to send command first, send "command" then send "streamon" then listen on port 11111.

I've also searched for a solution to this and came across a well implemented Tello java library, JTello # https://github.com/xrv0/JTello.
In the library the author make use of JCodec (http://jcodec.org/) to decode the H.246 stream:
For example:
// byte[] message refers to the content of the datagram received from the drone over port 1111
// Allocate output frame of max size
Picture out = Picture.create(1920, 1088, ColorSpace.YUV420);
Picture real = decoder.decodeFrame(ByteBuffer.wrap(message), out.getData());

You need to create an InetSocketAddress with IP "0.0.0.0" and port 11111 and then bind the socket to it. And also remove isStreamOn = false from the while loop.

Java TCP client repetitive connections result in EMFILE error

My Java application establishes TCP connection with a server and communicates with it every second by sending and receiving messages. Both server and client are run on the same Mac. In about 15-20 minutes, my server crashes with error "Errno::EMFILE Too many files open". Here is my client code:
package testtcp;
import java.awt.Dimension;
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.EOFException;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.Socket;
import java.net.UnknownHostException;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JOptionPane;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;
public class TestTCP extends JPanel
{
JFrame frame = new JFrame("Button Demo");
ScheduledExecutorService executorService;
private Socket socket = null;
private DataInputStream input = null;
private DataOutputStream output = null;
private BufferedReader br = null;
private boolean isMapUpdating = false;
public TestTCP()
{
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setPreferredSize(new Dimension(300,300));
frame.add(this);
JButton b1 = new JButton("BLACK");
b1.setPreferredSize(new Dimension(150,50));
b1.setFocusPainted(false); // get rid of border around text
add(b1);
b1.addActionListener((java.awt.event.ActionEvent evt) ->
{
startAcarsConnection();
});
frame.pack();
frame.setLocationRelativeTo(null);
frame.setVisible(true);
}
public void startAcarsConnection()
{
start();
}
public void start()
{
System.out.println("THREAD START");
// Default timer rate
int timerRate = 1;
executorService = Executors.newSingleThreadScheduledExecutor();
executorService.scheduleAtFixedRate(new Runnable()
{
#Override
public void run()
{
// Create new TCP connection if the map is not currently updating
if(isMapUpdating == false)
{
isMapUpdating = true;
communicateWithServer();
}
}
}, 0, timerRate, TimeUnit.SECONDS);
}
public void stop()
{
executorService.shutdown();
}
public void communicateWithServer()
{
// Create a message to the server
String messageToServer = makeMessageToServer();
// Connect to the client and receive the response
String messageFromServer = connectToClient(messageToServer);
SwingUtilities.invokeLater(() ->
{
messageReceived(messageFromServer);
});
}
public String connectToClient(String messageToServer)
{
String data = "";
// Message from the server that should terminate TCP connection
String terminator = "END_IGOCONNECT_DATA";
try
{
// Create socket and streams
socket = new Socket("192.168.1.2", 7767);
input = new DataInputStream( socket.getInputStream());
output = new DataOutputStream( socket.getOutputStream());
//Send message to the server
output.writeBytes(messageToServer);
System.out.println("MESSAGE TO SERVER FROM CONNECT TO CLIENT: "+messageToServer);
//Read Response
br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
StringBuilder sb = new StringBuilder();
String s = "";
int value;
// Process the message from the server and add to the StringBuilder
while((value = br.read()) != -1)
{
// converts int to character
char c = (char)value;
sb.append(c);
if(sb.toString().contains(terminator))
{
break;
}
}
// Create the final string
data = sb.toString();
}
catch (UnknownHostException e)
{
System.out.println("Sock:"+e.getMessage());
// Close Connection
cancelConnection();
// Pop-up message that the airport was not found
String message = "Application was not able to establish connection with X-Plane.\n"
+ "Check whether IP Address and Port number were correctly entered in Settings.\n"
+ "Check whether connection is not being blocked by your firewall.";
JOptionPane.showMessageDialog(new JFrame(), message, "TCP Connection Error: UnknownHostException",
JOptionPane.ERROR_MESSAGE);
data = "ERROR";
}
catch (EOFException e)
{
System.out.println("EOF:"+e.getMessage());
// Close Connection
cancelConnection();
// Pop-up message that the airport was not found
String message = "Application was not able to establish connection with X-Plane.\n"
+ "Check whether IP Address and Port number were correctly entered in Settings.\n"
+ "Check whether connection is not being blocked by your firewall.";
JOptionPane.showMessageDialog(new JFrame(), message, "TCP Connection Error: EOFException",
JOptionPane.ERROR_MESSAGE);
data = "ERROR";
}
catch (IOException e)
{
System.out.println("IO:"+e.getMessage());
// Close Connection
cancelConnection();
// Pop-up message that the server was not found
if(!e.getMessage().equals("Socket closed"))
{
String message = "Application was not able to establish connection with X-Plane.\n"
+ "Check whether IP Address and Port number were correctly entered in Settings.\n"
+ "Check whether connection is not being blocked by your firewall.";
JOptionPane.showMessageDialog(new JFrame(), message, "TCP Connection Error: IOException",
JOptionPane.ERROR_MESSAGE);
}
// "Connection reset"
data = "ERROR";
}
finally
{
// TO DO!!! DISABLED FOR NOW!! closeSocketPax();
}
return data;
}
public void cancelConnection()
{
executorService.shutdown();
closeSocketPax();
SwingUtilities.invokeLater(() ->
{
System.out.println("Cancel Connection");
});
}
private void closeSocketPax()
{
try
{
if(socket!=null) { socket.close();}
if(input != null) { input.close();}
if(output != null) { output.close();}
if(br != null) { br.close();}
}
catch (IOException ex)
{
String message = "Error closing socket.";
JOptionPane.showMessageDialog(new JFrame(), message, "TCP Connection Error: IOException",
JOptionPane.ERROR_MESSAGE);
}
socket = null;
input = null;
output = null;
br = null;
}
private String makeMessageToServer()
{
return "MESSAGE TO SERVER";
}
private void messageReceived(String message)
{
System.out.println("MESSAGE RECEIVED: "+message);
isMapUpdating = false;
}
public static void main(String[] args)
{
new TestTCP();
}
}
I have been trying to solve this for almost a month already!
Does anyone see a problem in the code and know how to mitigate the problem? Greatly appreciated!

Each connection you create uses a file descriptor. In any operating system there is a limit to the number of descriptors your process can have. For example, in the Linux environment I'm on the limit is 1024. Different O/S's have different limits but in Unix derived environments like Linux and Mac O/S you can run ulimit -n to see what the limit is.
In your code you do:
socket = new Socket("192.168.1.2", 7767);
in the connectToClient method. Each time you do that and you don't close the socket you use up a file descriptor. Eventually you reach the O/S defined limit and you get, in Mac O/S the Errno::EMFILE error.
You have two choices to fix this. The first is pretty much what you have commented out - close the connection when you're done with it. However, as you indicate in the comments this is occurring very frequently and you don't want to incur the overhead of opening and closing constantly.
That brings us to the second choice - reuse the connection. A socket can send data over and over again if the protocol you're designing handles it. Send the data back and forth over the protocol and reuse the Socket.
A warning though - if your physical connection is somehow severed - for example, you switch from Ethernet to Wi-Fi - your service will still need to deal with possible errors. Your code has most of that but you may want to consider closing and attempting to reconnect when this occurs.

How to create Java Socket object serving on localhost only?

I ask this question again here ( How to create Java socket that is localhost only? ) 'cause all before mentioned methods (simply to say to create one ServerSocket by 3 parameters method) can not solve my problem.
I am working in one big intranet in which, every time when I open one browser, I need enter my proxy account and password to access internet. This is why I hope to test my socket program on localhost.
Occasionally my Client side can connect the Server side, but usually I have to wait for a long time before she coming out. I suppose, it should be related with some of proxy/firewall.
Although I look over all following resources and believe all of them are well worth reading, but I still can not get my issues out.
http://zerioh.tripod.com/ressources/sockets.html
How to determine an incoming connection is from local machine
My Server Side Code
import java.net.Socket;
import java.net.ServerSocket;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.IOException;
import java.net.InetAddress;
public class Myserver {
private int serverPort = 8000;
private ServerSocket serverSock = null;
public Myserver(int serverPort) {
this.serverPort = serverPort;
try {
/*SocketAddress socketAddress = new InetSocketAddress(InetAddress.getByName("localhost"), serverPort);
ServerSocket serverSocket = new ServerSocket();
serverSocket.bind(socketAddress);
serverSocket.accept();*/
serverSock = new ServerSocket(serverPort, 0, InetAddress.getByName(null));
}
catch (IOException e){
e.printStackTrace(System.err);
}
}
public void handleConnection(InputStream sockInput, OutputStream sockOutput) {
while(true) {
byte[] buf=new byte[1024];
int bytes_read = 0;
try {
// This call to read() will wait forever, until the
// program on the other side either sends some data,
// or closes the socket.
bytes_read = sockInput.read(buf, 0, buf.length);
// If the socket is closed, sockInput.read() will return -1.
if(bytes_read < 0) {
System.err.println("Tried to read from socket, read() returned < 0, Closing socket.");
return;
}
System.err.println("Received "+bytes_read
+" bytes, sending them back to client, data="
+(new String(buf, 0, bytes_read)));
sockOutput.write(buf, 0, bytes_read);
// This call to flush() is optional - we're saying go
// ahead and send the data now instead of buffering
// it.
sockOutput.flush();
// sockOutput.close();
}
catch (Exception e){
System.err.println("Exception reading from/writing to socket, e="+e);
e.printStackTrace(System.err);
return;
}
}
}
public void waitForConnections() {
Socket sock = null;
InputStream sockInput = null;
OutputStream sockOutput = null;
while (true) {
try {
// This method call, accept(), blocks and waits
// (forever if necessary) until some other program
// opens a socket connection to our server. When some
// other program opens a connection to our server,
// accept() creates a new socket to represent that
// connection and returns.
sock = serverSock.accept();
System.err.println("Have accepted new socket.");
// From this point on, no new socket connections can
// be made to our server until we call accept() again.
sockInput = sock.getInputStream();
sockOutput = sock.getOutputStream();
}
catch (IOException e){
e.printStackTrace(System.err);
}
// Do something with the socket - read bytes from the
// socket and write them back to the socket until the
// other side closes the connection.
handleConnection(sockInput, sockOutput);
// Now we close the socket.
try {
System.err.println("Closing socket.");
sock.close();
}
catch (Exception e){
System.err.println("Exception while closing socket.");
e.printStackTrace(System.err);
}
System.err.println("Finished with socket, waiting for next connection.");
}
}
}
My Client Side Code
import java.net.Socket;
import java.net.ServerSocket;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.IOException;
public class MyClient {
private String serverHostname = null;
private int serverPort =0;
private byte[] data = null;
private Socket sock = null;
private InputStream sockInput = null;
private OutputStream sockOutput = null;
public MyClient(String serverHostname, int serverPort, byte[] data){
this.serverHostname = serverHostname;
this.serverPort = serverPort;
this.data = data;
}
public void sendSomeMessages(int iterations) {
System.err.println("Opening connection to "+serverHostname+" port "+serverPort);
try {
sock = new Socket(serverHostname, serverPort);
sockInput = sock.getInputStream();
sockOutput = sock.getOutputStream();
}
catch (IOException e){
e.printStackTrace(System.err);
return;
}
System.err.println("About to start reading/writing to/from socket.");
byte[] buf = new byte[data.length];
int bytes_read = 0;
for(int loopi = 1; loopi <= iterations; loopi++) {
try {
sockOutput.write(data, 0, data.length);
bytes_read = sockInput.read(buf, 0, buf.length);
}
catch (IOException e){
e.printStackTrace(System.err);
}
if(bytes_read < data.length) {
System.err.println("run: Sent "+data.length+" bytes, server should have sent them back, read "+bytes_read+" bytes, not the same number of bytes.");
}
else {
System.err.println("Sent "+bytes_read+" bytes to server and received them back again, msg = "+(new String(data)));
}
// Sleep for a bit so the action doesn't happen to fast - this is purely for reasons of demonstration, and not required technically.
try { Thread.sleep(50);} catch (Exception e) {};
}
System.err.println("Done reading/writing to/from socket, closing socket.");
try {
sock.close();
}
catch (IOException e){
System.err.println("Exception closing socket.");
e.printStackTrace(System.err);
}
System.err.println("Exiting.");
}
}
My Test Code
import java.net.*;
public class Mytest {
public static void main(String[] args) {
String hostname = "localhost";
int port = 8000;
try {
InetAddress add = InetAddress.getLocalHost();
System.out.println( add);
}
catch (UnknownHostException e)
{
e.printStackTrace();
}
byte[] data = "my program".getBytes();
MyClient client = new MyClient(hostname, port, data);
client.sendSomeMessages(10);
Myserver server = new Myserver(port);
server.waitForConnections();
}
}
I try telnet, but I can't connect it at all

The first problem is that the test code runs both the client and the server. In Mytest.main(), the main thread does the following:
Creates a client (I would have thought that this step would fail)
Tries to send some messsages (but no ServerSocket has been started)
The server is created
The server waits forever, blocking the main thread on accept()
As a starter to get your code working. Create two test classes TestServer and TestClient, both of these must have main() methods. Launch TestServer first in it's own Java process. Next launch TestClient in separate Java process. This should work!
After you've got everything working, you should introduce some concurrency into your server. The way that it's currently written it can only serve a single client at a time. Create new threads to manage new sockets returned from accept().
Good luck!

How do I run these two loops simultaneously? (Java)

I'm trying to write a simple console program that allows me to send and receive String messages. The problem I am encountering though, is that I don't know how to run the receiving code and the sending code simultaneously.
Individually, the classes are working. I can receive packets and send packets, but making them run at once seems impossible to me.
I've looked into multi-threading but since my knowledge is still very basic, I can't seem to understand how it really works.
This is the code I'm currently using. I wrote the Dialog class myself and found the other two classes on the internet.
Dialog class:
import java.util.Scanner;
public class Dialog {
Scanner scanner = new Scanner(System.in);
User user = new User();
Network net = new Network();
ThreadReceive tr = new ThreadReceive();
ThreadSend ts = new ThreadSend();
public void run() {
System.out.println("WELCOME");
System.out.print("Port: ");
while(!user.setPort(giveInput())) {
System.out.println("Enter a valid port.");
}
System.out.print("IP: ");
user.setIP(giveInput());
System.out.println();
System.out.println("--- CONVERSATION STARTED ---");
tr.receive(user.getIP(), user.getPort()); // Starts receiving loop (within ThreadReceive class).
while (true) { // Starts sending loop.
ts.sendMessage(giveInput(), user.getIP(), user.getPort()); // Sends packet when input is given.
}
}
private String giveInput() {
String input = scanner.nextLine();
return input;
}
}
Receiving class:
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
public class ThreadReceive extends Thread {
public void receive(String ip, int port) {
try {
// Create a socket to listen on the port.
DatagramSocket dsocket = new DatagramSocket(port);
// Create a buffer to read datagrams into. If a
// packet is larger than this buffer, the
// excess will simply be discarded!
byte[] buffer = new byte[2048];
// Create a packet to receive data into the buffer
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
// Now loop forever, waiting to receive packets and printing them.
while (true) {
// Wait to receive a datagram
dsocket.receive(packet);
// Convert the contents to a string, and display them
String msg = new String(buffer, 0, packet.getLength());
System.out.println(packet.getAddress().getHostName() + ": " + msg);
// Reset the length of the packet before reusing it.
packet.setLength(buffer.length);
}
}
catch (Exception e) {
System.err.println(e);
}
}
}
Sending class:
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
public class ThreadSend extends Thread {
public void sendMessage(String message, String ip, int port) {
try {
byte[] data = message.getBytes();
InetAddress address = InetAddress.getByName(ip);
DatagramPacket packet = new DatagramPacket(data, data.length, address, port);
DatagramSocket datagramSocket = new DatagramSocket();
datagramSocket.send(packet);
}
catch (Exception e) {
e.printStackTrace();
}
}
}
Also, is there any way to test if I can receive packets? I've been testing it with a friend but it would be much more convenient to do it myself.
Thanks!

You are not using Threads correctly.
The logic should be in the run method.
I suggest you use a queue such as an ArrayBlockingQueue to pass parameters to your threads. For instance, you could have a method to add elements to this queue
public void addMessage(String message) {
synchronized(inputQueue) {
inputQueue.offer(r);
inputQueue.notify();
}
}
And the run method will use these elements as so :
public void run() {
try {
while(!running)
synchronized (inputQueue) {
inputQueue.wait(); // you can have a timeout also...
String message = this.inputQueue.poll();
// use the message item....
// in your case send it to the other user.
}
}
}
} catch (Exception e) {
/////// your exception handler
}
}
Also Remember to start your threads :
Thread t = new MyThread();
t.start(); /// Start the thread !!!
PS : The messages can be any object here is used strings as i based this on some of my code where i'm using a
Queue<String>

Check out Beej's Guide to Network Programming: http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html -- This will give you some more examples to take a look at. As far as testing goes, you could set up a virtual machine or use another computer you own. Back when I had to learn networking in school we would ssh into two separate Linux boxes to test our code.
EDIT: Also to make sure you are receiving correctly you could make the sender and receiver both print out the packet data they receive. Or you could just have them print a simple string to say the packet was received.
You also might want to check out TCP rather than UDP if you are wanting a continuous stream. UDP just creates a datagram packet and sends it out on the network, whereas TCP creates a persistant connection between two hosts.

proxy server in java

i have the follwoing code of proxy server. IS if the right approach? Will this be able to handel load/trafffic if deployed comerially??
package proxyserver;
import com.sun.corba.se.spi.activation.Server;
import java.net.* ;
import java.io.* ;
import java.lang.* ;
import java.util.* ;
/**
*
* #author user
*/
public class Main {
/**
* #param args the command line arguments
*/
// Variable to track if an error occurred
boolean errorOccurred = false;
//Variables for the host and port parameters
public static void main(String[] args) {
// TODO code application logic here
int localPort = -1;
int remotePort = -1;
String remoteHost = "www.youtube.com";
System.out.print("dwdsw");
Integer parseLocalPort = new Integer(555);
Integer parseRemotePort = new Integer(80);
localPort =80 ;
remotePort = 80;
//Create a listening socket at proxy
ServerSocket server = null;
try
{
server = new ServerSocket(localPort);
}
catch(IOException e)
{
System.err.println("Error: " + e.getMessage());
System.exit(-1);
}
//Loop to listen for incoming connection,
//and accept if there is one
Socket incoming = null;
Socket outgoing = null;
while(true)
{
try
{
// Create the 2 sockets to transmit incoming
// and outgoing traffic of proxy server
incoming = server.accept();
outgoing = new Socket(remoteHost, remotePort);
// Create the 2 threads for the incoming
// and outgoing traffic of proxy server
ProxyThread thread1 = new ProxyThread(incoming, outgoing);
thread1.start();
ProxyThread thread2 = new ProxyThread(outgoing, incoming);
thread2.start();
}
catch (UnknownHostException e)
{
System.err.println("Error: Unknown Host " + remoteHost);
System.exit(-1);
}
catch(IOException e)
{
//continue
System.exit(-2);;
}
}
}
}
now proxy classs
package proxyserver;
/**
*
* #author user
*/
import java.net.* ;
import java.io.* ;
import java.lang.* ;
import java.util.* ;
class ProxyThread extends Thread
{
Socket incoming, outgoing;
ProxyThread(Socket in, Socket out)
{
incoming = in;
outgoing = out;
}
// Overwritten run() method of thread,
// does the data transfers
public void run()
{
byte[] buffer = new byte[5000];
int numberRead = 0;
OutputStream toClient;
InputStream fromClient;
try{
toClient = outgoing.getOutputStream();
fromClient = incoming.getInputStream();
while(true)
{
numberRead = fromClient.read(buffer, 0, 50);
if(numberRead == -1)
{
incoming.close();
outgoing.close();
}
String st = new String(buffer,"US-ASCII");
System.out.println("\n\nXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n\nXXXXXXXXXXXXXXXX\n\n" + st);
toClient.write(buffer, 0, numberRead);
}
}
catch(IOException e)
{
}
catch(ArrayIndexOutOfBoundsException e)
{
}
}
}

[ OK ... enough teasing :-) ]
It looks like it should work, though:
The stuff that you print to System.err in the Proxy class may be mangled. (As I said before, you cannot just assume that every web page is encoded in ASCII!!)
You should probably be reading much more than 50 bytes at a time .... especially if you want high throughput.
Your main class probably should be using a thread pool rather than creating and throwing away threads. And you probably should put an upper bound on the number of threads you want to allow at any given time.
You probably need to do something about servers that take a long time to deliver their responses, etcetera.
Finally, in response to this:
Will this be able to handle the
load/traffic if it is deployed commercially??
It is impossible to say how much load you could pump through this program. For a start, it will depend on your processor and network interface hardware.

It looks about right in principle but you should take a look at an open source version like TCP Proxy for pointers on maximizing throughput, increasing resilience, etc.