We Keep Coding

After retrieving multiple files I get an error on the next file '425 can't open data connection for transfer of file'

I'm using apache commons-net FTPClient (ver 3.3) and below error was produced on one of my clients machine (I've tried reproducing the error on my dev machine without luck using testing folder with the same requests on the same server with the same login)
I have a process that check's the remote FTP server for new requests in form of an XML-files. After listing all of those files i proceed in loop to check if they're in XML format. If the file is in this format I do first change their name by changing format from *.xml to *.xmlProcessing, retrive them to a input stream, parse them to my object and create a request in my queue and finally change the name and move them to subfolder working as an archive.
After downloading random amount of files I get stuck while calling retrieveFileStream on the next file, without a timeout or IO Exception.
I've managed to get logs from FTP server and it just says it can't open a data connection
05.06.2019 12:45:45 - > RNFR /folder/file.xml
05.06.2019 12:45:45 - > 350 File exists, ready for destination name.
05.06.2019 12:45:45 - > RNTO /folder/file.xmlProcessing
05.06.2019 12:45:45 - > 250 file renamed successfully
05.06.2019 12:45:45 - > PORT *ports*
05.06.2019 12:45:45 - > 200 Port command successful
05.06.2019 12:45:45 - > RETR /folder/file.xmlProcessing
05.06.2019 12:45:45 - > 150 Opening data channel for file download from server of "/folder/file.xmlProcessing"
05.06.2019 12:45:45 - > 425 Can't open data connection for transfer of "/folder/file.xmlProcessing"
I've already tried diffrent FTP modes. Active local, remote, passive etc. (currently stuck with passive local mode).
I've tried the data timeout but it looks like while i finally got stuck on one of the files the method took more than 1 minute on that file despite me setting the timeout on 30s.
ftp = new FTPClient();
ftp.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
ftp.connect(server, port);
int reply = ftp.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftp.disconnect();
throw new IOException("Exception in connecting to FTP Server");
}
ftp.login(user, password);
ftp.enterLocalPassiveMode();
ftp.setKeepAlive(true);
ftp.setDataTimeout(30000);
Collection<String> listOfFiles = listFiles(FOLDER_PATH);
for(String filePath : listOfFiles){
if (filePath != null && filePath.endsWith(".xml")) {
ftp.rnfr(folder + filePath);
ftp.rnto(folder + filePath + "Processing");
InputStream fileInputStream = ftp.retrieveFileStream(folder + filePath + "Processing");
ftp.completePendingCommand();
//Parsing file to an instance of my object and creating request
ftp.rnfr(folder + filePath + "Processing");
ftp.rnto(archiveFolder + filePath);
if(fileInputStream != null){
fileInputStream.close();
}
}
}
Is there's a bigger likehood that this is fault of the FTP Server, Firewall issues or something else ?
I've runed the same code from my dev machine and it processed all files from test folder (there were around 400 of them) i don't know if im being unlucky for error not occuring on my local dev machine or is it actually something wrong with communication of my contractor with the remote server ?

Everytime you do a file transfer or a directory listing with FTP, the server (or client if using an active mode) assigns a random port number out of a configured range to that transfer. The port number is not released immediately, when the transfer completes. There's some cooldown interval. If you do too many file transfers in a short time interval, it can happen that the server runs out of the available ports – Because all ports end up in the cooldown state.
If you can, check the server configuration and configure a larger range of ports.
Or as a workaround, you can try to slow down the transfer rate.
For some background, see:
How many data channel ports do I need for an FTP?
Why does FTP passive mode require a port range as opposed to only one port?
Though this is just a guess, you should check the server's log, as it can show more details.
Another possibility is, that there's simply a limited number of transfers the server allows for a specific user or source address in some time interval.

Got 200 FTP reply code but file not sent

Ive got a little problem which slowly makes me frustrated. I need to send file to FTP server over TLS, using org.apache.commons.net.ftp.FTPSClient and got a 200 reply code after calling storeFile() method but i don't see any result on a server.
FTP server is not running on my local host. When I do the same thing but using a Filezilla client then it's all done without any problems in both passive and active modes.
FTPSClient ftpsClient = new FTPSClient("TLS", false);
ftpsClient.connect(server, port);
boolean logged = ftpsClient.login(user, pass);
ftpsClient.enterRemotePassiveMode();
System.out.println(ftpsClient.getReplyCode());
ftpsClient.execPBSZ(0);
ftpsClient.execPROT("P");
ftpsClient.sendCommand("TYPE", "A");
System.out.println(ftpsClient.getReplyCode());
File file = new File("config.xml");
InputStream inputStream = new FileInputStream(file);
boolean done = ftpsClient.storeFile("config.xml", inputStream);
System.out.println(ftpsClient.getReplyCode());
inputStream.close();
result of this code execution is:
227
200
200
But maybe this STOR command hasn't overed in fact and this 200 code is related to command before?

If somebody is interested in solution of this problem, it was about resuming ssl session which was required by server and not handled by client. After unselecting this option on filezilla server it has started to work properly.

Android - Prevent chunking data written to OutputStream

Problem
I'm developing a remote app to control software on remote machine through sockets.
Between my app and the softwares there is a sort of router/reverse proxy (software) to pass my commands to the desired software.
Unfortunately, for now, this router doesn't support chunked requests, ie all the command must be received in the same TCP packet else it interprets it as two separates ones.
On some devices all goes well. But some others the command is separated in two separates packets and doesn't be interpreted correctly by the router.
I want to prevent request chunking. Is this possible (Android side)?
Technical details
Java code
In my app I'm using a Socket and a DataOutputStream to send commands, like this:
socket = new Socket(ip, port);
out = new DataOutputStream(socket.getOutputStream());
The protocole to communicate with the router is a little bit weird but I need to send an HTTP header first (for a sort of authentication) waiting for an OK from server and then send my commands (without header this time)
I'm doing this like this:
# Sending header
out.writeBytes("SOURCE content HTTP/1.0\n" +
"Content-type: type\n" +
"Authorization: Basic\n" +
"Nom: rouge\n\n");
# Getting server answer
String answer;
int charsRead;
char[] buffer = new char[256];
charsRead = in.read(buffer);
answer = new String(buffer).substring(0, charsRead);
# ... A little bit of computation of the answer to know if it's OK
# Sending my command to server
out.writeBytes("0 rouge violet getNext 1");
# Same as above to get and compute response
Devices used to test
"Chunking" device : LG Nexus 5X - Android 7.1.2 - API 25
"Non chunking" device : Kesa TM708 ($60 cheap tablet) - Android 5.1.1 - API 22
It's exactly the same code on this two devices
Wireshark captures
Here the two requests sent:
Here the chunked capture file (pcap)
Here the not chunked capture file (pcap)
Thank you :)

Getting a `500 Unknown Command` when invoking `GET` in Java

Alright, so I'm given a Java FtpClient class that I am supposed to finish/modify so that the finished product will serve as a WebServer.
The following is a method that lets me interact with the server through command lines.
/*
* Send ftp command
* #param command: the full command line to send to the ftp server
* #param expected_code: the expected response code from the ftp server
* #return the response line from the ftp server after sending the command
*/
private String sendCommand(String command, int expected_response_code){
String response = "";
try {
// send command to the ftp server
controlWriter.writeBytes(command);
// get response from ftp server
response = controlReader.readLine();
if (DEBUG) {
System.out.println("Current FTP response: " + response);
}
// check validity of response
if (!response.startsWith(String.valueOf(expected_response_code)))
{
throw new IOException(
"Bad response: " + response);
}
} catch (IOException ex) {
System.out.println("IOException: " + ex);
}
return response;
}
However, when I invoke the GET command, i.e.
sendCommand("get " + __file__name__ + "\r\l", 200);),
I get the following response:
500 Unknown command.
I am almost 100% sure this issue has nothing to do with the method I've posted above, but I only posted it so you'll know what I am referring to by the sendCommand method).
Has anyone had a similar issue with this command before? If so, how did you work around it?

I've done a very similar side project to the one you're doing here, and I've encountered the same problem you've discussed here. I still haven't figured out why I wasn't able to simly invoke GET and read off the data stream, but here's my get-around.
First, you'll need to use RETR instead of GET. If you're not familiar with what RETR does, it basically lets you send a file as a packet of bytes through a temporary port you'll generate for data transmission.
To instantiate a temporary port, you will need to be im Passive Mode. So, type in:
quote pasv
Your output would look something like the following:
227 Entering Passive Mode (127,0,0,1,143,155).
A quick glance at the numbers shown between the two parentheses will probably not mean anything to you, however, two details can be derived from there.
The first 3 numbers represent your localhost which is always 127.0.0.1, an the other two are referred to p1 and p2. In this case, we have p1 = 143 and p2 = 155. These two numbers can be used to figure out which port has been assigned to us for data transfer.
Fire up your command line interface
To find out the port number, plug in the numbers in the following formula:
PORT = p1 * 256 + p2
So, our port number in this case is (143 * 256) + 155 ==> (36763).
Now that we have a transfer port open for us and ready for data transfer, you can go ahead and instantiate a new Socket with the port number derived from the formula mentioned above (please note that the numbers will be different every time you run the quote pasv, so don't assume these are constants).
The next step here is to send the file from yourself to the client. Note that you're not directing the data packet to a specific client, rather, any client that's currently connected will be receiving the packets.
To send the file, type in the following command:
quote retr
now you can use the DataInputStream class from your Socket to read all the bytes, display them, then them into an identical copy of the original file, or do whatever you're planning to do with them.
Note.. Note... Note... : the commands listed above were meant to be entered from the command line But since you want your application to handle all the job (I assume), the same commands can be passed from your Java application with a little bit of tweaking around. You will basically only need to take the word quote out of all the commands we've used them in.
I think I've typed enough tonight. I am headed to bed now. Let me know if you need further help in a comment below and I will try to respond as soon as possible. Also, let me know if anything I have said is not making sense to you.

connect to a lacewing server chat

I'm trying to make a port of a chat program a friend of mine made with lacewing and multimedia fusion 2 for android device.
I've managed to create a socket connecting to the listening socket of the server successfully, but I cannot seem to be able to send data to login and enter the chat. The login for now just requires a name, but even if I send a String of data, the server doesn't seem to reply or accept that data to get me over the channel.
I know I could easily port this with other way like using the NDK of the multimedia fusion 2 exporter, but I just want to figure out how this works
PS: I'm using Java and libgdx for the development

You need to read the liblacewing relay protocol:
https://github.com/udp/lacewing/blob/0.2.x/relay/current_spec.txt
On initial connection, you have to send byte 0 to identify that you are not an HTTP client. After this, you can exchange normal protocol messages.
The first message you need to send is the connection request (which may be denied by the server with a deny message). This would be:
byte 0 (2.1.0 request)
(1.2 size)
byte 0 (2.1.0.0 connection request)
string "revision 3" (2.1.0.0 connection request -> version)
When the server responds with response 0 (2.2.0.0 Connect), you then have to set a name before you may join any channels. This is done with message 2.1.0.1 SetName, which is the same structure as above but instead of 2.1.0.0's byte 0, it is 2.1.0.1's byte 1, followed by the name as a string instead of the protocol version.
The server should then respond with 2.2.0.1 SetName, assuming it accepted your name change request. You should process this message in case the server gave you a different name than you requested. Finally, once you have a name, you can join a channel with 2.1.0.2 JoinChannel. The flags you specify here will be used if the channel doesn't exist yet (e.g. nobody is in the chat yet) - these should match the ones in the MMF2 project file. The name should also match.
After all that, you're still not done! You have to process more messages, etc. it's almost like writing the RelayClient class yourself. It's a tough task, but with the protocol specification in hand you should be able to work it all out.