We Keep Coding

Apache FTP Client running in Docker has problem with retrieving files contents from external server

I use Apache FTP Client to retrieve contents of files from FTP server. It runs on passive mode, retrieving with retrieveFileStream or retrieveFile and it works properly when I run it without Docker. There is a problem when I run this application from Docker container. It properly connects, log in, retrieving list of files. It has problem with retrieving content of file - it cannot retrieve full content of file, but only part of it. I thought that might be an buffer issue, I set values of all buffer settings to huge number and also tried to use buffer to read from stream (as was proposed on StackOverFlow when someone has problem with retrieving file), but without success. I also set the MTU settings of Docker to the value that my normal network uses, but also it did not help.
Here is some code:
FTPClient ftpClient = new FTPClient();
ftpClient.setControlKeepAliveTimeout(120);
ftpClient.setDataTimeout(120000);
ftpClient.setBufferSize(1024000);
ftpClient.setReceiveBufferSize(1024000);
ftpClient.setReceieveDataSocketBufferSize(1024000);
LOGGER.trace("Trying to connect to host {}.", connectionSettings.getHost());
ftpClient.connect(connectionSettings.getHost());
int replyCode = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(replyCode)) {
ftpClient.disconnect();
LOGGER.error("Received negative reply code from host {}. Disconnected.", connectionSettings.getHost());
throw new FtpClientNotReadyException();
}
ftpClient.enterLocalPassiveMode();
LOGGER.trace("Received positive reply from host {}. Trying to log in.", connectionSettings.getHost());
try {
if (!ftpClient.login(connectionSettings.getUsername(), connectionSettings.getPassword())) {
LOGGER.warn("FTP Client login failed for user {}", connectionSettings.getUsername());
ftpClient.disconnect();
throw new FtpClientNotReadyException();
}
} catch (IOException e) {
ftpClient.disconnect();
throw e;
}
LOGGER.trace("Successfully logged in to host {} on account {}.", connectionSettings.getHost(),
connectionSettings.getUsername());
try {
if (Strings.isNotBlank(connectionSettings.getWorkingDirectory()) &&
!ftpClient.changeWorkingDirectory(connectionSettings.getWorkingDirectory())) {
LOGGER.warn("FTP Client could not change worked directory to {}.", connectionSettings.getWorkingDirectory());
throw new FtpClientNotReadyException();
}
} catch (IOException e) {
ftpClient.logout();
ftpClient.disconnect();
throw e;
}
InputStream retrievedFileContentInputStream = ftpClient.retrieveFileStream(fileName);
byte[] fileContentByteArray = retrievedFileContentInputStream.readAllBytes();
retrievedFileContentInputStream.close();
try {
checkFileSize(fileName, expectedFileSize, fileContentByteArray.length);
} catch (FileCorruptedException | FileBeingCurrentlyUploadedException e) {
throw e;
} finally {
if (!ftpClient.completePendingCommand()) {
LOGGER.trace("Complete pending command was not successful.");
}
LOGGER.trace("Completed pending command.");
}
These are solutions that I tried:
1) Buffer to read from InputStream:
InputStream ins = ftpClient.retrieveFileStream(fileName);
ByteArrayOutputStream swapStream = new ByteArrayOutputStream();
byte[] buff = new byte[2048];
int rc = 0;
int value =2048;
while ((rc = ins.read(buff, 0, value)) > 0) {
swapStream.write(buff, 0, rc);
}
byte[] fileByte = swapStream.toByteArray();
ftpClient.completePendingCommand();
2) Changing output stream to input stream
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ftpClient.retrieveFile(fileName, outputStream);
outputStream.close();
byte[] fileContentByteArray = outputStream.toByteArray();
My typical log output when problem occurs:
Retrieved file size is lower than expected. Was: 228098. Expected: 236826.
Retrieved file size is lower than expected. Was: 1199. Expected: 1246.

Turns out the client on different platform than Windows received less bytes, because of different line endings (carriage return) that FTP server sent in ASCII MODE.

File not launching in Client Side Java (Server/Client) App no Web

Problem: My File not Launching which is requested from sever.
Okay I have written a server/client application but the problem is when i request for a file from the server it transfer over to the client but what I have notice is that I need to manually refresh the directory to get the file to be in the path or directory. So by saying that, I feel that this is why my code when request the file it doesn't launch.
My approach launching the file from the client after it had just been requested.
Here is my code below:
public static void receiveFile(String fileName) {
try {
int bytesRead;
InputStream in = sock.getInputStream();
DataInputStream clientData = new DataInputStream(in);
fileName = clientData.readUTF();
OutputStream output = new FileOutputStream((fileName));//need to state a repository
long size = clientData.readLong();
byte[] buffer = new byte[1024];
while (size > 0 && (bytesRead = clientData.read(buffer, 0, (int) Math.min(buffer.length, size))) != -1) {
output.write(buffer, 0, bytesRead);
size -= bytesRead;
}
output.close();
in.close();
File file = new File(fileName);
Desktop.getDesktop().open(file);
//System.out.println("File "+fileName+" received from Server.");
} catch (IOException ex) {
//Logger.getLogger(CLIENTConnection.class.getName()).log(Level.SEVERE, null, ex);
ex.printStackTrace();
}
}
Please can you look are tell me what you think I am doing wrong?
Server code:

Try flush the stream before close.
output.flush();
output.close();

I solve it my self the answer is to use the code below:
replace: Desktop.getDesktop().open(file);
with: Runtime.getRuntime().exec("rundll32 url.dll,FileProtocolHandler " + file);

Client Server File transfer Chat Java GUI

i have a client and a server,
1) client should send choose files and send them to server
2) client should send a command message so that the server knows that the incoming is a file not a message (thats why i have "SF" which stands for send file)
3) server receives files and store them somewhere in the system
Also, i dont want to close the socket after i send/receive files (because this is done when the client clicks on disconnect button)
Below is my code but it does not work for some reason, if someone can help me fix it.
Client
public void sendFiles(String file) {
this.out.print("SF");
this.out.flush();
File myfile = new File(file);
// Get the size of the file
long length = myfile.length();
if (length > Integer.MAX_VALUE) {
System.out.println("File is too large.");
}
byte[] bytes = new byte[(int) length];
FileInputStream fis;
try {
fis = new FileInputStream(myfile);
BufferedInputStream bis = new BufferedInputStream(fis);
BufferedOutputStream out = new BufferedOutputStream(sock.getOutputStream());
int count;
while ((count = bis.read(bytes)) > 0) {
out.write(bytes, 0, count);
}
System.out.println("count "+bytes.length);
// this.out.flush();
out.flush();
// out.close();
fis.close();
bis.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
Server
public void recvFile() {
InputStream is = null;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
int bufferSize = 0;
try {
is = sock.getInputStream();
bufferSize = sock.getReceiveBufferSize();
System.out.println("Buffer size: " + bufferSize);
fos = new FileOutputStream("/Users/day/Documents/Parallels/server.txt");
bos = new BufferedOutputStream(fos);
byte[] bytes = new byte[bufferSize];
int count;
while ((count = is.read(bytes)) > 0) {
bos.write(bytes, 0, count);
}
System.out.println("bytes "+bytes.length);
System.out.println("count "+count);
bos.flush();
bos.close();
// is.close();
// sock.close();
} catch (IOException e) {
System.out.println("ERROR" +e);
}
Also in the server side; this is how i jump to the method recvFile();
if (message.contains("SF")) {
recvFile();
}
MORE explanation about my problem:
It does not work in a sense that i dont know if the file is actually sent properly? or the file received properly because i get 0 bytes in the received file. Also, this is the thing i dont want to close the connection because this is a chat so how can i let the server know that this is the end of file?
Can someone help me making the code works because i dont know whats wrong? thanks

Could you kindly elaborate on the error that you are getting. That may help in answering. Because, prima facie the code structure looks fine.
Additionally, I presume that you are using TCP connection for file transfer.
P.S.: I couldn't add a comment to the question, so asking question here.

File transfer in Java is not working correctly

I was trying to write simple "FTP" program, but then suddenly an error occured. So this is a network with client and server and a server storages files uploaded from client, there is also a possibility to download files from server. But when I upload file it is saved in Server directory as an empty file, will someone help me find an error in code?
Here is Client
String nameOfFileToUp = fileFromFileChooser.getName();
System.out.println("fileChooserfile name= " + fileFromFileChooser.getName());
System.out.println("File path= " + fileFromFileChooser.getPath());
pw.println(nameOfFileToUp);
File sendFile = new File(fileFromFileChooser.getPath());
FileInputStream fis = new FileInputStream(sendFile);
int size =(int) fileFromFileChooser.length();
byte[] buffer = new byte[size+1];
int bytes = 0;
while((bytes = fis.read(buffer)) != -1)
{
out.write(buffer,0,bytes);
}
fis.close();
Where pw is PrintWriter,
And Server
FileOutputStream fos = new FileOutputStream(f);
DataOutputStream dops = new DataOutputStream(fos);
while(done)
{
fc = in.readLine();
if(fc == null)
{
done = false;
}
else
{
dops.writeChars(fc);
}
}
fos.close();
Can anyone help? Please

You need to flush/close the output stream.
Also, your server should not be reading by "line", it should be reading bytes (just like your client code).

URLConnection slow to call getOutputStream using an FTP url

I have this little piece of code below which uploads a file in java, the code functions correctly however it hangs for a long time when opening the output stream.
// open file to upload
InputStream filein = new FileInputStream("/path/to/file.txt");
// connect to server
URL url = new URL("ftp://user:pass#host/dir/file.txt");
URLConnection urlConn = url.openConnection();
urlConn.setDoOutput(true);
// write file
// HANGS FROM HERE
OutputStream ftpout = urlConn.getOutputStream();
// TO HERE for about 22 seconds
byte[] data = new byte[1024];
int len = 0;
while((len = filein.read(data)) > 0) {
ftpout.write(data,0, len);
}
// close file
filein .close();
ftpout.close();
In this example the URLConnection.getOutputStream() method hangs for about 22 seconds before continuing as normal, the file is successfully uploaded. The file is only 4 bytes in this case, just a text file with the word 'test' in it and the code hangs before the upload commences so its not because its taking time to upload the file.
This is only happening when connecting to one server, when I try a different server its as fast I could hope for which leads me to think it is a server configuration issue in which case this question may be more suited to server fault, however if I upload from an FTP client (in my case FileZilla) it works fine so it could be there is something I can do with the code to fix this.
Any ideas?

I have solved the problem by switching to use the Commons Net FTPClient which does not apear to have the same problems which changes the code to this below.
InputStream filein = new FileInputStream(new File("/path/to/file.txt"));
// create url
FTPClient ftp = new FTPClient();
ftp.connect(host);
ftp.login(user, pass);
int reply = ftp.getReplyCode();
if(!FTPReply.isPositiveCompletion(reply)) {
ftp.disconnect();
System.err.println("FTP server refused connection.");
return;
}
OutputStream ftpout = ftp.appendFileStream("text.txt");
// write file
byte[] data = new byte[1024];
int len = 0;
while((len = filein.read(data)) > 0) {
ftpout.write(data,0, len);
}
filein.close();
ftpout.close();
ftp.logout();