We Keep Coding

Uploading a file to testcontainer FTP server fails with Connection refused after being connected

I'm working with FTPClient against an FTP server using Testcontainers.
A reproducible code sample is here:
import org.apache.commons.net.PrintCommandListener;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPReply;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.images.builder.ImageFromDockerfile;
import org.testcontainers.junit.jupiter.Testcontainers;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import static org.assertj.core.api.Assertions.assertThat;
#Testcontainers
class FtpUtilsTest {
private static final int PORT = 21;
private static final String USER = "user";
private static final String PASSWORD = "password";
private static final int FTP_TIMEOUT_IN_MILLISECONDS = 1000 * 60;
private static final GenericContainer ftp = new GenericContainer(
new ImageFromDockerfile()
.withDockerfileFromBuilder(builder ->
builder
.from("delfer/alpine-ftp-server:latest")
.build()
)
)
.withExposedPorts(PORT)
.withEnv("USERS", USER + "|" + PASSWORD);
#BeforeAll
public static void staticSetup() throws IOException {
ftp.start();
}
#AfterAll
static void afterAll() {
ftp.stop();
}
#Test
void test() throws IOException {
FTPClient ftpClient = new FTPClient();
ftpClient.setDataTimeout(FTP_TIMEOUT_IN_MILLISECONDS);
ftpClient.setConnectTimeout(FTP_TIMEOUT_IN_MILLISECONDS);
ftpClient.setDefaultTimeout(FTP_TIMEOUT_IN_MILLISECONDS);
// Log
ftpClient.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
// Connect
try {
ftpClient.connect("localhost", ftp.getMappedPort(PORT));
ftpClient.setSoTimeout(FTP_TIMEOUT_IN_MILLISECONDS);
int reply = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftpClient.disconnect();
throw new AssertionError();
}
// Login
boolean loginSuccess = ftpClient.login(USER, PASSWORD);
if (!loginSuccess) {
throw new AssertionError();
}
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
} catch (IOException e) {
throw new AssertionError(e);
}
String remoteFile = "fileonftp";
try (InputStream targetStream = new ByteArrayInputStream("Hello FTP".getBytes())) {
assertThat(ftpClient.isConnected()).isTrue();
ftpClient.storeFile(remoteFile, targetStream);
}
}
}
This prints:
220 Welcome Alpine ftp server https://hub.docker.com/r/delfer/alpine-ftp-server/
USER *******
331 Please specify the password.
PASS *******
230 Login successful.
TYPE I
200 Switching to Binary mode.
PASV
227 Entering Passive Mode (172,17,0,3,82,15).
[Replacing PASV mode reply address 172.17.0.3 with 127.0.0.1]
then fails with:
Connection refused (Connection refused)
java.net.ConnectException: Connection refused (Connection refused)
...
at java.base/java.net.Socket.connect(Socket.java:609)
at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:866)
at org.apache.commons.net.ftp.FTPClient._storeFile(FTPClient.java:1053)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:3816)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:3846)
What I don't understand is that it fails after successfully connecting and logging in, and returning true for isConnected.
Turns out that when removing the ftpClient.enterLocalPassiveMode(); it works, but I need it to work with the passive mode.
I guess the failure is related when switching to a different port for the passive call.
but when trying to add the ports to the withExposedPorts the container fails to start with:
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [55600, 55601, 55602, 55603, 55604, 55605, 55606, 55607, 55608, 55609, 55598, 55599] should be listening)
Running against a docker (docker run -d -p 21:21 -p 21000-21010:21000-21010 -e USERS="user|password" delfer/alpine-ftp-server) works.
Local docker versions:
Docker version 20.10.11, build dea9396
Docker Desktop 4.3.1
Testcontainers - appears to behave the same both on 1.16.2 and 1.15.3
Link to testcontainers discussion

As you already figured out in the comments, the tricky part about FTP passive mode is that the server uses another port (not 21) for communication.
In the docker image you're using, it's a port from the 21000-21010 range by default. So you need to publish (expose) these additional container ports. In docker run command you used -p 21000-21010:21000-21010 for that.
However, Testcontainers library is designed to publish to random host ports to avoid the problem, when a desired fixed port (or a range of ports) is already occupied on the host side.
In case of FTP passive mode random ports on the host side cause problems, because afaik you can't instruct the ftp client to override the port, which FTP server returned for the passive mode. You'd need something like ftpClient.connect("localhost", ftp.getMappedPort(PORT)); but for passive mode ports as well.
Therefore the only solution I see here is to use a FixedHostPortContainer. Even though it's marked as deprecated and not recommended to use because of the mentioned issues with occupied ports, I think this is a valid use case for it here. FixedHostPortGenericContainer allows to publish fixed ports on the host side. Something like:
private static final int PASSIVE_MODE_PORT = 21000;
...
private static final FixedHostPortGenericContainer ftp = new FixedHostPortGenericContainer<>(
"delfer/alpine-ftp-server:latest")
.withFixedExposedPort(PASSIVE_MODE_PORT, PASSIVE_MODE_PORT)
.withExposedPorts(PORT)
.withEnv("USERS", USER + "|" + PASSWORD)
.withEnv("MIN_PORT", String.valueOf(PASSIVE_MODE_PORT))
.withEnv("MAX_PORT", String.valueOf(PASSIVE_MODE_PORT));
Keep in mind that this solution relies on the assumption that 21000 port is always free. If you're going to run this in the environment where it's not guaranteed, then you need to tweak it to find a free host port first. Like:
private static FixedHostPortGenericContainer ftp = new FixedHostPortGenericContainer<>(
"delfer/alpine-ftp-server:latest")
.withExposedPorts(PORT)
.withEnv("USERS", USER + "|" + PASSWORD);
#BeforeAll
public static void staticSetup() throws Exception {
Integer freePort = 0;
try (ServerSocket socket = new ServerSocket(0)) {
freePort = socket.getLocalPort();
}
ftp = (FixedHostPortGenericContainer)ftp.withFixedExposedPort(freePort, freePort)
.withEnv("MIN_PORT", String.valueOf(freePort))
.withEnv("MAX_PORT", String.valueOf(freePort));
ftp.start();
}

An answer similar to the accepted one but without using deprecated functionalities.
Note that we still have to use the fixed port 21000.
public static class FTPContainer extends GenericContainer<FTPContainer>
{
private static FTPContainer container;
private FTPContainer()
{
super(DockerImageName.parse("delfer/alpine-ftp-server:latest"));
}
#SuppressWarnings("resource")
public static FTPContainer getInstance()
{
if (container == null)
{
container = new FTPContainer().withEnv("USERS", "test|test|/home/").withExposedPorts(21)
.withCreateContainerCmdModifier(e -> e.getHostConfig()
.withPortBindings(new PortBinding(Ports.Binding.bindPort(21000), new ExposedPort(21000))))
.withEnv("MIN_PORT", "21000").withEnv("MAX_PORT", "21000");
}
return container;
}
#Override
public void start()
{
super.start();
}
#Override
public void stop()
{
// Handled when JVM stops
}
}
And then you can use your FTP server instance like
FTPContainer FTP = FTPContainer.getInstance();
FTP.start();

Running into an java.net.BindException: Address already in use (Bind failed) on server- client socket app [duplicate]

I am Running into a java.net.BindException: Address already in use (Bind failed) on a server-client socket app
I am trying to learn about Java sockets using a Youtube tutorial as a reference. My code seems to match everything in the video (except variables names) but, when trying to run the server and then the client sockets, I get:
Exception in thread "main" java.net.BindException: Address already in use (Bind failed)
I have tried even printing out the local port just to make sure I connect to the right available port but, nothing works. Is there any documentation I can look into to solve this problem? or any guidance?
Server.java
public class serverSocket {
public static void main(String args[]) throws IOException{
String message, serverResponse;
ServerSocket serverSocket = new ServerSocket(56789);
System.out.print(serverSocket.getLocalPort());
Socket acceptClientRequestSocket = serverSocket.accept();
Scanner serverScanner = new Scanner(acceptClientRequestSocket.getInputStream());
message = serverScanner.next();
System.out.println(message);
serverResponse = message.toUpperCase();
PrintStream newMessage = new PrintStream(acceptClientRequestSocket.getOutputStream());
newMessage.println(serverResponse);
}
}
Client.java
public class clientSocket {
public static void main(String args[]) throws UnknownHostException, IOException {
String message,outputMessage;
Scanner clientInput = new Scanner(System.in);
Socket clientSocket = new Socket("localhost",56789);
Scanner incomingStream = new Scanner(clientSocket.getInputStream());
System.out.println("Enter a message");
message = clientInput.next();
PrintStream printClientStream= new PrintStream(clientSocket.getOutputStream());
printClientStream.println(message);
outputMessage = incomingStream.next();
System.out.println(outputMessage);
}
}
Is there any documentation I can look into to solve this problem? or any guidance?

You have probably your previously exectued program still running. Check the running java processes. Kill the the previous one and try again.
If this wouldn't help try restarting your machine. If the problem persists after that then some service is already running on this port and is starting with the OS. In that case you can either change the port number in your app or disable that service.

Connection refused error when using sockets between C# (server) and Java (Client) applications

I am trying to create a socket connection between a .Net server application and Java Client Application.
I am getting an error from the java client application:
Connection refused: connect
Notes:
Communicating with a .Net Client Application, works fine.
I have disables the windows firewall
Undoubtedly, I am running the server application in the background and then I am running the client application
Following are my server code (C#):
public class Server
{
public Server()
{
CreateListener();
}
public void CreateListener()
{
// Create an instance of the TcpListener class.
TcpListener tcpListener = null;
IPAddress ipAddress = Dns.GetHostEntry("localhost").AddressList[0];
string output;
try
{
// Set the listener on the local IP address
// and specify the port.
tcpListener = new TcpListener(ipAddress, 13);
tcpListener.Start();
output = "Waiting for a connection...";
}
catch (Exception e)
{
output = "Error: " + e.ToString();
MessageBox.Show(output);
}
}
}
and client application code (Java):
public class smtpClient {
public void Send() {
Socket smtpSocket = null;
DataOutputStream os = null;
DataInputStream is = null;
try {
smtpSocket = new Socket("localhost", 13); // FAILURE
os = new DataOutputStream(smtpSocket.getOutputStream());
is = new DataInputStream(smtpSocket.getInputStream());
} catch (UnknownHostException e) {
System.err.println("Don't know about host: hostname");
} catch (IOException e) {
System.err.println(e.getMessage());
}
}
It fails at the following line in the Java Client Application:
smtpSocket = new Socket("localhost", 13);

I can't tell what is the issue you are facing, but you need to start with a solid foundation to discover these issues.
As a rule of thumb, you should always write one piece (typically the server) first and verify connectivity (say using telnet) and then write the other piece (typically client) and verify its connectivity.
I always keep a Standard Client and Server handy to test whether its my code or its the environment/configuration.
Below is a sample code that works fine to test connectivity.
using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading;
class ClientServer {
static void Main() {
new Thread(() => { StartServer("localhost", 5013); }).Start();
Thread.Sleep(100);
Console.WriteLine("\nPress enter to start the client...");
Console.ReadLine();
StartClient("localhost", 5013);
}
public static void StartServer(string serverInterface, int port) {
try {
IPHostEntry hostInfo = Dns.GetHostEntry(serverInterface);
string hostName = hostInfo.HostName;
IPAddress ipAddress = hostInfo.AddressList[0];
var server = new TcpListener(ipAddress, port);
server.Start();
Console.WriteLine($"Waiting for a connection at {server.LocalEndpoint}");
Console.WriteLine("Press ctrl+c to exit server...");
while (true) {
TcpClient client = server.AcceptTcpClient();
Console.WriteLine($"Server says - Client connected: {client.Client.RemoteEndPoint}");
ThreadPool.QueueUserWorkItem((state) => {
using (var _client = (TcpClient)state)
using (NetworkStream stream = _client.GetStream()) {
string msg = stream.ReadAsciiData();
if (msg == "Hello!") {
stream.WriteAsciiData($"Time:{DateTime.Now: yyyy/MM/dd HH:mm zzz}. Server name is {hostName}");
}
}
}, client);
}
} catch (Exception e) {
Console.WriteLine(e);
}
}
public static void StartClient(string serverInterface, int port) {
Console.WriteLine("Client started...");
try {
using (var client = new TcpClient(serverInterface, port))
using (NetworkStream stream = client.GetStream()) {
Console.WriteLine("Client says - Hello!");
stream.Write(Encoding.ASCII.GetBytes("Hello!"));
string msg = stream.ReadAsciiData();
Console.WriteLine($"Client says - Message from server: Server#{client.Client.RemoteEndPoint}: {msg}");
}
} catch (Exception e) {
Console.WriteLine(e);
}
Console.WriteLine("Client exited");
}
}
static class Utils {
public static void WriteAsciiData(this NetworkStream stream, string data) {
stream.Write(Encoding.ASCII.GetBytes(data));
}
public static string ReadAsciiData(this NetworkStream stream) {
var buffer = new byte[1024];
int read = stream.Read(buffer, 0, buffer.Length);
return Encoding.ASCII.GetString(buffer, 0, read);
}
public static void Write(this NetworkStream stream, byte[] data) {
stream.Write(data, 0, data.Length);
}
}
Now to your specific problem,
The choice of port 13, is not ideal for testing. Usually all ports below 1024 are considered privileged. i.e. a firewall or antivirus might block your attempt to listen on that port
Remember that IPV6 addresses plays a role. Your machine might have that enabled or disabled based on your configuration. You want to make sure that if your server is listening on a IPv6 interface, then your client also connects on the same
Which brings us to another related point: Irrespective of you are using IPv6 interface or not, the client needs to connect to the same interface the server is listening on. This might seem obvious, but is often missed. A typical machine
has at-least 2 interfaces: One for localhost (127...* called loopback interface) and another non local (typically 10...* or 192...*, but not restricted to it). It can so happen (especially when you pick the first available interface to bind your server without knowing which one it is) that server might be listening on non loopback interface like say 192.168.1.10 interface and the client might be connecting to 127.0.0.1, and you can see why the client will get "connection refused" errors
The sample code above works and you can test your code with it. You can us telnet for a client or just my sample code. You can play around changing the serverInterface values to some surprising discoveries which are accentuated by
ipAddress = hostInfo.AddressList[0] line
Hope this helps you with your debugging

Cannot connect through socket

I have a problem that's driving me crazy.
Let's say I have client and server (TCP connection):
public class ServerTCP {
public static void main(String args[]) throws IOException {
ServerSocket srvr = new ServerSocket(4380);
Socket skt = srvr.accept();
System.out.print("Client has connected!\n");
skt.close();
srvr.close();
}
}
public class ClientTCP {
public static void main(String args[]) throws IOException {
Socket skt = new Socket("myIPaddress", 4380);
}
}
If I replace IP address from ClientTCP with "localhost", everything works fine. When I start ServerTCP, go to http://www.yougetsignal.com/tools/open-ports/, enter 4380 and hit Check, I got message from terminal "Client has connected!". So that should mean that port forwarding is right and I can receive connections. But, when I try to connect from ClientTCP I cannot do it. No matter what I do (disabled ufw, tried different ports), it's just stuck and I eventually I get connection timeout.
I also tried same thing with Netty, same problem occurs. I am using XUbuntu
16.04 LTS, if that helps. I have no idea what else should I do.

Socket programming - Client (linux), server (Windows)

I'm trying to create simple socket application using sockets to send stream from linux (64x ArchLinux) to server (Windows XP).
Code I'm using I found on the internet, just to check if it is working. What is interesting the code works perfectly if I'm using Windows XP (server) and Win 8 (client), but when client is on ArchLinux it does not work. Is there some special way to connect Windows-Linux ?
Server.java
import java.lang.*;
import java.io.*;
import java.net.*;
class Server_pzm {
public static void main(String args[]) {
String data = "Toobie ornaught toobie";
try {
ServerSocket srvr = new ServerSocket(1234);
Socket skt = srvr.accept();
System.out.print("Server has connected!\n");
PrintWriter out = new PrintWriter(skt.getOutputStream(), true);
System.out.print("Sending string: '" + data + "'\n");
out.print(data);
out.close();
skt.close();
srvr.close();
}
catch(Exception e) {
System.out.print("Whoops! It didn't work!\n");
}
}
}
Client.java
import java.lang.*;
import java.io.*;
import java.net.*;
class Client {
public static void main(String args[]) {
try {
Socket skt = new Socket("192.168.224.78", 1234);
BufferedReader in = new BufferedReader(new
InputStreamReader(skt.getInputStream()));
System.out.print("Received string: '");
// while (!in.ready()) {} line removed
System.out.println(in.readLine());
System.out.print("'\n");
in.close();
}
/* lines removed catch(Exception e) {
System.out.print("Whoops! It didn't work!\n");
} */
// added exception handling
catch(UnknownHostException e) {
e.printStackTrace();
}
catch (IOException e){
e.printStackTrace();
}
}
}
EDIT
Sorry, I did not specify what I meant by not working. I meant I got an exception which later prints System.out.print("Whoops! It didn't work!\n"); as in the catch blok. Win 8 and Arch Linux are installed on the same laptop, while the code is on my dropbox folder in both systems (so the code is the same) - I will post the actual exception, after I get back to my laptop
EDIT 2:
I updated code and this is exception I got:
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)

java.net.ConnectException: Connection refused
This has two possible meanings.
There is nothing listening at the address:port you tried to connect to.
There is a firewall rule in the way.
More likely 1. Firewalls usually just drop the packets, which causes a connection timeout rather than a refusal.

Are you sure you can establish connection between those systems? I have compiled and run your code on Windows 7 and Linux Mint on Virtualbox and it works correctly.
What do you mean "It doesn't work"? Does it throw any exception? If you just don't have any output, try to run it again and wait about 30 seconds.
For me it's just a network problem. So you should also try to ping your windows machine from linux and then try to telnet to server.
Edit:
So we know it is a network problem. First try to ping ip server from Linux system.
ping 192.168.224.78
If it didn't work, you should check if both machines are in the same subnet 192.168.224.0 assuming the mask is 255.255.255.0. You need just to type ifconfig in console.
In next step you should try to disable windows firewall. Here is an instruction how to do that.