We Keep Coding

FakeFtpServer error: Connection refused (Connection refused)

I have a class that downloads files from FTP servers using the Apache FTP Client, and I want to test it. For doing so, I decided to use the FakeFtpServer class from MockFtpServer, but it always fails with the same error: Connection refused (Connection refused).
My test class:
public class FTPConnectionTest {
/** Mock FTP Server */
private static FakeFtpServer FAKE_FTP_SERVER;
/** Mock FTP Server address */
private static String ADDRESS = "localhost";
/** Mock FTP server user name */
private static String USERNAME = "ftpuser";
/** Mock FTP Server password */
private static String PASSWORD = "ftppasswd";
/** Mock FTP Server default root */
private static String ROOT;
/** Mock FTP Server port */
private static int PORT = 0;
/** Test directory in the mock server */
private static String DIRECTORY = "/directory";
/** Test file 1*/
private static String FILE1 = "/Human_satellite.txt";
/** Content of test file 1 */
private static String FILE1_CONTENT = "Ground control to Major Tom...";
/** Test file 1*/
private static String FILE2 = "/wall-e.txt";
/** Content of test file 1 */
private static String FILE2_CONTENT = "If lost in space, use fire extinguisher";
/** Test file 1*/
private static String FILE3 = "/buzz_lightyear.txt";
/** Content of test file 1 */
private static String FILE3_CONTENT = "To infinity, and beyond !";
/**
* Set up a mock FTP server before running the tests
*/
#BeforeClass
public static void setupMock() {
// Create Mock server
FAKE_FTP_SERVER = new FakeFtpServer();
FAKE_FTP_SERVER.setServerControlPort(0); // Automatically finds available port
PORT = FAKE_FTP_SERVER.getServerControlPort();
FAKE_FTP_SERVER.addUserAccount(new UserAccount(USERNAME, PASSWORD, ROOT));
// Get the path to the resources folder where to run the mock FTP
File mockDir = new File("src/test/resources/ftp/mock");
ROOT = mockDir.getAbsolutePath();
// Create mock files
FileSystem fileSystem = new UnixFakeFileSystem();
fileSystem.add(new DirectoryEntry(ROOT));
fileSystem.add(new FileEntry(ROOT + FILE1, FILE1_CONTENT));
fileSystem.add(new DirectoryEntry(ROOT + DIRECTORY));
fileSystem.add(new FileEntry(ROOT + DIRECTORY + FILE2, FILE2_CONTENT));
fileSystem.add(new FileEntry(ROOT + DIRECTORY + FILE3, FILE3_CONTENT));
FAKE_FTP_SERVER.setFileSystem(fileSystem);
FAKE_FTP_SERVER.start();
}
/**
* The the mock FTP Server once the tests are done
*/
#AfterClass
public static void stop() {
FAKE_FTP_SERVER.stop();
}
/**
* Test
*/
#Test
public void testFetchNewFiles () {
// Get output path
String outputPath = "src/test/resources/ftp/result";
File output = new File(outputPath);
outputPath = output.getAbsolutePath();
// Create the connection and get the files
FTPConnection conn = new FTPConnection(ADDRESS, PORT, USERNAME, PASSWORD, outputPath);
conn.fetchNewFiles();
// Check that the files have bin downloaded
File file1 = new File(outputPath + FILE1);
assertTrue(file1.exists());
}
}
And here is the part of the FTP class that fails:
public class FTPConnection {
/** Logger */
private Logger logger;
/** The FTP Client used to connect to the server */
private FTPClient client;
/** The address of the server */
private String server;
/** The port of the server (default 21) */
private int port;
/** The user name to use to connect to the server */
private String user;
/** The password to use to connect to the server */
private String password;
/** The directory where to save the downloaded files */
private String output;
/**
* Constructor
* #param server the address of the FTP server
* #param port the port of the FTP server
* #param user the user name to use to connect to the server
* #param password the password to use to connect to the server
* #param output the output directory where to download the files
*/
public FTPConnection (String server, int port, String user, String password, String output) {
this.logger = LoggerFactory.getLogger(FTPConnection.class);
this.server = server;
this.port = port;
this.user = user;
this.password = password;
this.output = output;
this.client = new FTPClient();
}
/**
* Constructor
* #param server the address of the FTP server
* #param user the user name to use to connect to the server
* #param password the password to use to connect to the server
* #param output the output directory where to download the files
*/
public FTPConnection (String server, String user, String password, String output) {
this(server, 21, user, password, output);
}
public void fetchNewFiles() {
// Connect to the server
try {
this.client.connect(server, port); // That's the line that fails
this.client.login(user, password);
} catch (IOException e) {
logger.error("Error while connecting to FTP server '" + this.server + "': " + e.getMessage());
e.printStackTrace();
return;
}
}
}
And finally an extract of what's going on in the console:
19:24:31.417 [Thread-1] INFO org.mockftpserver.fake.FakeFtpServer - Starting the server on port 0
19:24:31.419 [Thread-1] INFO org.mockftpserver.fake.FakeFtpServer - Actual server port is 34003
19:24:31.444 [main] ERROR com.my.project.importHandler.FTPConnection - Error while connecting to FTP server 'localhost': Connection refused (Connection refused)
*** STACK TRACE ***
19:24:31.457 [Thread-1] DEBUG org.mockftpserver.fake.FakeFtpServer - Cleaning up server...
19:24:31.457 [Thread-1] INFO org.mockftpserver.fake.FakeFtpServer - Server stopped.

Ok, I found it: FAKE_FTP_SERVER.setServerControlPort(0); sets the value of the server port to 0, and the automatic selection of an available port only happens during FAKE_FTP_SERVER.start();. I moved the line PORT = FAKE_FTP_SERVER.getServerControlPort(); after FAKE_FTP_SERVER.start(); and now it works.

OutOfMemoryError: Direct buffer memory when using websockets in WildFly

After a while on our WildFly 18 server, in production, we encountered this error:
[org.xnio.listener] (default I/O-1) XNIO001007: A channel event listener threw an exception:
java.lang.OutOfMemoryError: Direct buffer memory
at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
at org.jboss.xnio#3.7.3.Final//org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:57)
at org.jboss.xnio#3.7.3.Final//org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:55)
at org.jboss.xnio#3.7.3.Final//org.xnio.ByteBufferSlicePool.allocateSlices(ByteBufferSlicePool.java:162)
at org.jboss.xnio#3.7.3.Final//org.xnio.ByteBufferSlicePool.allocate(ByteBufferSlicePool.java:149)
at io.undertow.core#2.0.27.Final//io.undertow.server.XnioByteBufferPool.allocate(XnioByteBufferPool.java:53)
at io.undertow.core#2.0.27.Final//io.undertow.server.protocol.framed.AbstractFramedChannel.allocateReferenceCountedBuffer(AbstractFramedChannel.java:549)
at io.undertow.core#2.0.27.Final//io.undertow.server.protocol.framed.AbstractFramedChannel.receive(AbstractFramedChannel.java:370)
at io.undertow.core#2.0.27.Final//io.undertow.websockets.core.AbstractReceiveListener.handleEvent(AbstractReceiveListener.java:38)
at io.undertow.core#2.0.27.Final//io.undertow.websockets.core.AbstractReceiveListener.handleEvent(AbstractReceiveListener.java:33)
at org.jboss.xnio#3.7.3.Final//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at io.undertow.core#2.0.27.Final//io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener.handleEvent(AbstractFramedChannel.java:950)
at io.undertow.core#2.0.27.Final//io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener.handleEvent(AbstractFramedChannel.java:931)
at org.jboss.xnio#3.7.3.Final//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.jboss.xnio#3.7.3.Final//org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.jboss.xnio.nio#3.7.3.Final//org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.jboss.xnio.nio#3.7.3.Final//org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
We checked a JVM dump through jxray, and it seems websockets are the culprit:
Fact is that our websocket is kind of straightforward:
#ApplicationScoped
#ServerEndpoint(value = "/ws/messenger/{accountId}")
public class MessengerSocket implements Serializable
{
private static final long serialVersionUID = -3173234888004281582L;
#Inject
private Logger log;
#Inject
private MessengerHandler handler;
#OnOpen
public void onOpen(#PathParam("accountId") String accountId, Session session, EndpointConfig config)
{
log.debug("Opening for {}", accountId);
handler.subscribeSocket(session, UUID.fromString(accountId));
}
#OnClose
public void onClose(#PathParam("accountId") String accountId, Session session, CloseReason closeReason)
{
log.debug("Closing {}", accountId);
handler.unsubscribeSocket(session, UUID.fromString(accountId));
}
}
It's coupled with a simple handler, managing a map of users sessions:
#ApplicationScoped
public class MessengerHandler
{
#Inject
private Logger log;
// key: Account id
private Map<UUID, AccountMessengerSessions> sessions;
public void init()
{
sessions = new ConcurrentHashMap<>();
}
public void subscribeSocket(Session session, UUID accountId)
{
// build and store the account messenger session if new
AccountMessengerSessions messenger = sessions.getOrDefault(accountId, new AccountMessengerSessions(accountId));
messenger.getWsSessions().add(session);
sessions.putIfAbsent(accountId, messenger);
log.debug("{} has {} messenger socket session(s) (one added)", messenger.getAccountId(), messenger.getWsSessions().size());
}
/**
* Unsubscribes the provided WebSocket from the Messenger.
*/
public void unsubscribeSocket(Session session, UUID accountId)
{
if (!sessions.containsKey(accountId))
{
log.warn("Ignore unsubscription from {} socket, as {} is unknwon from messenger", session.getId(), accountId);
return;
}
AccountMessengerSessions messenger = sessions.get(accountId);
messenger.getWsSessions().remove(session);
log.debug("{} has {} messenger socket session(s) (one removed)", messenger.getAccountId(), messenger.getWsSessions().size());
if (!messenger.getWsSessions().isEmpty())
{
return;
}
// no more socket sessions, fully remove
sessions.remove(messenger.getAccountId());
}
}
Client side, we have a bit of javascript called when a page is loaded, again, nothing fancy:
var accountId = // some string found in DOM
var websocketUrl = "wss://" + window.location.host + "/ws/messenger/" + accountId;
var websocket = new WebSocket(websocketUrl);
websocket.onmessage = function (event) {
var data = JSON.parse(event.data);
// nothing fancy here...
};
Our users don't use a lot the feature offered by the websocket (an instant messenger), so what is really happening in production is basically websockets opening and closing at each page, with very few messages sent through.
Where could we get it wrong and create this buffer leak? Did we forget something critical?

Looking at this post this may happen if you have a lot of CPU. This was solved by decreasing the number of IO workers. Not sure if this can help in your case.

I also had a similar problem on our wildfly 18 (wildfly 19 suffers from it, too). It is probably triggered by a faulty xnio lib inside wildfly. After updating to wildfly 22 (using the latest xnio lib) the problem was gone.

JMS transaction exception thrown in nontransaction session

Hello I have problem with my jms code when I try to send over 1000 messages to MDB. Following code:
#Stateless(mappedName = "RequestProcessingQueue")
public class RequestProcessingQueue {
private static final Logger logger = Logger.getLogger(RequestProcessingQueue.class);
#Resource(mappedName = "jmsRequestsFactory")
private ConnectionFactory connectionFactory;
#Resource(mappedName = "jmsRequestsDestination")
private Queue queue;
public void add(String participant, String password, List<Long> documents) throws JmsAppException {
try {
logger.debug("requests to process " + documents);
Connection connecton = connectionFactory.createConnection();
connecton.start();
Session session = connecton.createSession(false, Session.AUTO_ACKNOWLEDGE);
QueueSender sender = (QueueSender) session.createProducer(queue);
Message msg = msg = session.createMessage();
msg.setStringProperty("participant", participant);
msg.setStringProperty("password", password);
for (Long id : documents) {
msg.setLongProperty("request", id);
sender.send(msg);
}
sender.close();
session.close();
connecton.close();
} catch (JMSException e) {
throw new JmsAppException(e);
} catch (Throwable e) {
throw new JmsAppException("Fatal error occured while sending request to be processed", e);
}
}
}
throws
MQJMSRA_DS4001: JMSServiceException on send message:sendMessage: Sending message failed. Connection ID: 2979509408914231552 com.sun.messaging.jms.ra.DirectSession._sendMessage(DirectSession.java:1844) / sendMessage: Sending message failed. Connection ID: 2979509408914231552 com.sun.messaging.jmq.jmsserver.service.imq.IMQDirectService.sendMessage(IMQDirectService.java:1955) / transaction failed: [B4303]: The maximum number of messages [1 000] that the producer can process in a single transaction (TID=2979509408914244096) has been exceeded. Please either limit the # of messages per transaction or increase the imq.transaction.producer.maxNumMsgs property. com.sun.messaging.jmq.jmsserver.data.handlers.DataHandler.routeMessage(DataHandler.java:467)'}
at jms.example.RequestProcessingQueue.add(RequestProcessingQueue.java:48)
I do not understand why cus when I create session I pass false as first param indicating that session is non transactional mode.

Your code does not work because the basic JMS API was designed to work in any environment, not just from within an EJB container. Runtime environment programming restrictions and behaviour are described in the EJB specifications and JavaDoc, in particular javax.jms.Connection.createSession(boolean transacted, int acknowledgeMode).
Your code can be simplified (assuming you're using at least Java 7) to:
#TransactionAttribute(TransactionAttributeType.NOTSUPPORTED)
public void add(String participant, String password, List<Long> documents) throws OgnivoException {
try (Connection connection = connectionFactory.createConnection();
Session session = connection.createSession();
// session.start() not required
MessageProducer sender = session.createProducer(queue)) {
logger.debug("requests to process " + documents);
for (Long id : documents) {
Message msg = msg = session.createMessage();
msg.setStringProperty("participant", participant);
msg.setStringProperty("password", password);
msg.setLongProperty("request", id);
sender.send(msg);
}
} catch (JMSException e) {
throw new JmsAppException(e);
}
// Don't catch throwable because it hides bugs
}
Remember that EJB methods are automatically associated with a transaction unless you specify otherwise. Additionally, be sure to check the javadoc for javax.jms.Connection.createSession() and associated methods, particularly the sections describing behaviour in different runtime environments.

How to unit test through the API in this case?

I have a class FTPOperation that I use as a base class to regroup methods that are common to FTP operations. One of these methods is connect().
public abstract class FtpOperation {
protected static final Log log = LogFactory.getLog(FtpOperation.class);
/**
* Hostname or IP address of the FTP server (e.g. localhost, 127.0.0.1).
*/
private String hostName;
private String username;
private String password;
protected FTPClient ftpClient = getFTPClient();
public void setHostName(String hostName) {
this.hostName = hostName;
}
public void setUsername(String username) {
this.username = username;
}
public void setPassword(String password) {
this.password = password;
}
/**
* Connect to the specified FTP server.
*
* #throws Exception
*/
protected void connect() throws Exception {
int reply;
// Connect to the FTP server
ftpClient.connect(hostName);
if (!ftpClient.login(username, password))
throw new Exception("Fail to log in with the given credentials.");
log.info("Connected to " + hostName + ".");
log.info(ftpClient.getReplyString());
// Check if the connection succeeded
reply = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply))
throw new Exception("Connection to FTP server failed with code "
+ reply + ".");
}
/**
* Used for mocking.
*
* #return
*/
protected FTPClient getFTPClient() {
if (this.ftpClient == null)
this.ftpClient = new FTPClient();
return ftpClient;
}
}
I want to write unit test for testing this method but I don't know how to test it. I use Mockito to create a mock object for the FTPClient instance.
First, I thought about testing the different cases where the ftpClient.connect() call returns a certain exception, but I think it's wrong since I'm testing by knowing the implementation of the connect() method and not trough the API.
On example of test I've done:
#Test(expected = SocketException.class)
public void testConnectSocketException() throws Exception {
downloadInitialFileTasklet.setHostName("hostname");
doThrow(new SocketException()).when(mockFtpClient).connect("hostname");
downloadInitialFileTasklet.connect();
}
Could someone explain me the right way to test this method?
Thanks

What is your test meant to be testing? If you're simply seeing that SocketException is not caught it seems a bit of a peculiar test.
If you were to wrap the exception then it makes a little bit more sense.
eg.
protected void connect() throws FTPException {
int reply;
// Connect to the FTP server
try {
ftpClient.connect(hostName);
} catch (IOException e) {
throw new FTPException(e, "unable to connect to: "+hostname);
}
...
}
With the test we're testing that connect correctly terminates early and throws an FTPException if the underlying client cannot connect
#Test(expected = FTPException.class)
public void ConnectFailsIfExceptionOnClientConnect() throws FTPException {
// setup
downloadInitialFileTasklet.setHostName("hostname");
when(mockFtpClient).connect(any(String.class)).doThrow(new SocketException());
// verify -- if something else throws an FTP exception later then the verify
// statements should fail the test because either connect was not called
// because or login was
verify(mockFtpClient).connect(any(String.class));
verify(mockFtpClient, never()).login(any(String.class), any(String.class));
downloadInitialFileTasklet.connect();
}

Create an interface for the FtpClient class, than wrap it into a new class that you will use in the production environment.
For tests instead you can implement a stub (a fake class) or a mock object of the wrapped FtpClient (I prefer the first way).
Pass the IFtpClient interface to the constructor of the FtpOperation class.

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException when using JDBC to access remote MySQL database

/**
*
*/
package ORM;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
/**
* #author Gwilym
* #version 0.0
*/
public class DatabaseConnection {
private String userName="";
private String password="";
private String host="";
Connection conn;
/**
* #param userName
* #param password
* #param host
*/
public DatabaseConnection(String userName, String password, String host) {
this.userName = userName;
this.password = password;
this.host = host;
}
public DatabaseConnection(String userName, String password, String host,boolean autoConnect) {
this.userName = userName;
this.password = password;
this.host = host;
if (autoConnect)
{
try {
Connect();
} catch (DatabaseConnectionException e) {
e.printStackTrace();
}
}
}
/**
* #return the connection
*/
public Connection getConn() {
return conn;
}
/**
* #param userName the userName to set
*/
public void setUserName(String userName) {
this.userName = userName;
}
/**
* #param password the password to set
*/
public void setPassword(String password) {
this.password = password;
}
/**
* #param host the host to set
*/
public void setHost(String host) {
this.host = host;
}
/**
* Connect, attempts to connect to the MySQL database
* with sun JDBC
* & MySQL driver
* #param none
* #return True iff connected;
* #return False for all else;
* #throws DatabaseConnectionException
*/
public boolean Connect() throws DatabaseConnectionException
{
// Attempt to load database driver
try
{
String url = "jdbc:mysql:"+host;
System.out.println(url);
//Load driver
Class.forName ("com.mysql.jdbc.Driver").newInstance ();
conn = DriverManager.getConnection (url, userName, password);
}
catch (ClassNotFoundException cnfe) // driver not found
{
conn=null;
System.err.println ("Unable to load database driver");
throw new DatabaseConnectionException(cnfe);
}
catch(InstantiationException ie)
{
conn=null;
System.err.println ("Unable to Create database driver");
throw new DatabaseConnectionException(ie);
}
catch (IllegalAccessException iae)
{
conn=null;
System.err.println ("Unable to Create database driver");
throw new DatabaseConnectionException(iae);
} catch (SQLException sqle) {
conn=null;
System.err.println ("SQL error");
throw new DatabaseConnectionException(sqle);
}
if (conn!=null)
{
System.out.println ("Database connection established");
return true;
}
else
{
System.out.println ("Database connection Failed");
return false;
}
}
/**
* Disconnects the System from the mySQL database
*
* #param none
* #return true, if successful
* #return false if not connection in existance
*/
public boolean Disconnect()
{
if (conn != null)
{
try
{
conn.close ();
conn=null;
System.out.println ("Database connection terminated normally");
return true;
}
catch (Exception e) {
//Ignore these errors as they all result in conn.close anyway
}
finally
{
conn=null;
System.gc();
// my removing the refrance to conncetion all calling the Garbage collecter we insure it is destoryed.
}
System.out.println ("Database connection terminated with errors");
return true;
}
else
{
System.out.println ("No Database connection present");
return true;
}
}
}
The above code is called by
DatabaseConnection db =new DatabaseConnection("USERNAME","PASSWORD","//tel2.dur.ac.uk:3306/dcs8s07_SEG",true);
for obvious reasons I have removed the user name and password , but they can be aassumed to be correct.
Right down to the problem its self I get a com.mysql.jdbc.exceptions.jdbc4.CommunicationsException when ever this code is run with the details "The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server."
My main problem at the moment is trying to discover what is actually going wrong.
In so far as I can tell the driver is being loaded correctly as my code does not throw a ClassNotFoundException, rather a SQLException of some kind.
So the problem is almost certainly the connection in some way. I can connect and query this database though a phpMyadmin located on the same server so I can assume that
1)The server is online
2)mySQL is working
3)the Username and password are correct
4) the database is present and i have the name correct
From this and "The driver has not received any packets from the server." I am wondering if the URL malformed?
URL= jdbc:mysql://tel2.dur.ac.uk:3306/dcs8s07_SEG
or there a simple setting that is incorrect on the server whihc is not allowing me to connect?
I have pondered on this problem and attempted several googles to no avail, so any idea would be of great help
thanks in advance SO!

This is a wrapped exception. What's the root cause of this exception? Look further in the stacktrace.
A very common root cause is java.net.ConnectException: Connection refused. I've seen this in almost 99% of the cases. If this is true in your case as well, then all the possible causes are:
IP address or hostname in JDBC URL is wrong.
Hostname in JDBC URL is not recognized by local DNS server.
Port number is missing or wrong in JDBC URL.
DB server is down.
DB server doesn't accept TCP/IP connections.
Something in between Java and DB is blocking connections, e.g. a firewall or proxy.
To solve the one or the either, follow the following advices:
Verify and test them with ping.
Refresh DNS or use IP address in JDBC URL instead.
Verify it based on my.cnf of MySQL DB.
Start it.
Verify if mysqld is started without the --skip-networking option.
Disable firewall and/or configure firewall/proxy to allow/forward the port.
The username and password are irrelevant in this problem. At this point the DB can't even be reached. You would have gotten a "Login failed" or "Not authorized" SQLException otherwise.

In addition to the last post you should also do a low-level telnet test, this is the best way to verify connectivity. This test will tell you if there is a firewall or other software blocking access to that port.
telnet tel2.dur.ac.uk 3306