We Keep Coding

How to convert FTP from Spring MVC to Spring Boot/Thymeleaf?

So I've been working on taking the logic from an older webapp and making a new Spring Boot application out of it. I've come to a stuck spot regarding an ftp connection and call. Since I don't have a bunch of experience with this, I'm curious if there is a better/more modern way to handle most of this ftp stuff using Spring Boot/Thymeleaf and ways to go ahead and set that up. Any advice/guidance would be fantastic.
This is the older code that I'd like to modernize a bit.
String serverName = getFtpServer();
// Connect to the server
try {
ftp.connect(serverName);
ftp.enterLocalPassiveMode();
String replyText = ftp.getReplyString();
System.out.println(replyText);
} catch (Exception e) {
e.printStackTrace();
return false;
}
// Login to the server
try {
ftp.login(userName, password);
String replyText = ftp.getReplyString();
System.out.println(replyText);
} catch (Exception e) {
e.printStackTrace();
return false;
}
// Tell server that the file will have JCL records
try {
ftp.site("filetype=jes");
String replyText = ftp.getReplyString();
System.out.println(replyText);
} catch (Exception e) {
e.printStackTrace();
return false;
}
// Submit and run the JCL
try {
System.out.println("TRYING TO START MAINFRAME JCL");
submitJcl(filename, serverName);
String replyText = ftp.getReplyString();
System.out.println(replyText);
} catch (Exception e) {
String replyText = ftp.getReplyString();
System.out.println(replyText);
e.printStackTrace();
return false;
}
// Quit the server
try {
ftp.quit();
} catch (Exception e) {
e.printStackTrace();
}
Storing the file
private String submitJcl(String remoteFile, String serverName) throws IOException {
String filePath = getFilePath();
String result = "";
String fileName = filePath + remoteFile;
System.out.println("filePath = " + fileName);
FileInputStream inputStream = new FileInputStream(fileName);
ftp.storeFile(serverName, inputStream);
return result;
}

For this I figured out that there may be better ways to change it into the newer ftp format for Spring Boot, but this still completely works.
Changes I made to it regardless:
Consolidated the try/catch blocks into one.
Pushed the ftp stuff into its own function and then just called it inside the try/catch block
Changed all of the sys.out's to info.debugs.
Changed the way it gets the filePath to more of a relative path with the file stored within the system instead of user files.

Database File 'is being used by another process' when all connections are closed

I am developing a system which uses a database access object (DAO) to link between the program and database. All connections must be made through this class, using getConnection(). However, I have a reset option which overrides the database file in the user's file system with one inside the application itself (i.e. in its jar when deployed). This reset() feature causes an exception saying that the database is already in use.
java.nio.file.FileSystemException: C:\iMProve\improveDB.accdb: The process cannot access the file because it is being used by another process.
With research, this means that the connections have not been closed properly. However, I will show you the whole code for this DAO class if you can spot possibly where lies the problem.
public class DAO {
private static final String dbDir = "C://iMProve";
private static final String dbName = "improveDB.accdb";
private static final String dbUrl = "jdbc:ucanaccess://" + dbDir + "//"+ dbName;
private ObservableList<Connection> allConnections = FXCollections.observableArrayList();
public DAO() { //constructor - called when object is made
try {
Class.forName("net.ucanaccess.jdbc.UcanaccessDriver");
} catch(ClassNotFoundException e) {
System.out.println("Cannot load ucanaccess driver");
e.printStackTrace();
}
File directory = new File(dbDir);
if(!directory.exists()) //create directory if not already
directory.mkdir();
File database = new File(dbDir + "//" + dbName);
if(!database.exists()) { //copy the database file into user's file system - if not already
try {
Files.copy(DAO.class.getResourceAsStream(dbName), database.toPath(), StandardCopyOption.REPLACE_EXISTING);
} catch(IOException ex) {ex.printStackTrace();}
}
}
public void reset() {
File database = new File(dbDir + "//" + dbName);
try {
Files.copy(DAO.class.getResourceAsStream(dbName), database.toPath(), StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
e.printStackTrace();
}
}
public Connection getConnection() { //create a connection to the database
Connection conn = null;
try {
conn = DriverManager.getConnection(dbUrl);
} catch (SQLException e) {
e.printStackTrace();
}
allConnections.add(conn);
return conn;
}
public void closeConnections() {
for(Connection conn: allConnections) {
if(conn!=null) {
try {
conn.close();
System.out.println("closed");
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
}
The constructor simply copies the database from the application into the user's file system if not there already.
The getConnection() method connects to the database.
Calling reset() on its own produces that error, so by adding the connections to a static collection we can closing each and every connection, using closeConnections().
Note: These connections should already be closed anyways because we always a try-with-resources.
For example:
try (
Connection conn = dao.getConnection();
PreparedStatement stmt = conn.prepareStatement("SELECT Title FROM Theory WHERE Grade <= ?");
) {
And although we have two levels for closing connections
try-with-resources
closeConnections() method
They still are ineffective. When calling closeConnections(), we still get outputs of
closed
closed
even when connections have been used inside try-with-resources blocks.
And even with closeConnections(), we still get the same error:
java.nio.file.FileSystemException: C:\iMProve\improveDB.accdb: The process cannot access the file because it is being used by another process.

SSL DB2 connection failed

I am trying to connect to a DB2 database using SSL on IBM Bluemix.
When I first tried to connect without SSL, it doesn't work. After reading the documentation, I have realized that it connects to the database with SSL enabled.
I tried using the following code to get it connect to the database:
public boolean connect() {
try {
String url = "jdbc:db2://" + serverName + ":" + port + "/" + dbName+
":securityMechanism=9";
connection = DriverManager.getConnection(url, userName, passWord);
st = connection.createStatement();
return true;
} catch (Exception e) {
System.err.println(e.getMessage());
}
return false;
}
Still I am not too sure on how to use the SSL certificate provided with the code above.
I tried searching for examples but most of the explanations are either unclear or used for another database system.

If you are using Liberty, a datasource is generated for you, and you can look it up using jndi.
#Resource(lookup = "jdbc/mydb")
private DataSource myDataSource;
Connection c = myDataSource.getConnection();
"mydb" is the name of the SQLDB service
https://developer.ibm.com/bluemix/2014/02/07/java-db2-10-minutes/

According to the SQLDB documentation, If you use the latest com.ibm.db2.jcc.DB2Driver with the JDBC connection, the current SSL certificate is bundled with the driver and does not need manually installing.
The following snippet shows you how to use the connection details available from VCAP_SERVICES to connect to SQLDB over SSL.
public class SSLTEST {
/**
* #param args
*/
public static void main(String[] args) {
String ServerName = "hostname or IP address";
int PortNumber = 50001;
String DatabaseName = "SQLDB";
String user = "your_user_id_from_VCAP_SERVICES";
String userPassword = "your_password_from_VCAP_SERVICES";
java.util.Properties properties = new java.util.Properties();
properties.put("user", "user ID that has access to SQLDB");
properties.put("password", "password for the user ID that has access to SQLDB");
properties.put("sslConnection", "true");
String url = "jdbc:db2://" + ServerName + ":"+ PortNumber + "/" +
DatabaseName + ":" + traceFileLocation + ";";
java.sql.Connection con = null;
try
{
Class.forName("com.ibm.db2.jcc.DB2Driver").newInstance();
}
catch ( Exception e )
{
System.out.println("Error: failed to load Db2 jcc driver.");
}
try
{
System.out.println("url: " + url);
con = java.sql.DriverManager.getConnection(url, properties);
if (con != null) {
System.out.println("Success");
} else {
System.out.println("Failed to make the connection");
}
con.close();
}
catch (Exception e)
{
if (con != null) {
try {
con.close();
} catch (Exception e2) {
e2.printStackTrace();
}
}
e.printStackTrace();
}
}

finally i use datasource to connect to the database.
Context ic = new InitialContext();
DataSource db = (DataSource)context.lookup("jdbc/MyDatabase");

connect to database use properties file in java web application

I write a code to connect to database from servlet.I want to use properties.but it does not work.i think i my code or properties file has problem.please help me to correct it.
try
{
Properties prop=new Properties();
FileInputStream in = new FileInputStream(System.getProperty("WEB-INF/dbConnection.properties"));
prop.load(in);
in.close();
String drivers = prop.getProperty("jdbc.drivers");
String connectionURL = prop.getProperty("jdbc.url");
String username = prop.getProperty("jdbc.username");
String password = prop.getProperty("jdbc.password");
Class.forName(drivers);
con=DriverManager.getConnection(connectionURL,username,password);
System.out.println("Connection Successful");
..
and this is my properties file
jdbc.driverClassName=oracle.jdbc.driver.OracleDriver
jdbc.url=jdbc:oracle:thin:#192.168.101.84:1521:orcl
jdbc.username=user1
jdbc.password=123

Instead of
FileInputStream in = new FileInputStream(System.getProperty("WEB-INF/dbConnection.properties"));
use
InputStream in = getClass().getResourceAsStream("dbConnection.properties");

try {
FileReader reader = new FileReader("Full Path");
Properties prop = new Properties();
prop.load(reader);
String drivers = prop.getProperty("jdbc.driverClassName");
String connectionURL = prop.getProperty("jdbc.url");
String username = prop.getProperty("jdbc.username");
String password = prop.getProperty("jdbc.password");
Class.forName(drivers);
Connection con = DriverManager.getConnection(connectionURL, username, password);
System.out.println("Connection Successful");
} catch (SQLException sqle) {
// TODO: Add catch code
sqle.printStackTrace();
} catch (FileNotFoundException fnfe) {
// TODO: Add catch code
fnfe.printStackTrace();
} catch (IOException ioe) {
// TODO: Add catch code
ioe.printStackTrace();
} catch (ClassNotFoundException cnfe) {
// TODO: Add catch code
cnfe.printStackTrace();
}catch (Exception e) {
// TODO: Add catch code
e.printStackTrace();
}

This part:
(System.getProperty("WEB-INF/dbConnection.properties"));
is actually referring to your classpath, try removing WEB-INF/ or just leave /, for checking root folder.
Or you can specify the fully qualified name, i.e : "C:\\foo\\test\\...\\dbConnection.properties" or "/app/blah/.../dbConnectionProperties"

For Class name, in the properties file you have defined the driver as 'jdbc.driverClassName'
but when passing variable you just pass as 'jdbc.drivers'

Reliable example of how to use SFTP using public private key authentication with Java

Recently a client of our unexpectedly shifted some important files we collect from an ftp to sftp server. Initially I was under the impression that it would be simple to write or find a java utility that can handle sftp, this has definitely not proven to be the case. What has also compounded this problem is that we are trying to connect to the sftp server from a windows platform (so the definition of where SSH_HOME is on the client gets very confused).
I have been using the apache-commons-vfs library and have managed to get a solution that reliably works for username/password authentication, but as of yet nothing that can reliably handle private/public key authentication.
The following example works for username/password authentication, but I want to adjust it for private/public key authentication.
public static void sftpGetFile(String server, String userName,String password,
String remoteDir, String localDir, String fileNameRegex)
{
File localDirFile = new File(localDir);
FileSystemManager fsManager = null;
if (!localDirFile.exists()) {
localDirFile.mkdirs();
}
try {
fsManager = VFS.getManager();
} catch (FileSystemException ex) {
LOGGER.error("Failed to get fsManager from VFS",ex);
throw new RuntimeException("Failed to get fsManager from VFS", ex);
}
UserAuthenticator auth = new StaticUserAuthenticator(null, userName,password);
FileSystemOptions opts = new FileSystemOptions();
try {
DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(opts,
auth);
} catch (FileSystemException ex) {
LOGGER.error("setUserAuthenticator failed", ex);
throw new RuntimeException("setUserAuthenticator failed", ex);
}
Pattern filePattern = Pattern.compile(fileNameRegex);
String startPath = "sftp://" + server + remoteDir;
FileObject[] children;
// Set starting path on remote SFTP server.
FileObject sftpFile;
try {
sftpFile = fsManager.resolveFile(startPath, opts);
LOGGER.info("SFTP connection successfully established to " +
startPath);
} catch (FileSystemException ex) {
LOGGER.error("SFTP error parsing path " +
remoteDir,
ex);
throw new RuntimeException("SFTP error parsing path " +
remoteDir,
ex);
}
// Get a directory listing
try {
children = sftpFile.getChildren();
} catch (FileSystemException ex) {
throw new RuntimeException("Error collecting directory listing of " +
startPath, ex);
}
search:
for (FileObject f : children) {
try {
String relativePath =
File.separatorChar + f.getName().getBaseName();
if (f.getType() == FileType.FILE) {
System.out.println("Examining remote file " + f.getName());
if (!filePattern.matcher(f.getName().getPath()).matches()) {
LOGGER.info(" Filename does not match, skipping file ." +
relativePath);
continue search;
}
String localUrl = "file://" + localDir + relativePath;
String standardPath = localDir + relativePath;
System.out.println(" Standard local path is " + standardPath);
LocalFile localFile =
(LocalFile) fsManager.resolveFile(localUrl);
System.out.println(" Resolved local file name: " +
localFile.getName());
if (!localFile.getParent().exists()) {
localFile.getParent().createFolder();
}
System.out.println(" ### Retrieving file ###");
localFile.copyFrom(f,
new AllFileSelector());
} else {
System.out.println("Ignoring non-file " + f.getName());
}
} catch (FileSystemException ex) {
throw new RuntimeException("Error getting file type for " +
f.getName(), ex);
}
}
FileSystem fs = null;
if (children.length > 0) {
fs = children[0].getFileSystem(); // This works even if the src is closed.
fsManager.closeFileSystem(fs);
}
}
I've got my private key stored in a known location and my public key has been distrubuted to the server (we have tested that these keys work succesfully when connecting using other tools)
I have played around with adding the following line
SftpFileSystemConfigBuilder.getInstance().setIdentities(this.opts, new File[]{new File("c:/Users/bobtbuilder/.ssh/id_dsa.ppk")});
This succesfully loads the private key into the whole framework but it never then uses that key to authenticate further.
Any help or direction most warmly received

After much digging around I finally got to the answer myself. Seems that much of my bother was to do with the format of the private and public key
privateKey must be in openSSH format
publicKey for whatever reason can only be pasted from the puttyGen window (exporting the public key always seemed to give it with missing headers meaning the freeSSHD windows server could not use it)
Anyhow below is my code I finally came up with including the javadoc, so hopefully should save some others the pain I went through
/**
* Fetches a file from a remote sftp server and copies it to a local file location. The authentication method used
* is public/private key authentication. <br><br>
* IMPORTANT: Your private key must be in the OpenSSH format, also it must not have a passphrase associated with it.
* (currently the apache-commons-vfs2 library does not support passphrases)<p>
*
* Also remember your public key needs to be on the sftp server. If you were connecting as user 'bob' then your
* public key will need to be in '.ssh/bob' on the server (the location of .ssh will change depending on the type
* of sftp server)
*
* #param server The server we care connection to
* #param userName The username we are connection as
* #param openSSHPrivateKey The location of the private key (which must be in openSSH format) on the local machine
* #param remoteDir The directory from where you want to retrieve the file on the remote machine (this is in reference to SSH_HOME, SSH_HOME is the direcory you
* automatically get directed to when connecting)
* #param remoteFile The name of the file on the remote machine to be collected (does not support wild cards)
* #param localDir The direcoty on the local machine where you want the file to be copied to
* #param localFileName The name you wish to give to retrieved file on the local machine
* #throws IOException - Gets thrown is there is any problem fetching the file
*/
public static void sftpGetFile_keyAuthentication(String server, String userName, String openSSHPrivateKey,
String remoteDir,String remoteFile, String localDir, String localFileName) throws IOException
{
FileSystemOptions fsOptions = new FileSystemOptions();
FileSystemManager fsManager = null;
String remoteURL = "sftp://" + userName + "#" + server + "/" + remoteDir + "/" + remoteFile;
String localURL = "file://" + localDir + "/" + localFileName;
try {
SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fsOptions, "no");
SftpFileSystemConfigBuilder.getInstance().setIdentities(fsOptions, new File[]{new File(openSSHPrivateKey)});
fsManager = VFS.getManager();
FileObject remoteFileObject = fsManager.resolveFile(remoteURL, fsOptions);
LocalFile localFile =
(LocalFile) fsManager.resolveFile(localURL);
localFile.copyFrom(remoteFileObject,
new AllFileSelector());
} catch (FileSystemException e) {
LOGGER.error("Problem retrieving from " + remoteURL + " to " + localURL,e );
throw new IOException(e);
}
}

I guess this is what you are looking for -
/**
* #param args
*/
public static void main(String[] args) {
/*Below we have declared and defined the SFTP HOST, PORT, USER
and Local private key from where you will make connection */
String SFTPHOST = "10.20.30.40";
int SFTPPORT = 22;
String SFTPUSER = "kodehelp";
// this file can be id_rsa or id_dsa based on which algorithm is used to create the key
String privateKey = "/home/kodehelp/.ssh/id_rsa";
String SFTPWORKINGDIR = "/home/kodehelp/";
JSch jSch = new JSch();
Session session = null;
Channel channel = null;
ChannelSftp channelSftp = null;
try {
jSch.addIdentity(privateKey);
System.out.println("Private Key Added.");
session = jSch.getSession(SFTPUSER,SFTPHOST,SFTPPORT);
System.out.println("session created.");
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect();
channel = session.openChannel("sftp");
channel.connect();
System.out.println("shell channel connected....");
channelSftp = (ChannelSftp)channel;
channelSftp.cd(SFTPWORKINGDIR);
System.out.println("Changed the directory...");
} catch (JSchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (SftpException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
if(channelSftp!=null){
channelSftp.disconnect();
channelSftp.exit();
}
if(channel!=null) channel.disconnect();
if(session!=null) session.disconnect();
}
}
See more at
http://kodehelp.com/sftp-connection-public-key-authentication-java/

This post and answer were very helpful, thank you very much.
I just want to add an integration to the statement "currently the apache-commons-vfs2 library does not support passphrases", as I did it also with passphrase and it worked.
You have to import the jsch library in your project (I used 0.1.49) and implement the interface "com.jcraft.jsch.UserInfo".
Something like this should be fine:
public class SftpUserInfo implements UserInfo {
public String getPassphrase() {
return "yourpassphrase";
}
public String getPassword() {
return null;
}
public boolean promptPassphrase(String arg0) {
return true;
}
public boolean promptPassword(String arg0) {
return false;
}
}
And then you can add it to the SftpFileSystemConfigBuilder this way:
SftpFileSystemConfigBuilder.getInstance().setUserInfo(fsOptions, new SftpUserInfo());
Hope this helps.