I need to copy a directory from my local machine to a remote machine via SFTP. I've done copying a file through JSCH API, but it doesn't work on directories. Any suggestions?
I'm using the following code:
JSch jsch = new JSch();
String filename = localFile.getName();
com.jcraft.jsch.Session sftpsession = jsch.getSession(username, hostname, 22);
sftpsession.setUserInfo(new HardcodedUserInfo(password));
Properties config = new Properties();
config.setProperty("StrictHostKeyChecking", "no");
sftpsession.setConfig(config);
sftpsession.connect();
ChannelSftp channel = (ChannelSftp)sftpsession.openChannel("sftp");
channel.connect();
channel.cd(remoteDirectory);
channel.put(new FileInputStream(localFile), filename);
channel.disconnect();
sftpsession.disconnect();
JSCH doesn't have a single function to recursively send or receive a directory through SFTP. Your code will have to build the list of files and directories to be created on the remote system, then call ChannelSftp.mkdir() and ChannelSftp.put() to create the directories and files.
Also remember that you need to create parent directories before you create subdirectories. For example, mkdir("/foo/bar/baz") will fail if directory /foo/bar doesn't exist.
You can copy folder recursively on remote server using JSCH java API.
Below is the sample code example for the same -
private static void recursiveFolderUpload(String sourcePath, String destinationPath) throws SftpException, FileNotFoundException {
File sourceFile = new File(sourcePath);
if (sourceFile.isFile()) {
// copy if it is a file
channelSftp.cd(destinationPath);
if (!sourceFile.getName().startsWith("."))
channelSftp.put(new FileInputStream(sourceFile), sourceFile.getName(), ChannelSftp.OVERWRITE);
} else {
System.out.println("inside else " + sourceFile.getName());
File[] files = sourceFile.listFiles();
if (files != null && !sourceFile.getName().startsWith(".")) {
channelSftp.cd(destinationPath);
SftpATTRS attrs = null;
// check if the directory is already existing
try {
attrs = channelSftp.stat(destinationPath + "/" + sourceFile.getName());
} catch (Exception e) {
System.out.println(destinationPath + "/" + sourceFile.getName() + " not found");
}
// else create a directory
if (attrs != null) {
System.out.println("Directory exists IsDir=" + attrs.isDir());
} else {
System.out.println("Creating dir " + sourceFile.getName());
channelSftp.mkdir(sourceFile.getName());
}
for (File f: files) {
recursiveFolderUpload(f.getAbsolutePath(), destinationPath + "/" + sourceFile.getName());
}
}
}
}
You can refer link here for more details on this code.
You can use this code to copy a directory:
copy(File localFile, String destPath, ChannelSftp clientChannel)
{
if (localFile.isDirectory()) {
clientChannel.mkdir(localFile.getName());
System.out.println("Created Folder: " + localFile.getName() + " in " + destPath);
destPath = destPath + "/" + localFile.getName();
clientChannel.cd(destPath);
for (File file : localFile.listFiles()) {
copy(file, destPath, clientChannel);
}
clientChannel.cd(destPath.substring(0, destPath.lastIndexOf('/')));
} else {
System.out.println("Copying File: " + localFile.getName() + " to " + destPath);
clientChannel.put(new FileInputStream(localFile), localFile.getName(), ChannelSftp.OVERWRITE);
}
}
Related
I want to check if a folder exist in the datalake. If exist create a file in it and If folder doesn't exist create folder and then create a file inside the folder
File directory = new File("/Raw/TEST_1/test");
System.out.println("check if directory exist");
if (directory.exists() == false) {
client.createDirectory("/Raw/TEST_1/test");
System.out.println("Directory created.");
OutputStream stream = client.createFile("/Raw/TEST_1/test/" + FuFileName, IfExists.OVERWRITE);
} else {
System.out.println("Directory exist.");
OutputStream stream = client.createFile("/Raw/TEST_1/test" + FuFileName, IfExists.OVERWRITE);
}
} catch (ADLException ex) {
printExceptionDetails(ex);
} catch (Exception ex) {
System.out.format(" Exception: %s%n Message: %s%n", ex.getClass().getName(), ex.getMessage());
}
every time directory.exists() gives me output false even if that folder exist if not giving directory.exists()output as true and not executing else statements
If you are using Java 7 or higher, then it is recommended to use java.nio in order to access the file system.
This very simple example can tell you if a given path is existing and if it is a directory or something else:
import java.nio.file.Path;
import java.nio.file.Paths;
...
public static void main(String[] args) {
Path dirPath = Paths.get("/Raw/TEST_1/test");
if (Files.exists(dirPath)) {
System.out.println(dirPath.toAbsolutePath().toString()
+ " exists on the file system");
if (Files.isDirectory(dirPath)) {
System.out.println(dirPath.toAbsolutePath().toString()
+ " is a directory");
} else {
System.err.println(dirPath.toAbsolutePath().toString()
+ " is not a directory");
}
} else {
System.err.println(dirPath.toAbsolutePath().toString()
+ " does not exist on the file system");
}
}
Is there an efficient way to check the existence of a file on a FTP server? I'm using Apache Commons Net. I know that I can use the listNames method of FTPClient to get all the files in a specific directory and then I can go over this list to check if a given file exists, but I don't think it's efficient especially when the server contains a lot of files.
listFiles(String pathName) should work just fine for a single file.
Using a full path to a file in listFiles (or mlistDir) call, as the accepted answer shows, should indeed work:
String remotePath = "/remote/path/file.txt";
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
if (remoteFiles.length > 0)
{
System.out.println("File " + remoteFiles[0].getName() + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
The RFC 959 in the section 4.1.3 in the part about the LIST command says:
If the pathname specifies a file then the server should send current information on the file.
Though if you are going to check for many files, this will be rather ineffective. A use of the LIST command actually involves several commands, waiting for their responses, and mainly, opening a data connection. Opening a new TCP/IP connection is a costly operation, even more so, when an encryption is used (what is a must these days).
Also LIST command is even more ineffective for testing an existence of a folder, as it results in a transfer of a complete folder contents.
More efficient is to use mlistFile (MLST command), if the server supports it:
String remotePath = "/remote/path/file.txt";
FTPFile remoteFile = ftpClient.mlistFile(remotePath);
if (remoteFile != null)
{
System.out.println("File " + remoteFile.getName() + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
This method can be used to test an existence of a directory.
MLST command does not use a separate connection (contrary to LIST).
If the server does not support MLST command, you can abuse getModificationTime (MDTM command) or getSize (SIZE command):
String timestamp = ftpClient.getModificationTime(remotePath);
if (timestamp != null)
{
System.out.println("File " + remotePath + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
This method cannot be used to test an existence of a directory.
The accepted answer did not work for me.
Code did not work:
String remotePath = "/remote/path/file.txt";
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
Instead, this works for me:
ftpClient.changeWorkingDirectory("/remote/path");
FTPFile[] remoteFiles = ftpClient.listFiles("file.txt");
public boolean isDirectory(String dstPath) throws IOException {
return ftpsClient.changeWorkingDirectory(dstPath);
}
public boolean exists(String dstPath) throws IOException {
if (isDirectory(dstPath)) {
return true;
}
FTPFile[] remoteFiles = ftpsClient.listFiles(dstPath);
return remoteFiles != null && remoteFiles.length > 0;
}
I am building a desktop app using javafx, I am downloading a file around 500 MB using ftp. I am choosing the download location using DirectoryChooser but after choosing the directory my application hang and doesn't response.
Though the file is downloaded.
here is my code:-
try {
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
success = ftpClient.changeWorkingDirectory(PATH + preset + "/" + file_to_download + offset);
System.out.println("Download Path:-" + PATH + preset + "/" + file_to_download + offset);
if (!success) {
System.out.println("Could not changed the directory to RIBS");
return;
} else {
System.out.println("Directory changed to RIBS");
}
FTPFile[] files = ftpClient.listFiles();
for (FTPFile file : files) {
if (file.getName().contains(".zip")) {
dfile = file.getName();
}
}
DirectoryChooser dirChooser = new DirectoryChooser();
File chosenDir = dirChooser.showDialog(tableView.getScene().getWindow());
System.out.println(chosenDir.getAbsolutePath());
OutputStream output;
output = new FileOutputStream(chosenDir.getAbsolutePath() + "/" + dfile);
int timeOut = 500;
ftpClient.setConnectTimeout(timeOut);
if (ftpClient.retrieveFile(dfile, output) == true) {
downloadButton.setDisable(true);
}
output.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
How can I improve this??
You are performing your download on the application thread which blocks the UI. Have a look at the documentation about JavaFX concurrency.
https://docs.oracle.com/javase/8/javafx/interoperability-tutorial/concurrency.htm
My requirement is to copy the file from local machine(Windows) to unix serevr. I have the code which was working fine with some Mbs data. i am using jsch lib to connect and to transfer.
But now i have to transfer the files of 1GB, 2Gb or may be 5GB.
When i am using the same approach.
Its getting failed. Its stucking at
channelsftpObj.put(from,to);
and then exception is "faliure". Nothing else coming in exception.
May i know the reason or how can i transfer these files?
private boolean executeCommand (String localDir, String remoteDir, String fileList, String actionFlg) {
boolean boolError = false;
String localPath, destinationPath;
ChannelSftp channelSFTPObj = (ChannelSftp) channelObj;
for (int i = 0; i < filelistArr.length; i++ ) {
localPath = localDir + "/" + filelistArr[i];
destinationPath = remoteDir + "/" + filelistArr[i];
try {
if (actionFlg.toLowerCase() == "upload") {
channelSFTPObj.put (localPath, destinationPath);
System.out.println ("Uploaded " + filelistArr[i] + " to " + remoteDir);
}
}
catch (SftpException e) {
System.out.println(e);
boolError = true;
}
}
channelSFTPObj.exit();
return boolError;
}
localDir is my path of my local system, remoteDir is the server path.
Successfully connected to the server.
Could anybody help me to copy file from shared folder to local drive? My code is:
import jcifs.smb.NtlmPasswordAuthentication;
import jcifs.smb.SmbFile;
import jcifs.smb.SmbFileInputStream;
import jcifs.smb.SmbFileOutputStream;;
public class smb {
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
String urlToBackUpFile = "smb://ip/backup$/test.txt";
System.out.println("smb folder of source file" + urlToBackUpFile);
NtlmPasswordAuthentication auth = new NtlmPasswordAuthentication(null, "login", "pass");
SmbFile dir = new SmbFile(urlToBackUpFile, auth);
System.out.println(dir.getDate());
SmbFile dest = new SmbFile ("C:/SQLRESTORESTAGE/v2.bak");
dir.copyTo(dest);
}
}
File file is not copied. I received a message "Failed to connect to server", but the programm shows dir.getDate() (and file name, and lenght) of source file. So I think the problem with destination folder (C:/SQLRESTORESTAGE/). Also I have proviledges only for reading source file. Could you help me to cirrect the code or advise something? Thank you.
maybe adding auth to the second file:
SmbFile dest = new SmbFile ("C:/SQLRESTORESTAGE/v2.bak",**auth**);
using SmbFile dest = new SmbFile ("C:/SQLRESTORESTAGE",auth).canWrite
you know if you have write permissions on the parent directory
After many trials and failures the only method that worked for me reliably was to go old school and use FileInputStream and FileOutputStream like so:
`SmbFile[] files = getSMBListOfFiles(sb, logger, domain, userName, password, sourcePath, sourcePattern);
if (files == null)
return false;
output(sb, logger, " Source file count: " + files.length);
String destFilename;
FileOutputStream fileOutputStream;
InputStream fileInputStream;
byte[] buf;
int len;
for (SmbFile smbFile: files) {
destFilename = destinationPath + smbFile.getName();
output(sb, logger, " copying " + smbFile.getName());
try {
fileOutputStream = new FileOutputStream(destFilename);
fileInputStream = smbFile.getInputStream();
buf = new byte[16 * 1024 * 1024];
while ((len = fileInputStream.read(buf)) > 0) {
fileOutputStream.write(buf, 0, len);
}
fileInputStream.close();
fileOutputStream.close();
} catch (SmbException e) {
OutputHandler.output(sb, logger, "Exception during copyNetworkFilesToLocal stream to output, SMP issue: " + e.getMessage(), e);
e.printStackTrace();
return false;
} catch (FileNotFoundException e) {
OutputHandler.output(sb, logger, "Exception during copyNetworkFilesToLocal stream to output, file not found: " + e.getMessage(), e);
e.printStackTrace();
return false;
} catch (IOException e) {
OutputHandler.output(sb, logger, "Exception during copyNetworkFilesToLocal stream to output, IO problem: " + e.getMessage(), e);
e.printStackTrace();
return false;
}
}`
I got it to work. I had to 'create' the destination file before doing the copy. try adding the middle line below into your original code-snippet and see if that works.
SmbFile dest = new SmbFile ("C:/SQLRESTORESTAGE/v2.bak");
dest.createNewFile();
dir.copyTo(dest);
This is for clarification.
"Logon failure: unknown user name or bad password." can show for example when you use 1.3.18 but not in 1.2.25.
it is probably because different compatibility settings:
jcifs.smb.lmCompatibility = 0 or 1: Send LM and NTLM 2)
jcifs.smb.lmCompatibility = 2: Send NTLM in both fields 3)
jcifs.smb.lmCompatibility = 3, 4, or 5: Send just LMv2
First way is to use it before NtlmPasswordAuthentication
jcifs.Config.setProperty( "jcifs.smb.lmCompatibility", "3");
It can resolve this issue.
you must use the file: protocol
SmbFile dest = new SmbFile ("file:" + "C:/SQLRESTORESTAGE/v2.bak");