My requirement is to copy the file from local machine(Windows) to unix serevr. I have the code which was working fine with some Mbs data. i am using jsch lib to connect and to transfer.
But now i have to transfer the files of 1GB, 2Gb or may be 5GB.
When i am using the same approach.
Its getting failed. Its stucking at
channelsftpObj.put(from,to);
and then exception is "faliure". Nothing else coming in exception.
May i know the reason or how can i transfer these files?
private boolean executeCommand (String localDir, String remoteDir, String fileList, String actionFlg) {
boolean boolError = false;
String localPath, destinationPath;
ChannelSftp channelSFTPObj = (ChannelSftp) channelObj;
for (int i = 0; i < filelistArr.length; i++ ) {
localPath = localDir + "/" + filelistArr[i];
destinationPath = remoteDir + "/" + filelistArr[i];
try {
if (actionFlg.toLowerCase() == "upload") {
channelSFTPObj.put (localPath, destinationPath);
System.out.println ("Uploaded " + filelistArr[i] + " to " + remoteDir);
}
}
catch (SftpException e) {
System.out.println(e);
boolError = true;
}
}
channelSFTPObj.exit();
return boolError;
}
localDir is my path of my local system, remoteDir is the server path.
Successfully connected to the server.
Related
Is there an efficient way to check the existence of a file on a FTP server? I'm using Apache Commons Net. I know that I can use the listNames method of FTPClient to get all the files in a specific directory and then I can go over this list to check if a given file exists, but I don't think it's efficient especially when the server contains a lot of files.
listFiles(String pathName) should work just fine for a single file.
Using a full path to a file in listFiles (or mlistDir) call, as the accepted answer shows, should indeed work:
String remotePath = "/remote/path/file.txt";
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
if (remoteFiles.length > 0)
{
System.out.println("File " + remoteFiles[0].getName() + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
The RFC 959 in the section 4.1.3 in the part about the LIST command says:
If the pathname specifies a file then the server should send current information on the file.
Though if you are going to check for many files, this will be rather ineffective. A use of the LIST command actually involves several commands, waiting for their responses, and mainly, opening a data connection. Opening a new TCP/IP connection is a costly operation, even more so, when an encryption is used (what is a must these days).
Also LIST command is even more ineffective for testing an existence of a folder, as it results in a transfer of a complete folder contents.
More efficient is to use mlistFile (MLST command), if the server supports it:
String remotePath = "/remote/path/file.txt";
FTPFile remoteFile = ftpClient.mlistFile(remotePath);
if (remoteFile != null)
{
System.out.println("File " + remoteFile.getName() + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
This method can be used to test an existence of a directory.
MLST command does not use a separate connection (contrary to LIST).
If the server does not support MLST command, you can abuse getModificationTime (MDTM command) or getSize (SIZE command):
String timestamp = ftpClient.getModificationTime(remotePath);
if (timestamp != null)
{
System.out.println("File " + remotePath + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
This method cannot be used to test an existence of a directory.
The accepted answer did not work for me.
Code did not work:
String remotePath = "/remote/path/file.txt";
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
Instead, this works for me:
ftpClient.changeWorkingDirectory("/remote/path");
FTPFile[] remoteFiles = ftpClient.listFiles("file.txt");
public boolean isDirectory(String dstPath) throws IOException {
return ftpsClient.changeWorkingDirectory(dstPath);
}
public boolean exists(String dstPath) throws IOException {
if (isDirectory(dstPath)) {
return true;
}
FTPFile[] remoteFiles = ftpsClient.listFiles(dstPath);
return remoteFiles != null && remoteFiles.length > 0;
}
I am building a desktop app using javafx, I am downloading a file around 500 MB using ftp. I am choosing the download location using DirectoryChooser but after choosing the directory my application hang and doesn't response.
Though the file is downloaded.
here is my code:-
try {
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
success = ftpClient.changeWorkingDirectory(PATH + preset + "/" + file_to_download + offset);
System.out.println("Download Path:-" + PATH + preset + "/" + file_to_download + offset);
if (!success) {
System.out.println("Could not changed the directory to RIBS");
return;
} else {
System.out.println("Directory changed to RIBS");
}
FTPFile[] files = ftpClient.listFiles();
for (FTPFile file : files) {
if (file.getName().contains(".zip")) {
dfile = file.getName();
}
}
DirectoryChooser dirChooser = new DirectoryChooser();
File chosenDir = dirChooser.showDialog(tableView.getScene().getWindow());
System.out.println(chosenDir.getAbsolutePath());
OutputStream output;
output = new FileOutputStream(chosenDir.getAbsolutePath() + "/" + dfile);
int timeOut = 500;
ftpClient.setConnectTimeout(timeOut);
if (ftpClient.retrieveFile(dfile, output) == true) {
downloadButton.setDisable(true);
}
output.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
How can I improve this??
You are performing your download on the application thread which blocks the UI. Have a look at the documentation about JavaFX concurrency.
https://docs.oracle.com/javase/8/javafx/interoperability-tutorial/concurrency.htm
I need to copy a directory from my local machine to a remote machine via SFTP. I've done copying a file through JSCH API, but it doesn't work on directories. Any suggestions?
I'm using the following code:
JSch jsch = new JSch();
String filename = localFile.getName();
com.jcraft.jsch.Session sftpsession = jsch.getSession(username, hostname, 22);
sftpsession.setUserInfo(new HardcodedUserInfo(password));
Properties config = new Properties();
config.setProperty("StrictHostKeyChecking", "no");
sftpsession.setConfig(config);
sftpsession.connect();
ChannelSftp channel = (ChannelSftp)sftpsession.openChannel("sftp");
channel.connect();
channel.cd(remoteDirectory);
channel.put(new FileInputStream(localFile), filename);
channel.disconnect();
sftpsession.disconnect();
JSCH doesn't have a single function to recursively send or receive a directory through SFTP. Your code will have to build the list of files and directories to be created on the remote system, then call ChannelSftp.mkdir() and ChannelSftp.put() to create the directories and files.
Also remember that you need to create parent directories before you create subdirectories. For example, mkdir("/foo/bar/baz") will fail if directory /foo/bar doesn't exist.
You can copy folder recursively on remote server using JSCH java API.
Below is the sample code example for the same -
private static void recursiveFolderUpload(String sourcePath, String destinationPath) throws SftpException, FileNotFoundException {
File sourceFile = new File(sourcePath);
if (sourceFile.isFile()) {
// copy if it is a file
channelSftp.cd(destinationPath);
if (!sourceFile.getName().startsWith("."))
channelSftp.put(new FileInputStream(sourceFile), sourceFile.getName(), ChannelSftp.OVERWRITE);
} else {
System.out.println("inside else " + sourceFile.getName());
File[] files = sourceFile.listFiles();
if (files != null && !sourceFile.getName().startsWith(".")) {
channelSftp.cd(destinationPath);
SftpATTRS attrs = null;
// check if the directory is already existing
try {
attrs = channelSftp.stat(destinationPath + "/" + sourceFile.getName());
} catch (Exception e) {
System.out.println(destinationPath + "/" + sourceFile.getName() + " not found");
}
// else create a directory
if (attrs != null) {
System.out.println("Directory exists IsDir=" + attrs.isDir());
} else {
System.out.println("Creating dir " + sourceFile.getName());
channelSftp.mkdir(sourceFile.getName());
}
for (File f: files) {
recursiveFolderUpload(f.getAbsolutePath(), destinationPath + "/" + sourceFile.getName());
}
}
}
}
You can refer link here for more details on this code.
You can use this code to copy a directory:
copy(File localFile, String destPath, ChannelSftp clientChannel)
{
if (localFile.isDirectory()) {
clientChannel.mkdir(localFile.getName());
System.out.println("Created Folder: " + localFile.getName() + " in " + destPath);
destPath = destPath + "/" + localFile.getName();
clientChannel.cd(destPath);
for (File file : localFile.listFiles()) {
copy(file, destPath, clientChannel);
}
clientChannel.cd(destPath.substring(0, destPath.lastIndexOf('/')));
} else {
System.out.println("Copying File: " + localFile.getName() + " to " + destPath);
clientChannel.put(new FileInputStream(localFile), localFile.getName(), ChannelSftp.OVERWRITE);
}
}
I am doing text file bulk upload in SQL Server. Whenever i tries to upload the files, gets Following Exception:
[Microsoft] [ODBC SQL Server Driver] [SQL Server]Could not bulk insert
because file 'C:/Form/Input_File/Form.txt' could not be opened.
Operating System error code 3(The system cannot find the path
specified).
Please find the below code:
public void uploadFiles()
{
File dir = new File(inputFilesPath);
String[] children = dir.list();
String completePathFileName = "";
System.out.println(" Inside Upload ::");
String saveFileNames = "";
PreparedStatement prepStat;
DBConnection dbConnection=new DBConnection();
Connection conHandler= dbConnection.getConnection();
if(null!=conHandler)
System.out.println(" Clear ::"+conHandler);
try
{
if (children != null)
{
for (int i = 0; i < children.length; i++)
{
String filename = children[i];
System.out.println(" children[i]::"+children[i]);
// File is validated based on some business rules.
if (isValidFile(filename) == 1)
{
String[] fileSplit = filename.split("E");
String[] extnSplit = fileSplit[1].trim().split(".TXT");
completePathFileName += (completePathFileName.equals(""))
? extnSplit[0] : "^" + extnSplit[0];
saveFileNames += (saveFileNames.equals(""))
? filename : "," + filename;
System.out.println(extnSplit[0]);
}
else
{
inValidFileNames += (inValidFileNames.equals(""))
? filename : ";\n" + filename;
}
}
if (!completePathFileName.trim().equals(""))
{
System.out.println(completePathFileName);
prepStat = conHandler.prepareStatement("Exec StartFileImport ?");
prepStat.setString(1, completePathFileName);
prepStat.execute();
saveFileNameToDatabase(saveFileNames);
}
}
}
catch (SQLException sql)
{
System.out.println(sql.getMessage());
}
}
Getting Connection Object from the below code:
public Connection getConnection()
{
System.out.println("In side DB Connection...");
try{
// get a database connection
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
System.out.println("Before Driver");
conn= DriverManager.getConnection("jdbc:odbc:form26qa","form26","form26");
System.out.println("After Driver");
if(conn != null)
{
System.out.println("Connection established...");
}//if
else
{
System.out.println("Connection failed...");
}//else
}//try
catch(Exception e)
{
System.out.println("Exception ocurred..."+e);
e.printStackTrace();
}//catch
return conn;
}
Explanation:
I am reading files from the input path and tried to get the fileName and file path and uploading file into SQL Server.
Application is able to find the input file in the specified path. while uploading i am getting the above mentioned Exception
Please check and suggest me to fix the issue.
The file needs to be accessable on the server. The file path is
relative to the server, not your PC. Also, if you are trying to use a
share or a mapped drive it will not work. You need to use the UNC
path.
UNC Name Examples
\\teela\admin$ (to reach C:\WINNT)
\\teela\admin$\system32 (to reach C:\WINNT\system32)
\\teela\temp (to reach C:\temp)
I am new for SFTP Server. I am tried to upload and download files using SFTP server. It's a stand alone application.
While downloading files from remoter server to local server, I able to download list of files under specified path using following SFTP Code:
sftp.lcd(details.get("LOCAL_DIR"));
sftp.cd(details.get("REMOTE_DIR"));
List<SftpFile> remoteFiles = sftp.ls();
for(int i = 0 ; i < remoteFiles.size(); ++i) {
if(remoteFiles.get(i).isFile()) {
String remoteFile = remoteFiles.get(i).getFilename();
sftp.get(remoteFile);
System.out.println("[SFTPOperations][downLoad] Downloaded: " + remoteFile);
System.out.println("Remote File: " + remoteFile);
System.out.println("Remote Archive Dir: " + details.get("REMOTE_ARCHIVE_DIR"));
sftp.rename(remoteFiles.get(i).getFilename(), details.get("REMOTE_ARCHIVE_DIR"));
System.out.println("[SFTPOperations][downLoad] Archived: " + remoteFile);
}
}
By using above code, its downloading all the files from remote directory to local directory. Here I want to download files from remote directory which are matching given pattern.
E.g. pattern: Query*.txt
I want to download text files which are starts with Query.
Please suggest how to do the mentioned above.
Did you try this one?
if (remoteFile.endsWith(".txt")) {
sftp.get(remoteFile);
}
However you could use some sort of regular expresion to validate if the file name is valid according to the pattern you are traing to evaluate
using Renci.SshNet;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Configuration;
namespace FileDownload
{
class Program
{
static void Main(string[] args)
{
string date = DateTime.Now.ToShortDateString();
using (StreamWriter writer = File.CreateText(ConfigurationManager.AppSettings["localDirectory"]+"U2000_CSVFile_Fetching_log_"+date+".txt"))
{
try
{
string host = ConfigurationManager.AppSettings["host"];
string username = ConfigurationManager.AppSettings["username"];
string password = ConfigurationManager.AppSettings["password"];
string remoteDirectory = ConfigurationManager.AppSettings["remoteDirectory"]+date+"/";
Console.WriteLine(password);
string localDirectory = ConfigurationManager.AppSettings["localDirectory"] + date;
System.IO.Directory.CreateDirectory(localDirectory);
Console.WriteLine(localDirectory);
using (var sftp = new SftpClient(host, username, password))
{
sftp.Connect();
writer.WriteLine("Connection OK");
var files = sftp.ListDirectory(remoteDirectory);
foreach (var file in files)
{
string remoteFileName = file.Name;
writer.WriteLine(file.Name);
if ((!file.Name.StartsWith(".")))
{
using (Stream file1 = File.OpenWrite(localDirectory + remoteFileName))
{
writer.WriteLine(file.Name + " Download Started");
sftp.DownloadFile(remoteDirectory + remoteFileName, file1);
writer.WriteLine(file.Name + " Downloaded Successfully_"+date);
}
}
}
}
}
catch (Exception ex)
{
writer.WriteLine(ex.Message);
}
}
Console.WriteLine("U2000 .csv files fetching Scheduler is Running");
Console.ReadLine();
}
}
}