I am new for SFTP Server. I am tried to upload and download files using SFTP server. It's a stand alone application.
While downloading files from remoter server to local server, I able to download list of files under specified path using following SFTP Code:
sftp.lcd(details.get("LOCAL_DIR"));
sftp.cd(details.get("REMOTE_DIR"));
List<SftpFile> remoteFiles = sftp.ls();
for(int i = 0 ; i < remoteFiles.size(); ++i) {
if(remoteFiles.get(i).isFile()) {
String remoteFile = remoteFiles.get(i).getFilename();
sftp.get(remoteFile);
System.out.println("[SFTPOperations][downLoad] Downloaded: " + remoteFile);
System.out.println("Remote File: " + remoteFile);
System.out.println("Remote Archive Dir: " + details.get("REMOTE_ARCHIVE_DIR"));
sftp.rename(remoteFiles.get(i).getFilename(), details.get("REMOTE_ARCHIVE_DIR"));
System.out.println("[SFTPOperations][downLoad] Archived: " + remoteFile);
}
}
By using above code, its downloading all the files from remote directory to local directory. Here I want to download files from remote directory which are matching given pattern.
E.g. pattern: Query*.txt
I want to download text files which are starts with Query.
Please suggest how to do the mentioned above.
Did you try this one?
if (remoteFile.endsWith(".txt")) {
sftp.get(remoteFile);
}
However you could use some sort of regular expresion to validate if the file name is valid according to the pattern you are traing to evaluate
using Renci.SshNet;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Configuration;
namespace FileDownload
{
class Program
{
static void Main(string[] args)
{
string date = DateTime.Now.ToShortDateString();
using (StreamWriter writer = File.CreateText(ConfigurationManager.AppSettings["localDirectory"]+"U2000_CSVFile_Fetching_log_"+date+".txt"))
{
try
{
string host = ConfigurationManager.AppSettings["host"];
string username = ConfigurationManager.AppSettings["username"];
string password = ConfigurationManager.AppSettings["password"];
string remoteDirectory = ConfigurationManager.AppSettings["remoteDirectory"]+date+"/";
Console.WriteLine(password);
string localDirectory = ConfigurationManager.AppSettings["localDirectory"] + date;
System.IO.Directory.CreateDirectory(localDirectory);
Console.WriteLine(localDirectory);
using (var sftp = new SftpClient(host, username, password))
{
sftp.Connect();
writer.WriteLine("Connection OK");
var files = sftp.ListDirectory(remoteDirectory);
foreach (var file in files)
{
string remoteFileName = file.Name;
writer.WriteLine(file.Name);
if ((!file.Name.StartsWith(".")))
{
using (Stream file1 = File.OpenWrite(localDirectory + remoteFileName))
{
writer.WriteLine(file.Name + " Download Started");
sftp.DownloadFile(remoteDirectory + remoteFileName, file1);
writer.WriteLine(file.Name + " Downloaded Successfully_"+date);
}
}
}
}
}
catch (Exception ex)
{
writer.WriteLine(ex.Message);
}
}
Console.WriteLine("U2000 .csv files fetching Scheduler is Running");
Console.ReadLine();
}
}
}
Related
I use java in my application. I use minio server to for file server. When I upload my file to the minio server, I save the url of the same file in the database. But after 7 days the same url is not working. I am using project in localhost and minio server free. I don't want to end the expiration date as 7 days. I want to show for life.
public Boolean uploadFile(List<MultipartFile> multipartFiles,
Product product) {
LinkedList<AppFile> appFiles = new LinkedList<>();
multipartFiles.forEach(multipartFile -> {
String fileName = multipartFile.getOriginalFilename();
String ext = FileNameUtils.getExtension(fileName);
AppFile appFile = new AppFile();
appFile.setName(fileName);
appFile.setExt(ext);
appFile.setDateCreated(new Date());
appFile.setUserId(UUID.randomUUID().toString());
appFile.setUuid(UUID.randomUUID().toString());
try {
UUID uuid = UUID.randomUUID();
// save minio server
minioClient.putObject(PutObjectArgs.builder()
.bucket(bucketName)
.object(uuid + "-" + multipartFile.getOriginalFilename())
.stream(multipartFile.getInputStream(), multipartFile.getSize(), -1)
.contentType(multipartFile.getContentType())
.build());
// file url
String url = minioClient.getPresignedObjectUrl(GetPresignedObjectUrlArgs.builder()
.method(Method.GET)
.bucket(bucketName)
.object(uuid + "-" + multipartFile.getOriginalFilename())
.build());
appFile.setUrl(url);
appFile.setProductId(product);
appFiles.add(appFile);
} catch (Exception e) {
throw new FileException(EnumFile.FILE_CAN_NOT_ADDED.getName() + " " + e.getMessage());
}
appFileRepository.saveAll(appFiles);
});
return true;
}
I am trying to download some files vis sftp using a java application which uses the sshj library. I am using this tutorial
My code is as follows:
try {
sftpClient = client.newSFTPClient();
remoteFileList = sftpClient.ls("gtdt/xml");
logger.debug("trying to explicitly donwload file");
sftpClient.get("remotepath/xml/somefile.xml", new FileSystemFile(localDownloadDir));
} catch (IOException e) {
logger.error("Error getting list of remote files on sftp server", e);
fail("error getting list or remote ftp files");
}
Later I loop over the files that were returned:
assertTrue(remoteFileList.size() > 0);
for (RemoteResourceInfo info : remoteFileList) {
logger.info(info.toString());
}
but when I try to download any files (have tried one and many) like so
// get first file in list and download
RemoteResourceInfo info = remoteFileList.get(0);
try {
String localCopyPath = localDownloadDir.getAbsolutePath() + "/" + info.getName();
FileSystemFile localCopyFile = new FileSystemFile(localDownloadDir.getAbsolutePath() + "/" + info.getName());
logger.debug("attempting to copy remote file " + info.getPath() + " to : " + localCopyPath);
sftpClient.get("remotepath/xml/somefile.xml", localCopyFile);
} catch (IOException e) {
logger.error("unable to copy file to local dir ", e);
fail("unable to copy " + info.getName() + " to local dir " + localDownloadDir.getAbsolutePath());
}
I get an error like this:
[main] WARN net.schmizz.sshj.xfer.FileSystemFile - Could not set permissions for c:\mypath\somefile.xml to 1a0
any idea how to fix this? The actual code that is being executed there starts at line 142 here. It wasn't clear to me if this is and issue on my local machine or something I need to configure in sshj.
thanks!
Is there an efficient way to check the existence of a file on a FTP server? I'm using Apache Commons Net. I know that I can use the listNames method of FTPClient to get all the files in a specific directory and then I can go over this list to check if a given file exists, but I don't think it's efficient especially when the server contains a lot of files.
listFiles(String pathName) should work just fine for a single file.
Using a full path to a file in listFiles (or mlistDir) call, as the accepted answer shows, should indeed work:
String remotePath = "/remote/path/file.txt";
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
if (remoteFiles.length > 0)
{
System.out.println("File " + remoteFiles[0].getName() + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
The RFC 959 in the section 4.1.3 in the part about the LIST command says:
If the pathname specifies a file then the server should send current information on the file.
Though if you are going to check for many files, this will be rather ineffective. A use of the LIST command actually involves several commands, waiting for their responses, and mainly, opening a data connection. Opening a new TCP/IP connection is a costly operation, even more so, when an encryption is used (what is a must these days).
Also LIST command is even more ineffective for testing an existence of a folder, as it results in a transfer of a complete folder contents.
More efficient is to use mlistFile (MLST command), if the server supports it:
String remotePath = "/remote/path/file.txt";
FTPFile remoteFile = ftpClient.mlistFile(remotePath);
if (remoteFile != null)
{
System.out.println("File " + remoteFile.getName() + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
This method can be used to test an existence of a directory.
MLST command does not use a separate connection (contrary to LIST).
If the server does not support MLST command, you can abuse getModificationTime (MDTM command) or getSize (SIZE command):
String timestamp = ftpClient.getModificationTime(remotePath);
if (timestamp != null)
{
System.out.println("File " + remotePath + " exists");
}
else
{
System.out.println("File " + remotePath + " does not exists");
}
This method cannot be used to test an existence of a directory.
The accepted answer did not work for me.
Code did not work:
String remotePath = "/remote/path/file.txt";
FTPFile[] remoteFiles = ftpClient.listFiles(remotePath);
Instead, this works for me:
ftpClient.changeWorkingDirectory("/remote/path");
FTPFile[] remoteFiles = ftpClient.listFiles("file.txt");
public boolean isDirectory(String dstPath) throws IOException {
return ftpsClient.changeWorkingDirectory(dstPath);
}
public boolean exists(String dstPath) throws IOException {
if (isDirectory(dstPath)) {
return true;
}
FTPFile[] remoteFiles = ftpsClient.listFiles(dstPath);
return remoteFiles != null && remoteFiles.length > 0;
}
My requirement is to copy the file from local machine(Windows) to unix serevr. I have the code which was working fine with some Mbs data. i am using jsch lib to connect and to transfer.
But now i have to transfer the files of 1GB, 2Gb or may be 5GB.
When i am using the same approach.
Its getting failed. Its stucking at
channelsftpObj.put(from,to);
and then exception is "faliure". Nothing else coming in exception.
May i know the reason or how can i transfer these files?
private boolean executeCommand (String localDir, String remoteDir, String fileList, String actionFlg) {
boolean boolError = false;
String localPath, destinationPath;
ChannelSftp channelSFTPObj = (ChannelSftp) channelObj;
for (int i = 0; i < filelistArr.length; i++ ) {
localPath = localDir + "/" + filelistArr[i];
destinationPath = remoteDir + "/" + filelistArr[i];
try {
if (actionFlg.toLowerCase() == "upload") {
channelSFTPObj.put (localPath, destinationPath);
System.out.println ("Uploaded " + filelistArr[i] + " to " + remoteDir);
}
}
catch (SftpException e) {
System.out.println(e);
boolError = true;
}
}
channelSFTPObj.exit();
return boolError;
}
localDir is my path of my local system, remoteDir is the server path.
Successfully connected to the server.
I am doing text file bulk upload in SQL Server. Whenever i tries to upload the files, gets Following Exception:
[Microsoft] [ODBC SQL Server Driver] [SQL Server]Could not bulk insert
because file 'C:/Form/Input_File/Form.txt' could not be opened.
Operating System error code 3(The system cannot find the path
specified).
Please find the below code:
public void uploadFiles()
{
File dir = new File(inputFilesPath);
String[] children = dir.list();
String completePathFileName = "";
System.out.println(" Inside Upload ::");
String saveFileNames = "";
PreparedStatement prepStat;
DBConnection dbConnection=new DBConnection();
Connection conHandler= dbConnection.getConnection();
if(null!=conHandler)
System.out.println(" Clear ::"+conHandler);
try
{
if (children != null)
{
for (int i = 0; i < children.length; i++)
{
String filename = children[i];
System.out.println(" children[i]::"+children[i]);
// File is validated based on some business rules.
if (isValidFile(filename) == 1)
{
String[] fileSplit = filename.split("E");
String[] extnSplit = fileSplit[1].trim().split(".TXT");
completePathFileName += (completePathFileName.equals(""))
? extnSplit[0] : "^" + extnSplit[0];
saveFileNames += (saveFileNames.equals(""))
? filename : "," + filename;
System.out.println(extnSplit[0]);
}
else
{
inValidFileNames += (inValidFileNames.equals(""))
? filename : ";\n" + filename;
}
}
if (!completePathFileName.trim().equals(""))
{
System.out.println(completePathFileName);
prepStat = conHandler.prepareStatement("Exec StartFileImport ?");
prepStat.setString(1, completePathFileName);
prepStat.execute();
saveFileNameToDatabase(saveFileNames);
}
}
}
catch (SQLException sql)
{
System.out.println(sql.getMessage());
}
}
Getting Connection Object from the below code:
public Connection getConnection()
{
System.out.println("In side DB Connection...");
try{
// get a database connection
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
System.out.println("Before Driver");
conn= DriverManager.getConnection("jdbc:odbc:form26qa","form26","form26");
System.out.println("After Driver");
if(conn != null)
{
System.out.println("Connection established...");
}//if
else
{
System.out.println("Connection failed...");
}//else
}//try
catch(Exception e)
{
System.out.println("Exception ocurred..."+e);
e.printStackTrace();
}//catch
return conn;
}
Explanation:
I am reading files from the input path and tried to get the fileName and file path and uploading file into SQL Server.
Application is able to find the input file in the specified path. while uploading i am getting the above mentioned Exception
Please check and suggest me to fix the issue.
The file needs to be accessable on the server. The file path is
relative to the server, not your PC. Also, if you are trying to use a
share or a mapped drive it will not work. You need to use the UNC
path.
UNC Name Examples
\\teela\admin$ (to reach C:\WINNT)
\\teela\admin$\system32 (to reach C:\WINNT\system32)
\\teela\temp (to reach C:\temp)