I am trying to read a file which is in the remote linux server. But I do not know how to get the inputstream of the file using java.
How can this be done?
Assuming that by "remote linux server" you mean "remote linux shell", you should use an ssh library like JSch. You can find a file download example here.
Maybe SSHJ can help you? https://github.com/shikhar/sshj
Features of the library include:
reading known_hosts files for host key verification
publickey, password and keyboard-interactive authentication
command, subsystem and shell channels
local and remote port forwarding
scp + complete sftp version 0-3 implementation
Assuming you have a working connection to the server and access to the file, you can create a File object with the URI of the file:
File f = new File(uri);
FileInputStream fis = new FileInputStream(f);
The URI should be the URI to the file, for example "file://server/path/to/file".
See also the Javadoc for File(URI) .
It depends on how is the file available. Is it by HTTP, FTP, SFTP or through a server you wrote yourself ?
If your want to get the file through HTTP, you can use this :
HttpURLConnection connec = (HttpURLConnection)new URL("http://host/file").openConnection();
if(connec.getResponseCode() != connec.HTTP_OK)
{
System.err.println("Not OK");
return;
}
System.out.println("length = " + connec.getContentLength());
System.out.println("Type = " + connec.getContentType());
InputStream in = connec.getInputStream();
//Now you can read the file content with in
There is also Jsch library which is very good for SFTP / SCP
You can use any ssh java lib, as was mentioned in other answers, or mount directory with file as NFS share folder. After mounting you can use usual java API to acsess file.
Example
Related
We are currently in the process of exploring the sshj library to download a file from SFTP path into ADLS. We are using the example as reference.
We have already configured the ADLS Gen2 storage in Databricks to be accessed as an abfss URL.
We are using scala within Databricks.
How should we pass the abfss path as FileSystemFile object in the get step ?
sftp.get("test_file", new FileSystemFile("abfss://<container_name>#a<storage_account>.dfs.core.windows.net/<path>"));
Is the destination supposed to be a file path only or file path with file name?
Use streams. First obtain InputStream of the source SFTP file:
RemoteFile f = sftp.open(sftpPath);
InputStream is = f.new RemoteFileInputStream(0);
(How to read from the remote file into a Stream?)
Then obtain OutputStream of the destination file on ADLS:
OutputStream os = adlsStoreClient.createFile(adlsPath, IfExists.OVERWRITE);
(How to upload and download a file from my locale to azure adls using java sdk?)
And copy from the first to the other:
is.transferTo(os);
(Easy way to write contents of a Java InputStream to an OutputStream)
I have a CSV file, and I need to copy it and rename it in the same path.
I tried this after the FTP login:
InputStream inputStream = ftpClient.retrieveFileStream(cvs_name +".csv");
ftpClient.storeFile(cvs_name2 + ".csv",inputStream);
But when I verify the file on the server, it's empty. How can I copy a file and rename it?
I believe your code cannot work. You cannot download and upload a file over a single FTP connection at the same time.
You have two options:
Download the file completely first (to a temporary file or to a memory).
The accepted answer to How to copy a file on the ftp server to a directory on the same server in java? shows the "to memory" solution. Note the outputStream.toByteArray() call.
Open two connections (two instances of the FTPClient) and copy the file between the instances.
InputStream inputStream = ftpClient1.retrieveFileStream(cvs_name + ".csv");
ftpClient2.storeFile(cvs_name2 + ".csv", inputStream);
It seems that my code won't work after learning that the machine that I'll be pointing the upload path to is a Linux box.
My use case is, a user logs in to the web app, chooses a file to upload, then click upload button. Is it possible to do this direct from the Java code to the Linux server using appropriate ssh or scp libraries if there is any?
EDIT: Here's my current code.
#Override
public void fileTransfer(File uploadedFile, String fileName, String pathTemp) {
File destFile = new File( pathTemp + File.separator + fileName);
try{
FileUtils.copyFile(uploadedFile, destFile);
String getTempFile = destFile.toString();
String tempPath = getTempFile.replace("\\", "\\\\");
File tempFile = new File(tempPath); // 1st file
String tempFileName = tempFile.getName();
String fileSave = getUploadPathSave().replace("\\", "\\\\");
tempFile.renameTo(new File(fileSave + tempFileName));
} catch (IOException ex) {
System.out.println("Could not copy file " + fileName);
ex.printStackTrace();
}
}
If your app is deployed at one place only (not mass distribution), the easiest way would be:
create samba share on linux machine
map samba share to logical drive on windows machine
do usual file copy with java functions.
attention: renameTo will not work between drives. You'll need to copy input stream to output stream or, better, use apache commons-io functions for that.
There are different possibilities:
If you can create a shared directory in linux and mount it under windows (see Samba. Then you can write to that directory like a local directory. File will go to the linux server.
Use a library like Jsch to upload the file from windows server to linux server.
There are certain things you can do:
1-> If you can program your linux server, then you can make a program that listens to user request on a port, and stores data in file. Then you can send you files to that port of server.
2-> The other way is you can use some sort of script to create ssh-connection to server and then you can just add file through ssh, but here your java program will not be useful.
I personally use my own program to share files between 2 machines in same network.
You can use it,if it will be useful for you: https://github.com/RishabhRD/xshare
File dir = new File(System.getProperty("user.home")+"\\Desktop\\" + svc);
dir.mkdir();
File f;
f = new File(System.getProperty("user.home")+"\\Desktop\\" + svc
+ "\\" + logFile + "_" + System.currentTimeMillis()
+ ".txt");
I am using this code to store the files in the user(client) machine.But it is storing the files in the server machine.Can anyone help me on this? I have deployed my war file in unix server.
Saving a file on a client machine from software running on a server is not as simple as that.
Servers do not have direct access to the file system of any client - it would be very insecure if that were the case.
The simplest way to do this is by making the server return a web page with a link which the user can click to download the file.
You could also do something more complicated, for example write an applet that downloads the file (using some file transfer protocol) and saves it in the local file system. The applet would need to have the appropriate permissions (by default, applets cannot access the local file system).
Seems like this code is part of a Web Application Server Side. In this case System.getProperty("user.home") will return Server's home directory.
I am trying to download a file from SFTP server to my local machine using JSch. It only downloads 16371 bytes of data regardless of the file size and ends the transfer. It does not throw any exception. If the file is smaller than 16371 bytes it is transfered sucesfully, but for any larger file the transfer results in a corrupted file.
Actually I managed to solve this problem. I replaced:
SftpProgressMonitor monitor = new MySftpProgressMonitor();
channelSftp.get(sourceFile, destFile, monitor);
with:
channelSftp.get(sourceFile, destFile);
After removing the progress monitor the transfer was sucesfull. I guess it is a bug inside Jsch.
I had the same problem with a special sftp server. My unique solution was use LFTP, wich is a linux command for automatize SFTP tasks. If you have a Linux environment it is very useful.
Example in PHP:
$command = set net:timeout 30; lcd $directorioDestino; cd /Usr/companies/cdrusr357901/CallCenterRecords/; mget {$prefijoArchivosAuris}*; bye
lftp -u {$this->user},{$this->password} -e '$command' sftp://{$this->host}