I am working on a project that automatically uploads files to the SFTP server. My project is now in production and functioning as per the initial requirement spec. My project picks-up a txt file from a predefined location. Now because of the size limit on the SFTP server, the txt file must now be Zipped before the upload.
I now want to change my camel route to cater not only for txt files but also for xls, csv and Zip files.
How do i do that?
Currently my route looks as follows:
from("quartz://myscheduler?cron={{cron}}")
.pollEnrich("file:{{pickuplocation}}?moveFailed=error/${file:name.noext}_${date:now:yyyyMMddHHmmssSSS}.${file:ext}&move=SFTPCompleted/${date:now:MMM}-${date:now:yyyy}/${file:name.noext}_${date:now:yyyyMMddHHmmssSSS}.${file:ext}")
.setHeader("CamelFileName", simple("${file:name}"))
.setHeader("RouteID",constant("Route ID"))
.multicast()
.to("sftp://"+username+"#"+SftpLocation+"password="+password+"&stepwise=false&disconnect=true&fileName=${file:name.noext}.txt")
.end()
TIA
I don't see why you use such a complicated way of copying files. If I'm not mistaken you could just use this:
from("file:{{pickuplocation}}?moveFailed=error/${file:name.noext}_${date:now:yyyyMMddHHmmssSSS}.${file:ext}&move=SFTPCompleted/${date:now:MMM}-${date:now:yyyy}/${file:name.noext}_${date:now:yyyyMMddHHmmssSSS}.${file:ext})
.to("sftp://"+username+"#"+SftpLocation+"password="+password+"&stepwise=false&disconnect=true")
This will pickup any file in the pickupLocation directory and the FTP compenent will write the file with the same name to the remote server.
You can use the 'include' parameter of the File component to set a regex to match only certain file types: http://camel.apache.org/file2.html
Related
I need to modify html file that is placed at server folder from my servlet.
No other way than read it by FileInputStream to byte[], convert to String[] splitting lines by "\n", change what I need and then rewrite it.
I don't see.
This is not possible by design. Your server might just have to serve a .WAR file. If the server is not configured to unzip it, your server will have to read all files directly from this archive. You can now guess that you cannot write at this location.
You would need to create some kind of working directory and also serve files from there, too. You can always use this directory as working directory:
File workingDir = (File)servletContext.getAttribute(ServletContext.TEMPDIR);
I am currently working on a project using jmeter-maven plugin. I need to use a CSV data config file from which some variables are to be read during execution. Although the created JMX file works in the JMeter GUI, the same isn't working in non-gui mode.
From the logs, I was able to gather that the data file isn't being opened (stored) during execution in the non-gui mode, which on the other hand, it happens in the GUI mode leading to successful execution in GUI mode.
I have checked the path of the file (the absolute path of the file - with forward slashes), permissions of the file and all the parameters set in the CSV data config element in the jmx file created (it has proper path to the csv file) and I couldn't possibly get the reason on why the CSV file is not being used during execution in non-gui mode.
I have tried having the CSV file in the bin folder (giving the complete path of to the bin folder and just the file name - both methods), having the csv in the same path as the jmx file.
Any ideas on what I might be missing?
CSV Config Data CSV configuration Image
I have a mule SFTP connector from which I want to download a ZIP file and store it locally on my computer. I am using only SFTP connector and file outbound.
The issue is that the file is somehow changed to a strange file with .dat extension. I assume this is because of InputSftpStream.
Any ideas how to download a ZIP file and save the same file without any change to computer?
This issue occurs when file name output pattern is not defined. Please update file outbound endpoint like outputPattern="#[message.inboundProperties.originalFilename]" , this will create the file with the same name as the SFTP file name or you can change it to any desired pattern like outputPattern="xyz.zip". Something like
<file:outbound-endpoint path="tmp" outputPattern="#[message.inboundProperties.originalFilename]" connector-ref="File" responseTimeout="10000" doc:name="File"/>
Hope this help.
As a small workaround, you could change the file extension after the fact with:
File.renameTo(File dest)
I've a project in netbeans which my employer wants me to add a bulk file upload.
The requirement was ,the user will put a csv file and some images in a folder in client side. on the file upload page he will choose the csv file and i have to upload the images in the folder along with the csv.
After a short research i found that the client side file location and details can't be accessed from the server side so instead of uploading a single file and the contents in the folder which it belong, the user will zip all the files and upload the zip file .
Now i'm conserned about the security risks of uploading a zip file.
What all are the measures should i take to prevent the upload of malecious scripts and files along with the zip?
Is it possible to validate the file content before it reach the server ?
Is it possible to validate it securely even after it reached the server side ?
1.- scan the zip files once they arrive at the server.
2.- unlikely. even if you use Javascript as front-end, it doesn't have access to the users file system.
3.- yes, for example with "clamav", but notice that no antivirus is 100% effective.
the steps are:
install clamav
configure clamav to update periodically its virus database
schedule a cron job that continuously scans all files that are in certain directory (the directory where you upload the users files)
delete files that contain viruses
for example:
# dnf install -y clamav
# dnf install -y clamav-update
$ clamscan java_error_in_IDEA_6451.log.zip
LibClamAV Warning: **************************************************
LibClamAV Warning: * The virus database is older than 7 days!
LibClamAV Warning: Please update it as soon as possible. *
LibClamAV Warning: **************************************************
java_error_in_IDEA_6451.log.zip: OK
----------- SCAN SUMMARY -----------
Known viruses: 4490129
Engine version: 0.99.2
Scanned directories: 0
Scanned files: 1
Infected files: 0
Data scanned: 0.37 MB
Data read: 0.03 MB (ratio 13.43:1)
Time: 6.239 sec (0 m 6 s)
Antivirus is optional. Everything you need here is to validate and safely extract files from archive to your server storage. I don't recommend to use system utilities for this (zip, etc.). It's better to take some library for working with zip archives and write code for its processing.
Here's possible algorithm for validating files in archive. You should iterate through every file and check if:
File size is too big.
Filename is too long.
Directory structure is too big.
Filename includes ../.
File extension is not in white list.
File has attributes of symlink.
Other rules.
If something is true for your current file, then it's better to stop processing of such archive and show error for client.
When you've checked everything, then you can start to extract files. From this moment you can try to validate content of file using file or exif tools.
Also, you can require archives with some special structure. For example, all images must be in images folder, archive must have only 1 level of folders and so on.
I'm trying to generate a zip file with java.util.zip API and I haven't found any way to set zip entry as read only file. I would like to create a new ZIP archive, put files inside it and set read only flag for all those files that appear inside of the ZIP file (on Windows platform).
I am aware that java.util.zip API works with Streams instead of File objects (File object has method setReadonly()).
I also tried with Apache Commons Compress API and haven't found solution as well.
Please help!