We Keep Coding

Transferring files from AWS lambda tmp folder to sftp server

I have created a AWS lambda function that takes some files from an S3 bucket, zips them and transfers the zipped file to a sftp server. When I look in the server, I see that the tmp folder has been carries over with the files and a tmp folder gets created inside the zip file. When I open the zip file, there is a tmp folder and inside that folder are the files that I had zipped. I have scoured the internet and AWS trying to figure out how to change the directory in AWS Lambda when I am retrieving the files to be zipped, but have not had any luck. I don't want to have a tmp folder in my zip file. When I unzip the zip file, I just want to see the files that I had selected to be zipped without any folders. Does anyone know how to do this? I am programming in Java.
My code is below.
private DownloadFile(){
File localFile = new File(fileName);
//pull data and audit files from s3 bucket
s3Client.getObject(new GetObjectRequest("pie-dd-demo/daniel20", fileName), localFile);
zipOS = new ZipOutputStream(fos);
//send files to be zipped
writeToZipFile(fileName, zipOS);
}
public static void writeToZipFile(String path, ZipOutputStream zipStream)
throws FileNotFoundException, IOException {
File aFile = new File(path);
FileInputStream fis = new FileInputStream(aFile);
ZipEntry zipEntry = new ZipEntry(path);
try {
zipStream.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while ((length = fis.read(bytes)) >= 0) {
zipStream.write(bytes, 0, length);
System.out.println(path + "write to zipfile complete");
}
} catch (FileNotFoundException exception) {
// Output expected FileNotFoundExceptions.
} catch (Exception exception) {
// Output unexpected Exceptions.
}
zipStream.closeEntry();
fis.close();
}

I think the problem is that you are creating a zip entry using new ZipEntry(path) and that means that the resulting zip file will contain the full path as the name of the zip entry.
You can retrieve the actual filename from a full path/file in Java as follows:
File f = new File("/tmp/folder/cat.png");
String fname = f.getName();
You can then use fname to create the zip entry by calling new ZipEntry(fname).

Copy files from one directory to another and append new files with timestamp instead of overwriting in Java

I want to copy files from source directory to destination. If the file already exists in the destination directory, then append the new file to be copied with its timestamp so that there is no overwrite. How do I check for duplicates and append timestamp to the new file name? Please help!
public static void copyFolder(File src, File dest)
throws IOException{
//list all the directory contents
String files[] = src.list();
for (String file : files) {
//construct the src and dest file structure
File srcFile = new File(src, file);
File destFile = new File(dest, file);
//recursive copy
copyFolder(srcFile,destFile);
}
}else{
//if file, then copy it
//Use bytes stream to support all file types
InputStream in = new FileInputStream(src);
OutputStream out = new FileOutputStream(dest);
byte[] buffer = new byte[1024];
int length;
//copy the file content in bytes
while ((length = in.read(buffer)) > 0){
out.write(buffer, 0, length);
}
in.close();
out.close();
System.out.println("File copied from " + src + " to " + dest);
}
}

//construct the src and dest file structure
File srcFile = new File(src, file);
File destFile = new File(dest, file);
while (destFile.exists()) {
destFile = new File(dest, file + '-' + Instant.now());
}
In one case the destination file got named test-file.txt-2018-03-14T11:05:21.103706Z. The time given is in UTC. In any case you will end up with a name of file that doesn’t already exist (if the loop terminates, but I have a hard time seeing the scenario where it doesn’t).
You may want to append the timestamp only to plain files and reuse existing folders (directories), I don’t know your requirements here. And you may want to append the timestamp before the extension if there is one (to get test-file-2018-03-14T11:05:21.103706Z.txt instead). I trust you to make the necessary modifications.

You can check if the file exists using File.exist() method, if exists, you can open the file in the append mode
The code is something like this
File f = new File(oldName);
if(f.exists() && !f.isDirectory()) {
long currentTime=System.currentTimeMillis();
String newName=oldName+currentTime;
// do the copy
}

Add suffix to filename if file already exist instead of overwriting it

I'm saving an uploaded file as below:
UploadItem item = event.getUploadItem();
File dir = new File("D:/FileUpload");
if (!dir.exists()) {
dir.mkdir();
}
File bfile = new File("D:/FileUpload" + "/" + item.getFileName());
OutputStream outStream = new FileOutputStream(bfile);
outStream.write(item.getData());
outStream.close();
But my question is when upload once file same old file in folder D:/FileUpload. In above function it will delete old file. Example first time, i upload file : test.doc (old file). Then i upload another file with same name : test.doc (new file). At folder FileUpload will has one file is test.doc (new file). I want function will process similar in window OS is : new file will be test (2).doc. How can i process it ? And all cases : D:/FileUpload have many file : test.doc, test (1).doc, test (2).doc, test (a).doc,...... I think we just check with format ....(int).doc. That new file will be :
test (3).doc (ignore test(a).doc)

Maybe you're looking for something like this? I list the files in your directory, compare the names of each to the name of your file. If the names match, increment a count. Then, when you come to create your file, include the count in its name.
UploadItem item = event.getUploadItem();
File dir = new File("D:/FileUpload");
if (!dir.exists()) {
dir.mkdir();
}
String [] files = dir.list();
int count = 0;
for(String file : files) {
if (file.startsWith(item.getFileName()) {
count++;
}
}
File bfile = new File("D:/FileUpload" + "/" + item.getFileName() + "(" + count + ")");
OutputStream outStream = new FileOutputStream(bfile);
outStream.write(item.getData());
outStream.close();

java.util.zip has problems with periods in filenames / directorynames?

I want to unzip an iPhone app .ipa file.
This is actually zip file that extracts normally.
But the actual app file in it is a folder with the ending .app ( as all mac applications are actually folders with the ending .app).
Now the period seems to be a problem for java.util.zip.
public static void main(String[] args) throws IOException {
ZipFile zipFile = new ZipFile("file.zip");
String path = "";
Enumeration files = zipFile.entries();
while (files.hasMoreElements()) {
ZipEntry entry = (ZipEntry) files.nextElement();
if (entry.isDirectory()) {
File file = new File(path + entry.getName());
file.mkdir();
System.out.println("Create dir " + entry.getName());
} else {
File f = new File(entry.getName());
FileOutputStream fos = new FileOutputStream(f); //EXception occurs here
InputStream is = zipFile.getInputStream(entry);
byte[] buffer = new byte[1024];
int bytesRead = 0;
while ((bytesRead = is.read(buffer)) != -1) {
fos.write(buffer, 0, bytesRead);
}
fos.close();
System.out.println("Create File " + entry.getName());
}
}
}
This is my output:
Exception in thread "main" java.io.FileNotFoundException: Payload/SMA Jobs.app/06-magnifying-glass.png (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
at Main.main(Main.java:27)
enter code here
Anyone knows how to handle those periods?

First of all, you should use mkdirs(), not mkdir().
second, zip files don't always include all the directory entries (or have them in the right order). the best practice is to make the directories in both branches of the code, so add:
} else {
File f = new File(entry.getName());
f.getParent().mkdirs();
(you should add some checking to make sure getParent() is not null, etc).

I don't think the period is the problem. Look at the absolute path of the file you are trying to output and make sure it is pointing to the correct place.

if (entry.isDirectory()) {
File file = new File(path + entry.getName());
....
} else {
File f = new File(entry.getName());
....
While creating directory, file path passed is path + entry.getName()
but while creating file, file path passed is entry.getName()
After changing file path to path + entry.getName(), code works for period file names and normal file names. :)

File Path appearing in the Zipped File

I have a java program as below for zipping a folder as a whole.
public static void zipDir(String dir2zip, ZipOutputStream zos)
{
try
{
File zipDir= new File(dir2zip);
String[] dirList = zipDir.list();
byte[] readBuffer = new byte[2156];
int bytesIn = 0;
for(int i=0; i<dirList.length; i++)
{
File f = new File(zipDir, dirList[i]);
if(f.isDirectory())
{
String filePath = f.getPath();
zipDir(filePath, zos);
continue;
}
FileInputStream fis = new FileInputStream(f);
ZipEntry anEntry = new ZipEntry(f.getPath());
zos.putNextEntry(anEntry);
while((bytesIn = fis.read(readBuffer)) != -1)
{
zos.write(readBuffer, 0, bytesIn);
}
fis.close();
}
}
catch(Exception e)
{
e.printStackTrace();
}
}
public static void main(){
String date=new java.text.SimpleDateFormat("MM-dd-yyyy").format(new java.util.Date());
ZipOutputStream zos = new ZipOutputStream(new FileOutputStream("Output/" + date + "_RB" + ".zip"));
zipDir("Output/" + date + "_RB", zos);
zos.close();
}
My query here is. The target folder(+date+_RB) to be zipped is present inside the folder named Output. After successful zipping, when I extract the zipped file, I find a folder Output inside which the (+date+_RB) required folder is present. I need not want that Output folder after the extraction of the zipped file, rather it should directly extract the required folder alone. Please advise on the same.
UPDATE:
I tried Isaac's answer. While extracting the resultant zip file, no folders are getting extracted. Only the files inside all the folders are getting extracted. I just dont need the folder "Output" alone in the resultant zip file. But what the program does is, it doesnt extracts all other folders inside the Output folder, rather it just extracts the files inside those folders. Kindly advise on how to proceed...

It happens because of this:
ZipEntry anEntry = new ZipEntry(f.getPath());
f.getPath() will return Output/ at the beginning of the string. This is due to the flow of your program and how it (mis)uses File objects.
I suggest you construct a File object called, say, tmp:
File tmp = new File(dirList[i]);
The change the construction of f:
File f = new File(zipDir, tmp.getPath());
Then, change this:
ZipEntry anEntry = new ZipEntry(f.getPath());
To this:
ZipEntry anEntry = new ZipEntry(tmp.getPath());
I didn't have time to actually test it, but in a nutshell, your problem is due to how the File object is constructed.