I'm using java Jclouds to upload to a container inside an OpenStack Swift, the upload is going fine on the root, but when i pass a path(contains folders then the file), the file is uploaded but also creates another folder with the same name of the file. the original file name is 8mb.bin
the code is:
try {
ByteSource fileBytes = Files.asByteSource(file);
File file = new File(filePath);
String name = "test/test2/" + file.getName();
Blob blob = blobStore.blobBuilder(name)
.userMetadata(ImmutableMap.of("ContentType", contentType, "test", String.valueOf(test)))
.payload(fileBytes)
.contentLength(file.length())
.contentType(contentType)
.build();
///sednig the request
blobStore.putBlob(ContainerName, blob, multipart());
return contentLength;
}
and in the designated path its like this:
the folder 8mb.bin has the path inside it /slo/1522766773.076000/8200000/33554432 and then a file called 00000000 with the same size of the original file size.
how to solve this?
Thanks
jclouds implements Swift multipart using Static Large Objects. This has the limitation that parts exist in the same namespace as the manifest and modifying or deleting the parts invalidates the manifest. JCLOUDS-1285 suggests putting the parts in another container to clean up object listing although this requires some extra logic for deletes and overwrites.
Related
I'm using the code from here:
https://developers.google.com/drive/api/v3/manage-downloads#downloading_a_file
The code snippet I'm using is the following and placed in the main method:
String fileId = "some file ID";
OutputStream outputStream = new ByteArrayOutputStream();
driveService.files().get(fileId)
.executeMediaAndDownloadTo(outputStream);
I have found no sign of the code actually downloading the file, nor do I know where the file is IF it actually downloads.
I'm not sure if I am using the proper scope to gain permission to download files. I am able to upload, list, and delete files as long as I know the fileID, but downloading seems to not work.
private static final List<String> SCOPES = Collections.singletonList(DriveScopes.DRIVE);
Alternatively, I'm trying to create a method to enact the download protocol like so:
private static void downloadFile(Drive service, File file (or String fileID)){
}
but am not sure on how to do so. I've tried looking for samples online but most are from v1 or v2 apis and don't seem to work for me.
Also, I've read somewhere that it is not possible to download a Folder. Instead, I have to download each item in the folder one by one.
So do I have to make an Arraylist/list/array of the fileIDs and iterate through it after initializing a variable to represent fileID?
Edit: Some progress has been made, but I still have some problems I'm trying to thrash out.
List<File> files = result.getFiles();
File newFile;
if (files == null || files.isEmpty()) {
System.out.println("No files found.");
} else {
System.out.println("Files:");
for (File file : files) {
System.out.printf("%s (%s)\n", file.getName(), file.getId());
String fileId = file.getId();
//System.out.println(fileId);
String fileName = file.getName();
//System.out.println(fileName);
OutputStream outputstream = new FileOutputStream();
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
outputstream.flush();
outputstream.close();
}
What I want:
The above code is in the main method. I don't know if this is the proper way to do it, but as the program fetches each file and executes the System.out.printf, I also want it to download that file (with the same mimeType, and pref the same name too) into the destination set in the OutputStream constructor (C://User//some name//Downloads).
What I've tried:
From what I've tested, it only downloads the first file exactly the way I want, but only because I specify the name and extension in OutputStream. I've initialized variables 'fileId' and 'fileName' so that they will change according to the info as the program fetches the metadata for the next file, but I don't know how to change or set multiple constructors into this code:
OutputStream outputstream = new FileOutputStream();
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
to download all the files.
My folder hierarchy in Google Drive is like this:
Logs
-- bin (folder)
---- bunch of .bin files
-- .xml file
-- .xml file
You are using a ByteArrayOutputStream object as the output of your download. If your program terminates without having saved the contents of this object somewhere, you will not be able to find this information in your computer's disk, as it is not written to it but rather saved in memory as a buffered byte-array (refer to the previous link for more information).
If you want to save the output of the download to the file, I suggest you use instead a FileOutputStream as the destination of your download. In order to do that, you have to modify your code as follows:
Add the appropriate import declaration:
import java.io.FileOutputStream;
Modify your outputStream variable assignment as follows:
OutputStream outputStream = new FileOutputStream('/tmp/downloadedfile');
Where the parameter passed to FileOutputStream should be the desired destination path of your download.
After writing any contents to your file, add the following lines of code:
outputStream.flush();
outputStream.close();
This will ensure that your file is being written to properly.
In regards to downloading a folder, you are completely right - you will first need to fetch the folder you want to download, and each of their children. In order to better understand how to do it, I suggest you check out the following answer: Download folder with Google Drive API
Edit - example downloading a folder
String destinationFolder = "/tmp/downloadedfiles/";
List<File> files = result.getFiles();
File newFile;
if (files == null || files.isEmpty()) {
System.out.println("No files found.");
} else {
System.out.println("Files:");
for (File file : files) {
System.out.printf("%s (%s)\n", file.getName(), file.getId());
String fileId = file.getId();
String fileName = file.getName();
OutputStream outputstream = new FileOutputStream(destinationFolder + fileName);
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
outputstream.flush();
outputstream.close();
}
}
I have uploaded a MultipartFile which is a video, and am trying to use capture a frame of it using some code. However, when it tries to open the file, it always gives the error java.io.IOException: File Not Found.
First I extract the multipart file to a normal file like so:
File convertedFile = new File(multipartFile.getOriginalFilename());
multipartFile.transferTo(convertedFile);
Then I set the file name of the video to the code to capture the frame like this:
"file://" + convertedFile.getName()
then it fails when it tries to create a URL out of the file name:
new URL(_videoFilename));
How can I make it find the file?
I haven't worked with MultipartFile but you can find this in the Javadoc
The file contents are either stored in memory or temporarily on disk.
In either case, the user is responsible for copying file contents to a
session-level or persistent store as and if desired. The temporary
storages will be cleared at the end of request processing.
Therefore, you certainly have to usetransferTo(File dest) to be able to handle your file on a determined place.
I have Spring MVC web app running on Tomcat.
I upload a file and save it in the /tmp folder on the file system.
Then I need to show a link to that file in the view (Thymeleaf), so that the user can download the file by clicking on the link. How to do that?
I've heard about configuring Tomcat to allow a specific context to link to a folder on the FS, but not sure how to do that or if that is the only solution. Please help.
The way I approach this is slightly different. Basically I use two controller actions for handling file uploads, one for uploading, and for downloading (viewing) files.
So upload action would save files to some preconfigured directory on the file system, I assume you already have that part working.
Then declare download action similar to this
#Controller
public class FileController {
#RequestMapping("/get-file/{filename}")
public void getFileAction(#RequestParam filename, HttpServletResponse response) {
// Here check if file with given name exists in preconfigured upload folder
// If it does, write it to response's output stream and set correct response headers
// If it doesn't return 404 status code
}
}
If you want to make impossible to download file just by knowing the name, after uploading file, save some meta info to the database (or any other storage) and assign some hash (random id) to it. Then, in getFileAction, use this hash to look for file, not the original filename.
Finally, I would discourage using /tmp for file uploads. It depends on the system/application used, but generally temp directory are meant, as name suggest, for temporary data. Usually it is guaranteed data in the temp directory will stay for "reasonable time", but applications must take into account that content of temp directory can be deleted anytime.
This is the precisely setup that worked for me (Tomcat 8, SpringMVC, boot):
server.xml:
<Context docBase="C:\tmp\" path="/images" />
In the controller:
public String createNewsSource(#ModelAttribute("newsSource") NewsSource source, BindingResult result, Model model,
#RequestParam("attachment") final MultipartFile attachment) {
new NewsSourceValidator().validate(source, result);
if (result.hasErrors()) {
return "source/addNewSource";
}
if (!attachment.isEmpty()) {
try {
byte[] bytes = attachment.getBytes();
BufferedOutputStream stream = new BufferedOutputStream(
new FileOutputStream(new File("/tmp/" + attachment.getOriginalFilename())));
stream.write(bytes);
stream.close();
} catch (Exception e) {
e.printStackTrace();
}
}
source.setLogo("images/" + attachment.getOriginalFilename());
newsSourceService.createNewsSourceIfNotExist(source);
return "redirect:/sources/list";
}
As you can see I am saving the file to /tmp, but in the DB (source.setLogo()), I am pointing to images as mapped in server.xml
Here's where I found about Tomcat config:
If the images are all located outside the webapp and you want to have
Tomcat's DefaultServlet to handle them, then all you basically need to
do in Tomcat is to add the following Context element to
/conf/server.xml inside tag:
This way they'll be accessible through http://example.com/images/....
SO answer to a similar question
Fixed: Instead of calling isFile() I used exists() and it seems to be working fine. If possible could someone explain why this change worked?
I'm attempting to write out to an excel file but am having a problem when trying to create that file if the name already exists.
Basically I am taking a file that is uploaded to a server, reading it, and then outputting a report file in a new location with the same filename. I tried to do this by simply checking if the file already existed and then adding a number onto the filename. My code works if the file doesn't exist or if it exists without a number (e.g. filename.xls). If a file exists with the name "filename1.xls" the server just seems to hang when trying to write the file. What can do to fix this?
Here is my code:
String destination = "c:/apache-tomcat-7.0.8/webapps/reports/" + fileName.substring( fileName.lastIndexOf("\\")+1, fileName.lastIndexOf(".")) + ".xls";
int filenum = 1;
while (new File(destination).isFile()) {
destination = "c:/apache-tomcat-7.0.8/webapps/reports/" + fileName.substring( fileName.lastIndexOf("\\")+1, fileName.lastIndexOf(".")) + filenum + ".xls";
filenum++;
}
WritableWorkbook workbook = Workbook.createWorkbook(new File(destination));
That will happen if some process is still keeping the file open. E.g. you've created a FileInputStream on the file to read it, but are never calling close() on it after reading.
Unrelated to the problem, the expanded WAR folder is not the best place to use as a permanent storage. All those files in the expanded WAR folder will get lost whenever you redeploy the WAR. Also hardcoding a servletcontainer-specific path in the code makes it totally unportable.
If your actual intent is to return the Excel file on a per-request basis to the client using a servlet, then you should be using
WritableWorkbook workBook = Workbook.createWorkbook(response.getOutputStream());
// ...
This way it writes to the response immediately without the need for an intermediate file.
Use the File.createTempFile(prefix, suffix, directory) API:
String localName = new File(fileName).getName();
String nameNoExt = localName.substring(0, fileName.lastIndexOf("."));
String extension = localName.substring(fileName.lastIndexOf(".")); // need to include the .
File directory = new File("c:/apache-tomcat-7.0.8/webapps/reports/");
File destFile = File.createTempFile(nameNoExt, extension, directory)
I am writting the code to upload a file using Spring's MultipartFile on server for that I have written following code
if(!partnersContentBean.getFile().isEmpty()){
MultipartFile file = partnersContentBean.getFile();
if(file.getOriginalFilename().endsWith(".jpeg")||file.getOriginalFilename().endsWith(".jpg")|| file.getOriginalFilename().endsWith(".gif")){
File dirPath = new File("//125.22.60.37/image/dev/cmt/");
if (!dirPath.exists()) {
dirPath.mkdirs();
}
URL url = new URL("http://125.22.60.37/image/dev/cmt/");
File destination = new File(url.toString());
file.transferTo(destination);
String url1 = request.getContextPath() + ApplicationConstants.imageUploadDirectory + file.getOriginalFilename();
System.out.println(url.getPath());
partnersContentBean.setPartnerImagename(file.getOriginalFilename());
partnersContentBean.setPartnerImagepath(destination.getPath());
}else
{
userModuleDetailBean.put("errorMessage", "File should be in type of jpg,Jpeg or GIF");
return new ModelAndView(new RedirectView("partnersAdd_CMT.htm"),"userModuleDetailBean",userModuleDetailBean);
}
}
but when I upload a file I get following exception java.io.FileNotFoundException: http:\125.22.60.37\image\dev\cmt (The filename, directory name, or volume label syntax is incorrect) dont know what path should i give to upload it
It looks like you're trying to transfer the uploaded file to another remote server (125.22.60.37). You can't do that - you can't represent an HTTP URL using a File object.
FileUpload is for storing the uploaded files to your local machine. Once there, you can worry about moving them to another remote server, but the two tasks are separate.