how to save large files inside the data base.? - java

I am using JSP form to upload and Servlet to store a large-sized file in a MySQL database. and because i am new to Java in general, i need a really urgent help.
I tried to store the file in a String value but i got error that string index is out of boundary.
I have the database table Files which has (varchar File_Name, Blob File_data, Varchar File_Date) .
And i need to store the uploaded file inside the
I had this code in servlet:
** BTW : i have no idea why "DiskFileUpload, parseRequest " have strikethrough line ?
try {
InputStream uploadedFile = null;
<strike>DiskFileUpload</strike> fu = new </strike>DiskFileUpload</strike>();
// If file size exceeds, a FileUploadException will be thrown
fu.setSizeMax(10000000);
List fileItems = fu.<strike>parseRequest</strike>(request);
Iterator itr = fileItems.iterator();
while (itr.hasNext()) {
FileItem fi = (FileItem) itr.next();
//Check if not form field so as to only handle the file inputs
//else condition handles the submit button input
if (!fi.isFormField()) { // If the form fiel is a file
uploadedFile = fi.getInputStream();
}
What type of data should I convert uploadFile cvariable to be able to store it in a Blob attribute ?

Strike through means API is deprecated and those classes/methods may not work as expected.
Storing file to database, you need to store the content as bytes in blob filed. See this discussion. Even though it is related to some error, OP posted code on how to do this.

Related

Delete a file while exiting out of the for loop in java

I created a method that downloads a file from the web, then using a for loop I get the absolute path of that downloaded file and load it into pdfbox to perform further actions. However, I want the file to be automatically deleted when the action is performed. But the code I wrote doesn't allow me to do so. However, if I use the for loop to delete the pdf it works. Any solution?
for (File pdfs : Objects.requireNonNull(folder.listFiles())) {
String desiredPdf = pdfs.getAbsolutePath();
System.out.println(desiredPdf);
pdfs.deleteOnExit();
File baseFile = new File(desiredPdf);
PDDocument document = PDDocument.load(baseFile);
document.setAllSecurityToBeRemoved(true);
String pdfContent = new PDFTextStripper().getText(document);
System.out.println(pdfContent);
Assert.assertTrue(pdfContent.contains("Imperfection"));
}

Google Drive API not downloading FIles (Java v3)

I'm using the code from here:
https://developers.google.com/drive/api/v3/manage-downloads#downloading_a_file
The code snippet I'm using is the following and placed in the main method:
String fileId = "some file ID";
OutputStream outputStream = new ByteArrayOutputStream();
driveService.files().get(fileId)
.executeMediaAndDownloadTo(outputStream);
I have found no sign of the code actually downloading the file, nor do I know where the file is IF it actually downloads.
I'm not sure if I am using the proper scope to gain permission to download files. I am able to upload, list, and delete files as long as I know the fileID, but downloading seems to not work.
private static final List<String> SCOPES = Collections.singletonList(DriveScopes.DRIVE);
Alternatively, I'm trying to create a method to enact the download protocol like so:
private static void downloadFile(Drive service, File file (or String fileID)){
}
but am not sure on how to do so. I've tried looking for samples online but most are from v1 or v2 apis and don't seem to work for me.
Also, I've read somewhere that it is not possible to download a Folder. Instead, I have to download each item in the folder one by one.
So do I have to make an Arraylist/list/array of the fileIDs and iterate through it after initializing a variable to represent fileID?
Edit: Some progress has been made, but I still have some problems I'm trying to thrash out.
List<File> files = result.getFiles();
File newFile;
if (files == null || files.isEmpty()) {
System.out.println("No files found.");
} else {
System.out.println("Files:");
for (File file : files) {
System.out.printf("%s (%s)\n", file.getName(), file.getId());
String fileId = file.getId();
//System.out.println(fileId);
String fileName = file.getName();
//System.out.println(fileName);
OutputStream outputstream = new FileOutputStream();
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
outputstream.flush();
outputstream.close();
}
What I want:
The above code is in the main method. I don't know if this is the proper way to do it, but as the program fetches each file and executes the System.out.printf, I also want it to download that file (with the same mimeType, and pref the same name too) into the destination set in the OutputStream constructor (C://User//some name//Downloads).
What I've tried:
From what I've tested, it only downloads the first file exactly the way I want, but only because I specify the name and extension in OutputStream. I've initialized variables 'fileId' and 'fileName' so that they will change according to the info as the program fetches the metadata for the next file, but I don't know how to change or set multiple constructors into this code:
OutputStream outputstream = new FileOutputStream();
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
to download all the files.
My folder hierarchy in Google Drive is like this:
Logs
-- bin (folder)
---- bunch of .bin files
-- .xml file
-- .xml file
You are using a ByteArrayOutputStream object as the output of your download. If your program terminates without having saved the contents of this object somewhere, you will not be able to find this information in your computer's disk, as it is not written to it but rather saved in memory as a buffered byte-array (refer to the previous link for more information).
If you want to save the output of the download to the file, I suggest you use instead a FileOutputStream as the destination of your download. In order to do that, you have to modify your code as follows:
Add the appropriate import declaration:
import java.io.FileOutputStream;
Modify your outputStream variable assignment as follows:
OutputStream outputStream = new FileOutputStream('/tmp/downloadedfile');
Where the parameter passed to FileOutputStream should be the desired destination path of your download.
After writing any contents to your file, add the following lines of code:
outputStream.flush();
outputStream.close();
This will ensure that your file is being written to properly.
In regards to downloading a folder, you are completely right - you will first need to fetch the folder you want to download, and each of their children. In order to better understand how to do it, I suggest you check out the following answer: Download folder with Google Drive API
Edit - example downloading a folder
String destinationFolder = "/tmp/downloadedfiles/";
List<File> files = result.getFiles();
File newFile;
if (files == null || files.isEmpty()) {
System.out.println("No files found.");
} else {
System.out.println("Files:");
for (File file : files) {
System.out.printf("%s (%s)\n", file.getName(), file.getId());
String fileId = file.getId();
String fileName = file.getName();
OutputStream outputstream = new FileOutputStream(destinationFolder + fileName);
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
outputstream.flush();
outputstream.close();
}
}

JSP Writing to server directory

So I am using resumable.js to upload files to a server.
The directory that I want to save to is something like
/dir/files/upload/
Obviously just made up, but this directory has user permissions to write to it.
I am using JSP to listen to the POST request that resumable.js makes, and writing the
.part
files to that directory.
Sample listener:
<% if(request.getMethod().equals("POST") && request.getParameter("resumableFilename") != null){
long chunkSize = StringUtils.isEmpty(request.getParameter("resumableChunkSize"))? 0:Long.parseLong(request.getParameter("resumableChunkSize"));
String fileName = request.getParameter("resumableFilename");
long totalSize = StringUtils.isEmpty(request.getParameter("resumableTotalSize"))? 0:Long.parseLong(request.getParameter("resumableTotalSize"));
String temp_dir = "/dir/files/upload/"+request.getParameter("resumableIdentifier");//Add in user_id
String dest_dir = temp_dir+fileName+".part"+request.getParameter("resumableChunkNumber");
File fDir = new File(temp_dir);
fDir.mkdirs();
if(ServletFileUpload.isMultipartContent(request)){
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setRepository(new File(temp_dir)); ServletFileUpload upload = new ServletFileUpload(factory);
List items = upload.parseRequest(request);
ArrayListIterator iter = (ArrayListIterator)items.iterator();
FileItem item = (FileItem)iter.next();
File fileWithNewDir = new File(dest_dir);
item.write(fileWithNewDir); // write file to dest_dir (fileName.part*CHUNK_NUM*)
}
}
%>
The script is hosted on
www.site.com/pubs/res.jsp
According to the JS itself for resumable, the process of uploading it gets completed, however, a new directory is not made at all. I know it's not the write permissions, so it must be something else.
Here is my call in javascript for a new resumable object
var resume = new Resumable({
target:"res.jsp",
resumableChunkSize: 1*1024*1024,
simultaneousUploads: 3,
testChunks: false,
throttleProgressCallbacks: 1
});
It seems to be hitting the jsp file, but nothing is happening.
I followed Apache's fileupload page in order to implement that listener, but maybe I went wrong at some point.
Apache's FileUpload
Resumable.js
Location of the directory matters. It has to be within the context of the WAR. You cannot write to any location outside the context of the container. If you look at the log you may be abe to see the error message which can explain this.

Convert binary contents in BLOB field in the database to files mySQL

I have got to retrieve binary in LONGBLOB field from the db. This field is storing all sorts of file formats such as txt, doc, xdoc, pdf, etc. I basically need to be able to convert the binary format into their actual file formats in order to allow my user to download these files.
Has anyone got any idea how to do this?
As others have said, it would be best have another field to store the format. You can do this by copying the extension (ie, everything after the last "." in the file name). The best way would probably be to get the file's mime type: see this for example.
You can then store the mime type in a field in the database. This will almost always work, whereas the extension of a file can be misleading (or vague).
Adding a field file_format indicating the file format of the file stored in LONGBLOB, then you are able to convert the binary according to the associated file format.
or, reserve the first several bytes for file format, after that is the actual content of file.
I think u should have another field to save the document type and tell which type should be converted. Use I/O InputStream to read/write file.
What I recommend is upload the client files to somewhere, save the path that's link to these file into db. That should be faster.
as others have suggested. create another column(FILETYPE_COL_NAME) in database which will tell you which type of file is stored in BOLB(BLOB_COL_NAME) field and then you extend the below code in try/catch block
String fileType = rs.getString("FILETYPE_COL_NAME");
DataOutputStream dos = new DataOutputStream(new BufferedOutputStream(new FileOutputStream("filepath."+fileType )));
ResultSet rs = statement.executeQuery("Select * from tablename");
BufferedInputStream bis = new BufferedInputStream(rs.getBinaryStream("BLOB_COL_NAME"));
byte[] buffer = new byte[1024];
int byteread = 0;
while((byteread = bis.read(buffer)) != -1){
dos.write(buffer, 0, byteread);
}
dos.flush();
dos.close();
bis.close();

multipart/form-data not working with servlets

I am not really sure why the html form with the tag enctype="multipart/form-data" does not pass the objects it should pass. this is the case with mozilla and firefox.
for IEs case, for example, I use html control to choose a file, it does get what it is supposed to get.
Now I just want to know if there are any alternatives I can use to pass in files through http request object because the enctype="multipart/form-data" seems to be having some compatibility issues but I am not really sure
any suggestions would be appreciated! :D
First of all, you have to provide a little bit of code to show what you have done, and to know what went wrong. Anyway, I am assuming that you have to upload a file to the server using HTML file upload control.
File upload or to say multipart/form-data encoding type support is not implemented in HttpServlet implementation. So, the request.getParameter() don't work with multipart/form-data. You have to use additional libraries which provide support for this. Apache Commons File Upload is a good example. Their using fileupload guide will help you to get started with the library. Here is a simple example (compiled from using file upload guide).
// Check that we have a file upload request
boolean isMultipart = ServletFileUpload.isMultipartContent(request);
if (isMultipart) {
// Create a factory for disk-based file items
FileItemFactory factory = new DiskFileItemFactory();
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload(factory);
// Parse the request
List /* FileItem */ items = upload.parseRequest(request);
// Process the uploaded items
Iterator iter = items.iterator();
while (iter.hasNext()) {
FileItem item = (FileItem) iter.next();
if (item.isFormField()) {
// Process form field.
String name = item.getFieldName();
String value = item.getString();
} else {
// Process uploaded file.
String fieldName = item.getFieldName();
String fileName = item.getName();
String contentType = item.getContentType();
boolean isInMemory = item.isInMemory();
long sizeInBytes = item.getSize();
if (writeToFile) {
File uploadedFile = new File("path/filename.txt");
item.write(uploadedFile);
}
}
}
} else {
// Normal request. request.getParameter will suffice.
}
got it. if someone else might have a problem like this, this was my problem. i checked the content type of the file for me to ensure that the objects passed is of a certain type. in IE, it returns application/x-zip-compressed THAT IS ONLY FOR IE but mozilla and chrome seems to be returning a different content type for the zip file which is application/octet-stream.
so i just added the application/octet-stream to the valid filetypes and it seems to be working now

Categories