I am currently storing and downloading my Thymeleaf templates in S3.
I am using the following function to retrieve the Template from S3:
public String getTemplateFile(String name, File localFile) {
ObjectMetadata object = s3Client.getObject(new GetObjectRequest(connectionProperties.getBucket(), name), localFile);
boolean success = localFile.exists() && localFile.canRead();
return localFile.getPath();
}
After doing this the file is successfully downloaded in the desired location.
But when trying to access the file from the FlyingSaucer PDF generator the file doesn't exist, despite it is already downloaded in FILE_LOCATION_PATH. (I can open the file... the file is there but the function doesn't see it)
String xHtmlStringDocument =
convertHtmlToXhtml(templateEngine
.process(FILE_LOCATION_PATH,
initializeLetterHtmlTemplateContext(letter)));
When I run the program again and again I get the same result. But when I STOP the program and RUN it AGAIN then everything works because the file form the last execution is now recognized by the program.
This sounds to me like an asynchronous function issue.
Does anybody know how can I fix this?
Thanks in advance.
EDITED (following suggestion)
New function: Same result:
(And the file was created, the Download from S3 was successful)
java.io.FileNotFoundException: ClassLoader resource "static/templates/template.html" could not be resolved
public String getTemplateFileN(String name, File localFile) throws IOException {
S3Object fullObject = null;
InputStream in = null;
try {
fullObject = s3Client.getObject(new GetObjectRequest(connectionProperties.getBucket(), name));
System.out.println("Content-Type: " + fullObject.getObjectMetadata().getContentType());
System.out.println("Content: ");
displayTextInputStream(fullObject.getObjectContent());
in = fullObject.getObjectContent();
System.out.println(localFile.toPath());
Files.copy(in, localFile.toPath());
} //then later
finally {
// To ensure that the network connection doesn't remain open, close any open input streams.
if (fullObject != null) {
fullObject.close();
}
if (in != null) {
in.close();
}
}
return localFile.getPath();
}
Checking javadoc
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#getObject-com.amazonaws.services.s3.model.GetObjectRequest-java.io.File-
I see not method signature ObjectMetadata getObject(GetObjectRequest getObjectRequest,String file)
There is
ObjectMetadata getObject(GetObjectRequest getObjectRequest,
File destinationFile)
Where you provide File (not String) as second argument. Make sure the file is not opened for write before you try reading it!
Related
Can someone explain what is the difference between the following two calls to ServletContext getRealPath() in Tomcat:
context.getRealPath("/") + "\\songModified.wav";
context.getRealPath("/" + "\\songModified.wav");
I have a very simple GET method on the server which reads a file on the server and copies the bytes into a new file in the location returned by the above call.
On the client side I have an audio tag that references an audio file on the server, calls this method that creates a new file and changes the reference of the audio tag to this new file. The thing is that in the javascript callback this new file is not immediately referenceable if I store the file to the path that is returned from the second case of the above getRealPath call. Basically it returns a 404. If I store it to the returned path of the first case of the call then it is immediately referenceable and the audio tag normaly references the new file.
Both of those calls to getRealPath() return exactly the same string:
C:\Users\Mihael\apache-tomcat-9.0.31\wtpwebapps\AudioSimulator\songModified.wav
I am passing this returned string to the FileOutputStream constructor further in the code.
Thing to note here is that this file does not exist at the moment of the getRealPath() call so I am confused why is it returning anything at all in the second case of the call.
I know this is not the recommended way of storing files so I am asking from a purely educational perspective. How can the second call to this method break my functionality if they both return exactly the same string to the rest of the code?
EDIT:
Here is a very simple Javascript and Java code for anyone who wants to test this.
Javascript:
<body>
<script>
function modifyRequest() {
var xhttp = new XMLHttpRequest();
xhttp.onload = function() {
var audio = document.getElementById("player");
var currentTime = audio.currentTime;
audio.src = "http://localhost:8080/AudioSimulator/bluesModified.wav";
audio.currentTime = currentTime;
audio.play();
};
xhttp.open("GET", "http://localhost:8080/AudioSimulator/rest/Test/testPath");
xhttp.send();
}
</script>
<audio id="player" src="http://localhost:8080/AudioSimulator/blues.wav"
controls>
Your browser does not support the
<code>audio</code> element.
</audio>
<button onclick="modifyRequest()">Test</button>
</body>
Java:
#Path("/Test")
public class Test {
#Context
ServletContext context;
#GET
#Path("/testPath")
public Response testPath() {
File fileIn = new File(context.getRealPath("/") + "\\blues.wav");
File fileOut = new File(context.getRealPath("/" + "\\bluesModified.wav"));
//if i write it like this it would work
//File fileOut = new File(context.getRealPath("/") + "\\bluesModified.wav");
FileInputStream fis = null;
FileOutputStream fos = null;
try {
fis = new FileInputStream(fileIn);
fos = new FileOutputStream(fileOut);
byte[] inArray = new byte[(int) fileIn.length()];
try {
fis.read(inArray);
fos.write(inArray);
} catch (IOException e) {
e.printStackTrace();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
try {
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return Response
.ok()
.entity("Success")
.header("Access-Control-Allow-Origin", "null")
.build();
}
}
I have taken the time to dive into Tomcat source to find the cause for this. It turns out that getRealPath, in addition to retrieving the system path for a given virtual path, also works a bit with the Tomcat cache.
NOTE:
I know that my file separator usage is not good, but Tomcat is smart enough to validate the above call to produce /bluesModified.wav. So even if I call it like #rickz mentioned in the comments, the result would be the same and therefore that was not the issue.
The issues I had with being unable to reference the file in the case of the following call
context.getRealPath("/" + "\\bluesModified.wav")
was the fact that in this case we are passing the file path to the method, while in the case that works we are passing in the directory path.
What happens is that the call to getRealPath() first checks the cache for the existence of the resource identified by the webapppath /bluesModified.wav. Since it does not exist at the moment of the call, Tomcat will create an instance of the EmptyResource class which is basically a wrapper around File class and represents a file that does not exist, and it will then store the reference to this file in its cache.
The issue here is that even though I create a file that will have the correct virtual path Tomcat will still have that empty resource representing a non existent file in its cache. In other words, if I reference the file from the client side like so
http://localhost:8080/AudioSimulator/bluesModified.wav
Tomcat will return the cached resource that represents the empty file, which actually means a 404 to the client even though the file exists.
Waiting for 5 seconds, which is the time to live of Tomcat cache entries, and then trying to reference the file will revalidate the cache entry and produce a FileResource instead of EmptyResource in which case the referencing will work normally.
It works in this case
context.getRealPath("/") + "\\bluesModified.wav"
since the path that is getting cached is a directory and the file name is simply concatenated. So the string I have here is just an absolute path to the file I am going to create with no cache entries colliding with it.
My mistake was assuming that getRealPath() is just some "pure" method that will return a string I can use to create files while in fact it has a bit of side effects. These side effects are not documented and even though I might have done some things incorrectly the bottom line is this method is not that predictable to use when doing File IO stuff.
The String returned by getRealPath from the ServletContext implementation is normalized.
So when you call getRealPath("/") + "\blues.wav") only the String "/" is normalized, and the String concatenation "\blues.wav" is not.
But when you call getRealPath("/" + "\blues.wav")) the full concatened String is normilized.
public String getRealPath(String path) {
if ("".equals(path)) {
path = "/";
}
if (this.resources != null) {
try {
WebResource resource = this.resources.getResource(path);
String canonicalPath = resource.getCanonicalPath();
if (canonicalPath == null) {
return null;
}
if ((resource.isDirectory() && !canonicalPath.endsWith(File.separator) || !resource.exists()) && path.endsWith("/")) {
return canonicalPath + File.separatorChar;
}
return canonicalPath;
} catch (IllegalArgumentException var4) {
}
}
return null;
}
You can see WebResource resource = this.resources.getResource(path) will try to validate your path and will return a validated path :
private String validate(String path) {
if (!this.getState().isAvailable()) {
throw new IllegalStateException(sm.getString("standardRoot.checkStateNotStarted"));
} else if (path != null && path.length() != 0 && path.startsWith("/")) {
String result;
if (File.separatorChar == '\\') {
result = RequestUtil.normalize(path, true);
} else {
result = RequestUtil.normalize(path, false);
}
if (result != null && result.length() != 0 && result.startsWith("/")) {
return result;
} else {
throw new IllegalArgumentException(sm.getString("standardRoot.invalidPathNormal", new Object[]{path, result}));
}
} else {
throw new IllegalArgumentException(sm.getString("standardRoot.invalidPath", new Object[]{path}));
}
}
Drive Quickstart: Run a Drive App in Java example works for uploading files fine. I want to download the files from Gdrive to local system by using java.
For download they are given a method
private static InputStream downloadFile(Drive service, File file) {
if (file.getDownloadUrl() != null && file.getDownloadUrl().length() > 0) {
try {
HttpResponse resp =
service.getRequestFactory().buildGetRequest(new GenericUrl(file.getDownloadUrl())).execute();
return resp.getContent();
} catch (IOException e) {
// An error occurred.
e.printStackTrace();
return null;
}
} else {
// The file doesn't have any content stored on Drive.
return null;
}
}
The above method,how can i give inputs? and from where i give the inputs? Can anyone give a complete code for download like Quickstart upload class.
any help will be appreciated.
you can use google drive api and send Http get request, you can see this tutorial
https://developers.google.com/drive/manage-downloads
Thanks Hanan it works fine.By using the retrieveAllFiles() i can list all the documents.And i have stored the retrieved documents in my local by using this below code.Is it a correct way to download.
for(File f:result){
i++;
System.out.println("File Name==>"+f.getTitle());
System.out.println("File Id==>"+f.getId());
System.out.println("File ext==>"+f.getFileExtension());
System.out.println("File size==>"+f.getFileSize());
InputStream in = downloadFile(service,f);
byte b[] = new byte[in.available()];
in.read(b);
java.io.File ff = new java.io.File("/home/test/Desktop/gdocs/"+f.getTitle());
FileOutputStream fout = new FileOutputStream(ff);
fout.write(b);
fout.close();
}
It stores all the documents in local. The text (.txt) files are open properly in my local, but the image files or pdf files are not open properly.It gives some error messages like file corrupted. There is no content in the image or pdf documents how can i get content and store it. Any suggestions
I am familiar with AWS Java SDK, I also tried to browse the corresponding Javadoc, but I could not realize how do I create a sub directory, i.e., a directory object within a bucket, and how do I upload files to it.
Assume bucketName and dirName correspond to already existing bucket (with public permission) and a new (object) directory which needs to be created within the bucket (i.e. bucketName/dirName/)
I have tried the following:
AmazonS3Client s3 = new AmazonS3Client(
new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY));
s3.createBucket(bucketName + "/" + dirName); //throws exception
which throws an exception on the second line.
A short snippet which creates a sub-directory and uploads files to it will be deeply appreciated.
There are no "sub-directories" in S3. There are buckets and there are keys within buckets.
You can emulate traditional directories by using prefix searches. For example, you can store the following keys in a bucket:
foo/bar1
foo/bar2
foo/bar3
blah/baz1
blah/baz2
and then do a prefix search for foo/ and you will get back:
foo/bar1
foo/bar2
foo/bar3
See AmazonS3.listObjects for more details.
Update: Assuming you already have an existing bucket, the key under that bucket would contain the /:
s3.putObject("someBucket", "foo/bar1", file1);
s3.putObject("someBucket", "foo/bar2", file2);
...
Then you can list all keys starting with foo/:
ObjectListing listing = s3.listObjects("someBucket", "foo/");
S3 doesn't see directories in the traditional way we do this on our operating systems.
Here is how you can create a directory:
public static void createFolder(String bucketName, String folderName, AmazonS3 client) {
// create meta-data for your folder and set content-length to 0
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(0);
// create empty content
InputStream emptyContent = new ByteArrayInputStream(new byte[0]);
// create a PutObjectRequest passing the folder name suffixed by /
PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName,
folderName + SUFFIX, emptyContent, metadata);
// send request to S3 to create folder
client.putObject(putObjectRequest);
}
As casablanca already said you can upload files to directories like this:
s3.putObject("someBucket", "foo/bar1", file1);
Read the whole tutorial here for details, and the most important thing is you will find info how to delete the directories.
In newer versions of the SDK, you can do something like this (no need to create empty InputStream) to create an empty folder:
String key = parentKey + newFolderName;
if (!StringUtils.endsWith(key, "/")) {
key += "/";
}
PutObjectRequest putRequest = PutObjectRequest.builder()
.bucket(parent.getBucket())
.key(key)
.acl("public-read")
.build();
s3Client.putObject(putRequest, RequestBody.empty());
Leaving this answer here just in case someone stumbles upon this. I have been using aws sdk version - 1.11.875 and the following successfully created a folder for me when trying to upload a file into S3 bucket. I did not have to explicitly create the folder as mentioned in the earlier answer.
private void uploadFileToS3Bucket(final String bucketName, final File file) {
final String fileName = "parent/child/" + file.getName();
final PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, fileName, file);
amazonS3.putObject(putObjectRequest);
}
This will create the parent and parent/child folders in the specified S3 bucket and upload the file into child folder.
This worked for me. I used spring boot and file uploaded according to Multipart mechanism. I wanted to save my images inside the photos folder in my aws s3 bucket. My need is save like this photos/mypic.jpg
----controller class method----
#PostMapping("/uploadFile")
public String uploadFile(#RequestPart(value = "file") MultipartFile file) throws IOException {
return this.amazonClient.uploadFile(file);
}
----service class (Implementation of controller)----
public String uploadFile(MultipartFile multipartFile) throws IOException {
try {
File file = convertMultiPartToFile(multipartFile);
String fileName = "photoes/"+generateFileName(multipartFile); //here give any folder name you want
uploadFileTos3bucket(fileName, file);
} catch (AmazonServiceException ase) {
logger.info("Caught an AmazonServiceException from GET requests, rejected reasons:");
}
return fileName;
}
The point is concatenate the folder name you want as prefix of the fileName
additionally I will show you how to delete folder. The point is give the folder name as the keyName(key name is uploaded object name in the s3 bucket.). I will show code snippet also.
----controller class method----
#DeleteMapping("/deleteFile")
public String deleteFile(#RequestPart(value = "keyName") String keyName) {
return this.amazonClient.deleteFile(keyName);
}
----service class (Implementation of controller)----
public String deleteFile(String keyName){
try {
s3client.deleteObject(new DeleteObjectRequest(bucketName, keyName));
} catch (SdkClientException e) {
e.printStackTrace();
}
return "deleted file successfully!";
}
for delete photos folder that we created , call method like this. deleteFile("photos/")
important:- / is mandatory
if You want to create folder then you need to use put command using following keys to create folder1 in:
in root of bucket -> folder1/folder1_$folder$
in path folder2/folder3/ -> folder2/folder3/folder1/folder1_$folder$
It is always all_previous_folders/folderName/folderName_$folder$
Im trying to access the example/web folder (see below in the image) in a jsf managed bean but cant seem to find a way to do it
thx
Try
FacesContext.getCurrentInstance().getExternalContext().getRequestContextPath()
for build relative url's to resources in your app.
If you want the real path...
ServletContext ctx = (ServletContext) FacesContext.getCurrentInstance()
.getExternalContext().getContext();
String realPath = ctx.getRealPath("/");
If you want to get it as a File for some reason, then you need ExternalContext#getRealPath(). This converts a relative web path to an absolute disk file system. Since you need the web's root folder, just pass in /:
String absoluteWebPath = externalContext.getRealPath("/");
File webRoot = new File(absoluteWebPath);
// ...
Unrelated to the concrete problem, whatever functional requirement you've had in mind for which you thought that having an absolute local disk file system path to the web folder is the right solution, it has most definitely to be solved differently. And indeed, as per your comment on the other answer,
because Im trying to upload some file inside the folder and using the relative path
you're going the wrong path. You should not store uploaded files in there if you intend to keep them longer than the webapp's deployment lifetime. Whenever you redeploy the webapp (and on some server configs even when you restart the server), the uploaded files would get completely lost, simply because they are not contained as part of the original WAR file. Even more, some heavy server configs don't expand the WAR on disk at all, but in memory instead, the getRealPath() would then always return null.
Rather store it in a fixed disk file system path outside the server's deploy folder. Add that path in turn as a new server context or docroot, so that it's accessible on a different (virtual) context path. Or homegrow a servlet which gets an InputStream of it from disk and writes it to OutputStream of the response. See also this related answer: Uploaded image only available after refreshing the page
Try:
String relativePath="/resources/temp/";
String absolutePath= FacesContext.getCurrentInstance.getExternalContext().getRealPath(relativePath);
File file = new File(absolutePath);
to get real path.
Create a tmp file in resources/temp/ to avoid any exception.
Just wanted to thank Balus C. Code Java with JSP, in Tomcat/Tomee server I the following code that works:
private Boolean SaveUserItemImage(Part ui, String bid) throws IOException {
Boolean fileCreate = false;
OutputStream out = null;
InputStream filecontent = null;
ExternalContext ctx = context().getExternalContext();
String absoluteWebPath = ctx.getRealPath("/");
String resource_path = absoluteWebPath + "\\resources\\";
String image_path = resource_path + "\\" + this.itemType + "_images\\";
String buildFileName = image_path + bid + "_" + getFileName(ui);
File files = null;
try {
files = new File(buildFileName);
fileCreate = true;
} catch (Exception ex) {
System.out.println("Error in Creating New File");
Logger.getLogger(ItemBean.class.getName()).log(Level.SEVERE, null, ex);
}
if (fileCreate == true) {
if (files.exists()) {
/// User may be using same image file name but has been editted
files.delete();
}
try {
out = new FileOutputStream(files);
filecontent = ui.getInputStream();
int read = 0;
final byte[] bytes = new byte[1024];
while ((read = filecontent.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
fileCreate = true;
} catch (FileNotFoundException fne) {
fileCreate = false;
Logger.getLogger(ItemBean.class.getName()).log(Level.SEVERE, "SaveUserItemImage", fne);
} finally {
if (out != null) {
out.close();
}
if (filecontent != null) {
filecontent.close();
}
files = null;
}
}
return fileCreate;
}
My program has a function that read/write file from resource. This function I have tested smoothly.
For example, I write something to file, restart and loading again, I can read that data again.
But after I export to jar file, I faced problems when write file. Here is my code to write file:
URL resourceUrl = getClass().getResource("/resource/data.sav");
File file = new File(resourceUrl.toURI());
FileOutputStream output = new FileOutputStream(file);
ObjectOutputStream writer = new ObjectOutputStream( output);
When this code run, I has notice error in Command Prompt:
So, My data cannot saved. (I know it because after I restarted app, nothing changed !!!)
Please help me solve this problem.
Thanks :)
You simply can't write files into a jar file this way. The URI you get from getResource() isn't a file:/// URI, and it can't be passed to java.io.File's constructor. The only way to write a zip file is by using the classes in java.util.zip that are designed for this purpose, and those classes are designed to let you write entire jar files, not stream data to a single file inside of one. In a real installation, the user may not even have permission to write to the jar file, anyway.
You're going to need to save your data into a real file on the file system, or possibly, if it's small enough, by using the preferences API.
You need to read/write file as an input stream to read from jar file.
public static String getValue(String key)
{
String _value = null;
try
{
InputStream loadedFile = ConfigReader.class.getClassLoader().getResourceAsStream(configFileName);
if(loadedFile == null) throw new Exception("Error: Could not load the file as a stream!");
props.load(loadedFile);
}
catch(Exception ex){
try {
System.out.println(ex.getMessage());
props.load(new FileInputStream(configFileName));
} catch (FileNotFoundException e) {
ExceptionWriter.LogException(e);
} catch (IOException e) {
ExceptionWriter.LogException(e);
}
}
_value = props.getProperty(key);
if(_value == null || _value.equals("")) System.out.println("Null value supplied for key: "+key);
return _value;
}