Environment: Ubuntu 18 + Spring boot 2.0 + JDK 11
It performs correctly when the application runs in IntelliJ, but it doesn't when reading the file after Spring boot maven plugin packaging file.
PS: The file indeed could be found in the packaged jar file!
java.io.FileNotFoundException: class path resource [jmxremote.password]
cannot be resolved to absolute file path because it does not reside in the
file system: jar:file:/home/XXX/YYY/target/YYY-1.0-Final.jar!/BOOT-
INF/classes!/jmxremote.password
You are most likely attempting to use java.io classes to load a class path resource. This won't work, the java.io classes are designed only for file system operations.
Either extract jmxremote.password outside of the JAR or use the regular InputStream to access it as a class path resource e.g. by using Class.getResourceAsStream():
try (InputStream in : getClass().getResourceAsStream("/jmxremote.password")) {
// read input
}
TL;DR
In the Spring Framework environment focus on using Spring utility tooling for handling resources (like the ResourceUtils class) which nicely encapsulate lower-level, operating system dependent IO operations. The ResourceUtils already contains multiple traps to figure out if the project you are running is exploded (run in IDE) or packaged (inside a JAR file).
The answer provided by Karol seems the easiest and is relatively bullet-proof, until you need a certain level of flexibility to specify file location (inside a jar file but with the possibility to define it externally and provide somewhere in the file system). Then the approach with getResourceAsStream() method won't work.
Standard Java IO (java.nio) is using FileSystemProvider classes to delegate IO operations (like creating, reading and deleting files).
A provider is identified by a URI scheme. The default provider is identified by the URI scheme "file." It creates the FileSystem that provides access to the file systems accessible to the Java virtual machine. The FileSystems class defines how file system providers are located and loaded.
So, if your file is somewhere on the filesystem, there are no issues, and everything works fine. Technically, the URL returned by Application.class.getResource("").toURI() starts with file:// and contains a valid filesystem path.
Having said that, when your file "land" inside a jar file, the Application.class.getResource("").toURI() returns something more like file://{jar-location}!/ (mind the exclamation mark), which is not a valid file schema path and Java doesn't know how to handle it. An additional file system provider needs registering.
FileSystems.newFileSystem(uri, emptyMap());
Java figures out (based on URI) the scheme and registers a new file system. From now on, standard java.nio file operations can be used.
For example, if you have some files in a /webapp folder which can (but don't need to) be inside a jar file and you'd like to list them.
// Load zip specific filesystem provider when run from inside a fat-jar
URI uri = Application.class.getResource("").toURI();
if (uri.toString().contains("!")) {
FileSystems.newFileSystem(uri, emptyMap());
}
URI rootFolder = Application.class.getResource("/webapp").toURI();
List<Path> banners = Files.list(Paths.get(rootFolder))
.collect(Collectors.toList());
Random rand = new Random();
Path path = banners.get(rand.nextInt(banners.size()));
log.info("Random image: {}", path.getFileName());
byte[] bytes = Files.readAllBytes(path);
Installation of a new file system provider is global and should be done only once.
Related
Context: I successfully parse a json file while debugging/running locally in my Windows PC that way
import com.fasterxml.jackson.core.JsonFactory
import com.fasterxml.jackson.core.JsonParser
import java.io.File
import java.nio.file.Paths
...
val basePath = Paths.get("").toAbsolutePath().toString()
val parser: JsonParser = JsonFactory().createParser(File("$basePath/app/src/main/resources/support/certain-file.json"))
When I deployed it to either docker or AWS I get this error
/app/src/main/resources/support/certain-file.json (No such file or directory)
Is there a more reliable way to "navigate" to a file placed in resources folder? I mean someway it wouldn't be affected either it is Windows/Linux/Kubernetes/etc?
The usual way to access resource files is via the classpath, not by going directly to the filesystem.
There are several advantages to this:
The resources will usually be deployed automatically, along with your compiled code; no need for manual copying. (Maven and Gradle usually know to treat the src/main/resources folder this way.)
If you build the application as a .jar/.war file, the resources will get included in that. (Your code doesn't need to know or care that they're in an archive; as long as you access the resource as a stream, loading just works.)
The application isn't tied to a particular directory layout or OS; it can run in any directory on any machine.
You can override files. Because the classpath can give several locations, a file in a later location can be overridden by adding a file into an earlier location.
It's easy enough to do: just call getResourceAsStream() on the relevant classloader. In this case, it could be as simple as:
val parser = javaClass.classLoader.getResourceAsStream("support/certain-file.json")?.use {
JsonFactory().createParser(it)
}
Here javaClass.classLoader gets the current class's ClassLoader. (You could use the system one instead, of course.)
use() takes care of closing the resulting InputStream once it's been read.
And luckily, there's a createParser() method which accepts the InputStream.
(Disclaimer: I don't know anything about Kubernetes, or about running Kotlin/Java programs on Windows. But the beauty of using the classloader is that you don't need to! If it can load classes, it can load resources too.)
In my web application I have to send email to set of predefined users like finance#xyz.example, so I wish to add that to a .properties file and access it when required. Is this a correct procedure, if so then where should I place this file? I am using Netbeans IDE which is having two separate folders for source and JSP files.
It's your choice. There are basically three ways in a Java web application archive (WAR):
1. Put it in classpath
So that you can load it by ClassLoader#getResourceAsStream() with a classpath-relative path:
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
InputStream input = classLoader.getResourceAsStream("foo.properties");
// ...
Properties properties = new Properties();
properties.load(input);
Here foo.properties is supposed to be placed in one of the roots which are covered by the default classpath of a webapp, e.g. webapp's /WEB-INF/lib and /WEB-INF/classes, server's /lib, or JDK/JRE's /lib. If the propertiesfile is webapp-specific, best is to place it in /WEB-INF/classes. If you're developing a standard WAR project in an IDE, drop it in src folder (the project's source folder). If you're using a Maven project, drop it in /main/resources folder.
You can alternatively also put it somewhere outside the default classpath and add its path to the classpath of the appserver. In for example Tomcat you can configure it as shared.loader property of Tomcat/conf/catalina.properties.
If you have placed the foo.properties it in a Java package structure like com.example, then you need to load it as below
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
InputStream input = classLoader.getResourceAsStream("com/example/foo.properties");
// ...
Note that this path of a context class loader should not start with a /. Only when you're using a "relative" class loader such as SomeClass.class.getClassLoader(), then you indeed need to start it with a /.
ClassLoader classLoader = getClass().getClassLoader();
InputStream input = classLoader.getResourceAsStream("/com/example/foo.properties");
// ...
However, the visibility of the properties file depends then on the class loader in question. It's only visible to the same class loader as the one which loaded the class. So, if the class is loaded by e.g. server common classloader instead of webapp classloader, and the properties file is inside webapp itself, then it's invisible. The context class loader is your safest bet so you can place the properties file "everywhere" in the classpath and/or you intend to be able to override a server-provided one from the webapp on.
2. Put it in webcontent
So that you can load it by ServletContext#getResourceAsStream() with a webcontent-relative path:
InputStream input = getServletContext().getResourceAsStream("/WEB-INF/foo.properties");
// ...
Note that I have demonstrated to place the file in /WEB-INF folder, otherwise it would have been public accessible by any webbrowser. Also note that the ServletContext is in any HttpServlet class just accessible by the inherited GenericServlet#getServletContext() and in Filter by FilterConfig#getServletContext(). In case you're not in a servlet class, it's usually just injectable via #Inject.
3. Put it in local disk file system
So that you can load it the usual java.io way with an absolute local disk file system path:
InputStream input = new FileInputStream("/absolute/path/to/foo.properties");
// ...
Note the importance of using an absolute path. Relative local disk file system paths are an absolute no-go in a Java EE web application. See also the first "See also" link below.
Which to choose?
Just weigh the advantages/disadvantages in your own opinion of maintainability.
If the properties files are "static" and never needs to change during runtime, then you could keep them in the WAR.
If you prefer being able to edit properties files from outside the web application without the need to rebuild and redeploy the WAR every time, then put it in the classpath outside the project (if necessary add the directory to the classpath).
If you prefer being able to edit properties files programmatically from inside the web application using Properties#store() method, put it outside the web application. As the Properties#store() requires a Writer, you can't go around using a disk file system path. That path can in turn be passed to the web application as a VM argument or system property. As a precaution, never use getRealPath(). All changes in deploy folder will get lost on a redeploy for the simple reason that the changes are not reflected back in original WAR file.
See also:
getResourceAsStream() vs FileInputStream
Adding a directory to tomcat classpath
Accessing properties file in a JSF application programmatically
Word of warning: if you put config files in your WEB-INF/classes folder, and your IDE, say Eclipse, does a clean/rebuild, it will nuke your conf files unless they were in the Java source directory. BalusC's great answer alludes to that in option 1 but I wanted to add emphasis.
I learned the hard way that if you "copy" a web project in Eclipse, it does a clean/rebuild from any source folders. In my case I had added a "linked source dir" from our POJO java library, it would compile to the WEB-INF/classes folder. Doing a clean/rebuild in that project (not the web app project) caused the same problem.
I thought about putting my confs in the POJO src folder, but these confs are all for 3rd party libs (like Quartz or URLRewrite) that are in the WEB-INF/lib folder, so that didn't make sense. I plan to test putting it in the web projects "src" folder when i get around to it, but that folder is currently empty and having conf files in it seems inelegant.
So I vote for putting conf files in WEB-INF/commonConfFolder/filename.properties, next to the classes folder, which is Balus option 2.
Ex: In web.xml file the tag
<context-param>
<param-name>chatpropertyfile</param-name>
<!-- Name of the chat properties file. It contains the name and description of rooms.-->
<param-value>chat.properties</param-value>
</context-param>
And chat.properties you can declare your properties like this
For Ex :
Jsp = Discussion about JSP can be made here.
Java = Talk about java and related technologies like J2EE.
ASP = Discuss about Active Server Pages related technologies like VBScript and JScript etc.
Web_Designing = Any discussion related to HTML, JavaScript, DHTML etc.
StartUp = Startup chat room. Chatter is added to this after he logs in.
It just needs to be in the classpath (aka make sure it ends up under /WEB-INF/classes in the .war as part of the build).
You can you with your source folder so whenever you build, those files are automatically copied to the classes directory.
Instead of using properties file, use XML file.
If the data is too small, you can even use web.xml for accessing the properties.
Please note that any of these approach will require app server restart for changes to be reflected.
Assume your code is looking for the file say app.properties. Copy this file to any dir and add this dir to classpath, by creating a setenv.sh in the bin dir of tomcat.
In your setenv.sh of tomcat( if this file is not existing, create one , tomcat will load this setenv.sh file.
#!/bin/sh
CLASSPATH="$CLASSPATH:/home/user/config_my_prod/"
You should not have your properties files in ./webapps//WEB-INF/classes/app.properties
Tomcat class loader will override the with the one from WEB-INF/classes/
A good read:
https://tomcat.apache.org/tomcat-8.0-doc/class-loader-howto.html
Suppose I had a directory containing resource files stored somewhere within the "src" source directory, containing things like templates, config files, etc.
I'm aware that from a Servlet I can access files by name like:
File file = new File(ServletContact.getResource("some/namespace/filename.txt").getPath());
And from a non-Servlet I can do:
File file = new File(Object.class.getResource("some/namespace/filename.txt").getPath());
But the problem is that I have code that needs to access these resource files and can be run independent of the runtime environment. e.g. Some code uses templates from within a servlet (under Tomcat 7). Other code runs as a Quartz background job and works with templates. If I try the Object.class.getResource() method in a Tomcat servlet, it returns null.
How can I access resources files in a safe way regardless of runtime environment, app engine, etc.?
To read file from classpath you can use:
getClass().getClassLoader().getResourceAsStream("path/to/resource");
Also there is simple and useful Spring utility ClassPathResource class:
Resource resource = new ClassPathResource("path/to/resource");
I would use any class (e.g. domain class) from your project, use getClassLoader() or getContextClassloader() and provide the path to your resource. Should work.
I'm working in Maven web application (Java). I need to read a directory (for ex: mydirectory) in my webapp folder as follows:
File file = new File("path of my directory");
I just specify the hierarchy of folders in my application.
MyApplication->src->main->webapp->MyDirectory
But I'm going to write my Java code I the package as follows,
MyApplication->src->main->java->com->mypackage->myfile.java
From myfile.java I need to read the directory "MyDirectory" in webapp. As new File("path of the directory")
But I don't know how to specify the path of the directory here.
Try
ServletContext context = getServletContext();
InputStream is = context.getResourceAsStream("/MyDirectory/FileInThatDir.txt");
alternatively use getResource() instead of getResourceAsStream()
At runtime, there is no maven anymore (thank God!), and there is no directory anymore. All you have is a war file, and everything is either in the war file, or somewhere outside of the app, on the file system.
Use ServletContext.getResourceAsStream() to load a file from the webapp's context.
You haven't mentioned what technology/framework you use, but I suppose you're using at least Java servlets, because you mentioned it's a web application. So you can use ServletContext.getRealPath() to get the path in the filesystem:
String fullPath = getServletContext().getRealPath("/MyDirectory");
System.out.printf("real filesystem path: %s", fullPath);
Update:
Please note that JavaDoc of the method says:
The real path returned will be in a form appropriate to the computer and operating system on which the servlet container is running, including the proper path separators.
Resources inside the /META-INF/resources directories of JAR files bundled in the application's /WEB-INF/lib directory must be considered only if the container has unpacked them from their containing JAR file, in which case the path to the unpacked location must be returned.
This method returns null if the servlet container is unable to translate the given virtual path to a real path.
This means that the returned path:
is not portable (e.g. in Windows with \, in Linux/macOS with /, etc).
might be null, e.g. if the resource is virtually in memory, in a JAR file, or so.
Therefore: please follow my answer only with those caveats in mind, and better follow the other answers using ServletContext.getResourceAsStream()
I couldn't access the servlet context since the service where i need to read data from the webapp folder is not aware of the current servlet context.
Anyone who is using Spring can never the less access the webapp folder by simply implementing the ResourceLoaderAware Interface provided by the spring framework.
public class SomeService implements ResourceLoaderAware {
private ResourceLoader resourceLoader;
#Override
public void setResourceLoader(ResourceLoader resourceLoader) {
this.resourceLoader = resourceLoader;
}
public readFromWebappFolder() {
Resource resource = resourceLoader.getResource("path/to/file/in/webappFolder");
}
}
I have to use a library that accepts path to a directory and loads it via File(path);. What should be the path in tomcat webapp context ?
I always work with classpaths, but this API is really not thought through...
I don't have experience with java execution context in tomcat, but it seems to me a bad idea doing something like this
System.getProperty("catalina.base") + "webapps/app/WEB-INF/classes/profiles"
Does tomcat provides java execution (AKA $PWD) path for resources in application context ?
Use ServletContext#getRealPath() to convert a webapp-relative path to an absolute disk file system path. Given your example, the following should do:
String path = getServletContext().getRealPath("/WEB-INF/classes/profiles");
File file = new File(path);
// ...
Note that this requires that the WAR is expanded, otherwise it will return null. Tomcat by default expands the WAR, but some other servletcontainers/configs don't. Keep this in mind with regard to portability.
Application resources do NOT necessary exist on file system, therefore the search from context path approach is simply wrong.
You should provide an external configuration to your application (a property file or DB), where you should define the path depending on environment.
Suggested order of searching for configuration:
System.properties: -Dmy.configuration=/home/tomcat/my.properties
System environment: MY_CONFIGURATION=/home/tomcat/my.properties
Default value: ./my.properties (for Tomcat it is usually bin folder, but not guaranteed)
Fail and complain.