We Keep Coding

Read external .properties file from the install directory (not the current directory)

I am distributing a Spring Boot application as a zipped "bootJar" using the Gradle Application plugin and the "distZip" task. The end-user will get the zip file, unzip it, and run it by just typing "myApp" (a shell script nicely created by the plugin).
I would like the end-user to create a "myapp.properties" file (a name I chose) and put it in the installation directory, or a "config" directory under the installation directory.
Suppose I set up my embedded (in the jar) application.properties file as follows:
spring.config.import = file:./myapp.properties will only read from the current directory
spring.config.import = file:/etc/myapp.properties will read from the specified directory -- but I don't know what this is at build time (the end-user determines it at installation time)
How can I set up my application so that Spring Boot can read properties from an external file whose location is specified later?
NOTE: I know I can play around with the generated scripts to pass in environment variables or Spring Boot properties, but I was hoping to do this completely within Spring Boot so I don't need to modify the nicely generated shell scripts.

spring.config.import = file:./myapp.properties will only read from the
current directory spring.config.import = file:/etc/myapp.properties
will read from the specified directory -- but I don't know what this
is at build time (the end-user determines it at installation time)
Why overcomplicate this.
Place inside the jar all the properties that you want to be statically configured as default values when you build the application.
Embedded application.properties
server.port = 8080
prop1.element = something
Then the client can create another file application.properties and place it in the same directory with the jar and define more properties which are not already defined.
prop2.element = something2
prop3.element = something3
By default Spring Boot will load properties both from the embedded file as well from the file in the current directory where the jar is placed during startup.
In the external application.properties you can also overwrite properties existing in the embedded application.properties. So if the external file in the current directory same as the jar is the following
prop2.element = something2
prop3.element = something3
prop1.element = something4 <--- this value here will overwrite the value 'something' from embedded file
According to spring doc
SpringApplication will load properties from application.properties
files in the following locations and add them to the Spring
Environment:
/config subdirectory of the current directory.
The current directory
classpath /config package
The classpath root The list is ordered by
Precedence (properties defined in locations higher in the list
override those defined in lower locations).
After having more input from the comments, it seems that you face another issue as well. You start the application from command line from another directory so that is counted as the directory where spring will look for the external configuration instead of where the jar is placed.
So for example let's say that the jar is placed inside the target folder that exists in current directory. You start the application using the following command:
java -jar target/demo-0.0.1-SNAPSHOT.jar
But then the external application.properties existing inside target folder is not loaded from spring because you executed the command from another directory. This can be solved if you start the application in the following way
java -jar -Dspring.config.additional-location=./target/ target/demo-0.0.1-SNAPSHOT.jar
This should not be difficult as you already provide the path where the jar exists in the command line.

How to load resources without using an absolute path [duplicate]

In my web application I have to send email to set of predefined users like finance#xyz.example, so I wish to add that to a .properties file and access it when required. Is this a correct procedure, if so then where should I place this file? I am using Netbeans IDE which is having two separate folders for source and JSP files.

It's your choice. There are basically three ways in a Java web application archive (WAR):
1. Put it in classpath
So that you can load it by ClassLoader#getResourceAsStream() with a classpath-relative path:
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
InputStream input = classLoader.getResourceAsStream("foo.properties");
// ...
Properties properties = new Properties();
properties.load(input);
Here foo.properties is supposed to be placed in one of the roots which are covered by the default classpath of a webapp, e.g. webapp's /WEB-INF/lib and /WEB-INF/classes, server's /lib, or JDK/JRE's /lib. If the propertiesfile is webapp-specific, best is to place it in /WEB-INF/classes. If you're developing a standard WAR project in an IDE, drop it in src folder (the project's source folder). If you're using a Maven project, drop it in /main/resources folder.
You can alternatively also put it somewhere outside the default classpath and add its path to the classpath of the appserver. In for example Tomcat you can configure it as shared.loader property of Tomcat/conf/catalina.properties.
If you have placed the foo.properties it in a Java package structure like com.example, then you need to load it as below
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
InputStream input = classLoader.getResourceAsStream("com/example/foo.properties");
// ...
Note that this path of a context class loader should not start with a /. Only when you're using a "relative" class loader such as SomeClass.class.getClassLoader(), then you indeed need to start it with a /.
ClassLoader classLoader = getClass().getClassLoader();
InputStream input = classLoader.getResourceAsStream("/com/example/foo.properties");
// ...
However, the visibility of the properties file depends then on the class loader in question. It's only visible to the same class loader as the one which loaded the class. So, if the class is loaded by e.g. server common classloader instead of webapp classloader, and the properties file is inside webapp itself, then it's invisible. The context class loader is your safest bet so you can place the properties file "everywhere" in the classpath and/or you intend to be able to override a server-provided one from the webapp on.
2. Put it in webcontent
So that you can load it by ServletContext#getResourceAsStream() with a webcontent-relative path:
InputStream input = getServletContext().getResourceAsStream("/WEB-INF/foo.properties");
// ...
Note that I have demonstrated to place the file in /WEB-INF folder, otherwise it would have been public accessible by any webbrowser. Also note that the ServletContext is in any HttpServlet class just accessible by the inherited GenericServlet#getServletContext() and in Filter by FilterConfig#getServletContext(). In case you're not in a servlet class, it's usually just injectable via #Inject.
3. Put it in local disk file system
So that you can load it the usual java.io way with an absolute local disk file system path:
InputStream input = new FileInputStream("/absolute/path/to/foo.properties");
// ...
Note the importance of using an absolute path. Relative local disk file system paths are an absolute no-go in a Java EE web application. See also the first "See also" link below.
Which to choose?
Just weigh the advantages/disadvantages in your own opinion of maintainability.
If the properties files are "static" and never needs to change during runtime, then you could keep them in the WAR.
If you prefer being able to edit properties files from outside the web application without the need to rebuild and redeploy the WAR every time, then put it in the classpath outside the project (if necessary add the directory to the classpath).
If you prefer being able to edit properties files programmatically from inside the web application using Properties#store() method, put it outside the web application. As the Properties#store() requires a Writer, you can't go around using a disk file system path. That path can in turn be passed to the web application as a VM argument or system property. As a precaution, never use getRealPath(). All changes in deploy folder will get lost on a redeploy for the simple reason that the changes are not reflected back in original WAR file.
See also:
getResourceAsStream() vs FileInputStream
Adding a directory to tomcat classpath
Accessing properties file in a JSF application programmatically

Word of warning: if you put config files in your WEB-INF/classes folder, and your IDE, say Eclipse, does a clean/rebuild, it will nuke your conf files unless they were in the Java source directory. BalusC's great answer alludes to that in option 1 but I wanted to add emphasis.
I learned the hard way that if you "copy" a web project in Eclipse, it does a clean/rebuild from any source folders. In my case I had added a "linked source dir" from our POJO java library, it would compile to the WEB-INF/classes folder. Doing a clean/rebuild in that project (not the web app project) caused the same problem.
I thought about putting my confs in the POJO src folder, but these confs are all for 3rd party libs (like Quartz or URLRewrite) that are in the WEB-INF/lib folder, so that didn't make sense. I plan to test putting it in the web projects "src" folder when i get around to it, but that folder is currently empty and having conf files in it seems inelegant.
So I vote for putting conf files in WEB-INF/commonConfFolder/filename.properties, next to the classes folder, which is Balus option 2.

Ex: In web.xml file the tag
<context-param>
<param-name>chatpropertyfile</param-name>
<!-- Name of the chat properties file. It contains the name and description of rooms.-->
<param-value>chat.properties</param-value>
</context-param>
And chat.properties you can declare your properties like this
For Ex :
Jsp = Discussion about JSP can be made here.
Java = Talk about java and related technologies like J2EE.
ASP = Discuss about Active Server Pages related technologies like VBScript and JScript etc.
Web_Designing = Any discussion related to HTML, JavaScript, DHTML etc.
StartUp = Startup chat room. Chatter is added to this after he logs in.

It just needs to be in the classpath (aka make sure it ends up under /WEB-INF/classes in the .war as part of the build).

You can you with your source folder so whenever you build, those files are automatically copied to the classes directory.
Instead of using properties file, use XML file.
If the data is too small, you can even use web.xml for accessing the properties.
Please note that any of these approach will require app server restart for changes to be reflected.

Assume your code is looking for the file say app.properties. Copy this file to any dir and add this dir to classpath, by creating a setenv.sh in the bin dir of tomcat.
In your setenv.sh of tomcat( if this file is not existing, create one , tomcat will load this setenv.sh file.
#!/bin/sh
CLASSPATH="$CLASSPATH:/home/user/config_my_prod/"
You should not have your properties files in ./webapps//WEB-INF/classes/app.properties
Tomcat class loader will override the with the one from WEB-INF/classes/
A good read:
https://tomcat.apache.org/tomcat-8.0-doc/class-loader-howto.html

HSQLDB in resources within a WAR

I've developed a small application using Spring Boot, which uses HSQLDB as a database to store users, for the time being. What I want to do is store the database within Maven's resource folder. My application.properties file contains the following configuration:
#...
spring.datasource.url=jdbc:hsqldb:file:src/main/resources/database/dashboard
#...
So, whenever I start the application with Java (right click on my Application's main() and Run As Spring Boot App), the database is stored right.
But, after packaging a WAR file with mvn clean install spring-boot:repackage, deploying it into Tomcat, e.g., and running it, the database is not stored within the exploded WAR.
My question is how can I tell spring.datasource.url=jdbc:hsqldb:file:* property to store the database within Maven Resources directory (which will point still after exploding the WAR)? Could I use some sort of environment variable as used in pom.xml?
EDIT
Given the answers by #Steve C and #fredt, I've realised that the database shouldn't be stored within the war. Instead, I'll store it within the user's Home dir spring.datasource.url=jdbc:hsqldb:file:~/tomcat_webapp_data/dashboard/database.. Thank you so much!

The directories and contents of a WAR file are read-only.
You can set the read_only flag in the HSQLDB database .properties file before including in the WAR. You access this kind of database with a jdbc:hsqldb:res:<path> URL.
If you want to store data in a persistent and updatable database, connect to the (at first non-existent) database within your application and set up its tables if they don't yet exist with data from a resource. You can then store data. The database path should be outside the directories that are used for jars and resources.
Contrary to one of the comments, HSQLDB is not limited to storing data in memory and can have disk-based tables, called CACHED tables.
You can include a variable in the database URL to pick up a pre-defined property from the web server. For example:
jdbc:hsqldb:file:${mydbpath}
See http://hsqldb.org/doc/2.0/guide/dbproperties-chapt.html#dpc_variables_url

Maven's src/main/resources directory is a build time path, not a runtime path.
If you're building a jar file, then everything in src/main/resources is copied to the root of the jar.
If you're building a war file, then everything in src/main/resources is copied to the root of the /WEB-INF/classes directory in the war.
Now, your JDBC URL jdbc:hsqldb:file:src/main/resources/database/dashboard is indicating a file with a relative path. At runtime this path is relative to the current working directory - and it's unlikely to exist at runtime anyway.
If you really want to build the database within your WAR, then given:
you're deploying an exploded WAR file (it's impossible to write to the content of the WAR file itself);
you really really want to store the database within the exploded WAR;
you want to put it in WEB-INF/database/dashboard (there's security implications if you store outside of the WEB-INF directory);
then you can compute the JDBC url using something like:
...
String databaseDirectoryName = servletContext.getRealPath("WEB-INF/database");
File databaseDirectory = new File(databaseDirectoryName);
if (!databaseDirectory.exists()) {
if (!databaseDirectory.mkdirs()) {
throw new RuntimeException("Failed to create database directory");
}
}
File databaseFile = new File(databaseDirectory, "dashboard");
String jdbcURL = "jdbc:hsqldb:" + databaseFile.toURI();
...
Getting that into your Spring configuration is an exercise for you; but using #Configuration and #Bean springs to mind as a way to do this - you just need to get access to the servletContext at Spring configuration time.

How to call filepath for files listed in a .properties file

I have a .properties file that is under a source folder I made called res (using Eclipse Mars 2). I have two other folders in there called messages and schemas.
I need help in giving a filepath so it works locally and on a server (e.g. JBoss) after making the project into a .war file. This is my .properties file:
# Credentials
user=flow
password=flow
# Path to schema for validation
schemaPath=schemas/Schema1.xsd
# Path to where you want to keep incoming and outgoing messages
messagePath=messages/
The above properties file will only work if I provide the full path to the two different *Path properties (above is not full path). However, I can't do that because it needs to work on the application server and on different operating systems.
In my code, I save the filepaths to Strings and use those Strings to specify where to write or read. How can I make it so it works after deploying to the server using a .war file?
I am using a Dynamic Web Project in Eclipse Mars 2.
EDIT: Since the properties is user configurable, they might give a full path. It should work whether the path is short as shown or the full path.

You have to make sure that the properties file is part of the classpath, that is usually including it in classes/ directory.
Mark the folder with properties file as Source Folder in your eclipse. If it is in a package, then use that package name in your path while loading the file.
For example, if the file is in config/data.properties, then load the file by .getResource("config/data.properties");

How to retrieve .properties?

Im developing desktop java application using maven.
I got a *.properties file that I need to retrive during execution (src/resources/application.properties).
The only thing comes to my mind is to use:
private Properties applicationProperties;
applicationProperties.load(new BufferedInputStream(new FileInputStream("src/resources/application.properties")));
This would work if I run my application directly from IDE.
I want to to keep outpout hierarchy clear, so I set maven to copy resources folder dircetly to target folder (which is a basedir for the output application). This way application.properties file won't load (since I have target/resources/application.properties but not target/src/resources/application.properties).
What is the best way to manage resources so they work both when I debug from IDE and run builded jar file directly?

Don't expect files to be in the src folder - it doesn't exist at runtime. The properties files go to /bin. But don't rely on that either. Because FileInputStream takes absolute paths only.
When you need a classpath-relative path, use:
InputStream is = YourClass.class.getResourceAsStream("/a.properties")`
(maven sends files from /src/main/resources to the root of the classpath)

You should load the property file from the classpath rather than from an explicit file system location:
applicationProperties.load(new BufferedInputStream(this.getClass().getResourceAsStream( "/application.properties" );
As long as your IDE is configured to include the resources directory on your classpath (this should be the default with Maven), then this will work whether you're running within the IDE or not since maven will copy the resources to the right place when packaging your archive.