So I have a microservice app that does image processing with ImageJ which I have created a microservice using spring boot.
Often the image I am trying to load is coming from a samba share mapped to a directory like p:/
I have an issue that is ONLY happening when I execute the spring boot app as a JAR directly. If I execute it directly from STS using the tool executors it works fine. As well, the file is readable, viewable etc.
File f = new File("P:\\Stamps\\_Temp\\Img001.jpg");
BufferedImage image = ImageIO.read(f);
This will result in
javax.imageio.IIOException: Can't read input file!
at java.desktop/javax.imageio.ImageIO.read(ImageIO.java:1308) ~[na:na]
For debugging purposes I had it print out the .exist() and .canRead() - when executed in STS (Eclipse) these both return true, however from the JAR it will return false. More over if I try to access the image directly from a local folder (say c:\my-images) it works fine. So my assumption is there is some thing restricting access to these Network shared files when accessed from within a Jar (only).
I have not been able to find any reference information via searches to this on the usage of File so I am wondering if there is a spring boot configuration that is blocking this access (mainfest setting etc), or if it is a restriction of executing class byte-code from within a JAR?
So networked Mapped Drives in Windows can be accessed if you track back to the remote name and replace that drive letter with the appropriate mapping name. This thread covers an example where they do that: https://gist.github.com/digulla/31eed31c7ead29ffc7a30aaf87131def they key here is to replace the "P:" with "\server\path"
Again does not explain why this fails via Jar access vs. class exploded access, but at least it covers a workaround. For my use I might just simply use a mapping file since while I use the Network Mapping, I do not know how common this would be for other users and asking them to set some configuration in application.properties does not seem ridiculous for those cases. Still if anyone has insights into WHY we get different behavior inside and outside the Jar execution I'd be curious (or whether there is some spring-boot property in the manifest that needs to be set)
I am using Docker and NuGet repositories in Artifactory. Current Artifactory REST API does not support every option on Docker or NuGet repositories.
It looks like the whole configuration created by calling appropriate REST API calls is stored in a file called artifactory.config.latest.xml.
Is it safe to restore the whole repository configuration just by copying that file?
The artifactory.config.xml that's stored in your filesystem is there mainly for recovery scenarios, although it is perfectly usable, yes.
The thing is, is that you have to restart your instance to have the filesystem changes re-imported into Artifactory (you also need to rename the file to artifactory.config.import.xml for Artifactory to pick it up at startup).
If you're looking for and option to modify repository configuration (or any other configuration Artifactory has for that matter) during runtime you can use this api which retrieves and persists the config.
It's perfectly safe to use as it represents all of the available configuration for all of your repos and the global configuration parameters as well.
Do take care though - it's always a good idea to backup a known working copy before you start playing around - there's no undo button.
Also, this configuration is subject to change as versions progress - you can always consult the config schema if you get lost (note the version - that's the one you have at the top of the xml file declaration).
I have a webservice that uses Java, REST, Jersey and runs on Tomcat8. The webservice requires access to a database. Depending on where we are in the process the we may be using a testdatabase, production database or something else. Ideally we would like to be able to set which database to use without requiring a code change and recompile.
The approach we have tried is to have a properties file defining the database parameters and use an environment variable to point to the file. This has proved troublesome, first we've had a hard time defining system properties on the Tomcat server that we can read from the application, also it seems like all the files will have to be defined on the classpath, i.e already configured ahead of time and part of the codebase.
This seems like fairly common scenario, so I'm sure there is a recommended way to handle situations like this?
Zack Macomber has a point here. Don't enable your app/service to look up its settings dynamically.
Make your build process dynamic instead.
Maven, Gradle and friends all provide simple ways to modify output depending on build parameters and or tasks/profiles.
In your code always link to the same file (name). The actual file will then be included based on your task and/or build environment. Test config for tests. Production config for production.
In many cases a complete recompilation is not necessary and will therefore be skipped (this depends on your tool, of course).
No code changes at all. Moreover the code will be dumb as hell as it does not need to know anything about context.
Especially when working on something with multiple people this approach provides the most stable long-term-solution. Customizable for those who need some special, local config and most important transparent for all who don't need or don't want to know about runtime environment requirements!
We have a similar case. We have created a second web service on the same endpoint (/admin) which we call to set a few configuration parameters. We also have a DB for persisting the configuration once set. To make life easier, we also created a simple UI to set these values. The user configures the values in the UI, the UI calls the /admin web service, and the /admin service sets the configuration in memory (as properties) as well as in the DB. The main web service uses the properties as dynamic configuration.
Note: we use JWT based authorization to prevent unauthorized access to /admin. But depending upon your need you can keep it unsecure, use basic HTTP auth or go with something more detailed.
Not sure if in this particular case it is wise, but it is possible indeed to create a .properties file anywhere on the filesystem - and link it into your application by means of a Resources element.
https://tomcat.apache.org/tomcat-8.0-doc/config/resources.html
The Resources element represents all the resources available to the web application. This includes classes, JAR files, HTML, JSPs and any other files that contribute to the web application. Implementations are provided to use directories, JAR files and WARs as the source of these resources and the resources implementation may be extended to provide support for files stored in other forms such as in a database or a versioned repository.
You would need a PreResources element here, linking to a folder, the contents of which will be made available to the application at /WEB-INF/classes.
<Context antiResourceLocking="false" privileged="true" docBase="${catalina.home}/webapps/myapp">
<Resources className="org.apache.catalina.webresources.StandardRoot">
<!-- external res folder (contains settings.properties) -->
<PreResources className="org.apache.catalina.webresources.DirResourceSet"
base="/home/whatever/path/config/"
webAppMount="/WEB-INF/classes" />
</Resources>
</Context>
Your application now 'sees' the files in /home/whatever/path/config/ as if they were located at /WEB-INF/classes.
Typically, the Resources element is put inside a Context element. The Context element must be put in a file located at:
$CATALINA_BASE/conf/[enginename]/[hostname]/ROOT.xml
See https://tomcat.apache.org/tomcat-8.0-doc/config/context.html#Defining_a_context
We build 3-tier enterprise solutions that typically consists of several webapp and ejbjar modules that all talk to a db and have several external integration points.
Each module typically needs its own configurations that can change over the solution's life time.
Deploying it becomes a nightmare because now we have 18 property files that must be remembered to copied over and configured also setting up data-sources, queues, memory requirements etc.
I'm hopeful but not optimistic that there can be a better way.
Some options we've considered/used, each with it's pros and cons:
Use multiple maven projects and continuous integration (eg. hudson or jenkins) to build a configuration jar that includes all the property files for each environment (dev, qa, prod) and then bundle everything up as an EAR. But then things can't be easily changed in production when needed.
Put most of the settings in the DB and have a simple screen to modify it. Internally we can have a generic configuration service EJB that can read and modify the values. Each module can have a custom extended version that have specific getters and setter.
Version control all the property files then check it out on production and check it into a production branch after making changes.
With all of these you still need to configure data-sources and queues etc. in a container specific way :(
Сonsider binding a custom configuration object to JNDI. Then lookup this object in your apps to configure them. Benefits - you can use custom configuration object instead of rather generic Map or Properties.
Another way is to use JMX to configure applications you need. Benefits - you can bind objects you have to configure directly to MBean Server and then use such a well-known tools as jconsole or visualvm to configure components of your application.
Both ways support dynamic reconfiguration of your applications at runtime. I would prefer using JMX.
I've gone through several cycles of finding ways to do this. I still don't have a definite answer.
The last cycle ended up with a process based on properties files. The idea was that each server instance was configured with a single properties file that configured everything. That file was read by the startup scripts, to set memory parameters, by the app server, and by the application itself.
The key thing, though, was that this file was not managed directly. Rather, it was a product of the build process. We had a range of files for different purposes, kept in version control, and a build step which merged the appropriate ones. This lets you factor out commonalities that are shared along various axes.
For example, we had development, continuous integration, QA, UAT, staging, and production environments, each with its own database. Servers in different environments needed different database settings, but each server in a given environment used the same settings. So, there was something like a development-db.properties, qa-db.properties, and so on. In each environment, we had several kinds of servers - web servers, content management servers, batch process servers, etc. Each had JVM settings, for heap size and so on, that were different to other kinds of servers, but consistent between servers across environments. So, we had something like web-jvm.properties, cms-jvm.properties, batch-jvm.properties, and so on. We also had a way to have overrides for specific systems - production-cms-jvm.properties sort of thing. We also had a common.properties that set common properties, and sensible defaults which could be overridden where needed.
Our build process was actually a bit more complicated than just picking the right options from each set; we had a master file for each server in each environment which specified which other files to include. We allowed files to specify other files to include, so we could build a graph of imports to maximise reuse.
It ended up being quite complicated. Too complicated, i think. But it did work, and it did make it very, very easy to make changes affecting many servers in a controlled way. We even merged a set of input files from development, and another from operations, which contained sensitive information. It was a very flexible approach.
I know this has already been answered and my answer is not necessarily generic, but here's my take on things:
Note, here I'm only considering system/resource properties, not application settings. In my view, application settings (such as a payment threshold or other settings should be stored in a database, so that the system can be reconfigured without having to restart a service or cause downtime by re-deploying or re-reading a properties file).
For settings that impact on how different parts of a system connect with each other (such as web service endpoints, etc), I would make use of the JNDI tree.
Database connectivity and JMS connectivity would then be set-up using the Websphere console and can be managed by the Websphere administrators. These can also be created as JACL scripts which can be put into version control if necessary.
In addition to the JNDI resources, for additional properties, such as usernames for web service calls to a backend, etc, I would use Websphere "Name Space Bindings". These bindings can be edited using the Websphere console and accessed via JNDI using the "cell/persistent/mypassword" name.
So I could create the "mypassword" binding (a string), and the management for it falls to the Websphere admin (away from developer eyes or other people who should not have access to production systems), while the same EAR file can be used on dev, test, preproduction and production (which is preferable to have different EAR files for different systems, as the likelihood of other differences creeping in is reduced).
The Java code would then use a simple JNDI lookup (and possibly cache the value in memory).
Advantages over properties files:
Not having a "vulnerable" file that would need to be secured because system properties contain passwords.
Not having to add Java security policies to allow access to that file location
Advantages over database properties:
Not tied to having one database tied to an application server.
Hope that helps
Use multiple maven projects and continuous integration (eg. hudson or
jenkins) to build a configuration jar that includes all the property
files for each environment (dev, qa, prod) and then bundle everything
up as an EAR. But then things can't be easily changed in production
when needed.
I think the config should be in the database of the application instance. Your local machine config may be diffrent to dev and to QA, PROD , DR etc.
What you need is a way of getting the config out the database in a simple way.
I create a separate project with a provided dependency of Apache commons-configuration
It has many ways of storing data, but I like databases and the configurations lives in the database environment.
import javax.sql.DataSource;
import org.apache.commons.configuration.DatabaseConfiguration;
public class MYConfig extends DatabaseConfiguration {
public MYConfig(DataSource datasource) {
super(datasource, "TABLE_CONFIG", "PROP_KEY", "PROP_VALUE");
}
}
Put most of the settings in the DB and have a simple screen to modify
it. Internally we can have a generic configuration service EJB that
can read and modify the values. Each module can have a custom extended
version that have specific getters and setter.
Commons configurations as a simple API, you may then write the GUI as you wish.
You can do the interface in anyway you wish. Or as a quick win have no interface.
Version control all the property files then check it out on production
and check it into a production branch after making changes.
Version control is great. Add another DatabaseConfiguration using composition. The class you extends is the active config and the composed one being the audit. There is another constructor can can have a version. Just overload the right methods to get the desired effect.
import javax.sql.DataSource;
import org.apache.commons.configuration.DatabaseConfiguration;
public class MYConfig extends DatabaseConfiguration {
final DatabaseConfiguration audit;
public MYConfig(DataSource datasource) {
super(datasource, "TABLE_CONFIG", "PROP_KEY", "PROP_VALUE");
audit = new DatabaseConfiguration("TABLE_CONFIG_AUDIT", "PROP_KEY", "PROP_VALUE");
}
#Override
public void addProperty(String key, Object value) {
Object wasValue = super.getProperty(key);
super.addProperty(key, value);
audit.put(key,wasValue);//add version code
}
}
http://commons.apache.org/proper/commons-configuration/
User a simple database table (Section, Key, Value). Add "Version" if you need it, and wrap the entire thing in a simple ConfigurationService class with methods like getInt(String section, String key)
Not a lot of work, and it makes the application code very neat, and tweaking with the configuration very easy.
Interesting alternative config file format: write a scala trait. Your config file can then just be a scala file that you compile and evaluate when the server starts.
http://robey.lag.net//2012/03/26/why-config.html
I have a web service that provides clients with configuration data. It receives a request containing a version (ex: "v1") of that configuration data and needs to retrieve the data associated with that version. New versions of configuration data can be added while the web service is running.
The directory structure of the configuration data is:
config
v1
...config files...
v2
...config files...
v3
...config files...
I know that web application best practices state I should not use java.io.File and should instead use ClassLoader.getResource(), so how do I go about doing that? I can't compile my config data into a JAR because it can be updated while the service is running, so do I place the data on the filesystem and add that directory to my classpath?
We didn't find any way to do this with Weblogic, so we updated the application to read the data from the filesystem instead of the classpath.