I am trying to replace a file in my resource folder (src/main/resources) with a new file generated by the Gradle build script. I'm having trouble doing this; the exclusion seems to be remembered, and preventing the addition of my new file.
Here's a short example that illustrates the behavior.
Project Structure:
TestProject
-- src/main/java
---- entry
------ EntryPoint.java
---- run
------ HelloWorldTest.java
-- src/main/resources
---- test.properties // FILE TO REPLACE
test.properties contents in src/main/resources:
Wrong File with extra text to make it obvious which one is being put into the jar based on size
build.gradle:
apply plugin: 'java'
task makeProp {
def propDir = new File(buildDir, "props")
ext.propFile = new File(propDir, "test.properties")
outputs.file propFile
doLast {
propDir.mkdirs()
propFile.createNewFile()
propFile.withWriter('utf-8') { writer ->
writer.writeLine 'Right File'
}
}
}
jar {
dependsOn('makeProp')
if (project.hasProperty('testExclude')) {
sourceSets {
exclude('test.properties')
}
}
from (makeProp.propFile) {
into '/'
}
}
JAR contents of ./gradlew build (both files included):
Archive: TestProject.jar
Length Date Time Name
-------- ---- ---- ----
0 08-07-15 14:27 META-INF/
25 08-07-15 14:27 META-INF/MANIFEST.MF
0 08-07-15 13:50 run/
499 08-07-15 13:50 run/HelloWorldTest.class
0 08-07-15 13:50 entry/
1413 08-07-15 13:50 entry/EntryPoint.class
95 08-07-15 14:27 test.properties
11 08-07-15 14:03 test.properties
-------- -------
2043 8 files
JAR contents of ./gradlew build -PtestExclude (neither file included):
Archive: TestProject.jar
Length Date Time Name
-------- ---- ---- ----
0 08-07-15 14:29 META-INF/
25 08-07-15 14:29 META-INF/MANIFEST.MF
0 08-07-15 13:50 run/
499 08-07-15 13:50 run/HelloWorldTest.class
0 08-07-15 13:50 entry/
1413 08-07-15 13:50 entry/EntryPoint.class
-------- -------
1937 6 files
I have done something very similar and this is what worked for me. The main objective is to make sure the task runs before your jar is created and your files are processed. Try this out.
// create a properties file add it to folder preprocessing
task propertiesFile << {
//
description 'Dynamically creates a properties file.'
// needed for the first pass
def folder = project.file('src/main/resources');
if(!folder.exists()){
folder.mkdirs()
}
//write it to a propertiess file
def props = project.file('src/main/resources/test.properties')
props.delete()
props << 'write this to my file!!!!'
}
processResources.dependsOn propertiesFile
The solution I went with in the end was similar to what was posted by Pumphouse, but instead of deleting the file, I renamed it to a temp name, excluded that temp name, and renamed it once I was done (I needed to preserve the file contents and position after the build was complete).
Modified build.gradle:
apply plugin: 'java'
def props = project.file('src/main/resources/test.properties')
def temp = project.file('src/main/resources/temp.properties')
task makeProp {
... // Unchanged
}
jar {
dependsOn('makeProp')
// Move the properties file to a temp location
props.renameTo(temp) // This returns a boolean; can perform conditional checks.
// Exclude the temp file
if (project.hasProperty('testExclude')) {
sourceSets {
exclude('temp.properties')
}
}
// Insert the right prop file
from (makeProp.propFile) {
into '/'
}
}
jar << {
// Restore the temp file
temp.renameTo(props)
}
The thing is that the exclusion patterns for the gradle SourceSet apply to all included paths and not only to specific paths. Therefore in your example above all files named test.properties will be excluded, regardless on their location.
What you can do is to have the default test.properties located somewhere else and make the build copy/generate the relevant version based on the scenario.
Related
I have a multi-module Java(Spring) project, which build by Gradle 6.7.1. And I use in Jetbrain IDEA to develop. The file Structure like this:
root
|--orm
| +---hibernates
|
|--web
|--mvc
|--rest
And then, I have tried some codes in my module project like below, what I get all are root path (/home/user/IdeaProjects/root/), not module path (/home/user/IdeaProjects/root/web/mvc). How can I get module path (/home/user/IdeaProjects/root/web/mvc) ?
new File("").getAbsolutePath()
Assuming for instance that your mvc project is setup like this in setting.gradle, in the root folder :
include 'mvc'
project(':mvc').projectDir = new File('./web/mvc')
Then, to get the path /home/user/IdeaProjects/root/web/mvc, just try this :
println project(':mvc').projectDir
Will prints :
/home/user/IdeaProjects/root/web/mvc
based on the answer of #ToYonos. We can do that by this:
settings.gradle gets the project path of every module.
write a key value into the info.properties in every module.
Spring Project read this properties file.
Code
Because struct of my project is:
root
|--orm
| +---mybatis
| +---jpa
| +---...
|--web
+--mvc
+--rest
+--...
So, I should loop twice to get the module name. And I exclude project without build.gradle.
file("${rootDir}").eachDir {
it.eachDirMatch(~/.*/) {
if (it.list().contains("build.gradle")) {
def moduleName = "${it.parentFile.name}:${it.name}"
println " ${moduleName}"
include moduleName
}}}
And then, read and write info.properties.
import java.nio.file.Paths
// read
def project_dir = project(":${moduleName}").projectDir
def propFile = Paths.get("${project_dir}", "src", "main","resources","info.properties").toFile()
propFile.createNewFile()
Properties props = new Properties()
propFile.withInputStream {
props.load(it)
}
// write
props.setProperty("project.dir","$project_dir")
props.store propFile.newWriter(), null
We have a build.gradle where the version is defined in it. I need to implement a version endpoint ( like /version) to get the version of the project. This version property in build.gradle has been there for a long time, I can't move it to the project.properties file. How can I access this version's values from my Java code?
There are many ways with which you can "plant" information into your code
First you can use the manifest file, and read from it
jar {
manifest {
attributes(
"lib-version": version
}
}
With conjunction to Reading my own Jar's Manifest
The other option is to add that info the your property files before the jar task
task('addVersion') {
doLast {
//append the version here, see example
file("src/main/resources/props.properties").append("version=$version")
}
}
jar.dependsOn(addVersion)
One of our project needs not only the version information but also build time etc. We use a copy task with template substitution.
task updateVersions(type: Copy) {
Properties props = new Properties()
props.load(new FileInputStream(file('build.properties')))
props.put("buildtime", new Date().format("dd MMM yyyy hh:mm aa"))
props.put("version", "${version}")
from project(':main').file('Version.tmpl')
into file('src/main/java').path
expand(props)
rename('Version.tmpl', 'Version.java')
}
I am developing a Spring Boot Application that serves REST HTTP(S) requests. (pretty common).
It works as it is supposed, but after the final (and working) jar is signed (by a valid certificate) all URL mappings stop working, returning only 404 to any request. (Note that the embedded Tomcat server starts without problems and I don't receive any exception)
After some debugging I found that the Java's default ClassLoader (Laucher$AppClassLoader) just doesn't return the classes within the packages I configurated (#ComponentScan) when the jar is signed.
//org.springframework.core.io.support.PathMatchingResourcePatternResolver
//Param 'path' has my valid, existing and desired package with #Controller or #Component inside
protected Set<Resource> doFindAllClassPathResources(String path) throws IOException {
Set<Resource> result = new LinkedHashSet<Resource>(16);
ClassLoader cl = getClassLoader(); //sun.misc.Laucher$AppClassLoader
Enumeration<URL> resourceUrls = (cl != null ? cl.getResources(path) : ClassLoader.getSystemResources(path));
//Empty enumeration when jar is signed
...
}
I tried to use a custom Class Loader without success; same problem.
Since it works when I sign the jar with a self signed certificate, I think that there may be a problem with the signing process that was done by another person. But I can't find any proof of that.
It appears that once signed, I can't list the package content...
I'll try some more tests and add here if I consider useful...
UPDATE
After debugging with the help of a custom Class Loader, I found that:
((java.net.JarURLConnection)new java.net.URL("jar:file:/home/user/my-app-with-dependencies_signed.jar!/META-INF/MANIFEST.MF").openConnection()).getJarEntry();
Ok. Works.
((java.net.JarURLConnection)new java.net.URL("jar:file:/home/user/my-app-with-dependencies_signed.jar!/META-INF/").openConnection()).getJarEntry();
Doesn't work! >.<
It throws
Exception occurred in target VM: JAR entry META-INF/ not found in /home/user/my-app-with-dependencies_signed.jar
java.io.FileNotFoundException: JAR entry META-INF/ not found in /home/user/my-app-with-dependencies_signed.jar
at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:142)
at sun.net.www.protocol.jar.JarURLConnection.getJarEntry(JarURLConnection.java:94)
...
This same second example works when trying to access a unsigned or self-signed jar.
This operation of opening the jar is performed by Spring when reading #Controller and #Component from the given packages in #ComponentScan.
In the same way, Java's Class Loader doesn't read directories content, only specified files.
this.getClass().getClassLoader(); //sun.misc.Launcher$AppClassLoader#18b4aac2
this.getClass().getClassLoader().getResources("META-INF/MANIFEST.MF").hasMoreElements(); //always true
this.getClass().getClassLoader().getResources("META-INF/").hasMoreElements(); //false when signed
UPDATE 2
I got the information about the signature. The people responsible for signatures and certificates actually uses a Windows applications that signs the jar with certificates from Windows-MY keystore and private key from a USB token.
Not that this is certainly the cause, but I think it is important to note that jarsigner is not used.
UPDATE 3
I created a github repository with a simple test case:
https://github.com/jesjobom/signed-jar-class-loader-test
I have reached a solution but the problem still exists.
When loading the classes I pointed via #ComponentScan Spring asks the ClassLoader (Laucher$AppClassLoader) for the java.net.URL for each package I informed. Since, for some unknown reason, I can't load packages/folders, I created a custom ClassLoader that always return the expected URL if the package is mine.
public class CustomClassLoader extends ClassLoader {
...
#Override
public Enumeration<URL> getResources(String name) throws IOException {
if(name.startsWith("com/my/package/")) {
readBasePath(); //obtains path to jar (e.g. "jar:file:/home/app.jar!/")
List<URL> resources = new ArrayList<>();
resources.add(new URL(basePath + name));
return Collections.enumeration(resources);
}
return fallback.getResources(name); //default classloader
}
...
}
Even so, later, Spring tries to load the ".class" from the packages and fails for the same reasons... So, I created a custom implementation of PathMatchingResourcePatternResolver that will list all the content of the jar (this I can do!) and selects only those within the given package.
public class CustomPathMatchingResourceLoader extends PathMatchingResourcePatternResolver {
#Override
protected Set<Resource> doFindPathMatchingJarResources(final Resource rootDirResource, URL rootDirURL, String subPattern) throws IOException {
try {
String searchBase = ...; //package within jar
String pathBase = ...; //path to jar
URLConnection conn = new URL(pathBase).openConnection();
Set<Resource> resources = new HashSet();
JarFile file = ((JarURLConnection) conn).getJarFile();
Enumeration<JarEntry> entries = file.entries();
while (entries.hasMoreElements()) {
JarEntry entry = entries.nextElement();
if (entry.getName().startsWith(searchBase) && !entry.getName().endsWith("/")) {
resources.add(new UrlResource(pathBase + entry.getName()));
}
}
return resources;
} catch (Exception e) {
e.printStackTrace();
}
return super.doFindPathMatchingJarResources(rootDirResource, rootDirURL, subPattern);
}
...
}
So it worked without any interference in the signing process... I am pretty confident that signing with jarsigner would solve the problem, but I think that it'd be difficult...
Anyway, although it worked, it's not a solution. Therefore, I will not accept this answer as the correct one...
The issue is that the signed jar doesn't contain the entries META-INF and com/jesjobom explicitly.
Using 7zip you can list the zip entries: e.g. 7za l signed-jar-class-loader-test_signed.jar
Your signed jar:
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2018-02-27 15:27:08 ..... 1907 999 com\jesjobom\Main.class
2018-02-27 15:27:08 ..... 2978 944 META-INF\maven\com.jesjobom\signed-jar-class-loader-test\pom.xml
2018-02-27 15:27:08 ..... 113 111 META-INF\maven\com.jesjobom\signed-jar-class-loader-test\pom.properties
2018-02-27 15:27:08 ..... 595 361 META-INF\MANIFEST.MF
2018-02-27 15:27:08 ..... 609 386 META-INF\BANCO_DO_BRASIL_SA.SF
2018-02-27 15:27:08 ..... 4520 3251 META-INF\BANCO_DO_BRASIL_SA.RSA
------------------- ----- ------------ ------------ ------------------------
2018-02-27 15:27:08 10722 6052 6 files
After it's compiled there are some D (directories) entries, that are missing in the signed version.
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2018-02-27 15:27:08 D.... 0 0 META-INF
2018-02-27 15:27:08 D.... 0 0 com
2018-02-27 15:27:08 D.... 0 0 com\jesjobom
2018-02-27 15:27:08 D.... 0 0 META-INF\maven
2018-02-27 15:27:08 D.... 0 0 META-INF\maven\com.jesjobom
2018-02-27 15:27:08 D.... 0 0 META-INF\maven\com.jesjobom\signed-jar-class-loader-test
2018-02-27 15:27:08 ..... 1907 999 com\jesjobom\Main.class
2018-03-09 14:15:16 ..... 3082 949 META-INF\maven\com.jesjobom\signed-jar-class-loader-test\pom.xml
2018-02-27 15:27:08 ..... 117 114 META-INF\maven\com.jesjobom\signed-jar-class-loader-test\pom.properties
------------------- ----- ------------ ------------ ------------------------
2018-02-27 15:27:08 5324 2221 4 files, 6 folders
I don't know why the folder entries are missing. I suspect that because they don't have a SHA signature in \META-INF\MANIFEST.MF they are removed from the signed JAR.
I have a few lines of a Gradle script and with some Groovy methods that I want to use as a base for testing a few ideas around project testing and configuration. There are existing scripts involved that need to use the (Java) construct: File (as opposed to the Gradle file() method).
The script runs perfectly on the command line and in Netbeans as long as I supply the complete Absolute File Path to the Properties.load() call.
Later, when run build (again in Netbeans) using a legal relative path and file name my little function fails to find the the file (at all!). That can happen. But in this case; the Java File.getAbsolutePath() method show the identical string in all cases. Working and the Fail case. I prepared this using Linux. The results are produced on Windows 10.
There must be a reason. I'm blowed if I can see what it could be at this point. The Gradle,
Working with Files section
says relative file paths are OK.
Java - Writing System Properties
System.setProperty()
I myself was a little dubious about using user.dir - However the docs seem to back it up, been there since Java 1.2
It works not only with Gradle; in Java and Groovy too.
Same as -Duser.dir=pathString
Failing output:
============ project ============
Gradle version: 3.4.1
Groovy version: 2.4.7
Java version: 1.8.0_121
Root project: 'try'
project dir: '/home/projects/sandbox/lab/gradle-lab/try-gradle'
cwd: '/home/projects/sandbox/lab/gradle-lab/try-gradle'
===================================
xxxxxxxxxxxxxxxxx[ loadProperties testing ]xxxx
cwd: '/home/projects/sandbox/lab/gradle-lab/try-gradle'
user.dir: '/home/projects/sandbox/lab/gradle-lab/try-gradle' ...
loading: 'common.properties' ...
loading: '/home/projects/sandbox/lab/gradle-lab/try-gradle/common.properties' ...
* No Such File: '/home/projects/sandbox/lab/gradle-lab/try-gradle/common.properties'.
xxxxxxxxxxxxxxxxx[ loadProperties end ]xxxx
Working output:
xxxxxxxxxxxxxxxxx[ loadProperties testing ]xxxx
cwd: '/home/projects/sandbox/lab/gradle-lab/try-gradle'
user.dir: '/home/projects/sandbox/lab/gradle-lab/try-gradle' ...
loading: '/home/projects/sandbox/lab/gradle-lab/try-gradle/common.properties' ...
loading: '/home/projects/sandbox/lab/gradle-lab/try-gradle/common.properties' ...
'ext.zzz' => "ext.zzz"
'zzz' => "zzz"
xxxxxxxxxxxxxxxxx[ loadProperties end ]xxxx
The Gradle build.gradle script is quite simple with a few lines to report on the runtime environment and software versions, etc.
build.gradle script:
import org.gradle.api.artifacts.*
System.setProperty( "user.dir", project.rootDir.toString() )
/****
* try-s project
*****/
println "";
apply plugin: 'base' // To add "clean" task to the root project.
println "\n ============ project ============";
println "Gradle version: "+ gradle.gradleVersion;
println "Groovy version: "+ GroovySystem.version;
println "Java version: "+ System.getProperty("java.version");
println "";
println "Root project: '${project.rootProject.name}'";
// println " root dir: '${project.rootDir}'";
println " project dir: '${project.projectDir}'";
println " cwd: '${(new File(".")).getCanonicalPath()}'";
// println " user dir: '${System.getProperty("user.dir")}'";
println "";
println " ===================================\n";
println " xxxxxxxxxxxxxxxxx[ loadProperties testing ]xxxx"
println ""
// Does Not Work:
// File.exists() --> false
//
loadProperties( "common.properties" );
// Works - fully qualified file path:
// File.exists() --> true
// and then loads properties from the file
//
loadProperties( "/home/projects/sandbox/lab/gradle-lab/try-gradle/common.properties" );
// Works - using project.rootDir property
//
loadProperties( "${project.rootDir}/common.properties" );
// Works - using a path relative to the Netbeans start directory
// Only works when `System.user.dir` has the
// same value as the start directory
// Obviously the path must be relative to
// Netbeans start-up directory -- Likely
// to change for different locations, etc.
//
loadProperties( "../../../sandbox/lab/gradle-lab/try-gradle/common.properties" );
println ""
println " xxxxxxxxxxxxxxxxx[ loadProperties end ]xxxx"
loadProperties() function:
private def loadProperties( String fileName )
{
File propFile = new File( fileName );
Properties fileProps = new Properties( );
// Check current directory
//
println " cwd: '${(new File(".")).getCanonicalPath()}'";
println " user.dir: '${System.getProperty("user.dir")}' ...";
println " loading: '${fileName}' ...";
println " loading: '${propFile.getCanonicalPath()}' ...";
println "";
if( propFile.exists() )
{
InputStream propStream = new FileInputStream( propFile ); // fileName );
fileProps.load( propStream );
fileProps.each { prop, val ->
println " '${prop}' => \"${val}\"";
}
}
else {
println " * No Such File: '${propFile.getAbsolutePath()}'."
}
} //loadProperties
Unfortunately without the if( propFile.exists() ) check Gradle reports an excepton: FileNotFoundException (oddly enough).
Looking at the output of the propFile.getAbsolutePath() and either the Fully Qualified file name string or the rootDir+"common.properties version of events -- All Four scenarios show the identical file-path-name string:
'/home/projects/sandbox/lab/gradle-lab/try-gradle/common.properties'
For me the bottom line is, How May two Identical File Paths be processed and ONE of FOUR, just one valid file pathname is Not Found by the JVM / Gradle partnership.
ps.
I already know the Gradle plugin has a bug concerning Not starting with the project dir as the current directory. By choice (apparently). I fix that with user.dir setting -- Unfortunately commenting that line makes no difference to the results. As long as the paths are correct.
Quite surprising. It is working.
build.gradle:
System.setProperty( "user.dir", project.rootDir.toString() )
apply plugin: 'base' // To add "clean" task to the root project.
println " user dir: ${System.getProperty('user.dir')}"
loadProperties('common.properties')
println "ends"
def loadProperties(file){
def properties = new Properties()
def propertiesFile = new File(file)
if( propertiesFile.exists() ) println 'file exists'
propertiesFile.withInputStream {
properties.load(it)
}
println properties
}
Output is seen as expected
$ gradle
Starting a Gradle Daemon (subsequent builds will be faster)
user dir: /home/rsettine/Documents/gradle
file exists
{abc=1, test=1}
ends
:help
Welcome to Gradle 3.3.
To run a build, run gradle <task> ...
To see a list of available tasks, run gradle tasks
To see a list of command-line options, run gradle --help
To see more detail about a task, run gradle help --task <task>
BUILD SUCCESSFUL
Total time: 8.263 secs
Some Snippet of my code is shown as below;
String log4jConfigPath = FileUtil.getFilePathFromClasspath("log4j.properties");
if (null != log4jConfigPath) {
PropertyConfigurator.configureAndWatch(log4jConfigPath);
} else {
logger.warn("log4j.properties not found on classpath!");
}
Config config = Config.getConfig("crawler.properties");
String mode = config.getString("crawler.mode").toLowerCase();
I m getting an error for both the files "log4j.properties" and "crawler.properties" not found in class path..i have this files residing in folders in my projects..Can someone please tell me how to add this files to class path compiler looks for both this properties files.
Thanks;
The folder, that contains log4j.properties has to be added to the classpath, either relative to your current working directory or absolute:
/
+-project
+-src
| +-com
| +-example
| +-Hello
+-resource
+-log4j.properties
now, if you current directory is /project, then you have to run your app with
java -cp src;resource com.example.Hello # relative paths
java -cp /project/src;/project/resource com.example.Hello # absolute paths