We Keep Coding

Maven build optimization - prevent building *-fat.jar locally

Right now I see this in my project:
I have a pretty optimized maven build using:
mvn -offline -T 9 package exec:java -DskipTests
the offline flag prevents it from looking for updates, uses 9 threads, and skips tests, but I wonder if there is a flag I can use to prevent it from creating the *-fat.jar?
I figure the fat.jar is a big file and if I avoid creating it until I need to, might save some time.

Maven is not creating something like "-fat.jar" by default. It mast be specific definition in the pom.xml: maven-assembly-plugin or maven-shade-plugin which do it.
So, you need to change your pom.xml: define special profiles: one(defualt) which will create "-fat.jar" and one which will not.
And then you will able to run something like "mav package -Pmy-no-fat-profile" to avoid "-fat.jar" creation.

Eclipse : disabling the warning only on the folder [duplicate]

I'm using a parser generator that creates somewhat ugly code. As a result my Eclipse project has several dozen warnings emanating from generated source files. I know I can use the #SuppressWarning annotation to suppress particular warnings in particular elements, but any annotations I add by hand will be lost when the parser generator runs again. Is there a way to configure Eclipse to suppress warnings for a particular file or directory?

Starting with version 3.8 M6, Eclipse (to be exact: the JDT) has built-in functionality for this. It is configurable through a project's build path: Project properties > Java Build Path > Compiler > Source
Announced here: Eclipse 3.8 and 4.2 M6 - New and Noteworthy, called Selectively ignore errors/warnings from source folders. That's also where the screenshot is from. This is the new feature developed on the previously linked Bug 220928.

There is a ticket for this, Bug 220928, that has since been completed for Eclipse 3.8. Please see this answer for details.
If you're stuck with Eclipse 3.7 or lower: The user "Marc" commenting on that ticket created (or at least links to) a plugin called 'warningcleaner' in comment 35. I'm using that with a lot of success while waiting for this feature to be integrated into Eclipse.
It's really quite simple:
Install plugin.
Right-click project and select "Add/remove generated code nature".
Open the project settings (right-click and select "properties").
Open the tab 'Warning Cleaner'.
Select the source folders you want to ignore the warnings from.

I solved this by using the maven regexp replace plugin - it does not solve the cause, but heals the pain:
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>maven-replacer-plugin</artifactId>
<version>1.3.2</version>
<executions>
<execution>
<phase>prepare-package</phase>
<goals>
<goal>replace</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>target/generated-sources/antlr/**/*.java</include>
</includes>
<regex>true</regex>
<regexFlags>
<regexFlag>MULTILINE</regexFlag>
</regexFlags>
<replacements>
<replacement>
<token>^public class</token>
<value>#SuppressWarnings("all") public class</value>
</replacement>
</replacements>
</configuration>
</plugin>
Note that I did not manage to get the ** notation to work, so you might have to specify path exactly.
See comment below for an improvement on how not to generate duplicate #SupressWarnings

I think the best you can do is enable project specific settings for displaying warnings.
Window -> Preferences -> Java -> Compiler -> Errors/Warnings
On the top of the form is a link for configuring project specific settings.

User #Jorn hinted at Ant code to do this. Here's what I have
<echo>Adding #SuppressWarnings("all") to ANTLR generated parser/lexer *.java</echo>
<echo> in ${project.build.directory}/generated-sources/antlr/</echo>
<replace dir="${project.build.directory}/generated-sources/antlr/"
summary="true"
includes="**/*.java"
token="public class"
value='#SuppressWarnings("all") public class' />
Note that Ant's <replace> does text replacement, not regular expression replacement,
so it cannot use the ^ meta-character in the token to match beginning of line as the maven regexp replace plugin does.
I'm doing this at the same time that I run Antlr from maven-antrun-plugin in my Maven pom, because the ANTLR maven plugin did not play well with the Cobertura maven plugin.
(I realize this is not an answer to the original question, but I can't format Ant code in a comment/reply to another answer, only in an answer)

I don't think Eclipse inherently provides a way to do this at the directory level (but I'm not sure).
You could have the generated files go into a separate Java project, and control warnings for that specific project.
I generally prefer to place automatically-generated code in a separate project anyway.

You can only suppress warnings at the project level. However, you can configure your problems tab to suppress warnings from files or packages. Go into the Configure Contents menu and work with the "On working set:" scope.

This small python script "patches" the M2E-generated .classpath files and adds the required XML tag to all source folders starting with target/generated-sources. You can just run it from you project's root folder. Obviously you need to re-run it when the Eclipse project information is re-generated from M2E. And all at your own risk, obviously ;-)
#!/usr/bin/env python
from xml.dom.minidom import parse
import glob
import os
print('Reading .classpath files...')
for root, dirs, files in os.walk('.'):
for name in files:
if (name == '.classpath'):
classpathFile = os.path.join(root, name)
print('Patching file:' + classpathFile)
classpathDOM = parse(classpathFile)
classPathEntries = classpathDOM.getElementsByTagName('classpathentry')
for classPathEntry in classPathEntries:
if classPathEntry.attributes["path"].value.startswith('target/generated-sources'):
# ensure that the <attributes> tag exists
attributesNode = None;
for attributes in classPathEntry.childNodes:
if (attributes.nodeName == 'attributes'):
attributesNode = attributes
if (attributesNode == None):
attributesNode = classpathDOM.createElement('attributes')
classPathEntry.appendChild(attributesNode)
# search if the 'ignore_optional_problems' entry exists
hasBeenSet = 0
for node in attributesNode.childNodes:
if (node.nodeName == 'attribute' and node.getAttribute('name') == 'ignore_optional_problems'):
# it exists, make sure its value is true
node.setAttribute('value','true')
#print(node.getAttribute('name'))
hasBeenSet = 1
if (not(hasBeenSet)):
# it does not exist, add it
x = classpathDOM.createElement("attribute")
x.setAttribute('name','ignore_optional_problems')
x.setAttribute('value','true')
attributesNode.appendChild(x)
try:
f = open(classpathFile, "w")
classpathDOM.writexml(f)
print('Writing file:' + classpathFile)
finally:
f.close()
print('Done.')

I'm doing this to a few ANTLR grammars, which generate a Java parser using Ant. The Ant build script adds the #SuppressWarnings("all") to one Java file, and #Override to a few methods in another.
I can look up how it's done exactly, if you're interested.

In the case of ANTLR 2, it is possible to suppress warnings in generated code by appenidng #SuppressWarnings before the class declaration in the grammar file, e.g.
{#SuppressWarnings("all")} class MyBaseParser extends Parser;

This can be done by excluding certain directories from the build path (The following example is given using Eclipse 3.5)
[1] Bring up the Java Build Path
Click on the projectin Package Explorer
Right click, properties
Select Java Build Path
[2] Add directories to exclude
The Source tab should contain details of the project source folders
Expand the source folder and locate the 'Excluded:' property
Select 'Excluded:' and click Edit
Add folders into the Exclusion patterns using the Add/Add Multiple options
Click Finish, then ok for Eclipse to rebuild.

It's been a while since I have released the warning-cleaner plugin, and now that I am using Eclipse 3.8, I have no need for it anymore.
However, for those who still need this plugin, I have released it on github with the update site on bintray.
If you are still using Eclipse 3.7 or before, this could be useful.
Check this site for installation details.

If the eclipse project is generated from gradle using Eclipse plugin's eclipse command the Selectively ignore errors/warnings from source folders option can be set by adding this on the top level of you build.gradle file:
eclipse.classpath.file {
whenMerged { classpath ->
classpath.entries.each { entry ->
if (entry.path.contains('build/generated/parser')) {
entry.entryAttributes['ignore_optional_problems'] = true
}
}
}
}
This assumes that generated sources are in build/generated/parser folder.

Is there a way to do source-to-source java refactoring in gradle?

I got some automatically generated Java code. I am willing to refactor automatically before compiling. It is mostly class rename and package modification.
Are there any gradle or ant tasks available for this?

I got some automatically generated java code I would like to
automaticaly refactor before compiling it. It is mostly class rename
and package modification.
[off topic comment: You should fix the code that generate the code. Generating code automatically and then modify it using another tool does not look like correct approach.]
Eclipse provides refactoring API that can be used in programs(without eclipse). Original tool was JDT(I had used it), I guess the new solution is LTK - not tried.

If you're trying to control the package and class names generated by xjc you can provide an external binding file.
You would say something like:
<jxb:bindings version="1.0" xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<jxb:bindings schemaLocation="your.xsd" node="/xs:schema">
<jxb:schemaBindings>
<jxb:package name="the.package.name.you.want" />
</jxb:schemaBindings>
</jxb:bindings>
</jxb:bindings>
And for each class name you want to customize you would have:
<jxb:bindings node="//xs:complexType[#name='UglyTypeName']">
<jxb:class name="MyNiceClass"/>
</jxb:bindings>
Save this up into a bindings.xjb file, for example, and run xjc like so:
xjc your.xsd -b bindings.xjb

as you said you use "xjc" to generate the code and you want "mostly class rename and package modification" maybe some of the "xjc" options would do what you want:
-p <pkg> : specifies the target package
-b <file/dir> : specify external bindings files (each <file> must have its own -b)
If a directory is given, **/*.xjb is searched
With "-p" you can define the target package to which the generated code should belong. For using the bindings option have a look here for further information http://docs.oracle.com/cd/E17802_01/webservices/webservices/docs/2.0/tutorial/doc/JAXBUsing4.html
edit Another approach could be: to move the generated files in the directory you want and then using the replace plugin to replace the package specification in the copied source files
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>copy-resources</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>the.directory.for.the.classes.to.move.to</outputDirectory>
<resources>
<resource>
<directory>the.directory.of.ajc.classes</directory>
... do here the appropriate settings ...
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.2</version>
<executions>
<execution>
<phase>process-sources</phase>
<goals>
<goal>replace</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/src/main/gen/**/**.java</include>
</includes>
<token>^package the.one.you.want.to.replace</token>
<value>package you.want.the.classes.to.be</value>
<regexFlags>
<regexFlag>MULTILINE</regexFlag>
</regexFlags>
</configuration>
</plugin>

What you want is a program transformation system (PTS).
A PTS reads source code, builds a program representation (usually an abstract syntax tree or AST), applies transformations (often stated in "if you see this, replace it by that" target langauge "surface" syntax) to the tree, and can regenerate valid source text (often prettyprinted) from the modified AST. An example surface syntax might be written literally as follows (syntax varies per system):
\x ** 2 ==> \x * \x
Usually a set of transformation rules working in cooperation, controlled by a metaprogram, are needed to achieve a more complex result. How metaprograms are written vary radically across the PTS.
You generally have to configure the PTS to parse/prettyprint the target language of choice. Stratego, TXL, and DMS (my tool) all have Java parsers/prettyprinters already and all have surface syntax rewrites. Given the choice of PTS, you would generate your source code, then launch a process to run the tool, using a set of transformations and corresponding metaprogram that you provide to achieve the specific code changes you want. Stratego, I believe, has a Java implementation, and you might be able to integrate it into your application, avoiding the separate process.
A complication is that often the transformations you want to do, require name resolution, type information, or some understanding of dataflow in the code. Stratego and TXL do not have this information built in, so you have to compute it on demand by writing additional transformations; that's actually a little hard because the language semantics are complex. Our tool, DMS, has these facilities already complete for Java with some incompleteness on the data flow.
If your problem is really just name substitution, and the names are unique in your generated code, then you might get away with transformations like:
uniquename1 ==> replacementpath1
e.g.
foo.bar ==> baz.bar.foo
(If this is really enough, you might get away with just text string substitution rather than a PTS. Most of my career has been spent discovering that nothing is ever as simple as I had hoped).
Eclipse's JDT might be an alternative. It certainly has a parser, but no surface syntax transformations so it isn't really a PTS). Instead transformations are coded in Java by walking up and down the tree and making changes using JDT APIs, so it is a bit painful. AFAIK, it provides access to name information but not expression types if I understand it, and has no specific support for data flow. I understand it isn't easy to isolate it from Eclipse for use as a module; YMMV.
There used to be a standalone tool called Spoon, that provided Java parsing, full name and type resolution, but has on procedural tree modifications. I don't know if has tracked modern dialects of Java (e.g., 1.5 and up with templates).

There is a tools named Spoon. This project is open source and enables you to transform and analyze Java source code with an approach source-to-source. It provides a complete and fine-grained Java metamodel where any program element (classes, methods, fields, statements, expressions...) can be accessed both for reading and modification.
You can have more information about it on its website: http://spoon.gforge.inria.fr/.
To run spoon with Gradle, there is plugin that you can insert on your project and apply Spoon on it. To use it, clone this project (https://github.com/SpoonLabs/spoon-gradle-plugin) and install the plugin on your local repository (./gradlew install). After, you can insert it on your project how is explained in the documentation of the plugin (https://github.com/SpoonLabs/spoon-gradle-plugin#basic-usage).
I hope this can help any new developer interested in the java transformation source to source! :)

You can do it after compiling with proguard. There is a gradle task for proguard.

Maven - synch "main" folder with "tests" folder

I'm just starting to use Maven with my project. All of my production code is of course inside the main directory. As I add unit tests to my tests directory, is there a way to synchronize the main dir with my tests dir?
For example, say I add a new package org.bio.mynewpackage. I have to go in my main folder and add the same package name... this is rather annoying.
Any ideas?

Here is a Groovy Script embedded in a GMaven plugin execution. It does exactly what you are asking for.
<plugin>
<groupId>org.codehaus.gmaven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<id>mirror-folder-structure</id>
<phase>generate-test-sources</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
<source>
<![CDATA[
static void createShadow(File base, File shadow){
if(base.exists()&&!shadow.exists())shadow.mkdirs();
base.eachDir { createShadow(it, new File(shadow, it.name))};
}
createShadow(pom.build.sourceDirectory,pom.build.testSourceDirectory);
]]>
</source>
</configuration>
</execution>
</executions>
</plugin>
The problem is: it won't run automatically. I have bound it to the phase generate-test-sources, but you can choose any other phase. You will however have to execute that phase manually, e.g. mvn generate-test-sources.
If you would however consider using Eclipse with the m2eclipse plugin, m2eclipse lets you define lifecycle phases that it runs automatically when you have saved a file, so that would be easier.

I don't know if I get your problem right: usually, by convention, maven will detect all class files in main/java and all class files in test/java. You don't have to declare the package names.
So if you add an new package an classes to "main", they will be compiled and packaged, if you add some new tests to "test", the will be autodiscovered and executed in the test phase.

Typically I would rely on the IDE to do this when I create tests.
Eg:
I create a new class org.bio.mynewpackage.MyNewClass in main/.
Now when I create a test org.bio.mynewpackage.MyNewClassTest, the IDE should automatically create the necessary directory tree.

I don't know if you were looking for something specific regards how maven might help you do this. Still i've always used rsync to match to target folders.
Something along the lines of:
rsync -Crit ./source ./target
where C ignores versioning
files/folders such as .svn
r is recursion i is information output.
t is timestamp. i've always put this
to ensure differences in files are
based on time stamp.
Add 'n' to run in test mode, it will output what will change rather than actually do it. Always do this first as rsync can totally mess things up if you don't have it right.
You can also add pattern matching rules, either in a file in each directory or once in the command line.

I don't know of any plugin that does this, but it should be pretty easy to write one. It could possibly even be done with some simple Groovy scripting using gmaven-plugin or the like.
Alternatively, this shell command should do what you want:
# ! -wholename '*/.*' excludes hidden dirs like .svn
$( cd src/main/java/ && find -type d ! -wholename '*/.*' -exec mkdir -p ../../test/java/{} \; )
EDIT:
Here's a simple Maven plugin (this plugin sorts entries of eclipse .classpath files by name) that should give you a quick start into Maven plugin development.

How to suppress Java warnings for specific directories or files such as generated code

I'm using a parser generator that creates somewhat ugly code. As a result my Eclipse project has several dozen warnings emanating from generated source files. I know I can use the #SuppressWarning annotation to suppress particular warnings in particular elements, but any annotations I add by hand will be lost when the parser generator runs again. Is there a way to configure Eclipse to suppress warnings for a particular file or directory?

Starting with version 3.8 M6, Eclipse (to be exact: the JDT) has built-in functionality for this. It is configurable through a project's build path: Project properties > Java Build Path > Compiler > Source
Announced here: Eclipse 3.8 and 4.2 M6 - New and Noteworthy, called Selectively ignore errors/warnings from source folders. That's also where the screenshot is from. This is the new feature developed on the previously linked Bug 220928.

There is a ticket for this, Bug 220928, that has since been completed for Eclipse 3.8. Please see this answer for details.
If you're stuck with Eclipse 3.7 or lower: The user "Marc" commenting on that ticket created (or at least links to) a plugin called 'warningcleaner' in comment 35. I'm using that with a lot of success while waiting for this feature to be integrated into Eclipse.
It's really quite simple:
Install plugin.
Right-click project and select "Add/remove generated code nature".
Open the project settings (right-click and select "properties").
Open the tab 'Warning Cleaner'.
Select the source folders you want to ignore the warnings from.

I solved this by using the maven regexp replace plugin - it does not solve the cause, but heals the pain:
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>maven-replacer-plugin</artifactId>
<version>1.3.2</version>
<executions>
<execution>
<phase>prepare-package</phase>
<goals>
<goal>replace</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>target/generated-sources/antlr/**/*.java</include>
</includes>
<regex>true</regex>
<regexFlags>
<regexFlag>MULTILINE</regexFlag>
</regexFlags>
<replacements>
<replacement>
<token>^public class</token>
<value>#SuppressWarnings("all") public class</value>
</replacement>
</replacements>
</configuration>
</plugin>
Note that I did not manage to get the ** notation to work, so you might have to specify path exactly.
See comment below for an improvement on how not to generate duplicate #SupressWarnings

I think the best you can do is enable project specific settings for displaying warnings.
Window -> Preferences -> Java -> Compiler -> Errors/Warnings
On the top of the form is a link for configuring project specific settings.

User #Jorn hinted at Ant code to do this. Here's what I have
<echo>Adding #SuppressWarnings("all") to ANTLR generated parser/lexer *.java</echo>
<echo> in ${project.build.directory}/generated-sources/antlr/</echo>
<replace dir="${project.build.directory}/generated-sources/antlr/"
summary="true"
includes="**/*.java"
token="public class"
value='#SuppressWarnings("all") public class' />
Note that Ant's <replace> does text replacement, not regular expression replacement,
so it cannot use the ^ meta-character in the token to match beginning of line as the maven regexp replace plugin does.
I'm doing this at the same time that I run Antlr from maven-antrun-plugin in my Maven pom, because the ANTLR maven plugin did not play well with the Cobertura maven plugin.
(I realize this is not an answer to the original question, but I can't format Ant code in a comment/reply to another answer, only in an answer)

I don't think Eclipse inherently provides a way to do this at the directory level (but I'm not sure).
You could have the generated files go into a separate Java project, and control warnings for that specific project.
I generally prefer to place automatically-generated code in a separate project anyway.

You can only suppress warnings at the project level. However, you can configure your problems tab to suppress warnings from files or packages. Go into the Configure Contents menu and work with the "On working set:" scope.

This small python script "patches" the M2E-generated .classpath files and adds the required XML tag to all source folders starting with target/generated-sources. You can just run it from you project's root folder. Obviously you need to re-run it when the Eclipse project information is re-generated from M2E. And all at your own risk, obviously ;-)
#!/usr/bin/env python
from xml.dom.minidom import parse
import glob
import os
print('Reading .classpath files...')
for root, dirs, files in os.walk('.'):
for name in files:
if (name == '.classpath'):
classpathFile = os.path.join(root, name)
print('Patching file:' + classpathFile)
classpathDOM = parse(classpathFile)
classPathEntries = classpathDOM.getElementsByTagName('classpathentry')
for classPathEntry in classPathEntries:
if classPathEntry.attributes["path"].value.startswith('target/generated-sources'):
# ensure that the <attributes> tag exists
attributesNode = None;
for attributes in classPathEntry.childNodes:
if (attributes.nodeName == 'attributes'):
attributesNode = attributes
if (attributesNode == None):
attributesNode = classpathDOM.createElement('attributes')
classPathEntry.appendChild(attributesNode)
# search if the 'ignore_optional_problems' entry exists
hasBeenSet = 0
for node in attributesNode.childNodes:
if (node.nodeName == 'attribute' and node.getAttribute('name') == 'ignore_optional_problems'):
# it exists, make sure its value is true
node.setAttribute('value','true')
#print(node.getAttribute('name'))
hasBeenSet = 1
if (not(hasBeenSet)):
# it does not exist, add it
x = classpathDOM.createElement("attribute")
x.setAttribute('name','ignore_optional_problems')
x.setAttribute('value','true')
attributesNode.appendChild(x)
try:
f = open(classpathFile, "w")
classpathDOM.writexml(f)
print('Writing file:' + classpathFile)
finally:
f.close()
print('Done.')

I'm doing this to a few ANTLR grammars, which generate a Java parser using Ant. The Ant build script adds the #SuppressWarnings("all") to one Java file, and #Override to a few methods in another.
I can look up how it's done exactly, if you're interested.

In the case of ANTLR 2, it is possible to suppress warnings in generated code by appenidng #SuppressWarnings before the class declaration in the grammar file, e.g.
{#SuppressWarnings("all")} class MyBaseParser extends Parser;

This can be done by excluding certain directories from the build path (The following example is given using Eclipse 3.5)
[1] Bring up the Java Build Path
Click on the projectin Package Explorer
Right click, properties
Select Java Build Path
[2] Add directories to exclude
The Source tab should contain details of the project source folders
Expand the source folder and locate the 'Excluded:' property
Select 'Excluded:' and click Edit
Add folders into the Exclusion patterns using the Add/Add Multiple options
Click Finish, then ok for Eclipse to rebuild.

It's been a while since I have released the warning-cleaner plugin, and now that I am using Eclipse 3.8, I have no need for it anymore.
However, for those who still need this plugin, I have released it on github with the update site on bintray.
If you are still using Eclipse 3.7 or before, this could be useful.
Check this site for installation details.

If the eclipse project is generated from gradle using Eclipse plugin's eclipse command the Selectively ignore errors/warnings from source folders option can be set by adding this on the top level of you build.gradle file:
eclipse.classpath.file {
whenMerged { classpath ->
classpath.entries.each { entry ->
if (entry.path.contains('build/generated/parser')) {
entry.entryAttributes['ignore_optional_problems'] = true
}
}
}
}
This assumes that generated sources are in build/generated/parser folder.