From a high level point of view I am looking to create a centralised list of libraries/dependencies that are either out of date and should not be used or can be used under warning.
We are using SonarQube for code inspection/coverage and my preference would be to use work the "Blacklist" into here through rules/analysis. This would be the preferred option as it is indeed centralised and allows the selection of severity e.g a library totally blacklisted as opposed to a warning. Though I have seen this mentioned on forums I have not seen a practical example of doing it.
My question so is there a way of doing this and if so how? All other opinions are welcomed.
I have looked at mavens bannedDependencies feature but I don't like the fact that it would break a build, allow the developer to edit and is not centralised.
The Disallowed dependencies should not be used rule template is what you want. It will let you specify dependencies by group and artifact id pattern, with or without version numbers.
This rule is available from version 3.10 of the Java plugin.
Related
I want to modify / make the rule target only public interfaces (not public classes etc). Is this possible ? Im using this rule in Java code but its too strict for my project and I would love to know if there is a way to change it a little bit.
Link for rule: https://rules.sonarsource.com/java/RSPEC-1213
For an existing ruleset on SonarQube, talk to your sonar administrator to change the rules that are enforced on the code and remove that particular one from global enforcement.
There have been a few times I've gone to the admins of the tool for the install that I use and said "this rule isn't one that I care about or will enforce and only makes it confusing" and had them remove that rule from the globally run ruleset.
Is it possible to write your own rule?
Yes, it is possible. From SonarQube's docs: Adding coding rules you have some options. Either you can write a plugin for SonarQube and add that to your instance (docs), or you can write an external application that analyzes the code which SonarQube consumes.
If you don't have your own instance of sonarqube or aren't up to writing the associated plugin or external tooling... you might want to instead lookout PMD (site).
For PMD, writing a custom rule can be much simpler (docs). One of the ways that PMD works is by 'compiling' the Java code into an XML representation of the abstract syntax tree for Java and then running xpath queries against that XML (tutorial).
The xpath rule can then be included in a project's configuration.
What about turning it off for the code that I'm working on?
If a specific rule is one that you don't want to invoke, you could suppress it with #SuppressWarnings("java:S106") (that particular spares warnings is for System.out.println use, but the same structure can be used for other warnings) or by adding // NOSONAR too strict on the line. There are spots where I have such comments where following the rule for a particular set of code is problematic and suppress it for that line, method, or class - with the comment about why that is done.
That particular rule... I'm gonna agree with the Java (and now Oracle) guidelines and follow it. The reason is that if anyone else works on the code, they'll expect it to follow that convention. Having a consistent understanding of what things should be where in code so that another developer doesn't need to go dig through an entire file to find the constructor when it is expected to be at the top (under the field definition) is a good thing. What's more, it limits the future cases where a developer goes through to make things consistent with conventions and results in a lot of style: updating code to follow style guide commits later.
Just as shown in the picture, one app (Java) referenced two third-party package jars (packageA and packageB), and they referenced packageC-0.1 and packageC-0.2 respectively. It would work well if packageC-0.2 was compatible with packageC-0.1. However sometimes packageA used something that could not be supported in packageC-0.2 and Maven can only use the latest version of a jar. This issue is also known as "Jar Hell".
It would be difficult in practice to rewrite package A or force its developers to update packageC to 0.2.
How do you tackle with these problems? This often happens in large-scale companies.
I have to declare that this problem is mostly occurred in BIG companies due to the fact that big company has a lot of departments and it would be very expensive to let the whole company update one dependency each time certain developers use new features of new version of some dependency jars. And this is not big deal in small companies.
Any response will be highly appreciated.
Let me throw away a brick in order to get a gem first.
Alibaba is one of the largest E-Commerces in the world. And we tackle with these problems by creating an isolation container named Pandora. Its principle is simple: packaging those middle-wares together and load them with different ClassLoaders so that they can work well together even they referenced same packages with different versions. But this need a runtime environment provided by Pandora which is running as a tomcat process. I have to admit that this is a heavy plan. Pandora is developed based on a fact that JVM identifies one class by class-loader plus classname.
If you know someone maybe know the answers, share the link with him/her.
We are a large company and we have this problem a lot. We have large dependency trees that over several developer groups. What we do:
We manage versions by BOMs (lists of Maven dependencyManagement) of "recommended versions" that are published by the maintainers of the jars. This way, we make sure that recent versions of the artifacts are used.
We try to reduce the large dependency trees by separating the functionality that is used inside a developer group from the one that they offer to other groups.
But I admit that we are still trying to find better strategies. Let me also mention that using "microservices" is a strategy against this problem, but in many cases it is not a valid strategy for us (mainly because we could not have global transactions on databases any more).
This is a common problem in the java world.
Your best options are to regularly maintain and update dependencies of both packageA and packageB.
If you have control over those applications - make time to do it. If you don't have control, demand that the vendor or author make regular updates.
If both packageA and packageB are used internally, you can use the following practise: have all internal projects in your company refer to a parent in the maven pom.xml that defines "up to date" versions of commonly used third party libraries.
For example:
<framework.jersey>2.27</framework.jersey>
<framework.spring>4.3.18.RELEASE</framework.spring>
<framework.spring.security>4.2.7.RELEASE</framework.spring.security>
Therefore, if your project "A" uses spring, if they use the latest version of your company's "parent" pom, they should both use 4.3.18.RELEASE.
When a new version of spring is released and desirable, you update your company's parent pom, and force all other projects to use that latest version.
This will solve many of these dependency mismatch issues.
Don't worry, it's common in the java world, you're not alone. Just google "jar hell" and you can understand the issue in the broader context.
By the way mvn dependency:tree is your friend for isolating these dependency problems.
I agree with the answer of #JF Meier ，In Maven multi-module project, the dependency management node is usually defined in the parent POM file when doing unified version management. The content of dependencies node declared by the node class is about the resource version of unified definition. The resources in the directly defined dependencies node need not be introduced into the version phase. The contents of the customs are as follows:
in the parent pom
<dependencyManagement>
<dependencies >
<dependency >
<groupId>com.devzuz.mvnbook.proficio</groupId>
<artifactId>proficio-model</artifactId>
<version>${project.version}</version>
</dependency >
</dependencies >
</dependencyManagement>
in your module ,you do not need to set the version
<dependencies >
<dependency >
<groupId>com.devzuz.mvnbook.proficio</groupId>
<artifactId>proficio-model</artifactId>
</dependency >
</dependencies >
This will avoid the problem of inconsistency .
This question can't be answered in general.
In the past we usually just didn't use dependencies of different versions. If the version was changed, team-/company-wide refactoring was necessary. I doubt it is possible with most build tools.
But to answer your question..
Simple answer: Don't use two versions of one dependency within one compilation unit (usually a module)
But if you really have to do this, you could write a wrapper module that references to the legacy version of the library.
But my personal opinion is that within one module there should not be the need for these constructs because "one module" should be relatively small to be manageable. Otherwise it might be a strong indicator that the project could use some modularization refactoring. However, I know very well that some projects of "large-scale companies" can be a huge mess where no 'good' option is available. I guess you are talking about a situation where packageA is owned by a different team than packageB... and this is generally a very bad design decision due to the lack of separation and inherent dependency problems.
First of all, try to avoid the problem. As mentioned in #Henry's comment, don't use 3rd party libraries for trivial tasks.
However, we all use libraries. And sometimes we end up with the problem you describe, where we need two different versions of the same library. If library 'C' has removed and added some APIs between the two versions, and the removed APIs are needed by 'A', while 'B' needs the new ones, you have an issue.
In my company, we run our Java code inside an OSGi container. Using OSGi, you can modularize your code in "bundles", which are jar files with some special directives in their manifest file. Each bundle jar has its own classloader, so two bundles can use different versions of the same library. In your example, you could split your application code that uses 'packageA' into one bundle, and the code that uses 'packageB' in another. The two bundles can call each others APIs, and it will all work fine as long as your bundles do not use 'packageC' classes in the signature of the methods used by the other bundle (known as API leakage).
To get started with OSGi, you can e.g. take a look at OSGi enRoute.
Let me throw away a brick in order to get a gem first.
Alibaba is one of the largest E-Commerces in the world. And we tackle with these problems by creating an isolation container named Pandora. Its principle is simple: packaging those middle-wares together and load them with different ClassLoaders so that they can work well together even they referenced same packages with different versions. But this need a runtime environment provided by Pandora which is running as a tomcat process. I have to admit that this is a heavy plan.
Pandora is developed based on a fact that JVM identifies one class by class-loader plus classname.
I am starting a new project which might be open-sourced later on and/or at least get some external contributors during its life-time.
I am now thinking about what the best approach to code-style / auto-formatting would be. I am a strong supporter of only having auto-formatted code committed to a project, as this eliminates the differences between individual developers and helps keeping individual commits clutter-free of reformatting issues.
My first approach was to use Eclipse built-in style for the project, but I really don't like the default style, because I think line-break at 80 characters is way out-dated for today's screen resolutions. Also, as the name suggests, it's available only for people using Eclipse as IDE.
So I was also thinking about using my own formatter settings and checking the exported settings into the project's repository so that any contributor can pick them up. Again, this would force most people to use Eclipse, as I am not aware of any formatting definition that can be read by multiple IDEs.
Any hint how this is handled in other projects? I searched some github repositories, but to me it seems that this issue is more or less ignored by a lot of projects.
I do understand that this question may be border-line for Stack Overflow, as I don't know if a definite answer is possible and if this triggers a discussion, but it is something I often struggle with when starting a new project.
While screens grow wider, they don't seem to grow taller.
Whatever you other drivers are, preserve vertical space. Put { and } on lines containing other language key words, if you can.
In any case, use a maven plugin or other automated tool in your compile chain to enforce the rules that you care about. That way they are unambiguous.
Also don't create too many rules that don't matter. Each rule costs time to make the code comply.
I understand your concern and in my opinion the best approach is to create code formatting preference file which can be shared along with the project.
For example in eclipse Using a file explorer, navigate to //.settings and copy org.eclipse.jdt.core.prefs to a new location. This file contains all your formatting settings. Hence this can be shared to maintain the code formatting consistencies.
If not that then you might have to rely on the editor specific code formatting.
I definitely look forward to other expert opinion on the same if what I have shared is not optimal as per the requirement.
I have a Java project, built with Maven, that aggregates several components, each one in its own Maven project. Any one of these components may evolve separately.
The structure of my project can be described as follows:
my-main-project that depends on:
my-component-1
my-component-2
etc.
Nowadays, all pom.xml are using "snapshot" versions, so, they are all using the "latest" version available in my repository.
But once I send a release version to my customer, I'm supposed to freeze the versions and make a TAG (or equivalent) in my source-control, so I can restore a previous state in case of maintenance.
So, my question is: should I change all pom.xml files before each release, give version numbers to the components, and tie everything with this dependency versions? Also, if I have many components (my project currenty has 30+ small subcomponents) would I have to renumber/reversion each one before each release? When a single component evolves (due to bug fix or enhancement), must I increase its version so that the changes do not affect pre-existing releases, right?
How people using maven generally handle this many-component versioning case?
Of course, I could just rely on my version-control tags to restore to a previous point-in-time and just tag every component on each release, but I don't like this approach, since the dependency versioning (with maven) gives me much more control and visibility about what is packaged, and relations of (broken-)compatibility and many more.
General Considerations
You may consider some relations between your components.
Are they really independant (each one vs each other) ? Or is there some kinds of relation ... some commons lifecycles ?
If you find some relationship between them, consider using maven multi-modules : http://www.sonatype.com/books/mvnex-book/reference/multimodule.html. In a few words, you will have a parent, with one version, and some modules (some jars .. in a way like Spring and its submodules). This will help you to reduce versions management.
You may consider using maven-release-plugin. It will help you to tag, build and deploy automatically your modules, dealing more easily with versionning and links with SCM and Repository.
Moreover, combine with multi-module it would drastically help you !
There is a lot of topic dealing with this on Stack Overflow.
I don't know if you already know that. I could explain it a lot further if you want, but you may have enough elements to search by yourself if you don't.
Straight Answers
So, my question is: should I change all pom.xml files before each release, give version numbers to the components, and tie everything with this dependency versions?
Yes you should. In Application Lifecycle Management follow the changes is REALLY important. So, as you could imagine, and as you point it out, you really should build and tag each of your components. It could be painful, but maven-realease-plugin and multi module (even with a Continuous Integration plateform) it could be easier.
would I have to renumber/reversion each one before each release?
For exactly the same reasons : yes !
must I increase its version so that the changes do not affect pre-existing releases, right?
Yes, you should too. Assuming you choose a common versionning like MAJOR.minor.correction, the first number indicate compatibilty breaks. Minor version would bring some breaks, but should not. Corrections whould NEVER affect compatibility.
How people using maven generally handle this many-component versioning case?
I cannot reply for every one, but my previous comments on release-plugin and multi-module considered as best pratices. If you want to a little bit further, you can imagine use more powerfull SCM (Clearcase, Perforce, ...), but maven integration is fewer, not "well" documented and community provide less examples than SVN or Git.
Maven Release Plugin
If you are using a multi-module pom.xml you should be able to do mvn release -DautoVersionSubmodules and have it do a "release" build of all your dependencies and remove the -SNAPSHOT versions and upload them to your repository. That is what the release plugin and its workflow exists solely to do.
I have a Java-based server, transmitting data from many remote devices to one app via TCP/IP. I need to develop several versions of it. How can I develop and then dwell them without need in coding for 2 projects?I'm asking not only for that project, but for different approaches.
Where the behaviour differs, make the behaviour "data driven" - typically by externalizing the data the drives the behaviour to properties files that are read at runtime/startup.
The goal is to have a single binary whose behaviour varies depending on the properties files found in the runtime environment.
Java supports this pattern through the Properties class, which offers convenient ways of loading properties. In fact, most websites operate in this way, for example the production database user/pass details are never (should never be) in the code. The sysadmins will edit a properties file that is read at start up, and which is protected by the operating system's file permissions.
Other options are to use a database to store the data that drives behaviour.
It can be a very powerful pattern, but it can be abused too, so some discretion is advised.
I think you need to read up on Source Control Management (SCM) and Version Control Systems (VCS).
I would recommend setting up a git or Subversion repository and adding the code initially to trunk and then branching it off to the number of branches (versions you'll be working on).
The idea of different versions is this:
You're developing your code and have it in your SCM's trunk (or otherwise known as a HEAD). At some point you consider the code stable enough for a release. You therefore create a tag (let's call it version 1.0). You cannot (should not) make changes to tags -- they're only there as a marker in time for you. If you have a client who has version 1.0 and reports bugs which you would like to fix, you create a branch based on a copy of your tag. The produced version would (normally) be 1.x (1.1, 1.2, etc). When you're done with your fixes, you tag again and release the new version.
Usually, most of the development happens on your trunk.
When you are ready with certain fixes, or know that certain fixes have already been applied to your trunk, you can merge these changes to other branches, if necessary.
Make any other version based on previous one by reusing code base, configurations and any other asset. In case if several versions should be in place at one time use configuration management practices. Probably you should consider some routing activities and client version checks on server side. This is the place where 'backward compatibility' comes to play.
The main approach is first to find and extract the code that won't change from one version to another. The best is to maximize this part to share the maximum of code base and to ease the maintenance (correcting a bug for one means correcting for all).
Then it depends on what really changes from one version to another. The best is that on the main project you can use some abstract classes or interfaces that you will be able to implement for each specific project.