I am trying to incorporate swagger-codegen in my new greenfield project, using Java (jaxrs-jersey2).
There are a lot of resources out there already documenting various parts of the project; however, I still haven't been able to find out any high-level advice on the best workflow to use with these tools.
As I understand it, swagger-codegen will be able to generate client-side code to interact with my API, such that I don't have to write this myself. This will happen by looking at a swagger.yaml (2.0) or openapi.yaml (3.0) file. This part is clear.
However, there seems to be multiple ways of generating this specification file. As I understand it, there are two primary ways:
Write a server implementation using a combination of jaxrs and Swagger annotations - have a maven plugin run as part of the compile step, generating a swagger.yaml specification file to be used by the client-generation plugin.
Write a swagger.yaml specification first, and generate server-stub code for Jersey, implementing only business logic, separate from all server boilerplate.
Which of these two ways is the recommended workflow? It sounds like (2) means writing less code and focusing on just application logic, without worrying too much about Jersey-specific glue to make the API work. This also means that the single source of truth for the API becomes a simple yaml file, rather than a bunch of Jersey code.
However, I'm unsure how to properly set this up:
Does my build need to have a compilation phase where server stubs are constantly regenerated?
Do I simply extend from the generated server stub and never worry about annotating API paths with #Path, #GET, etc?
Am I misunderstanding the use-case for server-stub generation? I.e. is the first approach (Jersey code-first) more appropriate?
If there is no real difference between the two approaches, when would you pick (1) over (2) and vice-versa?
Many thanks.
Are there any libraries out there that can generate the JSON/YAML file by analysing static code and generating the JSON/YAML files off of that?
Right now we're producing the Swagger files once the project has finished building. We hit the url at /api/swagger.yaml and do what we need with the file (this adds quite a bit of complexity to our automated builds)
Yes so as you are sort of doing already you can stick that in your CI which would startup this dummy Spring app then make a HTTP request to download the JSON/YAML then shut it down and get the file, as far as I am aware there are no tools that do this at compile time for Java and that's as good as it gets.
The other way is if you do things in opposite where you first design your API interface in swagger and then you generate code (your interfaces) which you then implement yourself.
From the sounds of it you want to be designing your API first, this will allow you to put it in version control and this will make it easier to manage your API versions life cycle and track it's changes, otherwise it can be quite difficult to see how things have changed if you have to search through annotated methods in code, that is what swagger is designed for.
You don't necessarily need to do codegen from your swagger as I have found this to be very awkward at times and it is extremely dumb and generating code, it's just moustache templating and IMO the tool is fundamentally flawed for this reason, but as my most hated saying goes:
it's free so you can't complain
I'm developing a library that needs to access layout items of the app implementing it. The only way I know how to do it is with reflection. In other words, if I create a constructor to my Library API like this:
public MyLibraryAPI(String packageName) {
Class appR = Class.forName(String.format("%s.R", packageName));
...
}
And the developer would instantiate the library with his package name as the parameter in the constructor.
What I ultimately need is to let my inner classes know the Android Views used in the developers layout (.xml files) - both the id and the type. Is there a way to achieve this without reflection and escape the performance overhead? I'm certain it cannot be done, but asking in case there's an expert that sees what I fail to notice.
EDIT: Additionally, proguard, by default, obfuscates the code for protection but, as a consequence, fails to provide JVM with means to achieve reflection at runtime, so if I use reflection I would have to ask the developer to turn off proguard obfuscation for his or her R class which is a bummer.
Reflection on Android is extremely costly. Some well-meaning and popular libraries like Roboguice have fallen over partly because of the performance cost of reflection.
I suspect some kind of code generation is the correct solution here. Dagger 2, Butter Knife and the Data Binding Library are successful examples of Android libraries that employ code generation. Since the data binding library performs inspections on the XML, it must be available to code generation libraries at that stage in the build and you may be able to base your implementation on that: here's a link to the source jars at Maven Central.
Apart from that, yes it seems there will be some compromise between ease of use and difficulty-to-implement. If you force your consumers to annotate their classes with your annotations, it becomes harder to use but probably much easier for you to implement. If you restrict yourself to inspecting XML and the generated R file and generating code from just that then I think your job will be a lot more difficult. On the other hand, using annotations has become rather commonplace and it may not be such an issue with your users.
Good luck!
Most of the time, I don't like Javascript and would prefer strict and compiled languages like Scala, Java, Haskell...
However, one thing that can be nice with Javascript is to be able to easily change code of external dependencies. For exemple, if you have a bug and you think it's one of your dependency library you can easily hack around and swap a library method by your own override and check if it's better. You can even add methods to Array ou String prototypes and things like that... One could even go to node_modules and alter the library code here temporarily if he wants to.
In the JVM world this seems to me like an heavy process to just get started:
Clone the dependency sources
Hack it
Compile it
Publish it to some local maven/ivy repository
Integrate the fixed version in your project
This is a pain, I just don't want to do that more than once in a year
Today I was trying to fix a bug in my app, and the lib did not provide me enough information. I would have loved to just be able to put a Logger on one line of that lib to have better insight of what was happening but instead I tried to hack with the debugger with no success (the bug was not reproductible on my computer anyway...)
Isn't there any simple alternative for rapidly altering the code of a dependency?
I would be interested in any solution for Scala, Java, Clojure or any other JVM language.
I'm not looking for a production-deployable solution, just a quick solution to use locally and eventually deployable on a test env.
Edit: I'm talking about library internals that are not intended to be modified by the library author. Please assume that the class to change is final, not replaceable by library configuration, and not injectable by any way into the library.
In Clojure you can re-bind vars, also from other namespaces, by using intern. So as long as the code you want to alter is Clojure code, that's a possible way to monkeypatch.
(intern 'user 'inc dec)
(inc 1)
=> 0
This is not something to do lightly though, since it can and will lead to problems with other code not expecting this behavior. It can be handy to use during development to temporarily fix edge cases or bugs in other libraries, but don't use it in published libraries or production code.
Best to simply fork and fix these libraries, and send a pull request to have it fixed in the original library.
When you're writing a library yourself that you expect people need to extend or overload, implement it in Clojure protocols, where these changes can be restricted to the extending/overloading namespaces only.
I disagree that AspectJ is difficult to use, it, or another bytecode manipulation library is your only realistic alternative.
Load-time weaving is a definite way around this issue. Depending on how you're using the class in question you might even be able to use a mocking library to achieve the same results, but something like AspectJ, which is specifically designed for augmentation and manipulation, would likely be the easiest.
I've recently been more and more frustrated with a problem I see emerging in my projects code-base.
I'm working on a large scale java project that has >1M lines of code. The interfaces and class structure are designed very well and the engineers writing the code are very proficient. The problem is that in an attempt to make the code cleaner people write Utility classes whenever they need to reuse some functionality, as a result over time and as the project grows more and more utility methods crop up. However, when the next engineer comes across the need for the same functionality he has no way of knowing that someone had already implemented a utility class (or method) somewhere in the code and implements another copy of the functionality in a different class. The result is a lot of code duplication and too many utility classes with overlapping functionality.
Are there any tools or any design principles which we as a team can implement in order to prevent the duplication and low visibility of the utility classes?
Example: engineer A has 3 places he needs to transform XML to String so he writes a utility class called XMLUtil and places a static toString(Document) method in it. Engineer B has several places where he serializes Documents into various formats including String, so he writes a utility class called SerializationUtil and has a static method called serialize(Document) which returns a String.
Note that this is more than just code-duplication as it is quite possible that the 2 implementations of the above example are different (say one uses transformer API and the other uses Xerces2-J) so this can be seen as a "best-practices" problem as well...
Update: I guess I better describe the current environment we develop in.
We use Hudson for CI, Clover for code coverage and Checkstyle for static code analysis.
We use agile development including daily talks and (perhaps insufficient) code reviews.
We define all our utility classes in a .util which due to it's size now has 13 sub-packages and about 60 classes under the root (.util) class. We also use 3rd party libraries such as most of the apache commons jars and some of the jars that make up Guava.
I'm positive that we can reduce the amount of utilities by half if we put someone on the task of refactoring that entire package, I was wondering if there are any tools which can make that operation less costly, and if there are any methodologies which can delay as much as possible the problem from recurring.
A good solution to this problem is to start adding more object-orientation. To use your example:
Example: engineer A has 3 places he needs to transform XML to String so he writes a utility class called XMLUtil and places a static toString(Document) method in it
The solution is to stop using primitive types or types provided by the JVM (String, Integer, java.util.Date, java.w3c.Document) and wrap them in your own project-specific classes. Then your XmlDocument class can provide a convenient toString method and other utility methods. Your own ProjectFooDate can contain the parsing and formatting methods that would otherwise end up in various DateUtils classes, etc.
This way, the IDE will prompt you with your utility methods whenever you try to do something with an object.
Your problem is a very common one. And a real problem too, because there is no good solution.
We are in the same situation here, well I'd say worse, with 13 millions line of code, turnover and more than 800 developers working on the code. We often discuss about the very same problem that you describe.
The first idea - that your developers have already used - is to refactor common code in some utility classes. Our problem with that solution, even with pair programming, mentoring and discussion, is that we are simply too many for this to be effective. In fact we grow in subteams, with people sharing knowledge in their subteam, but the knowledge doesn't transit between subteams. Maybe we are wrong but I think that even pair programming and talks can't help in this case.
We also have an architecture team. This team is responsible to deal with design and architecture concerns and to make common utilities that we might need. This team in fact produces something we could call a corporate framework. Yes, it is a framework, and sometimes it works well. This team is also responsible to push best practices and to raise awareness of what should be done or not, what is available or what is not.
Good core Java API design is one of the reason for Java success. Good third party open sources libraries count a lot too. Even a small well crafted API allows to offer a really useful abstraction and can help reduce code size a lot. But you know, making framework and public API is not the same thing at all as just coding an utility class in 2 hours. It has a really high cost. An utility class costs 2 hours for the initial coding, maybe 2 days with debugging and unit tests. When you start sharing common code on big projects/teams, you really make an API. You must ensure perfect documentation then, really readable and maintainable code. When you release new version of this code, you must stay backward compatible. You have to promote it company wide (or at least team wide). From 2 days for your small utility class you grow to 10 days, 20 days or even 50 days for a full-fledged API.
And your API design may not be so great. Well, it is not that your engineers are not bright - indeed they are. But are you willing to let them work 50 days on a small utility class that just help parsing number in a consistent way for the UI? Are you willing to let them redesign the whole thing when you start using a mobile UI with totally different needs? Also have you noticed how the brightest engineers in the word make APIs that will never be popular or will fade slowly? You see, the first web project we made used only internal frameworks or no framework at all. We then added PHP/JSP/ASP. Then in Java we added Struts. Now JSF is the standard. And we are thinking about using Spring Web Flow, Vaadin or Lift...
All I want to say is that there is no good solution, the overhead grows exponentially with code size and team size. Sharing a big codebase restricts your agility and responsiveness. Any change must be done carefully, you must think of all potential integration problems and everybody must be trained of the new specificities and features.
But the main productivity point in a software company is not to gain 10 or even 50 lines of code when parsing XML. A generic code to do this will grow to a thousand lines of code anyway and recreates a complex API that will be layered by utility classes. When the guy make an utility class for parsing XML, it is good abstraction. He give a name to one dozen or even one hundred lines of specialized code. This code is useful because it is specialized. The common API allows to work on streams, URL, strings, whatever. It has a factory so you can choose you parser implementation. The utility class is good because it work only with this parser and with strings. And because you need one line of code to call it. But of course, this utility code is of limited use. It works well for this mobile application, or for loading XML configuration. And that's why the developer added the utility class for it in the first place.
In conclusion, what I would consider instead of trying to consolidate the code for the whole codebase is to split code responsibility as the teams grow:
transform your big team that work on one big project into small teams that work on several subprojects;
ensure that interfacing is good to minimize integration problems, but let team have their own code;
inside theses teams and corresponding codebases, ensure you have the best practices. No duplicate code, good abstractions. Use existing proven APIs from the community. Use pair programming, strong API documentation, wikis... But you should really let different teams make their choices, build their own code, even if this means duplicate code across teams or different design decisions. You know, if the design decisions are different this may be because the needs are different.
What you are really managing is complexity. In the end if you make one monolithic codebase, a very generic and advanced one, you increase the time for newcomers to ramp up, you increase the risk that developers will not use your common code at all, and you slow down everybody because any change has far greater chances to break existing functionality.
There are several agile/ XP practices you can use to address this, e.g.:
talk with each other (e.g. during daily stand-up meeting)
pair programming/ code review
Then create, document & test one or several utility library projects which can be referenced. I recommend to use Maven to manage dependecies/ versions.
You might consider suggesting that all utility classes be placed in a well organized package structure like com.yourcompany.util.. If people are willing to name sub packages and classes well, then at least if they need to find a utility, they know where to look. I don't think there is any silver bullet answer here though. Communication is important. Maybe if a developer sends a simple email to the rest of the development staff when they write a new utility, that will be enough to get it on people's radar. Or a shared wiki page where people can list/document them.
Team communication (shout out "hey does someone have a Document toString?")
Keep utility classes to an absolute minimum and restrict them to a single namespace
Always think: how can I do this with an object. In your example, I would extend the Document class and add those toString and serialize methods to it.
This problem is helped when combining IDE "code-completion" features with languages which support type extensions (e.g. C# and F#). So that, imagining Java had a such a feature, a programmer could explore all the extension methods on a class easily within the IDE like:
Document doc = ...
doc.to //list pops up with toXmlString, toJsonString, all the "to" series extension methods
Of course, Java doesn't have type extensions. But you could use grep to search your project for "all static public methods which take SomeClass as the first argument" to gain similar insight into what utility methods have already been written for a given class.
Its pretty hard to build a tool that recognizes "same functionality". (In theory this is in fact impossible, and where you can do it in practice you likely need a theorem prover).
But what often happens is people clone clode that is close to what they want, and then customize it. That kind of code you can find, using a clone detector.
Our CloneDR is a tool for detecting exact and near-miss cloned code based on using parameterized syntax trees. It matches parsed versions of the code, so it isn't confused by layout, changed comments, revised variable names, or in many cases, inserted or deleted statements. There are versions for many languages (C++, COBOL, C#, Java, JavaScript, PHP, ...) and you can see examples of clone detection runs at the provided
link. It typically finds 10-20% duplicated code, and if you abstract that code into library methods on a religious base, your code base can actually shrink (that has occurred with one organization using CloneDR).
You are looking for a solution that can you help you manage this inevitable problem, then I can suggest a tool:
TeamCity: an amazing easy to use product that manages all your automated code building from your repository and runs unit tests etc.
It's even a free product for most people.
The even better part: it has built in code duplicate detection across all your code.
More stuff to read up:
Tools to detect duplicated code (Java)
a standard application utility project. build a jar with the restricted extensibility scope and package based on functionality.
use common utilities like apache-commons or google collections and provide an abstraction
maintain knowledge-base and documentation and JIRA tracking for bugs and enhancements
evolutionary refactoring
findbugs and pmd for finding code duplication or bugs
review and test utility tools for performance
util karma! ask team members to contribute to the code base, whenever they find one in the existing jungle code or requiring new ones.