thrift interface can be compiled across multiple languages. it's just text files, why there are no online tools like swagger hub? I don't want to copy paste interface across projects that use that interface
also i don't find it useful to package interface with jar file, because only jvm languages can resolve that interface and also it's not user friendly way. It's not only about thrift, it's about grpc also. I didn't find any docs concerned with this question and couldn't find any best practises
Assuming you have a .proto file with your interfaces, each sub project will need to know about the file. There are two main approaches to this problem: Vendor the file, or copy the file.
Vendor the File
In this option, you make an addition project (like a git repo) which stores all your interface definitions. Each project that needs to know about the interfaces will include a reference (git submodule or git subtree) which includes the interface project. When you build your project, the interfaces will need to be synced and then used to generate the necessary code.
The downside of this approach is that git subtree and submodule (or whatever version control you use) are more difficult to use, and require extra work by people building your code. If you make changes to the interface in the subproject it can be difficult to apply those changes back upstream to the interface project.
Copy the File
In this option, you manually copy the file around between projects, and manually keep them in sync. Each time you make a change, you'll want to apply that change to every other project that depends on the interface. When using Protobuf though, it is important to note that you don't have to do this. Protos are designed to be highly backwards compatible.
For example, code that is changing a proto definition from one form to the other can actually use both forms. Old code will look at the old form, and new code can decide on looking at the old or new form. Once all users have been upgraded, you can remove the old form.
The downside to this approach is that it pushes complexity into the decoding portion of your code. You end up needing to be backwards compatible, with an unknown number of older clients. Since not every project will be in sync with the interface definitions, all the users of the interface will need to be more flexible. This problem is not specific to Proto, but happens naturally; it happens to everyone.
A second downside is having to manually copy changes. You must make sure never to reuse field numbers or names. If you have a lot of projects that depend on the interface, its more work for you.
Which to Choose?
Neither approach is objectively better than the other. Each one pushes complexity into a different part of your build. From what I have seen, most people prefer to copy the file, since it is easier than learning advanced git commands.
Related
We have a large project consisting of the following:
A: C++ source code / libraries
B: Java and Python wrapping of the C++ libraries, using SWIG
C: a GUI written in Java, that depends on the Java API/wrapping.
People use the project in all the possible ways:
C++ projects using the C++ API
Java projects using the Java API
Python scripting
MATLAB scripting (using the Java API)
through the Java GUI
Currently, A, B and C are all in a single Subversion repository. We're moving to git/GitHub, so we have an opportunity to reorganize. We are thinking of splitting A, B, and C into their own repositories. This raises a few questions for us:
Does it make sense to split off the Java and Python SWIG wrapping (that is, the interface (*.i) files) into a separate repository?
Currently, SWIG-generated .java files are output in the source tree of the GUI and are tracked in SVN. Since we don't want to track machine-generated files in our git repositories, what is the best way of maintaining the dependency of the GUI on the .java/.jar files generated by SWIG? Consider this: if a new developer wants to build the Java GUI, they shouldn't need to build A and B from scratch; they should be able to get a copy of C from which they can immediately build the GUI.
Versioning: When everything is in one repository, A, B and C are necessarily consistent with each other. When we have different repositories, the GUI needs to work with a known version of the wrapping, and the wrapping needs to work with a known version of the C++ API. What is the best way to manage this?
We have thought deeply about each of these questions, but want to hear how the community would approach these issues. Perhaps git submodule/subtree is part of the solution to some of these? We haven't used either of these, and it seems submodules cause people some headache. Does anybody have stories of success with either of these?
OK, I looked in a similar problem like you (multiple interacting projects) and I tried the three possibilities subtree, submodules and a single plain repository with multiple folders containing the individual parts. If there are more/better solutions I am not aware of them.
In short I went for a single repository but this might not be the best solution in your case, that depends...
The benefit of submodules is that it allows easy management as every part is itself a repo. Thus individual parties can work only on their repo and the other parts can be added from predefined binary releases/... (however you like). You have to add an addtional repo that concatenates the individual repos together.
This is both the advantage and disadvantage: Each commit in this repo defines a running configuration. Ideally your developers will have to make each commit twice one for the "working repo" (A through C) and one for the configuration repo.
So this method might be well suited if you intent you parts A-C to be mostly independent and not changing too often (that is only for new releases).
I have to confess that I did not like the subtree method personally. For me (personally) the syntax seems clumsy and the benefit is not too large.
The benefit is that remote modifications are easily fetched and inserted but you loose the remote history. Thus you should avoid to interfere with the remote development.
This is the downside: If you intend to do modifications on the parts you have always to worry about the history. You can of course just develop in a git remote and for testing/merging/integrating change to the master branch. This is ok for mainly reading remote repos (if I am developing only on A but need B and C) but not for regular modifications (in the example for A).
The last possibility is one plain repo with folders for each part. The benefit is that no adminstration to keep the parts in sync is directly needed. However you will not be able to guarantee that each commit will be a running commit. Also you developers will have to do the administration by hand.
You see that the choice depends on how close the individual parts A-C are interconnected. Here I can only guess:
If you are in an earlier stage of development where modifications throughout the whole source tree are common one big repo is better handleable than a splitted version. If your interfaces are mostly constant the splitting allows smaller repos and a more strict separation of things.
The SWIG code and the C++ code seems quite close. Thus splitting those two seems less practical than splitting the GUI from the rest (for my guess).
For you other question "How to handle new developers/(un)tracking machine-generated code?":
How many commits are made/required (only releases or each individual commit)? If only releases are of interest you could go with binary packages. If you intent to share each single commit, you would have to provide many different binary versions. Here I would suggest let them compile the whole tree once consuming a few minutes and from there on rebuilding is just a short make that should not take too long. This could even automatized using hooks.
I'd like to do some versioning of some Android code.
I pushed the code (only the res and src directories) and now I want to pull it in some other package.
How can I manage that, given that the code in my git repository has it's source in src/com/some/package/ and I want to clone it and get the source in src/some/other/package/?
I'm aware of this question, but I can't see any answer that helps.
What you can do as an alternative to directly managing it with git, is to separate the common code into its own Android Library Project. Then you would be able to add that library to the other two projects, giving you access to the same code base.
This would have you end up with three projects, and three repositories. Two of each for the separate projects, and one of each for the library project.
One thing to be aware of, however, is that if there is specific code for one project inside of the library, it will also affect the other project. Some of this could be handled by subclassing and overriding methods/properties as needed. Another way around this, as suggested by jul, would be to have branches for each project.
A nice benefit of doing it this way is that if you make a bug fix in the library that you found while working on one project, you'll be able to get that in the other for practically free.
We currently have an application which is essentially a fully-functional demo for potential clients. All the functionality is there. However, we use generic branding/logos, call our own web services (which would later be swapped out for calls to client web-services), etc.
Here is my question. If we have two different clients, we would prefer as little duplicate code as possible. I understand that this could be done -- from a java perspective -- by simply including a shared JAR. However, we will need to change around resources. Also, one client may not want some functionality that another client does want. On top of this, if we are doing general bug fixes, we will normally want these fixes to be in both versions of the application.
We are using Git for version control and Maven for building the project.
One option we discussed is simply branching the project and maintaining separate versions. However, then we would have to manually merge changes that we want reflected in all versions of the app.
Another option we discussed is somehow swapping out resources, etc. using maven profiles. However, if we need to make any non-superficial changes to the code itself, this could be a problem. We might have to get into factories and different implementations.
Does anyone have recommendations on the best way to handle this?
We use a library project with git submodules to handle all of our similar projects. The master project is pretty hefty but we use a configuration file to determine what features should be in the finished product.
In Java web application, what is the exact meaning of the term "package structure" and "directory structure" ? Aren't they the same? I saw some articles have these two terms, but I am not sure about the exact meaning and difference.
Package is a collection of code that changes together, is used together and is shipped together. So a jar/war is a package.
Package Design Principles
I understand that you meant source package, which is more like directory structure. But I believe, a directory is a physical representation on hard drive.
EDIT: I had writtern original answer more than 3years back. But did not change as it was accepted. But changing it now so that any new visitor may benefit and also to avoid link rot. Some additional meaning of package may be extracted based on the discussion below. For example, is a jar a package?
Classes that get reused together should be packaged together so that the package can be treated as a sort of complete product available for you. And those which are reused together should be separated away from the ones those are not reused with. For example, your Logging utility classes are not necessarily used together with your file io classes. So package all logging them separately. But logging classes could be related to one another. So create a sort of complete product for logging, say, for the want of better name commons-logging package it in a (re)usable jar and another separate complete product for io utilities, again for the want of better name, say commons-io.jar. If you update say commons-io library to say support java nio, then you may not necessarily want to make any changes to the logging library. So separating them is better.
Now, let's say you wanted your logging utility classes to support structured logging for say some sort of log analysis by tools like splunk. Some clients of your logging utility may want to update to your newer version; some others may not. So when you release a new version, package all classes which are needed and reused together for migration. So some clients of your utility classes can safely delete your old commons-logging jar and move to commons-logging-new jar. Some other clients are still ok with older jar. However no clients are needed to have both these jars (new and old) just because you forced them to use some classes for older packaged jar.
Avoid cyclic dependencies. a depend on b; b on c; c on d; but d depends on a. The scenario is obviously deterring as it will be very difficult to define layers or modules, etc and you cannot vary them independly relative to each other.
Also, you could package your classes such that if a layer or module changes, other module or layers do not have to change necessarily. So, for example, if you decide to go from old MVC framework to a rest APIs upgrade, then only view and controller may need changes; your model does not.
In most Java applications, the package structure should be matched by the directory structure for the .java and .class files. However these directories are part of a larger directory structure, including other data than the source and/or the bytecode.
Depending on the context, the "package structure" might also refer to delivery packages, each containing an application or a library.
It is time to sub-divide a platform I'm developing and I'm looking for advice on how to handle cross-component dependencies. I spose there a many cases, so I'll give an example.
I have an Address class that I want to make visible to developers. It is also referenced by classes in my.Contacts, my.Appointments, and my.Location packages - each of which I want to be separately compiled, jar-d, and delivered. Of course I want Address to be a single class - an Address works across these platform components transparently.
How should Address be packaged, built, and delivered?
Thanks!
Two thoughts:
Address sounds like a common component that can be used in different deliverables and so should be available in some common or core library
It may make sense for your components to talk to an Address interface, and the implementation can be provided separately (e.g. provide an Address interface and an AddressImpl implementation). This will reduce the amount of binding between the core library and the library your developers will develop.
In this case Address is a part of a library which deserves its own jar. If you create a class named Address in my.Contacts, my.Appointments, and my.Location and you want to use all theses jar in a same application, you'll have a conflict for your Address class.
I suggest you don't "Deliver" these jars separately. Java has very subtle versioning issues that you don't want to run into. Build everything together and package it into one or two jars and always deliver both jars, or build them together and deliver a subset of jars (but never combine new and old jars--don't just try to send a single jar as an update).
If you must build them separately be very aware that final constants are compiled in and not referenced--so if you change one and deliver a new jar, any references from an older jar will not be updated.
Also method signatures that change will have strange, unpredictable results.
It sounds like you want a developer interface as well--that may be a set of interfaces and classes that reside in a separate jar. If you make that one jar well enough that you never have to rev it (and, of course, with no references to external constants) you can probably get away with not updating it which will keep your customer's extensions from getting crusty.