After years of programming, we all have a set of small functions used as helpers utilities that we wish it comes build-in so we can use it in any project and have ti taken care by more people (test and optimized).
I have quite a collection of these functions. I wonder how do you guys organize them? Do you have any tips?
This is how I do it. I put it in a separate project (an eclipse project) let say "MyUtils" and it referred to by other projects. This works but because the utils collection are getting bigger and bigger something it is kind of weird that the utils are bigger than the project code (for small projects). And to ship it in Jar, you have to select them all by hand (or include them all). Is there a better way?
Also, as Java requires all functions to be in a class so I have ton of static functions (those that does not fit in OOP) for example a function read text file from a file name. Like this:
package nawaman.myutil;
public class UText {
static public String ReadTextFile(String pFileName) {
...
}
static public String[] ReadLines_fromFile(String pFileName) {
...
}
static public String ReadLine_fromFile(String pFileName, int pLineNumber) {
...
}
...
}
So when I need to include all the functions goes when though it is not used.
Is there a better way to do this?
I use eclipse on Linux anyway if there is special technique for it but fell free to share if you have techniques with other tools.
I treat such utility classes just like other components external to the software that I develop:
For each component I create a Eclipse project and build it to a jar.
Classes are grouped logically in packages, e.g. [domain].util.net, [domain].util.text etc.
In a project I include the dependencies I need. Maven can help you here.
You write that utility classes have a lot of static methods. That's something I don't use a lot. For example the text functions you show can be refactored to a class or set of classes that extend or implement classes and interfaces from the collections framework. That makes it easier to integrate my code with other libraries.
This works but because the utils collection are getting bigger and bigger something it is kind of weird that the utils are bigger than the project code (for small projects). And to ship it in Jar, you have to select them all by hand (or include them all). Is there a better way?
For my projects I use javac to select all the classes from my util libraries. For this I compile all classes from my project to an empty output directory. javac automatically resolves the dependencies to the util libraries because I added the util library pathes as source pathes. Now I can create a jar that contains all classes of my project and only the needed classes of the util libraries.
Also, as Java requires all functions to be in a class so I have ton of static functions (those that does not fit in OOP) for example a function read text file from a file name.
I do it the same way. But I try have a lot of small util classes instead of a few big ones, so that I don't have to include tons of unneeded methods to my jars.
My "utilities" have their own package namespace and SVN repository. They are, in essence my own libraries: distinct projects which may be pulled in, shared, tagged, updated, whatever.
The organization used within each of these "libraries" depends on the scope and function in question.
Because I disagree with the structure being a slave to some potential class/JAR output:
If you are concerned about "method bloat" in the classes and/or JARs, please use an automated tool to combat this. ProGuards is just one example and, while it can obfuscate, it can work equally well at just "dead code elimination".
Split your utils module into smaller subprojects. Use Maven or other build system to track versions of all your util modules. They are crucial to your systems because I think they used are in almost all your projects. Use tools like Findbugs or PMD to mesure quality of your code.
Every project need to know which version of utils module is using. It unacceptable in my opinion to add to binaries/sources of one of yours 'nonutils' project some loosely coupled util classes.
Please, revise yours classes with other commons projects like Apache Commons. I assume that lot of your utility code is similiar. Think better of rewriting yours static metods, because they obstruct testing (I'm sure that Findbugs will be complaining a lot too).
To sum up - creating a utils library is a hard stuff and a lot of responsability. So requirements in area of code quality are very high. I hope that my advice will help.
You should be very careful with removing classes after compilation - you may end up in a class not found situation at runtime. If you never use reflection or Class.forName() you should be safe, but those introduce runtime dependencies which the compiler cannot help you with (like it can with "new").
Remember - those classes not used do not use memory in the running program, only uses bytes on disk.
Personally I've ended up at saying disk space is cheap, and the risk of accidntially removing a class defintion used causing a runtime break, is not worth it to me, so I say - all code used for compilation must be shipped.
I don't use Eclipse, but in Visual Studio you can add a reference to file without it being physically moved or copied. This allows you to define a file in the root of your source control that all of your projects can reference without it being included in every project or having to deal with the copying problem. With this kind of solution you can intelligently split your util methods into different files and selectively include them based on what individual projects need. Also you can get rid of the extra .jar.
That said, I have no idea if Eclipse supports this kind of file referencing, but it might be worthwhile to look.
Related
I'm aware that it isn't easily feasible to get all of the classes in a package using reflection, but I'm wondering if someone knows of a good solution/workaround, specifically for an Android project?
Given a package, I need to be able to retrieve all of the classes from it and process annotations from them using reflection.
Does anyone know of a way to do this? Are there any libraries available?
Scanning the filesystem as most solutions for non-Android Java do won't help on Android. Here's a (theoretical) solution that is android-specific: http://mindtherobot.com/blog/737/android-hacks-scan-android-classpath/
However, it remains a hack, since Java unfortunately does not directly support this.
Existing dependency injection solutions use reflection for processing the annotations, but still need the resources to be declared. See this example of DI using reflection.
If you are using Ant to build your artifacts, you could read the contents of your source directory using Bash or Java, and use this to regenerate the full hierarchy of classes automatically during each build. This might make things tricky if you rely on heavily on the Eclipse IDE though, since the list might be out of date until you run another Ant build. (Note: according to Pyscho you can make Eclipse use Ant by altering the project configuration, see comments)
Another option might be to process the AndroidManifest file using the AssetManager, but you would be limited to the resources declared in that file. The compiled classes themselves are in-lined and optimised in the classes.dex file, and as such you're unlikely to get much useful information from it.
I think you might find the answer here https://stackoverflow.com/a/1457971/1199538
there is a java file attached so you can download it and try it
short snippet from the answer following:
This method can only be used when:
You have a class that is in the same package you want to discover, This class is called a
SeedClass. For example, if you want to list all classes in 'java.io', the seed class may be java.io.File.
Your classes are in a directory or in a JAR file it has source file information (not source code file, but just source file). As far as I've tried, it work almost 100% except the JVM class (those classes come with the JVM).
Your program must have permission to access ProtectionDomain of those classes. If your program is loaded locally, there should be no problem.
You can do classpath scanning for Android at compiletime, before the JVM bytecodes have been converted to Dalvik bytecodes, e.g. using the ClassGraph library (I am the author):
https://github.com/classgraph/classgraph/wiki/Build-Time-Scanning
I recently started using Eclipse IDE and have read at a number of places that one shouldn't use the default(src) package and create new packages.
I just wanted to know the reason behind this.
Using the default package may create namespace collisions. Imagine you're creating a library which contains a MyClass class. Someone uses your library in his project and also has a MyClass class in his default package. What should the compiler do? Package in Java is actually a namespace which fully identifies your project. So it's important to not use the default package in the real world projects.
Originally, it was intended as a means to ensure there were no clashes between different pieces of Java code.
Because Java was meant to be run anywhere, and over the net (meaning it might pick up bits from Sun, IBM or even Joe Bloggs and the Dodgy Software Company Pty Ltd), the fact that I owned paxdiablo.com (I don't actually but let's pretend I do for the sake of this answer) meant that it would be safe to call all my code com.paxdiablo.blah.blah.blah and that wouldn't interfere with anyone else, unless they were mentally deficient in some way and used my namespace :-)
From chapter 7, "Packages", of the Java Language Spec:
Programs are organized as sets of packages. Each package has its own set of names for types, which helps to prevent name conflicts.
I actually usually start by using the default package and only move it into a real package (something fairly easy to do with the Eclipse IDE) if it survives long enough to be released to the wild.
Java uses the package as a way to differentiate between classes. By using packages, you can have an org.example.Something class and an org.example.extended.Something class and be able to differentiate between them even though they are both named Something. Since their packages are different, you can use them both in the same project.
By declaring a package you define your own namespace (for classes). This way if you have two identical classes using a different package name (namespace) will differentiate between which one you want to use.
The main reasons I can think of are:
It keeps things organised, which will help you (and others!) know where to look for classes/functionality.
You can define classes with the same name if they are in different packages.
Classes/etc in the default package cannot be imported into named packages. This means that in order to use your classes, other people will have to put all their classes in the default package too. This exacerbates the problems which reasons 1 & 2 solve.
From a java point of view, there are two general dev/deploy lifecycles you can folllow, either using ant to build and deploy, or the maven lifecycle. Both of these lifecycles look for source code and resources in local directories, and in the case of maven, in defined repositories, either locally or on the net.
The point is, when you set up a project, for development and eventually deployment, you want to build a project structure that is portable, and not dependent on the IDE, ie. your project can be built and deployed using either of your build environments. If you use a heavy dependence on the Eclipse framework for providing class variables, compile paths, etc.. you may run into the problem that your project will only build and deploy using that configurationj, and it may not be portable to another developers environment, so to speak.
When writing code in an Eclipse project, I'm usually quite messy and undisciplined in how I create and organize my classes, at least in the early hacky and experimental stages. In particular, I create more than one class with a main method for testing different ideas that share most of the same classes.
If I come up with something like a useful app, I can export it to a runnable jar so I can share it with friends. But this simply packs up the whole project, which can become several megabytes big if I'm relying on large library such as httpclient.
Also, if I decide to refactor my lump of code into several projects once I work out what works, and I can't remember which source files are used in a particular run configuration, all I can do it copy the main class to a new project and then keep copying missing types till the new project compiles.
Is there a way in Eclipse to determine which classes are actually used in a particular run configuration?
EDIT: Here's an example. Say I'm experimenting with web scraping, and so far I've tried to scrape the search-result pages of both youtube.com and wrzuta.pl. I have a bunch of classes that implement scraping in general, a few that are specific to each of youtube and wrzuta. On top of this I have a basic gui common to both scrapers, but a few wrzuta- and youtube-specific buttons and options.
The WrzutaGuiMain and YoutubeGuiMain classes each contain a main method to configure and show the gui for each respective website. Can Eclipse look at each of these to determine which types are referenced?
Take a look at ProGuard, it is a "java shrinker, optimizer, obfuscator, and preverifier". I think you'll mainly be interested in the first capability for this problem.
Yes it's not technically part of Eclipse, as you requested, but it can be run from an Ant script, which can be pretty easily run in Eclipse.
I create more than one class with a main method for testing different ideas that share most of the same classes.
It's better to be pedantic than lazy, it saves you time when coding :-)
You can have one class with a main method that accepts a command-line argument and calls a certain branch of functionality based on its value.
I guess this is kind of a follow-on to question 1522329.
That question talked about getting a list of all classes used at runtime via the java -verbose:class option.
What I'm interested in is automating the build of a JAR file which contains my class(es), and all other classes they rely on. Typically, this would be where I am using code from some third party open source product's "client logic" but they haven't provided a clean set of client API objects. Their complete set of code goes server-side, but I only need the necessary client bits.
This would seem a common issue but I haven't seen anything (e.g. in Eclipse) which helps with this. Am I missing something?
Of course I can still do it manually by: biting the bullet and including all the third-party code in a massive JAR (offending my purist sensibilities) / source walkthrough / trial and error / -verbose:class type stuff (but the latter wouldn't work where, say, my code runs as part of a J2EE servlet, and thus I only want to see this for a given Tomcat webapp and, ideally, only for classes related to my classes therein).
I would recommend using a build system such as Ant or Maven. Maven is designed with Java in mind, and is what I use pretty much exclusively. You can even have Maven assemble (using the assembly plugin) all of the dependent classes into one large jar file, so you don't have to worry about dependencies.
http://maven.apache.org/
Edit:
Regarding the servlet, you can also define which dependencies you want packaged up with your jar, and if you are making a stand alone application you can have the jar tool make an executable jar.
note: yes, I am a bit of a Maven advocate, as it has made the project I work on much easier. No I do not work on the project personally. :)
Take a look at ProGuard.
ProGuard is a free Java class file shrinker, optimizer, obfuscator, and preverifier. It detects and removes unused classes, fields, methods, and attributes. It optimizes bytecode and removes unused instructions. It renames the remaining classes, fields, and methods using short meaningless names. Finally, it preverifies the processed code for Java 6 or for Java Micro Edition.
What you want is not only to include the classes you rely on but also the classes, the classes you rely on, rely on. And so on, and so forth.
So that's not really a build problem, but more a dependency one. To answer your question, you can either solve this with Maven (apparently) or Ant + Ivy.
I work with Ivy and I sometimes build "ueber-jar" using the zipgroupfileset functionality of the Ant Jar task. Not very elegant would say some, but it's done in 10 seconds :-)
We have a developer who is in the habit of committing non-java files (xsd, dtd etc) in the java packages under the src/java folder in our repository. Admittedly, these are relevant files to that package, but I just hate to see non-java files in the src folder.
Is this is a common practice that I should get used to or are we doing something strange by maintaining these files like this?
The problem with putting non Java (or other languages) files that are closely tied to the code in a different place than the code is knowing where to find them. It is possible to standardize the locations then theoretically everyone will know where to go and what to do. But I find in practice that does not happen.
Imagine your app still being maintained 5 or 10 years down the road by a team of junior - intermediate developers that do not work at the company now and will never talk to anyone who works on your project now. Putting files closely linked to the source in the source package structure could make their lives easier.
I am a big proponent of eliminating as many ambiguities as possible within reason.
It's very common and even recommended as long as its justifiable. Generally it's justifiable when it's a static resource (DTD+XSLT for proprietary formats, premade scripts etc.) but it's not when the file is something that's likely to be updated by a third party like IP/geographic location database dump.
I think it gets easier if you think of 'src' as not specifically meaning 'source code'. Think of it as the source of resources that are things needed by your program at compile time and/or runtime.
Things that are a product of compile or build activities should not go here.
Admittedly, like most things, exceptions may apply :)
Update:
Personally, I like to break down src further with subdirectories for each resource type underneath it. Others may like that division at a higher level.
There is a lot of jar libraries that uses the same practice.
I think it is acceptable and comfortable.
In Eclipse it works well for us to have a src folder containing java classes, and a configuration folder (which is blessed as a source folder) containing property files etc. Then they all go in the output folder together and can be found in the classpath while still being in seperate folders inside Eclipse
One of the advantages of keeping all the auxiliary files next to the source is that version consistency is maintained between these 3rd party libraries and your source code. If you ever need to go back and debug a specific version, you can pull the entire set of source+config and have it all be the same version.
That being said I'd put them in a $project/config/ directory, or some such, rather than in $project/src/java itself. They're not source, nor java, really, so it's misleading having them in that directory.
When you really get down to it, though, this is an issue of personal style. There's no "Right" answer and you should be talking with those team members and understanding why they made this decision. Using this thread as evidence to support a unilateral decision probably won't go over well. ;)
Its pretty common, you can find it in really popular frameworks, e.g. xsd files for spring various schemas. Also people usually place hibernate mapping files in the same package as the model classes.
I think this is common as long as the files are necessary. The problems arise when people start committing files that are not needed with the source, such as design specs or random text files.
It is surely common, but incredibly lazy and sloppy. My skin crawls when I see it.
Using a tool such as Maven to build your products enables you to easily, and clearly separate code from resources.
Eclipse bundles can be similarly separated.