Related
I'm going to write my first Java based web app, and I'm sort of lost how to begin.
Firstly, I would like a web app and a desktop app that do pretty much the same thing, without the hackish idea of embedding a web browser into the desktop app because that doesn't allow to easily make changes to the desktop without affecting the web app and vice versa.
Now, here my questions.
Right now, I have a bunch of POJOs and they communicate with a single class that, right now, uses a flat file as a "database", of course, in production, I would use a legitimate database and just change that single class. Is this a good idea? Will I be able to go from POJOs to a web app?
Should I use a framework? I would like to have this app written pretty soon, seeing that all the buisness logic is there, I just need to wrap it so its usable, so, I don't want to spend an extreme amount of time learning, say, Spring (which AFAIK is huge), but, I don't want to keep reinventing the wheel throughout my app either. I can always just use JSP and scriptlets...
If you said yes to the above, what framework(s) do you suggest? Please note that I would like a framework that I can start using in maybe 3-4 weeks of learning.
Will I have to start from scratch with the POJOs that I have written? They're well over 30k LOC, so, if it is like that, I'll be hesitant.
You will need:
a web framework. Since you have Swing background, JSF 2 will be your best bet (everything will be painful, of course, but JSF will get you up and going quickly and will help you avoid the most tragic mistakes). Also, wrapping business pojos into web guis is the main use-case for JSF and it's biggest focus.
a "glue framework". One thing that is much different with web applications as opposed to desktop ones is that you cannot create view components by yourself - they must be created when browser requests a page. So you have to find a way to create the view objects and deliver all the references to the pojos that represent logic, some of which may have very different lifecycles (this is not a problem on desktop, but on web you have to distinguish between pojos that live along with the whole application, along with a single user session, along with a single request, and so on).
The "glue framework" could also provide the additional benefit of managing transactions. You have three choices:
Spring. It's not half as complex as you thing; you only need to learn some basic stuff.
EJB. You would need a real application server, like Glassfish or JBoss
bare JSF has good support for dependency injection, the only drawback is the lack of automatic transaction management.
If I were in your position, I would go with bare JSF 2.0 - this way you only need to learn one new technology. At first, try to avoid libraries like PrimeFaces - they usually work worse than advertised.
edit - and addendum
or - what is "dependency injection"(abridged and simplified)
When request comes to a web application, a new task starts in a new thread (well, the thread is probably recycled, but that's not important).
The application has already been running for some time and most of the objects you are going to need are already built and should not get created again: you have your database connection pool, maybe some parts of business layer; it is also possible that the request is just one of many request made during one session, and you already have a bunch of POJOs that the user is working on. The question is - how to get references to those objects?
You could arrange your application so that resources are available through some static fields. They may be singletons themselves, or they could be acquired through a singleton locator. This tends to work, but is out of fashion (hard to test, hard to refactor, hard to reuse, lifecycles are hard coded in application). The real code could look like this:
public void doSomething() {
Customer Service cs = AppManager.getInstance().getCustomerService();
System.out.println(cs.getVersion());
}
if you need clustering and session management, you could build a special kind of broker that would know and provide to anyone all kinds of needed objects. Each type of object would be registered as a factory under a different name. This also works and is implemented in Java as JNDI. The actual client code would look like this:
public void doSomething() throws Exception {
CustomerService cs = (CustomerService)new InitialContext().lookup("some_fancy_looking_name_in_reality_just_string");
System.out.println(cs.getVersion());
}
The last way is the nicest. Since your initial object is not created by you but by the server just after http request arrives (details depend on the technology you choose, but your entry point might be a JSF managed bean or some kind of action controller), you can just advertise which references you need and let the server take care of finding them for you. This is called "Dependency Injection". Your acts as if everything is taken care of before your code is ever launched. Spring or EJB container, or CDI, or JSF take care of the rest. The code would look like this (just an example):
#EJB
CustomerService cs;
public void doSomething() {
System.out.println(cs.getVersion());
}
Note:
when you use DI, it really uses one of the two former methods under the hood. The good thing is: you do not have to know which one and in some cases you can even switch them without altering your code;
the exact means of registering components for injection differs from framework to framework. It might be a piece of Java code (like in Guice), an XML file (classic Spring) or an annotation (classic EJB 3). Most of the mentioned technologies support different kinds of configuration.
You should definitely use a framework as otherwise sooner or later you'll end up writing your own.
If you use maven then simply typing mvn archetype:generate will give you a huge list of frameworks to choose from and it'll set up all of the scaffolding for you so you can just play with a few frameworks until you find the one that works for you.
Spring has good documentation and is surprisingly easy to get started with. Don't be put off by the pages of documentation! You could use JPA to store stuff in the database. You should (in theory) just be able to annotate your existing POJO's to denote primary keys and so on and it should just work. You can also use JSP's within Spring if that makes life easier.
... I a bunch of POJOs and they communicate with a single class that, right now, uses a flat file as a "database", of course, in production, I would use a legitimate database and just change that single class. Is this a good idea? Will I be able to go from POJOs to a web app?
qualified yes. if the pojo's are sane you should not have many problems. many people use hiberbate.
Should I use a framework? I would like to have this app written pretty soon, seeing that all the buisness logic is there, I just need to wrap it so its usable, so, I don't want to spend an extreme amount of time learning, say, Spring (which AFAIK is huge), but, I don't want to keep reinventing the wheel throughout my app either. I can always just use JSP and scriptlets...
probably. spring is huge, but things like grails or roo can help.
if you want to have a responsive web app, you will need to do some kind of rich client (AJAX). this may require a lot of your code to run on the client. this means writing a lot of javascript or using gwt. this will be a pain. it probably will not be so easy to just "wrap it". if you have written a swing app, then basically that code will need to run on the client.
If you said yes to the above, what framework(s) do you suggest? Please note that I would like a framework that I can start using in maybe 3-4 weeks of learning.
i like groovy and grails - grails uses spring-mvc, spring, hibernate. but there is roo, play and others.
Will I have to start from scratch with the POJOs that I have written? They're well over 30k LOC, so, if it is like that, I'll be hesitant.
the code that will run on the server can probably be mostly left alone. the code that has to run on the client needs to be rewritten in javascript or maybe you can get some reuse out of that code by using gwt,
The Play Framework is doing great things. I would recommend it highly. Having worked with EJB apps and Tomcat/Servlet/Spring apps it's a breath of fresh air. After framework installation you get a working app in a few seconds. Reminds me of Ruby on Rails or Node.js with the type-safety of Java.
Much quicker turnaround on getting started, faster development cycles, and a clearer configuration model than previous Java web app frameworks.
http://www.playframework.com/
I've recently been more and more frustrated with a problem I see emerging in my projects code-base.
I'm working on a large scale java project that has >1M lines of code. The interfaces and class structure are designed very well and the engineers writing the code are very proficient. The problem is that in an attempt to make the code cleaner people write Utility classes whenever they need to reuse some functionality, as a result over time and as the project grows more and more utility methods crop up. However, when the next engineer comes across the need for the same functionality he has no way of knowing that someone had already implemented a utility class (or method) somewhere in the code and implements another copy of the functionality in a different class. The result is a lot of code duplication and too many utility classes with overlapping functionality.
Are there any tools or any design principles which we as a team can implement in order to prevent the duplication and low visibility of the utility classes?
Example: engineer A has 3 places he needs to transform XML to String so he writes a utility class called XMLUtil and places a static toString(Document) method in it. Engineer B has several places where he serializes Documents into various formats including String, so he writes a utility class called SerializationUtil and has a static method called serialize(Document) which returns a String.
Note that this is more than just code-duplication as it is quite possible that the 2 implementations of the above example are different (say one uses transformer API and the other uses Xerces2-J) so this can be seen as a "best-practices" problem as well...
Update: I guess I better describe the current environment we develop in.
We use Hudson for CI, Clover for code coverage and Checkstyle for static code analysis.
We use agile development including daily talks and (perhaps insufficient) code reviews.
We define all our utility classes in a .util which due to it's size now has 13 sub-packages and about 60 classes under the root (.util) class. We also use 3rd party libraries such as most of the apache commons jars and some of the jars that make up Guava.
I'm positive that we can reduce the amount of utilities by half if we put someone on the task of refactoring that entire package, I was wondering if there are any tools which can make that operation less costly, and if there are any methodologies which can delay as much as possible the problem from recurring.
A good solution to this problem is to start adding more object-orientation. To use your example:
Example: engineer A has 3 places he needs to transform XML to String so he writes a utility class called XMLUtil and places a static toString(Document) method in it
The solution is to stop using primitive types or types provided by the JVM (String, Integer, java.util.Date, java.w3c.Document) and wrap them in your own project-specific classes. Then your XmlDocument class can provide a convenient toString method and other utility methods. Your own ProjectFooDate can contain the parsing and formatting methods that would otherwise end up in various DateUtils classes, etc.
This way, the IDE will prompt you with your utility methods whenever you try to do something with an object.
Your problem is a very common one. And a real problem too, because there is no good solution.
We are in the same situation here, well I'd say worse, with 13 millions line of code, turnover and more than 800 developers working on the code. We often discuss about the very same problem that you describe.
The first idea - that your developers have already used - is to refactor common code in some utility classes. Our problem with that solution, even with pair programming, mentoring and discussion, is that we are simply too many for this to be effective. In fact we grow in subteams, with people sharing knowledge in their subteam, but the knowledge doesn't transit between subteams. Maybe we are wrong but I think that even pair programming and talks can't help in this case.
We also have an architecture team. This team is responsible to deal with design and architecture concerns and to make common utilities that we might need. This team in fact produces something we could call a corporate framework. Yes, it is a framework, and sometimes it works well. This team is also responsible to push best practices and to raise awareness of what should be done or not, what is available or what is not.
Good core Java API design is one of the reason for Java success. Good third party open sources libraries count a lot too. Even a small well crafted API allows to offer a really useful abstraction and can help reduce code size a lot. But you know, making framework and public API is not the same thing at all as just coding an utility class in 2 hours. It has a really high cost. An utility class costs 2 hours for the initial coding, maybe 2 days with debugging and unit tests. When you start sharing common code on big projects/teams, you really make an API. You must ensure perfect documentation then, really readable and maintainable code. When you release new version of this code, you must stay backward compatible. You have to promote it company wide (or at least team wide). From 2 days for your small utility class you grow to 10 days, 20 days or even 50 days for a full-fledged API.
And your API design may not be so great. Well, it is not that your engineers are not bright - indeed they are. But are you willing to let them work 50 days on a small utility class that just help parsing number in a consistent way for the UI? Are you willing to let them redesign the whole thing when you start using a mobile UI with totally different needs? Also have you noticed how the brightest engineers in the word make APIs that will never be popular or will fade slowly? You see, the first web project we made used only internal frameworks or no framework at all. We then added PHP/JSP/ASP. Then in Java we added Struts. Now JSF is the standard. And we are thinking about using Spring Web Flow, Vaadin or Lift...
All I want to say is that there is no good solution, the overhead grows exponentially with code size and team size. Sharing a big codebase restricts your agility and responsiveness. Any change must be done carefully, you must think of all potential integration problems and everybody must be trained of the new specificities and features.
But the main productivity point in a software company is not to gain 10 or even 50 lines of code when parsing XML. A generic code to do this will grow to a thousand lines of code anyway and recreates a complex API that will be layered by utility classes. When the guy make an utility class for parsing XML, it is good abstraction. He give a name to one dozen or even one hundred lines of specialized code. This code is useful because it is specialized. The common API allows to work on streams, URL, strings, whatever. It has a factory so you can choose you parser implementation. The utility class is good because it work only with this parser and with strings. And because you need one line of code to call it. But of course, this utility code is of limited use. It works well for this mobile application, or for loading XML configuration. And that's why the developer added the utility class for it in the first place.
In conclusion, what I would consider instead of trying to consolidate the code for the whole codebase is to split code responsibility as the teams grow:
transform your big team that work on one big project into small teams that work on several subprojects;
ensure that interfacing is good to minimize integration problems, but let team have their own code;
inside theses teams and corresponding codebases, ensure you have the best practices. No duplicate code, good abstractions. Use existing proven APIs from the community. Use pair programming, strong API documentation, wikis... But you should really let different teams make their choices, build their own code, even if this means duplicate code across teams or different design decisions. You know, if the design decisions are different this may be because the needs are different.
What you are really managing is complexity. In the end if you make one monolithic codebase, a very generic and advanced one, you increase the time for newcomers to ramp up, you increase the risk that developers will not use your common code at all, and you slow down everybody because any change has far greater chances to break existing functionality.
There are several agile/ XP practices you can use to address this, e.g.:
talk with each other (e.g. during daily stand-up meeting)
pair programming/ code review
Then create, document & test one or several utility library projects which can be referenced. I recommend to use Maven to manage dependecies/ versions.
You might consider suggesting that all utility classes be placed in a well organized package structure like com.yourcompany.util.. If people are willing to name sub packages and classes well, then at least if they need to find a utility, they know where to look. I don't think there is any silver bullet answer here though. Communication is important. Maybe if a developer sends a simple email to the rest of the development staff when they write a new utility, that will be enough to get it on people's radar. Or a shared wiki page where people can list/document them.
Team communication (shout out "hey does someone have a Document toString?")
Keep utility classes to an absolute minimum and restrict them to a single namespace
Always think: how can I do this with an object. In your example, I would extend the Document class and add those toString and serialize methods to it.
This problem is helped when combining IDE "code-completion" features with languages which support type extensions (e.g. C# and F#). So that, imagining Java had a such a feature, a programmer could explore all the extension methods on a class easily within the IDE like:
Document doc = ...
doc.to //list pops up with toXmlString, toJsonString, all the "to" series extension methods
Of course, Java doesn't have type extensions. But you could use grep to search your project for "all static public methods which take SomeClass as the first argument" to gain similar insight into what utility methods have already been written for a given class.
Its pretty hard to build a tool that recognizes "same functionality". (In theory this is in fact impossible, and where you can do it in practice you likely need a theorem prover).
But what often happens is people clone clode that is close to what they want, and then customize it. That kind of code you can find, using a clone detector.
Our CloneDR is a tool for detecting exact and near-miss cloned code based on using parameterized syntax trees. It matches parsed versions of the code, so it isn't confused by layout, changed comments, revised variable names, or in many cases, inserted or deleted statements. There are versions for many languages (C++, COBOL, C#, Java, JavaScript, PHP, ...) and you can see examples of clone detection runs at the provided
link. It typically finds 10-20% duplicated code, and if you abstract that code into library methods on a religious base, your code base can actually shrink (that has occurred with one organization using CloneDR).
You are looking for a solution that can you help you manage this inevitable problem, then I can suggest a tool:
TeamCity: an amazing easy to use product that manages all your automated code building from your repository and runs unit tests etc.
It's even a free product for most people.
The even better part: it has built in code duplicate detection across all your code.
More stuff to read up:
Tools to detect duplicated code (Java)
a standard application utility project. build a jar with the restricted extensibility scope and package based on functionality.
use common utilities like apache-commons or google collections and provide an abstraction
maintain knowledge-base and documentation and JIRA tracking for bugs and enhancements
evolutionary refactoring
findbugs and pmd for finding code duplication or bugs
review and test utility tools for performance
util karma! ask team members to contribute to the code base, whenever they find one in the existing jungle code or requiring new ones.
We have an application at work that we'd like to monitor for performance. Actually, what we want to monitor is not our app's performance, but things like response time for external web services we invoke.
Years ago, using ATG Dynamo, you could instrument your code with something like...
Performance.monitorStart("my.operation");
try {
// code goes here
}
finally {
Performance.monitorEnd("my.operation");
}
; this generated a nice report of the time spent in diverse operations, in a friendlier way than hprof. Ideally, the time should be persisted (db or otherwise).
I recall seeing somewhere (here? Dzone? TSS?) about a new library that does this, but googling reveals nothing.
Thoughts?
Alex
What you're describing sounds a lot like Perf4J.
Springsource TC Server (which is a Tomcat++) with Insight enabled has been helpful to me
It will time your entire call-stack and give you nice reports. Here's a screencast http://www.youtube.com/watch?v=nBqSh7nVNzc
Since you are already showing an example where code changes are involved, you could simply roll your own using your probably existing logging facility.
Another option would be JMX beans for live statistics - this option is often used together with a 'professional' monitoring facility which aggregates these statistics.
I have a project right now that straddles the line on framework and pluggable program, and am worried about the sheer number of dependancies that this program rely's on.
Currently I have this:
Commons lang - Mainly for string utils and array utils
slf4j - Logging facade
slf4j-log4j - Redirects logging to log4j for GUI (note that the GUI is a module)
log4j - Log4j itself for the above reason
jpersist/EJP - Database abstraction layer
PircBot - IRC layer
A JDBC driver
Mozilla Rhino - For Javascript plugins
In all that totals 7, even without the GUI unless you don't want any logging. For me who's trying to pass this off as "lightweight", this seems like way too much.
So my questions:
Should I limit the amount of frameworks that I am using?
How should I distribute it? Would an independent jar for being used in other programs and a big combined jar for a single program be okay?
Is this many dependencies normal?
It does seem like quite a lot. Regardless of the issue of specifying numerous libraries, you're restricting your users wrt. the third-party libraries they can use in their project to the ones you specify.
Can you specify implementation-agnostic libraries ? e.g commons-logging, which will delegate to existing logging frameworks under the covers. If your users are already using something other than log4j, then this will permit them to carry on without having to switch.
Secondly, is your framework doing too much ? Instead of providing a chat implementation, why not provide a suitable API such that clients can plug in their own chat/notification mechanism. That way your framework becomes more generic and (again) your clients can choose what/how to implement features. A rich client API will give your users many more options and extend the usefulness of your framework.
Should I limit the amount of frameworks that I am using?
If you are really using/needing them, not really. I would just try to avoid overlapping libraries and adding a library if you're only using 1% of it.
How should I distribute it? Would an independent jar for being used in other programs and a big combined jar for a single program be okay?
Many projects are distributed as a zip/tar.gz distro. But for a framework, making it available as a Maven artifact would be a great plus (in which case, make log4j and the log4j binding optional).
Is this many dependencies normal?
Firstly, you don't have that many dependencies. Secondly, there is IMO nothing wrong with reusing a logging facade, a persistence library, utility classes, etc (not using such libraries and writing your own code to replace them would be stupid). Thirdly, most users don't care, especially if you are delivering nice features (instead of spending time reinventing the wheel and, ultimately, creating bugs).
the scale of your project ie what it accomplishes and in what environment it will be used will balance out against how many dependencies and ease of configuration in general when people assess the suitability of your project.
you haven't really hinted at what your project attempts to achieve so it's difficult to say whether you have a bloated stack. however, for something reasonably useful I personally wouldn't have a problem with most of those jars.
the only thing that rings alarm bells is the database layer and the jdbc driver. if your project is a 'framework' i fail to see how a particular jdbc driver fits the model, and persistence in general does not quite fit the model of a framework.
That might seem like a lot of dependencies, but I don't think it is in reality. Certainly, there doesn't seem to be much gratuitous duplication of functionality. Most of the dependencies are doing things that you'd otherwise need to implement yourself.
For me who's trying to pass this off as "lightweight", this seems like way too much.
Maybe you need to adjust your rhetoric. :-)
Seriously, if those dependencies are necessary, the only way you will be able to get rid of them is to either code equivalent functionality yourself (bad idea) or drop the corresponding functionality (maybe a bad idea). Which is more important to you; being lightweight or being functional?
EDIT
Functional is key in the end. I can have my own custom implementation of everything but it would be full of bugs I guess. However I would like to keep it small as small and easy do attract people.
Well you clearly understand the issues. The decision is yours to make. (But don't forget that while some people are put off by "bloat", others are attracted by lots of functionality.)
I suppose that there is a half-way solution. Keep the functionality, but make it optional and provide some way that people can configure it in / out. Of course, the downside is that this means that you have to test multiple permutations of configuration options, and it makes installation / configuration more complicated for your users.
I think you worry to much :) The number of dependencies is not relevant, the maturity of them it is. If you will drop functionality/usability/flexibility/etc just because you want to keep the number of dependencies "low" it would be a loss for you (and your clients).
We basically need to be able to adjust behaviour at start-up time, by providing desired classes to be produced by various factories inside our application (to avoid the hard binding of the "new" operator).
I am aware that this is provided by several large frameworks, but I was looking for something easily used by a stand-alone Java application without being gigantic.
Any suggestions?
Edit: It is my experience that frameworks tend to grow big as part of maturing (and complex too). I need this to be retrofittable to a legacy application as part of major refactoring (technical debt), so simplicity is essential of the used libraries. I do not mind having to do a bit of coding in our application, but it must be very visible what is going on. AOP has a tendency for moving stuff out of the way, and that may make the application harder to maintain.
Edit: We have now reached the point where we actually need to make a decision. The application will probably live for decades so we need to make a reversible decision with a framework that will be maintained for hopefully as long. I really like the static type check available with Guice, but not that the annotations bind explicitly to Guice instead of being external like in Spring. I also like that code appears to be more concise when using Guice as opposed to Spring. We need something that is robust and helpful. We do not need more than just DI at the moment. Is there a use case that definitive says go for one of these?
Edit 2011-07-27: The final decision was to use the JSR-330 API in code, and choose on a per-project basis if to use Spring, Guice or Weld. For stand-alone applications Guice has worked well so far as the JSR-330 implementation.
You can always use Spring Framework 2.5. It is a big one, but if you planning to use only DI you can use spring-core and spring-beans modules, which are pretty small (ca. 500KB and 300KB).
There is also Google Guice 2.0 which comes with a package with only basic stuff (no AOP) and it's 430KB.
Have you looked at the Google Guice framework? It's pretty lightweight and annotation-based, avoiding XML configuration files
There's also Pico- and Nano-container (from codehaus) which are quite lightweight although the last time I looked (admittedly a few years ago) the documentation was lacking.
I must say that I agree with others about what I assume is your presumption that Spring is massive and confusing. It's really a very simple IoC container and to be recommended.
There are a couple I know of you might find useful:
PicoContainer
Plexus (used in Maven)
I've found Plexus very useful in standalone apps as it has optional utility components for CLI interaction.
By "gigantic" I'm going to assume you're referring to Spring, but that's unfair, since you can cherry-pick the bits of Spring you want to use. If all you need is the IoC container, just use the appropriate JAR files and the appropriate bit of the API, and ignore the rest of it.
Most answers so far seem to be concerned with the size of the jar files to be added.
However I think the more important question is the impact on the project: How many lines of code must be added/changed in order to use the framework?
Even the "big" spring framework is actually very easy to use:
You basically need:
a xml file that describes your factories.
one line of code to initialize the container by loading the xml file
The nice thing is that spring is non-intrusive. So you do not have to implement specific interfaces or add any specific annotations or imports to your classes.
At best the single spot where you actually initialize the Spring container is the only
place in your application that has an actual dependency to spring classes.
I would strongly suggest to take a look at Spring ME. Although originally meant to be a way to use Spring on Java ME applications, it also works fine for standalone applications.
True, it doesn't give you all of the bells and whistles that Spring (Full) has to offer, but then again, Full Spring is much much more than a simple dependency injection framework.
On the plus side: it's based on a (compliant) subset of Spring's configuration files, and the footprint of the runtime is 0%. In fact, there isn't any. Spring ME will take your application context, and turn it into a class that has no dependencies on classes other than your own.
What's wrong with Spring?
These days it's packaged pretty well so you wouldn't need to take the whole kit and caboodle.
As an aside, I'm not a fan of the annotation based injection frameworks. This is because the annotations are bound to the class rather than the instance, the later being a pre-requisite, imho, for DI. This means every instance of a given class gets the same object(s) injected, which seems to defeat the point.
Also consider that DI doesn't even need a framework, what's wrong with your main method wiring together the application?
If you want something maximally simple and appropriate, then write some code that does what you want done. Presumably this involves wiring together factories based partly on fixed logic, and partly on run-time settings.
This has the advantage that the set of possible run-time configurations is known, and so documentable and testable.
It has the disadvantage that an deploying an unanticipated logic change inherently takes an extra second or so of compile time, and (more significantly) can't be sneaked into production without full testing by disguising it as 'just a configuration change'.
About a year ago I asked myself a question very like this. So I spend a few hours reading the Spring and Guice documentation. After about an hour with Spring I was left feeling that I could get a basic web app going, but had no idea how to use it in a stand alone application. After an hour with the Guice document everything had clicked and I could see just how I to do what I wanted to get done.
Now on to recommending Guice? Well no. What does your team already know? If someone already knows say Spring leaver that knowledge and have them spread it about. Like wise with Guice or Pico.
If you want something really light weight you might want to have a look at fuse it's fairly extendable so might be what you're looking for.
cheers
N