We Keep Coding

Java libraries and frameworks overviews

Which web sites could you recommend, where I shall find overviews of different java libraries and frameworks that are currently preferable to use in development of applications?
update: To be more precise, I've liked to find a site that will be like a magazine about java, where will be the overviews, comparisons, best practices, examples and other useful information about java (technics, libraries, frameworks and so on) for different purposes. The aim of magazines is not to cover all the things of their subject area, but to present more actual, interesting and useful things only.

I really like the design on Open Source Software in Java. They've got it laid out by type to start with, plus when you dig down you can find several competing projects for each category.
For example clicking on 'HTML Parsers' gives this (and more - this is just a partial clip):
I hope this helps.

The world of java has so many libraries that it would be practically impossible to recommend anything without some kind of understanding about what kind of application you are actually writing. Are you writing a blu-ray disc, a game for a java phone, an android app, a desktop application, a server side computation process, a web service, a web site.... the list of things that java can and does do is huge.
The same thing goes for frameworks too. I would always say (although many people on this site disagree) that you should only look for a framework when you are finding something difficult, and the framework makes that thing easier, without making other stuff harder. Some people say you pick your framework first.....
Perhaps if you gave more details on what kind of thing you were trying to attempt, the community might be able to point you towards some useful stuff.
As a side note, remember that the internet is full of people with opinions - just because they are loud, it doesn't make them right - after all http://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you%27re_a_dog

Document displayer architectural question

I am thinking of working on an online homework submission system where students can upload their text documents that can viewed by somebody(Teacher/Professor) with viewing rights. This person can annotate/mark up/highlight this document and then can pass this document on for further review. After the review is done, this document is sent back to the student with proper feedback.
There are a lot of moving pieces here and I am having a tough time trying to figure out where to start. I am using this project as an exercise to design a scalable/secure/fast web-application using entirely open source tools that supports desktop like user experience but besides the UI-framework(Vaadin) and language(Java), I am having a little bit of trouble figuring out the roadmap needed to make some progress with this.
It will be great if the SO gurus can mentor me through this or provide me with a nudge in the right direction.
Edit: Thanks for the response. This is the standard three-tiered architecture that was described in the response. I need massive scalability and since the application will be primarily document centric and I may have to retrofit search at some later date, I would like to steer away from RDBMS. Since we have a massive amount of users posting their documents at any given time(lets say *.txt) files for the time being, I would need some kind of message queue to process this massive information influx. There has to be some kind of a fast transformation layer that takes the documents in all its formats and displays it in a format suited for annotations and markup....and the list goes on. Starting with the Domain model and moving down would be ideal but I am a little bit of a skeptic.

I'm a Java guy. How I would approach this problem will be,
Figure out the data model. What are the Objects that the system will be using.
Based on that design your database. You can use ORM frameworks like Ibatis or Hibernate ( Ibatis has a code generator which generates pretty much all the DAO's to access the tables. It gives you the CRUD methods and you can add on top of it).
Once you are done with that, then you can start designing your service layer. Its not a good practice to expose the DAO's directly to the controllers( MVC pattern). This is where your business logic should go in.
The choose one of the web frameworks available. The most popular java web frameworks are Spring MVC. You can also try Google Guice.
Final step will be to design your front end. I think your project will include a lot of javascript. So look into JQuery or EXT JS
I hope this helps you to get started with.

Not really a "full answer" but hopefully some food for thought.
I am using this project as an exercise to design a scalable/secure/fast web-application using entirely open source tools...
That's a great thing to be doing - so my question would be why worry about the architecture? I admit this is a bit strange coming from someone who see's themselves as an advocate for architecture, but "learning architecture" and learning new technologies can both be demanding. Maybe you just want to get up-to speed with one first, then tackle the other?
Architecting, designing and building a system that is "scalable/secure/fast" isn't trivial. In a "real-world" business driven case you'd have a system context to fit in with, and non-functional requirements to hit: these might suggest a different technology stack. Different key drivers will have a huge affect on how you approach things and what decisions get made - and of course the architecure would be built around that.
Edit:
I'd start with what was more important to you; then as I start on the "other" topic I'd be constantly checking what I'd learnt / my assumptions.
Depending on the two subjects there might be some "synergies" which suggest a different approach. Part of me wants to recommend doing enough on one to get a good basic grasp and then bring the other one up to a similar level - then move on. That way you're kind of more consistent.
Another part of me says - just do what's most interesting!
Should I plumb the depths of a given technology stack before moving forward or get the blueprint right from a tool agnostic standpoint?
As a start, you'd definately want to get a broad but shallow view; maybe go into some specific areas in a little more detail if they were of particular importance to you.
Or is my blueprint going to change because of the technology constraints?
By this I'm assuming you mean that: if you discover a "given technology works in such a way" and how it works it different form what you were expecting - then will this affect the blueprint?
It could do - that's why a broad but shallow view of the relevant technology should help you avoid the worst / most obviously bad mistakes. I'm also assuming that you have enough overal experience to know what you're looking at when you start doing this - that you can recognise an area that needs more detailed study before committing to a design.
At a high level, all you're looking to do is ensure you're not restricting options further down the path.

Deciding among with Java technologies should use - beginner

I am not new to software developing, but in the last years I was almost completely involved with web applications development, most likely to PHP with MySQL/PostgreSQL.
Now I am changing focus. I would like to start programming for desktop applications, with Java. The decision about language was motivated by the following reasons: it is a mature language, has space into the market, it is multiplatform, full of resources, and has some very nice IDEs (I am using netbeans 6.9).
I am not new to OOP concepts (or most of them at least), but Java has a huge gamma of solutions (technologies, frameworks and so on), and I am getting lost with that. Can't decide what is the most appropriate solution to adopt and learn from.
That said, let's get to the point. I will describe the software targets, some things I has adopted so far, and expect someone to give me some trail to follow filling the messy points.
My application is a small business application. Usual stuff about products, sales, cash management. It needs to run into a small office, with some computers (not a fully standalone desktop application). Not every user can do everything. Some would have privileges to access financial stuff, others only sell, others only fulfill products, and so on.
Per above highlevel definition, we know that it will operate like a client-server application. One server will be installed into just one machine, all others will just have the client. I am using HSQL as backend for persistence.
Would like to use Swing to build the UI (to keep theming based on the OS). Possibilities for the client include sending email (not directly, but through some already-set email client [Thunderbird, Outlook]) and printing (to any available printer in the network).
For this I am using Desktop API.
Alright, finally we got were I am loosing the grip: which framework / technology do you recommend to make development quick (pre-built forms based on the available tables), and as modular as possible?
As I said, different users would have different privileges, but instead of just checking and disabling, I would like to keep modules separated. In different packs, or something like that.
EJB? BlueJ? JavaBeans? OSGi? Swing Application Framework? I'm kinda lost with those. Any help will be appreciated.

If you're lost, keep it simple.
For the kind of desktop apps you describe, you'll need Swing and JDBC. If you switch to web, substitute JSPs written using only JSTL and servlets. You'll have to know HTML and CSS to write JSPs. You can embellish those with JavaScript as needed.
You'll go a long way with just those.
The others you mentioned are just noise until you become more comfortable:
EJBs are for enterprise scale apps. You get to leave a lot of complex stuff like threading, object pooling and lifecycle, etc. to an app server and concentrate on expressing your business logic in distributed, transactional components called Enterprise Java Beans. Not necessary to start.
BlueJ is a beginners IDE designed to shield folks from the complexities of Java. I wouldn't recommend it.
OSGi - modular deployments for web apps. Not necessary to start.
Swing App Framework? I'm not familiar with it - I don't write Swing. If you mean Spring, I think it's an excellent framework, but not necessary to start.

This is a bit off-topic, but I think it should be said anyway.
My application is a small business application. Usual stuff about products, sales, cash management. It needs to run into a small office, with some computers (not a fully standalone desktop application). Not every user can do everything. Some would have privileges to access financial stuff, others only sell, others only fulfill products, and so on.
You are attempting something very ambitious here. There are countless existing applications for doing this kind of thing for small business, some of them household names. What makes you think that you can can do better? What makes you think you can compete?
The fact that you are new to Java makes this doubly ambitious.
My advice would be:
If you have dreams of making lots of money out of this, forget it. You are >10 years too late.
If you are doing this for some client, your company or yourself to use, they / you would probably be better off using an off-the-shelf business application.
If you are doing this for "the public good", you labors would be more fruitful if you joined some existing open source business application project.
If you are just doing this "for fun" ... go for it. (This doesn't strike me as a "fun" project though ...)

Starting off with java, swing desktop app is normal... I must say though that the components included in the default, i never liked, i thought they were too ugly. https://substance.dev.java.net/
Is a look and feel lib for java.
So do your app, think it through though and make sure you have some sort of design in mind.
Its easier and often faster if you have an idea of where you're going visually than if you just start coding and hope you reach somewhere you like.
Get paper and pencil out,scribble little notes and drawings of your GUI.
You don't necessarily need frameworks and the lot, if your app will need to store data then look at the JDBC driver to connect to mysql, or have a look at embedded databases such as HSQLDB...

Choose 'better' or more familiar technologies for a new project?

I am looking to start work on a brand-new project, something I've been thinking about for a while as my first independent sellable project.
It's broadly speaking a web-based service application, and my first choice, server-language is quite easy... I know Java pretty well from working on Java web-apps in the past.
However my experience doing web-apps involved JSP, Servlets and JSTL... I know the ideas behind newer technologies like Hibernate/Spring but have never used them. So we wrote our own DAOs, handled AJAX by writing special mini-JSP pages that generated XML/JSON pages, etc.
I'm not hugely into the idea that Spring/Hibernate are the 'only' or 'right' way to do any Java web-project, but they are widely used. On the other hand, not only would trying to learn these increase initial development time, but I'd be using my learning attempts to build a production system.
I remember one of Joel's early articles said (I'll paraphrase since I can't find it)
"regardless what's cool, always use
the technologies that the lead
developer (or dev team?) knows best"
I wondered what people thought about that?
ps: should this be CW?

I work as a consultant, and I've seen a lot of projects where the devs started out with servlets+JSP because that's what they knew, and it's pretty simple to get started with. However, it gives the team an opportunity/excuse to write a platform of their own, which is more fun than using someone else's and just writing an application.
As the project grows, the team reinvents more and more wheels, quite a few of which end up square. That's where I enter the picture - adding new stuff to this semi-flexible platform has become so complicated that the devs can't keep up adding features and fixing bugs without calling in reinforcements. Just to add insult to injury, the internal devs are usually the ones who get assigned to do the boring bug fixes because bug fixes require more knowledge of the gory entrails of what has become the team's proprietary persistence-and-web framework, and so those gosh-danged consultants get to do the new, fun stuff.
Now, you shouldn't use a framework just because people have been regurgitating each other's blog posts about the awesomeness of it, but you should also realize that there are very good reasons why those frameworks exist (and why they're used). If you haven't used any web frameworks at all, I'd recommend you to take Spring MVC, Wicket or whatever for a test drive. They don't solve all problems, and they do cause some of their own, but the grand total is usually a productivity increase, especially if you're making advanced user interfaces.
I have been on projects where plain JDBC has been quite sufficient for persistence, and where no more advanced web frameworks than servlets+JSP have been needed, but those projects are a minority. Without having used a framework or two, you'll never whether your project is part of that minority that doesn't need one, or if it is part of the grand majority that does.

Don't try everything all at once - take on one new technology at a time.

Beware the lure of cool new frameworks! I'm currently hacking on a tiny little web app that just has a login, a few mostly static pages, and a few forms to request some information by email. It would have taken me maybe two days to do as traditional Servlet/JSP in MVC style. Instead, since there was slack in the schedule, I decided to use this project to get up to speed in Spring, Spring MVC, and Spring WebFlow. While it's quite possible that I'm just dense, it took me several weeks to get my head around the right way of doing things, I'm still not totally confident that I'm doing everything correctly, and the application is still not done. Fortunately, due to slack, I'm not in danger of the overall project schedule slipping, but I'm always asking myself if I'm going to have to scrap it and start over.
I have learned my lesson, though: next time, I won't be the one pushing a new framework unless its one I've used for production projects before. That said, I'm glad I now understand Spring (or at least I think I do) and will not hesitate to use it again next time.
So how would I learn a new framework next time? If there's a project lead (in this case I'm a project lead of a team of one, no help there) I'd use the framework that they put in place. If there isn't, or if I want to learn a framework that the project lead isn't using, I'd use it for a side project on my own time. Learning is good. Putting company work at risk by throwing untested technology at it is not so good.

I can say for sure that Spring is worth considering. It gives you as much as you can take, but it doesn't bother you with things you don't need.
For example, at the very beginning you probably need dependency injection only. Then you'll need help with database interactions and transaction management. Then you'll decide to apply MVC patter to your web-application. After that may be you'll realize that components of your system are to send JMS to each other. And so on and so forth.
For all this cases Spring has it's own simple, intuitive, light-weight solution.

When starting a new project limit the number of unfamiliar technologies / frameworks to use. Every framework takes time to learn and every framework has issues especially if not implemented correctly.
If you can I would recommend you look into the Play framework. It is a web framework for Java that focuses on developer productivity. You can choose to use Spring / Hibernate if you want but you are not bound by that. It has a very easy to learn implementation and you should be able to get a good idea within a day of playing around with it if it is what you are looking for.

It depends what the customer wants (in the world of consultancy).
You have to learn new technologies. Does the customer wants to pay for that?
Not all the caveats of the new ones are known, whereas the older ones are proven a lot more.
Of course, if everybody thought like this we would all be stuck with VB these days. You have to look for the right balance, and learn a lot yourself too so you can get an objective view on the technologies available, their up and downsides.

i personally would definitely recommend looking into spring, i've found it's saved me countless hours. hibernate is also useful if you need an ORM layer (and spring has nice integrations with hibernate too). they're certainly not the 'only' or even the 'right' way to do things (that's quite subjective) but they have both saved me time and effort, especially spring.

There is one major trap with any unknown technology. You do not know where the dragons are, and you do not know how to rub the new technology "with the hairs".
Learning that will take time, and you need to have that in your estimates. Also your estimates will most likely be too low...

Does it make sense to use a framework for a simple java web app?

I've done lots of java web development using jsps and servlets, and I have found this approach to be straightforward and flexible. Some of the groundwork involved though - such as managing database connections - is rather tedious, and it takes a fair amount of work just to get a new web app off the ground.
I'm therefore considering using a framework for the first time, but my impression of frameworks is that they're mostly used for large J2EE applications and involve a lot of complex configuration. What I'm looking for is something simple that (after the initial learning curve) will enable me to get up and running with a new web app as quickly as possible.
So my question is - does it make sense to use a framework for a simple java web app?
Note that I'm not asking which framework to use (if indeed a framework is recommended), as it has already been asked here.

If you don't use a web framework you'll usually end up writing one - poorly.

It makes a lot of sense. My team has spent the better part of five years with our open source stack and no matter and we have a "seed" project (works like appfuse) that we use to create all new web apps. Even the simple two pagers from the pov of maintaining the app, it looks like every other app, just smaller.
The short is you won't get any return on the investment right now, but you will as the project evolves and you maintain it.

Yes, I would use a web framework for the following reasons:
Increased navigation capabilities and controls. Even though you may not need them, they are there for you to use should you require them at any time
As others have indicated, apps grow over time and you will feel the need for the framework at some point in time in the future. When you need to add additional pages and navigation
Features that allow you to plug into other frameworks such as security and DB access frameworks. Spring is a prime example in the Java world. You never have to use Spring but it plugs in so well with Struts, Spring MVC, Hibernate, Acegi etc. It ends up saving you the hassle of doing all of the plumbing on your own.
Support!!! Good frameworks almost always have a vibrant community to support them and ask and answer questions.
It may seem like too much hassle initially but definitely saves you much time in the future

I'd say it actually matters quite a lot which framework you go for. Something like Spring MVC is fairly unobtrusive in your code and allows you existing stuff to run quite a lot as-is. Other frameworks have much more specific ideas about how you should do things.

Yes it does make sense. Apps can grow and change and might require something a framework can provide easily in the future.
For example at my workplace we have simple jsp / servlet app. It needs to be rewritten just because of what I explained above. If someone would have taken the time to just get the framework setup we would be in better shape today.

Yes it makes sense. However implicit in your question is that the wrong (for you) framework can be more pain than it's worth - and that's true. There is a world of difference between some heavy J2EE framework and something light and cheerful like grails.

What's the alternative? Rolling your own? Embedding all the navigation and logic in the JSPs?
I agree with those who say that a web framework is worth it. There are literally hundreds now (e.g., Struts, JSF, Spring, Wicket, etc.). Pick one that suits you.

There are two kinds of applications:
1) the kind that you throw away and never use again, and therefore should not worry about modularity, maintainability and clarity.
2) the real kind.
It may sometimes seem that your app may never have to grow, scale, or service a larger feature set / user base than you currently are planning for....I assure you, that perception is always wrong.
Frameworks, specifically things like Struts for MVC in Java, Spring for MVC and Dependency Injection, Hibernate for Object-Relational Modeling are all extremely valuable tools that lead to modularity, maintainability and clarity in your code. So, to answer your original question.... Yes, emphatically.

No it doesn't unless:
you don't have any java experience
in your team
you are doing just a prototype that
must be ready in hours
you don't trust your developers
enough to let them write your
application
you don't plan to let your
developers learn and improve
Danger in using frameworks include but not limited to:
you'll get all framework defects on
top on your own
you'll be unable to do realistic estimates until you
know perfectly the framework
your team won't learn how to program a
web server
you'll find your team passing more
and more time on google instead that
solving problems by writing code.
The only thing that is worst that choosing a open source framework is having a separate R&D team in the company who should create the "big complete ultimate company framework".

but my impression of frameworks is that they're mostly used for large J2EE applications and involve a lot of complex configuration
Not necessarily true. The good frameworks are built to scale well so that they will take you from small apps to very large apps. Many of the frameworks today are moving towards zero configuration so you will find them easier and easier to use.
You are right that it does take an initial effort to learn the framework itself but that investment pays for itself in the very first application you build. And meta frameworks like AppFuse makes it even easier to get started since it pre-configures the frameworks for you.

Frameworks make sense for most applications. It may be necessary for you to build your own or adopt some other framework. The most important questions is the granularity of your data structures. What I mean by this is do you need to just enter data or do you need to parse, compile and execute dynamic code?
If you think of a frame works as a gauge and to the left is an all ready existing frame work open source or closed and to the right is all custom framework then super impose your code on top of that frame work where the level of complexity increases to the right. The farther right you go the more of a stuggle you will have with the frame work (IMO).
However, you can also slowly migrate from existing to custom framework.
There is also the question about your business. If you work for a business that does not look at software as it's core compentency such as a bank or hospital then you need to way that into how much of a frame work you want to build.
Bottom line a framework of some type will always be useful.

Using a framework may add some overhead to your web-app development due to the learning curve. However, depending on the framework you choose, you may be able to realize the following benefits:
Scalability
Maintainability
Clear Division of your model (e.g. MVC)
Out-of-the-box components(SessionManagement,authentication,etc )
Third party plugins
Modularity inherited from the framework
Many others.
Besides, the learning curve is a variable factor. Some frameworks may be easier to learn than others depending on your skills.

It definitely makes sense to look into using a framework even if for no other reason than it'll give you a new area of knowledge. If you've got the time to invest in learning a framework then you may well find in time that it's easier and faster to use that for new projects no matter how simple they are.
Also, if I could vote for SamBeran's post then I would. Getting started with a framework will involve several moments of "oh great! I don't need to write all that boilerplate again!".

I'd say use one - like you said, it's rather tedious to get stuff up and running for even a simple java web app. If the framework helps you get the job done faster and provides the services you need, I'd say it makes sense.