We Keep Coding

What is the advantage of using Direct Web Remoting over a REST web services design?

I have an enterprise Java web application deployed in tomcat.
I am looking for ways to integrate a HTML & JS UI with the mid tier java code. So far, from looking around I have read that there are two ways to do this which fit my requirements :
1) DWR - Direct web remoting
2) Use REST web services.
I am not sure which approach to use.
Edit: What is the advantage of using DWR over a rest WS design?
Is there a better approach available? Some people asked me to have a look at GWT, but I am not sure if these are scalable.
Edit 2: I will most likely be using dojo for the UI development(Works well with REST/DWR??), and I want to keep the java code chages minimal. I understand that by using web services, I would have to write an additional WS client layer on top of the existing java code.
Also, the usual enterprise needs, performance, scalability, etc.

Ok so you're building a web client frontend to your java backend, there are a few ways to go about this:
You want to use java standards as much as possible, preferably living in the java EE world: JSF. You will need some knowledge of HTML, XML and perhaps even javascript (though limited). Personally I have had a pretty buggy experience with JSF+richfaces to the extent that I don't use it anymore.
You are very well acquainted with java desktop application development but don't know much about javascript: vaadin: it allows you to write plain java using desktop application terminology which will be compiled to javascript. Currently I am working on a big vaadin project and it is very nice to be working with pure java in eclipse, the downside is however that you are far removed from the actual frontend, so tweaking can be tricky. Additionally everything is stored in sessions (afaik) and scales poorly.
You are primarily a frontend person with unrivaled javascript skills: use a REST interface in java and a pure js/html/css frontend. Personally I think this is the cleanest design and I have designed a few of my own applications like this. The downside is that managing large javascript projects tends to be hard because...well...javascript sucks. The upside is that this will always be the fastest most lightweight option available to you.
Ideologically I would definitely suggest the last approach but this can be hard for large projects. If you have the hardware to throw at it, vaadin is a nice option. My JSF 2 experience was a bit disappointing due to the bugs in (necessary) third party libraries like richfaces.
PS: I have never heard of DWR but the last stable release seems 2 years old and all it seems to do is expose java code as javascript methods which can be better handled with a REST interface.

Open interface standards like REST and SOAP make it easy to build code to consume these services if you are using frameworks to build a REST Client and a SOAP client code respectively. But the point is that you need to have this client code to make calls to these services.
DWR on the other hand, generates this client code. Your java classes are ready to be called (like you would do in Java).

What are my options for building a rich web application?

I'm working on a client-side Java application for which I want to create a web-based user interface (which will be served up to a browser from the local app). At the risk of oversimplifying, its a microblogging tool that will support functionality similar to Twitter and Facebook. Its open source, and you can read more on our wiki.
Its important that this app is extremely easy to use, and I also want it to look good relative to contemporary web applications (such as the examples available on the Chrome web store). Since I am not a web designer, I'm hoping to use tools that give me an attractive UI "out of the box". Oh, and I need to keep the distributable file reasonably small (un him so whatever I use needs to be reasonably lightweight.
So far, I've looked at Sproutcore and GWT.
Sproutcore apps definite have the appearance I'm hoping for, but I'm not that familiar with JavaScript, and after some experimentation is not entirely clear to me how I will achieve some of the more sophisticated functionality that I think I'm going to need.
Being Java, GWT is more within my comfort zone, however I had a hell of a time just getting my development environment set up properly. It seems that GWT wants to dictate the structure of my application, but since the user interface is only a small part of it I'm not willing to build the entire app around GWT.
Something else I've been thinking about is using CoffeeScript (which seems more palatable him and him him him him him than JavaScript, and there are Maven builders for it), perhaps together with JQuery-UI.
What other options should I consider?

To recommend two big javascript frameworks Sencha and Cappuccino.
These have an extensive UI toolkit out of the box. The latter has an MVC framework build in, the former has a general JavaScript library build in. Both come highly recommended.
Also as recommended jQuery UI is a reasonable toolkit. I would personally couple Sencha for its rich UI toolkit and a lightweight MVC framework like Backbone. I'm afraid this does require learning JavaScript well.
You can also instead drive your JavaScript UI toolkit mainly from the server and skip the clientside MVC. For that sencha should surfice as a UI toolkit, but if you want meaningful functionality you will need some more structure. I would also recommend RequireJS for modular file management.

Take a look on Vaadin its based on GWT but simpler to use. I use it since 2 years and i'am not a designer too.

Take a look at jQuery & jQuery UI (jQuery Modile if you are looking to deploy to hand held devices)
Also, since you mentioned that you are not a designer, you might want to implement the front end in an MVVM pattern to separate the UI from the business logic of your applications. Knockout is a nice way to do with with decelerative data binding

I don't like writing plain JavaScript, so I'll tell you little bit about Java based frameworks. Recently, I've picked Google Web Toolkit.
Pros:
plenty of materials
Eclipse + Google plugin
Very intuitive API
3rd party libraries
Fancy widgets + optimized JavaScript
lot of generics
very active community
awesome async callbacks
Cons:
very slow debugging, that will drive you mad sooner or later
lot of deprecated code for so young framework
Mixed feelings:
You don't have to write declarative UI (you can do Swing-like development), but your code grows bigger and less maintainable, so it is quite probable that you'll eventually learn some declarative UI like UI Binder
Visual designer for GWT is somehow nice, but very unstable (eclipse) :(
Some considerations:
Do some prototyping with Vaadin. Try some debugging. Then try GWT and debugging on similar sized project. Consider debugging as the VERY important decision factor, because GWT debugging was the most frustrating thing for me.
Consider JSF + some nice rich component framework for JSF like RichFaces, IceFaces, PrimeFaces etc (but you app won't be "light" and "small" anymore if you really care). Be sure to be perfectly comfortable with environment (IDE + plugins), nothing beats JSF there (I'd pick RichFaces + Seam today, if full Java EE is option).

If you don't want to learn javascript I recommend you GWT with Sencha-GWT (GXT), you can develop all your app in java language, and also debug your code in java.
It is very easy to create a web app with this technology, and provides almost all funcionality available in javascript version.
Also you can mix it with hibernate and giled to persist your data, and giled helps you to serialize hibernate objects.

I have been writing an application in SproutCore for the front-end and a Java backend. In my mind this works perfectly.
The backend can stop worrying about HTML and JavaScript contents on the client side, if the request is XHR or plain request/response. I can implement the backends using plain old Java Servlets, and only worry about the data that is passed between the client and the server (JSON for the most part).
The client becomed fast and responsive, and SproutCore has a rich amount of components that you can mix and match.
So in other words with SproutCore (and Cappuccino and other full-blows MVC JavaScript frameworks) you can let your client do what it does best : Render and display HTML, CSS and JavaScript, while leaving your server to worry about what it does best: retrieve, store and update data. If you are interested, the source code for this project is available with the GPLv3 licence: https://github.com/joachimhs/EurekaJ
I have written an article about the differnt approaches between server-side MVC and client-side MVC here: http://haagen.name/2011/05/24/The_RIA_MVC_Model.html.
Also SproutCore comes with the MIT licence, so its avaiable for use without (m)any restrictions.

Which Java technology should I use, if I want to build a website with thousands of users?

Sorry for asking this question, but I searched all Java-related questions, but I got more confused. I am still not clear what should I start with.
My main thing is building websites in Java, because someone told me that there are some machine learning or AI libraries available in Java which I can use in Java. So I decided to use that as it can reduce my work.
Now I don't know Java at all. Some people say the following are used to build websites, like:
Servlets alone can build a website
JSP alone can also build a website
Struts
Spring Framework with Hibernate
Seam
Java EE also for websites
I am confused. Where should I start from? Where does core Java fits in here?
I was thinking of perhaps learning Python as I know I have to learn Python only, not its variations.
So please guide me to one thing which can solve my basic purpose of using ready-made libraries of AI.
If I can do that with JSP then I will start with that.
But if I need to learn all of them, then it’s better if can start learning Python.
I have five months to finish the website.
I really don't know why there are many branches of Java for doing one thing.

To correct some misconceptions of yours:
servlets and JSP are Java EE
Seam and Struts can be used together with Hibernate as well
Spring MVC is a web framework. Spring as a dependency injection framework can be used together with any other framework.
there are many more options, like JSF, Tapestry, Click, Play, etc.
It depends on what will the site will be doing, but you can choose any.

Experience has shown that building complex HTML forms (especially validation) and Ajax functionality is hard to get right. You will most likely appreciate having a library to assist you with that. The same thing with database access.
Everything builds on servlets. Learn those in order to understand the libraries you will use.
I would suggest learning JavaServer Faces (JSF) with facelets. The easiest, while being powerful, is JSF 2 in Java EE 6. The easiest way to get started is to use NetBeans with GlassFish 3.

For static web pages, HTML alone is enough and if you want to make it more user-friendly and interactive, use JavaScript.
There are lot of frameworks you can use directly in JavaScript, like jQuery. Now if you want to make some more functionalities, like database interaction, then you will need either PHP, Java (JSP and Servlets) or .NET (there exist some more, but I’m aware of three only). Each of them have their library to interact with database.
Since you are interested in Java, go through Servlets, HTML, JavaScript, and JSP. Once this is done, you can look into JSF and EJB beans. So if you have knowledge of Java this will not take too much time, because the syntax is the same; the only difference is the concept.
So you will have a frontend (HTML, JS, and JSP) which will interact with the backend (Servlets and the database) which will be deployed in a Web container. To deploy, you can use an one of the available application servers, like Apache Tomcat, GlassFish, or WebSphere. To develop your application, you can use Eclipse as the IDE, which is one of the best IDEs I have used.
So decide your architecture and design, break it into components and start writing small applications first. Once you are experienced in small application, start writing an application where there are more things you have to care of, like load on database, availability, security, etc. For that, you can refer a very famous question on Stack Overflow, What technical details should a programmer of a web application consider before making the site public?.

The closest to core Java is servlets. No extra syntax, very minimal library, but no included functionality.

For building websites, the main thing is HTML and HTTP.
All technologies (including Enterprise Java or Server side Java) do the same—to understand an HTTP request and respond back with HTML. All that you mentioned do the same thing with a different level of sophistication.
Start with Wikipedia.org to know what technologies do what. If you are completely new to Java, go through the Sun tutorials, do sample programs, and get comfortable with Java packages, JVM and classes. (This might take a week or two).
To build websites using Java, you'll have to learn Java Servlets—everything else (JSP, Struts, JSF, and Spring(!)) are built on it. Get Head First books on Java and Servlets; they will be useful.

You can always build the page in Ruby or something that will give you a website very fast. For the AI stuff that analyzes the bought items for the recommendations, use something unrelated to the page that only takes the database to work with.
I would not mix the whole web page stuff with the AI computing. You can do the AI things that will take a long time and maybe should done on a different machine to have your webpage also responsive with C++ or Java in normal applications.
Choosing your front end technique because of later usage is a very dangerous thing. Using the whole Java EE concept and learning it from scratch will take you much longer than have a quick start with your page if you do it in Ruby or PHP.
You can then build the first version of the page and generate a user base. Start working on the AI stuff once you got some data with other technologies.
If your page has performance problems because your technology does not scale, you can always begin switching parts of the stuff out with a faster technology or maybe simply a bigger machine. Get your page running before you build it bulletproof for the thousand users you are expecting.

If you’re trying to build a website from scratch, with no prior knowledge, your best bet is to go with frameworks that provide a complete stack for web development.
If you aren't dead set on Java, you could probably start learning Ruby on Rails, which would be easier to start with (if so, go buy a Ruby on Rails tutorial which goes through all the necessary things for modern web development, from MVC to version control).
For Java, try:
Play Framework is very similar to Ruby on Rails. It will get you started very quickly, and hopefully (if you follow conventions) handle all of your peripheral needs.
Spring Roo is a very-fast-to-work-with framework, but it might get hard to maintain and extend later on.
These two are full web frameworks, that is, they handle database, MVC (presentation layer), configuration setting, etc... (I'll explain shortly). And could even ease your deployment process.
If you're going to go through any other route (using several frameworks together), you are probably not going to finish what you are doing in five months.
To understand why, and to clear some things you wrote in your question, you need to understand that modern web development (and enterprise development as a whole) is comprised of several different technologies:
Presentation: the term web frameworks has been more or less taken to mean frameworks that deal with this side of web development, i.e., how to create a web page (an HTML page). Most of the frameworks offer an MVC like approach (which you can read about elsewhere), and this is what most of the frameworks/technologies you gave sit in—JSP and Struts (which uses JSP) are examples of web frameworks. Servlets is a lower-level API that standardizes how to answer HTTP requests (which JSP and Struts sit on top of).
Persistence: being able to show a webpage doesn't mean squat these days if you can't save data to a database, and that is what the persistence frameworks do (they ease the way you can save data to the database). Hibernate is a persistence framework.
For Java there are also dependency injection (DI), which is what Spring DI is. It's tough to explain DI to people not familiar with the troubles of Java, so I won't try to explain it, but it's a very important part of modern Java development.
apart from these core technologies, there are also many items that you'll need to learn to actually deploy the application (unless your using services like Heroku or Google App Engine, which should simplify things), and troubleshoot it later (understanding how concurrency and HTTP works).
To address the other technologies you brought up:
I've never really touched Seam, so I'm not sure, but it looks like a collection of other frameworks (mainly EJB 3, which itself is a collection of APIs, and JSF).
Spring denotes a popular open source group (the SpringSource group) that provides various technologies, but most likely when you hear about it they refer to two things Spring's dependency injection (DI) and Spring's MVC (which is a web framework).
Java EE is merely a collection of high level APIs which other technologies implement (Hibernate implements JPA which is part of Java EE for instance).

Google Web Toolkit -- only for Gmail type applications?

I am trying to build a web application, similar in feel to something like GMail, with one persistent AJAXy page.
I am told that GWT is the way to do this most effectively. However, I also want a framework that I can use for other things besides single AJAXy pages. Is GWT more scalable than this, or should I spend the time and build this thing in ASP.net MVC or Django?
I should mention, I'm trying to LEARN a framework that will be suitable for me to do whatever in.

You can use GWT also for parts of the page (a single widget or two), but yes, you are right,
even then it is geared towards the case where you do not refresh the page it lives on.

GWT works fine for both cases. I've written a couple applications at work with it, and I haven't had any trouble with multi-page navigation. As Thilo also mentioned, you can use as much or as little of GWT for manipulating your page(s) as you need or want.

You can do both styles of applications in ASP.NET MVC or GWT.
GWT makes it easier for GMail-type applications eg: web applications which mimic desktop application functionality.
On the other side, ASP.NET MVC is easier for dynamic sites. EG: This site is built on ASP.NET MVC, and if you look at it, its behaves more like a dynamic and interactive web site rather than a desktop-like application.
So its just easier to do web sites in one, and web applications in the other. So I'd think about what type of app you'd rather spend your time on and choose the framework accordingly - Realising, of course, that you can do both types of applications in either.

The nice thing of GWT is that in the end all you have is HTML and JavaScript. You can combine it with any other technology.
You can use GWT for the one page that needs ajaxy stuff and the rest with something more traditional.
Or you could even write everything with another framework and then put a GWT widget that needs more AJAX behaviours in a dedicated location of your page.
If you want to stay in the Java world you can use JSP,JSF, Struts, Wicket, or any other framework.
If you want you could even just combine GWT with a .NET or PHP. Just don't use the GWT RPC in that case (use JSON or some simple XML encoding instead).

Best server-side framework for heavy AJAX Java application

There are zillions of Java web application frameworks.
95% were designed before the modern era of AJAX/DHTML-based development, and that means these new methods are grafted on rather than designed in.
Has any framework been built from the ground up with e.g. GWT + Extjs in mind?
If not, which framework has adapted best to the world of forms with dynamic numbers of fields and pages that morph client-side?

Echo2 / Echo3 by Nextapp (www.nextapp.com) is totally awesome.
Advantages over GWT:
1) It is not limited to a sub-set of java like GWT
2) It is easier (in my estimation) to learn
3) Has extremely robust design studio for almost drag and drop designing.
4) It is very fast, and works very well on all platforms browsers
5) You can write your application using either java script or java
6) It has great and straight forward methods for handling events and actions.
Personally I think that for any web-application in which you are trying to integrate java and speedy delivery I wouldn't hesitate to pick Echo3 or Echo2.

If you're starting from scratch. I'd have to say Google Web Toolkit. I have to say it is incredibly powerful. You get keep using most of your Java tools. Plus, you don't have to duplicate code that exists on both the server and the client, it just gets compiled differently for each area.

I'd consider REST-style frameworks as well as the other recommendations here- Restlet or Jersey may be good choices for the backend, while you use something like JQuery or GWT on the front end. Both frameworks can easily produce JSON, and the REST style provides a nice clean line of demarcation between your client application and your server source; I find that JSF can make that demarcation pretty muddy.

I use JSF and IceFaces. Although JSF has a few limitations, IceFaces seems to work pretty well and has ironed out a few of the problems with JSF.
I haven't used a really good AJAX Java framework as yet, although Echo2 looks interesting.

I like the stripes framework. It lets you use whatever javascript toolkit you want.
Here is their documentation on AJAX

GWT is quite powerful and easy to use (all Java, no Javascript/HTML/CSS coding). If Google has their way it will be a dominant framework/tool in web applications development, and for good reason. It already works with Google Gears (which allows offline access to web apps) - and more than likely will be optimized to work within Google Chrome.

DWR
I use this to dynamically populate drop downs, and even filter them on the fly based on user input in other places on the form.

I like the combination of JBoss Seam and Richfaces, especially with the JBoss tools that are extentions to Eclipse - makes building these sort of RIA's incredibly easy.
Wikipedia contains some useful comparisons:
Comparison of JavaScript frameworks
List of AJAX Frameworks
Your choice depends on several different factors including whether you want the "work" done client-side (most javascript frameworks) or server-side (echo2 etc.). Other things worth looking at are tools like OpenLaszlo that provide Flash (I think) out of the box, but drop back to DHTML if there is no Flash player present.
Unfortunately I think the decision comes down to balancing several competing cocerns. Check out the comparisons and try them out - most come with online demo's for you to try.

Aptana has a server side frame work called Jaxer. This is from their site:
Jaxer's core engine is based on the same Mozilla engine that you'll find in the popular Mozilla Firefox browser. This means that the execution environment you use on both the client and the server are the same. It's Ajax all the way through and through. That means you only need one set of languages -- the languages that are native to the browser -- to create entire applications.
This framework is open source and has a very nice IDE based on Eclipse. Aptana is also working on a Javascript implementation for ActiveRecord called ActiveRecordJS. Potentially you could use this both client and server side with their framework.

GWT is one of the best AJAX framework that I used ever. Most important thing about this framework is that its maintained by Google. And Everyone know "Who is Google ?"
GWT is used by many products at Google, including Google AdWords and Google
Wallet. It's open source, completely free, and used by thousands of
enthusiastic developers around the world.
GWT provide rich widgets that can be used to built any application. Almost all the widgets they have.
Another important point is GWT is continuously developing and its also have stable release which is very good thing. Another thing Google has also released GWT-Material which is again a very good thing because everyone is moving toward material.
I hope this will help you!!!