We Keep Coding

Struts2 - Isolating WebContents from Back-end Logic

I am totally new to Struts2 framework (and Java web services), and just went through few tutorials.
I have a general question here, in most tutorials, web contents like pages and back-end logic are stored in one single war package.
If I would need to make some quick changes to the pages (small ones, like updating text or picture), seems like I would need to deploy the war package again (according to the hello world tutorials).
In real life web applications, is there a better way to apply quick web page changes?
Thanks in advance.

It really depends.
There are companies using a Continuous Delivery approach, that allows them to release in production in minutes. For this lucky guys, it's not a problem.
Other companies are uglily drowned in burocracy, and every release means time, money and documentation, and hence this should be handled carefully.
Since JSPs (and Facelets) are not simple views but contain logic (not serverside logic like in the dark scriptlets days, but presentation logic), they should be part of a release process. Then you should keep them in the usual WAR, and extract from it only the pure presentation files, referencing them externally: .css, .js, images, HTML files and so on.
This way, if you will need to change the company logo, or the background color, you won't need to release anything except the static resources, if instead you'll need to add a new functionality, this will imply a release process.
This discussion can change with recent front-end frameworks (AngularJS, React, etc...) because HTML and .js there are part of the business logic. In those case, I'd take out just .css and images, keeping the rest under release control.

Best Dashboard Architecture

I need to build a dashboard for an application, the dashboard will have different dashlets and each dashlet can have any one of the following things:
Graphs (JFreeCharts and some Javascript Chart)
Table data from tables
Data from external sources
Maps
What can be a good architecture for such kind of application?
What I have currently in mind is:
Each dashlet should have its own lifecycle and when the dashboard loads it should just show the UI of the dashlets initially.
After the page load each dashlet sends a server call (based on its type) to fetch its data
After the data has been fetched, each dashlet (based on its type) renders the data.

First of all, there are plenty of front-end frameworks to get you started. Some of the more popular ones include:
Backbone
Javscript MVC
Sproutcore
A bit of Google searching can yeild pros and cons of each and I would weight your options accordingly.
That all being said, the basic problem you posed actually seems similar to ours. In the end, we built something a bit different in house. Many of the frameworks out there are optimized to display a singular canonical "view" based on a Model reflected by the DB and a Controller to manage small changes. A dashboard has, in essence, a variety of different modules that must be doing their own independent things as you've mentioned in your question. Because of the high number of independent modules, I feel like you might feel pains in some of the frameworks listed above.
I can't tell you exactly how to implement such a module architecture, but here are some rules of thumb we used when designing ours:
Module Design:
Module-based. (Login module, Map module, each Dashlet may be a module, etc.)
Modules must have one Model, may have no more than one Collection (which is-a Model), and may have one or more Views.
A module may be used in multiple places and pages. The singular Model should stay the same, but the Views are likely different.
Rendering:
Almost all HTML on the page is written and updated by javascript modules. The template files are almost empty except for headers and basic scaffolding.
All modules render their full HTML selves and replace themselves into the DOM. The module should have as complete of a static HTML representation ready to go before inserting into the DOM. This means the render functions use “.replaceWith()” instead of “.append()”.
If simple HTML replacing isn’t an option (i.e. needs to be animated) a transition function should be defined detailing how to go from one rendered state to another.
Because rendering is expensive, Views by default do not auto-refresh on all Model changes. Re-rending happens through events only. _render() is in-fact an internal method.
Orthogonality:
A single inter-module event dispatcher on the page Controller handles all cross-effects between modules.
Modules should never “reach outside” of their own DOM context. If an event in one module affects another, it should go through the page controller dispatcher.
Each module as orthogonal as possible. They depend on each other as little as possible.
Each module in its own file.
Connecting to backend:
All modules use the same global backend adapter. Modules never talk to the backend by themselves. This makes your front-end back-end agnostic.
Recursive:
Modules are commonly included in other modules.
Modules re-render recursively.
Testable:
Because modules render static HTML, they can be predictably tested.
Modules must be testable.
Standard input -> Module -> Predictable static HTML output.
Standard events -> Module -> Predictable static HTML output.
If anyone knows of other frameworks along these lines, please share!

Our web app is based exactly on this architecture and in production since end of last year. You can see it at http://beebole.com
We just optimized the calls to our own server.
There is a single call to get the common data needed by most widgets, each time a screen is loaded.
Then if a widget needs additional data, it makes a call itself to our server.
The external widgets call their own data too, but to another server.

I would advise against using a custom web framework when there are so many free ones available.
As mentioned in another answer, the traditional MVC style frameworks don't really fit well to your 'dashboard' desired style of UI. They are best used for creating static web sites based on data retrieved elsewhere. They don't handle user interaction well and you usually have to hand roll your own AJAX to do anything useful without a page request.
A better breed of web frameworks to look at are the Web 2.0 fraemworks, also known as the frameworks which help you build web applications. It is important to understand the difference between web site and web applications. They are usually differentiated by the latter being interactive and the former being mostly static. Websites which also have some interactive components are still web sites. A good way to think of it is ask yourself "Does this feel like a desktop app?".
For web application development in the Java (JVM) realm, I would use Vaadin. It lets you write Java code similar to Swing programming, with event based methods. You can even avoid writting HTML altogether if you'd like by defining your views programatically. This lets you unittest your view logic (in web apps, there is more than usual) which is not posible with regular HTML template based frameworks. The other main advantage is that it has built in methods which allow you to write Java code to handle dynamic, asynchronous functionality and it all gets translated to JavaScript automatically. No need to write 4 different languages while writing your web app, just write Java for everything! Try it out, it is fun to work with!
Another web app framework that is getting alot of attention is Lift. I do not have experience with it but many devs I have spoken with have promoted it to me. I believe it uses HTML templates with Java code as the back-end. It is also apparently really easy to get started and your web app spun up. It also has built in support for doing AJAX like functionality. Worth looking into at least.
There are probably many more web app frameworks out there that would suit your needs. These all have the advantage of being tested, independently maintained, updated, and secure*. If you roll your own framework for this project, you need to worry about everything yourself. Written a web framework that doesn't offer anything new would be like written yet another programming language that isn't innovative; it is just a waste of time.

I think what you are looking for is more along the lines of managing or controlling your dashboard. I am designing something similar. I would suggest you look at google app engine it can be used to automate and control this: https://developers.google.com/appengine/docs/whatisgoogleappengine
Also look at these open-source dashboards: https://github.com/twilio/stashboard

What is the best Java GUI methodology for ease of maintenance and enhancement?

I need advice on how to rewrite a java GUI. Ultimate goal is easier to maintain & enhance.
What I have built is a Java Applet Client interface that acts and behave similar to Eclipse. developer can design their data entry forms without using a single line of code (drag and drop), and define its attribute. This part is pretty well iron out. however, i am left with more than 40,000 lines of codes that is very difficult to maintain.
Each time a bug is occur or a new enhancement, i normally cant program in a more direct way. more than half the time, i need to workaround the problem and that adds up the lines of code.
Consideration:
-Java Web Applet (because it runs on any browser with J2RE installed)
-runs on slow machine
-deployment of around 200 nodes and growing
Problems that i currently have:
-Listeners are all over the place. sometimes is inside the element.AddListener(new listener..). Sometimes is outside of the class, could be in another package that contain all the rest of listener.
Question: is it always good idea to put all listener in another package? if that is that case, i cant use "this." to get the reference i need.
-JTable this is a killer to me :( the problem i had on Cannot access the Jtable column data after set invisible still persist. Imagine i have JTable with 3 column. First column is a dropDown, second and third column is a textfield. Whenever a value choose from dropdown, i need to base on the selected value, and update to the second column and third column. the problem is, if the user click and it click on other row very fast, it will update to a column that is in the wrong row.
-Currently the program is coded in the sense of it is single thread. whenever the user does a http connection to the server side, reading a file, writing a file and etc, i need to make it as asynchronous process so it doenst feel like "program hang". what is the best way to do this?
Really appreciate help here! Thanks!

Lots of questions here and I'm not sure where to start but I can sympathize with you one this one. Unless you have a well seasoned team that has already gone through the pains of Swing application development things can quickly become out of control and unmanageable.
Before you adventure into re-writing a project I would start with defining some simple standards for development. Like package structures and listeners. I would also recommend splitting the application up into well unit tested modules or sub projects.
Also, ask yourself if you really need to re-write the application or does it just need some TLC. As a consultant and Director of IT I see developers always wanting to re-write applications just because they've learned something new or don't think it's up to par. When they come to me and tell me that it's junk and needs to be re-written I usually send them back and ask them to come up with alternative solutions to a re-write and the impact of each solution including - doing nothing. In a lot of cases we didn't write the application at all.
[UPDATE]
Lastly, If you are going to re-write I would use a Domain Driven Design and MVC approach. Yes, I said MVC for desktop applications!. We've had great success with these methodologies. It keeps a good separation of concern and makes things easily re-usable. It also provides the structure to easily switch out the presentation layer. Most importantly it's easy to unit test and any developer that understands MVC can understand the basics of your project without knowing the details.
I have some more thoughts but i'll leave it at that for now. ;)

use dsl for gui:
swinghtmltemplate
swixml
yaml
there are some more of them
this will remove the need to describe listeners, allow binding in dsl manner

Why dont you just reuse the eclipse framework to build your own gui instead of writing it from scratch in Swing ?

Java, moving from desktop app to web app

I'm going to write my first Java based web app, and I'm sort of lost how to begin.
Firstly, I would like a web app and a desktop app that do pretty much the same thing, without the hackish idea of embedding a web browser into the desktop app because that doesn't allow to easily make changes to the desktop without affecting the web app and vice versa.
Now, here my questions.
Right now, I have a bunch of POJOs and they communicate with a single class that, right now, uses a flat file as a "database", of course, in production, I would use a legitimate database and just change that single class. Is this a good idea? Will I be able to go from POJOs to a web app?
Should I use a framework? I would like to have this app written pretty soon, seeing that all the buisness logic is there, I just need to wrap it so its usable, so, I don't want to spend an extreme amount of time learning, say, Spring (which AFAIK is huge), but, I don't want to keep reinventing the wheel throughout my app either. I can always just use JSP and scriptlets...
If you said yes to the above, what framework(s) do you suggest? Please note that I would like a framework that I can start using in maybe 3-4 weeks of learning.
Will I have to start from scratch with the POJOs that I have written? They're well over 30k LOC, so, if it is like that, I'll be hesitant.

You will need:
a web framework. Since you have Swing background, JSF 2 will be your best bet (everything will be painful, of course, but JSF will get you up and going quickly and will help you avoid the most tragic mistakes). Also, wrapping business pojos into web guis is the main use-case for JSF and it's biggest focus.
a "glue framework". One thing that is much different with web applications as opposed to desktop ones is that you cannot create view components by yourself - they must be created when browser requests a page. So you have to find a way to create the view objects and deliver all the references to the pojos that represent logic, some of which may have very different lifecycles (this is not a problem on desktop, but on web you have to distinguish between pojos that live along with the whole application, along with a single user session, along with a single request, and so on).
The "glue framework" could also provide the additional benefit of managing transactions. You have three choices:
Spring. It's not half as complex as you thing; you only need to learn some basic stuff.
EJB. You would need a real application server, like Glassfish or JBoss
bare JSF has good support for dependency injection, the only drawback is the lack of automatic transaction management.
If I were in your position, I would go with bare JSF 2.0 - this way you only need to learn one new technology. At first, try to avoid libraries like PrimeFaces - they usually work worse than advertised.
edit - and addendum
or - what is "dependency injection"(abridged and simplified)
When request comes to a web application, a new task starts in a new thread (well, the thread is probably recycled, but that's not important).
The application has already been running for some time and most of the objects you are going to need are already built and should not get created again: you have your database connection pool, maybe some parts of business layer; it is also possible that the request is just one of many request made during one session, and you already have a bunch of POJOs that the user is working on. The question is - how to get references to those objects?
You could arrange your application so that resources are available through some static fields. They may be singletons themselves, or they could be acquired through a singleton locator. This tends to work, but is out of fashion (hard to test, hard to refactor, hard to reuse, lifecycles are hard coded in application). The real code could look like this:
public void doSomething() {
Customer Service cs = AppManager.getInstance().getCustomerService();
System.out.println(cs.getVersion());
}
if you need clustering and session management, you could build a special kind of broker that would know and provide to anyone all kinds of needed objects. Each type of object would be registered as a factory under a different name. This also works and is implemented in Java as JNDI. The actual client code would look like this:
public void doSomething() throws Exception {
CustomerService cs = (CustomerService)new InitialContext().lookup("some_fancy_looking_name_in_reality_just_string");
System.out.println(cs.getVersion());
}
The last way is the nicest. Since your initial object is not created by you but by the server just after http request arrives (details depend on the technology you choose, but your entry point might be a JSF managed bean or some kind of action controller), you can just advertise which references you need and let the server take care of finding them for you. This is called "Dependency Injection". Your acts as if everything is taken care of before your code is ever launched. Spring or EJB container, or CDI, or JSF take care of the rest. The code would look like this (just an example):
#EJB
CustomerService cs;
public void doSomething() {
System.out.println(cs.getVersion());
}
Note:
when you use DI, it really uses one of the two former methods under the hood. The good thing is: you do not have to know which one and in some cases you can even switch them without altering your code;
the exact means of registering components for injection differs from framework to framework. It might be a piece of Java code (like in Guice), an XML file (classic Spring) or an annotation (classic EJB 3). Most of the mentioned technologies support different kinds of configuration.

You should definitely use a framework as otherwise sooner or later you'll end up writing your own.
If you use maven then simply typing mvn archetype:generate will give you a huge list of frameworks to choose from and it'll set up all of the scaffolding for you so you can just play with a few frameworks until you find the one that works for you.
Spring has good documentation and is surprisingly easy to get started with. Don't be put off by the pages of documentation! You could use JPA to store stuff in the database. You should (in theory) just be able to annotate your existing POJO's to denote primary keys and so on and it should just work. You can also use JSP's within Spring if that makes life easier.

... I a bunch of POJOs and they communicate with a single class that, right now, uses a flat file as a "database", of course, in production, I would use a legitimate database and just change that single class. Is this a good idea? Will I be able to go from POJOs to a web app?
qualified yes. if the pojo's are sane you should not have many problems. many people use hiberbate.
Should I use a framework? I would like to have this app written pretty soon, seeing that all the buisness logic is there, I just need to wrap it so its usable, so, I don't want to spend an extreme amount of time learning, say, Spring (which AFAIK is huge), but, I don't want to keep reinventing the wheel throughout my app either. I can always just use JSP and scriptlets...
probably. spring is huge, but things like grails or roo can help.
if you want to have a responsive web app, you will need to do some kind of rich client (AJAX). this may require a lot of your code to run on the client. this means writing a lot of javascript or using gwt. this will be a pain. it probably will not be so easy to just "wrap it". if you have written a swing app, then basically that code will need to run on the client.
If you said yes to the above, what framework(s) do you suggest? Please note that I would like a framework that I can start using in maybe 3-4 weeks of learning.
i like groovy and grails - grails uses spring-mvc, spring, hibernate. but there is roo, play and others.
Will I have to start from scratch with the POJOs that I have written? They're well over 30k LOC, so, if it is like that, I'll be hesitant.
the code that will run on the server can probably be mostly left alone. the code that has to run on the client needs to be rewritten in javascript or maybe you can get some reuse out of that code by using gwt,

The Play Framework is doing great things. I would recommend it highly. Having worked with EJB apps and Tomcat/Servlet/Spring apps it's a breath of fresh air. After framework installation you get a working app in a few seconds. Reminds me of Ruby on Rails or Node.js with the type-safety of Java.
Much quicker turnaround on getting started, faster development cycles, and a clearer configuration model than previous Java web app frameworks.
http://www.playframework.com/

Java/J2EE standard practices and design choices

I have a couple of design/architectural questions that always come up in our shop. I said "our", as opposed to "me" personally. Some of the decisions were made and made when J2EE was first introduced so there are some bad design choices and some good.
In a web environment, how do you work with filters. When should you use J2EE filters and when shouldn't you? Is it possible to have many filters, especially if you have too much logic in them. For example, there is a lot of logic in our authentication process. If you are this user, go to this site and if not go to another one. It is difficult to debug because one URL path could end up rendering different target pages.
Property resource bundle files for replacement values in JSP files: It seems that the consensus in the Java community is to use bundle files that contain labels and titles for a jsp parsing. I can see the benefit if you are doing development with many different languages and switching the label values based on locale. But what if you aren't working with multiple languages? Should every piece of static text in a JSP file or other template file really have to be put into a property file. Once again, we run into issues with debugging where text may not show up due to misspelling with property value keys or corrupt property files. Also, we have a process where graphic designers will send us html templates and then we convert them to jsp. It seems it more confusing to then remove the static text, add a key, add the key/value in a property file, etc.
E.g. A labels.properties file may contain the Username: label. That gets replaced by some key and rendered to the user.
Unit Testing for all J2EE development - we don't encourage unit testing. Some people do but I have never worked at shop that uses extensive unit testing. Once place did and then when crunch time hit, we stopped doing unit testing and then after a while the unit tests were useless and wouldn't ever compile. Most of the development I have done has been with servers, web application development, database connectivity. I see where unit testing can be cumbersome because you need an environment to unit test against. I think unit test manifestos encourage developers not to actually connect to external sources. But it seems like a major portion of the testing should be connecting to a database and running all of the code, not just a particular unit. So that is my question, for all types of development (like you see in CRUD oriented J2EE development) should we write unit tests in all cases? And if we don't write unit tests, what other developer testing mechanisms could we use?
Edited: Here are some good resources on some of these topics.
http://www.ibm.com/developerworks/java/library/j-diag1105.html

Redirection is a simpler way to handle different pages depending on role. The filter could be used simply for authentication, to get the User object and any associated Roles into the session.
As James Black said, if you had a central controller you could obviate the need to put this logic in the filters. To do this you'd map the central controller to all urls (or all non-static urls). Then the filter passes a User and Roles to the central controller which decides where to send the user. If the user tries to access a URL he doesn't have permission for, this controller can decide what to do about it.
Most major MVC web frameworks follow this pattern, so just check them out for a better understanding of this.
I agree with James here, too - you don't have to move everything there but it can make things simpler in the future. Personally, I think you often have to trade this one off in order to work efficiently with designers. I've often put the infrastructure and logic in to make it work but then littered my templates with static text while working with designers. Finally, went back and pulled all the static text out into the external files. Sure enough, found some spelling mistakes that way!
Testing - this is the big one. In my experience, a highly disciplined test-first approach can eliminate 90% of the stress in developing these apps. But unit tests are not quite enough.
I use three kinds of tests, as indicated by the Agile community:
acceptance/functional tests - customer defines these with each requirement and we don't ship til they all pass (look at FitNesse, Selenium, Mercury)
integration tests - ensure that the logic is correct and that issues don't come up across tiers or with realistic data (look at Cactus, DBUnit, Canoo WebTest)
unit tests - both defines the usage and expectations of a class and provides assurance that breaking changes will be caught quickly (look at JUnit, TestNG)
So you see that unit testing is really for the benefit of the developers... if there are five of us working on the project, not writing unit tests leads one of two things:
an explosion of necessary communication as developers try and figure out how to use (or how somebody broke) each other's classes
no communication and increased risk due to "silos" - areas where only one developer touches the code and in which the company is entirely reliant on that developer
Even if it's just me, it's too easy to forget why I put that little piece of special case logic in the class six months ago. Then I break my own code and have to figure out how... it's a big waste of time and does nothing to reduce my stress level! Also, if you force yourself to think through (and type) the test for each significant function in your class, and figure out how to isolate any external resources so you can pass in a mock version, your design improves immeasurably. So I tend to work test-first regardless.
Arguably the most useful, but least often done, is automated acceptance testing. This is what ensures that the developers have understood what the customer was asking for. Sometimes this is left to QA, and I think that's fine, but the ideal situation is one in which these are an integral part of the development process.
The way this works is: for each requirement the test plan is turned into a script which is added to the test suite. Then you watch it fail. Then you write code to make it pass. Thus, if a coder is working on changes and is ready to check in, they have to do a clean build and run all the acceptance tests. If any fail, fix before you can check in.
"Continuous integration" is simply the process of automating this step - when anyone checks code in, a separate server checks out the code and runs all the tests. If any are broken it spams the last developer to check in until they are fixed.
I once consulted with a team that had a single tester. This guy was working through the test plans manually, all day long. When a change took place, however minor, he would have to start over. I built them a spreadsheet indicating that there were over 16 million possible paths through just a single screen, and they ponied up the $10k for Mercury Test Director in a hurry! Now he makes spreadsheets and automates the test plans that use them, so they have pretty thorough regression testing without ever-increasing QA time demands.
Once you've begun automating tests at every layer of your app (especially if you work test-first) a remarkable thing happens. Worry disappears!
So, no, it's not necessary. But if you find yourself worrying about technical debt, about the big deployment this weekend, or about whether you're going to break things while trying to quickly change to meet the suddenly-urgent customer requirements, you may want to more deeply investigate test-first development.

Filters are useful to help move logic such as is the user authenticated, to properly handle this, since you don't want this logic in every page.
Since you don't have a central controller it sounds like your filters are serving this function, which is fine, but, as you mentioned, it does make debugging harder.
This is where unit tests can come in handy, as you can test different situations, with each filter individually, then with all the filters in a chain, outside of your container, to ensure it works properly.
Unit testing does require discipline, but, if the rule is that nothing goes to QA without a unit test then it may help, and there are many tools to help generate tests so you just have to write the test. Before you debug, write or update the unit test, and show that the unit test is failing, so the problem is duplicated.
This will ensure that that error won't return, and that you fixed it, and you have updated a unit test.
For resource bundles. If you are certain you will never support another language, then as you refactor you can remove the need for the bundles, but, I think it is easier to make spelling/grammar corrections if the text is actually in one place.

Filters in general are expected to perform smaller units of functionality and filter-chaining would be used to apply the filters as needed. In your case, maybe a refactoring can help to move out some of the logic to additional filters and the redirecting logic can be somewhat centralized through a controller to be easier to debug and understand.
Resource bundles are necessary to maintain flexibility, but if you know absolutely that the site is going to be used in a single locale, then you might skip it. Maybe you can move some of the work in maintaining the bundles to the designers i.e let them have access to the resource bundles, so that you get the HTML with the keys in place.
Unit testing is much easier to implement at the beginning of a project as opposed to building it into a existing product. For existing software, you may still implement unit tests for the new features. However, it requires a certain amount of insistence from team leads and the team needs to buy into the necessity of having unit tests. Code review for unit tests helps and a decision on what parts of the code need to be absolutely covered can help developers. Tools/plugins like Coverlipse can indicate the unit testing coverage, but they tend to look at every possible code path, some of which may be trivial.
At one of my earlier projects, unit tests were just compulsory and unit tests would be automatically kicked off after each check-in. However, this was not Test-driven development, as the tests were mostly written after the small chunks of code were written. TDD can result in developers writing code to just work with the unit tests and as a result, developers can lose the big picture of the component they are developing.

In a web environment, how do you work with filters. When should you use J2EE filters and when shouldn't you?
Filters are meant to steer/modify/intercept the actual requests/responses/sessions. For example: setting the request encoding, determining the logged-in user, wrapping/replacing the request or response, determining which servlet it should forward the request to, and so on.
To control the actual user input (parameters) and output (the results and the destination) and to execute actual business logic, you should use a servlet.
Property resource bundle files for replacement values in JSP files.
If you don't do i18n, just don't use them. But if you ever grow and the customer/users want i18n, then you'll be happy that you're already prepared. And not only that, it also simplifies the use of a CMS to edit the content by just using the java.util.Properties API.
Unit Testing for all J2EE development
JUnit can take care about it. You can also consider to "officially" do user tests only. Create several use cases and test it.