We Keep Coding

RPG (iSeries) Modernization using JTOpen - What is possible?

We would be in near future implementing a solution to modernize our iSeries applications
written as RPG programs with some stored procedures, and our preferred way is leveraging the latest and greatest of what Java has to offer in this space.
From googling and checking other questions here on STOVFlow, JTOpen seems to be the defacto
library/toolset which has worked for most and I was encouraged to see that Tomcat runs on an I-series box with out any issues.
With this as the background, I am thinking of the following as the high level sol arch
Install IBM JRE and use JTOpen's capabilities to invoke RPG Programs and in some cases directly call the stored procedures running on DB2
Have Tomcat host a modern web application built using Grails and other frameworks (Camel, Smooks) to provide an application logic layer which would fill any mediations, transformations required for the old functionality to be offered to the user from a browser
Questions-
If any one of you has been involved in such an exercise, please share the pitfalls with this approach
Is there a significant performance drop with respect to response times for the end user?
Would it be better to some how expose the JT400 code as web services and run the web app on a different machine altogether consuming these web services?

Be very careful with calling RPG from Java because RPG is not threadsafe without some changes.

When I was at COMMON, the best product I felt on the market was Profound UI. There are several others from a variety of vendors. Most of these products do not use Java. Java on the i tends to be slow. (There are things that can be done to make it faster, but native is always faster.) You'll pay the price for these products, but just imagine how much time it would take you to do this yourself. For the above, I was quoted in the $20+ thousand range. But like all i products prices vary greatly based on system.
To directly answer your questions:
I have been doing research on modernization as time allows, the products weren't quite there yet (at the time I looked) to use it for what we wanted to use it for (before COMMON 2011). Now it looks like it might work.
This really depends on your system. A newer system will have less problems than an older system. Web will always be slower than the green-screen. Hands-down entry people won't like it. Executives and younger people will love it.
Your slow point is running the business logic. It wouldn't matter which server the HTML is coming from.

I've found that for all practical purposes an AS/400 behaves like an AIX box seen from Java code, and you must use jt400 (jtOpen) to communicate with the AS/400 specific features like data queues, files etc. This works pretty well, but the slowness of invoking the JVM pressures Java based solutions to be long running.
Note also that QTEMP is generally unavailable as a mechanism to keep state due to the nature of prestarted jobs.
Under V6R1 Java 6 is available and runs pretty well in the "new technology" edition. You can then run almost all Java based solutions, including web servers like Jetty in it. Note that Java defaults to code page 819 when accessing IFS files directly. Windows clients using AS/400 as a network drive uses a compatible code page.

Drop-in replacement for MICO Corba?

We are currently using MICO to establish the communication between a server and a client, where the client is a simulator written in C++ and the server is a java program displaying an animation of what happens in the simulation. It seems that the developemt of MICO has slowed down to an almost halt and bugs that only allow us to hack around them (as we don't have the time to first figure out which parts of MICO are responsible for codegeneration and so on) keep us from making real progress.
So, does any one of you know of a good drop-in replacement? We would like to have the following:
The compiler can generate both C++ and Java-Code from the idl.
The project should still show considerable support.
The implementation should be open-sourced (GPL or BSD, or something alike), as we use our programs to teach students as well.
The migration from MICO should be as easy as possible. (This is not a hard requirement, but would be a good thing)
I found some other CORBA implementations, e.g. TAO, but I didn't find any of which I could
say that they are still supported. Correct me if I am wrong here.

The Free CORBA® Downloads page might be of interest to you.

Just naming:
orbit2 1, also pyorbit etc.
omniORB
TAO (has already been mentioned)
1 On my Ubuntu box, apt-rdepends -r liborbit2 returns 5530 lines...

I don't know where CORBA or MICO in particular has gone in the last 5 years, but back then a drop in replacement was not really possible, since differences between vendors where still there.
Not necessarily API differences (POA, etc.) but
in implementation behavior,
in custom extensions which were required to make it work in a real-world environment (threading, load balancing, security, etc.),
in how the development tools worked
and also about the whole deployment or runtime story.
We had Orbix ASP/2000/Whatever and ORBacus which were interchangeable having a small compatibility layer, some Makefile framework to hide differences in tools (e.g. the IDL compiler) and some scripts for wrapping ORB specific processes.
Unfortunately, ORBacus has long been bought by (then) IONA, which already make Orbix. IONA itself has been bought by someone else (I forgot). The original authors of ORBacus, plus some devs from IONA Orbix, changed their ways somewhat and produced Ice, which is not CORBA but somewhat alike - of course without the glitches ;-)
Concerning TOA, I think it would be the "best" choice concerning still being developed as it is driven by research on the Washington University. But last time I looked, they didn't have Java implementation but people seemed to use JacORB.
Maybe all this helps you little, unless it brings even more confusion :-)

TAO as C++ ORB is still actively supported and developed (see http://www.cs.wustl.edu/~schmidt/commercial-support.html). For Java I would propose JacORB.
On http://www.orbzone.org there is an overview of available corba implementations.

What does the future look like for Java Applets?

In the past, java applets were unreliable, due to the Microsoft/Sun JVM split. Flash took over, and Java applets became known for browser crashes and performance issues.
Now that the JVM is enjoying resurgence as a platform for dynamic languages like Clojure and Scala, what is the current and future outlook for the JVM for in-browser applets?
Are browser vendors continuing to address stability and performance issues?
How much market penetration does Java have?
Given the JVM's resurgence, will applets also see any kind of resurgence as an alternative to Flash, or are they going to become obsolete entirely despite the JVM's success?
(Yes, HTML5 is the way forward for many, many things. It will continue to displace Flash and Java in the browser wherever it is most appropriate. That's not what this question is about.)

Flash turned out to achieve what Applets tried to, and has done very well so. Small initial download, programs starts fast and run inside a web page.
That said, the abilities to launch applets were enhanced with the Java 6 update 10 revamping of the JNLP specification of Java WebStart, but I have not given it a closer look (we use Java WebSTart extensively for applications, not applets), and I suspect others haven't either.

With JavaFX Sun has improved the Applet plug-in in term of stability and start-up time.
The latest browsers support the new Java plug-in.
It's now also easier to create it's own start-up screen with progress bar.
In term of deployment 60% of people have Java 6 installed (riastats.com), 93% have Flash 10 installed
In term of usage, my opinion is that users don't care the technology used. They care about about things like: does it work? does it look good? does it start-up fast? does it do something useful/fun? is the application easy to use?

Java applets are dead. The lack of broadband early on contributed mightily to it's early demise. The fact that the Windows JVM was locked to a 1.1 version for so long was the other nail.
Java webstart provides something far better - on demand delivery of Java apps, caching, and automatic updates, without any of the browser integration woes.
Applets are dead. XMLHttpRequest came along and fulfilled most of the reason people were using applets to begin with.

To put it simply, you reach a lot of people through mobile devices now, and you'll be reaching people through mobile devices more in the future. Mobile device OS makers don't want to support the JVM on their mobile devices, and I doubt Oracle will want to either.
Stick with the standards. HTML5 and Javascript is still the way to go if you want to make a web app.
JVM languages are definitely enjoying a resurgence, and I'd imagine they continue to grow in popularity, but they will not (and, AFAIK, are not) using them to write applets.
Applets aren't integrated into the system, they're not well supported, and they're nasty resource hogs. Keep the JVM on your server where you get the full benefit from the JIT and the rest of the power of the JVM.

Is JavaEE really portable?

I'm just implementing a JavaEE assignment I was given on an interview.
I have some prior experience with EJB, but nothing related to JMS and MDBs. So here's what I find through the numerous examples:
application servers bind their topics and queues to different JNDI names - for example topic/queue, jms
the activationConfig property is required on JBoss, while in the Sun tutorial it is not.
after starting my application, jboss warns me that my topic isn't bound (it isn't actually - I haven't bound it, but I expect it to be bound automatically - in fact, in an example for JBoss 4.0 automatic binding does seem to happen). A suggested solution is to map it in some jboss files or even use jboss-specific annotations.
This might be just JBoss, but since it is certified to implement to spec, it appears the spec doesn't specify these these things. And there all the alleged portability vanishes.
So I wonder - how come it is claimed that JavaEE is portable and you can take an ear and deploy it on another application server and it magically runs, if such extremely basic things don't appear to be portable at all.
P.S. sorry for the rant, but I'm assume I might be doing/getting something wrong, so state your opinions.

Java EE, like (almost?) any standard, is something that implementers strive to advertise adherence to but desperately don't want to adhere to.
Consider this question: how does Red hat make money? By giving things away or by selling them? If the code you wrote could be easily transferred to another Java EE application server, this would interfere with them making money from you. The solution to this is the venerable "embrace and extend" technique that has been attributed to Microsoft but in reality has been the tool of choice for commercial software vendors since the first standard was published.
If you stick strictly to the Java EE APIs in your code then JBoss (or Geronimo (or JonAS (or ...))) will run it as well as any other compliant application server with the only changes being required in the server-specific deployment descriptors. This is the embrace stage.
Each server -- the commercial ones (like JBoss) in particular! -- also tends to add extra stuff to the API to "make things easier". (To be fair these often do make things easier.) Developers -- especially those not intimately familiar with the standard APIs -- frequently fall into the trap of relying on these extra APIs without wrapping them in any way, thus allowing these extensions to inundate their code to the point that they're difficult to remove should you wish to change platforms. This is the extend stage.
Name a standard from any point in software history and you'll find people embracing and extending (to the point that when people talk about "deadly embrace" I have to forcibly move my thoughts away from vendor lock-in problems to the proper terminology). You'll also find end-users (developer or otherwise) falling for it. Java EE is no different than any other technology in this regard.
Then you factor in just how badly-worded most specifications are and ...

Joel says "all non-trivial abstractions are, to some degree, leaky." I have found this to be very applicable to JavaEE. Consider that a JMS exception may be something transient such as a full queue. This is a typical fast-producer/slow-consumer problem and ideally the producer will throttle down to match the consumer's speed. But the error may also be fatal such as an authorization failure. In the first case retries eventually succeed (usually) whereas in the second case no amount of retry will help until humans intervene to fix the authorization failure.
So what do you do in your portable program to address this? One approach is to treat every JMS exception as fatal. Close all your objects and reinitialize the program. Sort of like killing a fly with a sledgehammer but very portable. Alternatively, you can check the JMS exception to see if it's a transient or fatal error and take some appropriate action. This is much more efficient but since the JMS exceptions are provider-specific it is hardly portable. Some of my clients have taken the approach of writing vendor-specific shims which catch JMS exceptions and do vendor-appropriate things with them so that the code can be "portable" (think: software equivalent of the Hardware Abstraction Layer).
And of course, this is just exception handling. Similar issues exist across the board. Consider reconnection details. Some transports bubble up a connection failure to the application or container. Some hide it away with the idea that the code should not need to know about this. But the reality is that virtually all messaging applications will need to provide an alert or log entry if the network is permanently down. You would not want the app to just hang forever if the network failed, right? So ultimately even an app running on a transport that provides transparent reconnect needs to code for connection failures. The specific features and behavior of the transport provider will leak up through the abstraction of JMS.
For my money, JavaEE makes the skills portable across transport providers. The application needs to be aware enough of the underlying transport provider to deal with the abstractions that bubble up to the surface. To the extent that you can avoid the leaks the app is portable, but no further.

That's just a partial answer but Java EE 6, more precisely EJB 3.1, finally specifies Portable Global JNDI Names. Prior to Java EE 6, JNDI naming wasn't standardized and each app server vendor was using its own rules which was indeed bad for portability (a kind of vendor lock in). Consequently, in the J2EE 1.4 world, if you wanted to ease the portability of your enterprise application it was necessary to implement various strategies, typically in your ServiceLocator class. The introduction of dependency injection in Java EE 5 reduced the need for lookups and somehow "improved" the portability but still, no standard for when JNDI lookups are needed and polyglot ServiceLocator are still required.

The core EE application can run unchanged. The external configuration is application server specific.

Any experience using Terracotta open source?

Does anybody have experience using the open source offering from Terracotta as opposed to their enterprise offering? Specifically, I'm interested if it is worth the effort to use terracotta without the enterprise tools to manage your cluster?
Over-simplified usage summary: we're a small startup with limited budget that needs to process millions of records and scale for hundreds-of-thousands of page views per day.

I am in a process of integrating Terracotta with my project (a sensor node network simulator). About three weeks ago I found out about Terracotta from one of my colleagues. And now my application takes advantage of grid computing using Terracotta. Below I summarized some essential points of my experience with Terracotta.
The Terracotta site contains pretty detailed documentation. This article probably a good starting point for a developer Concept and Architecture Guide
When you are stuck with a problem and found no answer in the documentation, the Terracotta community forum is a good place to ask questions. It seems that Terracotta developers check it on a regular basis and pretty responsive.
Even though Terracotta is running under JVM and it is advertised that it is only a matter of configuration to make you application running in a cluster, you should be ready that it may require to introduce some serious changes in you application to make it perform reasonably well. E.g. I had to completely rewrite synchronization logic of my application.
Good integration with Eclipse.
Admin Console is a great tool and it helped me a lot in tweaking my application to perform decently under Terracotta. It collects all performance metrics from servers and clients you can only think of. It certainly has some GUI related issues, but who does not :-)
Prefer standard Java Synchronization primitives (synchronized/wait/notify) over java.util.concurrent.* citizens. I found that standard primitives provide higher flexibility (can be configured to be a read or write cluster lock or even not-a-lock at all), easier to track in the Admin Console (you see the class name of the object being locked rather then e.g. some ReentrantLock).
Hope that helps.

At the moment, the Terracotta enterprise tools provide only a few features beyond the open source version around things like visualization and management (like the ability to kick a client out of the cluster). That will continue to diverge and the enterprise tools are likely to boast more operator-level functionality around things like managing and monitoring, but you can certainly manage and tune an app even with the open source tools.
The enterprise license also gives you things like support, indemnification, etc which may or may not be as important to you as the tooling.
I would urge you to try it for yourself. If you'd like to see an example of a real app using Terracotta, you should check out this reference web app that was just released:
The Examinator

You may want to take a look at JBossCache/PojoCache which is an in-memory distributed caching solution. The difference is it uses a simple API to propagate objects across your 'cluster' of caches, where as Terracotta works at the classloading/jvm level.
(They don't actually have their own JVM, but they modify classes as they are loaded to allow them to be 'clusterable')
Our company had a lot of luck with JBossCache, I'd recommend checking it out.

Update
What I see in the OP message is "well, I don't really know what we need (thus the lack of detailed requirements), but may be some enterprizey tool will magically solve all our problems, known and unforeseen? That would be awesome!"
With an architectural approach like this it's not gonna fly. No success stories from Teracotta would change that.
OSS is beneficial when the community around it can replace the commercial support. Suppose the guy have a problem in production. Community cannot help -- it's too small for the obscure product like this. Servers are down, business is in danger. You see? You need a commercial license up-front. No money? Well, then you're not a business, and probably not gonna become one (if nobody's willing to invest into it).
Sorry for interrupting your day-dreaming.
IMHO:
Terracotta is a clustering solution. Clustering is required for large, enterprise-grade applications. Large applications mean big budgets. Big budgets mean you can afford commercial license from Terracotta.
To put it in another way: if you don't have budget to buy it, it's probably not beneficial for your project.