We Keep Coding

Python egg that makes Java calls to a running Java application

I am making a proposal for a Python adapter to the Oracle NoSQL Database. The Oracle NoSQL Database runs as a stand alone java application, and at least in a Java program, you interface with it by telling your program the hostname and port to connect to, and some configuration settings. Then you make java calls off of the "kvstore" object that contains that configuration.
I'd like to make a Python library that essentially exposes Python versions of the java methods Oracle NoSQL has, and converts those to Java to speak with the running Oracle NoSQL application, but I'm not sure what technologies would be best to be able to do that.
Does anyone know what technology I would want to use? I'd rather not use Jython (so the application could run in a standard Python environment) or JNI (as it seems to have some nasty caveats.)
EDIT: The only potentially technology I've found so far is: Jpype
Would it work for me?
Also, here are the ideal requirements the library would have. I would consider using Jython or JNI if one of them really did best match these requirements.
Performance. The main benefits of Oracle NoSQL are performance and scalability, so that would be the most important component for the adapter.
Easy to implement for the Python users. In order for the library to actually be used by Python programmers, it would have to relatively easy for them to use in a natural sort of way.
Reliability. It would need to be possible for it to be trustworthy and bug free, while working on the platforms you naturally expect Python to work on. (This is what made me concerned about JNI. It sounds like it is platform dependent for its implementation, and can be error-prone.)
Development speed. The last point of importance is that it be relatively fast to develop. The team of developers would enjoy learning Python or C, but we know Java better than any other programming langauge right now.

I've tried to answer each point in order, with it's own related notes.
Performance: My opinion is that JNI would be the winner here, but I could be wrong, because Jython could be JITed as well.
Easy to Implement: I am taking this as you mean easy to consume the library? That would depend entirely on how you build the API, 1 to 1 method calls, object handles, etc.
Reliability: Jython is the clear winner here, because there is no room for error when you merely instantiate POJOs/ directly access the API right in the client code.
Development Speed: JNI can be very tedious. You are essentially learning CPython modules, C, Python extensions, JNI and then referencing an existing API built in Java.
All in all, if you can make the jump, I think you'd get more benefit in the short term embedding Jython, mostly because there you can directly manipulate the API. I have personally embedded IronPython in a .NET codebase with good success. Yes you lose native speed, but the tradeoffs are hard to justify with the amount of C coding needed for a working JNI bridge. That said, you may well find projects like the one you listed (Jpype) that can do much of the legwork for you.
I would be asking what features of a native CPython runtime I need that you lose when going to Jython. Is your existing codebase in CPython heavily reliant on CPython native features?
Anyway, there's my attempt at an answer.

Clojure libraries lacking (so use Java)...?

From Clojure it is easy enough to use Java libraries...but what libraries does Clojure not have that are best done with Java?

It isn't easy to give a straightforward question to this answer, because it would be first necessary to define the difference between a Clojure library and a Java library. (Even more so, because Clojure is a Java library :))
Ok, let's start with a premise that a Clojure library is any library written in Clojure and simply ignore the Java code in Clojure implementation itself. But, what if given library uses some Java dependency, like say one of Apache Commons libraries? Would it still qualify as a Clojure and not Java library?
My own criterion (and I am guessing yours, too) for the difference between the two is whether or not the library exposes a Clojure-style interface with namespaces, functions, sequences or a Java-style interface with classes, methods and collections.
It is almost trivial to write Clojure wrappers around such Java libraries. In my experience that is very useful if you want to fit in functionality of the library in overall functional design of your application. A simple example would be if you want to map a Java method against a sequence. You can either use an ad-hoc defined anonymous function to wrap the method call, or a named function from your wrapper layer. If you do such things very often the second approach may be more suited, at least for most commonly used methods.
So, my conclusion is that any Java library should be easy to convert to a Clojure library. All that is needed is to write a wrapper for it.
Another conclusion is that it may not be needed at all. If all you want is to call the method, you may still just call the method and avoid all the architecture astronautics. :)

One potential answer may be a bytecode library like ASM http://asm.ow2.org/
But honestly, with time, any library in Java can be written in clojure. Some Java code that compiled to different bytecode can be replicated if clojure uses ASM underneath.

I strongly prefer Clojure as a language for development in general, but there are several good reasons I have found for using Java libraries or writing Java code in preference to Clojure:
Leveraging mature Java libraries - some Java libraries are truly excellent and very mature. From a pragmatic perspective, you are much better off directly using Java libraries like Netty, Swing or Joda Time rather than trying to utilise or invent some Clojure alternative. Sometimes there are Clojure wrappers for these libraries but these are mostly still in a somewhat experimental / immature state.
High performance code - I do quite a lot of data and image processing where maximum performance in essential. This rules out pretty much any approach that adds overhead (such as lazy sequences, temporary object creation) so idiomatic Clojure won't fit the bill. You could probably get there with very unidiomatic Clojure (lots of tight imperative loops and primitive array manipulation for example...) but if you're going to write this kind of code it's often actually simpler and cleaner in Java
APIs with mutable semantics - if the APIs you are relying upon depend upon mutable objects, Clojure code to interface with these APIs can become a bit ugly and unidiomatic. Sometimes writing Java in these cases is simpler.
The good news is that because the interoperability between Clojure and Java is so good, there isn't really any issue with mixing Clojure and Java code in a project. As a result, most of my projects are a mix of Clojure and Java code - I use whichever one is most appropriate for the task at hand.

Libraries for building GUIs comes to mind.

Lots of APIs. In fact, Clojure itself is built on top many sturdy Java APIs like the java.util.Collection API. And well known Clojure APIs like Incanter are built on top of libraries like Parallel Colt, and JFreeChart.

I can't find the quote at the moment; but Rich said somthing to the effect of "clojure should use java where possible" and not wrap java unnecessarily. The principal being to embrace the java platform instead of fighting it. so the general advice becomes:
If a good java library exists use it, if not write one in clojure.

Best dynamic language to pair with Java on a Java project

What is the best dynamic language to pair with Java on a large Java project?
We are considering using a dynamic language for tests, controllers, services. Some options are Groovy, JRuby or Jython. What are the pros and cons of each for this? Ideally we'd be able to call Java from the dynamic language as well as call the dynamic language from Java.
EDIT: If it helps, we're using Hibernate with PicoContainer and Webwork.
Thanks,
Alex

There are really three dynamic languages that offer a very seamless interop with Java - scala, groovy and clojure. From there, I'd ask your team which language they would rather work in or have them try a prototype in each language and see what they think.
If the team efficiency isn't important in the beginning, look to what problem each language attempts to solve:
Groovy is going to be very loose but natural to experienced Java developers and allows fast prototype development due to it's duck typing.
Scala is going to enable you to write DSLs making it a good for frameworks and tools where you want to solve the problem in a language more akin to how you would describe the problem.
Clojure is going to impose lisp's functional programming and immutable state concepts and could be a very natural fit for problems in AI, natural language processing, etc.
Finally, I've gone down the path of looking for the perfect language to base projects on and have found there is no perfect language. All of the languages I've mentioned above compile to native JVM byte code and are quite solid. Sometimes you just need to pick a language that might not be as cool as the others but gets you on the way to solving your problem.

I recommend Groovy, principally because it interoperates seamlessly with Java, and is almost a superset of Java, so very easy for a Java developer to learn. I have absolutely no evidence to support this, but based on hearsay, guesswork, and personal experience, I suspect the Groovy community is much larger than that of either JRuby or Jython.
Incidentally, Groovy++ is way too immature to consider for production use, in my opinion.

The answer is, of course, going to depend somewhat on matters of taste and flexibility. If there are folk who don't have experience with Ruby or Python then Groovy is going to have a syntax much closer to Java (in fact it is a superset of Java), and consequently be a much easier sell.
I can't really speak to JRuby as I haven't used it.
Groovy gives you probably the easiest interop with Java of the three you listed. It also has a very nice BDD library in EasyB which I like a lot. On the negative side I don't think the features or syntax of Groovy really hang together very well. It can kind of feel like a whole bunch of separate extensions to Java.
Jython is of course Python so the syntax is different, but also has all the consistency of Python. Interop is very good at the script level but at least used to be a bit awkward if you wanted to write pre-compiled classes in Jython that you call from Java. The other main pro over Groovy for me is that is that you have a real REPL to interact with the Java project.
I would also mention Clojure, the syntax is even more different but the Java interop is excellent, probably the best of all, and again you have a REPL. On the down-side if folk have trouble adjusting to Ruby or Python syntax then a Lisp is probably right out.

Clojure is probably the best dynamic language for controllers and services. (depending on what you mean with "services".
Scala and Groovy++ has the best java interop, but those are not dynamic (well in Groovy++ you decide for yourself what is typed.). Scala has the look and feel of a dynamic language. Scala has good testing frameworks http://www.scalatest.org/ and Specs and AKKA is very mature for services and also has Java APIs

I'd suggest going with Jython. The syntax is clean, and you get whatever additional power/conveniences that Python gives you.
For example, if you were to go with Groovy, you are basically limited to only what Java will give you. Jython would add the powers of Python to that as well.
If it helps any, I've used Jython with Hibernate, SOAP, Corba, and EJBs and it is much easier than doing the same with just plain Java.

Scala - Java = ? (Or Clojure - Java = ?)

Is it possible for a developer to use Scala without knowing Java?
Is it possible for a developer to use Clojure without knowing Java?
Note: For example I am a C# developer and I use .NET without knowing any VB (Of-course WF 4.0 uses VB for statements, so I refuse any projects involving WF 4.0 :) ).
Summary I: Thanks to all for your answers. Now I can tell one can use these languages without knowing Java (The Language). Now I've got another question: How can one start with Java (JVM) ecosystem? What are beginner, intermediate and advanced knowledge-bases (bags!)? To be honest Java ecosystem was always a bit confusing to me, so it would be very helpful to provide some "reference references" and "defacto tutorials" for learning (curve!) JVM.
(OK! I will post it as another question!)

Scala, and Clojure even more so, are totally different languages from Java. The only thing they have in common is that they run on the Java virtual machine. You do not need to know the Java programming language to program in Scala or Clojure. It would be useful, however, to know something about the Java ecosystem (the Java virtual machine and runtime environment).

Sure you can. You'll lack the knowledge of the standard library at first, but that's not a big problem when you're starting with a new languages anyway. Clojure & Scala, however, put a heavy emphasis on the Java interop and do not try to hide the raw Java stuff, so you'd probably have to learn more about Java(not the language - the Java Platform) at some point to make maximum use of the language and the underlying platform.

Speaking from the perspective of knowing Java and Clojure (but not Scala), you can use Clojure just fine without knowing Java. You have access to the core language, which is fully-featured, and libraries (e.g. clojure-contrib).
However, one of the big benefits of using Clojure (versus other lisps) is access to the Java ecosystem - libraries, debuggers, JVM, etc.
So, yes you can use Clojure effectively and I wouldn't let a lack of Java knowledge hinder you, but you'll add to your bag of tricks if you eventually start to learn enough Java to take advantage.

Well, Clojure has been ported to the CLR. Probably easier thant the whole jump to the JVM since you are a .NET developer. :)
http://github.com/richhickey/clojure-clr

yes, thats me!
Has not been much of a problem for me except for when I needed a stupid simple swing gui. thats when I had to go back and review java.

Scala is totally different language than java. So yes, you can of course use scala without knowing java ( but of course you have to learn scala first). Even though, knowing java (or any OO language) + some functional language makes it easier to understand scala.

How safe would it be to use functional-java to add closures to a Java production project?

I would love to use closures in Java. I have read that they may or may not make it into Java 7. But an open-source project called functional-java has implemented functional features including closures.
How safe would it be to use such a library in an enterprise production app?
Is there a better way to add closures to Java currently?

Closures will definitely not make it into Java 7, due to a lack of consensus around a single implementation. See here.
The Functional Java library is thoroughly tested and actively developed. If you browse the source you will see that it's just simple bog-standard Java, and there's not a lot that could go wrong. I know at least one mission-critical enterprise app that uses it in production. This application has more than a million users.
Go for it.

Functional Java does not requires that you use closures -- the source compiles with any Java 1.5 compiler. Functional Java is far more complete than Google collections and just happens to allow you to use it well with BGGA.
Hope this helps.

I think you are looking at this issue from the wrong perspective.
I would love to use closures in Java. ... How safe would it be to use such a library in an enterprise production app?
Your decision making on what technology to use in an enterprise production app should be based on what is going to be best for the enterprise in the long term. You should be asking questions like:
Is adding the technology going to significantly improve the app from the standpoint of functionality?
What are the risks to the current project?
Does the technology claim to be "production ready"?
Is the technology supported. Is it likely to still be supported in 5 to 10 years time?
If you leave the company, is someone else going to take over maintaining your code?
What are the training / recruiting implications of using the technology?
In general, statements like "I would love to use Xxx" should have no place in enterprise decision making. (There are counter-examples; e.g. startups that bet the company on some new technology, but the real success rate is ... umm ... open to speculation.)
To sum it all up, if you are building enterprise production apps, you need to be conservative, and resist the temptation to use shiny new technology "for fun". Try to think like an IT manager.

If you are looking for a lighter solution to have closures in plain Java check out the lambdaj project:
http://code.google.com/p/lambdaj/

BGGA is still a prototype implementation, not sure how comfortable you are putting prototype code into production. check out google collections it gets you closer to the goal albeit more verbosely

Too old question to answer. Here we have lambdas in java8 to make your life easier.