We Keep Coding

Calling java code from python

I am trying to optimize the performance of some natural language processing in a python project I am currently working on. Basically I would like to outsource the computationally intensive parts to use apache OpenNLP, which is written in Java.
My question is what would be the recommended way to link Java functions/classes back to my python code? The three main ways I have thought about are
using C/C++ bindings in python and then embedding a JVM in my C program. This is what I am leaning towards because I am somewhat familiar writing C extensions to python, but using a triangle of languages where C only functions as an intermediary doesn't seem right somehow.
using Jython. My main concern with this is that CPython is the overwhelmingly popular python implementation as far as I know and I don't want to break compatibility with other collaborators or packages.
streaming input and output to the binaries that come with OpenNLP. Apache provides tokenizers and such as stand-alone binaries that you can pipe data to and from. This would probably be the easiest option to implement, but it also seems like the most crude.
I'm wondering if anyone who has experience interfacing python and java knows how much the performance is likely to differ between these options, and which one is "recommended" or considered best practice in such a situation - or of course if there is an entirely different way to do it that I haven't thought of.
I did search SO for existing answers and found this, but it's an answer from 3.5 years ago and mentions some projects that are either dead, hard to integrate/configure/install or still under development.
Some comments mentioned that the overhead for all three methods is likely to be insignificant compared to the time required to run the actual NLP code. This is probably true, but I'm still interested in what the answer is from a more general perspective.
Thanks!

Consider building a java server with existing language independent RPC mecahnism(thirift, ....). And use python as the RPC client to talk with the server. It has loose coupling。

Java SE and Scala Standard Library - cases when one of them is preferable

We all know that one can use Java libraries from Scala and vice versa. But even looking over the surface of Java SE and Scala standard library, we can notice that there are many parts in them that solve identical or at least similar problems. The trivial examples are collections, concurrency and IO. I am not an expert in either of two, but I suspect that in general Java SE is broader in size while Scala SL contains more conceptually advanced features (such as actors). The question is, if we have access to both libraries and have an opportunity to use both languages, are there some recommendations when we should choose Java SE features over Scala SL?

In general, when writing in Scala, I would advise always using the Scala libraries over the Java ones. My advice on specific areas would be:
Collections - Scala's are much better, and I would always prefer them over the Java equivalents. Scala does however lack a mutable TreeMap, so if you need that sort of structure you'll have to go back to Java.
Concurrency - Scala's concurrency features wrap Java's and are more advanced. I'd always pick them.
IO - I think this is one area where Scala is narrower in what it supports. I would generally use a Scala Source when possible, but there may be more unusual situations where you'll have to drop back to Java IO (or possibly use a third party library).
Swing - Last time I looked, Scala's swing wrapping wasn't complete, so if you're doing a lot of Swing related stuff, you might take the decision to use Java's swing components everywhere for consistency.

Scala Libraries fit into two general categories:
Original Scala Libraries. These are entirely (or almost entirely) written in Scala. Usually, people will write libraries from scratch for good reasons. Maybe Java lacks a similar library, or maybe whoever wrote the Scala one thinks the Java equivalent has serious limitations.
Collections is one such example.
Scala Wrappers over Java Libraries. In these cases, Scala uses an adapter pattern (or one of the other similar patterns) to provide a Scala-friendly API. These APIs are more fluent, integrate well with important Scala classes (such as collections and Option), and often make use of powerful Scala features such as traits to decrease boilerplate.
These libraries rarely offer more functionality than what Java provides, but reduces boilerplate enormously and makes code using them more idiomatic. Often, however, they present just a subset of the total functionality provided by Java. Depending on the library, it may or may not be possible or easy to extend it by accessing the underlying Java classes.
Scala Swing is great example of these.
In the particular case of scala.io, that is not so much a library as a crude wrapper just to handle simple common scripting tasks with an idiomatic Scala API. It's adequate for that -- and certainly much kinder on my eyes than java.io --, but not for any serious I/O. There's a real I/O library for Scala currently undergoing evaluation for adoption.
Another example I like a lot if scala.sys.process. It wraps over Java's Process and ProcessBuilder, providing almost all of the functionality, and adding some. Furthermore, you can use most of Java internals if needed (the sole exception is Process itself, which isn't really much useful).
My advice is to use Scala libraries were they exist and fit your needs, extend them if they are mostly adequate, but reach for Java libraries without hesitation otherwise. After all, having a high degree of interoperability with Java is a key feature of Scala.

Clojure libraries lacking (so use Java)...?

From Clojure it is easy enough to use Java libraries...but what libraries does Clojure not have that are best done with Java?

It isn't easy to give a straightforward question to this answer, because it would be first necessary to define the difference between a Clojure library and a Java library. (Even more so, because Clojure is a Java library :))
Ok, let's start with a premise that a Clojure library is any library written in Clojure and simply ignore the Java code in Clojure implementation itself. But, what if given library uses some Java dependency, like say one of Apache Commons libraries? Would it still qualify as a Clojure and not Java library?
My own criterion (and I am guessing yours, too) for the difference between the two is whether or not the library exposes a Clojure-style interface with namespaces, functions, sequences or a Java-style interface with classes, methods and collections.
It is almost trivial to write Clojure wrappers around such Java libraries. In my experience that is very useful if you want to fit in functionality of the library in overall functional design of your application. A simple example would be if you want to map a Java method against a sequence. You can either use an ad-hoc defined anonymous function to wrap the method call, or a named function from your wrapper layer. If you do such things very often the second approach may be more suited, at least for most commonly used methods.
So, my conclusion is that any Java library should be easy to convert to a Clojure library. All that is needed is to write a wrapper for it.
Another conclusion is that it may not be needed at all. If all you want is to call the method, you may still just call the method and avoid all the architecture astronautics. :)

One potential answer may be a bytecode library like ASM http://asm.ow2.org/
But honestly, with time, any library in Java can be written in clojure. Some Java code that compiled to different bytecode can be replicated if clojure uses ASM underneath.

I strongly prefer Clojure as a language for development in general, but there are several good reasons I have found for using Java libraries or writing Java code in preference to Clojure:
Leveraging mature Java libraries - some Java libraries are truly excellent and very mature. From a pragmatic perspective, you are much better off directly using Java libraries like Netty, Swing or Joda Time rather than trying to utilise or invent some Clojure alternative. Sometimes there are Clojure wrappers for these libraries but these are mostly still in a somewhat experimental / immature state.
High performance code - I do quite a lot of data and image processing where maximum performance in essential. This rules out pretty much any approach that adds overhead (such as lazy sequences, temporary object creation) so idiomatic Clojure won't fit the bill. You could probably get there with very unidiomatic Clojure (lots of tight imperative loops and primitive array manipulation for example...) but if you're going to write this kind of code it's often actually simpler and cleaner in Java
APIs with mutable semantics - if the APIs you are relying upon depend upon mutable objects, Clojure code to interface with these APIs can become a bit ugly and unidiomatic. Sometimes writing Java in these cases is simpler.
The good news is that because the interoperability between Clojure and Java is so good, there isn't really any issue with mixing Clojure and Java code in a project. As a result, most of my projects are a mix of Clojure and Java code - I use whichever one is most appropriate for the task at hand.

Libraries for building GUIs comes to mind.

Lots of APIs. In fact, Clojure itself is built on top many sturdy Java APIs like the java.util.Collection API. And well known Clojure APIs like Incanter are built on top of libraries like Parallel Colt, and JFreeChart.

I can't find the quote at the moment; but Rich said somthing to the effect of "clojure should use java where possible" and not wrap java unnecessarily. The principal being to embrace the java platform instead of fighting it. so the general advice becomes:
If a good java library exists use it, if not write one in clojure.

What languages would be a good replacement for Java?

I may be posting a premature question, and maybe I'm just freaking out for no reason, but the way Oracle is handling Java is not very promising. I am a nerd who fell in love with Java from the first sight, and use it all the time in my personal/freelance projects but now I am thinking of a replacement.
I am fluent in C#/VB.NET too but I am looking for something more like:
Open Source
Compiled
Cross-Platform
Object Oriented
Large standard library
Extensive documentation
Web development is a major plus
I was thinking about a compromise: Python/Django for web development (or PHP), and Qt for thick client development. Anyone with better thoughts?

Not so long ago, I decided to explore away from the JVM. I set foot on python, and even though i'm nowhere near the expert/ guru level, I dont regret it. Didn't choose C# (considered it) because I consider it to be more of the same. I alredy know (and like a lot) C++, so python seemed like something new, which is what I was looking for.
It fullfils many of your requirements. Particularly, i'm decided not to learn PHP, so the web frameworks in python came in great.
Not to mention, Python has a large community (also see here), always eager to help and teach, which I consider to be very important.
Just my two cents.

Might be worth loking at the other JVM languages - Clojure and Scala are the two I personally think are most promising.
Yes you are on the JVM, but you're pretty independent from Java the langauage and don't have to use any Sun/Oracle implementations if you don't want to.
Having said that - I think that you are worrying a little too much about Java, too many players (including Oracle!) have too much invested to let it go too far off course.

Try Scala. It looks extremely elegant and promising. Being object oriented and sharing a lot with java in a very concise manner.

Everything you said points to C#, except for the Open Source point.
To fix that, there's Mono.

You could try D. My one-sentence description of why it's an awesome language is that its generic programming/compile-time introspection/template metaprogramming facilities are good enough to give you almost flexibility of a duck-typed language, while its execution speed and static type checking rival or exceed C++ and C#.
I think it meets your requirements quite well.
Open source: The frontend to the reference DMD implementation is open source (the back end isn't due to restrictions beyond the author's control). Work is underway to glue the reference frontend to open source backends such as LLVM (LDC) and GCC (GDC). In the case of D1 (the older version of the language) the LLVM port is fairly mature.
Compiled: D is meant to be compiled to native machine code, i.e. raw, inscrutable hexadecimal numbers.
Cross-platform: The reference DMD compiler supports x86 Windows, Linux, Mac OS X and FreeBSD. GDC and LDC will likely support a lot more CPU architectures.
Object oriented: D isn't a "pure" OO language in the Ruby sense of everything being an object, or in the Java sense of not supporting any other paradigm. It does, however, fully support Java-style OO as a subset of the language, along with procedural and functional style programming.
Large standard library: D1 has Tango, which qualifies. D2 has Phobos, which is not "large" yet by modern standards but is larger than C or C++'s standard lib. However, recently there has been a large interest in contributing and Andrei Alexandrescu (its main designer) has accepted several new contributors, including myself.
Extensive documentation: The standard library and language are reasonably well documented at the Digital Mars website. There's also Andrei Alexandrescu's book "The D Programming Language".
Web development: This is an admitted weakness. D doesn't (yet) have a good web framework, though its native unicode support and excellent generic programming support should make writing one relatively easy.

I too would like another Java-like technology to come along. Lately I've been doing Flex/Actionscript. While I really enjoy it, Actionscript technology seriously lacks the elegance that Java has. Adobe can write some good cross platform APIs, but they just don't have the head capital to build elegant languages and compilers. I've also tried Ruby, but the VM for Ruby is really bad. I've gone back to Java after my flirtation with other technologies and I think it's because the language is good enough, but the JVM is by far the best out there.
So do you want to stay with the JVM or do you really want to the leave the JVM altogether? Staying on the JVM there are lots of options: JRuby, Scala, Groovy, Javascript, Clojure are the big players. However, there are tons of great languages that can take advantage of the JVM's features.
Leaving the JVM there are still good options like python, ruby, and erlang. But you give up some of the nice features of the JVM like performance (big one), and the ability to drop down to a nice language like Java if you need speed. Those others mean using C or nothing at all.
I finally stopped worrying about Java's future. Sun did all it could to screw it up and it still turned out pretty darn good. I think Opensource has a lot more influence over Java's success than Oracle or Sun could ever have had.

I can't post comments yet, so I'm posting an answer related to the Python discussion. Though Python isn't compiled to machine code, there is a Python-to-C compiler called Cython, which can compile nearly all valid Python -- closures are finally (!) in the latest development release. It's have a big impact on some parts of the Python commmunity, e.g., I was at Euroscipy recently, and over half the talks mentioned Cython.

I personally don't like PHP, but it does meet all of your requirements. It doesn't officially support compilation but there is the Hip Hop project which compiles PHP to C code. Facebook is currently heading up this project.
That said, I highly discourage you from using it :)

C# is the only thing that will meet your needs and not feel hopelessly archaic, or frustrate with limited library. For open source/non-windows, use mono. It's a good, mature implementation of most of what's important in the CLR.
Some things (WPF, WCF, etc) are "missing" from mono, but these aren't so much part of the platform as they are windows-specific proprietary toolkits. Some of them are being implemented slowly in mono, some aren't. Coming from java you won't miss them because you're looking for a platform and good standard libraries to build upon, not a gui toolkit or whiz-bang communication framework.
As far as a platform to build stuff with that's "like" java and offers similar levels of functionality, C# + CLR is the clearest option.

Using also Cython you get the best of the two worlds , the ability to code in python , the ability to code in C and C++ and of course compile your code and the ability to use both python a c/c++ libraries out of the box. And if you dont like C++ syntax , cython syntax is python syntax and more.
link text

How safe would it be to use functional-java to add closures to a Java production project?

I would love to use closures in Java. I have read that they may or may not make it into Java 7. But an open-source project called functional-java has implemented functional features including closures.
How safe would it be to use such a library in an enterprise production app?
Is there a better way to add closures to Java currently?

Closures will definitely not make it into Java 7, due to a lack of consensus around a single implementation. See here.
The Functional Java library is thoroughly tested and actively developed. If you browse the source you will see that it's just simple bog-standard Java, and there's not a lot that could go wrong. I know at least one mission-critical enterprise app that uses it in production. This application has more than a million users.
Go for it.

Functional Java does not requires that you use closures -- the source compiles with any Java 1.5 compiler. Functional Java is far more complete than Google collections and just happens to allow you to use it well with BGGA.
Hope this helps.

I think you are looking at this issue from the wrong perspective.
I would love to use closures in Java. ... How safe would it be to use such a library in an enterprise production app?
Your decision making on what technology to use in an enterprise production app should be based on what is going to be best for the enterprise in the long term. You should be asking questions like:
Is adding the technology going to significantly improve the app from the standpoint of functionality?
What are the risks to the current project?
Does the technology claim to be "production ready"?
Is the technology supported. Is it likely to still be supported in 5 to 10 years time?
If you leave the company, is someone else going to take over maintaining your code?
What are the training / recruiting implications of using the technology?
In general, statements like "I would love to use Xxx" should have no place in enterprise decision making. (There are counter-examples; e.g. startups that bet the company on some new technology, but the real success rate is ... umm ... open to speculation.)
To sum it all up, if you are building enterprise production apps, you need to be conservative, and resist the temptation to use shiny new technology "for fun". Try to think like an IT manager.

If you are looking for a lighter solution to have closures in plain Java check out the lambdaj project:
http://code.google.com/p/lambdaj/

BGGA is still a prototype implementation, not sure how comfortable you are putting prototype code into production. check out google collections it gets you closer to the goal albeit more verbosely

Too old question to answer. Here we have lambdas in java8 to make your life easier.