I am developing a framework where jars can be dropped into a folder and scanned for a function that can be called later. My first implementation used naive ClassLoader method where the jars were loaded and the class instance created. This is plugin architecture.
The problem I ran into is the versioning. Let's say for example my host app is using third-party lib that depends on org.joda time version 1.6 and the plugin is dependent on version 2.1 of the same (newer ) library.
I tried to use the Java Simple Plugin Framework but it does not seem to load my plug-ins using custom class loaders (which is what i assume i will need to overcome the version conflict and have the 2.1 version actually loaded).
My next step is to try osgi.
So, the question is: is this the right approach or is there a simple way that i don't know, I am coming from .net world and don't know java too well but i remember dll hell, and this seems to be the java version of it. I am developing in Scala btw, but that should not matter to the main question.
I had a similar use-case. I wanted a simple plugin framework much like you have described, but was not ready to jump into OSGi. I went the custom classloader route, but ran into much the same problems as you have. I did try a Parent-Last Classloader, which did help with some of the jar conflicts. That might be something to look into. I looked fairly seriously into what the CI Server Jenkins had done for their plugin system - and found this article interesting.
In the end, I needed to be able to track when services are coming and going, have a service registry, etc... and realized that I was re-inventing OSGi. I switched over to pure OSGi, and even though there is a learning curve and it can be a pain sometimes, I'm glad I did.
Try ScalaScriptEngine. It allows you to dynamically load and compile classes from source files and supports quite a few advanced features.
Related
How would you implement a Plugin-system for your Java application?
Is it possible to have an easy to use (for the developer) system which achieves the following:
Users put their plugins into a subdirectory of the app
The Plugin can provide a configuration screen
If you use a framework, is the license compatible with commercial developement?
First you need an interface that all plugins need to implement, e.g.
public interface Plugin {
public void load(PluginConfiguration pluginConfiguration);
public void run();
public void unload();
public JComponent getConfigurationPage();
}
Plugin authors should then bundle their plugins into JAR files. Your applications opens the JAR file and could then use an attribute from JAR manifest or the list of all files in the JAR file to find the class that implements your Plugin interface. Instantiate that class, the plugin is ready to go.
Of course you may also want to implement some kind of sandboxing so that the plugin is restricted in what it can and can not do. I have created a small test application (and blogged about it) that consists of two plugins, one of which is denied access to local resources.
Use OSGi.
It is the foundation of the Eclipse plug-in system. Equinox is Eclipse's implementation (licensed EPL) and Felix is the Apache Project's implementation (licensed Apache Public License).
Eclipse provides a concrete example that OSGi can cover the points you mentioned (or you could just build your application on top of Eclipse RCP if you want a full Eclipse/SWT/JFace stack).
Since 1.6, there's been java.util.ServiceLoader which can be used if you want to code your own simple system.
But if you want anything more than basic features, use one of the existing frameworks.
Use PF4J.
It has support for Web, Spring and Wicket.
Easy to use and build the applications
There is also JPF (Java Plugin Framework).
I worked on OSGi for a week--an intense, nothing but OSGi week. At the end it was like a bad dream but I learned a lot.
I was able to get OSGi working (not easy, all examples are out of date, everything on the net is at least three years old if not five), but I had serious trouble getting it integrated into an existing project because of issues with the jar manifests.
In short, there are only a few obscure tools used for building manifests and they are not well documented (BND Tools is hardly obscure, but it is designed for a certain process in Eclipse). Also, most of the OSGi information available is not targeted towards application developers who have an existing desktop application.
This makes a lot of the context for the information foggy or inappropriate. Neil Bartlett's blog posts were the biggest help, but even those failed to get a working system (I grabbed some code from the Felix tutorial and pieced it together to get the embedded framework rolling). I found his book draft that he posted for free years ago, which is excellent, but the examples in Eclipse do not work because of changes in Eclipse OSGi support.
I think that recommending OSGi for solving the above stated problem is extremely poor advice. OSGi is "the right choice" but for a scenario as the one above, I think either JPF or some homegrown minimalistic framework is sufficient.
Years ago I started a project like that and I hope soon will be ready.I got inspired by projects like NetBeans and Eclipse but meanwhile it changed to something a little bit different. OSGi looks like a good choice now, but I didn't had a chance to compare it with my project.It is similar with JPF mentioned above, but in the same time different in many ways.
The basic idea which motivated me is to be as easy as possible to build Java application, with no separation between web applications, desktop applications or applet/JWS applications(of course this doesn't cover the UI - yet) as a core functionality.
I built the project with a few goals in my mind :
it doesn't matter if you build a web application or a desktop application you should start the application in the same way, a plain main method, No fancy web.xml declaration(not that I'm against having a standard web descriptor, but it doesn't go well with a plug-in system, where you add "servlets" - I call them RequestHandler(s) - dynamic at your will).
easy to plug in "extensions" around an "extension point" - something from Eclipse but a different approach.
self-deployable, since all the plugins are registered(XML files) the application must be self-deployable independent of the build system - of course there is an Ant task and a Maven MOJO which are the links with the ourside world, but in the end it calls the application and instruct it to self-deploy itself at a specific location.
borrowed from Maven, it can download code from repositories(including Maven 1 & 2 repositories) so your application can be deployed as a single small jar as long as you have access to the repositories(useful sometime, and basically this provides support for auto-updates - don't you love the idea to be notified by your web application that there is a newer version, it was downloaded and it just needs your permission to install it? I know I love that).
basic application monitoring about system health, email notifications in case of failures
How would you implement a Plugin-system for your Java application?
Is it possible to have an easy to use (for the developer) system which achieves the following:
Users put their plugins into a subdirectory of the app
The Plugin can provide a configuration screen
If you use a framework, is the license compatible with commercial developement?
First you need an interface that all plugins need to implement, e.g.
public interface Plugin {
public void load(PluginConfiguration pluginConfiguration);
public void run();
public void unload();
public JComponent getConfigurationPage();
}
Plugin authors should then bundle their plugins into JAR files. Your applications opens the JAR file and could then use an attribute from JAR manifest or the list of all files in the JAR file to find the class that implements your Plugin interface. Instantiate that class, the plugin is ready to go.
Of course you may also want to implement some kind of sandboxing so that the plugin is restricted in what it can and can not do. I have created a small test application (and blogged about it) that consists of two plugins, one of which is denied access to local resources.
Use OSGi.
It is the foundation of the Eclipse plug-in system. Equinox is Eclipse's implementation (licensed EPL) and Felix is the Apache Project's implementation (licensed Apache Public License).
Eclipse provides a concrete example that OSGi can cover the points you mentioned (or you could just build your application on top of Eclipse RCP if you want a full Eclipse/SWT/JFace stack).
Since 1.6, there's been java.util.ServiceLoader which can be used if you want to code your own simple system.
But if you want anything more than basic features, use one of the existing frameworks.
Use PF4J.
It has support for Web, Spring and Wicket.
Easy to use and build the applications
There is also JPF (Java Plugin Framework).
I worked on OSGi for a week--an intense, nothing but OSGi week. At the end it was like a bad dream but I learned a lot.
I was able to get OSGi working (not easy, all examples are out of date, everything on the net is at least three years old if not five), but I had serious trouble getting it integrated into an existing project because of issues with the jar manifests.
In short, there are only a few obscure tools used for building manifests and they are not well documented (BND Tools is hardly obscure, but it is designed for a certain process in Eclipse). Also, most of the OSGi information available is not targeted towards application developers who have an existing desktop application.
This makes a lot of the context for the information foggy or inappropriate. Neil Bartlett's blog posts were the biggest help, but even those failed to get a working system (I grabbed some code from the Felix tutorial and pieced it together to get the embedded framework rolling). I found his book draft that he posted for free years ago, which is excellent, but the examples in Eclipse do not work because of changes in Eclipse OSGi support.
I think that recommending OSGi for solving the above stated problem is extremely poor advice. OSGi is "the right choice" but for a scenario as the one above, I think either JPF or some homegrown minimalistic framework is sufficient.
Years ago I started a project like that and I hope soon will be ready.I got inspired by projects like NetBeans and Eclipse but meanwhile it changed to something a little bit different. OSGi looks like a good choice now, but I didn't had a chance to compare it with my project.It is similar with JPF mentioned above, but in the same time different in many ways.
The basic idea which motivated me is to be as easy as possible to build Java application, with no separation between web applications, desktop applications or applet/JWS applications(of course this doesn't cover the UI - yet) as a core functionality.
I built the project with a few goals in my mind :
it doesn't matter if you build a web application or a desktop application you should start the application in the same way, a plain main method, No fancy web.xml declaration(not that I'm against having a standard web descriptor, but it doesn't go well with a plug-in system, where you add "servlets" - I call them RequestHandler(s) - dynamic at your will).
easy to plug in "extensions" around an "extension point" - something from Eclipse but a different approach.
self-deployable, since all the plugins are registered(XML files) the application must be self-deployable independent of the build system - of course there is an Ant task and a Maven MOJO which are the links with the ourside world, but in the end it calls the application and instruct it to self-deploy itself at a specific location.
borrowed from Maven, it can download code from repositories(including Maven 1 & 2 repositories) so your application can be deployed as a single small jar as long as you have access to the repositories(useful sometime, and basically this provides support for auto-updates - don't you love the idea to be notified by your web application that there is a newer version, it was downloaded and it just needs your permission to install it? I know I love that).
basic application monitoring about system health, email notifications in case of failures
Please give an advise on how to do "plugin" architecture for Java web application.
Currently we are using quite simple and standard Spring+Hibernate+Struts 2 in Tomcat servlet container. (Built with maven)
I need something like Redmine. Where any module can be enabled/disabled, updated
Please exclude heavy options like OSGi, Portlet.
OSGi is too heavy, there is no good adoption of the technology for web. I already looked at Eclipse Germini;
Portlet it just old, and never was popular.
I will try to provide several possible solution. I did spent some time preparing small PoCs for the project I'm working on, so let's hope the options below are relevant.
Important note: it is really easy to define some extension point, do resolve and find available implementations. There are a lot of solutions available, for example good and simple one -- JSPF
Resources are the main problem for WEB applications
OSGi
OSGi, is not that bad and can be useful. It seems to be heavy (and some implementations are heavy) but this is price of standardized platform. I would suggest to check Apache Felix. It can be used in a "lightweight" mode. By the way, it includes Web Console which is build as loosely coupled plugin-based application, could be helpful:
Some examples Extending the Apache Felix Web Console
The Web Console can be extended by registering an OSGi service for the
interface javax.servlet.Servlet with the service property
felix.webconsole.label set to the label (last segment in the URL) of
the page. The respective service is called a Web Console Plugin or a
plugin for short.
You can also check eie-manager which is clean and simple and uses OSGi to manage plugins. Could be a good example for you.
Custom plugin framework
I would suggest to review solution behind Jenkins/Hudson. I would say Jenkins plug-in system is quite mature and reliable. Can be used as a good example.
Please also check Hudson Plugin Architecture
Simple solution
For my project I've build plugin abstraction layer based on JSPF with custom dependency resolver.
PROS:
simple and small
clean concept
works good
CONS:
without proper plugin management can be slow (full classpath search)
provides very basic functionality
may require additional attention
I would suggest to use JSPF only if you really need some simplicity and want to control everything. JPF provides a lot of interesting features out of the box, for example:
Plug-ins can be "hot-registered" and even de-registered during
application execution. What's more, registered plug-ins can be
activated and deactivated "on the fly", minimizing runtime resource
usage.
The problem is JPF is dead.
Suggestion
Do spend some time with Apache Felix. It is mature enough, so your time investments may pay back a lot.
Check out the answers to this question: Best way to build a Plugin system with Java
If you don't trust the plugin code, you can implement sandboxing, as described here: Sandbox against malicious code in a Java application
The open-source Java Plug-in Framework project supports plugin deactivation, you can get inspired from it even if it is too heavy for your purposes.
Atlassian open sourced their plugin system here. I see it is being worked heavily by Atlassian team. Worth to explore its documentation
I have many questions about scala. I have done a bit of reading and googling and SO'ing and not found any solid answers readily available. I'm not at the experimentation/prototyping stage yet, so I thought I might as well just ask my questions and get some expert knowledge for everyone to share. Thanks in advance!
What is scala.exe really for? Can someone give a rundown about what scala.exe does differently than java.exe? Is there runtime black magic in scala.exe other than providing for an interpreter shell?
(UPDATE: There is no such thing as a scala.exe. Scala ships with a simple batch script launcher, scala.bat (or scala on *nix). The Scala runtime is java.exe with Scala's standard library jars in the CLASSPATH.)
Can I link in scala code to an existing java program if I launched the process with java.exe? If so, does my CLASSPATH need to change to link in the scala standard library jars? Also if I am launching with java.exe, are any new -javaagent parameters required to link in scala code? Is there a way to include scala code in jars in my existing web application .war file (or in WEB-INF/classes) and have it run?
Conversely, if scala.exe is required in my Java EE app server launcher to execute scala+java code, can scala.exe take all my esoteric -XX:InsertYourCrazySunPerfSwitchHere JVM command line parameters?
Finally, is incorporating new Scala code into an existing Spring Framework + JSF2 web application like trying to fit a round peg into a square hole? I see long lists of web frameworks designed for Scala that are available, but I wondered how smooth or how awkward Scala would be playing with commodity tools like JSF2. Are these Scala-based web frameworks the byproduct of non-Java developers migrating to the Scala community and wanting to recreate their framework du jour in the Scala language? Or, is there something intrinsic about the way JSF2 is designed such that, once I became an expert at Scala, I would see it's a sloppy mess trying to mix the two and I had wasted my time?
You have asked many questions. I will try to address them all, one at a time.
Question:
What is scala.exe really for? Can someone give a rundown about what
scala.exe does differently than java.exe? Is there runtime black magic
in scala.exe other than providing for an interpreter shell?
scala.bat can do several things depending on the arguments:
it can launch an interpreter shell
it can execute a Scala script file
it can execute a compiled Scala binary from a classpath
it can execute a jar file
When launching binaries, scala.bat will simply call java.exe with scala-library.jar added to the classpath. There is no magic of any kind.
Question:
Can I link in scala code to an existing java program if I launched the
process with java.exe? If so, does my CLASSPATH need to change to link
in the scala standard library jars? Also if I am launching with
java.exe, are any new -javaagent parameters required to link in scala
code?
All the Scala binaries are simple jars. The only difference is that they require the scala runtime library (scala-library.jar). So when launching with java.exe you simply follow the steps you would when launching a jar with dependencies. Again, there is no magic and no extra switches.
Question:
Is there a way to include scala code in jars in my existing web
application .war file (or in WEB-INF/classes) and have it run?
Scala jars are just like java jars, so you can treat them as such.
Question:
Finally, is incorporating new Scala code into an existing Spring
Framework + JSF2 web application like trying to fit a round peg into a
square hole? I see long lists of web frameworks designed for Scala
that are available, but I wondered how smooth or how awkward Scala
would be playing with commodity tools like JSF2. Are these Scala-based
web frameworks the byproduct of non-Java developers migrating to the
Scala community and wanting to recreate their framework du jour in the
Scala language? Or, is there something intrinsic about the way JSF2 is
designed such that, once I became an expert at Scala, I would see it's
a sloppy mess trying to mix the two and I had wasted my time?
Using Java libraries from Scala is at least as easy as using them from Java and often much smoother. On the other hand Scala is much richer that Java. So you will feel that Java libraries are lacking as soon as you start getting used to Scala.
I am making my first foray into scala for a production app.
The app is currently packaged as a war file.
My plan is to create a jar file of the scala compiled artifacts and add that into the lib folder for the war file.
My enhancement is a mysql-backed app exposed via Jersey & will be integrated with a 3rd party site via HttpClient invocations. I know how to do this via plain java. But when doing it in scala, there are several decision points that I am pussyfooting on.
scala 2.7.7 or 2.8 RC ?
JDBC via querulous Is this API ready for production ?
sbt vs maven. I am comfortable with maven.
Is there a scala idiomatic wrapper for HttpClient (or should I use it just like in java) ?
I'd love to hear your comments and experiences on starting out with scala.
I would use 2.8.0. There are just too many useful features in 2.8. Besides, 2.8 is closing in on a final release. If you're just starting out, why not start out with that? FWIW, I've been using 2.8.0 since Beta1, in various tools and libraries that I use daily. While there have been bugs, they haven't been enough to make me fall back to 2.7.7. YMMV, though.
This isn't going to make your decision any easier, but there are other possibilities for database access. I've been using SQueryL, for instance; I like it. ORBroker is another option.
If you're comfortable with Maven, then use it, by all means. Personally, I prefer SBT. I get the full power of a real programming language, when I need to implement special build logic. Just as useful, I don't have to deal with XML configuration files. (XML is good for data, but it's a crappy format for a human-edited configuration file.)
You might try Databinder Dispatch. See this article for a nice overview.
If you're only going to start development, Scala 2.8 GA will probably be available by the time you go production. Even if it's not, I would choose the freshest 2.8RC pack rather than sticking to 2.7.7. 2.8 not only has a number of great features, but also contains lots of 2.7.7 bugfixes.
There're not too many production-ready ORMs designed for Scala these days. I would probably choose Lift Persistence, because of the team of professionals and friendly community behind Lift Framework. But if you don't want to risk, you should consider using old good proven Java ORMs: Hibernate, JPA, iBatis (that was recently renamed to myBatis), etc.
You should give SBT a try! It's compatible with maven POMs, so migration to SBT shouldn't be too painful for you. Benefits from using SBT:
It's designed for Scala, so you will be relieved from the burden of maintaining countless number of plugins for Maven to make it work with Scala consistently
You will be able to write build scripts in Scala (it's an amazing experience compared with XMLs)
SBT has a killer feature - continuous whatever (building, testing, deploying). SBT monitors your code, detects when its changed, and triggers an action (test, re-deployment, etc.).