I just got a requirement to create a small (I assume standalone) utility to hit some code in our web application to do some custom processing of files from the app and then dump the files into a shared drive. My question is what is the best way for doing this? Do I just create a small app and then jar it up and run it off a command line or is there a better way?
Sorry, I didn't give enough detail. It's an old application, like over 10 years, so while it's been upgraded to jdk 1.6, most of the code uses the old collections, old loops, etc... There aren't any interfaces, very tightly coupled code that uses inheritance with lots of nested objects. The web app will do the processing. I think what they want is create some code outside of the application code that will login and then fire off the file processing code. Prior to this I had upgraded their version of Windward Reports in a separate branch and they want to make sure that the processed files: contracts, forms, etc.. don't get altered greatly as there are legal requirements on fonts and layouts. So this utility will go in, fire off the list of reports (a few thousand) dump it to a share drive so they can view them with another tool for comparision based on rules you can automate with that commercial tool, en masse. I was thinking create a small class with a main method, then jar it up and while the web server is running with my upgraded branch code, run the utility off the command line to fire it off.
There's not enough to go on here. How is the web app's functions exposed? If it's a REST interface then wget/curl/spring-rest-template are the way to go. If it's something like a JFS app then you're going to need something like Selenium to imitate a browser. If the functionality is in a shared library (JAR) then there web never even comes into play.
Well, I was originally looking at creating a standalone utility jar that I would run off the command line to connect with URLConnection to the app, but I found there is already testing code built into the application that I can run from a command line as long as I deploy the new code with the existing code. The utility will dump out the files to a shared drive and then XTest can be run to compare files. After reviewing the capabilities of XTest, it appears that it can handle the comparison of files well.
Related
For the past few days I'm stuck at trying to pass a PMML file with .xml extension that I created using R and pmml package. After creating a iris.xml PMML file I have no where to go, actually I don't know what to look for. And then I saw Alex Guazzelli's video here, where he tells us to use ADAPA from Zementis to run the PMML files that are created from R.
What I took away from the video is,
I want to do something similar as shown, like uploading my PMML file at some place and input my test files frequently to get the corresponding predicted values.
ADAPA looks great, but looks like it also costs some money. So I would like to look for an alternative, which might not be as user friendly as ADAPA.
Then I found this page where the author says,
So the first step was to get the engine running. I experienced some small problems but Scott fixed them very fast and offered a new war file ready for deployment inside tomcat. It can be obtained under http://code.google.com/p/openscoring/. If you still encounter some problems you could also try my war file. It was built with java version “1.6.0_26” and tested under tomcat 7. After installing the war file using the management console, you could test it by sending a POST request in XML like the following to ‘http://localhost:8080/OpenScoring/Scoring’
Then I found jpmml. The problem here is the package names are very confusing, I don't know which one to install. In fact I don't know how to install each of them. I have basic knowledge of Java, apparently it isn't helpful in the current scenario.
Desperately looking forward for any kind of help.
How do the data flows in your application look like? Zementis' ADAPA is a standalone web service, which lets you do scoring in a request/response fashion. This works for web applications, but is clearly sub-optimal if you're looking to score larger amounts of data.
The JPMML project provides "PMML runtimes" for most popular big data frameworks. For example, if your application is tied to Apache Spark, then you would go for the JPMML-Spark project, if tied to Apache Hive/Pig, then for the JPMML-Pig project, etc. For a simple REST web service you should check out the Openscoring project, which is functionally identical to Zementis' ADAPA.
All JPMML projects can be built from their source checkout using Apache Maven by issuing the following command:
mvn clean install
Some JPMML projects also have their binaries distributed via the Maven Central repository. If you're looking for the Openscoring WAR file, then you need to search for "openscoring-webapp" and on the results page click the "download WAR" link (at the time of writing this, you would get openscoring-webapp-1.2.12.war). This is a standard WAR file that can be dropped to the auto deploy folder of your favourite Java web server (Tomcat?) as it is. The REST API of the web service is fully documented in the README file of the Openscoring project.
So I have been tasked with integrating a program called "lightSIDE" into a hadoop job, and I'm having some trouble figuring out how to go about this.
So essentially, rather than a single JAR, lightSIDE comes as an entire directory, including xml files that are crucial to its running.
Up until now, the way the data scientists on my team have been using this program is by running a python script that actually runs an executable, but this seems extremely inefficient as it would be spinning up a new JVM every time it gets called. That being said, I have no idea how else to handle this.
If you are writing your own MapReduce jobs then it is possible to include all the jar files as as libraries and xml files as resources.
I'm one of the maintainers for the LightSide Researcher's Workbench. LightSide also includes a tiny PredictionServer class to handle predictions on new instances over HTTP - you can see it here on BitBucket.
If you want to train new models instead, you could modify this server to do what you want, drawing clues from the side.recipe.Chef class.
Since my attempt to set up a Dart project myself I think I miss something fundamental since I didn't succeeded. So I still need the help of the community.
Coming from GWT I am used to a single application forming a single JS file which is ran and will augment a HTML element once it is recognized by the application.
There will be usually two JS files, one for the user-frontend and the web applications backend application.
I want a solution with an incremental build during development time (which I guess Dart offers when used in Dartium)
I have an inhouse web framework that I want to be started and used to send the Dart files for the Dartium session. How this will integrate and interfere with the debug sessions?
Update regarding web framework:
The web framework is a component based rendering engine, including database and uses its own resource management including everything http related like setting the cache flags etc. Its about 1.5 MB with 1200+ tests. Its simply everything you need starting with a simple servlet. Its also using an embedded jetty.
The relevance here is that I need to know how the debugger connects to Dartium and how it finds the files once an instance is running and delivered a html file containing dartium sources, so how can I start my own web server at a given port and still have dartium debug capabilities?
Update regarding the former answers:
I tried it but after two days gave up to learn more and do some other stuff. I just don't know why it is just not possible to add a simple file to the root package of my Dart module like the good old package.html (javadoc) fil. I then just add the Dart libaries to my project and the Dart plugin adds the required Dart nature to the project and creates a builder entry, done. Why do I have to do all the fuzz. Or even better why cant I just annotate my Module's main class to form a module and so I can replace the extra file completely?
I guess the Dart plugin has a model of the Dart code already so discovery is done on the fly in Eclipse.
I also do not know why I cant put my dart code in a dart source folder like src/dart/main and src/dart/test.
Or is this possible? I am still trying to get this done. I will use a fresh Eclipse 3.8 install and check if I can get Dartium to work. Just installing the plugin seams not to do the trick.
Update regarding the JS generation:
I cannot understand why Dart is not offering an incremental build of JS files. Even if it is a single file. It should not be that hard to debundle the given compile steps. I guess it will be something like compile each source file independently and link those together, do some tree shaking and done. Would be awesome if this can be made possible. Remember one can hold a model of the output file in memory (or on disk) and know what part of the js relates to what source file. Then just look up the link symbol tables and write back the part that has changed.
For me the killer feature for Dart would be the ease of configuration as I outlined and the incremental build of JS files making co-developing in JS a no-brainer. I guess in the end both JS files will be just about 750kb combined. So all the stuff with additional compression would not force me to upgrade my 8GB memory or will stress my SSD at all (350MB/sec for writes in burst mode).
Is there any work planed on this? Would be great to have Dart as the final solution for JS creation but to be honest I do not understand why GWT is the way to create JS this way. An incremental build and easy setup for GWT would be also welcome.
Seems not to be a question ...
In Dart you have usually one JS file because Dart on the server runs native (without transpiling)
With Dartium you don't have a build at all because it also runs Dart natively.
You build to JavaScript only for deployment (and of course to test the build output before deployment).
The debugging is done by Dartium itself (you can use the Chrome DevTools debugger without DartEditor if you want). DartEditor access the debugger API of Dartium and acts as a remote display/control.
Debugging web clients loaded from other webservers is supported.
What might cause some work is setting up your custom web server so that it forwards requests to source files to pub serve the web server used by DartEditor (or standalone).
pub serve runs transformers (on the fly code transformations/generation). Some framework depend on transformers being run on the code to make it functional.
I have no idea what this means but I don't use Eclipse/Dart plugin.
[Update regarding the former answers] I tried it but after two
days gave up to learn more and do some other stuff. I just dont
know why it is just not possible to add a simple file to the
root package of my module like the good old package.html file
for the java docs and then all i do is add the Dart libaries
to my project and the Dart plugin adds the nature to it and
creates a builder entry, done. Why do I have to do all the fuzz.
Or even better why cant I just annotate my Module's main class
to form a module and so I can replace the extra files?
To integrate Dart with your Java project create the Dart project independent from your project and move the Dart build output to a directory where you have your other static files.
While development configure your web server to forward to pub serve as explained above.
As already stated in my first answer, this
[Update regarding the JS generation] I can not understand why
dartium is not offering an incremental build of JS files. Even
if it is a single file. It should not be that hard to debundle
the given compile steps. I guess it will be something like
compile a single file and link those then the magical tree
shake and done
is irrelevant. You don't do anything with JavaScript while developing.
If you load the page with a non-Dartium browser pub serve will serve
built JavaScript instead of Dart. Incremental build is in the works
to improve responsiveness. But incremental build is not available
for file generation (would make sense anyway IMHO).
When I add simple dart files anywhere in my existing Eclipse project, what do I have to do and what will be the limitiations and work arounds in order to make it work.
Update:
Some additional information might be helpful. What I use is a custom web server serving dart and js as simple files (dynamically on request). So I can not simply follow the dart way of things. The dart files would be rather sources only being used during development and converted to js files before the actual check in of the new version.
Therefore I would like to add the dart files in exactly the same mannor I add my other source. So i would love to have something like src/main/dart and src/test/dart or something.
I would also like being able to run dart in the stand alone vm (stand alone application) instead of the web / browser to simplify some development steps like the development of the model / business layer and add simplified testing. Also we want to evaluate if there is the possibility to start dart side processes on the server for additional simplification.
So the scenario is:
Put dart files anywhere (especially in the source tree)
The web framework is non-standard programmer friendly, we can simply add resources from anywhere under virtual directories (src/main/dart -> localhost/dart/)
The dart files will be converted to js prior to checkin
Dart should be able to pickup the right files on debug
Testing and debugging should be possible in standalone mode and in using Dartium
So how can I do this what should I be able to configure.
Update2:
Dart will be used as a JavaScript replacement here. So in the end this is a mid-size web project where the JS/Dart code provides the glue to make the pages functional. It also will drive the backend application which might be single paged but its not decided yet. Best would be to see the dart application as of two applications at once where the backedn application is distinct from the front end functionality that is dynamically added as needed.
For the compilation process to JS I would not mind to create a simple program copying files, compile and recopy files and done. Wouldnt take that long since it would be only needed once or twice a day.
You need to create a Dart pub package.
That is a directory containing the file pubspec.yaml.
You can put the Dart script and HTML files in the web and or lib subdirectory of the Dart package.
When you run pub build the deployable output is generated in the build/web subdirectory.
This deployable files can be served by your web server like any other static web content like HTML, CSS, JavaScript files.
update
I can't think of a way doing this
Dart isn't like JavaScript where you add some event handler script here and there.
I can't see how this would make sense anyway. More information about what you want Dart do for you is necessary to answer this - see my comment above.
You need the build step prior to serve generated JavaScript.
A Dart app is built entirely at once not file by file.
this is what pub build does. The output is written to my_dart_package/build/web
You can debug Dart as Dart (in Dartium, DartEditor or any other IDE that supports debugging Dart code) or after the build step using source maps in a browser like Chrome, FireFox, ...
You can debug non-web code standalone using the debugger of DartEditor, WebStorm, ...
You can create unit tests that invoke methods on Dart classes (model classes, PODO, ...) without a browser.
When you want to debug code that depends on the dart:html package you can only do it using a browser.
update2
You could create three packages
- server
- client
- shared
Dynamically loading functionality is already supported. The functionality has to be available at build time but can be loaded 'deferred' at first access or on request but this has nothing to do with the server part of Dart this is client-only functionality. Any web server can be used for that (for example Apache).
The server usually provided some data APIs (HTTP, REST, WebSocket, ...) for the client.
I'm a fan of a fairly big open-source program which is written in Java and uses Swing as a front-end client on series of web actions. As I've been working with Android development for a few months, I had an idea that I could create a new app that works similarly to the Swing application. The program is well-organized and separates interface from implementation, also offering a command line interface as well as Swing.
So ideally I'd like to be able to just inject a new Android package into the existing file system somewhere, make use of the back end that already exists, and have it work seamlessly with new updates to the program. The closest suggestion I found was this:
Q: How can I create a new project from an existing project, using Android command line tools?
A: Copy the directory tree of the old project into a new project.
This doesn't seem like it can be feasible advice for me, since I have to work with the existing SVN repository rather than starting the whole thing from scratch. Is there any way to pull this off?
since I have to work with the existing SVN repository rather than starting the whole thing from scratch. Is there any way to pull this off?
I doubt it. Android won't like the Swing code; standard Java won't know what to do with the Android code.
I suggest that you reorganize your code base into three:
One generates a JAR file, containing the common logic
One is the rest of your existing Swing/command-line logic, which uses the JAR
One is a new Android project, which also uses the JAR
Maybe you can pull this off by having src-jar/, src-swing/ and src/ (latter for Android) and associated build scripts. Personally, I'd have three totally separate projects.
Since SVN supports move operations (at least, it used to, last I used it, oh so many years ago), you should be able to accomplish this reorganization without losing any version history.