I have an ear application (myApp) that runs on a Websphere Application Server (WAS). I have a jar (myJar) that is loaded into the classpath of myApp when the WAS server is started. myJar has a class (MyInitClass) that reads from a db and loads a set of data (myData) into memory. This data gets read many times by myApp. The point is to get myData into memory to prevent doing a db call every time this data is used. This part works great!
The solution I am trying to provide is a manual initialization of MyInitClass. myData gets changed from time to time and I would like to be able to reinitialize MyInitClass from a command line so I don't have to restart the application. Is this possible?
myApp calls a class (MyClass) that has something like this:
public static MyInitClass initClass;
public boolean doStuff()
{
if (initClass == null)
{
// this method loads the data into initClass.myData array
initClass.dataInitializer();
}
else
// no need to reload initClass.myData
}
I have created code similar to this in another class (MyManualInit):
public static void main(String[] args)
{
MyClass.initClass = new MyInitClass();
MyClass.initClass.dataInitializer();
}
When I run MyManualInit from command line it prints all the same debug info that gets printed during the initialization from myApp. But myApp does not recognize that MyInitClass has been reinitialized. I have printed System.out.println(System.getProperty("java.home")) from both processes to validate that I am using the same JRE to run both. Am I doing something obviously wrong here or does it just not work like that? I assumed if I ran MyManualInit on the same JRE it would work. MyClass, MyInitClass and MyManualInit are all in myJar.
Please let me know if you need more info.
You are mixing things. Websphere runs in an instance of the JVM and your command line program instantiates a new one, and objects do not communicate between different JVMs (at least without some effort, bringing up sockets, etc.)
Actually your code does a lazy initialization of your initClass object, and it should be enough without any command line interaction. Why is it not enough for you?
Related
I have written an Azure Databricks scala notebook (based on a JAR library), and I run it using a Databricks job once every hour.
In the code, I use the Application Insights Java SDK for log tracing, and init a GUID that marks the "RunId". I do this in a Scala 'object' constructor:
object AppInsightsTracer
{
TelemetryConfiguration.getActive().setInstrumentationKey("...");
val tracer = new TelemetryClient();
val properties = new java.util.HashMap[String, String]()
properties.put("RunId", java.util.UUID.randomUUID.toString);
def trackEvent(name: String)
{
tracer.trackEvent(name, properties, null)
}
}
The notebook itself simply calls the code in the JAR:
import com.mypackage._
Flow.go()
I expect to have a different "RunId" every hour. The weird behavior I am seeing is that for all runs, I get exactly the same "RunId" in the logs!
As if the Scala object constructor code is run exactly once, and is re-used between notebook runs...
Do Spark/Databricks notebooks retain context between runs? If so how can this be avoided?
A Jupyter notebook spawns a Spark session (think of it as a process) and keeps it alive until it either dies, or you restart it explicitly. The object is a singleton, so it's initialized once and will be the same for all cell executions of the notebook.
You start with a new context every time you refresh the notebook.
I would recommend saving your RunId to a file to disk, then reading that file on every notebook run and then increment the RunId in the file.
I am trying to use py4j to open up a gateway that I can use to pass objects from java into python. When I try to open a gateway with the py4j function launch_gateway it does not seem to properly connect to my Java class. However, when I launch my java class in the command line and then connect to it in python using JavaGateway everything works as expected. I would like to be able to use the built in method as I am sure that I am not accounting for things that have already been considered in the design of py4j, but I'm just not sure what I'm doing wrong.
Let's say I wanted to create a gateway to the class sandbox.demo.solver.UtilityReporterEntryPoint.class. In the command line I can do this by executing the following:
java -cp /Users/grr/anaconda/share/py4j/py4j0.10.4.jar: sandbox.demo.solver.UtilityReporterEntryPoint py4j.GatewayServer
This launches as expected and I can use the methods in my class from within python after connecting to the gateway. So far so good.
My understanding of the py4j documentation would lead me to believe I should do the following to launch the gateway in python:
port = launch_gateway(classpath='sandbox.demo.solver.UtilityReporterEntryPoint')
params = GatewayParameters(port=port)
gateway= JavaGateway(gateway_parameters=params)
I get no errors when executing these three lines, but when I try to access my java class methods with gateway.entry_point.someMethod() it fails with the following error:
Py4JError: An error occurred while calling t.getReport. Trace:
py4j.Py4JException: Target Object ID does not exist for this gateway :t
at py4j.Gateway.invoke(Gateway.java:277)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:745)
Obviously something is not getting called correctly within launch_gateway or I am feeding it the wrong information.
In the py4j source code for launch_gateway you can see that given the inputs you provide and those constructed by the function, a command is constructed that eventually gets called by subprocess.Popen. So given the input passed to launch_gateway above the command passed into Popen would be:
command = ['java', '-classpath', '/Users/grr/anaconda/share/py4j/py4j0.10.4.jar:sandbox.demo.solver.UtilityReporterEntryPoint', 'py4j.GatewayServer', '0']
Passing this command to Popen returns the listening port as expected. However, connecting to this listening port still does not allow access to my class methods.
Finally, passing the command as a single string to Popen without the final argument ('0'), properly launches a gateway which again operates as expected. Having taken a glance at the Java source code for py4j.GatewayServer.class this makes no sense as the main method seems to indicate that the class should exit with status 1 if the length of arguments is 0.
At this point I'm kind of at a loss. I can hack my way into a workable solution, but as I said I'm sure that ignores important aspects of the gateway behavior and I don't like hacky solutions. I'd love to tag #Barthelemy in this one, but hopefully he reads this. Thanks in advance for any help.
EDIT
For now I have been able to work around this issue with the following steps.
Package entire project including all external dependencies into a single jar file magABM-all.jar, with 'Main-Class' set to UtilityReporterEntryPoint.
Include if...else block regarding presence of --die-on-exit exactly like it is in GatewayServer.java
Use subprocess.Popen to call the command to run the project jar.
UtilityReporterEntryPoint.java
public static void main(String[] args) throws IOException {
GatewayServer server = new GatewayServer(new UtilityReporterEntryPoint());
System.out.println("Gateway Server Started");
server.start();
if (args[0].equals("--die-on-exit")) {
try {
BufferedReader stdin = new BufferedReader(new InputStreamReader(System.in, Charset.forName("UTF-8")));
stdin.readLine();
System.exit(0);
} catch (java.io.IOException e) {
System.exit(1);
}
}
}
app.py
def setup_gateway()
"""Launch a py4j gateway using UtilityReporterEntryPoint."""
process = subprocess.Popen('java -jar magABM-all.jar --die-on-exit', shell=True)
time.sleep(0.5)
gateway = JavaGateway()
return gateway
In this way I can still use gateway.shutdown if necessary and if the python process that starts the py4j gateway dies or is closed the gateway will be closed.
N.B I would by no means consider this a final solution as py4j was written by much smarter individuals with a clear purpose in mind and I am sure that there is a way to manage this exact workflow within the confines of py4j. This is just a stopgap solution.
There are a few issues:
The classpath parameter in launch_gateway should be a directory or a jar file, not a class name. For example, if you want to include additional Java libraries, you would add them to the classpath parameter.
The error you receive when you call gateway.entry_point.someMethod() means that you have no entry point. When you call launch_gateway, the JVM is started with GatewayServer.main, which launches a GatewayServer with no entry point: GatewayServer server = new GatewayServer(null, port). It is not possible currently to use launch_gateway and specify an entry point.
When you start the JVM with java -cp /Users/grr/anaconda/share/py4j/py4j0.10.4.jar: sandbox.demo.solver.UtilityReporterEntryPoint py4j.GatewayServer I believe the JVM uses UtilityReporterEntryPoint as the main class. Although you did not provide the code, I assume that this class has a main method and that it launches a GatewayServer with an instance of UtilityReporterEntryPoint as the entry point. Note that there is a whitespace between the colon and the class name so UtilityReporterEntryPoint is seen as the main class and not as being part of the classpath.
I currently know Java and Ruby, but have never used JRuby. I want to use some RAM- and computation-intensive Java code inside a Rack (sinatra) web application. In particular, this Java code loads about 200MB of data into RAM, and provides methods for doing various calculations that use this in-memory data.
I know it is possible to call Java code from Ruby in JRuby, but in my case there is an additional requirement: This Java code would need to be loaded once, kept in memory, and kept available as a shared resource for the sinatra code (which is being triggered by multiple web requests) to call out to.
Questions
Is a setup like this even possible?
What would I need to do to accomplish it? I am not even sure if this is a JRuby question per se, or something that would need to be configured in the web server. I have experience with Passenger and Unicorn/nginx, but not with Java servers, so if this does involve configuration of a Java server such as Tomcat, any info about that would help.
I am really not sure where to even start looking, or if there is a better way to be approaching this problem, so any and all recommendations or relevant links are appreciated.
Yes, a setup it's possibile ( see below about Deployment ) and to accomplish it I would suggest to use a Singleton
Singletons in Jruby
with reference to question: best/most elegant way to share objects between a stack of rack mounted apps/middlewares? I agree with Colin Surprenant's answer, namely singleton-as-module pattern which I prefer over using the singleton mixin
Example
I post here some test code you can use as a proof of concept:
JRuby sinatra side:
#file: sample_app.rb
require 'sinatra/base'
require 'java' #https://github.com/jruby/jruby/wiki/CallingJavaFromJRuby
java_import org.rondadev.samples.StatefulCalculator #import you java class here
# singleton-as-module loaded once, kept in memory
module App
module Global extend self
def calc
#calc ||= StatefulCalculator.new
end
end
end
# you could call a method to load data in the statefull java object
App::Global.calc.turn_on
class Sample < Sinatra::Base
get '/' do
"Welcome, calculator register:#{App::Global.calc.display}"
end
get '/add_one' do
"added one to calculator register, new value:#{App::Global.calc.add(1)}"
end
end
You can start it in tomcat with trinidad or simply with rackup config.ru but you need:
#file: config.ru
root = File.dirname(__FILE__) # => "."
require File.join( root, 'sample_app' ) # => true
run Sample # ..in sample_app.rb ..class Sample < Sinatra::Base
something about the Java Side:
package org.rondadev.samples;
public class StatefulCalculator {
private StatelessCalculator calculator;
double register = 0;
public double add(double a) {
register = calculator.add(register, a);
return register;
}
public double display() {
return register;
}
public void clean() {
register = 0;
}
public void turnOff() {
calculator = null;
System.out.println("[StatefulCalculator] Good bye ! ");
}
public void turnOn() {
calculator = new StatelessCalculator();
System.out.println("[StatefulCalculator] Welcome !");
}
}
Please note that the register in here is only a double but in your real code you can have a big data structure in your real scenario
Deployment
You can deploy using Mongrel, Thin (experimental), Webrick (but who would do that?), and even Java-centric application containers like Glassfish, Tomcat, or JBoss. source: jruby deployments
with TorqueBox that is built on the JBoss Application Server.
JBoss AS includes high-performance clustering, caching and messaging functionality.
trinidad is a RubyGem that allows you to run any Rack based applet wrap within an embedded Apache Tomcat container
Thread synchronization
Sinatra will use Mutex#synchronize method to place a lock on every request to avoid race conditions among threads. If your sinatra app is multithreaded and not thread safe, or any gems you use is not thread safe, you would want to do set :lock, true so that only one request is processed at a given time. .. Otherwise by default lock is false, which means the synchronize would yield to the block directly.
source: https://github.com/zhengjia/sinatra-explained/blob/master/app/tutorial_2/tutorial_2.md
Here are some instructions for how to deploy a sinatra app to Tomcat.
The java code can be loaded once and reused if you keep a reference to the java instances you have loaded. You can keep a reference from a global variable in ruby.
One thing to be aware of is that the java library you are using may not be thread safe. If you are running your ruby code in tomact, multiple requests can execute concurrently, and those requests may all access your shared java library. If your library is not thread safe, you will have to use some sort of synchronization to prevent multiple threads accessing it.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
java/shellscript code to find out if a jar file is already running on current machine
I would love to get a cross-platform solution for this, but even if it's unix only- that would be fine.
The simple solution would be to do this from the shell (Pseudocode)(if ps -aux | grep myJar | print {awk 2}.contains myPID, don't run myProgram.
Now unfortunately our linux team doesn't want a script like that running in production since it can (admittedly) have undesired behaviors.
So what I need is to be able to have a file run, and when it runs see if another program is running. If the program is already running and it's below that time limit, it should prevent the program from running.
A bit of an example:
Myprog.jar -- timeout 5 min
Myprog.jar is in a cron that gets called every 4 minutes,
the first time it's called it launches, the second time it's called it's still running, but since it's not over the timeout, it's fine.
If it's still running when the third check comes through (at 8 minutes into execution) it's killed, and its process is replaced by itself afterwards.
If someone can help me understand how to do this (We've been trying to set up a lock file with limited success)
Thanks!
You could make your program open a dummy file for writing with a FileWriter when your program starts, and keep the file open until the program is finished.
When you now start a second instance of your program, it will also try to open this file for writing, which will throw an IOException, because only one process can have a write handle to a file at the same time.
You could use a port as a semaphore. See this question for more info on that. I think a port would be a good cross-platform solution
You can create a temporary file on a fixed location.
private static final File LOCK_FILE = new File("app.lock");
public static boolean checkIfAlreadyRunning()
{
return LOCK_FILE.exists();
}
public static void createLockFile()
{
LOCK_FILE.createNewFile();
Runnable shutDown = new Runnable()
{
public void run()
{
try
{
LOCK_FILE.delete();
} catch (Exception e) { /* Sad but true */ }
}
};
Runtime.getRuntime().addShutdownHook(new Thread(shutDown));
Thread.setUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler()
{
public void uncaughtException(Thread t, Exception e)
{
shutDown.run();
System.exit(-1);
}
});
}
I had exactly the same problem, and it can be pretty tricky to solve. Both File and Socket based approaches can be made to work, but it gets really tricky on some OS's (think of Windows with multiple users in multiple terminal server sessions etc.).
First, determine the scope where you want only one instance. Then decide on a solution.
The ServerSocket method with a fixed port number will allow you one instance per machine (maybe not exactly what you want).
The locking file approach can be tailored to create the locking file in the users temp directoy, so it gives one instance per session/user.
I personally use a combined approach where the locking file specifies a random port and a second instance connects to that port to pass command line parameter to the running instance.
I'm writing an Eclipse plugin with a custom launch configuration, i.e. a launch() method inside a subclass of LaunchConfigurationDelegate. This method essentially just calls Runtime.exec(), but when I write to System.out from within launch() it goes to the console of the Eclipse instance which is debugging the plugin, rather than to the console of the plugin instance itself. I've analysed the ILaunchConfiguration and ILaunch arguments to the method but cannot find anywhere that they specify any output/error streams I can write to.
As is recommended in the tutorials, I have 2 separate plugins running together; one which handles the UI stuff (LaunchConfigurationTab,LaunchConfigurationTabGroup,LaunchShortcut,) and the other which contains the LaunchConfigurationDelegate itself.
I created a console in my UI plugin using this code, and I can write to it fine from within the UI code. But I cannot figure out how to direct output generated in my non-UI plugin to the console created in my UI plugin.
I've read this post and this one, but they do not specify how to "get ahold" of the output which is generated within the launch() method in the first place.
Any pointers would be really welcome, I am stuck!
Well I finally managed to get something working as follows:
In my LaunchConfigurationDelegate I introduced the following static method:
public static void setConsole(PrintStream ps) {
System.setOut(ps);
System.setErr(ps);
}
Then when creating my console in my UI plugin's PerspectiveFactory I call it as follows:
private void createConsole() {
console = new MessageConsole("My Console", null);
console.activate();
ConsolePlugin.getDefault().getConsoleManager().addConsoles(new IConsole[]{ console });
MessageConsoleStream stream = console.newMessageStream();
MyLaunchConfigurationDelegate.setConsole(new PrintStream(stream));
}
This works, except everytime I close down Eclipse and restart it the console disappears. However when I reset my perspective, the console appears again. So obviously I need that code to be called on startup, not in the PerspectiveFactory itself.
Hope this helps someone.. and if anybody has some input for this last problem (or about my approach in general) please do comment!