i'm trying to build a wrapper around a Python module, to embed it in my java code.
looks like this module use many tricks like subprocess, threading and so on
(actually it is itself a module that control a C utility provided AS-IS and only as a binary, i am trying to avoid the overcost to recode the inner logic and others tools that this python wrapper already provided)
by the way, when instantiate my own wrapper it from Java i get :
------------------
Exception in thread "MainThread" Traceback (most recent call last):
File "<string>", line 1, in <module>
File "__pyclasspath__/mywrapper.py", line 303, in <module>
File "C:\jython2.5.2\Lib\subprocess.py", line 375, in <module>
import msvcrt
ImportError: No module named msvcrt
if i look on my harddisk, there is no msvscrt.py where is it suppose to live ?
i am launching my jython with :
PythonInterpreter interpreter = new PythonInterpreter(null, new PySystemState());
PySystemState sys = Py.getSystemState();
sys.path.append(new PyString("C:/jython2.5.2/Lib"));
sys.platform = new PyString("win32"); // this is a trick for the wrapper to not fail on a inner plateform test detection with java1.7.0_03
msvcrt is not available in Jython. In CPython on Windows, msvcrt is a built-in module compiled into the Python interpreter (you can check this with sys.builtin_module_names). There is no msvcrt.py file.
Why you need "a trick for the wrapper to not fail on a inner plateform test detection with java1.7.0_03", I can't say. But setting sys.platform to win32 makes Jython try to import msvcrt when using subprocess, which doesn't work.
Related
I am trying to work through what should be a simple problem, but am missing something. I have a very simple Java class:
package blah.blah;
public class Tester {
public String testMethod1() { return "GotHere"; }
public String testMethod2() { return "GotHereToo"; }
} // Tester
I am trying to get it to load into JPype:
import jpype
import jpype.imports
from jpype.types import *
path_to_jvm = "C:\\Java\\OracleJDK-8_241_x64\\jre\\bin\\server\\jvm.dll"
jpype.startJVM(path_to_jvm, classpath=["C:\\Users\\Administrator\\eclipse-workspace\\JpypeTest\\src;"])
from java.lang import System
print(System.getProperty("java.class.path"))
from java.io import ObjectInputStream
from blah.blah import Tester
tester = Tester()
print(tester.testMethod1())
print(tester.testMethod2())
jpype.shutdownJVM()
Everything works fine when I load the class directly using the above code. It stops working when I try to get it to load the Class file via a JAR.
jpype.startJVM(path_to_jvm, classpath=["C:\\Users\\Administrator\\JpypeTest.jar;"])
The error I get is:
Traceback (most recent call last):
File ".\jpype_test.py", line 16, in <module>
from blah.blah import Tester
ModuleNotFoundError: No module named 'blah'
PS C:\Users\Administrator> python .\jpype_test.py
C:\Users\Administrator\JpypeTest.jar;
Traceback (most recent call last):
File ".\jpype_test.py", line 16, in <module>
from blah.blah import Tester
ModuleNotFoundError: No module named 'blah'
I have built the JAR file a few different ways from Eclipse 2020/09 using JAR export and from the command line using:
C:\Users\Administrator\eclipse-workspace\JpypeTest\src>"C:\Java\OracleJDK-8_241_x64\bin"\jar cf C:\Users\Administrator\JpypeTest.jar blah\blah\*.class
I have confirmed that the JVM and the python interpreter are both 64 bit. I have also done my best to ensure the JVM is exactly the same between Eclipse, the command line and Python.
From what I can see, the JAR looks fine regardless of how I build it. It contains the blah\blah directory and the Tester.class file under it. The manifest is the only other file in the JAR.
I have also tried creating the JAR file with the class file at different directory levels (ie. one level of blah directories and no levels of blah directory).
Here are the versions of the python software:
Python 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Package Version
------------ -------------------
certifi 2020.6.20
JPype1 1.0.2
pip 20.2.3
setuptools 50.3.0.post20201006
wheel 0.35.1
wincertstore 0.2
I clearly have the base plumbing working since it can see the class file. Any thoughts on why the JAR is failing?
Thanks a bunch for your consideration.
Check that you don't have a directory called "blah" that python would look in for modules. This would hide your Java classes, as per https://jpype.readthedocs.io/en/latest/userguide.html#importing-java-classes :
One important caveat when dealing with importing Java modules. Python always imports local directories as modules before calling the Java importer. So any directory named java, com, or org will hide corresponding Java package. We recommend against naming directories as java or top level domain.
I am trying to install Matlab on a Linux machine, but setting LD_LIBRARY_PATH (as the installation requires) breaks other library files. I am not an Linux expert, but I have tried several things and cannot get it working correctly. I have even contacted Matlab support, got the issue elevated to the dev team, and was basically told "haha sucks to suck". I have seen a few other people online have had the same issue, but either their questions were never answered or they had a slightly different problem and their solution didn't apply to me.
Installing on a VM running Ubuntu:
I set LD_LIBRARY_PATH as the instructions say, then it breaks network files. I can ping google.com, but I cannot nslookup google.com or visit it in a browser. Nslookup provides this error:
nslookup: /usr/local/MATLAB/MATLAB_Runtime/v90/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/libdns.so.100)
03-Feb-2016 11:32:22.361 ENGINE_by_id failed (crypto failure)
03-Feb-2016 11:32:22.362 error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
03-Feb-2016 11:32:22.363 error:260B6084:engine routines:DYNAMIC_LOAD:dso not found:eng_dyn.c:447:
03-Feb-2016 11:32:22.363 error:2606A074:engine routines:ENGINE_by_id:no such engine:eng_list.c:418:id=gost
(null): dst_lib_init: crypto failure
The installation worked though (I can run my Java programs that reference compiled Matlab functions). Unsetting LD_LIBRARY_PATH fixes the network files but then I can't run programs anymore.
Installing on EC2 instance:
On an EC2 instance it does not break the network files (nslookup is fine). Instead it messes up Python library files. Trying to use any aws cli command, I get the error:
File "/usr/bin/aws", line 19, in <module>
import awscli.clidriver
File "/usr/lib/python2.7/dist-packages/awscli/clidriver.py", line 16, in <module>
import botocore.session
File "/usr/lib/python2.7/dist-packages/botocore/session.py", line 25, in <module>
import botocore.config
File "/usr/lib/python2.7/dist-packages/botocore/config.py", line 18, in <module>
from botocore.compat import six
File "/usr/lib/python2.7/dist-packages/botocore/compat.py", line 139, in <module>
import xml.etree.cElementTree
File "/usr/lib64/python2.7/xml/etree/cElementTree.py", line 3, in <module>
from _elementtree import *
ImportError: PyCapsule_Import could not import module "pyexpat"
Printing sys.path in Python shows lib-dynload is already there though, so it doesn't seem to the problem.
And when trying to run the program, I get:
Exception in thread "main" java.lang.LinkageError: libXt.so.6: cannot open shared object file: No such file or directory
at com.mathworks.toolbox.javabuilder.internal.DynamicLibraryUtils.dlopen(Native Method)
at com.mathworks.toolbox.javabuilder.internal.DynamicLibraryUtils.loadLibraryAndBindNativeMethods(DynamicLibraryUtils.java:134)
at com.mathworks.toolbox.javabuilder.internal.MWMCR.<clinit>(MWMCR.java:1529)
at VectorAddExample.VectorAddExampleMCRFactory.newInstance(VectorAddExampleMCRFactory.java:48)
at VectorAddExample.VectorAddExampleMCRFactory.newInstance(VectorAddExampleMCRFactory.java:59)
at VectorAddExample.VectorAddClass.<init>(VectorAddClass.java:62)
at com.mypackage.Example.main(Example.java:13)
I'm at a brick wall and really have no clue how to proceed.
Maybe something else already needs LD_LIBRARY_PATH set to work. Make sure you prepend not overwrite:
export LD_LIBRARY_PATH=new/path:$LD_LIBRARY_PATH
Edit:
OK, if LD_LIBRARY_PATH was initially empty, this suggests that Matlab comes with shared libraries that are incompatible with your system ones:
nslookup: /usr/local/MATLAB/MATLAB_Runtime/v90/bin/glnxa64/libcrypto.so.1.0.0: no version information available (required by /usr/lib/libdns.so.100)
suggests that /usr/lib/libdns.so.100 needs libcrypto.so.1.0.0, which is now being resolved to the one that comes with MATLAB, which is incompatible.
You can check the dependencies of a dll by
ldd /usr/lib/libcrypto.so.1.0.0
and hopefully you can find a configuration that keeps both MATLAB and your system happy. Unfortunately, this may involve a lot of trial and error.
If there is no such configuration, you can try setting LD_LIBRARY_PATH only when you run MATLAB:
LD_LIBRARY_PATH=$MATLAB_LD_LIBRARY_PATH matlab
Edit 2:
Well, for the Python issue, it seems to boil down to pyexpat, which is a wrapper around the standard expat XML parser. Try doing (name guessed since I don't have a Linux right now):
ldd /usr/local/lib/python2.7/site-packages/libpyexpat.so
and see what that depends on. Probably, it will be libexpat.so, which is now being resolved to MATLAB's version.
try the following command:
export LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v90/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v90/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v90/sys/os‌​/glnxa64:$LD_LIBRARY_PATH
Perhaps not helpful for OP but if you are generating a python package with MATLAB, you could modify the generated __init__.py file MATLAB creates for your package.
Specifically, the generated __init__.py file contains the following line (as of MATLAB 2017a):
PLATFORM_DICT = {'Windows': ['PATH','dll',''], 'Linux': ['LD_LIBRARY_PATH','so','libmw'], 'Darwin': ['DYMCR_LIBRARY_PATH','dylib','libmw']}
For Linux platform, you could simply replace LD_LIBRARY_PATH with something else such as MCR_LIBRARY_PATH to prevent mucking with your shared libs.
sed -i -e 's/LD_LIBRARY_PATH/MCR_LIBRARY_PATH/g' /MY/PACKAGE/BUILD/PATH/__init__.py
Then obviously export MCR_LIBRARY_PATH before using python.
After having spend 4 days on searching for a working solution, i guess i need to ask.
So far i'm successfully working withj jython 2.5.2 or 2.5.3, with a modifier thinClient.sh that loads what i need. It connects successfully to a DeploymentManager with either IPC or SOAP connector.
However it lacks the readline module:
wsadmin>import readline
WASX7015E: Exception running command: "import readline"; exception information:
com.ibm.bsf.BSFException: exception from Jython:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr2/produits/websphere7/AppServer/thinClient/lib/jython/Lib/readline.py", line 20, in <module>
raise ImportError("Cannot access JLineConsole")
ImportError: Cannot access JLineConsole
The goal is to make it interactive, with colored prompt and sofort.
I tried so far with absolutely no success:
org.python.util.JLineConsole / org.python.util.ReadlineConsole (misses readline module)
Readline-1.7 (does nothing)
java-readline / libreadline-java-0.8.0 (misses readline module)
JLine (won't load the jar)
pyreadline (won't integrate to jython)
So:
is it possible with jython 2.5.3 under IBM AIX 64, with a thinClient (jython-installer-2.5.3.jar) to have a real interactive shell with bash like completion and command recall using arrow keys, without having to build/compile something ?
if yes, please somebody describe a working solution:
what's the solution name ?
what's in your wsadmin.properties ?
which libraries to load in LIBPATH ?
which class to load in CLASSPATH ?
which command line to invoque with java ?
There are so many "solutions" or "propositions" to this frenquently asked question on the web, but nowhere did i found a well described or working solution. Too much information scattered all around just becomes a mess :(
thanks for any help !
ok i found a workaround, that was so easy to answer myself:
rlwrap -H $THIN_CLIENT_HOME/logs/rlwrap.history.log -f $THIN_CLIENT_HOME/etc/rlwrap.jython.words.txt -r -pBlue -z $THIN_CLIENT_HOME/etc/rlwrap.prompt.pl $CMDLINE
org.python.util.* and com.ibm.ws.scripting.WasxShell are mutually incompatible
I'm new to python and jython. I had a script in python which imports mmap, it works fine without any issues. I need to invoke a function in that script from a java class, which throws the following exception upon execution.
Exception in thread "main"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named mmap
I executed the same script in jython which gives me the below error.
C:\jython2.5.3>jython.bat d:\Python\AddConfToFile.py
Traceback (most recent call last):
File "d:\Python\AddConfToFile.py", line 9, in <module>
import mmap
ImportError: No module named mmap
I'm not able to find any suggestions through Google. Am I missing some thing here, kindly Help me fix this.
Thanks in Advance
Adithyan
The mmap module is not available under Jython. You will need to find an alternative.
It's just because the Jython doesn't support mmap still..As explained in the FAQ in the official site, in the paragraph below:
Modules that we don't support, and perhaps won't, include bsddb3, curses, dbm, gdbm,
ioctl, mmap, pty, Tkinter, and win32reg. Some of these might be readily doable, like
mapping the curses module against CHARVA, but given limited resources, are not likely to
be worked on. (Contributions welcome!)
I'm developing collection of monitoring script and templates for zabbix. It's called ZTC and all script are on python.
Now I want to add support for some java monitoring. I've not found the way to do it from CPython - only from java or jython. Since all project is on python, I've decided to write a simple script on jython, which would be called from my cpython classes.
Here is how my code looks like:
#!/usr/bin/env jython
#Java Dependencies
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;
import java.lang.management.ManagementFactory;
#Python Dependencies
import sys, cmd, socket
def usage():
print """Usage:
jmxclient.py -h
jmxclient.py <connect_url> <jmx_attribute_path> <jmx_property>"""
class JMXClient:
remote = None
def connect(self, connect_url):
if self.remote:
return True
#Establish Connection to JMX Server
url = javax.management.remote.JMXServiceURL(connect_url);
connector = javax.management.remote.JMXConnectorFactory.connect(url);
self.remote = connector.getMBeanServerConnection();
def getAttribute(self, mbean_path, attribute):
"""Query the mbean server for a specific attribute and return the
result"""
obn = javax.management.ObjectName(mbean_path);
result = self.remote.getAttribute(obn, attribute);
return result
if len(sys.argv) <= 1:
usage()
sys.exit(2)
if sys.argv[1] in ('-h', '--help'):
usage()
sys.exit(2)
if len(sys.argv) <> 4:
usage()
sys.exit(2)
(connect_url, mbean_path, attribute) = sys.argv[1:]
j = JMXClient()
j.connect(connect_url)
print j.getAttribute(mbean_path, attribute)
Ok, now I'm trying to get some attribute from terracotta server. It uses jmxmp with url service:jmx:jmxmp://0.0.0.0:9520.
So, I'm running my script as follows:
$ ./jmxclient.py service:jmx:jmxmp://localhost:9520 java.lang.ClassLoading LoadedClassCount
Traceback (innermost last):
File "./jmxclient.py", line 87, in ?
File "./jmxclient.py", line 61, in connect
at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:327)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:247)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:207)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
java.net.MalformedURLException: java.net.MalformedURLException: Unsupported protocol: jmxmp
(line numbers not relevant due to some stripped comments)
How do I add support of this jmxmp protocol?
I've found that it seems it might be enabled by jmxremote_optional.jar. How do I add this jar to my jython (pref. not system-wide)?
UPDATE:
As suggested, I've added jmxremote_optional.jar and jmxremote.jar from jmxremote-1_0_1-ri-bin-b58.zip reference implementation: jython -Djava.endorsed.dirs=. -Dpython.path=.../jmxremote_optional.jar:.../jmxremote.jar:.../jmissl.jar jmxclient.py service:jmx:jmxmp://localhost:9520 java.lang.ClassLoading LoadedClassCount, but still getting the same error. I'm sure that jmxremote_optional.jar is in classpath, and code seems to be very similar to reference examples.
After reading api docs, I've tried following changes:
url = javax.management.remote.JMXServiceURL('jmxmp', 'localhost', 9520);
connector = javax.management.remote.jmxmp.JMXMPConnector(url)
connector.connect()
self.remote = connector.getMBeanServerConnection();
which leads me to another exception:
Traceback (innermost last):
File "../src/jmxclient.py", line 87, in ?
File "../src/jmxclient.py", line 61, in connect
at com.sun.jmx.remote.opt.security.AdminClient.connectionOpen(AdminClient.java:209)
at com.sun.jmx.remote.generic.ClientSynchroMessageConnectionImpl.connect(ClientSynchroMessageConnectionImpl.java:72)
at javax.management.remote.generic.GenericConnector.connect(GenericConnector.java:177)
at javax.management.remote.jmxmp.JMXMPConnector.connect(JMXMPConnector.java:119)
at javax.management.remote.generic.GenericConnector.connect(GenericConnector.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
java.io.IOException: java.io.IOException: javax.management.remote.message.HandshakeBeginMessage
Jython version is 2.2 and I don't like to use later version, because this scripts is mostly being used on RHEL5 boxes, and they only have jython 2.2.1.
PS: marking question as answered, because I've decided to give up and use jmxterm or similar tool, which all seems to work with jmxmp by just adding -Djava.endosed.dirs=/path/to/dir_with_jmxremote_optional/.
But I'd still like to see jython solution.
Man, I read that post about 7 times .....
Anyways, here's what I think is going on. The jmxmp protocol is not packaged in the standard J2SE runtime. See this page. To quote:
Note – If you want to use a JMXMP connector, download the JSR 160 Reference Implementation from http://java.sun.com/products/JavaManagement/download.html, and add the jmxremote_optional.jar file to your classpath. You will find examples of use of the JMXMP connectors in the JMX Remote API Tutorial included with the JSR 160 Reference Implementation.
I'm not that familiar with jython, but it looks like this post should hook you up.
I've faced the same problem. The solution is to use a different method of importing jar jmxremote_optional.jar. It's described here https://stackoverflow.com/a/11638390/9209536
This code works in Jython 2.7
def importJar(jarFile):
from java.net import URL, URLClassLoader
from java.lang import ClassLoader
from java.io import File
m = URLClassLoader.getDeclaredMethod("addURL", [URL])
m.accessible = 1
m.invoke(ClassLoader.getSystemClassLoader(), [File(jarFile).toURL()])
importJar("opendmk_jmxremote_optional_jar-1.0-b01-ea.jar")
from javax.management.remote import JMXServiceURL
from javax.management.remote import JMXConnector
from javax.management.remote import JMXConnectorFactory
jmx_url = JMXServiceURL("service:jmx:jmxmp://server:port/")
jmx_connector = JMXConnectorFactory.connect(jmx_url)
Another option could be to use a different stack like Jolokia which exports JMX information via HTTP and JSON. There a already various client binding (Perl via Jmx4Perl, Javascript, Java) although no Python yet. But it shouldn't be hard to build one from scratch, the protocol is described in detail here.