In .bashrc and .profile I have my QHOME variable set to the directory that contains k4.lic, l64, q.k (validated by echo $QHOME).
When I startup q from the login shell all works fine, the license file is found.
When I start a q process programmaticaly from java, then I get the following output
[13:43:48][Step 1/2] WARN [main] (QSession.java:78) - Q Session not alive
[13:43:48][Step 1/2] INFO [main] (QSession.java:97) - QHOME: null
[13:43:48][Step 1/2] INFO [main] (QSession.java:98) - QLIC: null
[13:43:48][Step 1/2] ERROR [main] (QSession.java:101) - Error output
[13:43:48][Step 1/2] '2018.02.06T13:43:46.597 k4.lic
i.e. the license is not found because the QHOME env variable is undefined. This problem is described here: ".bashrc is only sourced in a login shell". The proposed solution is
"If you want a variable to be set in all Bourne shell derivatives regardless of whether they are interactive or not, put it in both .profile and .bashrc."
But I have already copied the content of .bashrc into .profile and still get the same error.
Unfortunately, there is no way to pass the path to the license as a command line argument for the q binary, so I have to work with QHOME.
What I could do is put a 32-bit version in my java project but obviously it is advantageous to use the 64-bit version.
Suggestions much appreciated!
Thanks
Thanks to #Jonathan McMurray!
The exact solution is to use
Runtime.getRuntime().exec(command, envp);
where command is for example q -p 5000 and envp is for example
String[] envp = {"QHOME="+qHomePath};
Related
I've tried everything I can think of to get WebDriverManager to detect the correct version of the chrome browser that I can think of, nothing works. I want to detect the browser driver version from the chrome.exe located at C:\Program Files\Google\Chrome Beta\Application. However, no matter what I do it runs the following command to detect the version:
io.github.bonigarcia.wdm.versions.Shell - Running command on the shell: [cmd.exe, /C, wmic, datafile, where, name="%PROGRAMFILES(X86):\=\\%\\Google\\Chrome\\Application\\chrome.exe", get, Version, /value]
I have tried so far:
For all the below:
sBrowserBinaryLocation = "C:\Program Files\Google\Chrome Beta\Application";
setting
options.setBinary(sBrowserBinaryLocation);
WebDriverManager.chromedriver().clearDriverCache().setup();
Setting:
options.setBinary(sBrowserBinaryLocation);
WebDriverManager.chromedriver().clearDriverCache().arch64().setup();
Setting:
options.setBinary(sBrowserBinaryLocation);
WebDriverManager.chromedriver().clearDriverCache().browserVersionDetectionCommand(sBrowserBinaryLocation + " --version").arch64().setup();
Every one of those produced the same
io.github.bonigarcia.wdm.versions.Shell - Running command on the shell: [cmd.exe, /C, wmic, datafile, where, name="%PROGRAMFILES(X86):\=\\%\\Google\\Chrome\\Application\\chrome.exe", get, Version, /value]
Which picked up the non-beta version 99. How do I tell WebDriverManager to use "C:\Program Files\Google\Chrome Beta\Application" rather than "C:\Program Files(x86)\Google\Chrome\Application"?
Although it is tricky, it can be done using the WebDriverManager method browserVersionDetectionCommand(). Supposing that Chrome Beta is in this path:
C:\Program Files\Google\Chrome Beta\Application\chrome.exe
... we need to run the following command in the shell:
cmd.exe /C wmic datafile where name="%PROGRAMFILES:\=\\%\\Google\\Chrome Beta\\Application\\chrome.exe" get Version /value
The problem is that, when running this command in Java, the blank space (in "Chrome Beta" folder) makes this command to be interpreted incorretly. The solution is to find the Windows path name using the ~ sintax. You can discover these names by running dir /X in the Windows shell (e.g, C:\Program Files becomes C:\PROGRA~1).
You can see a test using this feature here. The traces of this test are as follows:
2022-03-23 13:35:00 [main] DEBUG i.g.bonigarcia.wdm.versions.Shell.runAndWaitArray(65) -- Running command on the shell: [cmd.exe, /C, wmic, datafile, where, name="C:\\PROGRA~1\\GOOGLE\\CHROME~1\\APPLIC~1\\CHROME.EXE", get, Version, /value]
2022-03-23 13:35:00 [main] DEBUG i.g.bonigarcia.wdm.versions.Shell.runAndWaitArray(69) -- Result: Version=100.0.4896.46
2022-03-23 13:35:00 [main] DEBUG i.g.b.wdm.versions.VersionDetector.getDriverVersionFromRepository(127) -- Latest version of chromedriver according to https://chromedriver.storage.googleapis.com/LATEST_RELEASE_100 is 100.0.4896.20
2022-03-23 13:35:00 [main] INFO i.g.bonigarcia.wdm.WebDriverManager.resolveDriverVersion(1093) -- Using chromedriver 100.0.4896.20 (resolved driver for Chrome 100)
2022-03-23 13:35:00 [main] DEBUG i.g.b.wdm.cache.ResolutionCache.putValueInResolutionCacheIfEmpty(119) -- Storing resolution chrome=100 in cache (valid until 14:35:00 23/03/2022 CET)
2022-03-23 13:35:00 [main] DEBUG i.g.b.wdm.cache.ResolutionCache.putValueInResolutionCacheIfEmpty(119) -- Storing resolution chrome100=100.0.4896.20 in cache (valid until 13:35:00 24/03/2022 CET)
2022-03-23 13:35:00 [main] DEBUG i.g.bonigarcia.wdm.WebDriverManager.manage(1049) -- Driver chromedriver 100.0.4896.20 found in cache
2022-03-23 13:35:00 [main] INFO i.g.bonigarcia.wdm.WebDriverManager.exportDriver(1148) -- Exporting webdriver.chrome.driver as C:\Users\boni\.cache\selenium\chromedriver\win32\100.0.4896.20\chromedriver.exe
With regard to the Log4j JNDI remote code execution vulnerability that has been identified CVE-2021-44228 - (also see references) - I wondered if Log4j-v1.2 is also impacted, but the closest I got from source code review is the JMS-Appender.
The question is, while the posts on the Internet indicate that Log4j 1.2 is also vulnerable, I am not able to find the relevant source code for it.
Am I missing something that others have identified?
Log4j 1.2 appears to have a vulnerability in the socket-server class, but my understanding is that it needs to be enabled in the first place for it to be applicable and hence is not a passive threat unlike the JNDI-lookup vulnerability which the one identified appears to be.
Is my understanding - that Log4j v1.2 - is not vulnerable to the jndi-remote-code execution bug correct?
References
Apache Log4j Security Vulnerabilities
Zero-day in ubiquitous Log4j tool poses a grave threat to the Internet
Worst Apache Log4j RCE Zero day Dropped on Internet
‘Log4Shell’ vulnerability poses critical threat to applications using ‘ubiquitous’ Java logging package Apache Log4j
This blog post from Cloudflare also indicates the same point as from AKX....that it was introduced from Log4j 2!
Update #1 - A fork of the (now-retired) apache-log4j-1.2.x with patch fixes for few vulnerabilities identified in the older library is now available (from the original log4j author). The site is https://reload4j.qos.ch/. As of 21-Jan-2022 version 1.2.18.2 has been released. Vulnerabilities addressed to date include those pertaining to JMSAppender, SocketServer and Chainsaw vulnerabilities. Note that I am simply relaying this information. Have not verified the fixes from my end. Please refer the link for additional details.
The JNDI feature was added into Log4j 2.0-beta9.
Log4j 1.x thus does not have the vulnerable code.
While not affected by the exact same Log4Shell issue, the Apache Log4j team recommends to remove JMSAppender and SocketServer, which has a vulnerability in CVE-2019-17571, from your JAR files.
You can use the zip command to remove the affected classes. Replace the filename/version with yours:
zip -d log4j-1.2.16.jar org/apache/log4j/net/JMSAppender.class
zip -d log4j-1.2.16.jar org/apache/log4j/net/SocketServer.class
You can look through through the files in your zip using less and grep, e.g. less log4j-1.2.16.jar | grep JMSAppender
That being said, Apache recommends that you upgrade to the 2.x version if possible. According to their security page:
Please note that Log4j 1.x has reached end of life and is no longer supported. Vulnerabilities reported after August 2015 against Log4j 1.x were not checked and will not be fixed. Users should upgrade to Log4j 2 to obtain security fixes.
In addition to giraffesyo's answer and in case it helps anyone - I wrote this Bash script - which removes classes identified as vulnerabilities (link here to Log4j dev thread) and sets properties files are read-only - as suggested here on a Red Hat Bugzilla thread.
Note 1 - it does not check for any usage of these classes in properties it is purely a way to find and remove - use at own risk!
Note 2 - it depends on zip and unzip being installed
#!/bin/bash
DIR=$1
APPLY=$2
# Classes to be searched for/removed
CLASSES="org/apache/log4j/net/SimpleSocketServer.class
org/apache/log4j/net/SocketServer.class
org/apache/log4j/net/JMSAppender.class"
PROGNAME=`basename $0`
PROGPATH=`echo $0 | sed -e 's,[\\/][^\\/][^\\/]*$,,'`
usage () {
echo >&2 Usage: ${PROGNAME} DIR [APPLY]
echo >&2 Where DIR is the starting directory for find
echo >&2 and APPLY = "Y" - to perform purification
exit 1
}
# Force upper case on Apply
APPLY=$(echo "${APPLY}" | tr '[:lower:]' '[:upper:]')
# Default Apply to N
if [ "$APPLY" == "" ] ; then
APPLY="N"
fi
# Check parameters
if [ "$DIR" == "" ] ; then
usage
fi
echo $APPLY | grep -q -i -e '^Y$' -e '^N$' || usage
# Search for log4j jar files - for class file removal
FILES=$(find $DIR -name *log4j*jar)
for f in $FILES
do
echo "Checking Jar [$f]"
for jf in $CLASSES
do
unzip -v $f | grep -e "$jf"
if [ "$APPLY" = "Y" ]
then
echo "Deleting $jf from $f"
zip -d $f $jf
fi
done
done
# Search for Log4j properties files - for read-only setting
PFILES=$(find $DIR -name *log4j*properties)
for f in $PFILES
do
echo "Checking permissions [$f]"
if [ "$APPLY" = "Y" ]
then
echo "Changing permissons on $f"
chmod 444 $f
fi
ls -l $f
done
I am trying to run a basic robotframework test with RemoteSwingLibrary, but I can't seem to get it to work.
I was able to run the following test just fine with SwingLibrary:
*** Settings ***
Library SwingLibrary
*** Test Cases ***
Foobar Test
Start Application fully.qualified.name.of.my.class
I am running it from within Eclipse via the robot framework standalone jar. I put robotframework-3.0.2.jar and swinglibrary-1.9.7.jar on my class path, created a new run configuration using my projects default classpath with the main class being org.robotframework.RobotFramework. In the arguments, I specify the name of my test file.
So, that seems to work fine. Then I tried to do almost exactly the same thing with RemoteSwingLibrary:
*** Settings ***
Library RemoteSwingLibrary
*** Test Cases ***
Foobar Test
Start Application my_app java fully.qualified.name.of.my.class
I replaced the swinglibrary jar on my classpath with remoteswinglibrary-2.2.1.jar and I ran the same test. The output says:
console: Failed to install '': java.nio.charset.UnsupportedCharsetException: cp0.
==============================================================================
Remotetest
==============================================================================
Foobar Test | FAIL |
RemoteSwingLibraryTimeoutError: Agent port not received before timeout
------------------------------------------------------------------------------
Remotetest | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
Output: C:\<path>\output.xml
Log: C:\<path>\log.html
Report: C:\<path>\report.html
(note that the "console: Failed to install..." part is normal and I don't think I need to be concerned about it)
When I dig into the report for the Start Application Keyword, it says
13:48:40.070 INFO Link to stdout
13:48:40.070 INFO Link to stderr
13:48:40.070 INFO -javaagent:"C:<pathToProject>\__pyclasspath__"=127.0.0.1:63110
13:48:40.071 INFO Starting process:
java fully.qualified.name.of.my.class
13:49:40.145 INFO Failed to start application: Traceback (most recent call last):
File "__pyclasspath__/RemoteSwingLibrary.py", line 307, in start_application
File "__pyclasspath__/RemoteSwingLibrary.py", line 353, in _application_started
File "__pyclasspath__/RemoteSwingLibrary.py", line 373, in _get_agent_address
RemoteSwingLibraryTimeoutError: Agent port not received before timeout
13:49:40.155 INFO Waiting for process to complete.
13:49:40.165 INFO Process completed.
13:49:40.165 INFO STDOUT: Error occurred during initialization of VM
agent library failed to init: instrument
13:49:40.175 INFO STDERR: Error opening zip file or JAR manifest missing : C:<pathToProject>\__pyclasspath__
Picked up JAVA_TOOL_OPTIONS: -javaagent:"C:<pathToProject>\__pyclasspath__"=127.0.0.1:63110
Picked up _JAVA_OPTIONS: -Djava.security.policy="C:\<UserFolder>\
AppData\Local\Temp\2\grant_all_f_vfit.policy"
13:49:40.195 FAIL RemoteSwingLibraryTimeoutError: Agent port not received before timeout
I'm not sure if this hunch is really founded on anything, but my suspicion is that there's a classpath issue. I would think that when I call the Start Application keyword with "java fully.qualified.name.of.my.class" I would probably have to pass along my classpath. I was hoping that maybe I wouldn't since the non-remote version seems to handle that just fine (probably because its running my app in the same JVM as I launched robot framework in, and that already included everything I needed in my classpath).
So if that's the issue, I can probably fix it by using the -cp option in my java command. However, the classpath for this particular project is actually quite long (my project is dependent on a lot of other projects) and I would rather not have to list it all out. Also, when I add a new dependency down the road, I would rather not have to update the classpath in my test file as well.
Is there some magic way that I can pass the current classpath along to the remote app? Or perhaps some other solution to this problem?
Update
I tried again with -cp and my giant classpath and it gave me the same errors, so its possible that my classpath has nothing to do with whatever is going on.
I also tried specifying the port number explicitly and it gives me slightly different output (though it still doesn't work). I also discovered that there's a debug setting that gives a little more detail. Here's me new setup:
*** Settings ***
Library RemoteSwingLibrary debug=True port=8242
*** Test Cases ***
Foobar Test
Start Application my_app java -cp <classpath> <fully.qualified.name.of.my.class> remote_port=8242
And here is my new output:
13:48:40.070 INFO Link to stdout
13:48:40.070 INFO Link to stderr
13:48:40.070 DEBUG Picked old JAVA_TOOL_OPTIONS=''
13:48:40.070 DEBUG Picked old _JAVA_OPTIONS=''
13:48:40.070 INFO -javaagent:"C:<pathToProject>\__pyclasspath__"=127.0.0.1:51509:DEBUG
13:48:40.070 DEBUG Set JAVA_TOOL_OPTIONS='-javaagent:"C:<pathToProject>\__pyclasspath__"=127.0.0.1:8242:APPORT=8242:DEBUG'
13:48:40.070 DEBUG SET _JAVA_OPTIONS='-Djava.security.policy="C:\<UserFolder>\AppData\Local\Temp\4\grant_all_haud6t.policy"'
13:48:40.071 INFO Starting process:
java -cp <classpath> <fully.qualified.name.of.my.class>
13:48:40.071 DEBUG Process configuration:
cwd: C:<PathToProject>
shell: True
stdout: C:<PathToProject>\<someBigPath>.txt
stderr: C:<PathToProject>\<someBigPath>.txt
alias: my_app
env: None
13:48:40.100 DEBUG Returned old JAVA_TOOL_OPTIONS=''
13:48:40.100 DEBUG Returned old _JAVA_OPTIONS=''
13:48:40.102 INFO connection to started application at 127.0.0.1:8242
13:48:40.120 DEBUG remote swinglibrary instantiated
13:48:40.121 DEBUG remote services instantiated
13:48:40.121 INFO waiting for api at 127.0.0.1:8242
13:49:40.145 INFO Failed to start application: Traceback (most recent call last):
File "__pyclasspath__/RemoteSwingLibrary.py", line 307, in start_application
File "__pyclasspath__/RemoteSwingLibrary.py", line 357, in _application_started
File "__pyclasspath__/RemoteSwingLibrary.py", line 344, in _wait_for_api
RuntimeError: Connecting to api at 127.00.1:8242 has failed: ''
13:49:40.155 INFO Waiting for process to complete.
13:49:40.165 INFO Process completed.
13:49:40.165 INFO STDOUT: Error occurred during initialization of VM
agent library failed to init: instrument
13:49:40.175 INFO STDERR: Error opening zip file or JAR manifest missing : C:<pathToProject>\__pyclasspath__
Picked up JAVA_TOOL_OPTIONS: -javaagent:"C:<pathToProject>\__pyclasspath__"=127.0.0.1:8242:APPORT=8242:DEBUG
Picked up _JAVA_OPTIONS: -Djava.security.policy="C:\<UserFolder>\
AppData\Local\Temp\2\grant_all_f_vfit.policy"
13:49:40.195 FAIL Connecting to api at 127.0.0.1:8242 has failed: ''
13:49:40.195 DEBUG Traceback (most recent call last):
File "__pyclasspath__/RemoteSwingLibrary.py", line 498, in run_keyword
File "__pyclasspath__/RemoteSwingLibrary.py", line 307, in start_application
File "__pyclasspath__/RemoteSwingLibrary.py", line 357, in _application_started
File "__pyclasspath__/RemoteSwingLibrary.py", line 344, in _wait_for_api
Yet another update
I think that this may have something to do with the "__pyclasspath__" that shows up in the error message. I put my java command in a .bat file preceded by a "set" command to list my environment variables. I can run the .bat file from the command line just fine (it launches the java app).
Then I modified my .robot file to do:
Foobar Test
Start Application my_app cd robotframework/tests && Run.bat remote_port=8242
I get essentially the same error output that I got before:
Error opening zip file or JAR manifest missing : C:<pathToProject>\__pyclasspath__
Picked up JAVA_TOOL_OPTIONS: -javaagent:"__pyclasspath__"=127.0.0.1:8242:APPORT=8242:DEBUG
Picked up _JAVA_OPTIONS: -Djava.security.policy="C:\\AppData\Local\Temp\1\grant_all_lgirs7.policy"
The environment that my batch script prints out appears the same except for JAVA_TOOL_OPTIONS and _JAVA_OPTIONS (which match with what the error output says). There is no environment variable called __pyclasspath__ so I'm wondering if that's the issue.
I set JAVA_TOOL_OPTIONS in my cmd environment to match the output that I saw from my robot run and now I get the same error message when I try to run my .bat script from the command line, so that environment variable seems to be the issue.
Well, I came up with a fix, though its definitely not an ideal fix.
I found out that when the framework kicks of my remote app, it sets:
JAVA_TOOL_OPTIONS=-javaagent:"C:\<pathToProject>\__pyclasspath__
But __pyclasspath__ is not actually an environment variable. I figured out that the javaagent has to point to the remote framework library, so I decided to just set it explicitly.
So now I changed my robot file to something like this:
*** Settings ***
Library RemoteSwingLibrary
*** Test Cases ***
Foobar Test
Start Application my_app run.bat remote_port=1234
And then made a file called run.bat that does:
set JAVA_TOOL_OPTIONS=-javaagent:"C:\<pathToProject\lib\remoteswinglibrary-2.2.1.jar"=127.0.0.1:1234:APPORT=1234
java -cp <classPath> fully.qualified.name.of.class
This is obviously sort of a hacky solution and it will take a little work to make it portable (I'll need to get rid of the hardcoded "pathToProject" and also make it support Linux environments as well). It also requires hardcoding in a port, whereas it would be nice if the robotframework could just select a port for me.
So, I would definitely like to see a better solution if anyone can find one, but for now, this will at least work.
Can anyone help me how to integrate storm with zookeeper in windows.
I try to find the find a good installation steps for production in windows but I can't.
Write now I have installed stand alone zookeeper and I am trying to configure it in the storm.yaml.
sample code I tried :
storm.zookeeper.servers:
- "127.0.0.1"
- "server2"
storm.zookeeper.port: 2180
nimbus.host: "localhost"
If any body knows please help me.
Please follow the below steps for complete installation of storm with zookeeper in windows
1. create a zoo.cfg file under /conf directory of zookeeper.
here are the configuration entries
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=D:\\workspace\\zk-data
# the port at which the clients will connect
clientPort=2181
2.run the zookeeper by executing the zkServer.bat in /bin directory
3.create the storm.yaml file in /conf directory of the apache-storm
here are the configuration entries
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "localhost"
#supervisor.slots.ports:
# - 6700
# - 6701
# - 6702
# - 6703
storm.local.dir: D:\\workspace\\storm-data
nimbus.host: "localhost"
#
#
4.run the below commands from the command prompt
a. bin\storm nimbus
b. bin\storm supervisor
c. bin\storm ui
hit the browser with localhost:8080 to see the nimbus ui
6.make sure your JAVA_HOME path does not have any space. otherwise storm command
will not run.
i want to monitor c3p0 connection pool parameters with Icinga.
So i found for this the nagios plugin jmxquery.
There will be a patch for wildcard queries.
I've patched the plugin like described here,
but after that i'll get NullPointerException's on every query i run.
[root#hostname target]# ./check_jmx -U service:jmx:rmi:///jndi/rmi://<HOSTNAME>:9001/jmxrmi -O com.mchange.v2.c3p0:type=PooledDataSource[2rw2h791t5s2b210jnofo\|2ab68416] -A numConnectionsAllUsers -I numConnectionsAllUsers -vvvv -username monitorRole -password *******************
JMX CRITICAL - NullPointerException: null connecting to com.mchange.v2.c3p0:type=PooledDataSource[2rw2h791t5s2b210jnofo|2ab68416] by URL service:jmx:rmi:///jndi/rmi://<HOSTNAME>:9001/jmxrmijava.lang.NullPointerException
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1008)
at java.lang.Double.parseDouble(Double.java:540)
at jmxquery.JMXQuery.compare(JMXQuery.java:199)
at jmxquery.JMXQuery.report(JMXQuery.java:147)
at jmxquery.JMXQuery.main(JMXQuery.java:93)
Any Ideas??
Alternate tool to access jmx beans.
Jmxterm is a command line based interactive JMX client. It's designed to allow user to access a Java MBean server from command line without graphical environment. If this is useful please check.
JMXTerm