We Keep Coding

Java binary available, but still "No such file or directory"

When I run the command I got the error:
root#1290ff36814b:/opt/airflow/dags# python test.py
/opt/spark-3.3.1/bin/spark-class: line 71: /opt/jdk1.8.0/bin/java: No such file or directory
/opt/spark-3.3.1/bin/spark-class: line 96: CMD: bad array subscript
Traceback (most recent call last):
File "test.py", line 16, in <module>
.appName('test') \
File "/opt/spark-3.3.1/python/pyspark/sql/session.py", line 269, in getOrCreate
sc = SparkContext.getOrCreate(sparkConf)
File "/opt/spark-3.3.1/python/pyspark/context.py", line 483, in getOrCreate
SparkContext(conf=conf or SparkConf())
File "/opt/spark-3.3.1/python/pyspark/context.py", line 195, in __init__
SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
File "/opt/spark-3.3.1/python/pyspark/context.py", line 417, in _ensure_initialized
SparkContext._gateway = gateway or launch_gateway(conf)
File "/opt/spark-3.3.1/python/pyspark/java_gateway.py", line 106, in launch_gateway
raise RuntimeError("Java gateway process exited before sending its port number")
RuntimeError: Java gateway process exited before sending its port number
This is a docker container and /opt/jdk1.8.0 is a volume, and the volume is visible inside container and also de java binary.
root#1290ff36814b:/opt/airflow/dags# ls -l /opt/jdk1.8.0/bin/java
-rwxr-xr-x 1 uucp 143 8464 Dec 15 2018 /opt/jdk1.8.0/bin/java
root#1290ff36814b:/opt/airflow/dags# ls -l /opt/jdk1.8.0/
total 25996
-r--r--r-- 1 uucp 143 3244 Dec 15 2018 COPYRIGHT
-r--r--r-- 1 uucp 143 40 Dec 15 2018 LICENSE
-r--r--r-- 1 uucp 143 159 Dec 15 2018 README.html
-rw-r--r-- 1 uucp 143 112724 Dec 12 2018 THIRDPARTYLICENSEREADME-JAVAFX.txt
-r--r--r-- 1 uucp 143 153824 Dec 15 2018 THIRDPARTYLICENSEREADME.txt
drwxr-xr-x 2 uucp 143 4096 Dec 15 2018 bin
drwxr-xr-x 3 uucp 143 132 Dec 15 2018 include
-rw-r--r-- 1 uucp 143 5211154 Dec 12 2018 javafx-src.zip
drwxr-xr-x 5 uucp 143 4096 Dec 15 2018 jre
drwxr-xr-x 5 uucp 143 4096 Dec 15 2018 lib
drwxr-xr-x 4 uucp 143 47 Dec 15 2018 man
-rw-r--r-- 1 uucp 143 424 Dec 15 2018 release
-rw-r--r-- 1 uucp 143 21105955 Dec 15 2018 src.zip
root#1290ff36814b:/opt/airflow/dags#
Why this happen?
I already tried to set JAVA_HOME and PATH, but still the same problem.

I just realize it has one space inside my python code where I export the JAVA_HOME. After remove it finally the code worked.
import os
import sys
os.environ["SPARK_HOME"]='/opt/spark-3.3.1'
os.environ["JAVA_HOME"]=' /opt/jdk1.8.0'
sys.path+=[os.environ["SPARK_HOME"]+"/python"]
sys.path+=[os.environ["SPARK_HOME"]+"/python/build"]
sys.path+=[os.environ["SPARK_HOME"]+"/python/lib/py4j-0.10.9.5-src.zip"]

Keep link to /etc/alternatives/jre when switching java

I have 2 java on my server, where I installed java8 first, then java 11
by default, there is no jre -> /etc/alternatives/jre
[root#server-1:jvm]# ls -all
...
lrwxrwxrwx 1 root root 26 Mar 17 09:10 java -> /etc/alternatives/java_sdk
drwxr-xr-x 9 root root 215 Mar 17 09:10 java-11-amazon-corretto
drwxr-xr-x 3 root root 17 Mar 17 07:17 java-1.8.0-amazon-corretto.x86_64
While after I set the default java with alternative command, the link to /etc/alternatives/jre is created automatically
[root#server-1:jvm]# alternatives --config java
There are 2 programs which provide 'java'.
Selection Command
-----------------------------------------------
+ 1 /usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/jre/bin/java
* 2 /usr/lib/jvm/java-11-amazon-corretto/bin/java
Enter to keep the current selection[+], or type selection number: 2
[root#server-1:jvm]# ls -all
...
lrwxrwxrwx 1 root root 26 Mar 17 09:10 java -> /etc/alternatives/java_sdk
drwxr-xr-x 9 root root 215 Mar 17 09:10 java-11-amazon-corretto
drwxr-xr-x 3 root root 17 Mar 17 07:17 java-1.8.0-amazon-corretto.x86_64
lrwxrwxrwx 1 root root 21 Mar 17 09:21 jre -> /etc/alternatives/jre <<<<< i want to keep this
But once I switch back to java8, the jre -> /etc/alternatives/jre was removed.
How can I keep the link?
UPDATE: example problem what to be solved
A application have to access the cacert file in java, e.g.
***/usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/jre***/lib/security/cacerts
***/usr/lib/jvm/java-11-amazon-corretto.x86_64***/lib/security/cacerts
So I would like to have the /usr/lib/jvm/jre or /etc/alternatives/jre pointing to the java home
In case of corretto8
/usr/lib/jvm/jre -> /etc/alternatives/jre -> /usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/jre
In case of corretto11 (which the 'alternative' already did if I switch to )
/usr/lib/jvm/jre -> /etc/alternatives/jre -> /usr/lib/jvm/java-11-amazon-corretto.x86_64

zookeeper Starting error :'JAVA_HOME error in zookeeper.out'

When I do
bin/zkServer.sh start #It shows it has started
ZooKeeper JMX enabled by default
Using config: /data/sparkHA/zookeeper-3.4.9/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Afetr few second when i check status, I got
ZooKeeper JMX enabled by default
Using config: /data/sparkHA/zookeeper-3.4.9/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
My zookeeper.out says
nohup: failed to run command â/usr/bin/java/bin/javaâ: Not a directory
But my JAVA_HOME in bashrc is usr/bin/java, How come one extra /bin/java is added , that results in an invalid directory.
Also echo $JAVA_HOME outputs
/usr/bin/java
How to approach this error. Please Help. Thanks.
Also tried setting JAVA_HOME in zkServer.sh also by following Zookeeper not starting, nohup error but getting same error.

Your JAVA_HOME point to /usr/bin/java file but it should point to root directory of your JDK. For example for me this is:
➜ ~ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home
➜ ~ ll $JAVA_HOME
total 52064
-rw-rw-r-- 1 root wheel 3.2K Oct 1 09:00 COPYRIGHT
-rw-rw-r-- 1 root wheel 40B Oct 1 09:01 LICENSE
-rw-rw-r-- 1 root wheel 159B Oct 1 09:01 README.html
-rwxrwxr-x 1 root wheel 108K Sep 22 22:49 THIRDPARTYLICENSEREADME-JAVAFX.txt
-rw-rw-r-- 1 root wheel 173K Oct 1 09:01 THIRDPARTYLICENSEREADME.txt
drwxrwxr-x 46 root wheel 1.5K Oct 1 09:04 bin
drwxrwxr-x 9 root wheel 306B Oct 1 09:00 db
drwxrwxr-x 9 root wheel 306B Oct 1 09:00 include
-rwxrwxr-x 1 root wheel 4.9M Sep 22 22:49 javafx-src.zip
drwxrwxr-x 10 root wheel 340B Oct 1 09:02 jre
drwxrwxr-x 14 root wheel 476B Oct 1 09:02 lib
drwxrwxr-x 5 root wheel 170B Oct 1 09:01 man
-rw-rw-r-- 1 root wheel 529B Oct 1 09:01 release
-rw-rw-r-- 1 root wheel 20M Oct 1 09:01 src.zip
So, try to set valid path to JDK root directory. I think it should fix your problem.

Fixed Rest Endpoints

I have a war file deployed on tomcat ( /var/lib/tomcat7/webapps folder ), say, rest-api-webapp-0.0.1.war
To access the rest endpoint check, exposed in this war, I use curl in the format
curl -X POST -H "Content-Type:application/x-www-form-urlencoded" -d "remarks=Tester" https://localhost:8080/rest-api-webapp-0.0.1/check The problem I face is, whenever I up the patch/major/minor versions of my webapp, I need to change the curl appropriately (say the version is now 0.1.4, then the curl must change as curl -X POST -H "Content-Type:application/x-www-form-urlencoded" -d "remarks=Tester" https://localhost:8080/rest-api-webapp-0.1.4/check. I donot wish to change the way the client calls the endpoint (because it requires the client to upgrade their app, which they resent and see as high maintenance) Can this be avoided by doing something like this
Create a symbolic link as below ln -s rest-api-webapp-0.0.1.war rest-api inside the /var/log/tomcat7/webapps folder so that whennever I up the version, I just change the symlink to point to the new version and the client need not do anything to use the new version of the api. In effect, I need the api endpoint to be fixed and not change as and when I up the versions on the server. For Ex: I need the endpoint to be fixed as https:gva.atr.in/colouring-api/check and whenever I have a major change in the controllers and all that I need to do is just update the symlink and not change the endpoint. If you find that this approach is flawed, please show me the right direction as I have been trying to read about this from the past 2 days, but found very less articles that address my problem.

I read the tomcat documentation and realised that I need to create a symbolic link to the war file and give the symbolic file the war extension. So If this is what you have in your tomcat webapps directory
/var/lib/tomcat7/webapps user1$ ls
drwxr-xr-x 11 user1 wheel 374 Oct 19 21:52 .
drwxr-xr-x 9 user1 admin 306 Oct 19 19:36 ..
drwxr-xr-x 19 user1 wheel 646 Aug 29 20:19 ROOT
drwxr-xr-x 55 user1 wheel 1870 Aug 29 20:19 docs
drwxr-xr-x 4 user1 wheel 136 Oct 19 19:46 rest-api-webapp-0.2.1
-rw-r--r-- 1 user1 wheel 48258097 Oct 19 19:46 rest-api-webapp-0.2.1.war
drwxr-xr-x 7 user1 wheel 238 Aug 29 20:19 host-manager
drwxr-xr-x 8 user1 wheel 272 Aug 29 20:19 manager
Do this,
ln -s rest-api-webapp-0.2.1.war rest-api.war So that the directory looks like this (Wait for sometime for the tomcat engine to deploy the new war)
drwxr-xr-x 11 user1 wheel 374 Oct 19 21:52 .
drwxr-xr-x 9 user1 admin 306 Oct 19 19:36 ..
drwxr-xr-x 19 user1 wheel 646 Aug 29 20:19 ROOT
drwxr-xr-x 4 user1 wheel 136 Oct 19 21:52 rest-api
drwxr-xr-x 4 user1 wheel 136 Oct 19 19:46 rest-api-webapp-0.2.1
-rw-r--r-- 1 user1 wheel 48258097 Oct 19 19:46 rest-api-webapp-0.2.1.war
lrwxr-xr-x 1 user1 wheel 25 Oct 19 21:51 rest-api.war -> rest-api-webapp-0.2.1.war
drwxr-xr-x 7 user1 wheel 238 Aug 29 20:19 host-manager
drwxr-xr-x 8 user1 wheel 272 Aug 29 20:19 manager If needed restart your tomcat and you can use the curl command like this curl -X POST -H "Content-Type:application/x-www-form-urlencoded" -d "remarks=Tester" https://localhost:8080/rest-api/check totally not bothering about the major, minor, patch versions. All that you need to do once you have a new version is unlink rest-apiln -s rest-api-webapp-X.Y.Z.war rest-api.war

JConsole location in Linux

I am trying to run jconsole from my machine but I can't find the location of the .sh . I have tried finding it in different directories and setting JAVA_HOME to the jdk.
$which java
/usr/bin/java
$java -version
java version "1.7.0_51"
OpenJDK Runtime Environment (rhel-2.4.4.1.el6_5-x86_64 u51-b02)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
$whereis java
java: /usr/bin/java /etc/java /usr/lib/java /usr/share/java /usr/share/man/man1/java.1.gz
I looked in /usr/lib/jvm and I found
drwxr-xr-x. 3 root root 4096 Apr 23 2013 java-1.5.0-gcj-1.5.0.0
drwxr-xr-x. 3 root root 4096 Jan 27 2014 java-1.6.0-openjdk-1.6.0.0.x86_64
drwxr-xr-x. 3 root root 4096 Feb 5 2014 java-1.7.0-openjdk-1.7.0.51.x86_64
lrwxrwxrwx. 1 root root 21 Feb 5 2014 jre -> /etc/alternatives/jre
lrwxrwxrwx. 1 root root 27 Apr 23 2013 jre-1.5.0 -> /etc/alternatives/jre_1.5.0
lrwxrwxrwx. 1 root root 26 Apr 23 2013 jre-1.5.0-gcj -> java-1.5.0-gcj-1.5.0.0/jre
lrwxrwxrwx. 1 root root 27 Feb 5 2014 jre-1.6.0 -> /etc/alternatives/jre_1.6.0
lrwxrwxrwx. 1 root root 37 Feb 5 2014 jre-1.6.0-openjdk.x86_64 -> java-1.6.0-openjdk- 1.6.0.0.x86_64/jre
lrwxrwxrwx. 1 root root 27 Feb 5 2014 jre-1.7.0 -> /etc/alternatives/jre_1.7.0
lrwxrwxrwx. 1 root root 38 Feb 5 2014 jre-1.7.0-openjdk.x86_64 -> java-1.7.0-openjdk-1.7.0.51.x86_64/jre
lrwxrwxrwx. 1 root root 25 Apr 23 2013 jre-gcj -> /etc/alternatives/jre_gcj
lrwxrwxrwx. 1 root root 29 Feb 5 2014 jre-openjdk -> /etc/alternatives/jre_openjdk
and in java-1.7.0-openjdk-1.7.0.51.x86_64/jre/bin/, I only found
[root#portal01 bin]# ll
total 200
-rwxr-xr-x. 1 root root 9536 Jan 14 2014 java
-rwxr-xr-x. 1 root root 9656 Jan 14 2014 keytool
-rwxr-xr-x. 1 root root 9856 Jan 14 2014 orbd
-rwxr-xr-x. 1 root root 9688 Jan 14 2014 pack200
-rwxr-xr-x. 1 root root 10984 Jan 14 2014 policytool
-rwxr-xr-x. 1 root root 9648 Jan 14 2014 rmid
-rwxr-xr-x. 1 root root 9656 Jan 14 2014 rmiregistry
-rwxr-xr-x. 1 root root 9688 Jan 14 2014 servertool
-rwxr-xr-x. 1 root root 9896 Jan 14 2014 tnameserv
-rwxr-xr-x. 1 root root 92264 Jan 14 2014 unpack200
..even though there are 200 files. How can I find and run jconsole?

jconsole is part of the Java SDK, it appears you only have the JRE installed.

You can run jconsole directly from your terminal by running the command jconsole.
reena#IM-1500:~$ jconsole
It will launch java monitoring console dialog.

From https://serverfault.com/questions/179908/fedora-jconsole-location:
Try running: yum provides */jconsole
that will tell you what package you need as well as where it's located in the package.

Once you set the path of JDK in Linux then this command execute easily might be you had not included the JAVA_HOME with PATH variable.
plz follow this discussion

If you have Java JDK installed in your Linux machine, usually jconsole is located at /usr/bin/ as a symbolic link.
Example from my Linux VM running Centos6
$ which jconsole
/usr/bin/jconsole
$ ls -l /usr/bin/jconsole
lrwxrwxrwx. 1 root root 26 Sep 4 10:05 /usr/bin/jconsole -> /etc/alternatives/jconsole
$ ls -l /etc/alternatives/jconsole
lrwxrwxrwx. 1 root root 51 Sep 4 10:05 /etc/alternatives/jconsole -> /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin/jconsole
But you mentioned you got the a command not found error when you typed jconsole at the prompt. Can you find if jconsole is available in one of the jdk's bin directory?
$ ls -l /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin | grep jconsole
If yes, then add this in your PATH environment variable like so
$ export PATH=$PATH:/usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin
or put this in your ~/.bashrc if you are using Bash.
If jconsole is not there, then maybe you need to try another jdk which you have installed or re-install the jdk.

Please install JDK Development Package to get Jconsole