We Keep Coding

Unable to open flight recorder in JMC for Java application and Eclipse IDE

When I try to run flight recorder it takes a very long time in loading and then crashes, and throws an exception after a while:
Connection refused to host: 17.226.xx.yyyy; nested exception is:
java.net.ConnectException: Operation timed out (Connection timed out)
java.lang.RuntimeException: Could not connect to [1.8.0_181] Eclipse
(2092). Make sure the JVM is running and that you are using the
correct protocol in the Service URL
(service:jmx:rmi://127.0.0.1/stub/rO0ABXN9AAAAAQAlamF2YXgubWFuYWdlbWVudC5yZW1vdGUucm1pLlJNSVNlcnZlcnhyABdqYXZhLmxhbmcucmVmbGVjdC5Qcm94eeEn2iDMEEPLAgABTAABaHQAJUxqYXZhL2xhbmcvcmVmbGVjdC9JbnZvY2F0aW9uSGFuZGxlcjt4cHNyAC1qYXZhLnJtaS5zZXJ2ZXIuUmVtb3RlT2JqZWN0SW52b2NhdGlvbkhhbmRsZXIAAAAAAAAAAgIAAHhyABxqYXZhLnJtaS5zZXJ2ZXIuUmVtb3RlT2JqZWN002G0kQxhMx4DAAB4cHc4AAtVbmljYXN0UmVmMgAADTE3LjIyNi4yMC4yMjQAAMF44tzu6CRJNeORCVrBAAABZN4+pbCAAQB4).
I have tried running different version of jdks: from java 7 to java 8:
jdk1.7.0_80.jdk
jdk1.8.0_141.jdk
jdk1.8.0_144.jdk
jdk1.8.0_162.jdk
jdk1.8.0_181.jdk
Also, uninstalled eclipse and installed back STS back. Still the issue is there. Looks like issue is somehwhere in jmc not able to connect to eclipse and/or tomcat jvm.
Anyone who has resolved similar issue please help on that. Thanks.
Update: I am able to record flights using command terminal by using
jcmd $pid JFR.start name=MyRecording settings=default duration=60s
jcmd $pid JFR.dump name=MyRecording filename=/tmp/test_TIMESTAMP.jfr
Still I am clueless why it would not run from gui.

I am able to open flight recorder now with a "hack".
To make it work I had to add -Djava.rmi.server.hostname=localhost vm arg in the server configs of my JEE application.

Elasticsearch: Could not find or load main class org.elasticsearch.tools.launchers.JavaVersionChecker

I'm using CentOS and have downloaded Elasticsearch 6.2.1. I created a new user "elastic" and when I run ./bin/elasticsearch I get the error:
Could not find or load main class org.elasticsearch.tools.launchers.JavaVersionChecker
I tried placing this user in an admin group ("wheel"), and the same problem occurs. If I try it with "sudo ./bin/elasticsearch" I get:
[2018-02-15T17:42:39,776][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.2.1.jar:6.2.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.1.jar:6.2.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.1.jar:6.2.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.1.jar:6.2.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.1.jar:6.2.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.2.1.jar:6.2.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.2.1.jar:6.2.1]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:105) ~[elasticsearch-6.2.1.jar:6.2.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) ~[elasticsearch-6.2.1.jar:6.2.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.1.jar:6.2.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.1.jar:6.2.1]
I searched a bit and saw that it this error can be due to the java version, but it seems to be up to date:
[elastic#sandbox-hdp elasticsearch-6.1.1]$ sudo update-alternatives --config java
There are 3 programs which provide 'java'.
Selection Command
1 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
*+ 2 /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java
3 /usr/lib/jvm/jre-1.5.0-gcj/bin/java
This also happens if I try Elasticsearch 6.1
Any suggestions would be appreciated.

In case you still have this problem... or just for curiosity... take a look at the file ./bin/elasticsearch-env.sh, around the line 70, look for the command:
"$JAVA" -cp "$ES_CLASSPATH" org.elasticsearch.tools.launchers.JavaVersionChecker
Check if it is correctly "spelled" for your environment, write this command just before it and see what you've got:
echo "$JAVA" -cp "$ES_CLASSPATH" org.elasticsearch.tools.launchers.JavaVersionChecker
in my case... I was running in a Git Bash inside a Windows machine, and the command was pointing to:
/c/Users/Ualter/Developer/ELKStack/elasticsearch-6.2.2/lib/*
Changing to:
/Users/Ualter/Developer/ELKStack/elasticsearch-6.2.2/lib/*
As I was running on a Windows S.O. it get up and running OK the Elasticsearch.

I was trying hard to use elasticsearch 6.7.1 on Windows.
I had this exact error when running bin/elasticsearch.bat .
Long story short, there was probably an issue with my java version and elastic search, it couldn't run the jars (using Java 8 211).
I installed the last ElasticSearch 6.x from here :
https://www.elastic.co/downloads/past-releases#elasticsearch
And unset the existing ES_HOME and ES_CLASSPATH environment variables that I created, and all went well.

I also encounter such strange question, the first time it raises exception for running elastic as root, after I add a new user elastic and run it as elastic , I see this exception "Could not find or load main class org.elasticsearch.tools.launchers.JavaVersionChecker
".
I solved this problem by moving the program dirs to the non /root/ dirs, for example, I put it to /opt/elk/, and it works.
So I guess elastic may not allow to run as root for security , thus we can not put the program in the root user's dir. Hope it can make some tips for you.

I'm installing elasticsearch plugin es-sql on windows.
In my case, run the cmd as administrator role.

Seems like you don't have right permissions. Try to set them to 774 (rwxrwxr--) and check again:
sudo chmod 774 -R elasticsearch-6.3.2/

Jenkins windows ssh slave access denied

I want to build my projects on a Windows machine, so I created a slave there. First we tried the standard java application runner, but our build sets up its own environment and it always ended up deleting the existing Java installation, not giving any proper error messages, but failing. Then I switched to using Cygwin sshd and have Jenkins log in via ssh. This works for the connection (just as the standard method worked), but when I ran the build then it failed again to install the jdk. I ended up running the following command by hand:
C:\jenkins\tools\hudson.model.JDK\Latest\jdk.exe ADDLOCAL="ToolsFeature" REBOOT=ReallySuppress INSTALLDIR=C:\jenkins\tools\hudson.model.JDK\Latest /L
This is the same as Jenkins would run, but without the /s (silent) option. Now when I run the build, I get the following error:
Building remotely on jenkins-slave-win64 (Johnny5) (windows) in workspace C:\jenkins\workspace\continuous.xill
java.io.IOException: remote file operation failed: C:\jenkins\tools\hudson.model.JDK\Latest at hudson.remoting.Channel#200354fa:jenkins-slave-win64 (Johnny5): java.nio.file.AccessDeniedException: C:\jenkins\tools\hudson.model.JDK\Latest\bin\java.exe
at hudson.FilePath.act(FilePath.java:987)
at hudson.FilePath.act(FilePath.java:969)
at hudson.FilePath.deleteRecursive(FilePath.java:1171)
at hudson.tools.JDKInstaller.performInstallation(JDKInstaller.java:130)
at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:68)
at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:108)
at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206)
at hudson.model.JDK.forNode(JDK.java:132)
at hudson.model.AbstractProject.getEnvironment(AbstractProject.java:360)
at hudson.model.Run.getEnvironment(Run.java:2234)
at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:935)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1036)
at hudson.scm.SCM.checkout(SCM.java:485)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1277)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532)
at hudson.model.Run.execute(Run.java:1741)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:381)
Caused by: java.nio.file.AccessDeniedException: C:\jenkins\tools\hudson.model.JDK\Latest\bin\java.exe
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at hudson.Util.deleteFile(Util.java:247)
Of course I tried everything:
Run sshd as a different (Administrator) user
Checked every error mentioned here:
https://wiki.jenkins-ci.org/display/JENKINS/SSH+slaves+and+Cygwin
here
https://wiki.jenkins-ci.org/display/JENKINS/My+software+builds+on+my+computer+but+not+on+Jenkins
and here
http://www.noah.org/ssh/cygwin-sshd.html
And nothing works. Do you have a hint where to look?
UPDATE:
Meanwhile I managed to get a bit further in the build process by deactivating the option to "Automatically install jdk" in the Jenkins settings, git clone works and gradle starts up. However, now the JAVA_HOME environment variable somehow is not accessible to the slave (although we even tried setting it in .bashrc, just to be sure). So the build fails with this message":
FAILURE: Build failed with an exception.
* Where:
Script 'http://dl.bintray.com/content/shemnon/javafx-gradle/8.1.1/javafx.plugin' line: 60
* What went wrong:
A problem occurred evaluating script.
> Could not find JavaFX Packager Tools, please set one of [jfxrtDir in Gradle Properties, JFXRT_HOME in System Environment, JAVA_HOME in System Environment, java.home in JVM properties]

For your build error after "UPDATE" I think you have to add a JDK under Manage Jenkins > Configure System > JDK or in newer versions under Manage Jenkins > Global Tool Configuration > JDK

Running a script in apache storm

Is there a way to simply run a python script in Apache Storm?
I'm trying to figure out how to use storm to run scripts but am having trouble. It seems like I need to create a Java program to call the script and use it as a bolt but I simply want to send a very basic python script to storm to see if it is possible.
I read that the following command is helpful in sending topologies to storm but am having trouble understanding the syntax and if I am allowed to send any python code to storm or if it needs to have specific syntax.
Can someone clarify whether or not I can submit any python script to storm and if so what the following line of code means.
storm shell resources/ python topology.py arg1 arg2
When I try to submit a basic python script using the above code i get the following output.
956 [main] INFO backtype.storm.StormSubmitter - Uploading topology jar stormshell8691441.jar to assigned location: /home/scix3/apache/storm/data/nimbus/inbox/stormjar-ae0739f9-7c93-4f00-a02b-c4eceba3b005.jar
966 [main] INFO backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: /home/scix3/apache/storm/data/nimbus/inbox/stormjar-ae0739f9-7c93-4f00-a02b-c4eceba3b005.jar
Exception in thread "main" java.io.IOException: Cannot run program "simple.py" (in directory "."): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
at java.lang.Runtime.exec(Runtime.java:617)
at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:58)
at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:254)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:319)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:160)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:147)
at backtype.storm.util$exec_command_BANG_.invoke(util.clj:386)
at backtype.storm.command.shell_submission$_main.doInvoke(shell_submission.clj:29)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at backtype.storm.command.shell_submission.main(Unknown Source)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
... 10 more
The exact command I'm using (possibly incorrect) is storm shell resources/ simple.py
simple.py is merely a print 'Hello, world script.
I'm using storm version 0.9.4

Yes, you can run python on Storm. In fact you can run just about code from just about any language on a storm cluster, its just a matter of implementing the API.
However, there are some requirements for that to work, and so far as I can tell said requirements are not spelled out in the storm documentation. The fastest path to get up an running would be to take the splitsentence.py example from the storm source and run with it.

try pyleus (https://github.com/Yelp/pyleus) or streamparse (https://github.com/Parsely/streamparse), i will recommend using pyleus as it is simple.

Hive script issue

I am running the following shell script,
#!/bin/bash
HADOOP_HOME='/home/hadoop/hadoop-0.20.2'
HIVE_HOME='/home/hadoop/hive-0.8.0'
echo -e '1\x01foo' > /tmp/a.txt
echo -e '2\x01bar' >> /tmp/a.txt
HADOOP_CORE={{ls $HADOOP_HOME/hadoop-0.20.2-core.jar}}
CLASSPATH=.:$HADOOP_CORE:$HIVE_HOME/conf
for i in ${HIVE_HOME}/lib/*.jar ; do
CLASSPATH=$CLASSPATH:$i
done
java -cp $CLASSPATH HiveJdbcClient
In this code every variable is set properly. I have written code as per instructions given here. My hive version is 0.8.0.
When I run my code it gives me the following error,
./test.sh
./test.sh: line 8: /home/hadoop/hadoop-0.20.2/hadoop-0.20.2-core.jar}}: No such file or directory
Exception in thread "main" java.sql.SQLException: Could not establish connection to localhost:10000/default: java.net.ConnectException: Connection refused
at org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:98)
at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:104)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at HiveJdbcClient.main(HiveJdbcClient.java:22)
Can anyone please explain me what this error means?

The error you are seeing indicates that Java could not connect to the port with the JDBC server.
You should start it by following the instructions on the HiveServer wiki page. Something like:
$HIVE_HOME/bin/hive --service hiveserver
should work.

code Banger, here tricky thing is that Hive was not running in the background at all.
Check it is running.
First export the hive port, then start the hive.
$ export HIVE_PORT=10000
Good luck!