We Keep Coding

java.net.ConnectException when running MPJBOOT on linux cluster

I followed the steps on page http://mpjexpress.org/docs/guides/linuxguide.pdf, and I can start daemons on all machines. If I just run program on single server, it runs successfully
[ip1,master] MPJ Daemon started successfully with process id: xxx
[ip2,node] MPJ Daemon started successfully with process id: xxx
But when I use command mpjrun.sh -np 2 -dev niodev HelloWorld, it shows java.net.ConnectException: Connection refused and Multi-threaded starter: exceptionnull, but I can Ping this IP successfully.
Is anything wrong? Thanks for your comments.

SpringBoot2.2.1 deployment on cloud server always cannot run report error "Web server failed to start. Port 9090 was already in use."

I am new to SpringBoot and java.I have build a SpringBoot2 app,I want to deploy it in a cloud server. I have specify that port in Application-online.properties like
server.port=9090 but when I use IDEA MAVEN clean and package command to build a jar then copy to cloud server.
I use
nohup java -jar -Dserver.port=9090 -Dspring.profiles.active=online ./mall-0.0.1-SNAPSHOT.jar > /root/imooc_mall/mall.log 2>&1
then quickly the program exit. the mall.log report :
***************************
APPLICATION FAILED TO START
***************************
Description:
Web server failed to start. Port 9090 was already in use.
Action:
Identify and stop the process that's listening on port 9090 or configure this application to listen on another port.
[06:25 23:31:32.595] [INFO] [org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor] - Shutting down ExecutorService 'applicationTaskExecutor'
Noticably, 9090 is the second port I use, before that I use 8081 meet same error. Just like whatever I use any port,it always in use. And I have confirm that the 9090 and 8081 is not in use use netstat -tulpn and lsof -i :8081/9090 . I don't know what's the problem.Hope someone can give some ideals.Thanks advance!
The Environment are Centos7 and SpringBoot2.2.1, Java1.8,and Tomcat is SpringBoot in-build version
Another thing is that I have successed in starting the springboot app at first,But I use kill -9 to stop the program. After that the above errors occurs and never success .I wonder if that have any bad effect on the question.
But I am sure the port is not in use.

I still use SpringBoot 2.2.1.But not package it with built-in Tomcat.Instead, I install Tomcat server on the server. And Packaging SpringBoot application as war package. And I finally successed in running the application.

Jenkins kubernetes plugin: provided port:50000 is not reachable

I have set up jenkins on GKE using the official helm chart.
I have also created an nginx-ingress controller installation using helm and I am able to access jenkins via https://112.222.111.22/jenkins where 112.222.111.22 is the static IP I am passing to the load balancer.
I am also able to create jobs.
However, when I try to spin up inbound remote agent:
▶ java -jar agent.jar -noCertificateCheck -jnlpUrl https://112.222.111.22/jenkins/computer/My%20Builder%203/slave-agent.jnlp -secret <some_secret>
...
WARNING: Connect timed out
Feb 28, 2020 5:57:18 PM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: https://112.222.111.22/jenkins/ provided port:50000 is not reachable
java.io.IOException: https://112.222.111.22/jenkins/ provided port:50000 is not reachable
at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:303)
at hudson.remoting.Engine.innerRun(Engine.java:527)
at hudson.remoting.Engine.run(Engine.java:488)
Why is that?

I had similar issue. I have solved it enabling the "Use WebSocket". Jenkins Salve/Agent > Configure > Launch Method > Use WebSocket (enable) > Save.

Might be because port 50000 is not open in the Jenkins master.
You can try to open the port by Creating Inbound rule in Jenkins Master's firewall which allows Port 50000 (coming from other machine in this case Agent, hence Inbound rule).
Try following steps:
Go to Master Jenkins.
Open Wiindows Defender Firewall--> Advance setting --> create Inbound Rule allow port 50000
Note: In 1 inbound rule you can allow more than one port as shown in below.
Restart master Jenkins service.
Check if (step is valid only if you have created windows service on agent) agent service is running continuously.
Check Agent status in Jenkins !

looks like port 50000 is not open in the jenkins master. Try to open the port and restart the machine and instance ( i.e jenkins via url ) and see if it helps.

HBase 1.2.1 standalone in Docker unable to connect

I want to connect to HBase running in standalone in a docker, using Java and the HBase API
I use this code to connect :
Configuration config = HBaseConfiguration.create();
config.set("hbase.zookeeper.quorum", "163.172.142.199");
config.set("hbase.zookeeper.property.clientPort","2181");
HBaseAdmin.checkHBaseAvailable(config);
Here is my /etc/hosts file
127.0.0.1 localhost
XXX.XXX.XXX.XXX hbase-srv
Here is the /etc/hosts file from my docker (named hbase-srv)
XXX.XXX.XXX.XXX hbase-srv
With this configuration, I get a connection refused error :
INFO | Initiating client connection, connectString=163.172.142.199:2181 sessionTimeout=90000 watcher=hconnection-0x6aba2b860x0, quorum=163.172.142.199:2181, baseZNode=/hbase
INFO | Opening socket connection to server 163.172.142.199/163.172.142.199:2181. Will not attempt to authenticate using SASL (unknown error)
INFO | Socket connection established to 163.172.142.199/163.172.142.199:2181, initiating session
INFO | Session establishment complete on server 163.172.142.199/163.172.142.199:2181, sessionid = 0x15602f8d8dc0002, negotiated timeout = 40000
INFO | Closing zookeeper sessionid=0x15602f8d8dc0002
INFO | Session: 0x15602f8d8dc0002 closed
INFO | EventThread shut down
org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1560)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1580)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1737)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isMasterRunning(ConnectionManager.java:948)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3159)
at hbase.Benchmark.main(Benchmark.java:26)
However, if I remove the lines XXX.XXX.XXX.XXX hbase-srv from both /etc/hosts files I get the error unknown host : hbase-srv
I have also checked, I can successfully telnet to my hbase docker on the client port.
On the docker, all the ports used by HBase are opened and binded to the same number (60000 on 60000, 2181 on 2181, etc).
I also wanted to add that all was fine when I used this configuration on localhost.
If you can't give me an answer to my problem, could you at least give me a procedure to deploy a standalone hbase on a docker.
UPDATE : Here is my Docker file
FROM java:openjdk-8
ADD hbase-1.2.1 /hbase-1.2.1
WORKDIR /hbase-1.2.1
# ZooKeeper
EXPOSE 2181
# HMaster
EXPOSE 60000
# HMaster Web
EXPOSE 60010
# RegionServer
EXPOSE 60020
# RegionServer Web
EXPOSE 60030
EXPOSE 16010
RUN chmod 755 /hbase-1.2.1/bin/start-hbase.sh
CMD ["/hbase-1.2.1/bin/start-hbase.sh"]
My HBase shell is working, I also tried to open the port using iptables for tcp and udp but still the same problem

There are two problems with your Dockerfile:
use hbase master start instead of start-hbase.sh
regionserver is actually not running on 60020
The 2nd problem is not so easy to solve. If run hbase standalone with version >= 1.2.0 (not sure, I'm running 1.2.0), hbase will use ephemeral port instead of the default port or the port you provide in hbase-site.xml which makes it very hard to provide hbase service in docker using the original version.
I add a property named hbase.localcluster.port.ephemeral and managed to build a standalone hbase in docker, which you can reference here.

Timeout when submitting jobs to EC2 cluster

I've been trying to make this work with no luck thus far. I launch a cluster with
./spark-ec2 -k keyname -i ~/.keys/key.pem --region=us-east-1 -s 5 launch "my test cluster"
Then I submit a job with
bin/spark-submit --verbose --class com.company.jobs.AggregateCostDataWorkflow --master spark://ec2-54-157-122-49.compute-1.amazonaws.com:7077 --deploy-mode cluster --conf spark.executor.memory=5g /Users/my.name/scala-proj/target/scala-2.10/scala-proj-0.1.0.jar --outputPath,s3n://my-bucket/my-name/ec2-spark-test/
Where outPutPath is an argument to the main method. After a bit and some status output, I see an exception that looks like
15/06/05 16:09:33 INFO StandaloneRestClient: Submitting a request to launch an application in spark://ec2-74-141-162-19.compute-1.amazonaws.com:7077.
Exception in thread "main" java.net.ConnectException: Operation timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at [java socket stuff elided for brevity] org.apache.spark.deploy.rest.StandaloneRestClient.postJson(StandaloneRestClient.scala:150)
at org.apache.spark.deploy.rest.StandaloneRestClient.createSubmission(StandaloneRestClient.scala:70)
at org.apache.spark.deploy.rest.StandaloneRestClient$.run(StandaloneRestClient.scala:317)
at org.apache.spark.deploy.rest.StandaloneRestClient$.main(StandaloneRestClient.scala:329)
at org.apache.spark.deploy.rest.StandaloneRestClient.main(StandaloneRestClient.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:178)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
This is spark 1.3.1 (on my local machine) I can access the UI on the master machine and verify that the Spark processes are in fact up. I can also ssh into the master.
Any tips?

You need to open ports by editing security policies, if you want to access ports on your EC2 spark cluster. spark_ec2.py doesn't open ports 7077 and 6066 on master to be accessed from outside the cluster.
I use the other way - connect to master machine of your spark cluster with the command
./spark_ec2.py -k keyname -i ~/.keys/key.pem login "my test cluster"
Upload the your job file (with scp using same key) and submit job from there. This would ensure that your driver has access to a cluster master and slaves.
See "Running Applications" section of Running Spark on EC2 documentation