I have a java application running on an openshift remote cluster and I want to debug the app from my local machine with Intellij-Idea. The app is built by a Jenkinsfile on another remote jenkins server (gradle build, docker build and pushed to openshift, where it is automatically deployed).
The Dockerfile exposes port 9009 and therefore my Intellij Remote Debug Config looks like this:
Debug Config
With the localhost in the Debug Config I need openshift port-forwarding:
oc port-forward my-pod 9009
Forwarding from 127.0.0.1:9009 -> 9009
When I start the Debugger I get the following error in Intellij:
Error running 'DTC Remote Debug':
Unable to open debugger port (localhost:9009): java.net.ConnectException "Connection refused: connect"
At the same time the terminal with the port forwarding shows:
Handling connection for 9009
E0927 09:52:33.711817 5996 portforward.go:331] an error occurred forwarding 9009 -> 9009: error forwarding port 9009 to pod ad370...c010, uid : exit status 1: 2019/09/27 03:52:33 socat[129691] E connect(5, AF=2 127.0.0.1:9009, 16): Connection refused
Doing an Nmap scan against the url where I get the index.html of my application I got the following:
nmap -sS my-openshift-url
Starting Nmap 7.80 ( https://nmap.org ) at 2019-09-27 15:01 Mitteleuropõische Sommerzeit
Nmap scan report for my-openshift-url (IP-Address)
Host is up (0.0043s latency).
rDNS record for IP-Address: dispatch-my-domain
Not shown: 997 filtered ports
PORT STATE SERVICE
80/tcp open http
443/tcp open https
9009/tcp closed pichat
Nmap done: 1 IP address (1 host up) scanned in 6.10 seconds
I guess the problem is the closed 9009 port, but I have no clue how I can open that port on my openshift cluster. I already set several environment variables in the openshift web UI (just to be sure):
DEBUG TRUE
DEBUG true
DEBUGGING TRUE
DEBUGGING true
JAVA_DEBUG TRUE
JAVA_DEBUG true
JAVA_DEBUG_PORT 9009
But I can't get it to work. If I switch the port-forwarding to 8080 I can access the index.html via localhost:8080 from my browser. I don't know if I need to change something in the project code (gradle, docker, jenkins, etc.) or if I can just open the port on the deployed service in openshift somehow...
If anything isn't clear or if I missed something just tell me. I'm happy for every piece of advice.
Regards,
Christoph
Adding the following environment variable in openshift did the trick:
JAVA_TOOL_OPTIONS -agentlib:jdwp=transport=dt_socket,address=9009,server=y,suspend=n
All the other environment variables from above are absolete...
Related
I want to connect to HBase running in standalone in a docker, using Java and the HBase API
I use this code to connect :
Configuration config = HBaseConfiguration.create();
config.set("hbase.zookeeper.quorum", "163.172.142.199");
config.set("hbase.zookeeper.property.clientPort","2181");
HBaseAdmin.checkHBaseAvailable(config);
Here is my /etc/hosts file
127.0.0.1 localhost
XXX.XXX.XXX.XXX hbase-srv
Here is the /etc/hosts file from my docker (named hbase-srv)
XXX.XXX.XXX.XXX hbase-srv
With this configuration, I get a connection refused error :
INFO | Initiating client connection, connectString=163.172.142.199:2181 sessionTimeout=90000 watcher=hconnection-0x6aba2b860x0, quorum=163.172.142.199:2181, baseZNode=/hbase
INFO | Opening socket connection to server 163.172.142.199/163.172.142.199:2181. Will not attempt to authenticate using SASL (unknown error)
INFO | Socket connection established to 163.172.142.199/163.172.142.199:2181, initiating session
INFO | Session establishment complete on server 163.172.142.199/163.172.142.199:2181, sessionid = 0x15602f8d8dc0002, negotiated timeout = 40000
INFO | Closing zookeeper sessionid=0x15602f8d8dc0002
INFO | Session: 0x15602f8d8dc0002 closed
INFO | EventThread shut down
org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1560)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1580)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1737)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isMasterRunning(ConnectionManager.java:948)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3159)
at hbase.Benchmark.main(Benchmark.java:26)
However, if I remove the lines XXX.XXX.XXX.XXX hbase-srv from both /etc/hosts files I get the error unknown host : hbase-srv
I have also checked, I can successfully telnet to my hbase docker on the client port.
On the docker, all the ports used by HBase are opened and binded to the same number (60000 on 60000, 2181 on 2181, etc).
I also wanted to add that all was fine when I used this configuration on localhost.
If you can't give me an answer to my problem, could you at least give me a procedure to deploy a standalone hbase on a docker.
UPDATE : Here is my Docker file
FROM java:openjdk-8
ADD hbase-1.2.1 /hbase-1.2.1
WORKDIR /hbase-1.2.1
# ZooKeeper
EXPOSE 2181
# HMaster
EXPOSE 60000
# HMaster Web
EXPOSE 60010
# RegionServer
EXPOSE 60020
# RegionServer Web
EXPOSE 60030
EXPOSE 16010
RUN chmod 755 /hbase-1.2.1/bin/start-hbase.sh
CMD ["/hbase-1.2.1/bin/start-hbase.sh"]
My HBase shell is working, I also tried to open the port using iptables for tcp and udp but still the same problem
There are two problems with your Dockerfile:
use hbase master start instead of start-hbase.sh
regionserver is actually not running on 60020
The 2nd problem is not so easy to solve. If run hbase standalone with version >= 1.2.0 (not sure, I'm running 1.2.0), hbase will use ephemeral port instead of the default port or the port you provide in hbase-site.xml which makes it very hard to provide hbase service in docker using the original version.
I add a property named hbase.localcluster.port.ephemeral and managed to build a standalone hbase in docker, which you can reference here.
I've been following this tutorial which is how to setup remote debugging.
I have my Jar running debug mode listening on port 6065 on my server with the following setting:
-Xrunjdwp\:transport\=dt_socket,server\=y,address\=6065,suspend\=n
I start the JAR on my server with jar in debug mode with:
myApplicationThatContainsJar.exe -debug "my application"
> Now Starting JVM
> Listening for transport dt_socket at address: 6065
I have psping tool installed which I use to ping (IPaddress:port). I am able to ping my ipaddress example: 44.66.33.66:6065 from my dev box and get a reply. But when I try to initiate remote debug in Eclipse I get:
Failed to connect to remote VM. Connection refused. Connection
refused: connect
I have verified I've added permission for both inbound and outbound traffic for that port on both my DEV box and server.
When I start my JAR on my server and before I try to connect with Eclipse I do a:
psping 44.66.33.66:6065
And I get a response stating that it sent and recieved with 0% loss, meaning I am indeed getting a response.
In the eclipse debug configuration, which in the 'Debug Configurations' window I have the host and port listed properly and the 'Connection Type: Standard(Socket Attach)'.
As soon as I try to connect with Eclipse and get the connection refused error I try psping again from the command line, but now eclipse did something to the connection and I get in the command line:
The remote computer refused the network connection.
Any suggestions where else I should check? or where else I should troubleshoot? I'm trying to do remote-debugging for my JAR from the DEV box to my server.
-Xrunjdwp\:transport\=dt_socket,server\=y,address\=6065,suspend\=n
should be:
-Xrunjdwp\:transport\=dt_socket,server\=y,address\=6065,suspend\=y
notice the suspend\=y keep in mind I need to escape the equals and others may not need to do this. Once I set to suspend\=y the application waited for me to connect from eclipse and I was able to begin remote debugging.
Don't forget to open inbound and outbound traffic for the port you used for address\=6065
Check network setting in VM and set Attached to: Bridged Adapter
Check VM IP using ipconfig command
Run Java Application using following command
> java -Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=4000,suspend=n myapp
Above command myapp replace to your application name
In eclipse go to debug configuration -> remote java Application -> New
Host: remote VM ip
Port: given port that given in command
Apply -> Debug
I'm trying to attach my Eclipse debugger running on my Windows host to a Tomcat 8 server running in my Vagrant box running CentOS 7.
The issue is something to do with the networking setup of Vagrant I believe.
Here is the full error from the Eclipse logs:
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:179)
at java.io.DataInputStream.readFully(DataInputStream.java:189)
at java.io.DataInputStream.readFully(DataInputStream.java:163)
at org.eclipse.jdi.internal.connect.SocketTransportService.readHandshake(SocketTransportService.java:216)
at org.eclipse.jdi.internal.connect.SocketTransportService.access$7(SocketTransportService.java:212)
at org.eclipse.jdi.internal.connect.SocketTransportService$3.run(SocketTransportService.java:183)
at java.lang.Thread.run(Thread.java:761)
The output from vagrant up seems correct (port 8000 is the debugging port):
==> default: Preparing network interfaces based on configuration...
default: Adapter 1: nat
==> default: Forwarding ports...
default: 8000 => 8000 (adapter 1)
default: 8443 => 8443 (adapter 1)
default: 22 => 2222 (adapter 1)
I can access my application (using https://localhost:8443/app) via a browser from my host without issue; connecting the debugger is the issue. The debugger is configured for a Remote Java Application with connection properties:
Host: localhost
Port: 8000
Starting Tomcat of course displays the correct:
Listening for transport dt_socket at address: 8000
Based on the following questions, the answer maybe have something to do with configuring Tomcat to run on 0.0.0.0 instead of the default:
Cannot reach Jekyll server on Vagrant from outside
Vagrant port forwarding not working. Cups not accesible from host
Empty reply from server - can't connect to vagrant vm w/port forwarding
Connection Reset when port forwarding with Vagrant
So I tried configuring the Connectors in Tomcat's server.xml with the line address="0.0.0.0" but the result was the same.
Any other ideas about getting this to work? Also, assuming the answer is something to do with changing the address to 0.0.0.0, why would I need to do that to get the debugger to work if accessing the app via browser is already fine on localhost:8443?
P.S. It is not a firewall issue.
I was on the right track above changing the address to 0.0.0.0, but I was changing that on the Connector in server.xml, which has nothing to do with the debugger. The resolution is to change the JPDA_ADDRESS resulting in my bin/setenv.sh getting:
JPDA_ADDRESS="0.0.0.0:8000"
This is due to a change in Tomcat 8:
When starting Tomcat with the jpda option to enable remote debugging,
Tomcat 8 listens on localhost:8000 by default. Earlier versions
listened on *:8000. If required, this default can be overridden by
setting the JPDA_ADDRESS environment variable in, for example,
setenv.[bat|sh].
i'm using Glassfish 3.1.2.2 and Eclipse Kepler.
Glassfish configuration:
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009.
Debug: Enabled
Into Eclipse, in the Remote Java Application i've selected the project, and in the connection properties i've set:
Host: localhost
port: 9009
When i try to connect the debug, the message error is:
Failed to connect to remote VM. Connection refused.
Connection refused: connect
I've checked the port 9009 with a telnet, and is not used...
can someone help me?
tx
[SOLVED] it's a problem with the glassfish plugin in Eclipse! you must to start glassfish form the command line, not from the Eclipse window. After, you can connect like remote debug to your local glassfish ;)
In Host, you need to give IP address of the remote machine you want to connect.
localhost is your own machine.
I setup my remote WebLogic server to debug, using these options:
-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=${DEBUG_PORT},server=y,suspend=n -Djava.compiler=NONE"
I started WebLogic on the server and saw that it was listening on 8453 for dt_socket. I added the remote server on my local Eclipse and tested the connection. I also see that the server is STARTED and I can view the details. However, I'm unable to restart in DEBUG. I also stopped WebLogic on the server and now in Eclipse I see that it is stopped, but I'm not able to start it (the icons for debug and start are greyed out). When I set up my Remote Java Application in Eclipse I did check "Allow termination of remote VM".
Before stopping the server, the debug icon wasn't disabled and when I clicked it I got this message:
Cannot attach to remote server.
Please make sure the server myserver.com is running and listening for transport dt_socket at address: 8453
Failed to connect to remote VM. Connection refused.
Connection refused: connect
Instead of using java startup options, modify the following scripts:
setDomainEnv.sh -> make sure DEBUG_PORT is set ie. 8453
startWebLogic.sh -> set PRODUCTION_MODE=false and set debugFlag=true
Restart server, remote debug should work on port 8453.