Remote debugging in flink - java

Added one parameter in flink-conf.yaml :
env.java.opts.taskmanager: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=51005"
Then i started a jobmanager and a taskmanager using below commands on localhost :
flink-1.9.1_bin/bin/jobmanager.sh start
flink-1.9.1_bin/bin/taskmanager.sh start
Jobmanager runs and i can see flink UI dashboard on localhost:8081. Taskmanager waits for connection on 51001 port. It runs after i debug my code from IDE(IntelliJ) which has setup for remote debugging on localhost:51001. I can see tasks slots been added when i start debugging from IDE. After this i ran below command:
flink-1.9.1_bin/bin/flink run -c myapp.Main myapp.jar
I am expecting debug point to come to my local code in IDE, but it's not coming. My code goes in running state directly that i can in flink UI dashboard.
I am able do remote debugging for normal java projects but not for flink jobs.

Related

SpringBoot2.2.1 deployment on cloud server always cannot run report error "Web server failed to start. Port 9090 was already in use."

I am new to SpringBoot and java.I have build a SpringBoot2 app,I want to deploy it in a cloud server. I have specify that port in Application-online.properties like
server.port=9090 but when I use IDEA MAVEN clean and package command to build a jar then copy to cloud server.
I use
nohup java -jar -Dserver.port=9090 -Dspring.profiles.active=online ./mall-0.0.1-SNAPSHOT.jar > /root/imooc_mall/mall.log 2>&1
then quickly the program exit. the mall.log report :
***************************
APPLICATION FAILED TO START
***************************
Description:
Web server failed to start. Port 9090 was already in use.
Action:
Identify and stop the process that's listening on port 9090 or configure this application to listen on another port.
[06:25 23:31:32.595] [INFO] [org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor] - Shutting down ExecutorService 'applicationTaskExecutor'
Noticably, 9090 is the second port I use, before that I use 8081 meet same error. Just like whatever I use any port,it always in use. And I have confirm that the 9090 and 8081 is not in use use netstat -tulpn and lsof -i :8081/9090 . I don't know what's the problem.Hope someone can give some ideals.Thanks advance!
The Environment are Centos7 and SpringBoot2.2.1, Java1.8,and Tomcat is SpringBoot in-build version
Another thing is that I have successed in starting the springboot app at first,But I use kill -9 to stop the program. After that the above errors occurs and never success .I wonder if that have any bad effect on the question.
But I am sure the port is not in use.
I still use SpringBoot 2.2.1.But not package it with built-in Tomcat.Instead, I install Tomcat server on the server. And Packaging SpringBoot application as war package. And I finally successed in running the application.

Why some VPN clients break Java debugging and how to work around the issue?

I am using IntelliJ to develop a Scala project.
Due to client-server architecture of the system, my integration tests have to be run with the following settings in build.sbt:
fork in IntegrationTest := true
javaOptions in (IntegrationTest) ++= Seq("-Djdk.logging.allowStackWalkSearch=true", "-XX:PermSize=256M", "-XX:MaxPermSize=512M", "-Xmx1024m")
// for attaching with debugger to the processes under test
javaOptions in (IntegrationTest) += "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
Normally everything works fine - tests are being run, debugging works.
Sometimes I need to establish a VPN connection to access some resources of my employee company. I'm using Check Point Endpoint Security VPN (that's the officially recommended software of my employee and I'm not sure if anything else would work).
So, if I happen to be connected to the VPN and then run the integration tests, then SBT console starts getting stuck right after:
Listening for transport dt_socket at address: 5005
The exact message is:
Listening for transport dt_socket at address: 5025
[error] Uncaught exception when running tests: java.net.ConnectException: Connection timed out: connect
[trace] Stack trace suppressed: run last project/it:test for the full output.
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
When I run last project/it:test (I have to reload SBT console first because it has stuck in (busy) > state) I see this:
[debug] javaOptions: List(-Djdk.logging.allowStackWalkSearch=true, -XX:PermSize=256M, -XX:MaxPermSize=512M, -Xmx1024m, -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005)
[debug] Forking tests - parallelism = false
[debug] Create a single-thread test executor
[error] Uncaught exception when running tests: java.net.ConnectException: Connection timed out: connect
Sometimes when I disconnect the VPN and run the tests again, it starts working. But often disconnecting VPN doesn't help and I have to reboot my computer.
I have tried some less dramatic solutions - restarting IDE, killing all java and javaw processes, looking at netstat results to see anything still using the port 5005, changing the port to 5025 in build.sbt and reloading SBT console... nothing works, only reboot and only until the next time I need to connect to VPN.
That's a nightmare. I don't want to reboot my machine each time after I connect to VPN.
Is there any solution to this? Any Java flags? Any Windows network stack settings? Any VPN settings?

Spring Boot with debug args not close debug port after application stopped

I'm running my Spring Boot application with debug mode enabled. Like:
mvn -P dev spring-boot:run -Drun.jvmArguments="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005"
I can debug the application after that without any problem. But, after stopped (ctrl + c) and try to start again, I receive this error:
[INFO] Attaching agents: []
ERROR: transport error 202: bind failed: Address already in use
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)
JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [debugInit.c:750]
So, every time that I need to start the application in debug mode after that, I need to kill the process using the port 5005.
I would like to know why the port 5005 is still opened after stop the application.
I'm using Cygwin64 on Windows 10, Maven 3.5.2, Java 8 and Spring Boot 1.5.9.RELEASE.
Thanks!
This is not a problem of Spring Boot, but a problem of non-cygwin applications running in cygwin. And your application is not only leaving port 5005 open, it just keeps running in the background, having port 8080 also opened, but as on a restart the debug port is first opened, this is what you see in the error message.
This was discussed in a SpringBoot issue back in 2014 https://github.com/spring-projects/spring-boot/issues/773. I cite from this issue (the link in the citation dates from 2006):
Anyone still following this bug, this isn't a Spring Boot bug at all. It's a known problem with non-Cygwin programs (Java in this case) running on pty-based terminals.
See this link for a full explanation why this can't be fixed: http://cygwin.com/ml/cygwin/2006-12/msg00151.html
So you can basically use one of the following approaches:
use the native windows terminal and setup your java and maven environment for that
run your maven command from within an IDE which can handle this - you might use this for debugging as well
add the Spring Actuator to your app and use the /shutdown endpoint to stop your application

Launch spark master windows7

Using win7-64, jdk8, sparks1.6.2.
I have spark running, winutils, HADOOP_HOME, etc
Per documentation Note: The launch scripts do not currently support Windows. To run a Spark cluster on Windows, start the master and workers by hand. But does not say how?
How do I launch spark master on windows?
Tried running sh start-master.sh thru git bash : failed to launch org.apache.spark.deploy.master.Master: Even though it prints out Master --ip Sam-Toshiba --port 7077 --webui-port 8080 - So I don't know what all this means.
But when I try spark-submit --class " " --master spark://Sam-Toshiba:7077 target/ .jar -
I get errors:
WARN AbstractLifeCycle: FAILED SelectChannelConnector#0.0.0.0:
4040: java.net.BindException: Address already in use: bind
java.net.BindException: Address already in use
WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
17/01/12 14:44:29 WARN AppClient$ClientEndpoint: Failed to connect to master Sam-Toshiba:7077
java.io.IOException: Failed to connect to Sam-Toshiba/192.168.137.1:7077
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
Also tried spark://localhost:7077 - same errors
On Windows you can launch Master using below command. Open command prompt and go to Spark bin folder and execute
spark-class.cmd org.apache.spark.deploy.master.Master
Above command will print like Master: Starting Spark master at spark://192.168.99.1:7077 in console as per IP of your machine. You can check the UI at http://192.168.99.1:8080/
If you want to launch worker once your master is up you can use below command. This will use all the available cores of your machine.
spark-class.cmd org.apache.spark.deploy.worker.Worker spark://192.168.99.1:7077
If you want to utilize 2 cores of your 4 cores of machine then use
spark-class.cmd org.apache.spark.deploy.worker.Worker -c 2 spark://192.168.99.1:7077

Getting ScreenRenderException while running OfBiz

I am new to OfBiz.while running it in eclipse...I got the following in eclipse console...
Httpd started on port: 9989
Sessiond started on port: 9990
2014-02-04 12:57:37,538 (main) [ BeanShellContainer.java:100:INFO ] Started BeanShell telnet service on 9989, 9990
2014-02-04 12:57:37,538 (main) [ BeanShellContainer.java:101:INFO ] NOTICE: BeanShell service ports are not secure. Please protect the ports
2014-02-04 12:57:45,824 (org.ofbiz.service.jms.JmsListenerFactory#6ec213ad) [ JmsListenerFactory.java:75 :INFO ] JMS Listener Factory Thread Finished; All listeners connected.
after this I am trying to open url https://localhost:8080/webtools as mentioned in the tutorial but i am experiencing the following error message in the browser
:ERROR MESSAGE:
org.ofbiz.widget.screen.ScreenRenderException: Error rendering screen [component://common/widget/CommonScreens.xml#GlobalDecorator]: java.lang.IllegalArgumentException: Template location is empty (Template location is empty)
Help me to fix this...Thank You
That error usually happens when you run the ant build without any additional command and then execute the application
>ant
You can install the demo or a new installation, to install the demo run:
>ant load-demo
OR to install the new one run:
>ant run-install
if you need to clean before a new installation run:
>ant clean-all
With the information and templates loaded the application should work now, run from eclipse or with the command
startofbiz
Hope it helps
Regards

Categories