We Keep Coding

"Unavaliable io exception" when connecting to remote Bazel master on bazel-buildfarm

I want to setup a small POC remote area with 1x master (192.168.60.99) and 1x worker (192.168.60.98) using bazel-buildfarm. Both are CentOS 7 machines provisioned with Vagrant. When connection from a Ubuntu workstation (third machine) in the network, the following error occurs:
$ bazel build --verbose_failures //projects/myproj:app
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=229
INFO: Reading rc options for 'build' from /home/user/tests/ecommerce/.bazelrc:
'build' options: --strategy=TypeScriptCompile=worker --strategy=AngularTemplateCompile=worker --symlink_prefix=dist/ --define=compile=legacy --incompatible_strict_action_env --experimental_allow_incremental_repository_updates --distdir=third_party/_distdir
INFO: Reading rc options for 'build' from /home/user/.bazelrc:
'build' options: --spawn_strategy=remote --genrule_strategy=remote --strategy=Javac=remote --strategy=Closure=remote --remote_executor=192.168.60.99:8980
INFO: Writing tracer profile to '/home/user/.cache/bazel/_bazel_user/24700f1ad3e201a00a1c26bd59dc6502/command.profile.gz'
INFO: Invocation ID: 569b59ca-edcb-4922-92a0-b6f0b5ca2819
ERROR: Failed to query remote execution capabilities: UNAVAILABLE: io exception
The network connection is working and I even can connect to Bazel using telnet:
telnet 192.168.60.99 8980
Trying 192.168.60.99...
Connected to 192.168.60.99.
Escape character is '^]'.
.bazelrc file of the third Ubuntu machine:
$ cat ~/.bazelrc
build --spawn_strategy=remote --genrule_strategy=remote --strategy=Javac=remote --strategy=Closure=remote --remote_executor=192.168.60.99:8980
Buildfarm setup
Both got a clon of the buildfarm git repo. The example config files were used. Just on the server I replaced localhost by 192.168.60.99 (master server ip).
I know that bazel run is not recommended. But in lack of better alternatives that works, my idea is to get the documented way working first (Bazel itself doesn't mention any alternative). Since not even bazel run works, I think that something is wrong with my installation.
All machines use version 1.1.0, which is the latest stable one at the time of writing. It's definitely an issue with bazel-buildfarm, since the local build works fine on the Ubuntu machine.
Master server
bazel run //src/main/java/build/buildfarm:buildfarm-server $(pwd)/examples/server.config.example
Worker
bazel run //src/main/java/build/buildfarm:buildfarm-operationqueue-worker $(pwd)/examples/worker.config.example --distdir ~/distdir/
The distdir is a workaround for our company proxy, that manipulates files with MITM attacks. Since Bazel doesn't allow this, I downloaded the affected file for it's jdk manually:
[vagrant#localhost bazel-buildfarm]$ l ~/distdir/
total 188M
-rw-rw-r--. 1 vagrant vagrant 188M Jan 17 2019 zulu11.2.3-jdk11.0.1-linux_x64.tar.gz

If Bazel >= 1.0 is used , we need to specify the protocol grpc in .bazelrc like this:
--remote_executor=grpc://192.168.60.99:8980
Without the protocol, the UNAVAILABLE: io exception occurs. There is currently no documentation about this issue.

How to reference local Kafka and Zookeeper config on Spring Cloud Dataflow "Cloudfoundry" server start

Here's is what I have successfully done so far on SCDF Local Server
I have successfully deployed SCDF server on my local and also I have used Kafka and Zookeeper config parameters with it i.e
mymac$ java -jar spring-cloud-dataflow-server-local-1.3.0.RELEASE.jar
--spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.brokers=localhost:9092
--spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.zkNodes=localhost:2181
I was able to create my stream
ingest = producer-app > :broker1
filter = :broker1 > filter-app > :broker2
Now I need help to do the exact same thing on PCFDev
I have my PCFDEv running
I have to deploy SCDF-Cloudfoundry jar with my local kafka and zookeeper parameters to pcfDev but when I do the following steps it gives me an error that its
1.1) cf push -f manifest-scdf.yml --no-start -p /XXX/XXX/XXX/spring-cloud-dataflow-server-cloudfoundry-1.3.0.BUILD-SNAPSHOT.jar -k 1500M
this runs good...no problem. but 1.2
1.2) cf start dataflow-server --spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.brokers=host.pcfdev.io:9092 --spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.zkNodes=host.pcfdev.io:2181
gives me this error:--
Incorrect Usage: unknown flag `spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.brokers'
below is my manifest-scdf.yml file
---
instances: 1
memory: 2048M
applications:
- name: dataflow-server
host: dataflow-server
services:
- redis
- rabbit
env:
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL: https://api.local.pcfdev.io
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG: pcfdev-org
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE: pcfdev-space
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN: local.pcfdev.io
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME: admin
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD: admin
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION: true
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_SERVICES: rabbit
MAVEN_REMOTE_REPOSITORIES_REPO1_URL: https://repo.spring.io/libs-snapshot
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DISK: 512
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_BUILDPACK: java_buildpack
spring.cloud.deployer.cloudfoundry.stream.memory: 400
spring.cloud.dataflow.features.tasks-enabled: true
spring.cloud.dataflow.features.streams-enabled: true
Please help me. Thank you.

There are few options to supply Kafka credentials to Stream-apps in PCF.
1. Kafka CUPs
This option allows you to create CUPs for an external Kafka-service. While deploying the stream, you can then supply the coordinates to each application either individually as described in the docs or you can supply them as global properties for all the stream-apps deployed by the SCDF-server.
2. Inline properties
Instead of extracting from CUPs, you can also directly supply the HOST/PORT while deploying the stream. Again, this can be applied globally, too.
stream deploy myTest --properties "app.*.spring.cloud.stream.kafka.binder.brokers=<HOST>:9092,app.*.spring.cloud.stream.kafka.binder.zkNodes=<HOST>:2181
Note: The HOST must be reachable for the stream-apps; o'wise, it ill continue to connect to localhost and potentially fail since the apps are running inside a VM.

The error you're seeing is coming from the CF CLI, it's interpreting those (I'm assuming environment) variables you're providing as flags to the cf start command and failing.
You could either provide them in your manifest.yml or set their values manually using the CLI's cf set-env command by doing something like this:
cf set-env dataflow-server spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.brokers host.pcfdev.io:9092
cf set-env dataflow-server spring.cloud.dataflow.applicationProperties.stream.spring.cloud.stream.kafka.binder.zkNodes host.pcfdev.io:2181
After you've set them they should be picked up when you run cf start dataflow-server.
Relevant CLI docs:
http://cli.cloudfoundry.org/en-US/cf/set-env.html

"Uncaught Could not connect" error when using Openshift to deploy .war for PHP/Java bridge

Problem: Recieve 'Could not connect' error when using Openshift to deploy .war for PHP/Java bridge
Below I describe every step I've taken to get to this error, thanks in advance.
Full error: (IP concealed)
"Fatal error: Uncaught Could not connect to the JEE server W.X.Y.Z:8080. Please start it. Or define('JAVA_HOSTS',9267); define('JAVA_SERVLET',false); before including 'Java.inc' and try again. Error message: Connection refused (111) thrown in /home/stevenw1/public_html/softwareProject/real/inc/Java.inc on line 989"
Players: "Java_Bridge.war" = my java .war file that php will use to call methods from
"W.X.Y.Z" = Unchanging IP address of deployed .war file
"GUIProfile" = a Java class with #WebServlet("/GUIProfile") as public class GUIProfile extends HttpServlet
"tomcattest.php" = test file that calls GUIProfile's static method 'validate'
Deployed Java_Bridge.war and tomcattest.php work fine on my local tomcat, just not yet on Openshift.
Senario:
0. followed command line steps from Openshift 2012 java tomcat application
1. Using Openshift's rhc I have created my first app using "rhc app create -a tomcat6 -t jbossews-1.0"
2. cd ~/tomcat6/; git rm -rf ./src/ pom.xml
3. cp ~/Java_Bridge.war ./webapps/; git add ./webapps/Java_Bridge.war; git commit -m "first try"; git push
//notable output includes:
remote: Stopping jbossews cartridge
remote: Sending SIGTERM to jboss:386662 ...
remote: Skipping Maven build due to absence of pom.xml
remote: Preparing build for deployment
remote: Deployment id is bd83d6eb
remote: Activating deployment
remote: + '[' Java_Bridge.war ']'
remote: Starting jbossews cartridge
remote: Found W.X.Y.Z:8080 listening port //(not actual IP)
remote: Git Post-Receive Result: success
remote: Activation status: success
remote: Deployment completed with status: success
to ssh://thisisnottheactualuser#tomcat6-notthenamehereeither.rhcloud.com
4. Thinking the .war was deployed I ran a tomcattest.php that calls a java method 'validate', from java class GUIProfile that extends httpservlet, using the php/java bridge Java.inc
//of course using the same IP from above...
cat tomcattest.php;
<?php
define("JAVA_HOSTS", "W.X.Y.Z:8080");
define("JAVA_SERVLET", "/Java_Bridge/GUIProfile");
require_once("Java.inc");
$valiationOutput = java_context()->getServlet()->validate("hello");
echo $valiationOutput;
?>
EDIT: later tried this also in all instances: same results exactly
cat alternatetest.php
<?php
define("JAVA_HOSTS", "W.X.Y.Z:8080");
require_once("Java.inc");
echo java("java.lang.System")->getProperties();
?>
The output of calling tomcattest.php is the 'Fatal error' you see at the top.
Double Check:
5. ssh thisisnottheactualuser#tomcat6-notthenamehereeither.rhcloud.com
6. env | grep "IP"
OPENSHIFT_JBOSSEWS_IP=W.X.Y.Z
7. env | grep "PORT"
OPENSHIFT_JBOSSEWS_HTTP_PORT=8080
OPENSHIFT_JBOSSEWS_JPDA_PORT=8787
8. changed port in tomcattest.php to 8787, didn't change a thing of course.
9. restarted from the top, this time keeping ./src/ & pom.xml, same 'Deployment completed with status: success', same error using tomcattest.php with new listening IP...
10. tried solution from 'openshift youtube vid 2012' seemingly outdated. Followed step by step:
onced ssh'ed, no "tomcat" dir, I used cd $OPENSHIFT_DATA_DIR instead to access data dir
wget tomcat, changed ports to >15000, sh startup.sh && tail -f ../logs/*
notable errors in the result include:
SEVERE: Failed to initialize connector [Connector[AJP/1.3-15009]]
org.apache.catalina.LifecycleException: Failed to initialize component
[Connector[AJP/1.3-15009]] SEVERE: Failed to initialize connector
[Connector[AJP/1.3-15009]] org.apache.catalina.LifecycleException:
Failed to initialize component [Connector[AJP/1.3-15009]] SEVERE:
Failed to initialize end point associated with ProtocolHandler
["ajp-bio-15009"] java.net.BindException: Permission denied
:15009 SEVERE: Failed to initialize end point associated with
ProtocolHandler ["ajp-bio-15009"] java.net.BindException: Permission
denied :15009
11. restarted step 6, this time left IP's as they were, changing only localhost to tomcat-stevenwernercs.rhcloud.com, same result but with original IP's
12. ran out of options online, then I posed this question...
13. then I edited it a lot...
14. patiently wait :)
That is everything I have done, I am not sure why php isnt finding Java.
I looked at previous questions didn't find anything helpful.
Open to anything, thank you

Gears in a non-scable application are not allowed to communicate on any non http/ws port. So if your php java bridge is trying to communicate with the tomcat instance on 15009 (from what i see above) that is not going to work. You could only make requests on 80/443/8000/8443. Can you verify what port the java bridge works over?

Alternate solution: https://bitnami.com/cloud
I got it working using Bitnami with Amazon AWS, it provides free service for 1 year, documentation that is easy to follow, and plentiful and up to date, It took less than an hour.
Disclaimer: Using ssh tunnel for port 8080 i configured tomcat as if on localhost, and moved my hosted .php to the
same microserver that also hosted the tomcat. (I gave up on trying to host php and tomcat separately.)
I have hosted tomcat separately before on my university's tomcat server. But that was a node I had full access to.
I would still like to try Openshifts free solution, although the current resources and methods I've seen seem to no longer be valid. And unfortunately the workarounds are not intuitive enough and are currently over my head. Thank you.

Remote debugging on Tomcat with Eclipse

I'm trying to initiate a remote debugging session on my PC with Eclipse & Tomcat.
I managed to run tomcat (not through eclipse) with the following params:
set JPDA_TRANSPORT=dt_socket
set JPDA_ADDRESS=5050
cataline jpda start
The server starts ok, this is the outcome:
Using CATALINA_BASE: "C:\Java\Tomcat"
Using CATALINA_HOME: "C:\Java\Tomcat"
Using CATALINA_TMPDIR: "C:\Java\Tomcat\temp"
Using JRE_HOME: "C:\Java\jdk1.6.0_27"
Using CLASSPATH: ...
Listening for transport dt_socket at address: 5050
...
...
INFO: Server startup in 12502 ms
Now, I opened eclipse, and loaded the relevant project.
I set a new debugging configuration using Remote Java Application with localhost and the right port. however when I run it, I get this error message:
Failed to connect to remote VM. Connection refused.
Connection refused: connect
Can't really understand where is the problem. this is all local inside the PC, so there shouldn't be any firewalls involved, can someone think of something ?

Found the solution. apparently the connection was made, but Eclipse & Tomcat don'tt show any notification or status regarding it.
Then, when you initiate the connection again, you get the error because you're already connected.
I think that a small notice, either from Eclipse, or from Tomcat would be nice.

Try adding the following debug options directly to the JVM startup by directly modifying the catalina startup script
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5050

There could be problems when there are code mismatches between eclipse and tomcat. So when it matches the error will disappear

You can configure it in remote java application section in debug configuration.
For more details you can refer this link

Unable to deploy in Netbeans 6.7.1 and Glassfish v2.1.1

I am trying to deploy a simple WebService in Netbeans 6.7.1 and Glassfish v2.1.1 and am getting the following error. I am using GlassfishESBv2.2 and windows 7 machine. I have tried googling and implemented things as shown in
http://forums.netbeans.org/topic10055-0-asc-0.html . Still unable to deploy. Though the message says that application server is not started, from the Server tab , I am able to see a message which indicates Glassfish has started.Also , doing a netstat after trying to deploy returns this, which means that Glassfish is running.
C:>netstat -an | findstr "4848"
TCP 0.0.0.0:4848 0.0.0.0:0 LISTENING
I have been trying real hard to get this resolved. Any help is highly appreciated.
Error Message :
The Sun Java System Application Server could not start.
More information about the cause is in the Server log file.
Possible reasons include:
- IDE timeout: refresh the server node to see if it's running now.
- Port conflicts. (use netstat -a to detect possible port numbers already used by the operating system.)
- Incorrect server configuration (domain.xml to be corrected manually)
- Corrupted Deployed Applications preventing the server to start.(This can be seen in the server.log file. In this case, domain.xml needs to be modified).
- Invalid installation location.
C:\Users\xyz\Documents\NetBeansProjects\HWebService\nbproject\build-impl.xml:564: Deployment error:
The Sun Java System Application Server could not start.
More information about the cause is in the Server log file.
Possible reasons include:
- IDE timeout: refresh the server node to see if it's running now.
- Port conflicts. (use netstat -a to detect possible port numbers already used by the operating system.)
- Incorrect server configuration (domain.xml to be corrected manually)
- Corrupted Deployed Applications preventing the server to start.(This can be seen in the server.log file. In this case, domain.xml needs to be modified).
- Invalid installation location.
See the server log for details.
BUILD FAILED (total time: 29 seconds)

I've described the solution please visit here...
http://forums.netbeans.org/post-65058.html