I'm trying to set up a storm cluster but am having trouble running topologies on the cluster. I can run them locally on each machine but not together. I am using the following command from the storm starter kit to try and make it work:
$ storm jar storm-starter-*.jar storm.starter.RollingTopWords production-topology remote
I am getting two different but similar errors when I run the topology on one of the supervisor machines vs the nimbus machine. I have an instance of zookeeper that my supervisor nodes should be able to connect to but clearly that is not happening. Honestly, I don't even really understand how zookeeper manages the cluster.
Error thrown in supervisor:
scix3#M5:~/apache/storm/examples/storm-starter/target$ storm jar storm-starter-0.9.4.jar storm.starter.RollingTopWords production-topology remote
Running: /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -client - Dstorm.options= -Dstorm.home=/home/scix3/apache/storm -Dstorm.log.dir=/home/scix3/apache/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/scix3/apache/storm/lib/kryo- 2.21.jar:/home/scix3/apache/storm/lib/core.incubator-0.1.0.jar:/home/scix3/apache/storm/lib/commons-fileupload-1.2.1.jar:/home/scix3/apache/storm/lib/ring-servlet- 0.3.11.jar:/home/scix3/apache/storm/lib/clj-stacktrace-0.2.2.jar:/home/scix3/apache/storm/lib/jline-2.11.jar:/home/scix3/apache/storm/lib/servlet-api-2.5.jar:/home/scix3/apache/storm/lib/disruptor-2.10.1.jar:/home/scix3/apache/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/scix3/apache/storm/lib/clojure- 1.5.1.jar:/home/scix3/apache/storm/lib/commons-exec-1.1.jar:/home/scix3/apache/storm/lib/logback-core-1.0.13.jar:/home/scix3/apache/storm/lib/jetty-util-6.1.26.jar:/home/scix3/apache/storm/lib/slf4j-api-1.7.5.jar:/home/scix3/apache/storm/lib/carbonite-1.4.0.jar:/home/scix3/apache/storm/lib/compojure- 1.1.3.jar:/home/scix3/apache/storm/lib/minlog-1.2.jar:/home/scix3/apache/storm/lib/commons-lang-2.5.jar:/home/scix3/apache/storm/lib/tools.macro- 0.1.0.jar:/home/scix3/apache/storm/lib/reflectasm-1.07-shaded.jar:/home/scix3/apache/storm/lib/tools.cli-0.2.4.jar:/home/scix3/apache/storm/lib/math.numeric-tower-0.0.1.jar:/home/scix3/apache/storm/lib/logback-classic-1.0.13.jar:/home/scix3/apache/storm/lib/tools.logging-0.2.3.jar:/home/scix3/apache/storm/lib/asm-4.0.jar:/home/scix3/apache/storm/lib/jetty-6.1.26.jar:/home/scix3/apache/storm/lib/snakeyaml-1.11.jar:/home/scix3/apache/storm/lib/hiccup-0.3.6.jar:/home/scix3/apache/storm/lib/clj-time-0.4.1.jar:/home/scix3/apache/storm/lib/jgrapht-core-0.9.0.jar:/home/scix3/apache/storm/lib/clout-1.0.1.jar:/home/scix3/apache/storm/lib/chill-java-0.3.5.jar:/home/scix3/apache/storm/lib/commons-io- 2.4.jar:/home/scix3/apache/storm/lib/joda-time-2.0.jar:/home/scix3/apache/storm/lib/storm-core-0.9.4.jar:/home/scix3/apache/storm/lib/objenesis-1.2.jar:/home/scix3/apache/storm/lib/commons-logging-1.1.3.jar:/home/scix3/apache/storm/lib/ring-core-1.1.5.jar:/home/scix3/apache/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/scix3/apache/storm/lib/commons-codec-1.6.jar:/home/scix3/apache/storm/lib/json-simple-1.1.jar:/home/scix3/apache/storm/lib/ring-devel-0.3.11.jar:storm-starter-0.9.4.jar:/home/scix3/apache/storm/conf:/home/scix3/apache/storm/bin - Dstorm.jar=storm-starter-0.9.4.jar storm.starter.RollingTopWords production- topology remote
200 [main] INFO storm.starter.RollingTopWords - Topology name: production- topology
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Lists
at storm.starter.tools.Rankings.<init>(Rankings.java:33)
at storm.starter.bolt.AbstractRankerBolt.<init> (AbstractRankerBolt.java:69)
at storm.starter.bolt.AbstractRankerBolt.<init> (AbstractRankerBolt.java:56)
at storm.starter.bolt.IntermediateRankingsBolt.<init> (IntermediateRankingsBolt.java:41)
at storm.starter.RollingTopWords.wireTopology(RollingTopWords.java:68)
at storm.starter.RollingTopWords.<init>(RollingTopWords.java:52)
at storm.starter.RollingTopWords.main(RollingTopWords.java:122)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Lists
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 7 more
scix3#M5:~/apache/storm/examples/storm-starter/target$
Error thrown from nimbus:
scix3#M6:~/apache/storm/examples/storm-starter/targetstorm jar storm-starter-0.9.4.jar storm.starter.RollingTopWords production-topology remote
Running: java -client -Dstorm.options= -Dstorm.home=/home/scix3/apache/storm -Dstorm.log.dir=/home/scix3/apache/storm/logs - Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/scix3/apache/storm/storm-starter-0.9.4-jar-with- dependencies.jar:/home/scix3/apache/storm/storm-starter-0.9.4.jar:/home/scix3/apache/storm/lib/clj-stacktrace- 0.2.2.jar:/home/scix3/apache/storm/lib/clojure-1.5.1.jar:/home/scix3/apache/storm/lib/minlog- 1.2.jar:/home/scix3/apache/storm/lib/tools.logging-0.2.3.jar:/home/scix3/apache/storm/lib/commons-codec-1.6.jar:/home/scix3/apache/storm/lib/commons-io-2.4.jar:/home/scix3/apache/storm/lib/json-simple-1.1.jar:/home/scix3/apache/storm/lib/asm-4.0.jar:/home/scix3/apache/storm/lib/commons-fileupload-1.2.1.jar:/home/scix3/apache/storm/lib/commons-lang-2.5.jar:/home/scix3/apache/storm/lib/ring-servlet-0.3.11.jar:/home/scix3/apache/storm/lib/carbonite-1.4.0.jar:/home/scix3/apache/storm/lib/jetty-util-6.1.26.jar:/home/scix3/apache/storm/lib/tools.macro-0.1.0.jar:/home/scix3/apache/storm/lib/jgrapht-core-0.9.0.jar:/home/scix3/apache/storm/lib/kryo-2.21.jar:/home/scix3/apache/storm/lib/logback-core-1.0.13.jar:/home/scix3/apache/storm/lib/ring-devel-0.3.11.jar:/home/scix3/apache/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/scix3/apache/storm/lib/logback-classic-1.0.13.jar:/home/scix3/apache/storm/lib/commons-logging-1.1.3.jar:/home/scix3/apache/storm/lib/tools.cli-0.2.4.jar:/home/scix3/apache/storm/lib/reflectasm-1.07-shaded.jar:/home/scix3/apache/storm/lib/math.numeric-tower-0.0.1.jar:/home/scix3/apache/storm/lib/chill-java-0.3.5.jar:/home/scix3/apache/storm/lib/clout-1.0.1.jar:/home/scix3/apache/storm/lib/slf4j-api-1.7.5.jar:/home/scix3/apache/storm/lib/disruptor-2.10.1.jar:/home/scix3/apache/storm/lib/joda-time-2.0.jar:/home/scix3/apache/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/scix3/apache/storm/lib/compojure-1.1.3.jar:/home/scix3/apache/storm/lib/hiccup-0.3.6.jar:/home/scix3/apache/storm/lib/servlet-api-2.5.jar:/home/scix3/apache/storm/lib/commons-exec-1.1.jar:/home/scix3/apache/storm/lib/core.incubator-0.1.0.jar:/home/scix3/apache/storm/lib/jetty-6.1.26.jar:/home/scix3/apache/storm/lib/storm-core-0.9.4.jar:/home/scix3/apache/storm/lib/ring-core-1.1.5.jar:/home/scix3/apache/storm/lib/objenesis-1.2.jar:/home/scix3/apache/storm/lib/jline-2.11.jar:/home/scix3/apache/storm/lib/snakeyaml-1.11.jar:/home/scix3/apache/storm/lib/clj-time-0.4.1.jar:storm-starter-0.9.4.jar:/home/scix3/.storm:/home/scix3/apache/storm/bin -Dstorm.jar=storm-starter-0.9.4.jar storm.starter.RollingTopWords production-topology remote
202 [main] INFO storm.starter.RollingTopWords - Topology name: production-topology
237 [main] INFO storm.starter.RollingTopWords - Running in remote (cluster) mode
Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:38)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:116)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:70)
at storm.starter.util.StormRunner.runTopologyRemotely(StormRunner.java:45)
at storm.starter.RollingTopWords.runRemotely(RollingTopWords.java:78)
at storm.starter.RollingTopWords.main(RollingTopWords.java:129)
Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:83)
at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:63)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:47)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:43)
at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
... 5 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 11 more
scix3#M6:~/apache/storm/examples/storm-starter/target$
Although I've come this far setting up storm and zookeeper I don't really understand how they work and why I have issues. Any tips, regarding this issue or not, are more than welcome!
Related
i have two docker containers one is running an elasticsearch and another one is running my app which is using elasticsearch running in the container my app is working fine when both the elasticsearcj and my app working outside of docker in my machine
when i run both on docker i am getting java.lang.IllegalArgumentException: Self-suppression not permitted
exception
here is how i am doing
running ES on docker container
docker run --network=mynetwork -p 9200:9200 -p 9300:9300 --name docker-es -e "discovery.type=single-node" -e "cluster.name=my-cluster" elasticsearch:7.1.0
here is how i am running myapp container
docker run --network=mynetwork myapp
when i am trying to create a mapping for es doc i am getting this exception
12:04:19.347 11090 [run-main-0] HazelcastServer INFO - Starting ElasticSearch client and node in hazelcast Server on org.elasticsearch.client.RestHighLevelClient#c7484d8
12:04:19.377 11120 [pool-12-thread-1] PurchasingOrPledgeDetailsESIndexAndMappingMapping$ ERROR - index purchasingorpledgedetailsis not created with an exception {}
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
at java.base/java.lang.Thread.run(Thread.java:834)
Aug 22, 2019 12:04:19 PM org.apache.http.impl.nio.client.InternalHttpAsyncClient run
SEVERE: I/O reactor terminated abnormally
java.lang.IllegalArgumentException: Self-suppression not permitted
at java.base/java.lang.Throwable.addSuppressed(Throwable.java:1025)
at org.elasticsearch.client.RestClient.addSuppressedException(RestClient.java:489)
at org.elasticsearch.client.RestClient.access$1400(RestClient.java:95)
at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.trackFailure(RestClient.java:587)
at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:579)
at org.elasticsearch.client.RestClient$1.failed(RestClient.java:333)
at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.executionFailed(DefaultClientExchangeHandlerImpl.java:101)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:426)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.connectionRequestFailed(AbstractClientExchangeHandler.java:348)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(AbstractClientExchangeHandler.java:62)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler$1.failed(AbstractClientExchangeHandler.java:392)
at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$1.failed(PoolingNHttpClientConnectionManager.java:316)
at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
at org.apache.http.nio.pool.RouteSpecificPool.failed(RouteSpecificPool.java:162)
at org.apache.http.nio.pool.AbstractNIOConnPool.requestFailed(AbstractNIOConnPool.java:609)
at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSessionRequestCallback.failed(AbstractNIOConnPool.java:889)
at org.apache.http.impl.nio.reactor.SessionRequestImpl.failed(SessionRequestImpl.java:160)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:173)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171)
... 5 more
can anyone please guide me what is wrong here?
I have configured Apache Nutch 2.3.1 and crawled few websites. I have to index these documents to Solr (6.6.3) that is running in Cloud mode. When I execute solrindex command, I got following exception
2018-05-02 13:10:40,679 INFO [main] org.apache.hadoop.mapred.MapTask: Ignoring exception during close for org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector#3bd3d05e
java.io.IOException: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://10.11.22.156:8983/solr/collection2
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.close(SolrIndexWriter.java:103)
at org.apache.nutch.indexer.IndexWriters.close(IndexWriters.java:114)
at org.apache.nutch.indexer.IndexerOutputFormat$1.close(IndexerOutputFormat.java:54)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:670)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2019)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://10.11.22.156:8983/solr/collection2
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:559)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54)
at org.apache.nutch.indexwriter.solr.SolrIndexWriter.close(SolrIndexWriter.java:97)
... 11 more
Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://10.11.22.156:8983 refused
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:448)
... 17 more
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
Where is the problem? If I repeat the same job with solr without cloud mode, it works fines.
The error straight forward showing that you have an Apache Nutch server, which is unable to access this particular node and port of Apache Solr http://10.11.22.156:8983/solr/collection2.
You need access between these two servers to make them communicate with each other :
You need to provide outbound permission to solr server to do request response with Apache Nutch server.
Also, you need to provide inbound permission to Apache Nutch server to access the given solr IP and port.
I am using Netbeans 8.2 with the Java EE Base 1.31.1 and EJB and EAR 1.27.1 plugins installed. I installed Tomcat 9.0.4 and added it as a server to Netbeans. Netbeans can successfully start the tomcat server and launch my web app, but it will not debug. When I attempt to debug my webapp, I get the following error in my webapp log:
ant -f /Users/jryan/JDev/SVN/trunk/Java/CollaborationServer -Dnb.internal.action.name=debug -Ddirectory.deployment.supported=true -DforceRedeploy=false -Dnb.wait.for.caches=true -Dbrowser.context=/Users/jryan/JDev/SVN/trunk/Java/CollaborationServer debug
init:
deps-module-jar:
deps-ear-jar:
deps-jar:
library-inclusion-in-archive:
library-inclusion-in-manifest:
compile:
compile-jsps:
debug:
Stopping Tomcat process...
Waiting for Tomcat...
Tomcat server stopped.
Starting Tomcat process...
Waiting for Tomcat...
Tomcat server started.
connect-debugger:
/Users/jryan/JDev/SVN/trunk/Java/CollaborationServer/nbproject/build-impl.xml:1206: The following error occurred while executing this line:
/Users/jryan/JDev/SVN/trunk/Java/CollaborationServer/nbproject/build-impl.xml:1216:
org.netbeans.api.debugger.jpda.DebuggerStartException: Connection refused (Connection refused)
at org.netbeans.modules.debugger.jpda.JPDADebuggerImpl.waitRunning(JPDADebuggerImpl.java:405)
at org.netbeans.modules.debugger.jpda.JPDADebuggerImpl.waitRunning(JPDADebuggerImpl.java:387)
at org.netbeans.api.debugger.jpda.JPDADebugger.attach(JPDADebugger.java:324)
at org.netbeans.modules.debugger.jpda.ant.JPDAConnect$1.run(JPDAConnect.java:235)
at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.sun.tools.jdi.SocketTransportService.attach(SocketTransportService.java:222)
at com.sun.tools.jdi.GenericAttachingConnector.attach(GenericAttachingConnector.java:116)
at com.sun.tools.jdi.SocketAttachingConnector.attach(SocketAttachingConnector.java:90)
at org.netbeans.api.debugger.jpda.AttachingDICookie.getVirtualMachine(AttachingDICookie.java:231)
at org.netbeans.modules.debugger.jpda.actions.StartActionProvider.doStartDebugger(StartActionProvider.java:188)
at org.netbeans.modules.debugger.jpda.actions.StartActionProvider.access$200(StartActionProvider.java:80)
at org.netbeans.modules.debugger.jpda.actions.StartActionProvider$1.run(StartActionProvider.java:165)
... 4 more
BUILD FAILED (total time: 7 seconds)
The tomcat log says Listening for transport dt_socket at address: 8000
And the Netbeans Debugger Console says:
Attaching to localhost:8000
Connection refused.
Does anyone know what is happening here, and what I can do to fix it?
Thanks!
Perhaps try adding the flags detailed in this Answer.
This is how we currently do it.
Oops. It turns out that I had commented out the 127.0.0.1 localhost entry in my /etc/hosts file. When I re-enabled localhost, it worked fine! And as an aside, I thought I had to comment out the default localhost line because I had another line in there for a locally hosted site I was testing, but according to this answer, there's no need to do that!
i have the bin for spark-1.6.0-bin-hadoop2.6 im having issue trying to connect the slave to the master
so far i have tried(on ubuntu 14.04 live usb):
apt-get purge and install openssh-client and server on both systems
I have stated explicitly the ip address of the master in the spark url for the worker
spark://< master ip>:7077 and also tried changing the SPARK_MASTER_IP in /conf/spark-env.sh the worker executes but the log has the following error
im assuming there must be some ssh setup involved but i have tried ssh-keygen and ssh-copy-id # it also doesnt give any reuslts
16/02/22 07:49:16 INFO Worker: Connecting to master 192.168.0.208:7077...
16/02/22 07:49:16 WARN Worker: Failed to connect to master 192.168.0.208:7077
java.io.IOException: Failed to connect to /192.168.0.208:7077
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:167)
at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:200)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:187)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:183)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused: /192.168.0.208:7077
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:740)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
... 1 more
16/02/22 07:49:27 INFO Worker: Retrying connection to master (attempt # 2)
i am however able to open the master webUI by typing :8080 on my browser . i am also able to access the webUI of the slave from the master . im almost at the point of giving in so please helllppp.
Make sure that each master and worker has a firewall exception to allow connections for all other workers and masters.
Here's a simplified example from one of our master machines (master0):
$iptables -L
...
ACCEPT all -- worker0.company.com master0.company.com
ACCEPT all -- worker1.company.com master0.company.com
ACCEPT all -- master1.company.com master0.company.com
...
Of course you can also use IPs instead of hostnames.
I have a cassandra cluster on 4 nodes (each running ubuntu 10.04 and cassandra 0.7.4).
All nodes are connected using a SWITCH, and each is assigned a STATIC IP ADDRESS (no DNS is involved).
The node seems to be functioning properly (status is 'Normal', all nodes are up etc).
The problem is this: nodetool works, yet it only works when the nodetool command is run on the same node. When I try to run nodetool from STATIC_IP = 192.168.0.253 on STATIC_IP = 192.168.0.4 I get the following error:
(note: machine 's3789' has the static IP of 192.168.0.253)
user#s3789:/opt/cassandra/apache-cassandra-0.7.4$ bin/nodetool -host 192.168.0.4 ring
Error connection to remote JMX agent!
java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:128)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2343)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:296)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:137)
at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:107)
at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:511)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at java.net.Socket.<init>(Socket.java:392)
at java.net.Socket.<init>(Socket.java:206)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
Any hints how to solve this?
from conf/cassandra-env.sh:
# add this if you're having trouble connecting:
# JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=<public name>"
#
# see
# https://blogs.oracle.com/jmxetc/entry/troubleshooting_connection_problems_in_jconsole
# for more on configuring JMX through firewalls, etc. (Short version:
# get it working with no firewall first.)
Check if you started rmiregistry before starting the server
rmiregistry &
Getting started with RMI